The Unofficial Gemini 3 Showcase (2): Can We Trust AI Judges? I Checked My Ranking and I’m Not Top 50…

I Checked My Ranking and I’m Not Top 50…

While waiting for the final results, maybe you checked my previous post about The Unofficial Gemini 3 Showcase and noticed your ranking wasn’t as high as you’d hoped.

Original source: https://www.kaggle.com/competitions/gemini-3/discussion/668733

Now you’re wondering: Is this AI scoring even legit?

I ran some experiments to find out.

---

The Key Finding

Ah, you might be curious about how it works? Here is the diagram

Previously, each app got scored alone → rankings.

Now I put 4,000 app pairs head-to-head: if the “winner” disagrees with the original ranking, that’s a flip.

(Yep, I also checked if showing A before B matters. Next episode!)

When the score gap is small, AI judges are unreliable.

Score Gap	Flip Rate	Verdict
< 1 pt	20-35%	🔴 Basically a coin flip
1-2 pts	15-20%	🟡 Shaky
> 2 pts	< 8%	🟢 Reliable

So if you’re not in the top tier: it might just be noise, not skill.

---

Do the 8 AI Judges Agree?

Yes — correlations hit 0.70-0.88. Not perfect twins, but definitely siblings.

• 👀 Eagle-eyed readers might notice: Gemini 3 Pro’s data looks a bit… lonely? My API budget said “no more” before I could collect enough samples (scoring takes a lot of resources since it has to “read” all the source code).

• But here’s where YOU come in! I built a quick 1-min-eval app on aistudio — just drag in your project (or paste your AI Studio URL), and let Gemini 3 judge it.

• If you want, you can download json and create a PR here, we can make build it together (thinking of a better way) or just post it in the comments

---

How to Use This Leaderboard

✅ Get a rough sense of where you stand	❌ Judge your worth as a vibe coder
✅ Find your weak dimensions	❌ Predict official results
✅ Discover great projects to learn from	❌ Lose sleep over it like I did
✅ Try the “Shuffle” button when selecting a tier

---

I’m also working on personalized takeaways for each app — what you did well, what you could improve, and ideas to inspire your next project.

If you want something specific? Drop a comment below and let me know what would be most helpful!

---

More data (updating soon): topvibecoder.com/kaggle-gemini-3