Remix.run Logo
▲ wewewedxfgdf 2 hours ago

Gemini is so far behind that it is effectively useless compared to Claude.

It's a surprise that Google has let themselves lose the game given their infinite cash, massive computing resource, gargantuan information store/training data, and vast number of programmers.

The truckloads of ads revenue mean they don't have the single focus drive needed to win.

▲jjice 2 hours ago | parent | next [-]

We're like 3.5 years into this new era - I'm not counting winners or losers yet.

▲mattlondon 2 hours ago | parent | prev | next [-]

How is it far behind? The benchmarks published in the blog post show it is superior to Opus 5.5 and Astra 6?

Behind how?

▲wewewedxfgdf 2 hours ago | parent [-]

Within one question of their web interface, it has lost context and asks you to clarify what you are talking about.

I am very often giving the same programming task to multiple LLMs for various reasons - the answers from Google are so bad that I gave up.

I have no interest in benchmarks.

▲mattlondon 2 hours ago | parent | next [-]

So you have no experience of their latest model release then? Just repeating the usual tropes about Google having messed up? Or basing your opinions on their website chatbot?

If you have actual independent benchmarks and evidence about how this new model release is "so far behind" and refutes the stuff from their blog then please do share because I think we'd all love to see that?

▲wewewedxfgdf 2 hours ago | parent [-]

No I am commenting on my real world experience of using Gemini daily. I still ask it questions alongside Claude and OpenAI and Gemini is always the worst of the three.

▲mattlondon 2 hours ago | parent [-]

So you've not used this new release then? So how can you say that they are "so far behind" if you are not using the most recent model for your comparison. This is their first 4.0 model, that you are not using and instead basing all your opinions on on some ancient months-old model from a previous generation?

With respect, I don't find your arguement about them being "so far behind" especially convincing when you are using previous-gen releases and not actually using their current release.

▲fwip 2 hours ago | parent | prev | next [-]

Yeah, they've definitely got some recurring tooling/infrastructure problems around the models.

▲ 2 hours ago | parent | prev [-]
[deleted]
▲ASalazarMX 2 hours ago | parent | prev | next [-]

Funny how we start to see people supporting LLMs like we support sport teams, political parties, or celebrities.

- Person 1: X is garbage compared to Y!

- Person 2: Why?

- Person 1: Because I like Y.

▲bel8 2 hours ago | parent | prev | next [-]

I wonder if Google bans internal use of Claude/Codex.

And I wonder if Google's main monorepo is already in Anthropic/OpenAI training data because of some stubborn dev.

▲krat0sprakhar 2 hours ago | parent | next [-]

(I work at Google) Yes, internally we all use Jetski (internal version of Antigravity). Outside of Gemini, Opus models are supported and allowed for internal use. No OpenAI models since they are not on Vertex

▲heyjamesknight an hour ago | parent | prev | next [-]

No way to run OAI on a machine with monorepo access even if you wanted to. Claude runs on Vertex so it's not leaving Google infrastructure.

▲lunarboy 2 hours ago | parent | prev [-]

Claude used to be GDM only, but recently opened up Opus for all googlers

▲LoganDark 2 hours ago | parent | prev | next [-]

I've tasted Gemini through an intermediary and it feels far better at attention to detail than other models I've tested (Claude Opus/Sonnet, GPT whatever it's called nowadays). But it's less likely to get one-shots right.

▲gniv 2 hours ago | parent | prev | next [-]

They are playing a longer-term and more enterprise-oriented game.

▲ 2 hours ago | parent [-]
[deleted]
▲dhdjcjcjnd 2 hours ago | parent | prev | next [-]

Google's strategy is to let their competitors bankrupt themselves while they continue to offer good-enough models near breakeven.

▲georgemcbay 2 hours ago | parent | prev | next [-]

> Gemini is so far behind that it is effectively useless compared to Claude.

I fundamentally don't understand LLM "brand loyalty".

All of the models are constantly leapfrogging each other and always have been.

Google had a long lag between releases (and still hasn't released Argon), but why wouldn't they be able to compete? It isn't like any of this stuff requires secret knowledge, the Bitter Lesson has proved true again and again, and Google can certainly scale computation, it is like the one single thing they've always done well in spite of all their other foibles.

▲wewewedxfgdf an hour ago | parent | next [-]

Its not brand loyalty. I use them all the time and have no loyalty - I'd happily ditch an LLM for better results - that's how I got to Claude from ChatGPT.

▲786562354238 2 hours ago | parent | prev | next [-]

Gemini has never ever leapfrogged any competitor.

▲singingtoday 2 hours ago | parent | prev [-]

I hope it can. Today it is very far behind.

▲VirusNewbie 2 hours ago | parent | prev | next [-]

-

▲handfuloflight 2 hours ago | parent [-]

You have access to Argon?

▲osti 2 hours ago | parent | next [-]

Google employees do.

▲matthewfcarlson 2 hours ago | parent | prev [-]

Their profile says: > Currently at Google as a Sr. SWE SRE on the cloud.

▲ 2 hours ago | parent | prev [-]
[deleted]