Remix.run Logo
▲ Gemini 4 Neon is the new #1 on the Models Engineering Index(git-aarvin.github.io)
57 points by hn-aarvin 2 hours ago | 28 comments
▲postalcoder 13 minutes ago | parent | next [-]

I don’t think anybody who has commented has used the actual model. But, to be fair to the commenters, google has made it very hard for them to do so without spending a ton of money.

Neon is a really good model. It’s currently my favorite model to use. Coding-wise, it’s equivalent to Opus 5.5/Fable. Documentation-wise, it is great. If you have access to Neon have it take a crack at simplifying your claude/agents/gemini.md. It’s the first model I’ve run that criticized the amount of ceremony in my instructions and suggested solid simplifications. It also makes clean edits to documents leaving scars of the previous verbiage behind.

Art-wise, it’s more capable, in my experience, than Fable. There are some pieces of art that I’ve struggled to iterate with using fable that Neon handled with ease.

In terms of working together with a model and how it communicates to you, I love this model. The model knows how to clearly communicate to you. It is such a breath of fresh air over Opus and Fable which, despite the improvements, I feel is actively trying to damage my brain with its unnatural prose.

Neon has a big model smell, but the miracle and irony is it’s supposedly based on a smaller base model than Carbon. On google’s model page, they previously wrote that Neon shares the same base model as Hydrogen! It is thought that Argon and Carbon share the same base model.

My impressions will probably change as I use this for longer than a day, but my initial impression is: I don’t know how google pulled this off.

▲dzuc 11 minutes ago | parent [-]

it's pretty easy to use inside of cursor

▲largbae 23 minutes ago | parent | prev | next [-]

Neon is a contender. Neon "high" appears faster and cheaper than Opus 5.5 "medium" with similar results on long-running language porting work in a large monorepo.

The speed is the biggest win IMO. When Carbon came out, I tried to give it a shot, but it was so slow that I went back to Opus just for productivity, nevermind quality. Today I started Neon on one worktree and Opus on another, and Neon did so well that I have now switched for both.

Code quality looks great. Comment-wise it has its own Claudese(exclamations like "this function has a concurrency bug and it's a sharp one!"), but overall I like reading it more than Opus. Might just be because I am not yet tired of its quirks though.

Oh, after a productive day on this particular work I had only spent $30 of Neon vs. $70 of Opus yesterday. YMMV

▲johnmlussier 2 minutes ago | parent | prev | next [-]

Can’t use it for anything defensive audit oriented. Refuses too often. Google seemingly not trying at all on the CVP or Daybreak front.

▲YuechenLi 39 minutes ago | parent | prev | next [-]

Ok, so I can see that this is a benchmark the submitter made and submitted themselves, which is kinda misleading and should at least be disclosed.

But my question is, what does this benchmark actually measure about the LLM's engineering abilities? It just looks like a page of random bar graphs to me.

▲khimaros 24 minutes ago | parent [-]

this seems to be a composite index based on TerminalBench, CursorBench, vals.ai, etc: https://github.com/git-aarvin/mei/tree/main/source/jsonc

looking at the graphic below the table, it does seem like data is missing for Neon which is present for Fable, and isn't penalized for the missing benchmarks.

▲dw_arthur 8 minutes ago | parent | prev | next [-]

I prefer to use agents from the command line. Does Antigravity CLI deliver a good experience?

▲zf00002 21 minutes ago | parent | prev | next [-]

I've been impressed with my usage of Neon so far. Early impression at High and Extra is near enough to Opus 5.5 Extra for me. Usage has been pretty good for that effort level too.

▲andunieee 32 minutes ago | parent | prev | next [-]

How do I use it? Do I have to create a Google Console account and navigate through 15 menus in order to generate an API key? Or are there another 7 different ways to achieve similar outcomes?

▲dudus 26 minutes ago | parent | next [-]

The subscription is the most cost effective way to use. But if you want to avoid the monthly cost you can create an API key and pay per use.

There are a few ways to create/use an API key. Through Google cloud, through vertex, but by far the easiest way is to use AI studio. https://aistudio.google.com/ pretty good platform to try out any Google model or just create an API key and use in whatever you want without having to deal with Google Cloud UI.

Alternatively you can get from other sources as well such as openrouter.

▲Yahivin 27 minutes ago | parent | prev [-]

Openrouter if you just want to pay as you go at token rates.

https://one.google.com/about/google-ai-plans/ had a 75% off on the Pro plan ($5/mo for 3 months) and limits are decent.

▲_s_a_m_ 3 minutes ago | parent | prev | next [-]

I already used up the full quota for Neon. It is fine and close to Opus 5.5 on high to xhigh. At least the new quotas are much larger with the new models and you can get something done. I optimized multiple deep learning models and integrated them into an application pipeline and everything worked pretty much like Opus (almost).

▲quantisan an hour ago | parent | prev | next [-]

When and why should I trust this index over others?

▲ummonk an hour ago | parent [-]

I wouldn't trust any particular one, but obscure / indie indexes are going to be less affected by benchmaxxing than the well known ones, so it's a positive sign for Neon that it is performing so well on a lot of them including this one.

▲js4ever an hour ago | parent | prev [-]

Is it just me or all Google models are terribly bad? They all seems benchmaxed to hell and producing total crap compared to Claude or openai models from my own experience (mostly coding)

▲hadlock 39 minutes ago | parent | next [-]

After hearing some good things from my tokenmaxxing buddies (or whatever you want to call them, AI enthusiasts), I got a $20/mo sub to try out Neon and it's just breezed through some hardware graphics bring-up problems I've been struggling with on Sol/Astra and Fable. I'm gonna drop my $200 OpenAI sub this coming month in favor of Google's $200 plan just for more Neon tokens.

▲danpalmer 25 minutes ago | parent | prev | next [-]

It's just you. Since Argon, the Gemini models have been producing good results for me side-by-side with Opus/Fable models on the same prompts. Not always winning, but increasingly so with Carbon/Neon.

▲asadm 30 minutes ago | parent | prev | next [-]

this week I ran out of codex astra quota and switched to antigravity, boy is Gemini Carbon amazing model! It actually helped achieve a breakthrough I was stuck on for weeks with Astra.

▲etoxin 24 minutes ago | parent | prev | next [-]

I'm using AI heavily at work. I'm switching all my opus 5.5 sub-agents to neon. I'm quite impressed.

▲SwellJoe 42 minutes ago | parent | prev | next [-]

I think it's just you. Carbon is excellent but extremely slow. If Neon is better than or equal to Carbon while also being a lot faster, that'd be a great model. I haven't tried Neon, yet, but they seem to have understood the assignment with the latest round of models.

▲mjburgess 43 minutes ago | parent | prev | next [-]

And tokens/task is very high. They don't seem to have figured out making the models usable in a typical interactive workflow, ie., they're in a Opus-4.7 era.

Very high hallucination rates too.

▲horacemorace 14 minutes ago | parent | prev | next [-]

Carbon changed my mind.

▲Yahivin 36 minutes ago | parent | prev | next [-]

With Neon Gemini is good now.

▲copperx 43 minutes ago | parent | prev | next [-]

Are you saying Neon is just as bad?

▲CamperBob2 3 minutes ago | parent | prev | next [-]

Is it just me or all Google models are terribly bad?

Not this one.

▲azan_ 41 minutes ago | parent | prev [-]

Yes. Every Google model so far has been disappointment. And their biggest moat, the integration into other google products has been laughable. Today I've asked gemini in gmail to download all attachments from one e-mail address and it turned out it's unable to do it. It's unbelievable how incompetent they are, Google has such a massive edge over OAI and Anthropic and it's still losing.

▲andunieee 31 minutes ago | parent [-]

You mean they are not providing you with their best models for free?

▲azan_ 2 minutes ago | parent [-]

No, why would you think that? I'm paying customer.