Remix.run Logo
▲ vikramkr 2 hours ago

Theo is quite the twitter shitposter but this tweet lines up directly with what I've seen: https://x.com/theo/status/2114483219538296946

Aggressively staggeringly high rates of straight up making shit up and covering up the lies. Gemini 3.5 flash was comical because it would delusionally scream nonsense like "we're go-go ready!!! :tada:" while the code didn't compile (I swear the only reason people thought that was a good model was because they didn't use it) and their models just got better at making tool calls without fixing it's tendency to cheat.

Gemini models are not looking all that good here: https://maxxbench.unlap.ai/

They got a lot of shit for it at the argon launch and you'll notice the carbon release blog has a lot of detail on how they focused on misalignme t and fabrication rates: https://blog.google/innovation-and-ai/models-and-research/ge...

So Google themselves is not behaving like they think people are making stuff up about their models behavior.

Google itself is also just behaving in such a sleazy way. "Carbon is the best model ever but also you can't use it yet." Watch them release it now that fable is out and nobody cares anymore so nobody checks to see how much they benchmaxed lol. They're clearly under some sort of internal pressure to pretend they're at the frontier and that's bleeding through I to the frenetic pace of model releases they've had in the past couple months through Argon and Carbon. Bleeding through in the sense of the models also only really care about benchmarks and good news cycles. Again they did claim they tried to solve a lot of this for carbon but they didn't let people use the model and once you have a base model that cheats benchmarks you have to wonder if it actually lies less or just gets caught less, but hopefully they got things back on track?

▲vikramkr 44 minutes ago | parent [-]

> Watch them release it now that fable is out and nobody cares anymore so nobody checks to see how much they benchmaxed lol

Fucking called it: https://blog.google/innovation-and-ai/models-and-research/ge...