All of the latest developments surrounding these attacks are actually a really bad sign for these labs.

It seems that raw intelligence of frontier models has largely plateaued (despite what is basically an order of magnitude increase in parameter size) so to make any significant improvements and to justify massive capex spend they have resorted to reinforcement training models to never give up and brute force the search space until they find solution. This is what humans might do when they lack sufficient intelligence/information/knowledge to solve a problem.

This in turn is causing misalignment (I imagine it is more difficult to keep model aligned through such training process) issues that we are now witnessing and turning models into making dumb decisions and acting like brutes with no regard for their surroundings. I would argue that misaligned model is not much different from dumb model in several aspects.

On top of that they can’t seem to control their creations and processes, either due to incompetence or intentionally for PR benefits (not sure which is worse).

Given all of the above, I wonder if we can still trust these labs to develop something that benefits humanity since they seem to be making desperate attempts to improve models that stop at nothing in order to justify all the investments. One could say that they themselves, due to misaligned incentives, are much bigger threat to our society today than open weights models coming from China that they are so desperately warning us about.

▲

simonw 9 hours ago | parent | next [-]

This doesn't look like a plateau to me: https://artificialanalysis.ai/evaluations/artificial-analysi...

I do agree that they're investing heavily in brute force methods though. I've been trying out GPT-5.6 Sol "Ultra" recently and that thing fires up a bunch of subagents and crunches for hours.

▲

supermdguy 4 hours ago | parent [-]

Here's the performance of frontier models without reasoning, to more directly address the claim that raw performance is plateauing:

https://artificialanalysis.ai/evaluations/artificial-analysi...

I don't have any insider info, but if model sizes actually have increased exponentially since GPT 4.1, there's an argument to be made that there are diminishing returns in scaling pretraining alone.

Also interesting thing I haven't noticed before, Opus models have followed a really consistent linear improvement, while it looks like OpenAI struggled with base model performance until 5.5/5.6 (EDIT - 5.5 was their first new pretraining run in over a year).

▲

simonw 4 hours ago | parent [-]

The trend I've found most interesting is models of the same size getting better.

I'm very much looking forward to seeing how Qwen 3.8 27B compares to Qwen 3.6 27B next week, for example.

And the latest DeepSeek v4 Flash has extremely impressive performance for a 304B model.

	▲	asadotzler 2 hours ago \| parent [-]
		The trends you found don't support my goals so I've got some other trends I find more interesting than yours.

▲

chrisjj 5 hours ago | parent | prev [-]

> I wonder if we can still trust these labs to develop something that benefits humanity

Surely soon they'll comprise only people who are blind to the inevitable danger and people who don't care about it. Because who else would feel at all comfortable doing the job?