Yeah, I think you can't just throw money randomly at problems and expect results unless you know a line of attack that can get you all the way. OpenAI chose the line of attack only after it became known to them via rumors. They "front-ran" the researchers.

Yes. What the headlines hailed as an AGI discovery the facts show more to be someone spending years mining for gold, rumor gets to OpenAI that there might be gold in this specific place, they mine there and instantly discover gold, then tell the world they’ve developed the worlds best gold finding/mining machine.

Separate from all the allegations of more nefarious actions and ethical issues, that’s the most charitable version of what happened here.

You can’t do anything novel with these models from scratch and let it fly. I’ve observed something over the past few months

Work on something novel -> llm is kinda useless and low value-add -> Keep at it and in the process feed it more information -> keep doing this periodically -> a few months go by and you realise the model outputs are almost like-for-like regurgitations of what was inputted in some prior period.

Once it’s accumulated new info can it produce something automated that is somewhat useful? Sure.

But by itself - absolutely not.

I clearly see humans will be needed - the best ones that is. For ‘rote work’ and stuff that is not IP sensitive firms will be ok with employees putting that as inputs into models.

But I’d wary about trusting the labs. They will push the letter of the law to the max.

Personally I’ve stopped doing anything novel with these models. If I do use a model on something adjacent but not totally novel I have to craft the inputs in a strategic way not to give much away.

I’d wager firms will soon realise this and that growth rate of revenues of the frontier labs will become questionable. The economic cost that firms have brought out thus far is only financial. There’s a whole bunch of other costs people aren’t talking about.

Almost but not quite I think. You can throw money at parts of problems. I think it's helpful to think it kind of like supercomputer MD/MC or electronic structure calculations. A tool that can get you valuable answers but not necessarily aid understanding. Simulations can be used to aid understanding also, and are integral to theory development. In the same way the approach to this result is.

Worth noting they claim they did not choose the line of attack. Of course we don’t know whether that is true.

but the researchers were also largely relying on AI