If you believe the bitter lesson, all the handwavy "engineering" is better done with more data. Someone likely would have written the same thing as this 8 years ago about what it would take to get current LLM performance.

So I don't buy the engineering angle, I also don't think LLMs will scale up to AGI as imagined by Asimov or any of the usual sci-fi tropes. There is something more fundamental missing, as in missing science, not missing engineering.

Even more fundamental than science, there is missing philosophy, both in us regarding these systems, and in the systems themselves. An AGI implemented by an LLM needs to, at the minimum, be able to self-learn by updating its weights, self-finetune, otherwise it quickly hits a wall between its baked-in weights and finite context window. What is the optimal "attention" mechanism for choosing what to self-finetune with, and with what strength, to improve general intelligence? Surely it should focus on reliable academics, but which academics are reliable? How can we reliably ensure it studies topics that are "pure knowledge", and who does it choose to be, if we assume there is some theoretical point where it can autonomously outpace all of the world's best human-based research teams?

> all the handwavy "engineering" is better done with more data.

How long until that gets more reliable than a simple database? How long until it can execute code faster than a CPU running a program?

A lot of the stuff humans accomplish is through technology, not due to growing a bigger brain. Even something seemingly basic like a math equation benefits drastically from being written down with pen&paper instead of being juggled in the human brain itself (see Extended mind thesis). And when it comes to something like running a 3D engine, there is pretty much no hope of doing it with just your brain.

Maybe we will get AIs smart enough that they can write their own tools, but for that to happen, we still need the infrastructure that allows them writing the tools in the first place. The way they can access Python is a start, but there is still a lack of persistence that lets them keep their accomplishments for future runs, be it in the form of a digital notepad or dynamic updating of weights.

So the "Bitter Lesson" paper actually came up recently and I was surprised to discover that what it claimed was sensible and not at "all you need is data" or "data is inherently better"

The first line and the conclusion is: "The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin." [1]

I don't necessary agree with it's examples or the direction it vaguely points at. But it's basic statement seems sound. And I would say that there's lot of opportunity for engineer, broadly speaking, in the process of creating "general methods that leverage computation" (IE, that scale). What the bitter lesson page was roughly/really about was earlier "AI" methods based on logic-programming and which including information on the problem domain in the code itself.

And finally, the "engineering" the paper talks about actually is pro-Bitter lesson as far as I can tell. It's taking data routing and architectural as "engineering" and here I agree this won't work - but for the opposite reason - specifically 'cause I don't just data routing/process will be enough.

[1]https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson...

The missing science to engineer intelligence is composable program synthesis. Aloe (https://aloe.inc) recently released a GAIA score demonstrating how CPS dramatically outperforms other generalist agents (OpenAI's deep research, Manus, and Genspark) on tasks similar to those a knowledge worker would perform.

I'd argue it's because intelligence has been treated as a ML/NN engineering problem that we've had the hyper focus on improving LLMs rather than the approach articulated in the essay.

Intelligence must be built from a first principles theory of what intelligence actually is.

I agree with your comment and the article. LLMs should be part of the answer, but the core of the progress should probably dive back into neural networks in general. Language is how we communicate as well as with other senses, but right now we're stuck at LLMs that just seem to be blown out elizas trained with other actual humans work. I remember early on, training of simple neural networks was done with rules in their environment and they evolved behavior according to criteria set, like genetic algorithms. I think the current LLMs are getting a "filtered" view of the environment, and that filter behaves like the average IQ of netizens lol

This isn’t really what the bitter lesson says.

Aye. Missing are self correction (world models/action and response observation), coherence over the long term, and self-scaling. The 3rd are what all the SV types are worried about, except maybe Yann LeCun who is worried about the first and second.

Hinton thinks the 3rd is inevitable/already here and humanity is doomed. It's an odd arena.

8 years is a pretty short perspective. The current growth phase was unlocked by more engineering. We could’ve had some of these kinds of capabilities decades ago, as illustrated by how cutting edge AI research is now trickling down all the way to microcontrollers with 64MHz CPU and kBs of RAM.

Once we got the “Attention is all you need” paper I don’t remember anyone saying we couldn’t get better results by throwing more data and compute at it. But now we’ve pretty much thrown all the data and all (as much as we can reasonably manufacture) at it. So clearly we’re at the end of that phase.

I think the gist of TFA is just that we need a new architecture/etc not scaling.

I suppose one can argue about whether designing a new AGI-capable architecture and learning algorithm(s) is a matter of engineering (applying what we already know) or research, but I think saying we need new scientific discoveries is going to far.

Neural nets seems to be the right technology, and we've now amassed a ton of knowledge and intuition about what neural nets can do and how to design with them. If there was any doubt, then LLMs, even if not brain-like, have proved the power of prediction as a learning technique - intelligence essentially is just successful prediction.

It seems pretty obvious that the rough requirements for an neural-net architecture for AGI are going to be something like our own neocortex and thalamo-cortical loop - something that learns to predict based on sensory feedback and prediction failure, including looping and working memory. Built-in "traits" like curiosity (prediction failure => focus) and boredom will be needed so that this sort of autonomous AGI puts itself into leaning situations and is capable of true innovation.

The major piece to be designed/discovered isn't so much the architecture as the incremental learning algorithm, and I think if someone like Google-DeepMind focused their money, talent and resources on this then they could fairly easily get something that worked and could then be refined.

Demis Hassabis has recently given an estimate of human-level AGI in 5 years, but has indicated that a pre-trained(?) LLM may still be one component of it, so not clear exactly what they are trying to build in that time frame. Having a built-in LLM is likely to prove to be a mistake where the bitter lesson applies - better to build something capable of self-learning and just let it learn.

> If you believe the bitter lesson, all the handwavy "engineering" is better done with more data

Id say better model architechture than more data. A human can learn to do things more complex than an LLM with less data. I think modelling the world as a static system to be representation learned in an unsupervised fashion is blocked on the static assumption. The world is dynamical, that should be reflected in the base model

But yeah, definitely not an engineering problem. Thats like saying the reason a crow isnt as smart as a person is becauss they dont have the hands to type of keyboards. But its also not because they havent seen enough of the world like your saying. Its be ause their brain isnt complex enough

> There is something more fundamental missing

I am thinking we need a foundation, something that is concrete and explicit and doesn't do hallucination. But has very limited knowledge outside of absolute Maths and basic physics.

It seems similar to the fermi paradox.

The underlying assumption is that it exists in the first place. Or rather, one must first accept an axiom.

In fermi, its that interstellar signals can be detected and further travel is possible.

In AGI, its that intelligence is a isolateable process which we can bootstrap in minimal time.

Both assumes human progress are templates of unlimited exponential growth.

The bitter lesson was "general methods that leverage computation" win rather than more data. Like rather than just LLMs you could maybe try applying something like AlphaEvolve to finding better algorithms/systems (https://news.ycombinator.com/item?id=43985489).

Sometimes I think the fundamental thing could be as ‘simple’ as something like introducing l an attention/event loo, flush to memory, emotion driven motivation. There are quite a few fairly obvious things that llms don’t have that might be best not to add just in case.

Indeed. The Bitter Lesson has proved true so far. This sounds like going back to the 60s expert systems concept we're trying to get away from. The author also just describes RAG. That certainly isn't AGI, which probably isn't achievable at all.

What will it scale up to if not AGI? OpenAI has a synthetic data flywheel. What are the asymptotics of this flywheel assuming no qualitative additional breakthrough?

The counter argument is that we were working with thermodynamics before knowing the theory. Famously the steam engine came before the first law of thermodynamics. Sometimes engineering is like that. Using something that you don’t understand exactly how it works.

[dead]

Too bad about all those chumps designing better, faster architectures and kernels to make models run faster...