> Another thing to consider about LLMs is that the nature of the training and the core capability of transformers is to mimic the function of the processes by which the training data was produced; by training on human output, these LLMs are in many cases implicitly modeling the neural processes in human brains which resulted in the data. Lots of hacks, shortcuts, low resolution "good enough" approximations, but in some cases, it's uncovering precisely the same functions that we use in processing and producing information.

I would argue this is deeply false, my classic go-to examples being that neural networks have almost no real relations to any aspects of actual brains [1] and that modeling even a single cortical neuron requires an entire, fairly deep neural network [2]. Neural nets really have nothing to do with brains, although brains may have loosely inspired the earliest MLPs. Really NNs are just very powerful and sophisticated curve (manifold) fitters.

> Could be very TARS like, lol.

I just rewatched Interstellar recently and this is such a lovely thought in response to the paper!

[1] https://en.wikipedia.org/wiki/Biological_neuron_model

[2] https://www.sciencedirect.com/science/article/pii/S089662732...

>> I would argue this is deeply false

I am making the case that this is distinctly and specifically true, for these types of models. They're eliciting many of the underlying functions and processes that brought about the data; transformers are able to model the higher degrees of abstraction that previous neural architectures could not. This was one of the major features of transformers that make them so powerful.

It's comparable to the idea that if you trained a model to output human sounding speech, many of the functions that shape the voice will correspond to the physical attributes that affect the sound of actual human voices. Volume of the mouth, shape of the lips, position of teeth, what the tongue does, etc - some of those things will be captured, others will be mashed into "good enough" , and others will be captured as an optimization possible in silicon but not for flesh and blood. It's not a one to one correspondence, but capturing process semantics and abstractions is why we have ChatGPT with transformers and not CNNs (although RNNs could have pulled it off back in the 90s, see: RWKV)

Anyway - the training methods, the paradigm of next token generation (in contrast to things like diffusion) and other aspects of LLMs restrict them to a subset of human capabilities, but it's reasonable to make the claim that many of the same functions that operate in Werncke's area and Broca's area in the human brain are resident in transformers. Many of the same associations between language and emotion and those abstract correlations - unspecified, implicit context that exists in the training data, but only as a deep subtext, sometimes even distributed across many texts, like cultural trends and so forth - are modeled by LLMs, not as an explicit feature of the data, but an implicit feature or function of the processes which produced the data.

Plus, there seems to be some support for the idea that for intelligent systems, modeling the world will result in comparable structures, networks, and features for similar concepts and knowledge - because you're modeling consistent, persistent things using modalities that are shared, or overlap, the way in which things are modeled converges on "universal" forms, simply due to constraints of utility and efficiency.

https://arxiv.org/abs/2405.07987