I have to agree. It is messed up that transformers can just talk, and it been pretty normalized. We are only talking about the impact they will have and whether they can do what people say they can, but we arent talking about how crazy it is that they can talk

▲

kirrent 11 days ago | parent | next [-]

I come back to this every so often as well. After so many years of looking at Markov chain outputs that almost looked like they made sense or chatbot systems rewriting your sentence back at you, software which can simply talk is a heady thing.

▲

Lplololopo 11 days ago | parent [-]

I would say that the LLM is something completly different especiayll as its not a normal algorithm but is very close to what brains do.

▲

yencabulator 11 days ago | parent [-]

Ah yes, matrix multiplication is "not a normal algorithm", surely.

▲

Lplololopo 11 days ago | parent [-]

The matrix multilication underneath a large language model is the hardware but the data and the forming weights is not.

A quick sort sorts a list. A LLM depends on its learning data.

You train a model and then you use the model.

▲

yencabulator 11 days ago | parent [-]

The LLM is still a "normal algorithm", just one with a fairly large dataset to use. Assigning the algorithm part of LLMs magical properties hinders understanding. The work needed to pick the next output token is very much a classical algorithm.

Algorithms can be based on training and/or use data just fine, too. https://arxiv.org/abs/1712.01208

(Now, the weights used, those we kinda really don't understand the same way we understand the processing, and the approach to looking for structures in weights sometimes looks more like archeology or anthropology than computer science.)

It sounds like you're trying to express some kind of "but LLMs are so much more" thought. Yes, very much, they are. It's because of the size of the data, there's interesting emergence there. They're still a normal algorithm. (And our brains aren't quite like that; biological things are much more random/chaotic and generally non-reproducible. And the data and algorithm aren't separate.)

	▲	Lplololopo 10 days ago \| parent [-]
		Anthropic has a blog article on how they analyse their LLM to understand better how the LLM is doing math and estimates. For this they needed extra tools to do so. This 'algorithm' of how the lLM does that, was unknown before their research. Our brains are not that chaotic though. They have even more complexity to size for sure and the issue that its hard to look into a humans brain.

▲

shepherdjerred 11 days ago | parent | prev | next [-]

LLMs have really changed the world. I didn’t think something like then would be possible in my lifetime

▲

dyauspitr 11 days ago | parent [-]

It came out of nowhere. It’s all emergent. I’m convinced this is possible with just about anything given enough data. We will be seeing a near magical physical outputs LLM in the near future. It’s going to take in video and sounds and spit out physical movements that will be just as mind blowing as when 3.5 came out and it will come out of nowhere.

	▲	zhoBEENG 10 days ago \| parent [-]
		I can't agree enough, and I am increasingly struggling to understand why people are not grasping this. It's just a matter of sensors in the right places and compute. sufficient telemetry + sufficient compute = AI solution to any problem From the Universal Approximation Theorem for neural nets, we know that if we have the right training method and net architecture we can get approximate any function with a NN. Of course, that doesn't imply that we actually have a sufficient training method and net architecture for the problem at hand, but we have been able to demonstrably solve at least two engineering domains: physical world navigation (Waymo) and language (GPT). It turns out a robust enough language model is sufficient for reasoning. Given these results, I am personally stumped to come up with a problem humans can solve now that we can't solve with a computer given the correct telemetry and sufficient compute.

▲

modzu 11 days ago | parent | prev [-]

if youve ever seen a pile of wrinkly mush and wondered.. pretty damn crazy too

https://web.mit.edu/people/dpolicar/writing/prose/text/think...