Remix clone Hacker News

> I couldn’t believe they whiffed it

Why should we expect a general-purpose instruction-tuned LLM to get this right in the first place? I am not at all surprised it didn't work, and I would be more than a little surprised if it did.

▲

sangnoir 9 days ago | parent [-]

> Why should we expect a general-purpose instruction-tuned LLM to get this right in the first place?

The argument goes: Language encodes knowledge, so from the vast reams of training data, the model will have encoded the fundamentals of electromagnetism. This is based in the belief that LLMs being adept at manipulating language, are therefore inchoate general intelligences, and indeed, attaining AGI is a matter of scaling parameters and/or training data on the existing LLM foundations.

▲

TheOtherHobbes 9 days ago | parent | next [-]

Which is like saying that if you read enough textbooks you'll become an engineer/physicist/ballerina/whatever.

	▲	sph 8 days ago \| parent [-]
		A huge number of people in academia believe so. The entire self-help literary genre is based upon this concept. In reality, and with my biases as self-taught person, experience is crucial. Learning on the field. 10,000 hours of practice. Something LLMs are not very good at. You train them a priori, then it's a relatively static product compared to how human brains operate and self-adjust.

▲

garyfirestorm 9 days ago | parent | prev | next [-]

This could be up for debate -

https://www.scientificamerican.com/article/you-dont-need-wor...

▲

Vampiero 8 days ago | parent | prev [-]

Yeah but language sucks at encoding the locality relations that represent a 2D picture such as a circuit diagram. Language is a fundamentally 1D concept.

And I'm baffled that HN is not picking up on that and ACTUALLY BELIEVES that you can achieve AGI with a simple language model scaled to billions of parameters.

It's as futile as trying to explain vision to a blind man using "only" a few billion words. There's simply no string of words that can create a meaningful representation in the mind of the blind man.