Ah right, you don't like AI and don't care to understand how it works.

bunderbunder 3 hours ago | parent | next [-]

I’ve been working in AI - and specifically NLP - since 2003. I am no stranger to how weird quirks can sneak into overparametrized models, nor am I a stranger to how good humans can be at inferring meaning where there is none in specific language model behaviors. So, yeah, I am inclined to assume non-teleological causes are more parsimonious than inferring the presence of a strange loop, because that continues to be the winning bet. Even for generative LLMs.

▲

bigfishrunning 3 hours ago | parent | prev [-]

Ah right, so you like AI and don't care to understand how it works.

It doesn't "decide" anything or "need" any semantic. It derives the likelihood of the token, and "bearing" is likely to come after "load".

▲

Myrmornis 3 hours ago | parent [-]

Sure but the question is why "load" after X?

	▲	bigfishrunning 3 hours ago \| parent \| next [-]
		Because, for some high number of contexts, its likelihood comes out high in the big tree of multiplies that is claude's model. For some sets of 500 words (or whatever), the next word is "load". The classifier that decides which sets of 500 (or whatever) words is a prefix for "load" is returning "true" too often.
	▲	bunderbunder 3 hours ago \| parent \| prev [-]
		More-or-less the same principle, but scaled up massively, and with context-dependent probability conditioning maps.