There was a paper recently that demonstrated that you can input different human languages and the middle layers of the model end up operating on the same probabilistic vectors. It's just the encoding/decoding layers that appear to do the language management.

So the conclusion was that these middle layers have their own language and it's converting the text into this language and this decoding it. It explains why sometime the models switch to chinese when they have a lot of chinese language inputs, etc.

▲

DrewADesign 7 hours ago | parent | next [-]

Ok — that sounds more like a theory rather than an open-and-shut causal explanation, but I’ll read the paper.

▲

trenchgun 3 hours ago | parent [-]

You’re a literature cycle behind. ‘Middle-layer shared representations exist’ is the observed phenomenon; ‘why exactly they form’ is the theory.

You are also confusing ‘mechanistic explanation still incomplete’ with ‘empirical phenomenon unestablished.’ Those are not the same thing.

PS. Em dash? So you are some LLM bot trying to bait mine HN for reasoning traces? :D

▲

DrewADesign an hour ago | parent [-]

Oh, Jesus Christ. I learned to write at a college with a strict style guide that taught us how to use different types of punctuation to juxtapose two ideas in one sentence. In fact, they did/do a bunch of LLM work so if anyone ever used student data to train models, I’m probably part of the reason they do that.

You sound like you’re trying to sound impressive. Like I said, I’ll read the paper.

	▲	cyanydeez an hour ago \| parent [-]
		Congrats on reading.

▲

skydhash 5 hours ago | parent | prev [-]

Pretty obvious when you think that neural networks operate with numbers and very complex formulas (by combining several simple formulas with various weights). You can map a lot of things to number (words, colors, music notes,…) but that does not means the NN is going to provide useful results.

	▲	DrewADesign an hour ago \| parent [-]
		Everything is obvious if you ignore enough of the details/problem space. I’ll read the paper rather than rely on my own thought experiments and assumptions.