What's the relationship with the Platonic Representation Hypothesis?

unionjack22 2 days ago | parent | next [-]

I hope someone much smarter than I answers this. I’ve been noticing an uptick platonic and neo-platonic discourse in the zeitgeist and am wondering if we’re converging on something profound.

	▲	nowittyusername 2 days ago \| parent \| next [-]
		I've been noticing that as well....
	▲	2 days ago \| parent \| prev [-]
		[deleted]

▲

MarkusQ 2 days ago | parent | prev | next [-]

From what I can tell, they are very closely related (i.e. the shared representational structures would likely make good candidates for Platonic representations, or rather, representations of Platonic categories). In any case, it seems like there should be some sort of interesting mapping between the two.

	▲	brillcleaner 2 days ago \| parent [-]
		My first thought was that this was somehow distilling universal knowledge. Platonic ideals. Truth. Beauty. Then I realized- this was basically just saying that given some “common sense”, the learning essence of a model is the most important piece, and a lot of learned data is garbage and doesn’t help with many tasks. That’s not some ultimate truth, that’s just optimization. It’s still a faulty LLM, just more efficient for some tasks.

▲

altairprime 2 days ago | parent | prev [-]

Same hat, except 18 months later, assuming it survives peer review, reproduction, etc. (or: "The newer one proposes evidence that appears to support the older one.")

https://arxiv.org/abs/2405.07987