Remix clone Hacker News

>but I don't think that LLMs can create synthetic a priori knowledge.

Do you think that a LLM has the ability to identify a new a priori knowledge?

It seems like it would be a lower threshold to meet but If you combine that with a stochastic process then it seems inevitable that it would be able to ruminate until it came up with new a priori knowledge.

▲

viccis a day ago | parent [-]

I've said this in another comment but an example would be to train an LLM on a corpus with ALL mathematic content removed. Nothing at all. Then ask it what the shortest distance between two points is. That's an example of synthetic a priori knowledge.

	▲	Lerc 16 hours ago \| parent [-]
		I think it would be nearly impossible to prove that your corpus had no mathematical content. In fact it would be extremely contentious as to what was considered mathematical content. Do you remove all reference to numbers and counting? How about the words and, or and not. I think the only criteria of no math that would satisfy some people would by definition fail because the model would have no concept of points or distance because some would certainly count those as mathematical content.