> so the proof to the unit-distance problem was on the manifold, given it was outputed by a LLM?

By design this must be the case, even when you account for stochastic sampling i.e. 'temperature'. All it's outputs are a highly-dimensional combinatorial interpolation (I'm talking about GPTs here)

That's probably why Claude is very good at producing plausible nonsense rather than the often correct response of 'I don't know'.

What you’re saying here makes no sense.