So basically the map projection problem [1] in higher dimensions?

[1] https://en.m.wikipedia.org/wiki/Map_projection

Worse. Map projection means that you cannot have a mapping that preserves elements of the internal geometry: angles and such.

Violation of topology means that a surface wrongly is mapped to one intersecting itself: Think Klein Bottle.

https://en.wikipedia.org/wiki/Klein_bottle

▲

bravura a day ago | parent [-]

Can you share an actual example demonstrating this potential pathology?

Like many things in ML, this might be a problem in theory but empirically it isn’t important, or is very low on the stack rank of issues with our models.

	▲	Nevermark 16 hours ago \| parent [-]
		A fold means two different regions of topology get projected across each other. It's a problem for the simplest of reasons, information is lost. You cannot reconstruct the original topology. In terms of the model, it now can't distinguish between what were completely different regions. From the Klein bottle perspective, a 4D shape gets projected into a 3D shape. On most of the bottle, there is still a 1 to 1 topological mapping from 3D to 4D versions. But where two surfaces now intersect, there is now no way to distinguish between previously unrelated information. The model won’t be able to anything sensible with that. TLDR; We don't like folding.