Einstein's discoveries were based (to a large degree) on negating very specific parts of scientific orthodoxy and then taking the steps forward to carefully derive results with those rejections in place.

LLMs are aggressively trained to reproduce facts and consequently struggle to reject orthodoxy. There isn't any reason they can't, in principal, make big new discoveries just by getting lucky, which is sort of also how humans do it, but its ok to acknowledge that current AIs aren't so good at certain things.

That doesn't disagree with this article. Proving a theorem that a human already proposed in an existing discipline of math - math, the most formalized and easiest discipline to involve computers in even before LLMs - is very different from expanding the boundaries of science.

They interpolate data in an XYZ dimensional space. The implications of that is beyond our comprehension.

I have a hard time believing that all novel concepts yet to be discovered are contained within that space, though.