>This isn't how LLM works.

I know how an LLM works. I've built one. At best we only know surface level stuff like the fact that it involves a feed forward network and is using token prediction.

But the emergent effect of how it an LLM produces an overall statement that reflects high level conceptual understanding is something we don't know.

So your claim of "This isn't how an LLM works" which was said which such confidence is utterly wrong. You don't know how it works, no one does.