But my half baked instinct is that LLMs in and of themselves will never constitute true AGI.

I agree. But... LLM's are not the only game in town. They are just one approach to AI that is currently being pursued. The current dominant approach by investment dollars, attention, and hype, to be sure. But still far from the only thing around.