The people who refer to LLMs as stochastic parrots generally do so to imply significant limits on an LLMs ability, not as an abstract statement about the underlying mechanism of how they work. Probably the defining thing that is surprising about LLMs is that they do in fact gain significantly more capability than you would expect from such a simple underlying mechanism!