I get the impression that folks who have a strong negative reaction to the phrase "stochastic parrot" tend to do so because they interpret it literally or analogously (revealed in their arguments against it), when it is most useful as a metaphor.

(And, in some cases, a desire to deny the people and perspectives from which the phrase originated.)