No, they use "the same cadence and cliches" because they inflate a short and ambiguous prompt into long and specific prose by making statistical assumptions about what best fills in the gaps. It's not a training problem, it's an information theory problem, and it's not really surmountable.

Any given model will always have some distinct implicit voice that its biased towards for that infill content, and so a popular model will always become exhaustingly common, painfully familiar, and cliche. Users can use more elaborate prompts that shift the voice away from the most normative and towards some other nodes, but they need to put in special effort for that, and what people-at-scale specifically want from these tools is to put in very little effort, so we can expect that overwhelming number of casual and naive users will always be generating cliche slop with them.

Code escapes this problem not because of training but because it specifically benefits from cliche (boilerplate, patterns, etc) and so an model whose code "voice" reflects your own taste as a coder (or your toolchain's taste as a vibecoder) is going to feel like productive output rather than slop. But it's still cliche.

No. Reinforcement Learning is doing a lot here. Anyone who played with these models before the Davinci intstruct-tuning (completion) era can tell you the same. In some ways, SOTA models have gotten better at writing, but the neuroticism of instruct-tuning has still not been resolved.

> No, they use "the same cadence and cliches" because they inflate a short and ambiguous prompt into long and specific prose by making statistical assumptions

Even if LLM output has to largely follow some statistical rules, yet, first of all, some amount of randomness is normally injected during token generation, and, secondly same true for human speech.

> about what best fills in the gaps. It's not a training problem, it's an information theory problem, and it's not really surmountable.

This is not true, as LLM has internal knowledge storet in its weight. Unless you force it to produce 2000 words doc out of 3 word prompt, you would end up adding some sense information.

>Users can use more elaborate prompts that shift the voice away from the most normative and towards some other nodes, but they need to put in special effort for that, and what people-at-scale specifically want from these tools is to put in very little effort, so we can expect that overwhelming number of casual and naive users will always be generating cliche slop with them.

True, here I agree with you. But using finetuned or simply less popular models like Kimi, Hy etc. should take care of that.