We used to use markov chain generators of various types to spit out absolute insanity, in the early years of the century, and this is all I ever wanted from LLMs. I bet if you used a slightly older model for your text generation, you'd get really clean output. Sonnet 4.whatever, I think, actually does a superb job at text generation, despite it being only an ok coding model.

Author here. Surprised to see this posted on HN. we had a Markov chain irc bot who was quite a jokester

https://charlesleifer.com/blog/building-markov-chain-irc-bot...

Extremely small or very heavily quantized LLMs can generate hilarious results. You can also get wild shit if you dial temperature really high.