> write novel length stories

This would never work. Anything longer than 1500 words gonna be bad. To get proper quality you should generate piece by piece then stitch.

Sure, one shot generation doesn't work, you have to split into sections and generate each section independently, then usually do a "seam" pass to check continuity between two bits.

Though, I'd also agree that if you're not providing feedback between each piece, the result is gonna suck, or at least, it's not really going to be "more than the sum of its prompt".

I experimented with "introducing randomness" in the form of web search + using older LLMs like EleutherAI's GPT-{J,NeoX} to try and inject novelty into the generation, but I never really got that to work either.