Why not generate some asset libraries to help with some of the rote generation? You could theoretically serve the same asset for a pack of rats to multiple campaigns.

Yes! I am doing a lot of this where it won't break the illusion. Not everyone needs a unique innkeeper generated in every town, but I want to avoid that "Officer Jenny" effect like in Pokemon where she looks the same in every town they visit.

I have a private vs public flag for assets that I'm considering more unique or sensitive, at the AI GM's discretion. I'm using embeddings from there to try and parse if an asset already exists in the public pool or not, and reuse it if possible. The thinking is that eventually I will have pretty decent asset coverage on most standard campaigns. I can't account for people going way off book though.

I have an asset pipeline that tries to determine player intent and pre-generate assets before they're needed. That way we can attempt to hide the "load screens" like retro games did with elevators. I have a kind of sliding scale for player coherency, and if the player has too many "misses" on the pre-generation pipeline it will increase its requirements for when it starts generating.

I may have wildly over-engineered this but I love it. =)