If you're going to do that you need completely different LLMs to base the agents on. The ones I've tried have "mode collapse" - ask them to emulate different agents and they'll all end up behaving the same way. Simple example, if you ask it to write different stories they'll usually end up having the same character names.

It may depend on the domain. I tend to use LLMs for things that are less open ended, more categorization and summarization response than pure novel creation.

In these situations, I’ve been able to sufficiently program the agent that I haven’t seen too much of an issue as you described. Consistency is a feature.