It's interesting that you chose to measure how well each LLM did in talking to other NPCs, and having each NPC also use an LLM to react to each input. Why not have the LLM fight NPCs and loot items (both a significant part of the MUD experience, neither requiring LLMs on the server side), then measure character progression in experience and stats?

Just starting small. Those are all ideas for Phases 2 & 3.