To me LLM are one of the greatest advancement in gaming. I don’t understand how there isn’t already any big game that’s using it to have ‘live’ npc.

It would make single player game so much more alive, even railroaded one like red dead redemption because the npc could adapt more easily to what you just did, how you behave in game and so on.

Games are already demanding on gpu, running a very tiny local LLM and only when people interact with npc wouldn’t require so much power.

I’m sure there are issues which explain why it’s not already being used but the first one to do it will be remembered.

▲

netruk44 4 days ago | parent [-]

I wanted the same, so I hacked/bolted on an LLM to Morrowind (the open source recreation OpenMW).

The biggest problem I faced at the time (during ChatGPT 3 era) was that, without a good context, LLMs are the most vanilla roleplayers you’ve ever seen. By themselves, LLMs are just not interesting enough for a player to choose to talk to in-game.

If you want them to be “interesting” to talk to, you must provide (or generate and keep track of): a backstory, chat history, the scene, NPC inventory, the NPC’s current emotional state, the weather, literally everything needs to be given to the model before it generates messages for the player.

At which point you’ve got a big task. You need a way to automatically get the relevant data to the model for the specific conversation you’re having. There might be tools to pick appropriate text documents from a db given a conversation topic, but I didn’t/don’t know how to make that work for games.

I’m sure there’s a way to accomplish this with more modern tools and models. (Maybe instead of providing all that data up front, you would now give the model tools to call to retrieve that data on-demand?) But that’s what made me give up in 2022.

	▲	aucisson_masque 4 days ago \| parent [-]
		Of course it would require the game engine to provide context to the LLM, it certainly is quite a bit of work but nothing technically impossible and the result are endless possibilities and endless replayabilities. You could very well let the LLM decide if an npc should agree to give an object to the player or not, even let the player try to convince the npc. I mean it looks like to me the next big step in gaming after 3D and yet this is being ignored.