This is really cool! I'd like to understand how you're dealing with the costs. Since you're using an LLM and this is getting viral on HN, aren't API costs getting higher? I'm working on some LLM project myself and trying to figure out how to make the demo accessible without spending too much.

Great job.

▲

alecf 2 days ago | parent [-]

At least for the sake of this demo, the costs are actually pretty small, especially given that many of the ... even at $1.75/1M tokens you'd have to have 2 megabytes of text across all users before this even broke $1...

▲

storystarling 2 days ago | parent [-]

The unit price looks low but the cost driver in these apps is usually the context window. You have to re-send the full history every turn to maintain state, so you end up paying for the same tokens dozens of times. By the end of a long session the cost per interaction is significantly higher than at the start since you're re-processing the entire game log for every new command.

	▲	selcuka 2 days ago \| parent [-]
		Cached tokens are cheaper. Also there are ways to compress the context (not sure if this game employs any of those techniques).