How do a few extra requests with the same prefix make it more expensive for everyone else?

SwellJoe 2 hours ago | parent | next [-]

Keeping a conversation with a very large model active requires hundreds of GB of memory. If my conversation can never be swapped out, like when I go to lunch or take an afternoon walk, that's several hundred GB not available for other users. If everybody does that, Anthropic needs even more infrastructure than the quarter trillion dollars in infra they're already using.

▲

s0ck_r4w 2 hours ago | parent [-]

The idea here is that your conversation is being swapped out for no good reason. The session is still active, as the subagents continue to make progress.

How is this comparable to going to lunch or taking a walk?

▲

SwellJoe 2 hours ago | parent [-]

Is that actually happening, though?

	▲	devnonymous an hour ago \| parent [-]
		Yes https://github.com/izeigerman/claude-thermos/blob/main/src%2...

▲

sznio 2 hours ago | parent | prev [-]

you use more memory?