Can you host the model for a lower cost per token than you'd pay Anthropic or OpenAI for a similar level of intelligence? I doubt you're beating their efficiencies of scale.

▲

zbendefy 2 hours ago | parent | next [-]

I dont have estimates on the cost of running models, but I think openai and anthropic are running on subsidized prices. At actual prices it might be worth it in the future.

▲

anthonypasq 2 hours ago | parent [-]

how is this idea still so persistent? The fact people are able to run open models with about the same performance at 1/10th the cost should make it glaringly obvious that Anthropic has massive inference margins at api pricing.

	▲	entrope 2 hours ago \| parent [-]
		I think the idea conflates price discrimination -- where people on individual subscriptions pay a much lower price per token than corporate accounts pay -- with using venture capital funding for opex. Both are subsidies in some senses, but the former is sustainable indefinitely.

▲

criley2 3 hours ago | parent | prev [-]

No, and the reason is simple: Usage is bursty and if you don't maximize usage of the hardware you're going to lose on price.

Ok you can host this model once. What if I want a dozen subagents? Ok you can host it 12 times at once. What if we go a whole week only using max 4 at a time? Etc etc. The limits imposed by self-hosting might be bearable for a variety of reasons, but it's going to be more expensive and less convenient/useful.