Imagine you're a mid sized company and you can host this model locally. Suddenly there are zero reasons to pay a single red cent to the bloodsucking American AI cartel.

▲

cavemandaveman 2 hours ago | parent | next [-]

Can you host the model for a lower cost per token than you'd pay Anthropic or OpenAI for a similar level of intelligence? I doubt you're beating their efficiencies of scale.

▲

zbendefy an hour ago | parent | next [-]

I dont have estimates on the cost of running models, but I think openai and anthropic are running on subsidized prices. At actual prices it might be worth it in the future.

▲

anthonypasq an hour ago | parent [-]

how is this idea still so persistent? The fact people are able to run open models with about the same performance at 1/10th the cost should make it glaringly obvious that Anthropic has massive inference margins at api pricing.

	▲	entrope an hour ago \| parent [-]
		I think the idea conflates price discrimination -- where people on individual subscriptions pay a much lower price per token than corporate accounts pay -- with using venture capital funding for opex. Both are subsidies in some senses, but the former is sustainable indefinitely.

▲

criley2 2 hours ago | parent | prev [-]

No, and the reason is simple: Usage is bursty and if you don't maximize usage of the hardware you're going to lose on price.

Ok you can host this model once. What if I want a dozen subagents? Ok you can host it 12 times at once. What if we go a whole week only using max 4 at a time? Etc etc. The limits imposed by self-hosting might be bearable for a variety of reasons, but it's going to be more expensive and less convenient/useful.

▲

bhouston 2 hours ago | parent | prev | next [-]

Whether it is "open" or not seems to be in question. While it was initially called an "open" model, it seems that "open" mentions have been scrubbed from website.

▲

kingleopold 2 hours ago | parent | prev [-]

hardware, electricity cost and other extra time consuming deployment, are they joke to you? ROI needs to positive otherwise open models have still BIG COST.