Not specifically Anthropic but why are we allowing billing to take place in tokens that are nebulous and fully controlled by the operators who have no aligned incentives?

If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?

Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.

*to clarify my rambling... We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.

▲

bob1029 4 hours ago | parent | next [-]

> the operators who have no aligned incentives

The model providers are quite aligned with concerns like customer retention. These arguments only work if there is no competition. We exist in a marketplace of black boxes. There's not just "the one" you must suffer. You have options. You can build your own too.

▲

boredumb 4 hours ago | parent [-]

there are roughly three of them and they all use the same pricing model. I am also not in the position to build a frontier model company these days.

	▲	olibhel an hour ago \| parent \| next [-]
		You could try: 1. Self hosting 2. Chinese models 3. Running it locally. Requires upfront cost and compromises on TPS.
	▲	hn_throwaway_99 2 hours ago \| parent \| prev \| next [-]
		A per-token model roughly aligns with the providers' costs, and it is an objective measure, so it seems a reasonable way to charge. I see posts all the time on HN about which models from which providers offer the most bang-for-the-buck, and how to minimize token usage and still get optimal results, so it appears that competition is working.
	▲	surgical_fire 3 hours ago \| parent \| prev [-]
		There are more than 3. Hell, I use 3 different providers, and I currently don't give a dime to Anthropic or OpenAI.

▲

demibabs 4 hours ago | parent | prev | next [-]

How else would they bill tho? Their operating cost is per token.

▲

dijit 4 hours ago | parent | next [-]

Charge on the input tokens, then you will naturally optimise for fewer output tokens.

Theoretically.

In reality, one sessions output tokens become the next sessions input tokens (at least if you continue the topic) so, its not as aligned as all that.

But the parent is right, when incentives are not aligned, friction will happen. Its inevitable.

▲

demibabs 4 hours ago | parent | next [-]

“Claude, spend the next 10 hours trying to solve the Reimann Hypothesis”.

I agree that incentives are misaligned but there’s several competing model providers. If one gets funny with their costs people will jump ship, especially if the gap between the top 2 labs and everyone else keeps shrinking.

▲

dijit 3 hours ago | parent [-]

“I can take current sources and tell you how solved this is, but I am not willing to work to a timeframe or to solve things that aren’t yet solved by mathematicians or science”

These safeguards already exist when they get a whiff that you might be using Claude to fix security issues. Doesn’t seem farfetched given the incentives I outlined that they would apply to this kind of abuse.

How loose those controls are becomes a market force.

▲

dullcrisp 3 hours ago | parent [-]

Perfect, a coding agent that refuses to do things that haven’t already been done before.

	▲	dijit 3 hours ago \| parent [-]
		Hahahaha, I think you’ve misunderstood what LLMs are.

▲

cthaeh 4 hours ago | parent | prev [-]

[dead]

▲

xmcp123 4 hours ago | parent | prev | next [-]

NeuralWatt just does it on energy consumption.

▲

ACCount37 4 hours ago | parent | prev [-]

And tokens can be metered reliably. Unlike something like "task completion".

▲

3 hours ago | parent | prev | next [-]

[deleted]

▲

eh_why_not an hour ago | parent | prev | next [-]

One guess is that their "primary" target audience/market is the large corporations that get their employees unlimited tokens, and not the individual developer who may worry about spending and token accounting.

	▲	taude an hour ago \| parent [-]
		It's the opposite. The enterprises have all the tooling to monitor token usage of employees, and to limit access. For example, we have a $300 month limit, and then need to file exception tickets when we need more to justify the cost. Pretty similar at other non-silicon valley company process. I don't know any enterprise who'se on unlimitaged token budget for their employees. that's not how enterprises sign contracts. https://code.claude.com/docs/en/admin-setup#set-up-usage-vis...

▲

daishi55 4 hours ago | parent | prev [-]

For OAI and Anthropic at least you can set a spend limit per response. Also tokens are well-defined.

▲

boredumb 3 hours ago | parent [-]

I'm not worried about the volatility in the definition, i'm worried that I give it 1 token today and receive 2 token output, tomorrow I receive 40. If i'm doing this a hundred thousand times a day it is difficult to price this in for users downstream or in the extreme cases be able to absorb that at all short of going into a failmode with degraded access until someone goes and buys more tokens or gets the bill. The alternative is just pass the buck and bill your non-technical customers with a "tokens" line iteim every month.

	▲	Aurornis 2 hours ago \| parent \| next [-]
		> If i'm doing this a hundred thousand times a day it is difficult to price this in When you’re doing this 100K times per day you get an extremely good idea of what it costs. You also have all the tools to see when something starts changing quickly. This change is for Claude Code the harness. If you’re using the API at scale and paying full price then you get exactly what you put into the request.
	▲	sroussey 3 hours ago \| parent \| prev [-]
		No, those doing this 100k times a day have very good data on this, good estimators and modeling. And the API has various knobs to change and evals will give you actionable data.