I rather enjoy OpenAI's focus on cost-per-task.

I'm optimistic about them tackling frontier intelligence next. Astra 6.1 please?

mapontosevenths 4 hours ago | parent | next [-]

They're doing its because they recently cut plan usage by about 50%, leaving Anthropic providing about 5x the value on their plans. [0]

The "cost per task" metric is just a weaselly way to sound like they didn't just double the cost for most users, and aren't having their proverbial lunch money stolen. They can no longer claim to provide more tokens, more capable models, or more usage in general. So they made up a new metric where "task" could be defined to mean whatever they need it to mean.

They had to become VERY specific in their wording to present any kind of value proposition that will entice people.

[0] https://newsletter.semianalysis.com/p/anthropic-subscription...

▲

Archit3ch 4 hours ago | parent | next [-]

> leaving Anthropic providing about 5x the value on their plans

5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.

> like they didn't just double the cost for most users

Most users aren't on their $200 plan. Most users (including on the $200 plan) continue being subsidized.

▲

mapontosevenths 2 hours ago | parent [-]

> 5x is calculated in terms of API costs, nobody pays those extreme Anthropic prices outside enterprise users.

Ignore value then. Anthropic still provides about 3x as many raw tokens. The 20x\200 plans provide about 10B for openai versus almost 30B for Anthropic.

Do you really think that Opus uses 3x as many tokens as Sol per task? I don't.

> Most users aren't on their $200 plan.

Fair enough. I should have said most heavy users.

> Most users (including on the $200 plan) continue being subsidized.

They continue to be fifty percent less subsidized, I guess.

Look, I don't fault them for trying to become profitable. I fault them for the weird weaselly business speak about cost per task. It's BS game playing and we all know it.

I'm honestly rooting for them, and hope it all works out. For now though I'm pretty salty that my bill jumped by 150% in one month. Gaslighting about made up metrics just makes me more angry.

	▲	ComputerGuru 2 hours ago \| parent [-]
		People in HN live in a bubble. Plenty of real world use cannot use subscription plans (not even talking about enterprise agentic use) and is absolutely at the mercy of the cost per task metric. We agonize over cost per task to the extreme for services/functionality built atop of APIs and can absolutely guarantee you that it is the real world metric that counts most right after “is the model good enough for the task at hand,” and furthermore that, yes, in fact, it is quite often that we see certain models/providers easily using 3x or more tokens on average for the same tasks (or even the exact same task) than a supposedly similarly matched model from another provider. Once you become intimately familiar with the thinking traces from specific models, you can preempt this to some extent with manipulations to the system prompt (essentially heading off entire thinking branches) but the nuking of reasoning traces in the name of preventing against distillation attacks has made this harder.

▲

_aavaa_ 4 hours ago | parent | prev [-]

Cost per task, time per task, quality (good luck measuring objectively between models) are the metric that matter. I don't care how much a token costs, I'm not buying tokens, I'm buying solutions.

	▲	wj 2 minutes ago \| parent \| next [-]
		Exactly! I called that the Operations Equation and wrote a tiny bit more here about it: https://zero2data.substack.com/p/the-operations-equation Once you are inserting models into non-creative (and software development is creative work) it is imperative to start creating your own small evals. Not just to check how different models perform in that equation, but to catch model drift over time.
	▲	agoodusername63 32 minutes ago \| parent \| prev \| next [-]
		the point is that "task" is a dubiously defined unit that the frontier labs adjust to whatever makes them look good.
	▲	mapontosevenths 3 hours ago \| parent \| prev [-]
		Are OpenAI's models delivering 50% (or more) completed tasks than they were last month? Are they delivering 50% more than Anthropic per token? Nope to both. Not even close. The only way it works is if you define "task" in a way that OpenAI is good at it and Anthropic isn't.

▲

oh_no 3 hours ago | parent | prev [-]

I kind of expect them to go right to Astra 6.2, 6.1 was cancelled over the cybersecurity hubbub. I eagerly await either one.