Remix.run Logo
aeyes 4 hours ago

https://arcprize.org/leaderboard

$13.62 per task - so we need another 5-10 years for the price to run this to become reasonable?

But the real question is if they just fit the model to the benchmark.

onlyrealcuzzo 3 hours ago | parent | next [-]

Why 5-10 years?

At current rates, price per equivalent output is dropping at 99.9% over 5 years.

That's basically $0.01 in 5 years.

Does it really need to be that cheap to be worth it?

Keep in mind, $0.01 in 5 years is worth less than $0.01 today.

willis936 2 hours ago | parent [-]

Wow that's incredible! Could you show your work?

onlyrealcuzzo 2 hours ago | parent [-]

https://epoch.ai/data-insights/llm-inference-price-trends

golem14 42 minutes ago | parent | prev | next [-]

A grad student hour is probably more expensive…

re-thc 3 hours ago | parent | prev | next [-]

What’s reasonable? It’s less than minimum hourly wage in some countries.

willis936 2 hours ago | parent [-]

Burned in seconds.

gowld 39 minutes ago | parent [-]

Getting the work done faster for the same money doesn't make the work more expensive.

You could slow down the inference to make the task take longer, if $/sec matters.

igravious 4 hours ago | parent | prev [-]

That's not a long time in the grand scheme of things.

throwup238 4 hours ago | parent [-]

Speak for yourself. Five years is a long time to wait for my plans of world domination.

tasuki an hour ago | parent | next [-]

This concerns me actually. With enough people (n>=2) wanting to achieve world domination, we have a problem.

gowld 38 minutes ago | parent [-]

n = 2 is Pinky and the Brain.

amelius 3 hours ago | parent | prev [-]

Yes, you better hurry.