How exactly will models get cheaper?

▲ himata4113 4 days ago | parent | next [-]

Compare the performance of a 980 and a 5050 and I am sure that will answer your question.

Also models baked into the silicon are able to achieve efficiency that is simply impossible to achieve with programmable circuits, there is a general slowdown in the raw capabilities that transformers can achieve and agentic tool use is simply an amplifier that will reach a wall eventually. It wouldn't surprise me if we saw within 5 to 10 years accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s as well as api costs going down to what we already see with subscriptions today.

There has been quite a lot of off-ramping going on where people feel satisfied with the performance they're getting out of the models and simply staying there instead of using SOTA.

▲ andrekandre 4 days ago | parent [-]

  > accelerator cards that you're able to purchase and plug into via usb-c that are able to achieve thousands of tok/s

how do you update that baked-in model for things that have happened in the last say 2 months?

if i'm a programmer for example, even being a couple months old is a huge annoyance because programming languages and frameworks are changing all the time...

	▲	HDBaseT 4 days ago \| parent [-]
		When was the last time you heard about someone talking about "Knowledge Cutoff" dates? OpenAI used to make a huge deal about it every release, now its not even mentioned. We give agents tools, the ability to read a man page, the ability to use web search. Knowledge cut-off is far less important than it used to be.

▲ 4 days ago | parent | prev [-]

[deleted]