If you looked at a graph of GPU power in consumer hardware and model capability per billion parameters over time, it seems inevitable that in the next few years a "good enough" model will run on entry-level hardware.

Of course there will always be larger flagship models, but if you can count on decent on-device inference, it materially changes what you can build.

▲

physicsguy 6 hours ago | parent | next [-]

It also massively changes the value economics of the frontier models. In a lot of cases, you really don't need a general purpose intelligence model too.

	▲	bensyverson 5 hours ago \| parent [-]
		Exactly… as hn readers, we sometimes forget that a lot of people are using these tools to search for the best sunscreen, or rewrite an email.

▲

dakolli 5 hours ago | parent | prev [-]

No offense, this is a crazy delusional statement.

▲

5 hours ago | parent | next [-]

[deleted]

▲

afro88 5 hours ago | parent | prev [-]

No offense, this is a crazy worthless contribution to the discussion.

Why?

▲

dakolli 4 hours ago | parent [-]

Because everyone in these replies is in complete denial about the physical limits of memory and scaling in general. Ya'll literally living in an alternate reality where model capability increases with a decrease in size, its simply not the case. There will be small focused models that preform well on very narrow tasks, yes, but you will not have "agents" capable of "building most things" running on consumer hardware until more capable (and affordable) consumer hardware exists.

▲

bensyverson 4 hours ago | parent [-]

Ah, you haven't realized that consumer hardware gets more capable over time

	▲	adrian_b 2 hours ago \| parent [-]
		Not this year, when many vendors either offer lower memory capacities or demand higher prices for their devices.