A single, used 3090 costs more than I have ever spent on a computer.

Karrot_Kream an hour ago | parent | next [-]

I keep coming back to this: why do I need to run a local model on my own GPU? Open models can run in dedicated clouds and while, yeah, they may be more expensive per token than my own GPU, when accounting for depreciation, energy usage, and opportunity cost (money not spent on my GPU will instead sit in my portfolio appreciating with its particular blend of returns), I'm pretty sure I break even or even net lose money with a GPU.

Don't get me wrong, there are advantages to a fully local model in that, I can have agents looping 24/7 even when my internet is not working. But this is niche enough that if I had to price the advantages they don't seem worth it.

If I'm willing to pay the Openrouter tax, I can fire up Openrouter today and just get access to whatever model I want, and still pay a fraction for tokens as what I'm paying with the big guys.

▲

9cb14c1ec0 5 hours ago | parent | prev | next [-]

Yes, the tunnel vision around local models on this site is crazy. The percentage of people in the world who can afford the hardware is extremely low.

▲

layer8 5 hours ago | parent | next [-]

It seems roughly similar to the pricing level of personal computers in the early eighties (i.e. IBM PC and Apple Macintosh). I’d expect prices to come down significantly over the next few years. Not so much in the next year or two, but after that.

▲

oblio 4 hours ago | parent [-]

Just like for warships, the complexity and cost of building cutting edge hardware has grown exponentially up to a point where a significant chunk of the world's computing is dependent on 2 companies: ASML, TSMC. We shouldn't extrapolate linearly from examples from the 80s.

	▲	layer8 2 hours ago \| parent [-]
		No, but I wouldn’t expect it to stagnate like with Intel in the 2010s either. Maybe the biggest caveat is that most people will be fine with using cloud providers, so the market for non-server hardware won’t be subject to as much competition.

▲

regularfry 3 hours ago | parent | prev | next [-]

There's a non-small contingent who lucked into the periodic games machine upgrade at the right time to snag a {3,4,5}090 rig just before everything exploded. It's a small contingent now but it was less so then. And now those people can add a second card for roughly what that whole system would have cost new originally.

	▲	hparadiz 3 hours ago \| parent [-]
		The current supply chain problems will eventually pass.

▲

jermaustin1 4 hours ago | parent | prev | next [-]

I don't think there is tunnel vision. I'm just saying that I have a couple 3090s I invested in a handful of years ago, and they are still going strong today as multiple GPU-needing technologies emerged.

I'm not saying everyone has to run local LLMs, because the APIs are in a race to the bottom, and my $10 of OpenRouter credits I bought months ago is down to $8.94 because most models give you MILLIONS of tokens for a US Quarter.

▲

BoxOfRain 4 hours ago | parent | prev | next [-]

It's a decreasing pool as well I'd say, the dev machine I built last summer would make less financial sense to me now for example.

▲

zahlman 4 hours ago | parent | prev [-]

I mean, I'm not rushing out to buy that kind of hardware myself, but it is a matter of perspective. People commonly spend an order of magnitude more on a car, and that's just the sticker price.

▲

throwaway219450 38 minutes ago | parent | prev | next [-]

Unless you value privacy, pay for openrouter. You still get the benefits of cheap tokens and programmatic usage.

3090 pricing is something of a wild card. Since the only big-mem consume cards are the xx90s, and a 5090 is pushing $5000, resale value has gone way up. The bottom hit ~$700 last year. It's still a very good GPU, if power hungry.

▲

wafflemaker 5 hours ago | parent | prev [-]

My single 3080 runs so hot I don't need to warm my room in winter, and have to play games in my underwear in summer.