I really want capable models local but it’s impossible for me to afford the hardware. If there is a magical way to run glm 5.3 without quantization at 80-100tps with a total spend of less than 10K I’d be happy but that’s impossible now.

▲

maipen 4 hours ago | parent [-]

$200 a month is only $2400 a year. That’s 4.16 years until you spend 10k.

I don’t find it reasonable at all. Subsidized Subscriptions or api for cheaper models, is the best value.

But the future of AI is so uncertain, regarding model sizes. I don’t think making huge hardware investments is reasonable for individuals.

▲

EddieRingle 3 hours ago | parent [-]

> $200 a month is only $2400 a year. That’s 4.16 years until you spend 10k. I don’t find it reasonable at all. Subsidized Subscriptions or api for cheaper models, is the best value.

If you buy your own hardware, then after those four years you still have that hardware, and it likely still holds significant value. Plus you know the local model you're running today won't change or suddenly degrade in four months, let alone four years. Whereas after four years of paying subscriptions and API costs, you have literally nothing.

▲

cadamsdotcom 2 hours ago | parent | next [-]

Energy costs (including capex if you buy batteries or solar) complicate things.

▲

SoftTalker 3 hours ago | parent | prev [-]

Four year old hardware has almost zero value.

	▲	D13Fd 2 hours ago \| parent \| next [-]
		Please send me all of your 4090's.
	▲	argee 2 hours ago \| parent \| prev \| next [-]
		Oh? How come the four year old 4090 RTX cannot be purchased for less than (or even at) MSRP then? Can you buy me one? I'm willing to pay $500 even if it is worth zero in your eyes, that's a stellar deal.
	▲	Espressosaurus 2 hours ago \| parent \| prev [-]
		Tell that to the 3090s still selling at $1500 how many years after manufacture, used?