You sure can, https://github.com/lxe/dsx gets you this deepseek on 2x3090s and 128 gb ram at about 30 t/s

At around 75% of the full model intelligence

About half the t/s of an online provider, heat, noise, power consumption and on top of that you only get 75% of the intelligence? Sounds like a waste of time to me.

▲

akmarinov 6 hours ago | parent | next [-]

Half the token/s - sure, no way around beating big ass data centers.

Heat, noise and power consumption are actually quite ok - i'm running 2x3090s, capped at 250W and i haven't seen heat and noise being an issue, sitting right next to it. Might be helping that it's getting colder.

75% of the intelligence with 2x3090s, with a 5090 if you have it it's around 90% and these are first day builds too, typically people hone in and improve these all the time.

It's true local doesn't make sense value wise against online, but if you have the hardware, having "unlimited" tokens with no 5h limit, no weekly limit, is super nice.

	▲	origini 2 hours ago \| parent [-]
		I have not capped my 2x4090s so my room gets cosy warm. But here my price per kwh is 40 cents (euro). But if you integrated it with solar system and a heat exchange, this will be feasable

▲

nirvdrum 6 hours ago | parent | prev | next [-]

As luck would have it, you can just pay the provider and use their service. Others may want to run locally for privacy reasons or just for the fun of it. I think at-home AI is incredibly awesome and I applaud the enthusiasts pushing it forward. Just because that's where it's at today doesn't mean it's the best we can do.

Plus, you know, this thread was about running AI on "lame consumer hardware". There are more power efficient options available.

	▲	egeozcan 5 hours ago \| parent [-]
		I use only providers because I don't have the hardware, but if, and that's a big if, the only moat left for humans is our ability to steer and make bigger picture decisions with our personality, I think it makes sense not to want to send all that conversation data to a 3rd party.

▲

celrod 6 hours ago | parent | prev [-]

[dead]