| ▲ | herf 3 hours ago | |||||||
I love local AI, but even so, I admit the economics point the other way. Unlike gaming, AI is a uniquely good match for data centers because the applications today aren't latency-sensitive at all (except perhaps DLSS/self-driving/robotics). Most importantly, utilization is massively better with always-on servers doing parallelism (in-flight batching). Also in some areas (like California) our local electricity is quite expensive, so moving the work 100ms away can help. The privacy implications are horrifying, though. | ||||||||
| ▲ | younglunaman 3 hours ago | parent | next [-] | |||||||
I tend to disagree here. Think about how useful it would be to have a nice llm in real-time debugging your applications loaded entirely on your computer. That sounds pretty sweet to me. I don't see how that's not really cool / immediately useful to you? Maybe your point is the majority of the time that's not needed, fair, then maybe the approach is both? | ||||||||
| ||||||||
| ▲ | drdexebtjl 3 hours ago | parent | prev | next [-] | |||||||
> the applications today aren't latency-sensitive at all This is circular reasoning. We don’t have latency-sensitive applications because sufficiently powerful local AI is not profitable to entertain such applications. | ||||||||
| ||||||||
| ▲ | musha68k 3 hours ago | parent | prev [-] | |||||||
The economics point the other way because capital is aligned against you. Should this be a bubble that pops we will hopefully see prices come down again. MooresLawIsDead and his spidey web / senses thinks it already popped: | ||||||||