Remix.run Logo
▲ bee_rider an hour ago

I think ultimately local LLMs will win but only because models will get good enough that the “cost” of running locally on a 2030 chip will be less than the “cost” of talking to the network. Especially given that building wider/lower precision LLM accelerators is a perfect task for hardware companies (embarrassingly parallel in an era where sequential performance improvements have long dried up).

In terms of self-hosting vs remote-hosting regular software… I mean, a lot of regular software is fundamentally about communication (social media and office suites) so that goes on the server. LLMs don’t benefit much from being remotely hosted, other than that it makes the flops run in somebody else’s machine. But that will be hard to fund in a universe where the upper-limit to what can be charged is defined by how much additional cost “good AI hardware” adds to the price of your phone. We’ll see I guess.