| ▲ | paxy 3 hours ago |
| I don't see any difference between the conversations around self-hosting local AI and self-hosting regular software, which have been going on for decades. Yes, there are a hundred valid reasons to self-host. But 99%+ of consumers aren't going to care. People don't even care when the cost of hosting is ~free, like a simple home server. How many are ready to buy thousands of dollars worth of AI hardware, which could very well be obsolete a couple of years from now? |
|
| ▲ | abdullahkhalids 16 minutes ago | parent | next [-] |
| One of my favorite stats is that Limewire used to be installed on 1/3rd of computers worldwide [1], i.e. people were self-hosting. Why? Because Limewire was easy enough to install that even grandma could do it. There is not a single way to self-host even a simple html website without learning half a dozen software skills. People will self-host when it is click-and-play. [1] https://marquettewire.org/3777966/tribune/marquee/the-life-a... |
|
| ▲ | wry_xy 3 hours ago | parent | prev | next [-] |
| The self-hosting crowd, Linux fans, OSS contributors, etc, are our own worst enemies. If we care about self-hosting and open source and decentralization because we believe it leads to a better world, and not simply because we want to be control freaks and snobs, then we need to learn how to build shit that is actually convenient and usable and understandable to normal people. I think it's possible. I think many people would want this, and care, if we could package it up in a digestible way. |
| |
| ▲ | lilbigdoot 2 hours ago | parent | next [-] | | Linux has been more user friendly and accessible to me for normal use than Windows for years now. I really do not understand this comment | | |
| ▲ | guhidalg 2 hours ago | parent [-] | | Talk to people not in tech. Windows never ever ever makes you open a terminal to do anything, that’s scary to non-tech people. Windows “just works” and most people just want to minimize their computer use for work and go watch tiktok on their phone. | | |
| ▲ | rangerelf an hour ago | parent | next [-] | | "Windows just works" is such a lie. | |
| ▲ | drdaeman an hour ago | parent | prev | next [-] | | Windows makes you open a terminal just to install it properly. | |
| ▲ | luqtas an hour ago | parent | prev [-] | | which scenario someone needs to open terminal if all they need is a browser and an office suite? |
|
| |
| ▲ | pelagicAustral 2 hours ago | parent | prev | next [-] | | I'd say one of the major frictions with self-hosting is that for the most part it does always going to take a lot of tinkering around and maintaining things going. A world were we have a lot more people self-hosting would be great, but as it stands, it's an utopia, because normal users will get annoyed the fuck out of the environment the second their infra is busted because of any of the million reasons things break. | | |
| ▲ | pydry 2 hours ago | parent | next [-] | | Self hosting is in pretty much the same state smartphones were in before the iPhone. I remember trying to install apps and it was annoying for all the same reasons self hosting is these days. It was niche, only hobbyists did it, "nobody cared" and the presumption was that for normal non-nerds, phones were exclusively for calls and texting. There are some spectacular successes in terms of the actual self hosted software (immich and home assistant) but the overall experience is still N95 not iPhone. The part where self hosting falls down isnt the apps it's config, networking, reliability and backups. | |
| ▲ | japbujan 2 hours ago | parent | prev [-] | | [flagged] |
| |
| ▲ | davedx 2 hours ago | parent | prev | next [-] | | Ollama and friends are pretty damn easy to use already. Hardware is the real barrier | | |
| ▲ | lurking_swe 2 hours ago | parent [-] | | let's see here, you already make a crucial mistake that a product manager wouldn't. (I say this as a senior software engineer that has worked with product managers a lot) If I tell my mom to download and install Ollama on her Mac, do you think she's gonna have any clue which model to download from all of the 5000 models she can choose from? Do you think she understands which ones will even work well on her machine? What if she downloads one that's highly recommended online but it doesn't work on her 64 GB MacBook (hypothetical example)? Because it's too too large/slow... and can Ollama, out of the box, do actual work on her computer, similar to the ChatGPT app with its "Work" mode, and dozens of built in connectors (google, etc)? you call this easy to use for a normal person? You just made their point. The ChatGPT and Claude apps that my mom can download, set the standard for her. Her standard of good/easy is not what you and I as nerds find easy. you need to really put yourself in the shoes of a tech illiterate person. we haven't even touched on the fact that Claude and ChatGPT ship with easy to use iOS and Android apps that sync up perfectly with her Mac. | | |
| ▲ | anonymousDan 2 hours ago | parent [-] | | This is a bit of a strawman argument. Who cares about your Mom (no offense)? The important part for me is that technical people (e.g. software developers etc) have access to competitive open source and/or local models. | | |
| ▲ | 2 hours ago | parent | next [-] | | [deleted] | |
| ▲ | guhidalg 2 hours ago | parent | prev [-] | | That may be personally important to you, but the market for tech-literate people is dwarfed by everyone else. | | |
| ▲ | graemep an hour ago | parent [-] | | In terms of work usage its the technical people who will install it for other people. "everyone else" does not know how to set up a server or install an OS etc. its all done for them by someone else. The consumer market for LLMs is small. compared to the business market. |
|
|
|
| |
| ▲ | petra 2 hours ago | parent | prev | next [-] | | What percentage of techies use self-hosting for a significant part of their services? Before that number is high, can we realistically expect non-techies to use self-hosting? | |
| ▲ | forshaper 2 hours ago | parent | prev | next [-] | | At some point there is a trade-off between those far away controlling "your" stuff and convenience. It is much easier for someone to make something convenient if they have control over it. For example, troubleshooting when you own the stack versus troubleshooting when you don't. However, if someone wants someone far away to help them troubleshoot, then it will be easier if the one far away controls the stuff. There is a trade-off there. | |
| ▲ | zoeysmithe 2 hours ago | parent | prev | next [-] | | Its worth mentioning "normal people" take hours upon hours of classes and trainings to use typical office productivity software and other job-related software, on top of watching and learning from tutorials and family and friends to use personal software and websites and apps. And heaven knows how many support calls and tickets to just do what is supposed to be "easy" in the closed source world. Also, none of this is intuitive and a lot of popular closed stuff is harder to use (and vastly disliked by the very same normal users) than some popular open source stuff. I'd even mention 'normies' want self hosted Word and Excel and regularly reject the web-based versions of them. Normies don't want streaming games but a gaming PC or console. Normies may buy streaming services like Netflix but get upset when their favorite shows are removed, so they actually don't like netflix. Normies complain about censorship on twitter and reddit and discord and would love an alternative they control, but may be unaware that's an option and seem stuck on those services and hate them. Normies want self-hosted all the time too. Their ability to understand and do this is what's lacking. So that's a conversation on its own, but normies aren't actually happy. Form what I can tell most people hate 50% of the stuff they use right now and most of those complaints revolve around the cloud-hosted/lack of control/privacy and other issues self-hosting or federated services or FOSS apps could arguably solve for them. This is a big conversation but imho tends to boil down to who has the marketing and sway to get you to install or use something, not that typical end users are savvy judges of software and useability working purely on merit and informed facts. Does your grandma actually like Facebook, or is she just lonely and its the lowest friction way to not be lonely? Do you really like Chrome or Word or is it just easy to get and popular? Is the big mac the best burger or is just easy to get and popular? | |
| ▲ | nemomarx 3 hours ago | parent | prev | next [-] | | There's a decent opportunity where AI coding agents seem good at creating a very "average" and understandable UI. Maybe not perfect, but probably better than Gimp? | |
| ▲ | goda90 3 hours ago | parent | prev | next [-] | | The problem with open source is that it is often written by people who are fixing issues they have or making features they miss. To get to the point of contributing, you have to go through the initial setup process. Once you've done that, it's not longer a problem you care about, so you're less motivated to contribute code to make that process easier. | |
| ▲ | betterunix2 2 hours ago | parent | prev | next [-] | | > shit that is actually convenient I mean...sure...but how convenient was that Office 365 outage? How convenient is it to find that a feature you actually depended on is gone because the company that runs your software decided to get rid of it? It is always a back-and-forth. It is not as if centralized computing is a new idea, in fact it was the original way to use computers and people found that issues of control mattered and switched to PCs; the pendulum swung back because people realized that there are advantages to centralization (e.g. not having a failed hard drive destroy high-value documents). The cycle will probably continue for a long time to come because the advantages and disadvantages of both approaches are fundamental. | | |
| ▲ | bendergarcia 2 hours ago | parent | next [-] | | That inconvenience is still less inconvenient than not being able to figure out your dns or vpn or tail scar settings or why it won’t render and why the machine isn’t reachable. When running your own services is easier than it currently is and lot of tools are built around notifications and debugging that are actually understandable, then you’ll have more adoption of open source software. | |
| ▲ | coldtea 31 minutes ago | parent | prev [-] | | >I mean...sure...but how convenient was that Office 365 outage? One outage in a blue moon? Bloody very convenient. People have more outage days per year on their own laptops. |
| |
| ▲ | templar_snow 2 hours ago | parent | prev | next [-] | | This. The FOSS/etc crowd don't even try to bother with outreach on tools that the majority of people would want if they were marketed correctly instead of kept as niche projects for people to be smug about. Consider the EU rolling out Matrix in a huge way, or people moving to Mastodon. For most normies, "Use this to escape Elon Musk" is more than enough sales pitch. | |
| ▲ | pyrale 2 hours ago | parent | prev | next [-] | | > not simply because we want to be control freaks and snobs It’s not so much about being snobs as it is about not having time and energy for a second job. The level of quality you demand simply requires a level of attention, investment and involvement that can’t be provided without working full-time on it, for profit. Telling a bunch of hobbyists that they should provide service of a quality equivalent to that of a trillion-dollar corporation if they are serious about it is gaslighting. | | |
| ▲ | dotancohen 2 hours ago | parent [-] | | I hate to say this, but Claude Code has turned my home projects and personal scripts from "I tolerate this because I need it" to "This works so nicely I might make it available to others". AI could really help here. I'll go purify myself now. |
| |
| ▲ | agile-gift0262 2 hours ago | parent | prev [-] | | > we want to be control freaks and snobs That's a mirror that hurts to look at that you just held in front of my face. Guilty on both counts, even though I try to convince myself I'm not |
|
|
| ▲ | nsbk 2 hours ago | parent | prev | next [-] |
| Hard disagree. I'm running Qwen3.8-Flash-Next using Strata on a Mac Studio and it covers all my needs. One drag and drop install. Nothing like self-hosting your email. |
| |
| ▲ | 30 minutes ago | parent | next [-] | | [deleted] | |
| ▲ | mortalapeman 2 hours ago | parent | prev | next [-] | | Seconding this. If you know how to build with a dumber model it scales crazy well. And Qwen 3.8 Flash Next is just wildly good even at low quants. I'm trying to find it's limits and I've been using it for over a week building and haven't hit a wall yet. | | |
| ▲ | choxi 2 hours ago | parent | next [-] | | Are you using it for coding? I'm looking for a good coding model to use for Xcode and NextJS apps | |
| ▲ | 3836293648 29 minutes ago | parent | prev [-] | | Really? I hit a wall immediately, though I'm using it to implement things originally written in a proof language, and wasn't that good at prompting to begin with. |
| |
| ▲ | paxy an hour ago | parent | prev | next [-] | | What are you disagreeing with exactly? Qwen 3.8 flash costs a few cents per million tokens from cloud providers. You can get effectively unlimited usage on a $6/month plan. You were okay with buying a $5000+ Mac Studio for it because you have the money, but you are in the tiny minority. | | |
| ▲ | nsbk 29 minutes ago | parent [-] | | I disagree with your statement saying there's no difference between self-hosting local AI and self-hosting regular software. Regular services (which I guess is what you're referring to) require uptime, to be accesible from the Internet, backups, security patches, the list goes on. The main argument against self-hosting has never been cost, it is a complex task that requires knowledge and dedication. On the other hand, you can self-host on a Raspberry Pi or ancient laptop for close to nothing. A local AI can be as simple as a program that runs in your computer [0]. Any machine with 12GB+ of VRAM can run the same model I'm using. I has a chat interface, can see images, control your browser, generate artifacts, work on projects and workspaces. You name it. [0] https://stratallm.org |
| |
| ▲ | xandrius 2 hours ago | parent | prev [-] | | How much was your Mac Studio? | | |
| ▲ | nsbk 26 minutes ago | parent [-] | | This is not about cost. Any machine with 12GB+ of VRAM can run Qwen3.8-Flash-Next at usable speeds using Strata |
|
|
|
| ▲ | __MatrixMan__ 2 hours ago | parent | prev | next [-] |
| > But 99%+ of consumers aren't going to care. They don't now, but they might soon. It used to be that bad behavior on the part of our service providers was annoying because you'd get ads and spam. Nowadays the leaked data gets aggregated into apps that help government spooks abduct your neighbors. The effort remains constant, the threats it helps you avoid are getting more dire. Eventually those lines will cross. |
|
| ▲ | bee_rider 2 hours ago | parent | prev | next [-] |
| I think ultimately local LLMs will win but only because models will get good enough that the “cost” of running locally on a 2030 chip will be less than the “cost” of talking to the network. Especially given that building wider/lower precision LLM accelerators is a perfect task for hardware companies (embarrassingly parallel in an era where sequential performance improvements have long dried up). In terms of self-hosting vs remote-hosting regular software… I mean, a lot of regular software is fundamentally about communication (social media and office suites) so that goes on the server. LLMs don’t benefit much from being remotely hosted, other than that it makes the flops run in somebody else’s machine. But that will be hard to fund in a universe where the upper-limit to what can be charged is defined by how much additional cost “good AI hardware” adds to the price of your phone. We’ll see I guess. |
|
| ▲ | jbjbjbjb 2 hours ago | parent | prev | next [-] |
| It’s more about self-hosting the hardware which is more desirable than thin clients. |
|
| ▲ | wmichelin 2 hours ago | parent | prev | next [-] |
| Self hosting is not free, it's actually pretty hard to setup and keep running sometimes. Time and cognitive overhead are money! |
| |
| ▲ | Zambyte 2 hours ago | parent | next [-] | | > cognitive overhead I told a remote model to set up strata on my local machine. With the best options for my hardware. If someone can figure out using remote models, it's trivial to bootstrap local intelligence assuming available hardware. The hardest part was knowing that strata exists. Ninja edit: I've even had my agent tell me about suboptimal hardware configurations and told me what to do to fix it (XMP settings, having hardware in suboptimal slots). Cognitive load is simply a non-issue. The only issue is time, but in the grand scheme of things, it's a trivial amount of time to get up and running. | |
| ▲ | mortalapeman 2 hours ago | parent | prev | next [-] | | Just get a 5070 and 64 GB of RAM and download Strata. It's the ollama of actually usable local AI. | |
| ▲ | nullsanity 2 hours ago | parent | prev [-] | | [dead] |
|
|
| ▲ | barbazoo 3 hours ago | parent | prev | next [-] |
| It's so sad that we are not empowered to act as more than just consumers as if it was some kind of shared religion. How do we get out of this cycle? |
|
| ▲ | random3 2 hours ago | parent | prev | next [-] |
| People complaining about how difficult self-hosting is seem to be stuck in the past and missing the point. It's the very AI that we're talking about that removes most if not all friction points related to software configuration and maintenance. While this doesn't remove the upfront cost (which could be fixed with basic financing like Apple does) and physical manipulation (e.g. replacement on failure, etc.) it does lower the barrier. Finally, there's another path I find interesting and interested to think more of which is sharing hardware locally, from your house to your neighborhood, etc. We've done this in the late 90s - early 00s with Internet and it worked really well. I think it could work great for compute. I see a lot of opportunities in this direction, actually and most are actually enabled by the latest improvements in AI, so most of the "old" thinking doesn't apply anymore. |
|
| ▲ | bobajeff 2 hours ago | parent | prev | next [-] |
| That's like the argument on why people don't care about privacy. Just because most people choose that, all things considered, they are willing to sacrifice sovereignty over their own computer usage doesn't mean they don't care about it. By that reasoning you'd also conclude that no one cares about affordable housing or having a good work life balance or having your own lake made out of solid gold. |
|
| ▲ | lrvick an hour ago | parent | prev | next [-] |
| This is like owning a car vs using a train. Both are valid choices depending on your goals. I really do not care if everyone does it my way. What matters is that the choice exists for those that want more ownership, freedom, and privacy. That said, most next gen laptops will be shipping with capable inference hardware more than most consumers will ever need. Not using the GPU people have in front of them would be silly, but people are silly. |
|
| ▲ | redanddead 2 hours ago | parent | prev | next [-] |
| It comes down to the incentives Think of who uses local AI, it’s either -power users upset with the limits
—-cost of intelligence is going down -people and corps that want stabiltiy
—-a massive self own by the labs, this is justified -American labs seemingly snitching on their users (wtf?)
—-huge incentive for people to use Chinese inference, which is cheaper along with other benefits Regardless, we’re talking about a subset of a subset of people, 1% of 1%. The big bet is that everyone will become AI power users. I think it’s accurate, forget openAI, fuck Anthropic, let’s look at us, the people. Entire workflows like coding have been buried, there’s no going back to how things were before. Is it reasonable to say that will happen to the other work streams? There’s no indication not, and it’s a total gamble if it’s yes |
|
| ▲ | jmyeet 2 hours ago | parent | prev | next [-] |
| Here's one difference: currently lawyers using cloud LLMs violates attorney-client privilege. And you have to be really careful sending confidential medical information to one so as to not violate HIPAA. More broadly, those prompts and the responses have value. There's a whole shadow economy in China of aggregating queries across Claude (etc) and then using that as training data. It's like your entire search history is for sale. Clould infra has typically been opaque to the provider. You run an MySQL instance on an Amazon VM and Amazon doesn't have easy access to the unencrypted traffic and what you store. Cloud LLMs don't operate that way. AI hardware is improving at a rapid pace. By 2030 I believe a significant amount of LLM usage will be local for these and other reasons. |
|
| ▲ | 2 hours ago | parent | prev | next [-] |
| [deleted] |
|
| ▲ | jimbokun 2 hours ago | parent | prev | next [-] |
| Enough that Apple currently build enough Mac Minis and Mac Studios to satisfy the demand. |
|
| ▲ | adammarples 2 hours ago | parent | prev | next [-] |
| I don't know about this whole argument that hardware will become obsolete. If it can run Opus 5.5 today it will always be able to run Opus 5.5 until it burns out, and the quality of the models it can run will continue to increase as the software improves. |
|
| ▲ | nullsanity 2 hours ago | parent | prev [-] |
| [dead] |