| ▲ | maxdo 3 hours ago | ||||||||||||||||||||||||||||||||||
i'll share a lonely voice on hn, i care very little about local ai. I feel like people who consider it as an option thinks of narrow scenarios vs ai is part of our life like internet is. How do you things on mobile for example with your local models? AI is very competitive, I use various models, I like to be on cutting edge and to be productive, its the opposite from doing something permanent, and not changing, where local models can be good. I like to do things on the go, and the last thing any normal person would want is to have DC at their basement. | |||||||||||||||||||||||||||||||||||
| ▲ | cableshaft 2 hours ago | parent | next [-] | ||||||||||||||||||||||||||||||||||
I'm not willing to feed my personal data (emails, files, etc) to a non-local AI model (well, beyond what I can't avoid, like Google and my Gmail account). But I would like to have an AI that does that, like synthesize my journals and notes and provide a local personal knowledgebase I can query, things like that. I don't think I have a good enough GPU to manage a local model powerful enough to be worth the effort just yet, though, so I'm waiting a bit longer. For coding and bouncing ideas off of though? Yeah sure, online is fine. | |||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||
| ▲ | ale42 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
I think that many people want similar things. But (at least personally), not giving my data to BigTech is more important that this. | |||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||
| ▲ | segmondy 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
I run local AI, there are tons of local models with no cloud alternatives, the only way to unlock and have these functionality is by being able to run local. So you are not on the cutting edge by using cloud only. | |||||||||||||||||||||||||||||||||||
| ▲ | NoidFonsense 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
I just set up a wireguard tunnel to my home network | |||||||||||||||||||||||||||||||||||
| ▲ | musha68k an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
They will need to realize quantum proof, performant and verifiable zero trust compute to stay competitive with the sovereign front. Yet this would subvert one, if not the main pillar of funding - which is total self imposed (of course) surveillance. | |||||||||||||||||||||||||||||||||||
| ▲ | tencentshill 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
Local (Free) beats cloud (free) for pretty much any task except live web search (which will be less useful as more cloud AI scrapers are blocked). You run into the free limits in one or two prompts most days if you try to do anything even mildly complex. | |||||||||||||||||||||||||||||||||||
| ▲ | jakswa 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
You think you will be doing those things? And if you do keep doing those things forever, won't AI be smart enough to figure it out for you, whatever needs to happy locally? | |||||||||||||||||||||||||||||||||||
| ▲ | codybontecou 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
Whisper and other stt models work fine on mobile. Mobile is far behind frontier coding agents but they are already solving valuable problems with competitive quality and privacy, one-time payments as viable options. | |||||||||||||||||||||||||||||||||||
| ▲ | nater5000 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
First, I'll say that I, too, care little about local AI. I do like toying around with local AI, and I do see how it can be a very important option in some contexts. I also loosely agree that it's good for keeping frontier models on their toes, prevent this technology from being locked away behind a paywall, etc. But for me, as of now, they're not very useful, and I use AI quite a bit. However, I do think your perspective on local models is a bit narrow. For a lot of people, they want AI tools but do not need cutting edge AI. They'd prefer to just have their own AI which is capable enough and under their control. They'd run it on a small server at home and connect to it as needed for their work (including on their phone). It's worth noting that much of the consumer-grade infrastructure definitely doesn't exist for this, but one could blame the lack of consumer-grade hardware as the source of this rather than just a general lack of demand. You can definitely see people picking up on this by running their various assistants, like OpenClaw, on Mac Minis from their home. It's definitely not accessible or cheap enough for most consumers at this point, but it's also not impossible to imagine us getting to the point where it is more ubiquitous. I'll say, personally, that I run Qwen 3.8 27B on an RTX 3090 from my home server, and it operates pretty well for what it is. I'm certainly not giving up my Claude subscription anytime soon, but it is nice to have the option and I can chat with it directly on my phone securely (using Tailscale) and using a nice UI. Regardless, I think the whole point of these kinds of discussions is to point out that, right now, local models aren't viable, but not because they can't be viable, but because companies are actively preventing them from being more viable. I can see for myself that they work well enough that proper investment could easily push them into the "good enough" territory for many users, and I suppose it is inevitable that more and more people will end up running local models in one way or another. But I definitely agree with the sentiment that most consumers simply do not care about it. They don't want to deal with hardware, they don't want to think about software, and they're happy to pay $20 per month for a subscription that "just works." I definitely don't blame them. | |||||||||||||||||||||||||||||||||||
| ▲ | yieldcrv 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
I disagree, I think local is the supercycle here once it gets out of enthusiast territory We have LLMs baked directly into silicon getting 20,000 tokens/sec with low power consumption, Taalas was bought by AMD weeks after that demo. In that world, people get a new LLM only on hardware refreshes, and only after LLMs are good enough to be in stasis for years Apple’s M7 Pro/Max if it meets promised will be very expensive, I see some enterprises rolling them out to some staff Some language models in the Apple Ecosystem are already baked into iOS and MacOS, and its ubiquity is invisible Thats the direction I see things going, but like not anytime soon to alleviate supply chain crunch. 2029, 2032 | |||||||||||||||||||||||||||||||||||
| ▲ | dofm 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
> How do you things on mobile for example with your local models? The same way you can screenshare or forward from your local Mac? Paseo, LM Studio and LM Studio Bionic all support this kind of tailscale-ish access to local models from phone apps. (Haven't tried any of these myself) I do, personally, have "narrow scenarios" for AI. I have to use it for work, and want to be informed and engaged on that level, but I want to keep it out of my personal life as much as is practical. If you don't, that's fine, but I reframed my mass social media use down to an extremely minimal pattern in 2014 and I am healthier for it; I intend to keep the same distance from AI in my hobbies, for several of the same reasons. Paradoxically, for me that has involved taking the control over it that local models offer. I use the smallest, simplest things that get the job done and I try to retain my engagement with my work. It's all pretty low-effort and low-tinkering; no more than setting up local VMs for dev work. Old M1 Max macbook, Unsloth, Pi, a model in the Muse Glimmer/Qwen 27B/Nemotron Bolt sort of bracket. Good enough to be a rubber duck that talks back, write the minimal kind of AI-generated support code I want, and teach me how it all works. I might one day buy an M5 Pro Mac Mini, or an M5 Max Mac Studio, or stick an R9700 in a homebuild PC, or I might just use openrouter for the few things the Macbook won't do. I am a cynic: I believe AI has some value, but I also believe it should be a commodified technology rather than a branded SaaS subscription product, and it should demonstrate that value without me letting a couple of large corporations FUD me into pushing it into every corner of my life at once, or simply handing over control of my job to it. It is a matter of personal discipline, for me: I am going to resist the endless FOMO of new model benchmarks, learn how it works, keep it in a box, and take it out when I need it. | |||||||||||||||||||||||||||||||||||
| ▲ | betterunix2 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
It would be a lot easier to do local AI if we were not using natural language as the interface. In fact there is already plenty of local AI, including on mobile, that most users do not even realize is there -- because nobody has "conversations" with those models. Niche? Sure, but let's be honest here, a lot of AI use right now is forced, which is usually a sign that the product is not actually in demand or has questionable value for users. The big, natural-language-interpreting models are also niche -- a minority of people find them extremely useful and a majority find them to be "meh" at best. | |||||||||||||||||||||||||||||||||||
| ▲ | lubujackson 2 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
I agree. It reminds me a lot of building your own desktop computer from parts - it started as a requirement, then a cost-effective choice, then became for power users only, and now it has little place beyond tinkerers/purpose-driven/privacy tinhats. I refused to use Macs for a decade because of how offensive I found it that they locked down their computers from even adding RAM yourself. But this is the direction of the tide and swimming against it will only continue to become more and more arduous. At best, we will have some Tor-like hosted AI, but I am not holding my breath or wasting my money/energy. | |||||||||||||||||||||||||||||||||||
| ▲ | xscott an hour ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||
Online models will eventually enshitify to serve you advertisements. They'll give you worse answers and steer you towards commercial products to solve your problem. | |||||||||||||||||||||||||||||||||||
| ▲ | neutronicus 2 hours ago | parent | prev [-] | ||||||||||||||||||||||||||||||||||
No, I'm the same way. I started trying to spec out a local AI box, looked at the numbers, and it felt like trying to run my house off a generator. Obviously worse than the centralized option, and not the hobby I want. I'd rather just use Cloud tokens help me do what I actually want to do. | |||||||||||||||||||||||||||||||||||