| ▲ | rienbdj 7 hours ago |
| There will come a point where the local models are good enough (and iPhones powerful enough) for this to be possible without privacy downsides. |
|
| ▲ | thatsabadlook 5 hours ago | parent | next [-] |
| They are that good now. The computer is just too expensive. The computer is too expensive because of meta, openai, anthropic, grok, etc. it's funny how this works... |
|
| ▲ | raz32dust 7 hours ago | parent | prev | next [-] |
| A lot of use cases for agents require long running sessions. And you're likely to have many agent sessions running in parallel. These aren't going to be running on your personal devices |
| |
| ▲ | brabel 5 hours ago | parent | next [-] | | I agree. I currently do run a bunch of dev containers on my local Docker but that is not good enough, they neeed to be able to work while I am on the run and close my laptop. So we are trying having personal containers on a managed Kibernetes cluster for every developer. We’re still working on making this work but it seems promising, the hardest thing being just permissions to let the agent use MCP servers, http APIs etc. If anyone has done this, how did you manage that?? I know there are products like OpenCLI that help with the problem but we want something we can build ourselves since AI makes it so much easier to build things! | | |
| ▲ | devrwoody 4 hours ago | parent [-] | | Use ai to build it a poc and read the code after, learn, rebuild until you find something that works. I host all of my custom applications on a machine in my home with Tailscale, k8, infrastructure as code It’s wired, always on, has access to my gpu running on another machine or falls back to cloud inference Self hosted GitHub action runners on each different platform for build/deploy of various projects: tauri v2, apis, web uis, native macOS apps, Unity 3d None of this existed a month ago and it’s been a blast pretending to run my own platform |
| |
| ▲ | ctolsen 4 hours ago | parent | prev | next [-] | | Durability and parallelism aren't that hard, and you won't be needing frontier models for much of this stuff. The parameter efficiency of models is increasing rapidly. I don't see why a (near-)future device wouldn't be capable of running a perfectly capable personal assistant. | |
| ▲ | spwa4 5 hours ago | parent | prev [-] | | Why not? Why wouldn't your phone be able to do this? The other advantage long running sessions have is that very low tok/s are perfectly acceptable. In fact, hardly noticeable. (aside, of course, from Google and Apple both having incentives to prevent this from happening, as both depend on selling centralized services to you and advertisers, and the local model breaks both) |
|
|
| ▲ | reacharavindh 6 hours ago | parent | prev | next [-] |
| I’m sure there is probably a skunkworks like team secretly working at Apple trying to make the hardware needed to accelerate local models enough to handle personal data processing needs without decimating the battery. I’d guess another team is probably tasked at researching a E2E encrypted remote inference mechanism where the personal data that does get out of the iPhone/ipad/Mac is encrypted with something Secure Enclave on processor, and “even Apple can’t read it”.. Apple is my long term bet for personal AI computing. No way in hell am I letting FaceBook/OpenAI/Anthropic/Deepseek/anybody have complete access to my personal digital life. That’s just dumb. Just look at OpenAI digging through ChatGPT logs to get the idea to solve the math problems etc. I’m sure all the subscription plans and subsidies are aimed at gathering enough breadth of real people comms to get the models to improve on wider signals… Worst case, I don’t want to be voluntarily giving up my digital footprint to get profiled a certain way and be fed what AI overlords (or the companies that wield them really) think I should get as an answer/response. My thoughts and activity stays private as much as practical and feasible. |
| |
| ▲ | r_lee 3 hours ago | parent [-] | | > where the personal data that does get out of the iPhone/ipad/Mac is encrypted with something Secure Enclave on processor, and “even Apple can’t read it”.. there already is technology for this, which is Confidential Computing which is implemented on Nvidia etc. it requires transparency to audit it properly and if there's an exploit which weakens it then the security guarantees fall apart but in terms of commercial viability, it's the most promising way to tackle this and it seems like Apple is already pushing for it with their Private Cloud Compute |
|
|
| ▲ | jayd16 7 hours ago | parent | prev [-] |
| Ok but look at the list... Google likely already has all that. It's not really something a normie cares about self hosting. |
