| ▲ | barrkel 4 hours ago |
| I remain puzzled why anyone would want to do inference on a laptop if they have a home internet connection and a phone. If you really want to do DIY inference, plug it in at home and set up tailscale so you can access it on the go. No huge battery drain or toasty hot laptop required. Lower cost laptop. Lighter laptop. Longer battery life laptop. Win win win win. |
|
| ▲ | solyin 4 hours ago | parent | next [-] |
| Not everyone grew up in a situation where home was a stable dependable place. Not everyone has a home to go to. If you have it with you, they can't mess with it or take it away from you when you're not there. If you grew up in a broken home, you would understand. |
| |
| ▲ | Gigachad 3 hours ago | parent | next [-] | | And those people are buying $10k laptops to run local AI? | | |
| ▲ | jeroenhd 3 hours ago | parent | next [-] | | Those people won't be buying the Spark laptop, but asking "why anyone would want to do inference on a laptop" is broader than what the article covers. | | |
| ▲ | c7b an hour ago | parent [-] | | Local AI is an expensive hobby. You're paying big money for small performance (so mainly for the feeling of "owning it", which is a great motivator). It can't match cloud services, not absolutely nor bang-for-buck, especially if you look a bit for cheaper and freebie options. As someone who loves and promotes local AI, in the situation you described my plea would be to use that money to improve your situation more directly (move out, get support,...). The cost of a laptop that can run somewhat decent models should buy you at least several months of therapy (@1 session/week). | | |
| ▲ | throw1234567891 6 minutes ago | parent [-] | | You can't put a price of sending all your IP to a dubious third-party. Sorry. You argument apples vs oranges. |
|
| |
| ▲ | 3 hours ago | parent | prev | next [-] | | [deleted] | |
| ▲ | Danox 3 hours ago | parent | prev [-] | | The spark laptop in its present form is not very good, in fact, it’s disappointing. |
| |
| ▲ | barrkel 24 minutes ago | parent | prev [-] | | If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere. |
|
|
| ▲ | alerighi 2 hours ago | parent | prev | next [-] |
| Because maybe you don't want to send your private data to a company that could use it for whatever it wants? |
| |
| ▲ | anonym29 a minute ago | parent | next [-] | | GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself. | |
| ▲ | barrkel 34 minutes ago | parent | prev [-] | | Can you explain what data is going where? | | |
| ▲ | entrope 8 minutes ago | parent [-] | | Your prompts and whatever data they collect from your dev environment go to the inference provider, and typically the customer has neither control nor assurance over what the inference provider (re-)uses that data for. |
|
|
|
| ▲ | KeplerBoy 3 hours ago | parent | prev | next [-] |
| The RTX Spark is also sold in a mac mini form factor. That's pretty well suited for home inference. |
| |
| ▲ | barrkel 30 minutes ago | parent [-] | | Exactly. Get yourself a DGX spark and plug it in at home. I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived. Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside. |
|
|
| ▲ | dlahoda 3 hours ago | parent | prev | next [-] |
| private memory. unique local data and know how not shared with cloud. |
| |
|
| ▲ | pixelpoet 3 hours ago | parent | prev | next [-] |
| I have been saying this for years, and the answer hits a nerve that gets you downvoted into oblivion: people prefer the aesthetics of laptops, simple as that. They will absolutely go to the grave arguing that they need the mobility etc, then just leave it plugged in. They simply prefer how laptops look, and they absolutely hate it when someone dares say it. |
| |
| ▲ | monegator 3 hours ago | parent | next [-] | | I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something) | |
| ▲ | barrkel 26 minutes ago | parent | prev | next [-] | | To be absolutely crystal clear: I am not saying don't use a laptop. I am not saying you have to use cloud inference. I am saying using a laptop for inference is dumb. If you want private inference: - use a dgx spark or similar plugged in at home - set up tailscale so you can connect from anywhere - use your phone as a hotspot on the go - work on your laptop, using tailscale to connect to your home inference server if you're on the go This gets you everything except for the few hours you might be on an airplane. | |
| ▲ | dumberquestions 2 hours ago | parent | prev [-] | | If you do any travel whatsoever a laptop is usually not optional. |
|
|
| ▲ | 3 hours ago | parent | prev [-] |
| [deleted] |
