Yes, iphone, ipad, mac. Maybe your phone makes the inference requests to either Apple servers (requiring iCloud+ and credit based?) or your Mac (Tailscale equivalent).

My assumptions are that inference costs continue to decline (and/or the difficulty level of routing requests declines) and hardware acceleration continues into the future. These assumptions might be wrong, but that is the idea the thesis is based around.

(have built workflows and harnesses that currently target LLM provider APIs under the assumption I'll be able to point "locally-ish" eventually, my career had a strong focus on distributed computing about 10-15 years ago, think managing data and work clusters for LHC detector data processing, and I grew up on hobbyist distributed.net and BOINC projects, all the lenses I see this problem through; queues, jobs, sync vs async performance optimization, etc; recovered sysadmin|infra/devops engineer a lifetime ago)

I can have full conversations with Gemma4 on my phone, no server requests, with voice.