I made a completely free 100% on device translation app https://apps.apple.com/us/app/arda-translate/id6778970560 and hard to image a world where TTS and STT will not be done locally in the future

cool app, and agreed that on-device keeps eating the single-user cases, we are seeing dictation and translation are exactly where local models shine. We benchmark the open models on the same boards as the hosted ones, but there are still a few: https://benchmarks.speko.ai/open.

production phone agents are a different shape today: the call terminates server-side, three models plus turn-taking under one latency budget, and per-language quality still swings a lot from our tests