▲ | kcb 3 days ago | |
There's no real physical reason to run those LLMs on the end device. LLM interaction is not particularly latency sensitive as in single digit ms like gaming. So if there's no real usability drawback from hopping over the Internet then that's the direction it will go. |