I was looking for a similar scheme, and though far from perfect I found you can run tmux+ttyd. ttyd lets you share your terminal over http. That lets you use your phone's browser (and speech-2-text).