I think the other side of things is why have the graphic program run over ssh, just have raw text cli tools to manipulate what ever then have a program driving it. This for me is where emacs excels and some programs already fit well with this model and the fantastic (albeit hacky) tramp.