Can you explain how this works? Doesn’t this also stop you from connecting to it over ssh via vs code?

Yes that's the point. If vscode didn't insist on installing potentially gigabytes of blobs then this wouldn't be necessary.

(I'm a vimmer anyway... And emacs is too bloated to fit too, conveniently.)

Sounds like a feature.