I use a very customized emacs/evil setup, and before that a very customized vim/neovim setup, but I've never had trouble jumping onto systems where only plain Jane vi is available. I think it's because of the color scheme, or lack thereof, that I don't have any problem automatically switching. If I don't see my color scheme, by fingers just automatically use vi compatible bindings and I don't find myself expecting my other customizations to be present.

Maybe not everybody can switch so easily, but I think it's worth trying out. (One change that absolutely does trip me up is capslock->control. If I try using somebody else's computer, I constantly enable capslock by accident.)