Is this how Stadia was supposed to work?

Not exactly, since they aren't opinionated in the game engine, they can't control what "primitives" are being used to render.. they probably just encode video