I agree. Six years ago during COVID I wrote a document describing my idea of a dream personal computing environment, where all functionality is accessible using an API, enabling scripting and customizable UIs. UIs are simply shells covering functionality provided by various objects.

Unfortunately I haven't had the time to implement this vision, but Smalltalk environments such as Squeak and Pharo appear to be great environments to play around with such ideas, since everything is a live object.

A lot of Linux programs are command line only, with multiple GUIs available to use them. Sounds similar to what you're describing.

It's not a novel idea: I've also invented that, as have most people I know who've thought about this problem. (This is a good thing: it means it'll be fairly easy to bootstrap a collaborative project.) I never got as far as writing up a full document, though: only scattered notes for my own use. Would you mind sharing yours?