It would be simple, but somewhat incomplete. It would not look like a virtual memory subsystem where merely accessing an address can allocate physical RAM to it.

The API is already there! AllocMem(, MEMF_PUBLIC) allocates shared memory, otherwise you get private memory.

You can then enforce memory protection around that, but obviously the entire OS is still built on message-passing between tasks where the message itself remains at exactly the same (public) address and isn't copied between private memory spaces, and there's a gentlemens' agreement as to who currently owns it (PutMsg/ReplyMsg)

Software like Enforcer and MuGuardianAngel already checked this, and dinged your program for accessing non-allocated memory.

I think "somewhat incomplete" is underplaying it as such a large proportion of message exchange tends to include pointers to shared structures. That the message handling uses intrusive lists doesn't help.

I still think it's possible but doing it to a meaningful extent would be a long tedious effort.

You have Janus-UAE which sandboxes/emulates M68k apps for AROS "rootlessly" in that it bridges a number of structures etc. That mostly solves the problem on modern hardware for legacy apps, though it's a painful solution.

Then you could aggressively upgrade message formats for "native" apps to replace pointers with opaque handles.

If done at the time you could of course had relied on more of the developers of various apps still being active and willing to update their code and might have been able to avoid the Janus-UAE approach, but there was never a point at that time where anywhere close to a majority of users had MMU's,