Microkernels are normally pretty slow. It takes a lot of extra effort to make them fast.

Realistically there's no reason Linux wouldn't be fine on its own for AR and in fact I'm typing this on Linux on some AR glasses right now.