What about running the whole Android infrastructure, but on top of a non-Android Linux distro? And this, on top of a Android kernel (otherwise you won't have the drivers yo uneed)

The advantage being, we can manage packages using a regular Linux distro

A lot of "non Android Linux distributions" like sailfish os actually use a lot of Android infrastructure to keep working.

Drivers doesn't just mean the kernel. It's the user space binary blobs and services that need to talk to the kernel to enable the hardware.

Other than that there's waydroid, alien dalvik etc.. that run another Android instance in a container.

The thing is a lot of Android applications use safety net/other methods to make sure they only run on. "Approved"/stock hardware.

This is exactly where my head goes. There's nothing special about this hardware other than its small form factor. Sure, some desktop apps would likely want a different skin, but thats hardly limiting.

You can't because a lot of apps check that the phone is controlled by Google with Play Integrity.

Google thought about this, don't worry. They learned their lesson after CyanogenMod tried to compete by offering an alternative. Non-Google Android are now dead except in China.