Have you tried scrcpy? https://github.com/Genymobile/scrcpy

I ran this with a custom fork to expose the device screen as a VNC server for years, no problems

Thanks for reminding me about scrcpy. I knew it but I've completely forgot about it. Will give it a shot. Why the custom fork? You added the VNC support to scrpy?