It would be difficult, but it would at least be possible. See projects like Asahi Linux where they did the work to reverse engineer the drivers for ARM-based Macs.

And I'm sure a lot of that work could be reused for the A-series SoCs.