Perhaps a trusted execution environment based anti-cheat system could be possible.

I think Valve said something about working with anti-cheat developers to find a solution for the Steam Deck, but nothing happened. Perhaps they will do something this time.

With a TEE, you could scan the system or even completely isolate your game, preventing even the OS from manipulating it. As a last resort, you could simply blacklist the machine if cheats are detected.

There would probably still be some cheaters, but the numbers would be so low as to not be a problem.

Maybe the user friction would be too much, but I'd be happy for the system to just straight up reboot for games which require anti cheat. So while that game is running, the system is in a verified state. But once you close the game all of your mods and custom drivers can be loaded just fine.