If you could get every hardware manufacturer in the world onboard with such an interface, perhaps. But even if 90% of them were onboard there would be edge cases that people and companies would demand support for and there goes your standard.

Drivers exist to ultimately turn actual hardware circuits off and on, often for highly specialized and performance-critical applications, and are often written based on the requirements of a circuit diagram. So any unified driver platform would also involved unified hardware standards, likely to the detriment of performance in some applications, and good luck telling Electrical Engineers around the world to design circuits to a certain standard so the kernel developers can have it easier.