Do you really need to break the abstraction? Current scenario where SPIR-V is let's say compiled by Mesa into NIR and then NIR is compiled into GPU specific machine code works pretty well, where optimizations can happen on different phases of compilation.