Hey, first author here. You are correct when you say that GPU programs do not have the right amount of information to do compiler style optimizations, which is why its important to find the right abstraction level. Even compilers do not directly optimize assembly as assembly has little to no information about the original source code; they usually do it all on an IR that is carefully engineered to hold all the useful information needed to optimize programs.

GPU programs as they are currently authored. It's because there isn't a runtime or DSL at the right conceptual level that is still practically useful against modern hardware and APIs. This is what I wanted to tackle with goldy.

Goldy has the high level structure of the shader graph and exchanges with the outside world (host memory, surfaces, etc) which is turned into a backend specific graph IR (different backends have different rules for command list retention and other properties). But my library also embeds Slang and uses Slang IR to inject and introspect on the user written shaders. I can then at runtime turn it into CUDA graphs (slang can turn any shader into a CUDA kernel) but with the added benefit that the runtime can manipulate the kernels and graph as more information is available. For example if a host upload that usually is scheduled between kernels isn't done on a specific graph submission, the graph partitioner can fuse kernels and remove a fence.

I plan to take advantage of the compiler and runtime introspection to implement some interesting features like shader coroutines without sacrificing GPU residency.