>LLMs are still bad at low-level hardware optimizations, but really good at high-level composition.

I disagree. While yes they don't have all the architectural quirks of every GPU memorized, they are able to extract such optimizations from ISA docs and online guides. Now with 1M context available on frontier models, they can even fit the whole ISA definition in context (RDNA 3.5 here specifically) and spit out swathes of optimizations to try. The rest is just bruteforcing a single goal which they are extremely good at.

Or that's how simple it'll look until you have subtle bugs to solve somewhere deep in your stack.

Anyways, low-level hardware optimized GPU kernels has been an exceptionally good use case for agents in my opinion. They have far more trouble in other domains like doing GUI.