This is one of the dilemmas that I am trying to wrap my head around. I love optimizing software pipelines, which often boils down to figuring out the operational constraints that the compiler and the generic libraries can’t assume. Then I exploit these to squeeze out performance. But in a world I can start from scratch and code a domain specific solution from line zero in a matter of hours/days, I do not need general libraries as much as I used to. On one hand the code won’t be as well tested as a good general library. On the other hand, it also won’t have a plethora of library bugs that are there because the code is generic and opaque. One counter argument is that things are never static and you can’t have specific code for too long. A counter to that is that you can then change the code to be specific to the new reality at very low cost. This is the mental loop I ride constantly. Disclaimer: My circumstances are definitely not general, I am not writing code that is truly large scale.

Many open-source communities are wrestling with similar questions. In many ways the generic frameworks end up being reference implementations for the specific solutions to copy and hill-climb towards, and it's as you say unclear what the point is to maintain generic things beyond that.

What I feel is lacking with the solutions that have for example agent-generated and tuned GPU kernels is that the use-cases for them are unclear. If you are a researcher on second-order optimizers, you probably want to be able to handle variable input shapes to experiment, you also might want something readable to understand intermediate steps and perhaps build on that. If you are a neolab running massive training runs for 80% of your VC funding, you need to know that every line in your training code is bitwise identical to the theory/reference because a divergent run from some LLM-generated numerical bug will set you back or bankrupt you, so you can't just plop in a random kernel even if it promises good performance.

So who is the agentically-looped end result for? Except for Openai and Anthropic of course who sold the tools.