Maybe I'm a control freak, but asking agents to one-shot random apps is nothing like how I actually use AI in software engineering.

Absolutely yes, but that's how you become twitter/X famous

Yeah, the models have all been really good at generating greenfield apps for a really long time (in the scope of LLM time).

I suppose it’s interesting to see how they make better greenfield apps. But I am much more interested in how they solve hard problems in existing gnarly codebases.

Man I do ponder this all the time.

It's not, but it's trying to bring any level of objective measure in this realm, vs just going off of vibes.