What I'm wondering is, why couldn't the AI generate this solution? And implement it all?

Why did they need to spend human time and effort to experiment, arrive at this solution and implement it?

I'm asking genuinely. I use GenAI a lot, every day, multiple times a day. It helps me write emails, documents, produce code, make configuration changes, create diagrams, research topics, etc.

Still, it's all assisted, I never use its output as is, the asks from me to the AI are small, so small, I wouldn't ever assign someone else a task this small. We're not talking 1 story point, we're talking 0.1 story point. And even with those, I have to review, re-prompt, dissect, and often manually fix up or complete the work.

Are there use-cases where this isn't true that I'm simply not tackling? Are there context engineering techniques that I simply fail to grasp? Are there agentic workflows that I don't have the patience to try?

How then, do models score so high on some of those tests, are the prompts to each question they solve hand crafted, rewritten multiple times until they find a prompt that one-shot the problem? Do they not consider all that human babysitting work as the model not truly solving the problem? Do they run the models with a GPU budget 100x that they sell us?

> What I'm wondering is, why couldn't the AI generate this solution? And implement it all?

My read of the blog post is that is exactly what happened, and the human time was mostly spent being confused why 40MB/s streams don't work well at a coffee shop.