This thought completely neglects the idea that Haskell probably needs significantly less compiler runs because every run catches more errors and gives more information about them.

And that is not even considering how often the agent needs to run tests to get it right.

It seems pretty clear that they do only minimal live testing during the "open a ticket, implement something, deploy it in production, all while the customer is still on the call" cycles. So your second concern is probably not relevant in this particular setting.

Regarding the first, I think you're probably right, but then again, if there is a 15-minute base cost, it's hard to amortize that through fewer incremental runs of the compiler.

(Which isn't to say that I think they are doing the right thing.)

The number of compiler runs doesn't matter as much as the total elapsed time it takes to finish the task. In just about every test we ran, LLMs are faster at building in Python than Haskell.