just the other week I asked Fable 5 to diagnose the cause of some intermittent latency spikes on an API that queries an OpenSearch cluster at work. I encouraged it to look at the datadog metrics, splunk, the whole works. I let it loose to look at whatever it wanted.

End result - 2 hours later it produced a convincing theory with lots of references, and burned a bunch of tokens too of course. just for fun we tried its suggestions and deployed them to prod. Guess what? Didn’t fix the issue. Alas, a human was needed after all.

either everyone’s working on toy problems, or they’re working on very cookie-cutter code. I’m really not sure. I DO remain impressed with Fable 5 but the idea that we’ll all be unemployed in 2 years is hilarious delusion. we’re already at the point where many organizations are scaling back some of their AI spend.

By any chance, is the following true: you had no empirics in the loop. It couldn’t validate any of its theories by experimentation. If the solution requires being able to make actual changes in order to gather more information and it is not allowed to, and this is the only way to solve the problem, by definition it couldn’t do it, nor could you. Or was it something that could be worked out entirely on an a priori basis from the available data?

And while we’re talking about hilarious delusions, perhaps you should look at the current capability curve of AI and weigh it against the constant stream of arguments for why it couldn’t have continued at every point and yet has.