Basically, at place we've a coding agent in a while loop.

What it does is pretty simple. You give it a problem, setup enviornment with libraries and all.

It continuously makes changes to the program, then checks it output.

And iteratively improves it.

For example, we used it to build a new method to apply diffs generated by LLMs to files.

As different models are good at different things, we managed to run it against models to figure out which method performs best.

Can a human do it? I doubt.

What kind of problems have you been throwing at it?