Not a useful mental model.

It is useful. If you can whip up something complex fairly quickly with an AI agent, it’s likely because it’s already been done before.

But if even the AI agent seems to struggle, you may be doing something unprecedented.

▲

simonw 2 hours ago | parent [-]

Except if you spend quality time with coding agents you realize that's not actually true.

They're equally useful for novel tasks because they don't work by copying large scale patterns from their training data - the recent models can break down virtually any programming task to a bunch of functions and components and cobble together working code.

If you can clearly define the task, they can work towards a solution with you.

The main benefit of concepts already in the training data is that it lets you slack off on clearly defining the task. At that point it's not the model "cheating", it's you.

	▲	aix1 an hour ago \| parent [-]
		Simon, do you happen to have some concrete examples of a model doing a great job at a clearly novel, clearly non-trivial coding task? I'd find it very interesting to see some compelling examples along those line.