I feel a really odd dissonance with all these accounts. I have access to top models via Cursor at work. Every time I think to myself, here’s a tedious but well defined task, with clear success criteria, where you can achieve a lot with persistent iterations. I’ll give it to Claude to do.

First, it takes me ages to describe the task properly. All these clear success criteria, well, instead of writing code from a clear spec in my head, I’m writing tons of prose, and trying to make it unambiguous. I’m programming in English++.

But then, one time in 5, it produces something that kinda works. Maybe not perfect but good enough. Two times out of 5 it kind of sort of looks alright, but actually ignores 80% of the spec, or pays it minimal lip service in the comments.

And sometimes, maybe a bit less than 2/5, it just completely diverges into total shit. Starts writing scripts that load the ast of the main script and pickle it then serialise to base64 for no good reason. Encode some stuff in strings then check ord(string[i]) repeatedly for string comparison. Eventually runs out of context and develops the LLM equivalent of severe dementia.

I truly cannot reconcile my experience with people who seem to say “hey computer write this” and it’s a good use of their time.

Take a step back and give it the essential functional requirements.

Or, consider what information is driving your extremely detailed instructions and give it the raw source if you can. Meeting transcripts, messy notes on some adhoc conversation, etc.

The more you try to micromanage it, the more impossible to work with it it is.

Of course not every situation calls for just giving it high level requirements. But not every situation calls for trying to define how everything is done yourself either.

This is similar to my experience when cursor first came out. But now it's even trickier because LLM actually writes working code that pass the test. The issue is when i dig in deeper, there's often small security or design issues that will cause problem down the road when the app gets more traffic.

These issues are usually in really tedious places that I didn't bother to describe what to do to the agent

I've found a more productive workflow is to ask claude how it would do the task, go back and forth with it a bit until you both agree on the steps, then have it work through them. i also emphasise the need to have every step in its own commit, then go back and read through the stack suggesting code improvements and refactoring, letting the AI do the tedious work of fixing conflicts and restacking. I'm usually pretty happy with the process and the end result.

[dead]