davydm nails it. The gap isn't in generating code - it's in everything else that makes software actually work.

I've been building accounting tools for years. AI can generate a function to parse a bank statement CSV pretty well. But can it handle the Barclays CSV that has a random blank row on line 47? Or the HSBC format that changed last month? Or the edge case where someone exports from their mobile app vs desktop?

That's not even touching the hard stuff - OAuth token refresh failures at 3am, database migrations when you change your mind about a schema, figuring out why Xero's API returns different JSON on Tuesdays.

The real paradox: AI makes starting easier but finishing harder. You get to 80% fast, then spend longer on the last 20% than you would have building from scratch - because now you're debugging code you don't fully understand.

When I got hired at my current job, they handed me an AI generated app. It did a pretty reasonable job on the frontend, I think (I'm not a React guy), but the backend was a disaster. Part of it involved parsing a file and they had somehow fed the AI a test file with the first 20B truncated. I can tell that the AI tried hard to force the parser to match the file spec and ended up inserting checks for magic byte values that made no sense.

It took me a few days to realize what was happening. Once I got some good files it was just a couple hours to understand the problem. Then three weeks untangling the parser and making it actually match the spec.

And then three months refactoring the whole backend into something usable. It would have taken less time to redo it from scratch. If I'd known then what I know now, I would have scrapped the whole project and started over.

LLMs (well, most of the frontier and popular open-source models) are actually quite good at abiding by weird formats like this given that your prompt describes them clearly enough. The real problem is that you'll have to manually spot-check the results, as LLMs are also very good at adding random incorrectness. This can take just as long (or longer!) than writing the code + tests yourself.

I would say that handling these edge cases for minor changes that a human can easily understand, but is very hard to program rules for, is exactly where AI is the perfect fit.

Interesting, but isn't the real issue here how external systems can/will update their output at random? Given you are probably a domain expert in this situation, you can easily solve the issue based on past experience. But, what if a junior person encountered these errors? Do you think they have enough background to solve these issues faster than an AI tool?

Exactly. That last 20% is engineering. Handling edge cases, integrating with quirky APIs, optimizing for performance under load. An LLM excels when all conditions are perfect, but the real world is a mess of imperfections

As someone that's currently building accounting (and many many other) tools for myself: yes, it can.

But with a big fat asterisk that you: 1. Need to make it aware of all relevant business logic 2. Give it all necessary tools to iterate and debug and 3. Have significant experience with strengths and weaknesses of coding agents.

To be clear I'm talking about cli agents like Claude Code which IMO is apples and oranges vs ChatGPT (and even Cursor).

No, it cant handle those perfectly - but it can help you to develop the required code to do that correctly much faster :-)

all of these problems are better articulated at the level you just explained them. the code for these issues is convoluted and is only of use when an entity (human or not) can actually manipulate the symbolic text that achieves that task. a random oauth stub is of 0 use to the most skilled programmers without documentation as to what contracts and invariants are. bits in a file is just a means

Shit in'n'out. If a 'notech' throws a sample CSV at anyone and asks to create a parser for it, an engineer who doesn't care and LLM will give similar shitty result.

Parsers and data serialization in general is mature and more standardized area of software engineering. Can AI write a good parse? May be. Will it though?

I've heard AI coding described as "It makes the first 80% fast, and the last 20% impossible."

...which makes it a great fit for executives that live by the 80/20 rule and just choose not to do the last 20%.

AI makes starting easier but finishing harder. You get to 80% fast, then spend longer on the last 20% than you would have building from scratch - because now you're debugging code you don't fully understand.

I run a SaaS solo, and that hasn't really been my experience, but I'm not vibe coding. I fully understand all the code that my AI writes when it writes it, and I focus on sound engineering practices, clean interfaces, good test coverage, etc.

Also, I'm probably a better debugger than AI given an infinite amount of time and an advantage in available tools, but if you give us each the same debugging tools and see who can find and fix the bug fastest, it'll run circles around me, even for code that I wrote myself by hand!

That said, as time has gone on, the codebase has grown beyond my ability to keep it all in my head. That made me nervous at first, but I've come to view it just like pretty much any job with a large codebase, where I won't be familiar with large parts of the codebase when I first jump into them, because someone else wrote it. In this respect, AI has also been really helpful to help me get back up to speed on a part of the codebase I wrote a year ago that I need to now refactor or whatever.