> If you ask AI to write a document for you, you might get 80% of the deep quality you’d get if you wrote it yourself for 5% of the effort. But, now you’ve also only done 5% of the thinking.

This, but also for code. I just don't trust new code, especially generated code; I need time to sit with it. I can't make the "if it passes all the tests" crowd understand and I don't even want to. There are things you think of to worry about and test for as you spend time with a system. If I'm going to ship it and support it, it will take as long as it will take.

▲

jdjdjssh 31 minutes ago | parent | next [-]

Yep, this is the big sticking point. Reviewing code properly is and was the bottle neck. However, with humans I trusted, I could ignore most of their work and focus on where they knew they needed a review. That kind of trust is worth a lot of money and lets you move really fast.

> I need time to sit with it

Everyone knows doing the work yourself is faster than reviewing somebody elses if you don’t trust them. I’d argue if AI ever gets to the point where you fully trust it, all white collar jobs are gone.

▲

simianwords 38 minutes ago | parent | prev | next [-]

Honest question: why is this not enough?

If the code passes tests, and also works at the functionality level - what difference does it make if you’ve read the code or not?

You could come up with pathological cases like: it passed the tests by deleting them. And the code written by it is extremely messy.

But we know that LLMs are way smarter than this. There’s very very low chance of this happening and even if it does - it quick glance at code can fix it.

▲

kranner 15 minutes ago | parent | next [-]

You can't test everything. The input space may be infinite. The app may feel janky. You can't even be sure you're testing all that can be tested.

The code may seem to work functionally on day 1. Will it continue to seem to work on day 30? Most often it doesn't.

And in my experience, the chances of LLMs fucking up are hardly very very low. Maybe it's a skill issue on my part, but it's also the case that the spec is sometimes discovered as the app is being built. I'm sure this is not the case if you're essentially summoning up code that exists in the test set, even if the LLM has to port it from another language, and they can be useful in parts here and there. But turning the controls over to the infinite monkey machine has not worked out for me so far.

▲

jdjdjssh 28 minutes ago | parent | prev | next [-]

> If the code passes tests, and also works at the functionality level

Why doesn’t outsourcing work if this is all that is needed?

	▲	jmathai 15 minutes ago \| parent \| next [-]
		We haven’t fully proven that it is any different. Not at scale anyway. It took a decade for the seams of outsourcing to break. But I have a hypothesis. The quality of the output, when you don’t own the long term outcome or maintenance, is very poor. This is not the case with AI in the same sense it is with human contractors.
	▲	simianwords 15 minutes ago \| parent \| prev [-]
		Why do we have managers if managers don’t have accountability?

▲

throwup238 27 minutes ago | parent | prev [-]

It depends on the scale of complexity you’re working at and who your users are going to be. I’ve found that it’s trivial to have Claude Code spit out so much functionality that even just proper manually verifying it becomes a gargantuan task. I end up just manually testing the pieces I’m familiar with which is fine if there’s a QA department who can do a full run through of the feature and are prepared to deal with vibe coding pitfalls, but not so much on open source projects where slop gets shipped and unfamiliar users get stuck with bugs they can’t possibly troubleshoot. Writing the code from scratch The Old Way™ leaves a lot less room for shipping convincing but non functional slop because the dev has to work through it before shipping.

The most immediate example I can think of is the beans LLM workflow tracker. It’s insane that its measured in the 100s of thousands of LoC and getting that thing setup in a repo is a mess. I had to use Github copilot to investigate the repo to get the latest method. This wouldn’t fly at my employer but a lot of projects are going to be a lot less scrupulous.

You can see the effects in popular consumer facing apps too: Anthropic has drunk way too much of its own koolaid and now I get 10-50% failure rates on messages in their iOS app depending on the day. Some of their devs have publicly said that Claude writes 100% of their code and its starting to show. Intermittent network failures and retries have been a solved problem for decades, ffs!

▲

layer8 an hour ago | parent | prev | next [-]

Yes, regression tests are not enough. One generally has to think through code repeatedly, with different aspects in mind, to convince oneself that it is correct under all circumstances. Tests only point-check, they don’t ensure correct behavior under all conceivable scenarios.

▲

slfreference an hour ago | parent | prev [-]

I think what LLMs do with words is similar to what artists do with software like cinema4d.

We have control points (prompts + context) and we ask LLMs to draw a 3D surface which passes through those points satisfying some given constraints. Subsequent chats are like edit operations.

https://youtu.be/-5S2qs32PII

	▲	catdog an hour ago \| parent [-]
		An LLM is an impressive, yet still imperfect and unpredictable translation machine. The code it outputs can only be as good as your prompt is precise, minus the often blatant mistakes it makes.