Just as "use code for contracts" failed for crypto currencies, "use AI output as prod" will fail for AI. Both is based on "just don't make catastrophic mistakes anymore".

You also wrongly assume that requirements can always easily expressed as natural language.

Another point: Software Engineering always starts where tooling capabilities stop. You don't get a competitive advantage by building without engineers what anybody everybody else can build without engineers.

▲

throwaway041207 2 hours ago | parent | next [-]

> Just as "use code for contracts" failed for crypto currencies, "use AI output as prod" will fail for AI. Both is based on "just don't make catastrophic mistakes anymore".

What I think will happen is AI will write code and it will do the best it can to mitigate mistakes prior to rollout, but once rollout time occurs, rollout will be incremental and it will self monitor by defining success conditions at rollout time. The nature of the code will mitigate "catastrophe" to a small group at worst, but most likely initial rollout will just run new versions of the code in a simulated context (language design could benefit from this) and analyze potential outcomes without affecting current functionality.

But when the code goes live... it will be slowly scope changes progressively (think feature/experiment flags) and if it fails in the initial cohort, it will redirect. If success is positive, it will increase the rollout cohort.

This is a normal software engineering practice today, but it's labor and process intensive when driven by humans. But in a world where humans are less involved, this process is scalable.

▲

bulbar 2 hours ago | parent | next [-]

This assumes failures can be detected and fixed more easily than generating the corresponding change. I am not convinced that's the case.

Counter points to my own arguments:

1. We don't know yet in detail what AI is good at.

2. AI doesn't need to be perfect, just "good enough", whatever that means for a specific project. More failures while saving hundreds of thousands dollars each year might be acceptable, for example.

	▲	throwaway041207 an hour ago \| parent [-]
		> 2. AI doesn't need to be perfect, just "good enough", whatever that means for a specific project. More failures while saving hundreds of thousands dollars each year might be acceptable, for example. This I think is the unexplored aspect of what's happening right now. Guardrails around "good enough" systems is where the future value lies. In the future code will never be as good as when the artisans were writing it, but if you have an automated process to validate/verify mediocre code (and kick it back to AI for refinement when it fails) before it's fully productionized, then you have a pathway to scaling agentic coding.

▲

amoss 30 minutes ago | parent | prev [-]

> rollout will be incremental and it will self monitor by defining success conditions at rollout time.

This sounds a lot like allowing an LLM to define tests as well as implementation, and allowing the LLM to update the tests to make the code pass. Recently people have come to understand (again?) that testing and evaluation works better outside of the sandbox.

▲

throwaway041207 2 hours ago | parent | prev [-]

> Another point: Software Engineering always starts where tooling capabilities stop. You don't get a competitive advantage by building without engineers what anybody everybody else can build without engineers.

I'd note here that the long arc of software engineering has been commodifying the discipline into tooling. Ask any unix greybeard how shitty modern abstractions are and they'll give you all you can stomach and yet the wheel turns despite their treasured insights.