You can. We know the properties of materials based on experimentation. In the same way, we can statistically quantify the results that come out of any kind of spaghetti box, based on repeated trials. Just like it's done in many other fields. Science is based on repeated testing of hypotheses. You rarely get black and white answers, just results that suggest things. Like the tensile strength of some particular steel alloy or something.

> Engineering uses repeatable processes to produce expected results

this is the thing with LLMs, the response to a prompt is not guaranteed to be repeatable. Why would you use something like that in an automation where repeatability is required? That's the whole point of automation, repeatability. Would you use a while loop that you can expect to iterate the specified number of times _almost_ every time?

Practically everything engineers have to interact with and consider are equivalent to a software black box. Rainfall, winds, tectonic shifts, material properties, etc. Humans don't have the source code to these things. We observe them, we quantify them, notice trends, model the observations, and we apply statistical analysis on them.

And it's possible that a real engineer might do all this with an AI model and then determine it's not adequate and choose to not use it.

This is off the cuff, but comparing software & software systems to things like buildings, bridges, or real-world infrastructure, there's three broad gaps, I think:

1) We don't have a good sense of the "materials" we're working with - when you're putting up a building, you know the tensile strength of the materials you're working with, how many girders you need to support this much weight/stress, etc. We don't have the same for our systems - every large scale system is effectively designed clean-sheet. We may have prior experience and intuition, but we don't have models, and we can't "prove" our designs ahead of time.

2. Following on the above, we don't have professional standards or certifications. Anyone can call themselves a software engineer, and we don't have a good way of actually testing for competence or knowledge. We don't really do things like apprenticeships or any kind of formalized process of ensuring someone has the set of professional skills required to do something like write the software that's going to be controlling 3 tons of metal moving at 80MPH.

3. We rely too heavily on the ability to patch after the fact - when a bridge or a building requires an update after construction is complete, it's considered a severe fuckup. When a piece of software does, that's normal. By and large, this has historically been fine, because a website going down isn't a huge issue, but when we're talking about things like avionics suites - or even things like Facebook, which is the primary media channel for a large segment of the population - there's real world effects to all the bugs we're fixing in 2.0.

Again, by and large most of this has mostly been fine, because the stakes were pretty low, but software's leaked into the real world now, and our "move fast and break things" attitude isn't really compatible with physical objects.

Act like creating a merge-request to main can expose you to bankruptcy or put you in jail. AKA investigate the impact of a diff to all the failure modes of a software.

What is the factor of safety on your code?

https://en.wikipedia.org/wiki/Factor_of_safety