> [P]eriodically there is a discussion on Hacker News that boils down to "all of the other engineering disciplines can make reliable predictions and deadlines; why can't software?" or "why is this company's code so shoddy?" or "why are we drowning in technical debt?".

You'd find those also have more trouble with predictions when machinery needed for the goal hasn't been delivered before.

While most software jobs today are a bespoke configuration of a solved problem, the practice of software development is new enough to remember when most software was to solve a physical problem in a new digital way. Discovering/inventing are predicted on the search space being unknown in advance, making discovery and invention unlikely to be estimated accurately.

Note, though, most software today has already been writting, and the lack of predictable delivery is because the process doesn't rigorously enforce a "first, apply known/solved software" approach.

If, in software, materials use was third party inspected and certified as it is in physical or electrical engineering, you'd find software get more predictable.