This is closely related to why Sussman and Abelson stopped teaching SICP: it is not possible to engineer software any more because systems are too complicated to completely understand and abstractions hide too many behaviors. So now we do "programming by poking" to understand what the system does instead of making it correct by construction.

http://lambda-the-ultimate.org/node/5335

We just tinker. That's all we can do to get stuff done.

Helps to have more of the quote:

He said that programming today is “More like science. You grab this piece of library and you poke at it. You write programs that poke it and see what it does. And you say, ‘Can I tweak it to do the thing I want?'”. The “analysis-by-synthesis” view of SICP — where you build a larger system out of smaller, simple parts — became irrelevant.

Software is the one discipline we can have perfection. In every other engineering there are tolerances, unknowns we have to yield to nature. In software we are god. We're just a lazy and stupid god.

Anecdotal, I had a coworker working on a new feature in a huge code base. I was trying to ignore the pseudocode planning they put on the board. After a moment of frustration I wrote down 6 conditionals that simplified what would become hundreds of new lines, to those 6 statement, and capture a few more corner cases they had not worked out yet.

Months later I got a green light on a total overhaul of the larger code base and we turned a massive code base into a fully automated and effectively bullet proof distributed system because I took the time to apply engineering principles to.the problem. The code wasn't always the most proper (I did things downright sinful) but the system was organized like an engineering project with interdependent subsystem instead of a rube Goldberg kludge of new.function on old. That overhaul too less time than any of the prior incremental feature updates prior.

There have been serious efforts at software engineering, like the OOP movement in the 1980s and 90s to construct software very methodically.

Programmers hate it and rejected it.

To be fair, it does tend to create its own pathology. Instead of a layer cake made of congealed spaghetti, you tend to get over-engineering.

https://github.com/Hello-World-EE/Java-Hello-World-Enterpris...

Software engineering leads to software over-engineering because unlike in physical material engineering there is no capital or material cost to push back against complexity. You can just add things, and add things, and add things, forever, and it costs very little (a bit of RAM and CPU but that's cheap).

I have this weird hypothesis that part of why methodical "correct" software engineering fails is that it succeeds. It is able to manage complexity, which allows complexity to grow without bound. A mountain of ugly shit will start crashing in mysterious ways if it gets too complex, which has the virtue of limiting complexity.

A root problem is that programmers tend to add complexity, not remove it, and the incentive structure of the software business tends to encourage this. Each new bit of complexity or layer is something you could build a business around, or a feature you could sell. Nobody pays for simplicity. It has value, often massive value, but it's not intuitive. In what other domain would you pay more for the absence of something? This would make sense in software since simplicity is harder than complexity, but it feels weird and wrong.

Which means software is trapped behind the evolutionary lens. And without a direct connection between goal and method, it's like an out of control path to extinction.

If that is the case any biological engineering is also impossible since no one understands the full system.

Software as science.

I’ll stick my neck out and say SICP has never been relevant to me. I’ve tinkered from embedded, to OS, to databases, to desktop applications, drivers, compilers, web applications, and all sorts of shit inbetween when I’m not reversing some archaic binary compiled in a pre-standard C++. I’ve done both idiomatic programming in a language as well as applying idioms from other languages that I liked. I’ve turned C# into Lisp because it couldn’t do what I needed it to do. I read through ARM opcode docs while I’m daydreaming of writing Ruby.

When I looked at SICP I saw one thing: an introduction to “thinking about programming” for people who will have to re-learn everything and will maybe use a few things here and there. The fundamentals matter but the application is the “everyone has a plan until they get punched in the face” phase of actual learning. There are much better ways to go about studying and learning and I found SICP lacking, among other course work that I think is taught backwards and makes it hard to reason and apply.

Read “Great Programmers” by Bram Cohen. The wisdom is there, but it’s lost on people.

Just my 2c.