From 20y experience and CS degree, I see software engineering as a constant struggle against accidental complexity. Like quicksand, every movement you make will pull you deeper, even swimming in the right direction. And like entropy, it governs all things (there are no subfields that are free of complexity). It even seems impossible to give a meaningful, useful definition, perhaps by necessity. All is dark.

But now and then, something beautiful happens. Something that used to be dreadful, becomes "solved". Not in the mathematical strict sense, but some abstraction or some tool eliminates an entire class of issues, and once you know it you can barely imagine living without it. That's why I keep coming back to it, I think.

As a species, I think we are in the infancy stages of software engineering, and perhaps CS as well. There's still lots of opportunity to find better abstractions, big & small.

▲

mmcnl 2 days ago | parent | next [-]

I'm a tech lead and pushing back against accidental complexity is basically my fulltime job.

▲

onion2k 2 days ago | parent [-]

I'm an Engineering Manager, and I think I have a similar role just applied to people processes rather than code. One nuance though - a lot of the time I suspect it's deliberate complexity designed to obfuscate how little people actually do.

▲

macrocosmos 2 days ago | parent [-]

Thats projection.

▲

onion2k 2 days ago | parent [-]

Well, maybe. It's projection, because I certainly don't make simple processes myself a lot of the time, but I do try to optimize them afterwards. I have a few decades of seeing people implement processes than I've had to use, and then had to simplify as I moved into more senior roles. I've had people push back quite forcefully when I've pointed out they do things like writing reports that no one reads or gathering data that teams ignore. People often fight for added complexity because their perception is that it's important, and that means they must be important because they're the one in control of it.

There is an element of projection because there is in most things people talk about; I'm speaking about this through my filters and biases after all. But it's grounded in a fair chunk of experience.

▲

AgentMatt 2 days ago | parent [-]

Maybe you are saying the same thing, but couldn't that be explained better by those people being afraid to be made obsolete? Or at least, afraid if having to retrain?

	▲	onion2k 2 days ago \| parent [-]
		This isn't new. I've seen it for decades, including in situations where no one is at risk. I don't think it's often a fear thing.

▲

appplication 2 days ago | parent | prev | next [-]

This was really well written and I agree with you completely. Though I am not so optimistic as a species we have much runway left to get meaningfully much farther out of that infancy.

As tech progresses and those abstractions become substantially more potent, it only amplifies the ability of small groups to use them to massively shape the world to their vision.

On the more benign side of this is just corporate greed and extraordinary amplification of wealth inequality. On the other side is authoritarian governments and extremist groups.

▲

Droobfest 2 days ago | parent | next [-]

Wealth inequality is a direct cause of authoritarianism and is not benign.

	▲	appplication 2 days ago \| parent [-]
		Agree, I probably worded it poorly but I intended more like “seemingly benign” increase in corporate profits.

▲

Joel_Mckay 2 days ago | parent | prev [-]

Perhaps, but generally annoying millions of technology people tends not to end well for firms. Usually the market simply evolves to better match the fiscal conditions.

https://en.wikipedia.org/wiki/Competitive_exclusion_principl...

The Internet itself will likely further fracture into different ecosystems. =3

▲

vbezhenar 2 days ago | parent | prev [-]

Can you provide some examples of these beautiful abstractions or tools?

▲

d0mine 2 days ago | parent | next [-]

To get perspective(we know what worked), here’s some 50+ years abstractions:

A file is a simple stream of bytes in Unix. (If you think what else it might be then compare to Multics’ segments). Separate processes that may be connected using simple standard I/O streams [pipe] (vs everything is DLL in Multics) — the concept of shell itself (policy vs. mechanism separation http://www.catb.org/esr/writings/taoup/html/ch01s06.html ).

https://retrocomputing.stackexchange.com/questions/15685/wha...

For comparison, you need a new app on iOS for what might have been a shell pipeline (hierarchical file system is absent at user level).

▲

stephbook 2 days ago | parent | prev | next [-]

Memory garbage collection, borrow checker, compile-time static typing in dynamic languages (Typescript, Python).

Language specific for JavaScript: Strict comparison operator === that disables type coercion, together with banning ==.

== allows "5" equals 5.

	▲	Izkata 2 days ago \| parent [-]
		HTML attributes are all strings, so javascript's type coercion in general was this (doesn't just apply to ==) - a way to avoid having to do explicit conversions and make values act semantically equal without having to think about types. A strict ban has always felt to me like we're leaving behind useful functionality.

▲

kelsey98765431 2 days ago | parent | prev | next [-]

Take message queues. ZMQ and the like have basically solved message passing which was a ghastly thing to worry about for many years.

	▲	convolvatron 2 days ago \| parent [-]
		I disagree. I worked at a protocol designer and implementor for years before people settled on the message queue as the universal abstraction. at the bottom end dumping serialized objects into tcp connections gets you most of the way. and at the top end there is so much leverage around locality, addressing, and transport that we are leaving a lot on the table. message queues arent at all bad, but they come with additional complexity (most of it operational), and come with a set of limiting assumptions. so my frustration is that they are now the default answer for everything, and we're ignoring this lovely design space, one that becomes increasingly important when talking about scale.

▲

ramraj07 2 days ago | parent | prev | next [-]

Its outrageous that no one pointed out git. Its the perfect example of this. I consider code version control a solved problem post git.

	▲	klabb3 16 hours ago \| parent \| next [-]
		GP here. I agree, git is the best example in the spirit of my comment. Maybe the reason it wasn’t pointed out is precisely because it’s so obviously good that it’s no longer a conscious choice, and then we forget life before it. Even those of us who experienced it.
	▲	bmitc 15 hours ago \| parent \| prev [-]
		What about source files that aren't lines of text?

▲

kccqzy 2 days ago | parent | prev | next [-]

Build tools that enforce hermeticity (cannot depend on files not declared as a dependency) and hashes files (as opposed to using timestamps). This eliminates whole classes of complaints against make.

▲

braebo 2 days ago | parent | prev | next [-]

Svelte for eliminating countless categories of complexity introduced by React.

▲

brabel 2 days ago | parent | prev | next [-]

The actor model for concurrency.

▲

whattheheckheck 2 days ago | parent | prev [-]

Read The Linux Programming Interface book