The premise of "the purpose of a system is what it does" is that if it did something the people in control of the system didn't want then they would change it, from which the implication is that they like what it does and want it to keep doing that, i.e. that's the system's purpose.

There are real instances of that happening, e.g. the purpose of zoning rules is to constrain the housing supply. It isn't an unintended consequence, there are people in actual reality who lobby for things like density restrictions under disingenuous pretenses because "we want to charge higher rents to tenants" or "we want housing prices to go up forever even if it means young people get priced out of home ownership" are not sympathetic motives.

The problem is naturally that the quip is false whenever the reason the system fails is incompetence, lack of resources/information, etc. rather than ulterior motives. Which is also why it's a meme, because it's essentially the inverse of Hanlon's razor and therefore becomes the thing people reach for when they want to insinuate malice rather than stupidity.

The problem with both is, of course, that it isn't always malice but it also isn't never malice.

The example from the thread is basically perfect, because bureaucracy is both at the same time. Some of the rules were put there by honest people with honest goals, but not all of them. Meanwhile this (see "General Interference with Organizations and Production"):

https://www.gutenberg.org/ebooks/26184

"The purpose of a system is what it does" is, in my view, a statement about accountability. It's an answer to "it causes this or that harm? Well, that's not the purpose of the system". We should reject those arguments: The purposes is what it does.

It takes a particular kind of rationalist-cult-addled weaponized autism to strawman this basic and important position. (not referring to you, of course :))

It might well be literally false in any specific case-- systems often have unintended and unexpected consequences and in large systems the unintended effects are often more significant than the primary effects. But it is never acceptable to dismiss those on the basis that they're unintended. We are no less responsible for the systems we create, control, and (often) impose upon others because of some argument about their "purpose"-- and sure, sometimes the effect truly is intended and someone is just lying about it but we don't need to go down that rathole. If a system's 'purpose' was somehow different than what it does we must adapt it or abolish it, or otherwise we must admit that we accept what it does, that we expect and intend what it does to continue, and that we who are responsible for it are responsible for all its consequences-- what it does is its purpose either by actual intent or because we failed to eliminate the system.

If the standard of care of treatment for cancer treatment results in 30% mortality, and a facility achieves that-- then we can say that it's purpose is to provide that standard of care. The purposes is what it does doesn't imply that such the facility it trying to kill people. It's purposes is to provide a service which includes in its effect a level of mortality, and its existence is desirable in spite of the death because we believe the alternative is worse, because e.g. we believe without it the mortality in those patients would be 50% instead. The statement is a rejection of often self-serving claims about motivations as excuses.

This is particularly acute for regulatory regimes (or the procedural artifacts of them)-- as they're imposed on people against their will and "just don't do that" is always an alternative. To continue existing any such scheme must justify its existence on the basis of "what it does" vs where we'd be without it-- not on some lofty argument about the intentions behind it.