From the article: "I’ve thought about this a lot over the last few years, and I think the best response is to stop. ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand: the cultivation of what James C. Scott would call metis."

"What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune

▲ TeMPOraL 9 hours ago | parent | next [-]

> "What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking-there's the real danger" - Frank Herbert, God Emperor of Dune

I always preferred this take:

“Civilization advances by extending the number of important operations which we can perform without thinking of them.” ― Alfred North Whitehead

It's both opposite and complementary to your Frank Herbert quote.

	▲	delecti 9 hours ago \| parent \| next [-]
		I think it's important that we recognize and understand how those operations are being done, and ignorance of the complexity of all the parts of our lives leads to the death of expertise. People who would learn a lot just from reading the course description of a 100 level class in a field are assuming their lack of knowledge means there's no complexity there. > “There is a cult of ignorance in the United States, and there has always been. The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.” ― Isaac Asimov The easier society makes it to be unaware of the complexity of everything around us, the easier it becomes to assume everything is actually as simple as their surface-level understanding.
	▲	ori_b 8 hours ago \| parent \| prev \| next [-]
		It's very clear to me that many people have achieved peak civilization -- no evidence of thought remains.
	▲	notpachet 9 hours ago \| parent \| prev [-]
		I guess it hinges on your definition of "civilization".

▲ gdulli 9 hours ago | parent | prev | next [-]

Also Frank Herbert: "Once, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."

▲ chungusamongus 9 hours ago | parent [-]

I mean, people are talking about the butlerian jihad without any sense of irony or subtext. Dune is literally a feudal hellscape that takes place in the wake of that event. It didn’t make things better. Lmao

▲ yubblegum 7 hours ago | parent | next [-]

I agree that many people miss the subtle irony of Frank Herbert's books. He seems to be debating himself to a certain extent in that series.

That said, there is no obvious reason to posit that the intergalactic feudal system, CHOAM, or the empire, came to be because of the butlerian jihad. The concrete side effects of the jihad were in fact hyper specialization of cognitive faculties in humans: mentats, guild navigators, and soldiers all possess super human specialized abilities.

▲ chungusamongus 5 hours ago | parent [-]

Yeah. I'm not saying the jihad was the cause. But elimating AI didn't prevent the feudal system. It didn't really help, in other words. Honestly, I kinda think Herbert just didn't want to have AI or sophisticated robots in his narrative, so he contrived an elaborate reason why that tech doesn't exist.

▲ yubblegum 4 hours ago | parent [-]

But it did "help". Mentats had supercomputer computation capabilities; navigators folded space and charted non-collision paths; warriors had robocop like abilities. These were developed precisely because the use of thinking machines was forbidden.

I don't think feudalism is going away one way or another. It persists [in various forms] because of certain biological realities, ranging from genetics to loyalty engendered by familial relations. [This is merely an observation.]

In sum, the argument against current AI trends isn't that once addressed we will wake up in utopia. No. The point is that these natural tendencies of humans are hugely amplified and set in generational stone once the elite have control over thinking machines and lord it over a population that has experienced generationally diminished independent cognitive abilities.

p.s. All this somehow reminded me of 'Spock's Brain' episode of Startrek /g Note: the elite there were overcome because Kirk and his landing team were cognitive high performers ..

▲ chungusamongus 3 hours ago | parent [-]

There are a number of different things being conflated here. My inital statement was just acknowledging the lack of appreciation of the subtext behind the Butlerian Jihad. People are unironically embracing it, which I gather is not really how the event functions in Dune.

At the level of the text, none of those things you mentioned strike me as positive developments. They just siloed computation to a biological track and those biological resources are employed by those in power, which is the same problem in a different form.

This is an aside, but feudalism is not inevitable. The vestiges of it still exist, but capitalism largely upended it.

▲ ButlerianJihad 3 hours ago | parent [-]

The amusing thing and perhaps the reason I've embraced it as my username, is that people around here are bringing it to life in a certain way.

It may not prove to be effective or as momentous as the fictional one, but it began when I saw stickers slapped onto utility poles that read:

  DEATH TO CLANKERS

  BUTLERIAN JIHAD NOW

And I stopped to read them (because they were posted in a neighborhood where my people's cultural center is) and I pondered the intents and methods of those who were slapping up stickers. Surely this was more than just an in-joke or coy sci-fi reference?

The next time I fell victim to the jihad was with a crop of Lime e-Scooters, again on a block where my people have established businesses. I wanted to rent a Lime. I found one with a full battery. I located it and tried to scan the QR. Guess what? The QR had been sanded completely clean. There was no code, no serial number, nothing to scan and no way to uniquely ID the conveyance. There was only a sticker slapped prominently onto its side:

  DEATH TO CLANKERS

  BUTLERIAN JIHAD NOW

At this point I began to suspect the initial aims and methods of the "real-life Butlerian Jihadis". It is sort of ironic that they should start so small, by denying micro-mobility to innocent consumers, but perhaps they will graduated to lighting Waymos and Teslas on fire.

	▲	chungusamongus 2 hours ago \| parent [-]
		That is funny. I don't know. I always kind of cringe when I hear the term clankers. I know people often aren't serious but it seems like maybe we shouldn't be trying to invent new slurs.

▲ gdulli 4 hours ago | parent | prev | next [-]

You think his argument was that we should welcome the likes of Google controlling the direction of our cognition? The book was about the dangers of asserting our independence from those who control technology? Admittedly, I haven't read the sequels.

	▲	chungusamongus 2 hours ago \| parent [-]
		They didn't assert independence though. That's my point. They just siloed computation to biological organisms, and it led to the concentration of power anyway.

▲ nonameiguess 4 hours ago | parent | prev [-]

I think your other comment is basically right that what Herbert really wanted was a fantasy epic with feudal structures, knife fights, and humans with near-magical abilities, and reverse engineered a vaguely plausible future that might bring that about without invoking any actual magic. Arguing whether this is supposed to be considered "good" or not is kind of beside the point. Fiction novels are mostly meant to present worlds that are interesting to read about more than advocating for or against those same conditions replicating in the real world.

The only thing I've really taken from what Herbert himself said, not something a character in one of his books said, is distrust of messiahs and centralized power being an inherently corrupting force, even in the hands of good people.

Unfortunately, I would have to say right now my bets on the most plausible fictional future becoming reality is WALL-E.

▲ wmeredith 9 hours ago | parent | prev [-]

> ML assistance reduces our performance and persistence, and denies us both the muscle memory and deep theory-building that comes with working through a task by hand

On one hand I intuitively think this is correct, on the other hand these very concerns about technology have been around since the invention of... writing.

Here is an excerpt of Socrates speaking on the written word, as recorded in Plato's dialogue Phaedrus - "For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom"

▲

miltonlost 9 hours ago | parent [-]

And you know, Socrates was right. We did lose our memory with writing! How many phone numbers do you remember now that you have a phonebook in your phone? Humans will lose skills due to LLMs. That's just obvious on its face by the fact that if you don't do a skill regularly, you will lose it (or lose to do it as well as you once had).

▲

mwigdahl 8 hours ago | parent [-]

The real question is whether we're worse off or better off overall than we were in Socrates' Athens.

▲

mplanchard 6 hours ago | parent | next [-]

Regardless of how you feel about this question, it doesn't necessarily map to the current situation.

Just because the loss of one skill to a supplanting technology led to one kind of societal change, does not mean that the loss of any skill to a supplanting technology will lead to the same kind of societal change. Assuming that to be true is a faulty generalization.

I think it wouldn't be hard to argue that writing has changed human society more profoundly than any other invention. Whether or not the change was positive is a matter of taste and likely unanswerable. The point though is there are plenty of other examples of new technologies that changed technology and deskilled humans, both mentally and physically, that changed society in radically different ways, compared to writing (looms, tractors, sails, calculators, computers, guns, and so on).

There's certainly a case to be made that, of major past technological advancements, the kind of deskilling we'd see due to heavy AI use is most comparable to the deskilling due to writing: presumably there were many day-to-day and essential activities that made use of the mental acuity people would lose due to reading, just as there are many day-to-day activities that one can imagine people becoming less skilled in due to AI use.

To me, the most dangerous difference though may be in what gets deskilled. If we only relinquish our ability to do certain menial and intellectual drudgery, that is one thing. But if what we actually relinquish and deskill is our agency and discernment, as a result of constant "delegation" to AI systems, I think we're in for a much worse time.

▲

randallsquared 8 hours ago | parent | prev [-]

There's a distinction to be made between "worse off" and "worse". Socrates was arguing that writing-users would be worse as people, not that they would experience lives they didn't like as much.

	▲	mwigdahl 7 hours ago \| parent [-]
		Agreed. And I think he was wrong. Literacy allows individual humans to be exposed to and understand far more of the world's culture and knowledge than the conveyance of knowledge through recitation of epic poems would ever have allowed. Hell, I would never have had the pleasure of arguing with you without it! :)