> I don't understand why people will necessarily cooperate with any intelligence, human or machine, that tells them things they don't want to hear [1]

LLMs are already more persuasive than most human writers. And LLMs are not the only form of AI: the algorithmic feeds which people now find addictive in the sense that they struggle to stop themselves from wasting time on them, got this way in part due to AI constantly optimising for just that.

Still, this is not universal. I am fortunate, because my mind is alien to the models these algorithms have of normal humans. Not autism, whatever oddity I am does not seem to be anything that anybody cared about enough to turn into diagnostic criteria, but I am definitely odd, and in this decade that's a good thing.

> This entire analysis is predicated on the assumption that intelligence alone confers some unlimited power, when both theory and observation tell us that that is not the case.

I don't see that.

Humans didn't need "unlimited power" to make it so all the other species face much the same flowchart, even if they themselves can't read or place themselves on it.

We don't need to know how a butterfly flapping its wings will change the course of a hurricane a month out because we can just build for hurricanes as if they're Poisson distributed, and the butterfly isn't better off for our ignorance.

> [2]: Not only are the most intelligent people not the most powerful people,

There is a general correlation, however. The personality has to be power seeking to keep power if they get it, but for all we like to point and cast aspersions at our opponents, high office seems to always come with an above average IQ, even in the really extreme cases of stupid behaviour like Pol Pot he was multilingual and got a scholarship.

> but there are biological species that are more capable of destroying humanity than the other way around, even though they are far less intelligent than humans.

Humanity? All of it? Name one.

▲

pron 42 minutes ago | parent [-]

> LLMs are already more persuasive than most human writers.

And writers are well known to be the true holders of power in the world... In any event, you can see that even among humans, the most persuasive aren't always the most intelligent.

> the algorithmic feeds which people now find addictive in the sense that they struggle to stop themselves from wasting time on them, got this way in part due to AI constantly optimising for just that.

There are many addictive things, most of which have little to do with intelligence.

> because we can just build for hurricanes as if they're Poisson distributed

Hurricanes are not particularly important components of power in society. The economy on the other hand is very relevant. Even to the extent it can be controlled, which is a fairly large extent, there's unlikely to be a consensus on the desired outcome.

Anyway, people will have to choose to listen to a machine that claims it's very intelligent even though it will make plenty of mistakes (as intelligence often does). Is that how you know people to behave?

> Humanity? All of it? Name one.

Some viruses. Probably some bacteria. Possibly some insects. I don't necessarily mean one single species, as they're all less intelligent than people.

▲

ben_w 9 minutes ago | parent [-]

> And writers are well known to be the true holders of power in the world...

Martin Luther, Adam Smith, Karl Marx. All kinda famous for things they wrote, which changed the world because of those writings.

> In any event, you can see that even among humans, the most persuasive aren't the most intelligent.

Can you? Again, correlation. The village idiots don't generally get heard; the professors tend to be.

> There are many addictive things, most of which have little to do with intelligence.

Missing my point: these are not natural things found in the wild whose addictive nature is pure coincidence of biology, these things were made addictive by applied intelligence.

> The economy on the other hand is very relevant. Even to the extent it can be controlled, which is a fairly large extent, there's unlikely to be a consensus on the desired outcome.

An AI that has taken over doesn't need consensus, it only needs "the economy" to the extent that the economy provides specifically what it needs.

> Anyway, people will have to choose to listen to a machine that claims it's very intelligent even though it will make plenty of mistakes (as intelligence often does). Is that how you know people to behave?

Yes. Look at all the people who keep listening to Musk despite https://en.wikipedia.org/wiki/List_of_predictions_for_autono...

Or same for Trump or Bush, both re-elected. Trump's second egregiously so, if you think people punish mistakes. And those two were both in a democracy; the British before universal suffrage had "Their's not to reason why, Their's not to make reply, Their's but to do and die"(https://en.wikisource.org/wiki/Maud,_and_other_poems/The_Cha...), while Chairman Mao had the disastrous Four Pests, Stalin the Holodomor, and Pol Pot… where do you even begin with him?

> Some viruses. Probably some bacteria. Possibly some insects. I don't necessarily mean one single species, as they're all less intelligent than people.

You said "biological species that are more capable of destroying humanity than the other way around".

None of those are more capable of destroying humanity than the other way around. We have, in fact, entirely eliminated e.g. smallpox, but no disease, not smallpox, not HIV, not even the Black Death, was ever capable of wiping out the entire human species. We've got plans for gene-drives to wipe out (malarial) mosquitoes and the only reason we're holding back is uncertainty about the side effects on the broader ecosystem. If anything, our biggest problem in this regard is wiping out species and then regretting it later.

	▲	defrost 4 minutes ago \| parent [-]
		> The village idiots don't generally get heard; the professors tend to be. Populists persuading masses and leading them on don't tend to be professors. See "Professors" Mussolini, Hitler, Yaxley-Lennon, Trump, et al - not exactly the sharpest crayons in the sun.