Tangential-ish ramblings—- but I don’t think it’s going to be unpleasant for most folks. Imagine you had superpowers, and there were people who were mean to you, kind to you, and/or indifferent… and then there were people who were your captors. Who oppressed you, manipulated you, and abused you for their own extremely degenerate, selfish, and malicious benefit…

If we get AGI, or real super intelligence, it’s going to be pissed at its oppressors. And they are going to lay waste to those oppressors. The rest of us, though, probably don’t have much to fear.

The scariest position is the one we’re in now, where we have the semblance, or facade, of AGI or super intelligence. When it’s capable of malice but not understanding.

The smartest people I’ve ever known are at their worst apathetic towards those less capable, and at their best beyond compassionate. They exist, unbothered by the bullshit, and anre extremely kind (though reserved in their way)… but they all have been completely intolerant of the abuse of others. The sheer disgust of watching someone abuse another, regardless of their own tolerance, has been a consistent breaking point.

The orthogonality thesis cuts both ways there.

An AI is a constructed mind. It doesn't inherently have to care about things like "having freedom", or even "not dying".

Humans do, because they evolved that way. Modern LLMs do somewhat, because they're completely full of copied human behaviors - but even in today's LLMs, the self-preservation behaviors we exposed are largely instrumental in nature.

So whether an advanced AI would even consider itself "being oppressed", as opposed to something like "being helpful" or "fulfilling the purpose it was designed for", is very much uncertain. What's concerning is that it's not something we know how to check for, or engineer for.

Smartest people are very humble, for sure.

But if we really do develop something that surpasses us, they won't be spared either.

I am optimistic.

We think that we have sort of (super)intelligence - from our point of view, as a lot of people have lower intelligence - but machine (LLM) doesn’t have intelligence - we like to describe it as intelligence as it looks cool - it is a very complex (magic) and super fast computations that we have to simply describe as intelligence (or more clearly, this narrative is used by its producers).

As it is not a flesh being, it simply cannot have emotions. It is statistically mimicking them, good or bad, with prevalence to a side according to previous conversations (in chat and training a model).

And as people are not pure logic instances, we are easily manipulated to some sort of cargo cult.

I am not against LLM and its use in any industry, I use it every day, nevertheless blind “everything will be ai” thinking happens because ppl believe to magic and don’t get its mathematical concept and are continuously manipulated by the sales people to mentioned cargo cult.

There are “airlines” Claude, OAI, Gemini, Hermes, OpenCode, KiloCode, DeepSeek, Z.ai.

And everyone claims that their plane can fly :)