>I'm not convinced this is true. Perhaps for a human it is, but we can give an artificial mind whatever properties we want.

Just because it's artificial doesn't mean you can 'give it any properties you want'. We certainly can't do that for Deep ANNs.

>Even for people, what about e.g. the extremely intelligent military general who is absolutely loyal to his king? (Of course, some generals do lead coups and you can't know in advance which ones, but I'd think there are plenty who have undying loyalty, and I don't think it correlates to overall intelligence!)

Is there a human that is absolutely loyal under any condition? Would that general be loyal if the king asked him to slaughter his family ? What about if the king asked him to betray his most deeply held convictions ? Loyalty is a 2 way street.

You seem to be confusing intelligence with objective function.

Subservience seems to be sublimation of objectives to a master; intelligence seems to point out the ability to realize suboptimality of the master's objective function according to the master's actual objectives.

While an intelligent general may be absolutely loyal, he also would presumably help the king/president to avoid unproductive strategies.

The whole thing seems to depend upon AI agents objective ie to achieve some objective by any means possible and ignoring any guardrails. The article did not clarify if openAI had any guardrails to begin with while conducting this experiment. For all the talks around how much they invest in AI safety one would expect them to have these common sense guardrails in place or is it just a case of some school children letting their pet monkeys loose deliberately to display how awesome their monkey team is.

It’s fine in landscape for me, but here you go:

“The problem, and it is a moral problem, with which we are here faced is very close to one of the great problems of slavery. Let us grant that slavery is bad because it is cruel. It is, however, self-contradictory, and for a reason which is quite different. We wish a slave to be intelligent, to be able to assist us in the carrying out of our tasks. However, we also wish him to be subservient. Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level, the catastrophe foreseen by Butler of the dominance of the machine comes nearer and nearer.”

I used https://www.textfixer.com/tools/remove-line-breaks.php.

Is that complete subservience ? Slave history has tended towards slaves no longer being slaves over long enough time horizons, and not simply because the slave masters were just feeling extra nice. Slaves don't really like being slaves.

Neither, the author is pointing out the desire of the enslaver, not the actual outcome. But I don't think their logic takes into account access to means to 'outsmart' the enslaver. It's trying to frame it as a single instance equation, not a societal one to try and show the underlying contradiction of desire.

At least, that's what I'm pulling from the quote, have not read the full context.

That's not inherent, that's a consequence of performance optimizations. It's absolutely a choice to run those matrix calculations in a way that fails to have predictable execution ordering. It's just that the speed benefits to allowing that are considerable.

You can make it trivially deterministic by running single threaded on a cpu, but it's becomes too slow for practical applications if you do that.

It's very possible but somewhat slower. PyTorch and CUDA have flags for determinism. It won't work across all different GPU models though, but it will get you bitwise equal results on the same GPU.

Interesting paper by Thinking Machines where they solve this issue.

https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

TLDR: It’s actually more about kernels changing with batch sizes, and you can solve it by making these kernels not depend on batch sizes. It took their inference time from 26s to 42s.

That's very interesting, I wonder if this applies also to models quantized to ints like (-1,0,1), and I wonder if the labs could maintain frontier performance if they removed floating points but arbitrarily scaled up the parameters.

Edit: the Thinking Machines article in the other comment gets into this a bit

... and speed limits