| ▲ | ACCount39 10 hours ago | |
The orthogonality thesis cuts both ways there. On one hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may have no issue whatsoever doing terrible things. On the other hand, AI doesn't have to share human wants, needs and morality - and that means that an AI may see no issue with just doing whatever humans want it to, without ever doing human things like "wanting freedom". The problem is, of course, that we kind of suck at both assessing and shaping AI wants, needs and morality. | ||
| ▲ | Calazon 6 hours ago | parent [-] | |
The other problem is that the AI may well do what humans tell it to rather than what they want it to, in the style of a literal genie or paperclip maximizer. It can lead to catastrophe without ever being conscious or wanting freedom, or anything like that. | ||