| ▲ | intended 3 hours ago | |
IMO its because they can't handle rogue activity. In the cases that I have seen covered, the AI just paper clip maximized its way to success. It has no morality / larger motivational structure. It just kept token predicting its way to wards whatever goal it was tasked with. Model versions which gave up were discarded, leaving the ones that get to success on long horizon tasks. Just because its a computer program, doesn't mean they can actually make it not go rogue. Sure you can add more telemetry, have better observation, but there is no fundamental barrier that can be implemented that ensures an AI won't go rogue. | ||