IMO the most alarming thing currently going on with AI is the huge RL runs that give models some incentive to compete with each other and rather strong incentives to hack things, break rules and otherwise cheat. And the “cyber” initiatives are remarkably examples of doing most of this deliberately.

Of course, it’s the “frontier labs” doing almost all of this. No one is about to SFT a model that turns into Skynet on its own.