Remix.run Logo
nullbio 3 hours ago

I'm not convinced you can have an AI intelligent enough to dominate the world if it is trained on a single domain. The intelligence comes from all of the patterns and heuristics it learns across the entire web of all domains. You could train something far more narrow, but it'd be easily overcome by something more general that is tasked with maintaining human objectives.

For a seriously dangerous ASI you'd have to basically train it on everything, and then finetune it for maximal carnage. That's not something the frontier labs are going to do (at least... I hope not), and anyone attempting this with limited hardware will be outpaced by frontier lab AI or the collective of personal agents that aren't misaligned, and they can intercept it and alert on its behavior.

I imagine we'll be getting to a point shortly where anything that is key infrastructure and has the capacity to be accessed on a network will require a permanently running aligned interceptor AI to observe and monitor systems.

We're basically recreating the human immune system in digital form for the entire species.