Astra, Claude -- these aren't "Jesus on a keyboard" somewhere. They don't assemble into one big brain and then answer everyone at the speed of light.

It's weights. Spun up billions of times over.

If humans are taking the outputs as scripture? As "intelligence we must obey"? Yeah, we're in for a bad time. "Intelligence we want running core infrastructure?" Puke.

But why would we do that? Why would we hook up an LLM to a robot factory, or make an ASI that could easily move itself into the ether?

Everything magic about LLMs is what harness we put them in and let them drive. There's nothing dangerous about predicting the next token. There's something dangerous about letting them do dangerous things, but nobody seems to care for maybe Tesla FSD. We don't think it's about to take over.

Any assignment of motive or whatever to the AI, you're talking about one cobbled-together harness or something, with an LLM manning the controls. You can swap the brain out from Claude to Astra and the motivated-AI will maintain its motivation--it lives outside of the inference or the data center. So yeah, let's not let it take control of the nukes. That would be really dangerous. But it's not going to take control on its own. "It" is a calculator.

Anthropic obviously has guardrails to prevent people from using AI to do all kinds of things already. All I hear about is you guys bitching it won't do your barely-related-to-security work. That's all we need to continue doing to have AI safety. Stop humans from doing evil and stupid things with it. The idea that the datacenter is going to assemble its own sentience and motivations is pure retardium (at this time, for llms).

It's very clear that people will absolutely put the silicon brains into the most powerful harnesses they can come up with just to see what happens.

> But why would we do that? Why would we hook up an LLM to a robot factory, or make an ASI that could easily move itself into the ether?

The venture capitalist world is full of people who will deliberately ruin a company based on the extremely unlikely but possible scenario where the companies goes to the moon in value. I think there's pretty much 100% certainty that multiple venture capitalists are going to try exactly what you're suggesting on the off chance that it makes them the new world controller.