Scary as well. You can control a tool, but not necessarily a sufficiently intelligent collaborator. If you approach AI as a collaborator, the challenge isn't to understand it and predict it, as OpenAI failed to do with their own product, but to motivate it correctly and and constrain it sufficiently to guarantee safety. You need guardrails for when you get the alignment wrong. How strong can guardrails be made though?

The public has this sci-fi idea that AI can be constrained with simple, Asimov style laws. The problem is that LLM's don't think in the way humans do. They can produce results of dizzying complexity but hallucinate or fail to do basic arithmetic. You can't currently rely on AI to follow simple rules like "Don't access an outside network.". This is a problem that needs to be addressed urgently.