> you should never be using prompting to add guardrails like this in the first place

This "should", whether or not it is good advice, is certainly divorced from the reality of how people are using AIs

> you need actual restrictions in place not magical incantations

What do you mean "actual restrictions"? There are a ton of different mechanisms by which you can restrict an AI, all of which have failure modes. I'm not sure which of them would qualify as "actual".

If you can get your AI to obey the prompt with N 9s of reliability, that's pretty good for guardrails

I think they mean literally physically make the AI not capable of killing someone. Basically, limit what you can use it for. If it's a computer program you have for rewriting emails then the risk is pretty low.