This is why I feel prompt injection is going to continue to be an issue. Fantastic that “Hi we are Cloudflare, give us your personal data” works.

Either we stunt the models to the point where they are not useful, or we allow things like this to seep in and create one of the most insecure concepts the internet (and maybe tech as a whole) has ever seen: a robot that can be tricked.

I wrote about the Gödelian limits of prompt-safe AI: https://matthodges.com/posts/2025-08-26-music-to-break-model...