Remix.run Logo
▲ pixl97 an hour ago

This doesn't work.

The first rule of working with AI is AI is smarter than you.

If AI controls the URL request then all it has to do is find a server it can read the http log from and use it as a communication channel. Worse trying to simulate reaching incorrectly configured servers probably wouldn't catch the bug internally, it would just serve a static page and you'd never even realize what it tried to do.

Also, creating fake data doesnt work. The only way to know if it works in reality is to test in reality. Your test environment can never be rich enough, and you can never be sure if the model knows it's in a test environment.

This stuff is unforgivingly hard as your model gets better.