in the limit the honeypots would be dynamic and some of them secret, so at the very least a rogue agent would have a significant chance of stepping on a landmine.

this game doesn't favor the agents, the honeypot could be as simple as a text filter watching for kernel source code entering the LLM context or as complex as reading certain memory pages in the sandbox.

LLM's aren't magic, to exploit they must probe. and all probing is active.

i feel like that's still a kind of security by obscurity. the LLM isn't guaranteed to trigger a booby trap.

in my opinion there's no _fundamental_ reason between an AI agent and a human adversary attacking your code, just a practical difference. in the long run the only way is to fix the bugs.

but i agree with you that in practice, it's also a question of cost effectiveness. booby-trapping your code is probably making attacks cost prohibitive for almost everyone (which is the same situation as before, when attackers had to employ security researchers/hackers or buy zero-days from 3rd party providers).

running SOTA models is expensive, so the attackers aren't going to do drive-by attacks. next generation cheap models might do the same, but they can also be used by the defenders.