Agents can read the binary that makes up a compiled file and detect behavior directly from that. I've been doing it to inspect my own builds for the presence of a feature.

Yeah. I think people are deluding themselves as to how capable these models are now.