Is AI also superior to humans at black box challenges and attacking actual targets on the internet? That seems like a really important question.

No, the search space is much more vast and the feedback loop almost nonexistent.

The reason LLMs can do CTFs so well is partially because the challenges are usually designed to avoid wasting time and to introduce a single concept without noise.