Can you do something with fail2ban? I.e. after a small number of requests resulting in 401 or 404, or a series of requests that's too fast to be reasonably from a human, ban that address for 48 hours.

If a legitimate request later comes from that address, it will probably get through.

But not sure how well iptables handles checking every connection against a list of possibly hundreds of thousands of addresses.

Fail2ban doesn’t work well against residential bot nets. A single ip generally only makes a couple of requests before rotating.