> A popular theory is that this is because of sloppy coding, AI companies are too rich to care, but then again that doesn't really add up

I can substantiate this a bit. Verified traffic from Amazonbot is too dumb to do anything with 429s. They will happily slam your site with more traffic than you can handle, and will completely ignore the fact that over half the responses are useless rate limits.

They say they honor REP, but Amazonbot will still hit you pretty persistently even with a full disallow directive in robots.txt

How do you know it's Amazonbot?