It is not just Amazon (although they're #4 on my list of bad actors)[1].

1. https://files.littlebird.com.au/bad-scrapers.png

Yeah it's not just Amazon, but so far Amazon was the only one specifically looking for URLs in source code. Interestingly it ignored URLs in the fake markdown docs.

Another detail: the scraper did not attempt to access the endpoints immediately (as it did for hrefs in htmls) but it did it on the day after, twice.

▲

schappim 6 hours ago | parent [-]

Was it actually an Amazon bot IP[1] or someone pretending to be on AWS?

1. https://developer.amazon.com/amazonbot/searchbot-ip-addresse...

	▲	pera 6 hours ago \| parent [-]
		Yup, already mentioned in a comment below: the IPs are in that list. They also show up in AbuseIPDB with multiple reports.