| ▲ | pera 7 hours ago | |||||||
Yeah it's not just Amazon, but so far Amazon was the only one specifically looking for URLs in source code. Interestingly it ignored URLs in the fake markdown docs. Another detail: the scraper did not attempt to access the endpoints immediately (as it did for hrefs in htmls) but it did it on the day after, twice. | ||||||||
| ▲ | schappim 6 hours ago | parent [-] | |||||||
Was it actually an Amazon bot IP[1] or someone pretending to be on AWS? 1. https://developer.amazon.com/amazonbot/searchbot-ip-addresse... | ||||||||
| ||||||||