Which companies are we talking about here? In my case the traffic was similar to what was reported here[1]: these are crawlers from Google, OpenAI, Amazon, etc. they are really idiotic in behaviour, but at least report themselves correctly.

[1]: https://pod.geraspora.de/posts/17342163

▲

nemothekid 6 days ago | parent [-]

OpenAI/Anthropic/Perplexity aren't the bad actors here. If they are, they are relatively simply to block - why would you implement an Anubis PoW MITM Proxy, when you could just simply block on UA?

I get the sense many of the bad actors are simply poor copycats that are poorly building LLMs and are scraping the entire web without a care in the world

▲

rnhmjoj 6 days ago | parent [-]

> why would you implement an Anubis PoW MITM Proxy, when you could just simply block on UA?

That's in fact what I was asking: I've only seen traffic from these kind of companies and I've easily blocked them without an annoying PoW scheme.

I have yet to see any of these bad actors and I'm interested in knowing who they actually are.

	▲	whatevaa 6 days ago \| parent [-]
		Huawei. Be happy that you haven't been hit by them yet.