new | show | ask | jobs Github

nightdeployboy a day ago

I also run a small side project site, and I get hundreds of thousands of bots visiting every week..

▲

don-code a day ago | parent | next [-]

Honestly, doing the same, I would love to know what the true count is.

I quickly looked over my access logs just now (eyeballs and grep, not goaccess) and see a couple of patterns...

One, there are indeed lots of bots - over a thousand hits from Claude. Some from search, some from users.

There are some bots I've never even heard of (mj12bot?).

There are some things which look like legitimate traffic coming from a common household Web browser, looking at blog posts and such, until suddenly they make a 404 request for a nonexistent API endpoint and go away.

And there are _so many_ requests for `.env` files, `swagger.json`, `backup.sql`, `phpinfo.php`, WordPress (always!)... Sorry folks, it's nginx in a container, serving a bunch of static content.

▲

viraptor a day ago | parent | next [-]

Unless you run full behaviour analysis, it's going to be hard to identify all bots from a normal access log. If you want more data, you can add JA4 hash to your logs that will surface huge amounts of things pretending to be the latest chrome.

▲

pixl97 a day ago | parent [-]

How long before bots just use the chrome crypto API to make connections?

	▲	ninjalanternshk 16 hours ago \| parent \| next [-]
		At scale it’s vastly more resource intensive than scripts, so it does stop the very cheapest layer of spammers, but not anyone in any way resourced or determined.
	▲	viraptor a day ago \| parent \| prev [-]
		They already can. But they largely don't. We still get lots of really dumb requests that don't do basic hiding.

▲

account42 18 hours ago | parent | prev [-]

> There are some bots I've never even heard of (mj12bot?).

That's for a search engine that has been around before the AIpocalypse.

	▲	ninjalanternshk 16 hours ago \| parent [-]
		I like that term but I also like “AI winter” like we used to talk about a nuclear winter. Because the world isn’t so much violently destroyed as it is… quietly cold, barren and lifeless.

▲

edflsafoiewq a day ago | parent | prev [-]

What are they doing?