Remix.run Logo
palsecam 4 hours ago

I do something quite similar with nginx:

  # Nothing to hack around here, I’m just a teapot:
  location ~* \.(?:php|aspx?|jsp|dll|sql|bak)$ { 
      return 418; 
  }
  error_page 418 /418.html;
No hard block, instead reply to bots the funny HTTP 418 code (https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/...). That makes filtering logs easier.

Live example: https://FreeSolitaire.win/wp-login.php (NB: /wp-login.php is WordPress login URL, and it’s commonly blindly requested by bots searching for weak WordPress installs.)

jcynix 3 hours ago | parent | next [-]

418? Nice I'll think about it ;-) I would, in addition, prefer that "402 Payment Required" would be instantiated for scrapers ...

https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/...

kijin 3 hours ago | parent | prev [-]

nginx also has "return 444", a special code that makes it drop the connection altogether. This is quite useful if you don't even want to waste any bandwidth serving an error page. You have an image on your error page, which some crappy bots will download over and over again.

palsecam an hour ago | parent [-]

Yes @ 444 (https://http.cat/status/444). That’s indeed the lightest-weight option.

> You have an image on your error page, which some crappy bots will download over and over again.

Most bots won’t download subresources (almost none of them do, actually). The HTML page itself is lean (475 bytes); the image is an Easter egg for humans ;-) Moreover, I use a caching CDN (Cloudflare).