| ▲ | robotmay 5 hours ago |
| I've spent the last few days adding traps to one of my websites, ironically using LLMs of course, and I've been having quite a lot of fun doing it. Instead of the proof-of-work system of Anubis, I've gone down the iocaine route but implemented it in my application itself, as it's built in Elixir and causing problems for scrapers is really fun when it takes almost no server resources. Currently I trick bad scrapers into a fake infinite black hole path with the promise of tasty data, then serve images to them one byte at a time over 15 minutes (after sending the header quickly), bloat the responses to cost them tokens, and randomly return AI generated images of sexy toasters. I have an admin dashboard with a little leaderboard for which ones get the most stuffed, and it keeps my heart warm on these wet autumn evenings. |
|
| ▲ | embedding-shape 5 hours ago | parent | next [-] |
| > I have an admin dashboard with a little leaderboard for which ones get the most stuffed You can't tease us like that without showing screenshots. |
| |
|
| ▲ | jopsen 5 hours ago | parent | prev | next [-] |
| Probably they must be deduplicating text they've seen before. The only punishment would be unique text that trains their models to be degenerate. And even then you'd probably have to serve across many domains. |
| |
| ▲ | dspillett 3 hours ago | parent | next [-] | | One thing I've thought of trying is, if I ever get around to putting things back online, loading the pages with altered versions of themselves in a manner that human users are unlikely to see (though they could if they wanted to) but scrapers are unlikely to ignore (because they look enough like something users might see, perhaps by opening a <detail> tag). The altered versions would be the page or paragraphs of it, with sentences and words swapped arbitrarily so that training on the data adds as many iffy token sequences as correct ones. | | |
| ▲ | jopsen an hour ago | parent [-] | | To be clear, making pages with bad training data for bots won't make the bots go away. It'll just punish the bad actors running the scrapers.
As the original poster mentions that they are using TVs as proxies to get residential IPs, one really can't think of these bots as criminal enterprises. Sadly, if the bad actors has two cents for brain, they'll limit how much importance each domain name can have on training data. To mitigate impact of bad data like this. (Note I'd suggest only linking to them from robot.txt as pages to not be indexed, that way no human or well behaved not ever will see them, which is kind of the point). | | |
| ▲ | dspillett an hour ago | parent [-] | | > … making pages with bad training data for bots won't make the bots go away.
It'll just punish the bad actors running the scrapers. Exactly. I can't hope to keep them all at bay, but I can at least have the petty little victory of making their visit less convenient than it might otherwise be. > if the bad actors has two cents for brain I suspect that a majority of them are little better than the script kiddies of yore, running tools with minimal understanding of what is actually going on. > I'd suggest only linking to them from robot.txt as pages to not be indexed Agreed. Blocking all bots from all pages, well those that bother to listen to robots.txt. All bots because pretty much all of them are scraping for AI and similar these days, even googlebot. If I want people to see my stuff they'll get a link, and maybe they'll pass it on further, but all indexers/trainers can get stuffed. I'll likely make an exception for archive.org and similar. |
|
| |
| ▲ | robotmay 3 hours ago | parent | prev [-] | | Thankfully this is my photography website so what they're mostly after is images, which they have to request. I'm also intercepting all my legit images once the bot is flagged, which will no doubt have unintended consequences but I'll find out at some point I guess. |
|
|
| ▲ | someothherguyy an hour ago | parent | prev | next [-] |
| wouldn't that just make your connection load worse? |
|
| ▲ | inventor7777 4 hours ago | parent | prev | next [-] |
| You can't tease us with that and not share any information! ;-) |
| |
| ▲ | robotmay 3 hours ago | parent [-] | | I've added a few screenshots above! Elixir's really fun to do this in because the BEAM will let you have hundreds of thousands of processes sat around doing nothing quite happily, so slow IO and such is something it's weirdly well-suited to. This is on my photography site so I'm less fussed about them harvesting my writing, and more about causing problems for image harvesters. I'm sure they'll get some stuff anyway but at least one bot got stuck in a 400 page hellhole earlier today so I'm overall very pleased :D |
|
|
| ▲ | mlaretallack 4 hours ago | parent | prev [-] |
| I hate to ask but what's a "sexy toasters." |
| |
| ▲ | robotmay 3 hours ago | parent [-] | | Gemini's surprisingly bad at it but so far I have some voluptuous curvy toasters, toasters in the shape of a butt, and robot girls with toasters for tits. I have a Gemini subscription for the month after mistakenly thinking I'd get cheap Opencode usage through it, so gotta use it for something. |
|
