| ▲ | Aurornis 2 hours ago |
| > One tanh call on the right input is a per-OS signature. Claim macOS, return Linux math bits, and you have contradicted your own User-Agent. They (or rather the LLM that wrote this) missed that this is possibly fingerprintable to browser version range, which is slightly more interesting. Most users aren't spoofing their user agent headers to be a different operating system. Most fingerprinting solutions aren't trying to infer your operating system, they only care about semi-unique things that show up. It's an interesting finding. I wish they had taken some time to have a real person write it up. This is too heavily LLM written to ignore. |
|
| ▲ | jeroenhd 2 hours ago | parent | next [-] |
| > Most users aren't spoofing their user agent headers to be a different operating system. The people behind the LLM behind this blog post are. They're trying to pretend their robots are people to sell other websites' data to their customer. It's easier to pass bot detection gates if you pretend to be a physical machine running Windows or macOS than if you honestly admit you're using Linux on a VM. |
| |
| ▲ | nradov 18 minutes ago | parent | next [-] | | The HTTP User-Agent header was a mistake from the beginning. There is no legitimate need for the server to know what software the client is (or claims to be) running. | | |
| ▲ | esrauch a few seconds ago | parent [-] | | I feel like this is with 2026 view where browsers are so mutually compatible. In the bad old days there were so many differences between html, css and js behaviors that if you wanted your site to be correct you had to change it for that site. Feature detection was rarely viable for any of this. No user agent would probably have only entrenched IE6 dominance even more by blocking you from making a site that works at all on other browsers and even made it so IE7+ couldn't make the standards conformance progress that they did. |
| |
| ▲ | acters 2 hours ago | parent | prev | next [-] | | It's sometimes easier to lie than to tell the truth, and being on Linux telling the truth gets me more scrutiny than those pretending to be legit. | |
| ▲ | baby_souffle 23 minutes ago | parent | prev | next [-] | | Some scrape activity happens because I can't obtain the data any other way. I would be thrilled if certain retailers had price and availability data as an API so I could not bother with the bulk of scrape and process. | |
| ▲ | reactordev 2 hours ago | parent | prev [-] | | The Internet is a cesspool of scams now | | |
| ▲ | pocksuppet 2 hours ago | parent | next [-] | | scraping, however, is not intrinsically a scam. | | |
| ▲ | mplewis 2 hours ago | parent | next [-] | | It is when you're doing it like the LLM companies are: at scale, to the degree that you're taking down my site, without my consent by masking your user-agent, for the purpose of stealing data I didn't authorize you to have. | | |
| ▲ | Bender 38 minutes ago | parent | next [-] | | I documented some crude methods that can stop most of that without a CDN. [1] There will be some false positives so I guess it depends on ones priorities which methods if any to implement or test on a throw-away test site. Not perfect, nothing is. I am watching hundreds of bots sending SYN's and the daemons are oblivious to them. The only method I have not played around with yet is #7 ssl fingerprinting. There are additional methods I chose not to document such as limiting access to logged in accounts that require double-opting-in to acceptable use policies and terms of use, not that most scrapers would give a toss. That it too much whack-a-mole for me personally. That method requires progressively adding friction to account creation and that comes with some pros and cons. [1] - https://nochan.net/b/Internet-Crap/20260606-How-To-Block-Som... | |
| ▲ | nradov 14 minutes ago | parent | prev | next [-] | | What data is being stolen? Are you referring to copyright violation or something else? If you don't want LLM companies to scrape a site then just restrict access to authorized users. Simple. | |
| ▲ | lsaferite an hour ago | parent | prev | next [-] | | Unfortunately the response to that right now is to nuke everything that isn't within a strict set of constraints. That helps with the bad bulk-scrapers, but hits everyone else as well as collateral damage. | |
| ▲ | VorpalWay an hour ago | parent | prev | next [-] | | Something like Anubis in front of the server to protect it might be an option. It sucks that we have to resort to that yes, but it seems the least bad option currently (better than the entire internet going through Cloudflare at least). | | | |
| ▲ | userbinator an hour ago | parent | prev [-] | | The idiotic "stealing" argument again? At least use "piracy" if you want to be correct... The moment you openly publish information on the Internet, you have already given consent. There are other solutions to bandwidth usage. User-agent discrimination should be illegal. All it does is further the control that Big Tech has, and help authoritarian governments with their control too. | | |
| ▲ | pdpi an hour ago | parent [-] | | As with anything IP-related, the data is licensed under whatever terms you choose to impose. That’s the extent of the consent you’ve provided. | | |
|
| |
| ▲ | duped 33 minutes ago | parent | prev [-] | | No it's just intrinsically antisocial behavior | | |
| ▲ | GaryBluto 22 minutes ago | parent [-] | | Scraping has never "intrinsically" been anti-social behaviour, you just don't like it. |
|
| |
| ▲ | Mistletoe 16 minutes ago | parent | prev | next [-] | | Has the internet made the average human’s life better? As someone that lived in both eras I have to say no. | |
| ▲ | d1ss0nanz 2 hours ago | parent | prev [-] | | Always was. |
|
|
|
| ▲ | joahnn_s 2 hours ago | parent | prev | next [-] |
| You can only assert >148 at the moment, but there are better vectors to strictly assert the version by simply checking the addition of v8/blink on each chromium version (and since ~120 it's the case), so by checking if xxx is present and yyy is not present in js userland or css feature, the inference is 100% for the major version And for the LLM writing, yes, it's written in the article and blog, it's not hidden or pretending, otherwise I would never publish an article due to lack of time, and I stand by it |
| |
| ▲ | Arch-TK 42 minutes ago | parent | next [-] | | It takes less time to write the prompt, you could just publish that? It's an important topic, and I am glad you wrote about it, but even half a page of notes would have been enough to convey this. It would save me literally skim reading headings just to get past all the fluff. | |
| ▲ | georgemcbay an hour ago | parent | prev [-] | | > otherwise I would never publish an article due to lack of time, and I assume Didn't even have time to finish their HN reply. |
|
|
| ▲ | comex 2 hours ago | parent | prev | next [-] |
| This can be used to fingerprint version range, but so can a million other things. Browsers are constantly adding new features and fixing bugs, most of which can be detected from JavaScript. |
|
| ▲ | userbinator an hour ago | parent | prev [-] |
| The LLM has a good point. I personally don't care what or who wrote it if it's true. |
