Remix.run Logo
▲ creata 2 hours ago

> Bots can use async or virtual threads and not even have to hold on to a stack while they are waiting for this bullshit, clicking on buses, etc.

Don't the bots still need to do the proof of work for Anubis? They can't "async" their way out of that work.

> Another web is possible but web populists don't really believe it.

How? Sharing knowledge with the world used to be an obviously good thing. Now it's a way to help tech companies replace people with machines. What alternative web could change this state of things?

▲dspillett 2 hours ago | parent | next [-]

> Don't the bots still need to do the proof of work for Anubis? They can't "async" their way out of that work.

The implication is that while they are waiting for the async loop to complete for one site, the bot can be scraping others that have already had the challenge completed (usually the one challenge stays in force for some time afterwards) or that have no such challenge.

Memory hard challenges will reduce the effectiveness of this, but not remove it completely.

Though that these methods are working at all implies that most (almost all?) bots are not bothering to use such async methods as there are sufficient resources not protected by them that this complication is not yet worth implementing.

▲kotaKat 2 hours ago | parent | prev | next [-]

Or they could just be more 'powerful' compute that can clear the challenge faster.

I know I'm incensed at all of this because I feel like I'm the one that endlessly waits in front of the Anubis challenge for 5-10 seconds, or keeps getting extremely difficult or brain-draining hCaptchas thrown at me with no real solutions.

And I'm stuck with low-end compute for probably the rest of time at this point - not a single machine in the house has more than 8GB of RAM. So I'm just doomed to wait and burn my time and my sanity some more.

▲jon128 an hour ago | parent [-]

The "you need to enable cookies to view this site", cookies that will be limited to a browser-container and will be thrown away when the browser is closed anyway, is a sign that this modern nu web is broken in a fundamental way.

▲jon128 an hour ago | parent | prev [-]

> They can't "async" their way out of that work.

Sharing cookies along a strata of inter-team bots is one trick.

_

Web scraping is (among the other pieces) also for the more technically inclined among the "little people" to have computers do things that people they don't have the money to hire might do. Making web scraping more difficult, which won't stop the big players who will work around the hurdles one way or another, is "pulling up the ladder".

_

> What alternative web could change this state of things?

A different architecture for trafficking web contents, one that doesn't need the "shield" of something like Cloudflare or "tar pit" of something like Anubis in the first place.

IPFS is or is something like one piece of that picture but there are other essential pieces to complete that picture.