I don’t see the sliding scale - if the website didn’t want a bot they could have blocked you. They clearly don’t mind so what’s the issue? You still used a bot.

▲

baby_souffle 2 hours ago | parent | next [-]

> I don’t see the sliding scale - if the website didn’t want a bot they could have blocked you.

The point is that not all bots are bad. Assuming as much / implementing policies to that effect won't block all bots but it _will_ block the portion of bots that represent users considering giving you money.

I have a series of bots that monitor various eCom sites to monitor prices over time for big-ticket items I am considering as well as staples/groceries and everything in between. I have this little scrape/ingest pipeline because there's no other way to obtain this data... not even an API that I can pay for access to.

One of the large appliance sellers that I have in the scrape queue has gotten _hyper_ aggressive with bot detection to the point where even my personal head-full chrome instance doesn't always get to load the page. Guess who I will never buy that ~$2000 appliance from.

	▲	volkl48 13 minutes ago \| parent \| next [-]
		If this is the case, they probably outright do not want you as a customer. Significant sales (beyond for excess stock clearance/discontinued items) exist largely to generate some excitement from potential customers and to lure them in to taking a look at the site/store and to hopefully get them to buy some extra stuff that's less discounted while they're there. The actual on-sale item is potentially losing money for the business or at break-even. ------ A person who's running a series of bots like this to track the exact lowest price on the internet for items over time and to always only buy at the lowest price it's ever been sold for....is probably the least likely kind of customer to buy anything extra that's actually profitable for the business.
	▲	jbs789 an hour ago \| parent \| prev \| next [-]
		The fundamental issue here is the way we use the internet is changing… and what we are seeing is some friction as that works its way through. I have a business where I clearly don’t want any bots. A retailer, maybe they do.
	▲	compiler-guy an hour ago \| parent \| prev [-]
		The tradeoff here is a classic false-positive vs false-negative issue. If the cost of the bad-bot false positive (which blocks your bot out) is lower than the cost of the bad-bot false negatives (which allow bad bots in), then it is still a good tradeoff, if a suboptimal situation.

▲

3 hours ago | parent | prev [-]

[deleted]