| ▲ | petesergeant 6 hours ago | |
I wish blockers would distinguish between crawlers that index, and agentic crawlers serving an active user's request. npm blocking Claude Code is irritating | ||
| ▲ | specialp 6 hours ago | parent | next [-] | |
Agentic crawlers are worse. I run a primary source site and the ai "thinking" user agents will hit your site 1000+ times in a minute at any time of the day | ||
| ▲ | klodolph 5 hours ago | parent | prev [-] | |
I think of those two, agentic crawlers are worse. | ||