I can’t find the measurement methodology for Brightedge’s whitepaper, which concerns me because the margin for error seems huge, but Cloudflare’s numbers do make sense.

Both are however in line with my personal experience and observations regarding how people around me are using the web now.

I find myself using web crawling in LLMs a lot more, and search a lot less. My reasoning follows, and I think most people would agree.

- When I’m looking for some relatively obscure information which I’m not sure where to find or which would require hours of research for me to find, I use ChatGPT (usually o3 with deep research) and refer to citations for more information regarding a topic. This saves me hours of investigating, which I usually don’t have for something that’s just a curiosity. A friend also used deep research to find papers highly relevant to a topic he was working on for his med PhD in minutes, claiming that just searching through PubMed to find such papers would take him days - and probably less successfully.

- When I’m looking for something specific in regards to a topic I’m relatively familiar with, I use search (usually Kagi, unless at work where it is banned!) to quickly find reference material.

- LLMs (and engines like Kagi) let you skip through the SEO spam you’d usually skip through when using Google, as well as letting you search more easily (due to better natural language understanding than classical query engines). The quality of search results had been diminishing for years (geeks4geeks ranks higher than SO on Google), so it’s not surprising people turn to tools which produce better results. It’s like being shocked that people are driding cars instead of riding donkeys.

A particular example is that I looked up a DTC that my car was throwing. I googled it first, and got a results page consisting of forums that said nothing, paywalled generated sites that also didn’t provide any info, scammy Scribd clones hosting diagnostic manuals for the wrong car model and ad-ridden garbage sites that just claimed “oh it could be anything, just take it to a mechanic”. ChatGPT gave me an exact (and correct) answer in seconds.

This is an expected result of what we have done to the web, and it should surprise nobody. I’m only sad that genuine, small online communities are dying in favour of walled gardens, but that’s an entirely separate discussion.