They don't generate keyword search queries very well. They can overcome this by brute force but if you watch what they search you will cringe.

nhl toronto scores nhl hockey toronto scores "nhl hockey" toronto score today nhl "hockey score toronto" "hockey" who won toronto

etc.

Somehow being good at semantic search makes them bad at keyword search, for whatever reason.

I suspect that this behavior is a learned adaptation. And that it's most likely a feature not a bug.

Based on personal usage, I think it reflects functional degradation of search engines. I've found LLM keyword combinations are more likely to find the results I want with most search engines than mine. Including the big one.

The big one had solved this issue a long time ago by generating those associated keywords based on your input keywords, but somehow, something, somewhere has degraded that system to the point of inanity. And so here we are.

Can confirm; Claude is quite bad at this by default. Need a special skill

I’ve noticed this too but it hasn’t been obvious to me that this style of search is not a learned behavior. Tool calling is very much part of the post training phase, I would expect that these style searches just naturally emerge during training. This is just my prior though.

that and always putting the “current year” at the end of the search term (so the results are more recent, I guess?), except that “current year” consistently ends up being 2-3 years ago since I guess that’s what’s in the training data (even on a harness that injects the current date)

Reminds me of using AltaVista search back in the day. Yes, it was that bad.