What? The Google LLM assisted search experience is... not the best option by a long shot? It's laughably incorrect in many cases, and infuriatingly incorrect in the others. It forces itself into your queries above the fold without being asked, and then bullshits to you.

A recentish example, I was trying to remember which cities' buses were in Thessaloniki before they got a new batch recently. They used to rent from a company (Papadakis Bros) that would buy out of commission buses from other cities around the world and maintain the fleet. I could remember specifically that there were some BVG Busses from Berlin, and some Dutch buses, and was vaguely wondering if there were some also from Stockholm I couldn't remember.

So I searched on my iPad, which defaulted to Google (since clearly I hadn't got around to setting up a good search engine on it yet). And I get this result: https://i.imgur.com/pm512HU.jpeg

The LLM forced its way in there without me prompting (in e.g. Kagi, you opt in by ending the query with a question mark). It fundamentally misunderstands the question. It then treats me like an idiot for not understanding that Stockholm is a city in Sweden, and Thessaloniki a city in Greece. It uses its back linking functionality to help cite this great insight. And it takes up the entire page! There's not a single search result in view.

This is such a painful experience, it confirms my existing bias that since they introduced LLMs (and honestly for a couple years before that) that Google is no longer a good first place to go for information. It's more of a last resort.

Both ChatGPT and Claude have a free tier, and the ability to do searches. Here's what ChatGPT gave me: https://chatgpt.com/share/68b78eb7-d7b4-8006-81e0-ab2c548931...

A lot of casual users don't hit the free tier limits (and indeedI've not hit any limits on the free ChatGPT yet), and while they have their problems they're both far better than the Gemini powered summaries Google have been pumping out. My suggestion is that perhaps you haven't surveyed the market before suggesting that "by far the best LLM-assisted search experience today is available for free at the Google prompt".

> The LLM forced its way in there without me prompting

I agree this is annoying but other than that I really can't follow your argument: You're comparing a keyword-like "prompt" given to Google's LLM to a well-phrased question given to ChatGPT and are surprised the former doesn't produce the same results?

It's so frustrating the way AI argumentation goes. People will cherry pick outrageously specific items and extend to crazy generalization. I mean... your phrasing was 100% ambiguous! There's no such thing as a "Stockholm bus", or "Stockholm rolling stock".

There are buses in Stockholm, and buses in Thessoloniki, and buses manufactured in Sweden, and buses previously used in Stockholm that are now in operation in Thessoloniki. And one LLM took one path through the question, answering it correctly and completely. And the other took a different one[1]. As it happened your (poorly phrased) intended question was answered by one and not the other.

If I ask the same question with a more careful phrasing that (I think!) matches what you wanted to know: "Where did buses used in Thessoloniki come from originally?"

...I get correct and clear answers from both. But the Google result also has the Wikipedia page for the transit operator and its own web page immediately to the right.

Again, cherry picking notwithstanding I think in general the integrated experience of "I need an AI to help me with this problem" works much better at google.com, it just does.

[1] It's worth pointing out that the result actually told you that your question didn't make sense, and why. I suspect you think this was a bug since the other LLM guessed instead, but it smells like a feature to me.