This is what bothers me a lot. For the people who doesn't know how it's made or want to believe, it's a miracle.

For me, it's a resource wasting text generator. I'll not lie, I don't use OpenAI, Mistral or Anthropic's models, even for coding. I prefer to read my API docs and cry once.

I used Gemini, five or six times in total. Twice I asked a couple of very specific things, and it unearthed them. Since they were not products, but information, that was helpful. Twice, it has given wrong information. When I "told" it, there was another way, it said "of course there are two ways", etc. Tasteless and time wasting.

I don't like using an LLM all day long, or offload my thinking to them. It's the ultimate self-poisoning incident.

And as you say, these algorithms can't know right/wrong/logical/bullshit, etc. They just spew out text.

Something I’ve also seen multiple times is an LLM giving wrong information, I tell it it’s not right, then it tells me I’m “absolutely right” and it provides the exact same answer and tells me that one will work.