At work I had this kind of discussion on a conference call, someone looked something up about a internal company policy and it came back with a hallucinated wrong result.

So I said, don't ever trust the output of an LLM without verification. However this caused me some hassle with the AI adoption manager. We have minimum-use AI KPI's for employees and he asked me to stop saying these things or people will use it less.

In the end I just hated the company a little bit more. I'm just sick of fighting against idiot. And he does have a point, our leadership is pretty crazy about the AI hype, they want everyone to be on it all the time. They don't seem to care whether it adds value or if it even detracts.

If the tool provides a link to the raw source, you can just click that. The LLM is basically just a better search engine here.

That sounds insufferable and I'm sorry your coworker and your bosses have gotten this hypnotized. There are actual use cases, but this type of productivity cargo cult drives me up the wall.

At least if they were into filling the office with magic crystals, it would be decorative and easily ignored. This is just forcing people to spend a bunch of tokens in a dull ritual to make the line go up.