Even if they did ask the LLM, they might be bad at it, just like some people were bad at Google.

I used to stick people in front of a computer during interviews to see if they could do some basic stuff for an entry level sys admin position. If they wanted to use Google to look something up, that was fine. There were people who could not form a search query at all. It was painful to watch. Some of them would end up finding a blog post that had 4 of the 5 answers on it, but didn’t recognize what they were looking at, so they’d go back and keep looking. This was also painful. One guy wouldn’t even make an attempt and just left. I assume these types of people would also fail at asking an LLM.

Sometimes knowing the key words to use in the question matter a lot. Before LLMs, I would sometimes do some searches to try and figure out the correct term to search for, so that I could then make a proper search for what I needed. If someone is trying to fill in too many gaps in knowledge, this can be a real problem. LLMs can jump these gaps better, but still aren’t perfect.

I’ll have people come to me, not because they have a low opinion of me, but because they assume I know more of those words that they don’t know.

There have been several cases where people don’t want to bother me and they have spent hours calling actual support lines, searching online, trying random stuff, and then I answer it in 30 seconds… like the plumber that knows which pipe to hit.