I was talking to some friends that are heavy Claude users and suggested it was bullshitting, and got some pushback along the lines of it wasn't a person and didn't have agency. I'm not sure it's that simple.

The jargon can be functionally like bullshitting. It's preferentially aping language that's over everybody's heads in hopes of not being criticized. Whether that is a person's toxic habits (it's not, there's nobody in there) or an epiphenomenon of the design of the machine is less important: it's bullshitting if it's a social behavior there to manipulate people.

(jargon included where relevant, in hopes that people who know it will see it's the right word to use :) )

If you use the definition from On Bullshit, then bullshit is a statement made without regard for truth value because of its utility to the speaker.

As such, LLMs can only bullshit.