> If LLMs get better at writing

They definitionally cannot, for the definitions implied in the argument. The point is that good writing is a thing humans are capable of doing because they are human.

> It's a sort of a claim that certain thoughts can be attributed to a human being when in fact they can't.

Yes. And this is a requirement of "good writing" as understood here.