You can literally read their thoughts if you run an open model, they look like pretty human thoughts to me, albeit a neurotic human.

These aren't thoughts how humans literally think them.

I can write a program to produce a string that looks like human thinking, is it human thinking? Of course it isn't. It's such a silly comparison.