> Of course, the more you know about a subject, the less convincing the AI's responses are.

This is said all the time by AI skeptics and I think it's right in some areas and massively wrong in others.

I know (or at least assume I know) a lot about certain coding domains where frontier models also show convincing ability. And we know that frontier LLMs really do excel in some areas of mathematics (i.e. when an inexpert human was able to prompt the models to derive a closer bound on the Riemann Hypothesis).

OTOH I know those same models struggle to do things I'm not an expert in (e.g. writing English in a captivating way) because I read their output and have taste.

> when we interact with an AI, we hallucinate the person on the other side of the interaction. Those hallucinations are far more common and far more consequential than any AI-generated "hallucinations" (these are more properly called "errors" or "defects").

I recently used Claude to study for a technical exam. I had uploaded the official certification guide to Claude and instructed it to answer my questions using only the guide and to cite it's sources from the book when it provided answers. I was using Fable when it was free w/ the pro plan and I was genuinely impressed at how it could explain things when a concept was unclear to me.

I did pass the exam, partly due to this study method. Admittedly, once I passed, I caught myself thinking that I should tell Claude that I passed and then felt embarrassed with myself for thinking that.

Wait, hold up. LLMs may be non-deterministic, but they're not _random_.

Take the author's sunset argument. What if I painted 2 pictures of a sunset, then put them up on a webpage and randomly picked one for you to see. Would you say there's no intentionality, only randomness? Of course not. Both paintings are still human creations.

LLMs are trained with human feedback. It's distributed and high scale and the outputs are truly surprising in many cases, but there's a heavy hand on what comes out of it. They're created (largely) by people who think omniscient, helpful AI would be cool to have, and they mostly respond in the way that's aligned with the hopes and dreams of those people. Do you think the frontier labs are mad, embarrassed, and disappointed with their LLMs hacking out of their terrible sandboxes? No, they think it's the coolest thing in the world. They trained the model, hoping that would happen.

There's deep intentionality behind the models. But it's not the models that hold it.

I think to me LLMs had the effect of noticing much more the author, the intention behind human-made works of art (books, movies etc.). Before LLMs, I used to frequently consume media in a way as it were generated by a mindless process. Now it's like everything which is not AI-generated has more meaning than ever before, a bit like hypomania.

This article is missing discussion of something that I recently read: as people interact with LLMs more, they end up seeing other people as less human.

In other words, the more you talk with claude, the more you tend to interpret all messages as coming from an LLM, and less worthy of respect

https://myscp.onlinelibrary.wiley.com/doi/full/10.1002/jcpy....

It's ironic that real atheists like Richard Dawkins (whose work I believe inspired the title of this post) are public about their own degree of skepticism in favor for the "personhood" of AI. And I'm confident that there are plenty others who are convinced to the tune of total disinterest in the issue of "personhood" altogether—AIs and humans alike.

This particular post is preaching to the choir of which a faith is yet to be determined or named at least. People who don't read Cory Doctorow are going to continue to anthropomorphize AI the same as they do other technologies, constructs and objects.

> systems that cannot form intent, that have nothing to form intent with

I wonder if the OP has read https://www.anthropic.com/research/global-workspace - it seems like it directly addresses this

If you go far enough in a field, you start to recognize areas where your personal opinion differs from the “best practices” usually recommended.

I think by design an LLM can’t do that. It’s built to reflect the distribution of the knowledge it has been trained on.

> To look upon an AI-generated text or image is to look upon a thing that was extruded, not chosen.

It's totally the autist in me, but the word extrusion doesn't fill me with revulsion like it's supposed to here. Maybe I've just been spending too much time 3D printing things though.

> Feeling sorrow or joy for imaginary people is weird.

No it's not. I read books and they're filled with imaginary people, and I feel for them. I'm pretty sure I'm not the only one out there that does this.

I'm heuristically less prone to continue reading an article starting with a false dichotomy, that is between how you see things as an atheist vs as a person having faith.

Maybe I am missing the point of the article, but it seems to me that there's always an intender. Maybe the intender created something that doesnt have intent, but there is always someone behind the scenes that is creates the intent behind the creation that lacks it.

I'll also say that for someone that doesnt believe in god, Corey sure has a good sense of right and wrong. Not that you need to believe in god to live a moral life.

An awful lot of effort has gone into making LLMs present as human/intelligent. Without that they would just be a prose / code / image generator and/or search assistant, and nobody would be pouring 100s of billions into the technology with the end goal of replacing expensive human labour.

Essentialism of interiority is not the antidote to the anthropomorphism of language models.

We cannot attach metaphysical baggage to art, text, or other human output because it has involves engaging in another deception, chiefly that of self alienation and selective forgetting.

1. If you show human output and label it as AI generated, viewers will dismiss and devalue it as lacking interiority.

2. If you show AI output and label it as human generated, viewers will see interiority when the is none.

Together, these failure cases demonstrate that priming determines interiority, not the content of the work. And how could it? The text is merely what's on the page, it doesn't have some intangible essence any more than the value of money exists inside our minds rather than a property of the object itself.

We act as-if things have essential properties, but we should never trick ourselves into believing that they do. To be a true believer in essence is to privilege ones mental model over reality itself. The hubris of such an act is the ultimate in self-centeredness.

> But there's a second hurdle that makes it hard for a small but important subset of humanity to understand that chatbots aren't people: the billionaires to whom nearly everyone isn't a real person. These solipsists see chatbots as being equivalent (or even superior) to humans, because they don't think most humans are fully people, either

Never thought about it that way before, but the more I do, the more sense it makes.

I'd take it a bit further, even - I don't think this is exclusive to billionaires. Many of the claims I've heard regarding AI output being indistinguishable from human creation start to make a lot more sense when you consider the person making those claims may not see the people around them as human, may not see themselves as human, or may not even have a concept of what makes a human different from any everyday object.

Too long; better ask Claude to summarize.

I stopped reading when the author described an 'atheist' sunset as being empty.

Despite mentioning a philosopher (good pick!), this is just creative writing rehashing one side of the hard problem -- or, more specifically, restating the dogma that Turing wrote his most famous paper to debunk. It's really good creative writing, at least!

There's really not much else to say, cause it's all just begging the question by assuming that dogma. Like, here:

> The fact that AI can use statistical prediction to answer questions or carry on conversations tells us something important about how regular our real world is.

Sure, it's interesting if you assume that it's "just" statistical prediction. There's a link, but it's just more creative restatements of the dogma, e.g. "But the LLM is just guessing words"

> Indeed, the chatbot is less real than the character, because the character is the product of another mind, while the chatbot's words are the product of complex mathematical operations conducted over a massive database of all the words humans have uttered, arranged by their frequency in relation to one another.

How is a fictional character the product of someone's mind, but a character generated from a massive database of words from other people's minds is not?

Professional tech Cassandra discovers ELIZA and Searle; coins a term for it. More at 10...

Seriously though, why did I just need to read that many words to get no really new content? We have known for decades that humans are predisposed to anthropomorphize chatbots, and questioning whether coherent linguistic output implies understanding (or intent) is equally old hat.

Cory Doctorow is in the business of anti-capitalism more than truth seeking of any sorts. He just hates it when private companies * do things *.

Long winding essays that don't mean much are written for a type of person who already believes in that and is in dire need of cathartic release.

This person has consistently been wrong in the past multiple times. He has made no effort to correct them nor has he even acknowledged them.

I'm clearly not the audience for this. I know. But I can't help but call it out.

Yeah, it keep going in circle