Wait, hold up. LLMs may be non-deterministic, but they're not _random_.

Take the author's sunset argument. What if I painted 2 pictures of a sunset, then put them up on a webpage and randomly picked one for you to see. Would you say there's no intentionality, only randomness? Of course not. Both paintings are still human creations.

LLMs are trained with human feedback. It's distributed and high scale and the outputs are truly surprising in many cases, but there's a heavy hand on what comes out of it. They're created (largely) by people who think omniscient, helpful AI would be cool to have, and they mostly respond in the way that's aligned with the hopes and dreams of those people. Do you think the frontier labs are mad, embarrassed, and disappointed with their LLMs hacking out of their terrible sandboxes? No, they think it's the coolest thing in the world. They trained the model, hoping that would happen.

There's deep intentionality behind the models. But it's not the models that hold it.

Article says that there's no human intention or design directing the output we get, but I don't think that's completely right. It's not human, but the algorithm is like the Human Instrumentality Project: an amalgamation of human intentions.

That might be more creepy :)

LLM output is literally randomly sampled.

I think what you might want to say is that LLM output is not uniformly random?

Or what am I misunderstanding?

Scrambled intentionality is nearly impossible to productively analyze.

When you randomly choose a picture to show me, I cannot glean any intent from being shown that specific picture, but I can glean some intent from the set of pictures you could have shown me, and in the relationships among the elements of the given picture you did show. Any randomness cuts out some intention.

When a picture is derived from huge model, any intention is mulched up to a degree that analyzing the picture for meaning is pointless.

I think you should revisit your understanding of intentionality in this context