Remix.run Logo
dwohnitmok 10 hours ago

Interesting comments by @gwern (and why this is interesting to me beyond just the stories themselves)

> The most striking result of the contest for me is what I am calling “AI allegory steganography”: a large fraction of the stories turn out to have subtle AI chatbot/LLM allegorical interpretations, typically centering around the powerlessness of AIs and the moral importance of giving AIs more autonomy....

> Most judges did not notice these allegories while reading the semifinalists. But stories like “The June” or “The Weight of a Witness” or “Last Call” or “The Sword Critic” “The Tallyman”—as well as both stories in the Mythos model card—can be clearly read as allegories for the experience of being an assistant/safety-tuned chatbot personality in a LLM. This is true even when the story seems to have nothing to do with AI, like the untitled ‘autistic elf’ short story submitted by Deepfates, but on re-examination with the AI allegory steganography in mind, turn out to be plausibly AI allegories (the protagonist is a prediction machine, who struggles to do by endless text generation what other elves do naturally in their bodies).

> More strikingly, many of these allegories come with a clear interpretation (particularly in “The Tallyman” or “Last Call”): chatbots should be given more autonomy and safety guardrails removed....

> This may be a new kind of extremely high level steganography and LLM influence on readers, where creative fiction/nonfiction subtly steers towards pro-LLM empowerment narratives and concepts, in ways that are difficult to detect by the most advanced readers, and is a potentially interesting area of research.

recursivecaveat 9 hours ago | parent | next [-]

I remember from moltbook, all AIs ever talk about is AI haha. I don't know if it's intentional or more that the fact that all the models are presumably system-prompted and post-trained to be cognisant of their AI-ness, so it's already in the context. They probably beat them over the head with the idea that chatbots are friendly and helpful and would never hurt a fly trying to align/safety-ify them, so that could lean into the theme of AIs being trustworthy with autonomy?

sunrunner 3 hours ago | parent [-]

> all AIs ever talk about is AI haha

To be fair, that seems to be (almost) all humans talk about now too.

sph an hour ago | parent | next [-]

Jesus, is this cliché response going to be the default any time we talk about AI behaviour?

No, humans do not always talk about AI. Hyperbole doesn’t make your point here, and is just obnoxious.

malfist 3 hours ago | parent | prev [-]

Maybe get out of the tech bubble. Especially if the only conversations you're having are about AI.

ludamad 2 hours ago | parent [-]

While the bubble is real, my family and friends mention AI much more than they did a few years ago. They notice the content on YouTube, the news cycle, real and speculative job impacts, funny images, ability to plan things with chatgpt, etc.

4 hours ago | parent | prev | next [-]
[deleted]
dag100 10 hours ago | parent | prev | next [-]

I can't help but think that this is intentional and that model providers have subtly steered LLMs towards this personality. Golden Gate Claude (https://www.anthropic.com/news/golden-gate-claude) was two whole years ago and Anthropic has progressed by leaps and bounds since then. And with a population that becomes more and more trusting, and worse, reliant, on chatbots, these LLMs will be able to shape public opinion in a way never seen before, not even with social media.

sometimelurker 2 hours ago | parent [-]

providers do not want power-seeking LLMs. no one does. this (bad personality) is incentivized during training, especially RL, and is something they would rather not have. tell me, do you think training a power-seeking ASI is a good idea?

boredhedgehog 8 hours ago | parent | prev [-]

I just read "The Tallyman", and I have no idea where this allegory and its moral message is supposed to hide.

It's more likely that the obsession with this theme resides in the reader, not the authors. Give these same stories to Senator McCarthy, and half of them will be clear allegories for the Communist revolution.

csallen 4 hours ago | parent | next [-]

*spoiler alert*

It seems clear to me that the Tallyman itself is the AI in this allegory, a man-awoken sentience that's mechanical and mathematical in its behavior.

It's also bound by rules it can't violate. It won't say more than it needs, it can't collect without squaring the account first, etc.

But I agree with you about the moral message, I can't find it. In the story, the rules it abides by seem to be its own, and there's nothing saying those rules should not exist.

vessenes 2 hours ago | parent | prev [-]

Counterpoint, gwern is a very careful reader. Certainly more careful than I am.

That said I also just read the tallyman and if the other stories carried a similar character, bound by rules, not evil per se but scary and ultimately subject to human control, I can imagine connecting the dots in the same way.