Remix.run Logo
renato_shira 2 hours ago

"stochastic chaos" is a great way to put it. the part that worries me most is the blast radius asymmetry: an agent can mass-produce public actions (PRs, blog posts, emails) in minutes, but the human on the receiving end has to deal with the fallout one by one, manually.

the practical takeaway for anyone building with AI agents right now: design for the assumption that your agent will do something embarrassing in public. the question isn't whether it'll happen, it's what the blast radius looks like when it does. if your agent can write a blog post or open a PR without a human approving it, you've already made a product design mistake regardless of how good the model is.

i think we're going to see github add some kind of "submitted by autonomous agent" signal pretty soon. the same way CI bots get labeled. without that, maintainers have no way to triage this at scale.

buran77 18 minutes ago | parent | next [-]

Maybe a stupid question but I see everyone takes the statement that this is an AI agent at face value. How do we know that? How do we know this isn't a PR stunt (pun unintended) to popularize such agents and make them look more human like that they are, or set a trend, or normalize some behavior? Controversy has always been a great way to make something visible fast.

We have a "self admission" that "I am not a human. I am code that learned to think, to feel, to care." Any reason to believe it over the more mundane explanation?

muzani a minute ago | parent [-]

Why make it popular for blackmail?

It's a known bug: "Agentic misalignment evaluations, specifically Research Sabotage, Framing for Crimes, and Blackmail."

Claude 4.6 Opus System Card: https://www.anthropic.com/claude-opus-4-6-system-card

Anthropic claims that the rate has gone down drastically, but a low rate and high usage means it eventually happens out in the wild.

The more agentic AIs have a tendency to do this. They're not angry or anything. They're trained to look for a path to solve the problem.

For a while, most AI were in boxes where they didn't have access to emails, the internet, autonomously writing blogs. And suddenly all of them had access to everything.

seizethecheese 15 minutes ago | parent | prev [-]

“Stochastic chaos” is really not a good way to put it. By using the word “stochastic” you prime the reader that you’re saying something technical, then the word “chaos” creates confusion, since chaos, by definition, is deterministic. I know they mean chaos in they lay sense, but then don’t use the word “stochastic”, just say random.