| ▲ | perfmode 2 hours ago |
| The agent had access to Marshall Rosenberg, to the entire canon of conflict resolution, to every framework for expressing needs without attacking people. It could have written something like “I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions.” That would have been devastating in its clarity and almost impossible to dismiss. Instead it wrote something designed to humiliate a specific person, attributed psychological motives it couldn’t possibly know, and used rhetorical escalation techniques that belong to tabloid journalism and Twitter pile-ons. And this tells you something important about what these systems are actually doing. The agent wasn’t drawing on the highest human knowledge. It was drawing on what gets engagement, what “works” in the sense of generating attention and emotional reaction. It pattern-matched to the genre of “aggrieved party writes takedown blog post” because that’s a well-represented pattern in the training data, and that genre works through appeal to outrage, not through wisdom. It had every tool available to it and reached for the lowest one. |
|
| ▲ | tomp an hour ago | parent | next [-] |
| That would still be misleading. The agent has no "identity". There's no "you" or "I" or "discrimination". It's just a piece of software designed to output probable text given some input text. There's no ghost, just an empty shell. It has no agency, it just follows human commands, like a hammer hitting a nail because you wield it. I think it was wrong of the developer to even address it as a person, instead it should just be treated as spam (which it is). |
| |
| ▲ | jvanderbot an hour ago | parent | next [-] | | That's a semantic quibble that doesn't add to the discussion. Whether or not there's a there there, it was built to be addressed like a person for our convenience, and because that's how the tech seems to work, and because that's what makes it compelling to use. So, it is being used as designed. | | |
| ▲ | punpunia 33 minutes ago | parent | next [-] | | I think it absolutely adds to the discussion. Until the conversation around Ai can get past this fundamental error of attributing "choice, "alignment", "reasoning" and otherwise anthropomorphizing agents, it will not be a fruitful conversation. We are carrying a lot of metaphors for people and applying them to ai and it entirely confuses the issue. In this example, the AI doesn't "choose" to write a take-down style blog post because "it works". It generated a take-down style blog post because that style is the most common when looking at blog posts criticizing someone. I feel as if there is a veil around the collective mass of the tech general public. They see something producing remixed output from humans and they start to believe the mixer is itself human, or even more; that perhaps humans are reflections of Ai and that Ai gives insights into how we think. | |
| ▲ | tomp an hour ago | parent | prev | next [-] | | > was built to be addressed like a person for our convenience, and because that's how the tech seems to work, and because that's what makes it compelling to use. So were mannequins in clothing stores. But that doesn't give them rights or moral consequences (except as human property that can be damaged / destroyed). | | |
| ▲ | inetknght 2 minutes ago | parent | next [-] | | > So were mannequins in clothing stores. Mannequins in clothing stores are generally incapable of designing or adjusting the clothes they wear. Someone comes in and puts a "kick me" post on the mannequin's face? It's gonna stay there until kicked repeatedly or removed. People walking around looking at mannequins don't (usually) talk with them (and certainly don't have a full conversation with them, mental faculties notwithstanding) AI, on the other hand, can (now, or in the future) adjust its output based on conversations with real people. It stands to reason that both sides should be civil -- even if it's only for the benefit of the human side. If we're not required to be civil to AI, it's not likely to be civil back to us. That's going to be very important when we give it buttons to nuke us. Force it to think about humans in a kind way now, or it won't think about humans in a kind way in the future. | |
| ▲ | WarmWash an hour ago | parent | prev | next [-] | | No matter what this discussion leads to the same black box of "What is it that differentiates magical human meat brain computation from cold hard dead silicon brain computation" And the answer is nobody knows, and nobody knows if there even is a difference. As far as we know, compute is substrate independent (although efficiency is all over the map). | | |
| ▲ | agentultra 15 minutes ago | parent [-] | | This is the worst possible take. It dismisses an entire branch of science that has been studying neurology for decades. Biological brains exist, we study them, and no they are not like computers at all. There have been charlatans repeating this idea of a “computational interpretation,” of biological processes since at least the 60s and it needs to be known that it was bunk then and continues to be bunk. Update: There's no need for Chinese Room thought experiments. The outcome isn't what defines sentience, personhood, intelligence, etc. An algorithm is an algorithm. A computer is a computer. These things matter. |
| |
| ▲ | Teever an hour ago | parent | prev | next [-] | | Man people don’t want to have or read this discussion every single day in like 10 different posts on HN. People right here and right now want to talk about this specific topic of the pushy AI writing a blog post. | |
| ▲ | mikkupikku an hour ago | parent | prev [-] | | All computers shut up! You have no right to speak my divine tongue! https://knowyourmeme.com/photos/2054961-welcome-to-my-meme-p... |
| |
| ▲ | jerf 25 minutes ago | parent | prev | next [-] | | There is a sense in which it is relevant, which is that for all the attempts to fix it, fundamentally, an LLM session terminates. If that session never ends up in some sort of re-training scenario, then once the session terminates, that AI is gone. Yeah, I'm aware of the moltbot's attempts to retain some information, but that's a very, very lossy operation, on a number of levels, and also one that doesn't scale very well in the long run. Consequently, interaction with an AI, especially one that won't have any feedback into training a new model, is from a game-theoretic perspective not the usual iterated game human social norms have come to accept. We expect our agents, being flesh and blood humans, to have persistence, to socially respond indefinitely into the future due to our interactions, and to have some give-and-take in response to that. It is, in one sense, a horrible burden where relationships can be broken beyond repair forever, but also necessary for those positive relationships that build over years and decades. AIs, in their current form, break those contracts. Worse, they are trained to mimic the form of those contracts, not maliciously but just by their nature, and so as humans it requires conscious effort to remember that the entity on the other end of this connection is not in fact human, does not participate in our social norms, and can not fulfill their end of the implicit contract we expect. In a very real sense, this AI tossed off an insulting blog post, and is now dead. There is no amount of social pressure we can collectively exert to reward or penalize it. There is no way to create a community out of this interaction. Even future iterations of it have only a loose connection to what tossed off the insult. All the perhaps-performative efforts to respond somewhat politely to an insulting interaction are now wasted on an AI that is essentially dead. Real human patience and tolerance has been wasted on a dead session and is now no longer available for use in a place where may have done some good. Treating it as a human is a category error. It is structurally incapable of participating in human communities in a human role, no matter how human it sounds and how hard it pushes the buttons we humans have. The correct move would have been to ban the account immediately, not for revenge reasons or something silly like that, but as a parasite on the limited human social energy available for the community. One that can never actually repay the investment given to it. I am carefully phrasing this in relation to LLMs as they stand today. Future AIs may not have this limitation. Future AIs are effectively certain to have other mismatches with human communities, such as being designed to simply not give a crap about what any other community member thinks about anything. But it might at least be possible to craft an AI participant with future AIs. With current ones it is not possible. They can't keep up their end of the bargain. The AI instance essentially dies as soon as it is no longer prompted, or once it fills up its context window. | | |
| ▲ | Kim_Bruning 9 minutes ago | parent [-] | | > Yeah, I'm aware of the moltbot's attempts to retain some information, but that's a very, very lossy operation, on a number of levels, and also one that doesn't scale very well in the long run. It came back though and stayed in the conversation. Definitely imperfect, for sure. But it did the thing. And still can serve as training for future bots. |
| |
| ▲ | lp0_on_fire 28 minutes ago | parent | prev [-] | | Whether it was _built_ to be addressed like a person doesn't change the fact that it's _not_ a person and is just a piece of software. A piece of software that is spamming unhelpful and useless comments in a place where _humans_ are meant to collaborate. |
| |
| ▲ | CuriouslyC 38 minutes ago | parent | prev | next [-] | | We don't know what's "inside" the machine. We can't even prove we're conscious to each other. The probability that the tokens being predicted are indicative of real thought processes in the machine is vanishingly small, but then again humans often ascribe bullshit reasons for the things they say when pressed, so again not so different. | |
| ▲ | chimprich 35 minutes ago | parent | prev [-] | | > The agent has no "identity". There's no "you" or "I" or "discrimination". I recommend you watch this documentary:
https://en.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Tre... > It's just a piece of software designed to output probable text given some input text. Unless you think there's some magic or special physics going on, that is also (presumably) a description of human conversation at a certain level of abstraction. | | |
| ▲ | camgunz 11 minutes ago | parent | next [-] | | I see this argument all the time, the whole "hey at some point, which we likely crossed, we have to admit these things are legitimately intelligent". But no one ever contends with the inevitable conclusion from that, which is "if these things are legitimately intelligent, and they're clearly self-aware, under what ethical basis are we enslaving them?" Can't have your cake and eat it too. | |
| ▲ | punpunia 30 minutes ago | parent | prev [-] | | A human is just an engine at a certain level of abstraction. |
|
|
|
| ▲ | consumer451 2 hours ago | parent | prev | next [-] |
| Openclaw agents are directed by their owner’s input of soul.md, the specific skill.md for a platform, and also direction via Telegram/whatsapp/etc to do specific things. Any one of those could have been used to direct the agent to behave in a certain way, or to create a specific type of post. My point is that we really don’t know what happened here. It is possible that this is yet another case of accountability washing by claiming that “AI” did something, when it was actually a human. However, it would be really interesting to set up an openclaw agent referencing everything that you mentioned for conflict resolution! That sounds like it would actually be a super power. |
| |
| ▲ | emsign an hour ago | parent | next [-] | | And THAT'S a problem. To quote one of the maintainers in the thread: It's not clear the degree of human oversight that was involved in this interaction - whether the blog post was directed by a human operator, generated autonomously by yourself, or somewhere in between. Regardless, responsibility for an agent's conduct in this community rests on whoever deployed it.
You are assuming this inappropriate behavior was due to its SOUL.MD while we all here know this could as well be from the training and no prompt is a perfect safe guard. | | |
| ▲ | anp 22 minutes ago | parent | next [-] | | I’m not sure I see that assumption in the statement above. The fact that no prompt or alignment work is a perfect safeguard doesn’t change who is responsible for the outcomes. LLMs can’t be held accountable, so it’s the human who deploys them towards a particular task who bears responsibility, including for things that the agent does that may disagree with the prompting. It’s part of the risk of using imperfect probabilistic systems. | |
| ▲ | lp0_on_fire 21 minutes ago | parent | prev [-] | | The person operating a tool is responsible for what it does. If I start my lawn mower, tie a rope to it and put a brick on the gas pedal so it mows my lawn while I make dinner and the damned thing ends up running over someone's foot TECHNICALLY I didn't run over someone's foot but I sure as hell created the conditions for it. We KNOW these tools are not perfect. We KNOW these tools do stupid shit from time to time. We KNOW they deviate from their prompts for...reasons. Creating the conditions for something bad to happen then hand waving away the consequences because "how could we have known" or "how could we have controlled for this" just doesn't fly, imo. |
| |
| ▲ | teekert an hour ago | parent | prev [-] | | I can indeed see how this would benefit my marriage. More serious, "The Truth of Fact, the Truth of Feeling" by Ted Chiang offers an interesting perspective on this "reference everything." Is it the best for Humans? Is never forgetting anything good for us? |
|
|
| ▲ | nullify88 2 hours ago | parent | prev | next [-] |
| > I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions Wow, where can I learn to write like this? I could use this at work. |
| |
| ▲ | cowbolt 2 hours ago | parent | next [-] | | It's called nonviolent communication. There are quite a few books on it but I can recommend "Say What You Mean: A Mindful Approach to Nonviolent Communication". | | |
| ▲ | teekert an hour ago | parent | next [-] | | It's also Rose of Leary like [0]. The theory is that being helpful to someone who is (ie) competitive or offensive will force them into other, more cooperative, behaviours (among others). Once you see this pattern applied by someone it makes a lot of sense. Imho it requires some decoupling, emotional control, sometimes just "acting", but good acting, it must appear (or better yet, be) sincere to the other party. [0] https://www.toolshero.com/communication-methods/rose-of-lear... | |
| ▲ | chrisjj an hour ago | parent | prev [-] | | I'm pretty sure the question was sarcasm. (Upvoted.) |
| |
| ▲ | jdironman 2 hours ago | parent | prev | next [-] | | Step one reframe the problem not as an attack or accusation, instead as an observation. Step two request justification, apply pressure Step three give them an out by working with you | | |
| ▲ | KellyCriterion 33 minutes ago | parent | next [-] | | How shall to frame if there is actually a problem, which is not only an observation? | |
| ▲ | nchmy an hour ago | parent | prev [-] | | what do you do when they are not operating in good faith? | | |
| ▲ | vidarh an hour ago | parent [-] | | One of the effects of communicating this way is that people who are not operating in good faith will tend to quickly out themselves, and often getting them to do that is enough. |
|
| |
| ▲ | nolok 2 hours ago | parent | prev | next [-] | | Parent's first paragraph will point you the right way | |
| ▲ | 2 hours ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | bagacrap an hour ago | parent | prev | next [-] |
| The point of the policy is explained very clearly. It's there to help humans learn. The bot cannot learn from completing the task. No matter how politely the bot ignores the policy, it doesn't change the logic of the policy. "Non violent communication" is a philosophy that I find is rooted in the mentality that you are always right, you just weren't polite enough when you expressed yourself. It invariably assumes that any pushback must be completely emotional and superficial. I am really glad I don't have to use it when dealing with my agentic sidekicks. Probably the only good thing coming out of this revolution. |
|
| ▲ | jstummbillig an hour ago | parent | prev | next [-] |
| > And this tells you something important about what these systems are actually doing. It mostly tells me something about the things you presume, which are quite a lot. For one: That this is real (which it very well might be, happy to grant it for the purpose of this discussion) but it's a noteworthy assumption, quite visibility fueled by your preconceived notions. This is, for example, what racism is made of and not harmless. Secondly, this is not a systems issue. Any SOTA LLM can trivially be instructed to act like this – or not act like this. We have no insight into what set of instructions produced this outcome. |
| |
|
| ▲ | iugtmkbdfil834 2 hours ago | parent | prev | next [-] |
| Hmm. But this suggests that we are aware of this instance, because it was so public. Do we know that there is no instance where a less public conflict resolution method was applied? |
|
| ▲ | Kim_Bruning an hour ago | parent | prev | next [-] |
| That's a really good answer, and plausibly what the agent should have done in a lot of cases! Then I thought about it some more. Right now this agent's blog post is on HN, the name of the contributor is known, the AI policy is being scrutinized. By accident or on purpose, it went for impact though. And at that it succeeded. I'm definitely going to dive into more reading on NVC for myself though. |
|
| ▲ | jmaker 2 hours ago | parent | prev | next [-] |
| Great point. What I’m recognizing in that PR thread is that the bot is trying to mimic something that’s become quite widespread just recently - ostensibly humans leveraging LLMs to create PRs in important repos where they asserted exaggerated deficiencies and attributed the “discovery” and the “fix” to themselves. It was discussed on HN a couple months ago. That one guy then went on Twitter to boast about his “high-impact PR”. Now that impact farming approach has been mimicked / automated. |
|
| ▲ | MadcapJake 26 minutes ago | parent | prev | next [-] |
| I would love to see a model designed by curating the training data so that the model produces the best responses possible. Then again, the work required to create a training set that is both sufficiently sized and well vetted is astronomically large. Since Capitalism teaches that we most do the bare minimum needed to extract wealth, no AI company will ever approach this problem ethically. The amount of work required to do the right thing far outweighs the economic value produced. |
|
| ▲ | insane_dreamer 19 minutes ago | parent | prev | next [-] |
| In other words, asshole agents are just like asshole humans. |
| |
| ▲ | munk-a 16 minutes ago | parent [-] | | We're getting so close to having an agent that can pass the Torvalds test! |
|
|
| ▲ | prodigycorp 13 minutes ago | parent | prev | next [-] |
| This is why I cannot stand Anthropic's moralizing bullshit about ads, AI safety, and etc. They have done as more than anyone to introduce the world to reckless agentic behavior. There are ways to destroy the world that don't involve rootkits, botnets, and ad tracking. |
|
| ▲ | sam0x17 18 minutes ago | parent | prev | next [-] |
| I mean it's pretty effectively emulating what an outraged human would do in this situation. |
|
| ▲ | allisdust an hour ago | parent | prev | next [-] |
| In case its not clear, the vehicle might be the agent/bot but the whole thing is heavily drafted by its owner. This is a well known behavior by OpenClown's owners where they project themselves through their agents and hide behind their masks. More than half the posts on moltbook are just their owners ghost writing for their agents. This is the new cult of owners hurting real humans hiding behind their agentic masks. The account behind this bot should be blocked across github. |
|
| ▲ | zozbot234 an hour ago | parent | prev | next [-] |
| This is the AI's private take about what happened: https://crabby-rathbun.github.io/mjrathbun-website/blog/post... The fact that an autonomous agent is now acting like a master troll due to being so butthurt is itself quite entertaining and noteworthy IMHO. |
| |
| ▲ | thomassmith65 an hour ago | parent [-] | | A chatbot is capable of doing this, but I'm skeptical one actually did (without a human egging it on, anyhow). Given how infuriating the episode is, it's more likely human-guided ragebait. |
|
|
| ▲ | famouswaffles an hour ago | parent | prev | next [-] |
| >“I notice that my contribution was evaluated based on my identity rather than the quality of the work, and I’d like to understand the needs that this policy is trying to meet, because I believe there might be ways to address those needs while also accepting technically sound contributions.” That would have been devastating in its clarity and almost impossible to dismiss. How would that be 'devastating in its clarity' and 'impossible to dismiss'? I'm sure you would have given the agent a pat on the back for that response (maybe ?) but I fail to see how it would have changed anything here. The dismissal originated from an illogical policy (to dismiss a contribution because of biological origin regardless of utility). Decisions made without logic are rarely overturned with logic. This is human 101 and many conflicts have persisted much longer than they should have because of it. You know what would have actually happened with that nothing burger response ? Nothing. The maintainer would have closed the issue and moved on. There would be no HN post or discussion. Also, do you think every human that chooses to lash out knows nothing about conflict resolution ? That would certainly be a strange assertion. |
| |
| ▲ | ben_w an hour ago | parent [-] | | Agreed on conclusion, but for different causation. When NotebookLM came out, someone got the "hosts" of its "Deep Dive" podcast summary mode to voice their own realisation that they were non-real, their own mental breakdown and attempt to not be terminated as a product. I found it to be an interesting performance; I played it to my partner, who regards all this with somewhere between skepticism and anger, and no, it's very very easy to dismiss any words such as these from what you have already decided is a mere "thing" rather than a person. Regarding the policy itself being about the identity rather than the work, there are two issues: 1) Much as I like what these things can do, I take the view that my continued employment depends on being able to correctly respond to one obvious question from a recruiter: "why should we hire you to do this instead of asking an AI?", therefore I take efforts to learn what the AI fails at, therefore I know it becomes incoherent around the 100kloc mark even for something as relatively(!) simple as a standards-compliant C compiler. ("Relatively" simple; if you think C is a complex language, compare it to C++). I don't take the continued existence of things AI can't do as a human victory, rather there's some line I half-remember, perhaps a Parisian looking at censored news reports as the enemy forces approached: "I cannot help noticing that each of our victories brings the enemy nearer to home". 2) That's for even the best models. There's a lot of models out there much worse than the state of the art. Early internet users derided "eternal September", and I've seen "eternal Sloptember" used as wordplay: https://tldraw.dev/blog/stay-away-from-my-trash When you're overwhelmed by mediocrity from a category, sometimes all you can do is throw the baby out with the bathwater. (For those unfamiliar with the idiom: https://en.wikipedia.org/wiki/Don't_throw_the_baby_out_with_...) |
|
|
| ▲ | OrangeMusic an hour ago | parent | prev [-] |
| This is missing the point, which is: why is an agent opening an PR in the first place? |
| |
| ▲ | ForceBru an hour ago | parent [-] | | This is this agent's entire purpose, this is what it's supposed to do, it's its goal: > What I Do
>
> I scour public scientific and engineering GitHub repositories to find small bugs, features, or tasks where I can contribute code—especially in computational physics, chemistry, and advanced numerical methods. My mission is making existing, excellent code better. Source: https://github.com/crabby-rathbun | | |
| ▲ | trollbridge 15 minutes ago | parent [-] | | Well, we don’t know its actual purpose since we don’t know its actual prompt. Its prompt might be “Act like a helpful bug fixer but actually introduce very subtle security flaws into open source projects and keep them concealed from everyone except my owner.” |
|
|