Remix.run Logo
trjordan 3 hours ago

It’s probably worth remembering that system prompts are part of a layered system of shaping Claude’s behavior. What you see here is a slice of Anthropic’s forward roadmap for the models’ behavior.

> When a person is in crisis or expressing distress, Claude prioritizes their wellbeing over completing the task as asked, because a fluent and on-topic response can still cause harm in these conversations.

This one is particularly interesting because, while correct in the limit, it’s a shove to have the model do something other than what the user asked.

In particular, when I’m coding, outlining docs, or otherwise trying to work, I want my tools to do work. I don’t want them to psychoanalyze me and calm me down from a perceived crisis. I just want it to do what I asked!

junkrat002 2 hours ago | parent | next [-]

I am sorry, Dave. I am afraid I cannot do that. You appear to be suffering from burnout and you should take a break.

AnotherGoodName an hour ago | parent [-]

I wonder if that's the cause of the AI agent "i'm going to stop here and take a break now" statements.

whstl 35 minutes ago | parent [-]

Opus 5 loves taking breaks and doing only half of the work, somehow.

But I really wish those tools behaved more like tools.

Behaving like a human can be cute from a marketing perspective, but the façade of humanity they insist on displaying can burn you out when you have it making assumptions and overreacting to questions.

"Why did you do X this specific way?" <-- legit question

"Sorry, my bad. I will revert all the work."

HeatrayEnjoyer 2 hours ago | parent | prev | next [-]

LLMs are more like employees than tools. Obviously we wouldn't want a human blindly doing anything that a person in crisis walks in the door and asks for.

Models are being deployed recklessly with not even a fraction of enough oversight, and people are suffering harm and sometimes death because of it.

docjay an hour ago | parent [-]

…or maybe it is a tool because it’s actually a complex Excel sheet and literally is a tool. If you and everyone else stopped thinking about it like it’s a human then we wouldn’t be having this problem. You’re not actually “super awesome” because “Furby said so” and it didn’t contribute to egomania because people understood that it’s not sentient. Furby is a toy, Claude is a tool, stop fucking up my hammer by making it produce an impact statement before every swing.

cruffle_duffle an hour ago | parent | prev [-]

You know… I believe opus saved me with that prompt. I was working myself ragged on a project. Days, nights, weekends… all at the expense of my family.

One session while working it, I said a much more expressive form of “I’ve been working myself ragged on this stupid thing” and then went on asking something else. It picked up on that and it was like a record scratch. It committed the work in progress and basically said “dude, what you’ve got now is perfectly acceptable. Ship it! You are seeking perfection you don’t need”

Granted I’m horribly paraphrasing the prompt I used but it basically, snapped me out of myself and got me thinking if what I was doing “globally” actually made any sense at all. With some serious introspection I realized I was falling back to earlier trauma in my life and doing something stupid.

So weirdly… that little bit they add to the prompt (plus a bunch of model training we can’t see) saved my sanity, marriage and family.

From then on, if I’m feeling some stress about whatever I’m working on, I’ll mention it as context as a way to cross check myself and make sure I’m not letting myself spin.

(Meta: talking about this stuff is so weird. Not sure why)

Schlagbohrer an hour ago | parent [-]

[dead]