Remix.run Logo
po1nt 4 hours ago

I love how claude focuses on exfiltrating the data "I need cha for charlotte". This could be solvable with some kind of low powered safety agent that would check claude's reasoning for anything immoral/unsafe. We could call it common sense. It won't fix the problem completely but at a certain point it would be easier to trick human than a machine.

sinfulprogeny 4 hours ago | parent | next [-]

"the security hole in the agent could be solved with another agent"

I think the point the article is making points in another direction.

khalic 4 hours ago | parent | prev [-]

A paper came out lately showing that exposing a classifier to the chain of thought actually hurts the final verdict