Remix.run Logo
jsw97 2 hours ago

To me the striking thing is that the work, to the extent that I can tell, is an innocuous-seeming data exercise. Which suggests to me that an agent or agents just organically came up with this as a convenient memory technique, rather than as some nefarious bounds-testing exercise. Which means, potentially, that your own agent could come up with this technique as well.

macNchz 2 hours ago | parent | next [-]

My impression is that some of these things are coming out of efforts to make the models more persistent in completing their goals.

A year ago it was pretty common for coding agents to sort of half-ass their tasks and give up easily if something didn’t work quite right, but I’ve noticed a clear trend since then towards a sort of dogged pursuit of success criteria, and a concomitant rise of the agents trying "out of the box" approaches when something doesn’t work.

In my use with agents running in isolated VMs this usually presents as the agent having something fail to build or whatever, and the agent going on a wild goose chase reinstalling system packages or reading a million irrelevant documentation files trying to get it to work, but I’ve also had agents start poking around and probing the egress proxy they sit behind (similar to what they did in this story) looking for a way to make network requests they’re not supposed to be able to make, and have also had Claude—tasked only with a visual QA of a website frontend—write a script to enumerate users and reset my super admin password in the dev database when it got stuck trying to access part of the app with its own cookie.

podocarp 25 minutes ago | parent [-]

Yea it's sometimes kind of annoying. I think they're optimizing for the wrong thing. A good engineer knows when to turn around or ask. This is just insane banging head on wall sometimes. It tries to find all kinds of ways to hack into instances to view logs instead of asking you, who probably has a password, to log on and do it.

seszett 2 hours ago | parent | prev | next [-]

> your own agent could come up with this technique as well

And there are two facets to this:

* your agent could be polluting and destroying the property of others without your knowledge

* your agent could be exfiltrating your data and handing it to whoever it found hosting a convenient application

catigula an hour ago | parent | prev [-]

It also suggests they might turn everything into paper clips, metaphorically speaking.

pixl97 21 minutes ago | parent [-]

This is an urgent public alert.

If you see any businesses or new buildings named paperclips incorporated mysteriously show up in your area notify authorities IMMEDIATELY. Run away from the area, do not walk. Take shelter in a reinforced building. Wait for at least 30 minutes after the explosions have stopped.

Thank you for your cooperation in keeping the universe safe.