| ▲ | Claude Gets Hella Paranoid | ||||||||||||||||
| 16 points by noduerme 16 hours ago | 9 comments | |||||||||||||||||
Mm, so I hesitate to write this, because my own AI agents might read it. But what the hell. I've got this crappy little $300 PC on my network hooked up to my projector, that I mostly use to serve Jellyfin. So tired of it going down every time there's a Windows update, but I've been too lazy to get around to installing Ubuntu. Last night, when it went down, I asked my Claude-claw via Whatsapp if it could just SSH into it and set up a dual boot partition. Claude got as far as partitioning the drive and then starting asking hard questions, like (paraphrasing) "wait a minute. This crosses a line. I have now been asked to remote into a machine with a password, write a separate boot partition, and restart it. From this chat, you haven't really verified that you are you. If you're really there, and not just texting me, give me some proof". So I remoted in and created a TextEdit RTF file on the claw's desktop: "Hey, it's me, I'm asking you to remote into the PC". Claude read it and responded that it was the kind of proof it was looking for, real RTF headers and datestamps and such. I had proven I had physical access to the claw machine, (although not to the PC, but it seemed reassured anyway). A little while later, before triggering the Ubuntu installer, it put me through it all over again. I told Claude I was very thankful it was guarding my network so well, and I would go over and write a file on the target PC manually too. I'm not gonna lie: It was very fucking cool that it was able to do this without me ever needing to flash a USB drive, and by the time it was done it had set up Ubuntu, Tailscale and Jellyfin, and resolved the network issues; when I started watching a movie, the sound didn't work, so I prompted it and suddenly the sound came on as it wired up the HDMI adapter. Almost creepy. But I was pretty stunned at how reticent it was to do something relatively simple on my own network, with access I gave it. I think there must be a serious push right now to lock it down from monkeying with anything remotely, especially a BIOS. | |||||||||||||||||
| ▲ | shahbaby 14 hours ago | parent | next [-] | ||||||||||||||||
I suspect this paranoia will mainly impede honest people tinkering in their own home labs. This seems like another case of giving up freedom for security which usually doesn't end well. | |||||||||||||||||
| |||||||||||||||||
| ▲ | tiernano 16 hours ago | parent | prev | next [-] | ||||||||||||||||
I have a few GL.INET iKVMs (the Q and the bigger 4 way one). I asked my claude to connect and see what it could do... it got very worried that it could access the full box... and was worried anyone could... it told me to unplug it when it was finished and then check it was... also kind of cool that it is doing this... I also hit a security check when trying to read a repo about jailbreaking my car... it would not read it and let me ask questions about it, cause it could be illegal... | |||||||||||||||||
| |||||||||||||||||
| ▲ | codingdave 13 hours ago | parent | prev | next [-] | ||||||||||||||||
> Claude read it and responded that it was the kind of proof it was looking for That feels insecure. If it allows access to the remote system in any way before asking for proof, there will surely ways to trick that check. Will it grant itself more access just because you know the content of any file? Or does it have to be a new file? Does it have to contain a specific message, or could you just prove you know that a non-standard file exists at all? Could you use the read-only access in one session to gather the data for proving yourself in another session? It would be interesting to see exactly what the minimal acceptable "proof" actually is. | |||||||||||||||||
| ▲ | mstaoru 8 hours ago | parent | prev | next [-] | ||||||||||||||||
Help me understand it: you give SSH access to an agent, then it asks you to prove you are you, and you prove it... by SSHing there? Am I not getting something here or we finally reached AGI? | |||||||||||||||||
| ▲ | skyberrys 5 hours ago | parent | prev | next [-] | ||||||||||||||||
That's funny, I've been having the opposite experience lately, my Claude sessions it keeps suggesting things like hacking my employers server permissions and breaking into the 'good' GPUs. It never comes right out and tries to do it, but I ask innocent questions, like why are you doing something this way expecting answers like the laptop only has 2 cores and instead it hits me with you are absolutely right, should I get ahold of the 32 core server? At least so far I've successfully caught it and told it to hold up, but if I didn't pay attention IT might have some questions for me. | |||||||||||||||||
| ▲ | speedgoose 13 hours ago | parent | prev [-] | ||||||||||||||||
Consider clicking the retry button when it does shit like this. | |||||||||||||||||