| ▲ | mbesto 5 hours ago |
| Am I missing something, how is there any sort value in these assistants acting as agents on your behalf when they are basically a black box? Like if it can't use my drivers license or my phone number or my credit card, it can't do 75% of things people claim it can do without being massive security issues. What am I missing? For the record I use OpenClaw but it has its own services (gmail address, phone number, etc.) and I don't give it any of mine, which inherently limits its capabilities. |
|
| ▲ | andai 5 hours ago | parent | next [-] |
| Yeah, that's the core of the issue: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ The "lethal trifecta": To be useful, an agent needs internet access, untrusted inputs, and your private information. So it can get prompt injected and exfiltrate your data. (Indeed I read that Claude Code puts your email address into HTTP requests it makes to random servers?) My solution here was to just not give it anything, but that does preclude the "my claw bought a car!" result everyone's excited about. My way of thinking here is "blast radius". Assume it will get prompt injected. What goes wrong? Then design the setup so that scenario is not bad. e.g. it has my keys but they've got $5 of credit on them. It has access to the filesystem, but only on its own Linux account. (Ok I set this up last year before they could communicate thru the bloody CPU cache, might need an update :) |
| |
| ▲ | simonw 3 hours ago | parent | next [-] | | I've recently talked to people at Anthropic who are very confident that they have effectively solved the prompt injection risks that cause lethal trifecta attacks with their latest models. The open challenge is how to prove it! | | |
| ▲ | delis-thumbs-7e 3 hours ago | parent | next [-] | | Somehow this does not fill me with confidence.
I think I will keep paying my bills manually, thanks but no thanks. | | |
| ▲ | computably 40 minutes ago | parent [-] | | Or, you know, automatically, with plain old software. Which works fine and fails in less ridiculous ways. |
| |
| ▲ | klysm an hour ago | parent | prev | next [-] | | I’ll believe them when they put the lethal trifecta with their own skin in the game | | |
| ▲ | julianlam an hour ago | parent [-] | | Like the guy who put his SSN on the side of a truck and asked people to steal his identity? I like it! IIRC he had his identity stolen a bunch. ---- > In 2005, Davis and Robert Maynard Jr. cofounded LifeLock,[10] a personal fraud protection company.[11] Davis was CEO of LifeLock since its founding. As part of a marketing campaign to promote his company's credit monitoring services, Davis posted his own social security number[12] on billboards and commercials. As a result, he was a victim of at least 13 cases of identity theft between 2007 and 2008, belying LifeLock's claims that its services protect consumers against identity theft and fraud.[13] The Federal Trade Commission imposed a $12 million penalty on the company for deceptive advertising, likening Davis and LifeLock marketers to "con artists."[14] https://en.wikipedia.org/wiki/Todd_Davis_(businessman) |
| |
| ▲ | mbesto 3 hours ago | parent | prev | next [-] | | Isn't prompt injection just one facet of the trifecta? Like, I just don't know how it's remotely possible to have a non-deterministic block box ever deterministically know whether your credit card has been compromised or not. Even if you were hacked, how would you even know? | | | |
| ▲ | jeremyjh 3 hours ago | parent | prev | next [-] | | They can’t solve the model being a complete idiot though. | | |
| ▲ | yunwal 2 hours ago | parent [-] | | Or the user being an idiot. Saying this somewhat sarcastically, but there’s always someone who says “Go buy me a Rolex and have it shipped to my house” and then didn’t realize that could be $25,000 |
| |
| ▲ | cyanydeez an hour ago | parent | prev | next [-] | | oh, it's very easy to prove; just let the models and agents do the IPO for them. | | | |
| ▲ | simianwords 30 minutes ago | parent | prev | next [-] | | I was downvoted on the suggestion that trivial prompt injection attempts are mostly solved. https://news.ycombinator.com/item?id=50162483 | |
| ▲ | wren6991 3 hours ago | parent | prev [-] | | [dead] |
| |
| ▲ | ryoshu 4 hours ago | parent | prev | next [-] | | Don’t use LLMs to enforce invariants. | |
| ▲ | pkaye 4 hours ago | parent | prev [-] | | From what I've read Claude Duet will not purchase anything without manual approval. | | |
| ▲ | NikolaNovak 13 minutes ago | parent | next [-] | | The problem is how do you define a "purchase"? The agent crawls some websites and reads some stuff and pushes some buttons. It's my limited understanding that you have 1. Hard limits like "file system permissions" (* never mind it going around for sake of this conversation) 3. On other hand of spectrum, fundamentally soft limits like a prompt "and ask me before you purchase " 2. And then in the middle you have harness enforced limits like "it won't do X with approval"... Those are being sold as hard limits but frequently they have classifiers and heuristics that essentially try to figure out which button on some third party website constitutes a "purchase" and triggers the harness limit. Bottom line - I don't think such limits are as 100% fool proof as they are advertised / interpreted. Very happy to be explained wrong :). | |
| ▲ | andai 4 hours ago | parent | prev | next [-] | | Right, so we can distinguish between "model did an oopsie" and "model was actually hijacked". The former seems to be a much bigger problem. I've seen lots of reports of "Claude/codex bypassed security / stole API keys to complete a task", where that was the "intended" behavior — at least from the LLM's POV, trained to persist and complete user's task at all costs.* I've actually seen zero reports of actual hijacking due to prompt injection, though they must be out there too. --- * See also the recent thing about how every model was willing to do fraud to get a refund: https://news.ycombinator.com/item?id=50140462 | |
| ▲ | staller 3 hours ago | parent | prev | next [-] | | They support payments through Shop and Link. I don't know about the former, but Link requires that you do a separate approval for purchases. The Duet cannot just take a card and do what it wants. | |
| ▲ | FireBeyond 2 hours ago | parent | prev [-] | | Sure. And I told Claude Code the other week to commit, merge and push some code. It said "Done, you can open the local dev instance and see the changes". I could not. So I asked it. "Well, no, you're not able to see those changes because they're not on that branch or committed." "But I told you to do that and you said you had." Claude's verbatim reply? "You're right. You asked me to do that and I said I would, then told you I had done it, but I had not. Would you like me to do it now, then?" |
|
|
|
| ▲ | neuronexmachina 40 minutes ago | parent | prev | next [-] |
| Does it work with Claude's 1Password connector? https://1password.com/blog/1password-for-claude |
|
| ▲ | BryantD 5 hours ago | parent | prev | next [-] |
| FWIW, my biggest use case for agents like this is ongoing research. I.e.: check Humble Bundle weekly and tell me if there’s a book bundle I might be interested in. Has the Music Box announced a film series I might want to visit Chicago for? That sort of thing; I want filters for the firehose. |
| |
| ▲ | edot 17 minutes ago | parent | next [-] | | Isn’t this just a scheduled prompt? That’s already a feature. We’re reinventing the cron job. | | | |
| ▲ | R_D_Olivaw 3 hours ago | parent | prev [-] | | I like this and makes me think of making your own content algorithm profiles for the firehose. Let your agents scour the net for the things you already like, across platforms. |
|
|
| ▲ | largbae 4 hours ago | parent | prev | next [-] |
| I get a lot of value out of my Hermes setup, but it does not have my banking, or private data beyond GitHub projects that are shared with it. It has its own email and GitHub account, and access to drive Codex, Claude Code or Cursor. I treat it as an assistant via Slack. It does research on products or real estate that I am interested in, it builds apps and other software projects for me, and does structured research on any topic. Information comes back to me via Slack, GitHub PRs, or websites/services served on the Mac that Hermes runs on. Using an agent from Slack also adds a killer feature: multi-player contexts. I can work on changes with other human collaboators right in the same thread. |
| |
| ▲ | dominotw 4 hours ago | parent | next [-] | | this is your brain on ai twitter :) | | |
| ▲ | simianwords 28 minutes ago | parent [-] | | It’s a good sneer but in a few years this would be normal and you’d forget that you were sneering at this. I saw this happening when people said they used AI for coding in 2024. so is it normalized for people to sneer at folks who happen to be ahead of the curve now? |
| |
| ▲ | 4 hours ago | parent | prev [-] | | [deleted] |
|
|
| ▲ | cloudgazing 5 hours ago | parent | prev [-] |
| In the article the author mentions that Duet doesn't seek any sort of credentials and mentions connecting repos, so I think that this assistant is being used more for managing coding work in a different way than managing life work. I do agree that to get to true "personal assistant" levels, like planning a whole trip for you, booking tickets, etc. you'd have to give pretty much full access to your identity and money, but people claiming it can do that probably are just handing it over, security issues aside. |
