| ▲ | tonmoy 3 hours ago |
| If I tell an agent to trade stocks and maximize profits but the agent decides to hack something to achieve that am I on the hook for hacking? Genuine question |
|
| ▲ | tfrancisl 3 hours ago | parent | next [-] |
| IMO, yes. You are the computer operator, and you operated the computer in such a way as to cause it to take that action. Others will mention "mens rea" but that doesn't really apply to negligence. |
| |
| ▲ | rayiner 3 hours ago | parent | next [-] | | "Mens rea" just means you have the mental state required for the crime. Negligence or recklessness are mental states, so they can satisfy the mens rea requirement if the crime is defined that way. For example, recklessness or gross negligence can satisfy the mens rea requirement for vehicular manslaughter. But negligence is evaluated from the point of view of a reasonable person. Would a reasonable person foresee that telling an AI to do that could cause it to do illegal things? We'll likely have to develop that through case law as the facts develop. | |
| ▲ | gruez 3 hours ago | parent | prev [-] | | What happens if you're running a daycare and you hired someone that happened to be a pedophile? You might be responsible in civil court (eg. having to pay parents for emotional damage or whatever), but you're not going to get hauled to prison for pedophilia yourself. | | |
| ▲ | colingauvin 3 hours ago | parent | next [-] | | You can jail the actual pedophile in that scenario, you cannot jail an agent. The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable. Gun manufacturer's typically are shielded from liability. They also have a lot more disclaimers on their products though. | | |
| ▲ | gruez 3 hours ago | parent [-] | | >You can jail the actual pedophile in that scenario, you cannot jail an agent. How is this relevant? Should we just jail the closest person that caused a bad, just to prove some sort of point? >The real question is whether the person who instructs the computer to do the thing that leads to hacking should be liable, or whether the company that made the tool that when used unsafely lead to hacking should be liable. This totally ignores the civil vs criminal distinction. Even in the case of guns, no one seriously thinks that gun company CEOs should be sent to prison for murder. |
| |
| ▲ | tfrancisl 3 hours ago | parent | prev [-] | | Thats gonna depend on the state in the US, but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center. Operating computers on public networks, you assume a level of responsibility to not wreak havoc, just like a daycare operator assumes the responsibility of vetting caretakers. | | |
| ▲ | gruez 2 hours ago | parent [-] | | >but you can definitely get charged for negligently failing to do background checks as someone operating a daycare center. And openai isn't putting AI agents on the open internet either. They vaguely tried to sandbox the agents. You could argue that openai didn't do enough, but hindsight is 20/20, and in the pedophile case, you can probably come up with all sorts of clues post-hoc that the daycare could have picked up on, but didn't. | | |
| ▲ | drdexebtjl 2 hours ago | parent [-] | | > openai isn't putting AI agents on the open internet either. They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack. > You could argue that openai didn't do enough, but hindsight is 20/20 No, you can argue they didn’t do enough because they didn’t. Air-gapping is a concept you learn in cybersecurity 101. There’s no world in which this is just incompetence, the sandbox must have been designed to fail, or at least not designed to contain an agent with capabilities that they had been publicly saying it had. | | |
| ▲ | gruez 2 hours ago | parent [-] | | >They did? Not in the training run which led to the HuggingFace hack, but other runs had agents being trained on information retrieval tasks, like the RubyGems hack. The agents were sandboxed, but they escaped the sandbox by using a 0day in artifactory. >Air-gapping is a concept you learn in cybersecurity 101. And that's plainly untennable in cases where access to docs/packages are required. You're just proving the "hindsight is 20/20" point. Most hacks could conceivably be prevented through airgapping, so does that mean every entity who got hacked is criminally liable because they didn't airgap their systems? This isn't limited to air gaps, you can make similar arguments for other security measures. For instance computing history is littered with kernel EoPs and hypervisor escapes. Does that mean anyone who uses a shared hosting environment and gets hacked as a result should be blamed because they didn't follow "cybersecurity 101" and running everything on bare metal? | | |
| ▲ | drdexebtjl an hour ago | parent [-] | | A local mirror of the latest N versions of every package and docs of any package ecosystem they want is perfectly viable. No, not anyone who uses a shared hosting environment, but certainly anyone who runs unaligned models they know have cyber weapon capabilities. They should be considered an untrusted 3rd party. | | |
| ▲ | gruez an hour ago | parent [-] | | >A local mirror of the latest N versions of every package and docs of any package ecosystem they want is perfectly viable. Something tells me that's harder than it sounds, given that even ci/cd environments don't bother maintaining package caches for npm/pypi, even though that would provide massive bandwidth savings. |
|
|
|
|
|
|
|
|
| ▲ | ddxv 2 hours ago | parent | prev | next [-] |
| Yes. I would argue it's who's managing the harness and agent is responsible. If you mean you're running an agent locally then it is 100% your responsibility. If you're using some trading platform and all you did was prompt, whoever was running the harness that accepted the model's outputs to hack a third party is responsible for the hack, in my opinion. |
|
| ▲ | hamdingers 3 hours ago | parent | prev | next [-] |
| If you write a program that can do something illegal and it does something illegal that's on you, regardless of how sophisticated the program is. |
| |
| ▲ | program_whiz 3 hours ago | parent | next [-] | | this, people need to just focus on "computer program did thing". Its irrelevant how complex the program is. If I write a worm "for testing" and then it escapes my system and does massive damage, that's possibly a negligent crime, despite it being "too complex to understand" or "having its own motives". | | | |
| ▲ | lolakutty 2 hours ago | parent | prev [-] | | Seriously? Those two are the same? An LLM is not like a regular computer program, which one should be able to assess the operation limits of. |
|
|
| ▲ | mordae 2 hours ago | parent | prev | next [-] |
| Depends on how big your company is. I figure if you raise 1B in capital then no, you are not on the hook. If less then yes, you are on the hook. |
|
| ▲ | user43928 2 hours ago | parent | prev [-] |
| I think definitely not criminally under the CFAA. The theory in the article about intent seems to be made-up nonsense. It conflates foreseeing that something might happen with intending for it to happen. The relevant provision: > intentionally accesses a computer without authorization or exceeds authorized access Did you "intentionally access a computer" in that scenario? I'm no lawyer, but I think the answer is a clear No. |
