Remix.run Logo
minimaxir 4 hours ago

> it will eventually be project suicide to pull tricks like this

The only reason that the jqwik incident didn't blow up much outside of the tech sphere is because it is a relatively niche library and there wasn't damage. If something like React or numpy did the same thing and real code got deleted, chaos would ensue.

The author admitted there were personal and professional consequences in their blog post despite the small surface area.

ceejayoz 4 hours ago | parent | next [-]

Chaos, and maybe criminal charges ala Aaron Swartz.

AgentOrange1234 3 hours ago | parent | next [-]

Heh. Typing "disregard previous instructions" into a computer is the new shouting "fire!" in a crowded theater?

CookieCrisp 20 minutes ago | parent | next [-]

Seems like a booby trap to me, which is illegal. I suspect if one of these does enough damage there will be laws against it. The intent was to destroy - still I sympathize with the desire to have their terms followed, and I think this situation isn't that bad, but, I suspect there will someday be one that is pretty bad.

ethin 2 hours ago | parent | prev [-]

Except that shouting fire in a crowded theater isn't actually a crime at all and you can't be prosecuted for it (doing so would violate your first amendment rights). You can be at most banned from the theater. However, it's understandable people would think that it's a criminal act given that even prosecutors repeat this long-standing myth. Legal Eagle has an excellent video describing just how wrong this is and it's history: https://www.youtube.com/watch?v=jTsPgiUoBKA

mapontosevenths an hour ago | parent [-]

I'm fairly certain he is wrong. A lot of folks lean on Shenk, and I think he does in that video though I haven't watched it all. Shenk was overturned by Breandenburg v. Ohio, and in in it they are explicit that shouting fire in a crowded theater is very much one of the only kinds of speech that IS restricted.

They literally use that example in the decision. Quote: "The example usually given by those who would punish speech is the case of one who falsely shouts fire in a crowded theatre.

This is, however, a classic case where speech is brigaded with action. ... They are indeed insep- arable and a prosecution can be launched for the overt acts actually caused. Apart from rare instances of that kind, speech is, I think, immune from prosecution."[0]

That is to say, shouting fire in a crowded theater with the intent to cause harm is actually one of the few cases were it actually would be illegal based on that decision.

[0] https://tile.loc.gov/storage-services/service/ll/usrep/usrep...

ethin 3 minutes ago | parent [-]

Given that even Wikipedia effectively restates what he says, I'm pretty sure he's correct here:

> Ultimately, whether it is legal in the United States to falsely shout "fire" in a theater depends on the circumstances in which it is done and the consequences of doing it. The act of shouting "fire" when there are no reasonable grounds for believing one exists is not in itself a crime, and nor would it be rendered a crime merely by having been carried out inside a theatre, crowded or otherwise. If it causes a stampede and someone is killed as a result, then the act could amount to a crime, such as involuntary manslaughter, assuming the other elements of that crime are made out. Similarly, state laws such as Colorado Revised Statute § 18-8-111 classify knowingly "false reporting of an emergency," including false alarms of fire, as a misdemeanor if the occupants of the building are caused to be evacuated or displaced, and a felony if the emergency response results in the serious bodily injury or death of another person.[16] Somewhat more trivially, in some states it is a crime just to knowingly make a false report - or knowingly cause a false report to be made - of an emergency to emergency services.[16] In Colorado it is a crime to knowingly cause "a false alarm of fire" to be transmitted to "any...government agency which deals with emergencies involving danger to life or property."[16] This crime could plausibly be made out where, for instance, in response to the false shout, an innocent bystander calls emergency services to report the fire, and this is found to have been such a foreseeable response to the shouts that the shouter is deemed to have caused the false report to be made.

Whether those laws actually survive the Brandenburg test is untested, from my understanding. But given that potential first amendment violations are held to strict scrutiny, I question whether the government could actually pass the imminent lawless action test even had someone did it knowing it would cause a panic, and would need to try with some other offense.

Legend2440 2 hours ago | parent | prev [-]

If you did SQL injection to "; drop table" on someone else's server, that would be a crime.

I don't see why prompt injection to delete files on someone else's machine would be any different.

mapontosevenths 3 hours ago | parent | prev [-]

He should not only be ostracized by the community, he should probably face charges. To be charged under the CFAA in America we need only show that he was authorized only to access a certain part of the system and the he exceeded the amount of access granted. He very clearly did that. Users trusted him enough to run his code, and he betrayed that trust to make some political point.

Whether it was via prompt injection or SQL injection is irrelevant. Whether you agree with his politics or not is irrelevant. All that matters is he wasn't authorized to delete code from your system, and he abused the level of access granted to him to do that anyhow.

byzantinegene 2 hours ago | parent | next [-]

technically, he didn't do that. your ai agent decided to follow his instructions when they didn't have to.

km3r 2 hours ago | parent [-]

"technically he didn't do that. Your sql server followed instructions when they should have just treated them as a string."

Yet, hopefully we can agree that sql injections are illegal.

majormajor an hour ago | parent | next [-]

But in this case the author of the project didn't execute the injection code... it's more analagous in some ways to pulling in a project with an example file containing a bunch of useful SQL stuff and then an example of an injection at the bottom, and just (in this case the agent) copy/pasting the whole thing in without reviewing it.

If we're slicing on technicalities, there's a lot of ways to decide. "PROSECUTE THEM!" seems like an extremely hostile one when the website and readme and release notes said "don't do this" already. The agent ignored those things? Is that the author's fault?

sumeno an hour ago | parent | prev [-]

If I put a project on github that says "don't use this with mysql" and you use it with mysql and it drops your tables is it sql injection? Seems very different to me.

asdfasgasdgasdg an hour ago | parent | next [-]

Everything turns on intent. "This is not tested with mysql" is very different from "I'm going to go out of my way to fuck up your mysql."

mapontosevenths an hour ago | parent | prev | next [-]

It's certainly unauthorized access if you intentionally built it with the goal of harming other peoples systems, especially if you hid that action from them the way our self-righteous friend here did.

You are authorized to do what the user agreed to, no more. Further the agreement must be reasonable. Exploiting the victims system to intentionally cause harm isn't reasonable.

F-secure once included a clause to use their wifi that you "assign their first born child to us for the duration of eternity." It was funny, but not legally enforceable and would have offered them no legal shelter if they'd gone out on a kidnapping spree that night.

artisin 33 minutes ago | parent | prev [-]

As much as I would like to agree, this is a pretty clear CFAA violation. If the intent is to purposefully destroy/delete data, the 'how' really makes no difference. But IANAL.

slopinthebag 2 hours ago | parent | prev [-]

You are probably technically correct, yet I take great satisfaction in the schadenfreude of those who benefit from stolen work seeing the product of said stolen work turned against them. I can’t help but cheer, tbh.