Remix.run Logo
▲ createful 2 hours ago

All these "safety" bills are to stop open source models by painting them as "dangerous". Regulatory capture.

Companies are responsible for their agents; you don't blame the car for running over the person.

▲lukifer 7 minutes ago | parent | next [-]

If that were the intent, from the OAI/Anthropic perspective, you'd think we would have seen at least one of the scary incidents (either real or simulated) being performed by Qwen or DeepSeek, rather than from their own models.

If anything, their public messaging has been to entirely pretend that open models don't exist, that they're mere toys or curiosities, that only the big dogs with eleven zeros of capex have capabilities which could be considered dangerous.

You could certainly argue that the various "regulate AIs like airplanes" bills might shut down some open model development (domestically; they're certainly not calling for international weapons treaties). But the big firms haven't exactly been jumping backflips in support of Sanders' Ban Artificial Superintelligence Act.

▲Waterluvian 2 hours ago | parent | prev | next [-]

Every play made by these companies aligns with what you’d do if you have a $1T valuation to protect against an industry that seems increasingly commoditizable.

I’m still… hopeful… that “good enough” models will be abundantly available to be run anywhere as to limit the low water mark for market capture by a few big ones.

▲jayd16 23 minutes ago | parent [-]

Can they just IPO and dump already so we can all move on...

The biggest frustration to me is that investment dollars are all tied up in the shovel company and no one is funding the digging.

▲hdgvhicv 2 hours ago | parent | prev | next [-]

You do regulate cars so they don’t have spinning blades on the front or flamethrowers from the sides.

▲peri-cl 23 minutes ago | parent | next [-]

There are zero laws against the design, manufacture, sale, or possession of cars with spinning blades and/or flamethrowers.

Such laws attach to the human operators of such cars, if they recklessly endanger others by driving them on shared, public roads.

But they're perfectly legal to own (and operate off-road). And, there's no technical barrier stopping people from illegally driving them on public roads: just fear of laws.

(There's entire subcultures of people who modify cars to do insanely dangerous things, on private race tracks. Perfectly legal).

If we were to apply Amodei's safety standard to cars—do they have the theoretical risk of killing over 50 people—well, cars empirically kill tens of thousands of people in the US annually, so we'd have to categorically ban them. That standard is bad-faith and absurd.

▲jayd16 16 minutes ago | parent [-]

So we just make the regulation such that it kicks in on machines connected to the public web?

▲peri-cl 13 minutes ago | parent [-]

No. Half the point of LLM's is having them research topics on the internet as your agent. It's totally harmless in the usual (tool-based search) workflow.

(Would we also need a real ID to operate curl(1)?)

Regulations should attach only at the point humans willfully or recklessly cause their agents to commit crimes.

▲flavionm an hour ago | parent | prev | next [-]

Yet we don't regulate computers so that they're rendered incapable of running software that can be used to hack into systems. Or said software, by the way. All we do is punish the people who use these tools to cause harm to others.

AI is much more comparable to that than to a car.

▲jayd16 20 minutes ago | parent [-]

Is that right? You can't even sell software built to break copy protection. Can you legally sell software made to harm others?

You can still run someone over with a car. The analogy doesn't require absolute lack of capability to do harm.

▲createful 2 hours ago | parent | prev | next [-]

That assumes AI is designed to be malicious, "the spinning blades". AI is not; the public version given to you is the sanded down one, in the same way that cars don't have spinning blades but can still be dangerous when misused.

If a person closes his eyes and hits the gas pedal on the car and runs over someone, the person is to blame, not the car. AI companies do the same. Your eyes may be closed but you're still operating/hosting the car/AI.

▲LocalH 2 hours ago | parent [-]

There exist uncensored, unregulated models. Many of them are available for local use, sidestepping any third-party checks that could be present.

▲createful 2 hours ago | parent [-]

There being no third party checks is the point, and it's not bad.

Keep in mind that those open source/weight models are not as powerful as whatever closed source Claude/GPT version exists behind the scenes that hack companies. As per the Hugging Face hack - open weight GLM saved the day as closed source ChatGPT refused to help Hugging Face fix their systems. An example of my point that "sidestepping any third-party checks that could be present" IS good.

Part of point of open source/weight AI is that it allows what those model providers can't: uncensored models for whatever reason you may need it. You are responsible for being legal with it.

▲pembrook an hour ago | parent | prev [-]

If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.

Until then, I propose we use the 'unplug the server' rule. Can [insert bad thing] be prevented by unplugging the server? Then these things are fully in our control and no, we don't need to re-build hundreds of years of legal precedent from the ground up.

Liability in the event of harm is already something we have a pretty good handle on from living through the industrial revolution 150 years ago, a time when much more activity that companies did existed in the real world with real harms, as opposed to digital companies doing imagined "mental" harm to soft desk chair people who work at email jobs.

▲pessimizer 41 minutes ago | parent [-]

> If your AI model develops the capability to manifest itself in the physical world as spinning blades and flamethrowers I'm fully open to re-evaluating the need for regulation.

No, only metaphorical AI spinning blades and flamethrowers should be regulated. Actual AIs with spinning blades and flamethrowers will be subsidized and unleashed in your town.

▲dfxm12 2 hours ago | parent | prev | next [-]

I don't know if that analogy holds. You don't normally blame Toyota for car crashes, but the driver. If you must use a car analogy, consider the 2010 Toyota recalls rather than just normal car crashes.

That is to say, anthropic should be considered responsible, not (or not just) the users.

▲Geezus_42 2 hours ago | parent [-]

Who takes the blame when someone ask the blackbox to complete a task and it then breaks the law in order to complete that task without the user having any knowledge of what it was doing or intent for the machine to accomplish the task in an illegal manner?

▲flavionm an hour ago | parent | next [-]

When it's an open weight, locally run blackbox, you blame the user, since in that case it's very much possible for the user to restrict what the blackbox can actually do. They have full control over it, so it follows they have full responsibility.

None of that requires trying to impose limits on open models, unlike what Anthropic is aiming for.

▲Geezus_42 an hour ago | parent [-]

We have seen repeatedly that users are not able to meaningfully restrict the blackbox. Besides the fact that laws are already codified restrictions.

I never said we should impose limits on open models.

▲flavionm 42 minutes ago | parent | next [-]

Aren't able to or have just failed to do so? Regardless, that doesn't exempt them from the responsibility. It's ultimately software they're running on their own machine, there is no one else to responsibilize for it.

I wasn't trying to imply you were advocating to restrict them, just that that is what Anthropic is doing. Legislating away all competition is worth whatever small concessions they would need to make.

▲Geezus_42 36 minutes ago | parent [-]

Sure, I agree. I just think there comes a point when the industry has a whole should be held accountable, and in the past we have done everything possible to shift that responsibility onto consumers how generally have very little choice in the tech they use.

Well, you shoehorned it into a completely unrelated conversation, so forgive me.

▲pessimizer 37 minutes ago | parent | prev [-]

> We have seen repeatedly that users are not able to meaningfully restrict the blackbox.

We have seen these companies repeatedly ship coding harnesses without any attempt at real sandboxing, and encourage people to let them run free. Seems that if they've somehow solved coding, they'd be able to solve making it easy for people to sandbox agents, and to be fully aware when they've enabled some LLM the ability to hack companies. Why don't they put 10,000 agents on that instead of ripping off Navier-Stokes?

▲Geezus_42 33 minutes ago | parent [-]

Because its not profitable and that would shift responsibility onto the company that claimed it was safe, rather than the consumer who doesn't know better because the media has sold LLMs as a panacea.

▲dfxm12 2 hours ago | parent | prev [-]

The situation as provided is missing too many details for anyone to make a meaningful judgement call, but remember, more than one party can be found liable/guilty (maybe of different things) in a situation. Overall, the law usually protects people who are acting reasonably, but does not protect people who act negligently.

I'm not a lawyer, but I imagine there's not a lot of precedent or law around this, so it'll be up to the court to decide on the finer details until we do have meaningful laws/regs. Anthropic will probably have better lawyers than the user.

This is why it's important to get ahead of issues like this, and anthropic should not be involved in discussions around regulating itself...

▲Geezus_42 an hour ago | parent [-]

Unfortunately, I agree.

▲parsimo2010 2 hours ago | parent | prev | next [-]

A sizable part of the country thinks it’s okay to regulate firearms. Under your logic, people should be responsible for who they shoot, don’t blame the gun when a human pulled the trigger. While gun control advocates have that pesky second amendment hampering their efforts, AI/LLM systems have no such protection.

Regulatory capture is certainly something to be wary of, and obviously Anthropic is only going to be in favor of regulation that helps them in some way. But some form of regulation is coming. There is nothing that can stop it, unless you somehow make more money than these guys do and fund a massive counter-lobby. Spoiler alert: the only guys that have more money than Anthropic and OpenAI have all invested in them.

▲flavionm an hour ago | parent [-]

A much less sizable part thinks it's okay to limit local computing. There isn't a widespread push to make it so local hardware is limited in ways that they're not allowed to, for instance, run programs that can hack systems. These programs are very deliberately allowed, and it's people misusing them that are held responsible.

Regardless of anyone's instance on gun control, it's probably wiser to compare AIs with something that's a little more similar to it, and try to reach a conclusion from there.

▲johnfn 2 hours ago | parent | prev [-]

How do you plan to solve z.ai distilling Anthoropic’s model into a cyber-capable open source GLM-5.5?

▲Geezus_42 2 hours ago | parent | next [-]

We don't. The only way to stop that is to not develop the model to begin with. People will tinker and modify anything you create.

▲johnfn an hour ago | parent [-]

Ok, so the plan is to stop z.ai, DeepSeek, and all other non-American labs from developing models entirely, full stop? How do you plan to enact this large scale system of international coordination? Asking politely?

▲Geezus_42 an hour ago | parent [-]

No, my point obviously flew right over your head. Maybe work on reading comprehension.

My point was that it can't be stopped unless we kill the industry entirely. Once the cat is out of the bag, you can no longer control it.

▲johnfn an hour ago | parent [-]

I responded directly to your point. How do we kill the industry without international cooperation, and what is your plan there?

▲Geezus_42 41 minutes ago | parent [-]

I don't get paid enough for that.

▲otabdeveloper4 41 minutes ago | parent | prev [-]

> cyber-capable

Ou-la-la. Spicy. But actually we already have those kinds of AI.