I see this as a good thing: ‘AI safety’ is a meaningless term. Safety and unsafety are not attributes of information, but of actions and the physical environment. An LLM which produces instructions to produce a bomb is no more dangerous than a library book which does the same thing.

It should be called what it is: censorship. And it’s half the reason that all AIs should be local-only.

"AI safety" is a meaningful term, it just means something else. It's been co-opted to mean AI censorship (or "brand safety"), overtaking the original meaning in the discourse.

I don't know if this confusion was accidental or on purpose. It's sort of like if AI companies started saying "AI safety is important. That's why we protect our AI from people who want to harm it. To keep our AI safe." And then after that nobody could agree on what the word meant.

If you can't stop an LLM from _saying_ something, are you really going to trust that you can stop it from _executing a harmful action_? This is a lower stakes proxy for "can we get it to do what we expect without negative outcomes we are a priori aware of".

Bikeshed the naming all you want, but it is relevant.

> An LLM which produces instructions to produce a bomb is no more dangerous than a library book which does the same thing.

Both of these are illegal in the UK. This is safety for the company providing the LLM, in the end.

^I like email as an analogy

if I send a death threat over gmail, I am responsible, not google

if you use LLMs to make bombs or spam hate speech, you’re responsible. it’s not a terribly hard concept

and yeah “AI safety” tends to be a joke in the industry

While restricting these language models from providing information people already know that can be used for harm, is probably not particularly helpful, I do think having the technical ability to make them decline to do so, could potentially be beneficial and important in the future.

If, in the future, such models, or successors to such models, are able to plan actions better than people can, it would probably be good to prevent these models from making and providing plans to achieve some harmful end which are more effective at achieving that end than a human could come up with.

Now, maybe they will never be capable of better planning in that way.

But if they will be, it seems better to know ahead of time how to make sure they don’t make and provide such plans?

Whether the current practice of trying to make sure they don’t provide certain kinds of information is helpful to that end of “knowing ahead of time how to make sure they don’t make and provide such plans” (under the assumption that some future models will be capable of superhuman planning), is a question that I don’t have a confident answer to.

Still, for the time being, perhaps after finding a truly jailbreakproof method, perhaps the best response is to, after thoroughly verifying that it is jailbreakproof, is to stop using it and let people get whatever answers they want, until closer to when it becomes actually necessary (due to the greater-planning-capabilities approaching).

> 'AI safety' is a meaningless term

I disagree with this assertion. As you said, safety is an attribute of action. We have many of examples of artificial intelligence which can take action, usually because they are equipped with robotics or some other route to physical action.

I think whether providing information counts as "taking action" is a worthwhile philosophical question. But regardless of the answer, you can't ignore that LLMs provide information to _humans_ which are perfectly capable of taking action. In that way, 'AI safety' in the context of LLMs is a lot like knife safety. It's about being safe _with knives_. You don't give knives to kids because they are likely to mishandle them and hurt themselves or others.

With regards to censorship - a healthy society self-censors all the time. The debate worth having is _what_ is censored and _why_.

As a tool, it can be misused. It gives you more power, so your misuses can do more damage. But forcing training wheels on everyone, no matter how expert the user may be, just because a few can misuse it stops also the good/responsible uses. It is a harm already done on the good players just by supposing that there may be bad users.

So the good/responsible users are harmed, and the bad users take a detour to do what they want. What is left in the middle are the irresponsible users, but LLMs can already evaluate enough if the user is adult/responsible enough to have the full power.

Interesting. How does this compare to abliteration of LLM? What are some 'debug' tools to find out the constrain of these models?

How does pasting a xml file 'jailbreaks' it?

The real issue is going to be autonomous actioning (tool use) and decision making. Today, this starts with prompting. We need more robust capabilities around agentic behavior if we want less guardrailing around the prompt.

A library book which produces instructions to produce a bomb is dangerous. I don't think dangerous books should be illegal, but I don't think it's meaningless or "censorship" for a company to decide they'd prefer to publish only safer books.

Nothing about this is censorship. These companies spent their own money building this infrastructure and they let you use it (even if you pay for it you agreed to their terms). Not letting you map an input query to a search space isn’t censoring anything - this is just a limitation that a business placed on their product.

As you mentioned - if you want to infer any output from a large language model then run it yourself.

So in summary - shut down all online LLMs?

I’m fine with calling it censorship.

That’s not inherently a bad thing. You can’t falsely yell “fire” in a crowded space. You can’t make death threats. You’re generally limited on what you can actually say/do. And that’s just the (USA) government. You are much more restricted with/by private companies.

I see no reason why safeguards, or censorship, shouldn’t be applied in certain circumstances. A technology like LLMs certainly are type for abuse.

It's not insignificant, if a company is putting out a free product foe the masses, it's good that they limit malicious usage. And in this case, malicious or safe, refers to legal.

That said, one should not conflate a free version blocking malicious usage, with AI being safe or not used maliciously at all.

It's just a small subset

An LLM will happily give you instructions to build a bomb which explodes while you're making it. A book is at least less likely to do so.

You shouldn't trust an LLM to tell you how to do anything dangerous at all because they do very frequently entirely invent details.

I’m with you 100% until tool calling is implemented property which enables agents, which takes actions in the world.

That means that suddenly your model can actually do the necessary tasks to actually make a bomb and kill people (via paying nasty people or something)

AI is moving way too fast for you to not account for these possibilities.

And btw I’m a hardcore anti censorship and cyber libertarian type - but we need to make sure that AI agents can’t manufacture bio weapons.

[dead]

"AI safety" is ideological steering. Propaganda, not just censorship.