This is dangerous. I think of talking to an LLM like playing a game - it's a complex system and it's fun to mess with it and role play.

Taking what somebody says in an LLM chat as the truth and using it in the real world is a slippery slope. The fears about thought crime are real.

▲

cldellow a day ago | parent | next [-]

In my view, LLM companies are in a tough spot here.

I grew up in Dawson Creek, BC. An hour's drive away is Tumbler Ridge. In February 2026, someone did a mass shooting, killing 6 people and injuring 2 dozen more.

The shooter had previously discussed this with ChatGPT, but OpenAI did not report it to the authorities at the time.

In hindsight, that was an error.

Should the LLM companies be the ones determining if threats are real, or should they just default to letting the police investigate?

I'm fine with the latter. Again, I absolutely believe that these companies have immense power to do evil, I just don't think the linked article is strong evidence of it.

▲

rottencupcakes a day ago | parent [-]

By allowing the monitoring of AI conversations, we are creating a dystopian world where essentially all recorded or spoken thought is monitored (given that AI has or will have access to almost all conversations).

Is this the world we want to live in?

▲

SoftTalker a day ago | parent | next [-]

Like anything else online, if you want something to be private, don't post it.

Absolutely nothing you view, send or post on any online service can be assumed to be private.

If someone told you they were planning a mass shooting and wanted advice for how to carry it out, would you report them?

▲

NewJazz a day ago | parent [-]

So do you think offline language models should be outlawed or restricted?

▲

SoftTalker a day ago | parent [-]

I don't know how you inferred that. I said nothing about restricting offline computing.

▲

NewJazz a day ago | parent [-]

Ok, so the rules for someone who selfhosts vs. Hosts for others should be different? What if I provide inference services to my family members, should I monitor their interactions and report them if I find anything dangerous?

▲

SoftTalker a day ago | parent [-]

It's more about what I can expect.

If I keep a diary offline on paper I can expect that to be private.

If I keep a diary on a hosted diary-as-a-service, I cannot.

	▲	NewJazz 20 hours ago \| parent [-]
		Many people have different expectations. Isn't that what this whole conversation is about? You didn't answer my question.

▲

cldellow a day ago | parent | prev | next [-]

No, I don't want to live in that world, and I would definitely read an article discussing how to ensure we don't reach that world.

That said, I think that world is quite a few steps away from the scenario being discussed, which is "when talking to X, X might report my conversation to the police".

You comment elsewhere that this might be a have vs have-not divide -- if you can afford to self-host a powerful LLM, you will have more liberties.

I think this isn't that trenchant of an observation for two reasons: (1) the wealthy have always had an advantage in exercising their rights over the poor, but more importantly, (2) the capability vs cost ratio of self-hosted LLMs continues to improve at an insanely rapid pace.

▲

Hizonner a day ago | parent [-]

... and the advantage of the frontier models over anything you can possibly self-host continues to increase at an even more insane pace.

▲

serf a day ago | parent [-]

It's my perspective that open models are closing the gap faster than the frontier is widening it.

I'm not alone in that opinion.

	▲	Hizonner 4 hours ago \| parent [-]
		By "open models", do you perhaps mean the approximately trillion-parameter open models that have been getting released recently? Because basically nobody can actually self-host those. And even at that size it's debatable that they're closing the gap. Talk to me when you can do it with a 24B model quantized to 8 bits.

▲

palmotea a day ago | parent | prev [-]

> Is this the world we want to live in?

Do you think you have a choice? It's Sam, Dario, and their investors' decision about what kind of world they want to build.

▲

4d4m a day ago | parent | prev | next [-]

The legal fallout from LLMs becoming spy will overwhelm any profit from these misguided attempts at short lived contracts.

	▲	palmotea a day ago \| parent [-]
		> The legal fallout from LLMs becoming spy will overwhelm any profit from these misguided attempts at short lived contracts. Then the law must be changed to protect the profit.

▲

generic92034 a day ago | parent | prev | next [-]

While I mostly agree, was that really different with googling certain topics, before modern AI?

▲

rottencupcakes a day ago | parent | next [-]

I had to give this thought, but yes.

I could use Tor or Google from the library if I wanted. Maybe I would accidentally leak my paper trail, like the Dread Pirate Roberts (Ross Ulbricht) did. But it was possible.

The AI systems require an account.

	▲	generic92034 a day ago \| parent [-]
		I can still use chatgpt without an account. Maybe it will be a limited model, but for discussing/analyzing some topics it still does fine, most of the time. You just cannot code with it.

▲

mrguyorama a day ago | parent | prev [-]

The big problem that bothers me is the ability for post-hoc justification.

Any interaction with cops will result in them curating your entire digital presence to craft a damning story about how deviant you are, for doing things like googling TNT

Every time they shoot someone now there's an entire media blitz dedicated to how that person once did something bad.

▲

Analemma_ a day ago | parent | prev [-]

I don't think you should think of it like "playing a game", you should think of it as "posting to a web service you don't control". If you posted on a web forum about your plans for violence against the CEO of that forum, they'd probably contact the police too, and I don't particularly think there's anything sinister about this.

▲

rottencupcakes a day ago | parent [-]

So we're creating a system where people who can afford their own hardware can speak to AI freely but the plebes will be monitored by the elites for any attempted uprising?

	▲	AnonyX387 a day ago \| parent \| next [-]
		> Richard Stallman was right about everything. He might be right about this but everything is really pushing it.
	▲	Analemma_ a day ago \| parent \| prev [-]
		"Creating"? I don't think there has ever been a system where you could expect to be able to plan an uprising against the elites in public, or using infrastructure those elites own and control. I wouldn't expect to be able to plan a peasant revolt in the lord's manor.