> Sure, I suppose that's something that someone who doesn't understand the discussion might say.

I'm going to be frank, you are the person who misunderstands (and are being rather rude about it). You are responding to an argument no one is making.

To put a fine point on it, you said this:

> Entropy may mean all ceilings collapse eventually, but that doesn't mean we aren't able to make useful ceilings.

But you were responding to a comment saying this:

> Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material.

Emphasis added. They are saying maintenance is necessary, not that a safe ceiling is unachievable. It's obviously achievable, we've all seen it achieved.

They further say:

> It boggles the mind to let an LLM have access to a production database without having explicit preventative measures and contingency plans for it deleting it.

Emphasis added. When they say it boggles the mind to deploy an LLM without the proper measures, the implication is that it does make sense to deploy it with the proper measures.

> ...the point that stands is that LLMs and ceilings are both useful, neither are doomed such that no one should use them, ...

I have not seen a single person in this subthread say that LLMs aren't useful or that they are doomed. People say that. But the people you're talking to haven't.

I try to avoid these petty "I brought the receipts" comments, but I don't like the way you're being snarky to people who's crime is engaging with the premises you set up. The faults you are finding are faults you introduced. I'd appreciate if you would avoid that in the future.

▲

margalabargala 2 hours ago | parent [-]

If that's what you got out of the above conversation that is about as fundamental a misunderstanding as the one at the top of this thread saying "It is fundamental to language modeling that every sequence of tokens is possible". I could say something rude here about both mistakes being made by the same person, but since you brought it up I won't.

If you want to take a comb to it, the comment saying this:

> Except your ceiling can and will fall on you unless you take preventative measures, entirely due to molecular interactions within the material

Was already off the plot. What was being discussed wasn't some specific molecular process, it was the false premise "oh molecules move around randomly so your ceiling might just collapse of its own accord because the beam decided to randomly disintegrate". That's not something that happens.

You said "The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use". This is analogous to "the ceiling could just collapse on you due to random molecular motion, no matter how much maintenance you do or what materials you use".

Make sense now?

Your edit at the bottom of your top comment does better than your original statement.

▲

maxbond 2 hours ago | parent [-]

> I could say something rude here about both mistakes being made by the same person, but since you brought it up I won't.

I'd encourage to desist from rudeness, not just when people point it out to you, but at all times.

> You said "The sequence of tokens that would destroy your production environment can be produced by your agent, no matter how much prompting you use". This is analogous to "the ceiling could just collapse on you due to random molecular motion, no matter how much maintenance you do or what materials you use".

If prompt engineering is effective (analogous to performing the necessary maintenance and selecting the correct materials), I'm curious what your explanation is for the incident in the article?

▲

margalabargala 2 hours ago | parent [-]

> I'd encourage to desist from rudeness, not just when people point it out to you, but at all times.

I desire neither to be inauthentic, nor to suppress my emotions.

> If prompt engineering is effective (analogous to performing the necessary maintenance and selecting the correct materials), I'm curious what your explanation is for the incident in the article?

Keeping with the analogies, the original article doesn't say whether they built the roof properly or if the just used some screws to hold up a piece of quarter inch plywood and called it a day.

It's no surprise that a terribly built roof may fall down. It's possible to get shoddy materials from a supplier without knowing.

Calling a curl command isn't something that would be within the model's training as "this deletes things don't do it". The fact that this happened is not, to me, evidence that the model might have equally run `sudo rm -rf --no-preserve-root /` under similar circumstances.

It sounds like the phrase "NEVER FUCKING GUESS!" was in the prompt as well, which could easily encourage the model towards "be sure of yourself, take action" instead of the "verify" that was meant.

As mentioned elsewhere in this thread, the fact that the article focuses so strongly on "the model confessed! It admitted it did the wrong thing!" doesn't lead me to put a ton of stock into the capability of the author to be cautious.

	▲	maxbond an hour ago \| parent [-]
		[dead]