wiring up an RNG to your CLI has fairly obvious risks, the root of the problem is ~everyone's treating GenAI as if it's AGI - the rest is popcorn fodder.

▲

kbrkbr 4 hours ago | parent | next [-]

That.

"And it confessed in writing" - no, it created probabilistically token after token based on the context without any other access to what happened.

LLMs can't explain themselves in the manner relevant here, much less confess.

▲

tantalor 5 hours ago | parent | prev [-]

New rule: Roll a 1 on a D20 -> you accidentally delete your own database

▲

saghm 4 hours ago | parent | next [-]

This is actually a fun way to describe it. I've being saying for a little while now that using AI for things where there's consequences if it fails is a bad idea, but it never occurred to me that this is basically the same concept as some rules in tabletop RPGs.

In D&D 3.5 edition, there was a rule about how you could "take 20" on a d20 roll to get a guaranteed 20 by taking 20 times as long in-game to perform the action, but only if it was a check that didn't have consequences for failure, since it was essentially a shortcut to skip the RNG of rolling until you rolled a 20. Maybe framing it like this might make sense to people a bit more, but if not, I'll at least have more fun making my case.

▲

pc86 4 hours ago | parent | prev [-]

It seems closer to "roll two or three successive 1s on a D100 and have your LLM hooked directly into your production systems and have your LLM user have DELETE permissions" and probably 1 or 2 other things I'm forgetting.

▲

kennywinker 4 hours ago | parent [-]

It pulled an api key from an unrelated file. It wasn’t given delete permission, it found it.

Not picking on you specifically, but in general the comments here have me wondering if AI has stolen our basic reading comprehension, or if we were always this bad.

Anyway, take “LLM user had delete permission” off your list and add “deleting the production db also deletes all the backups” to the list.

	▲	pc86 2 hours ago \| parent [-]
		Fair criticism mate. I'll only say that if your backups aren't in a completely separate system you don't really have backups.