Literally all of software is artificial? Being explicit and reasoned about how you choose to allow or deny a particular computation is, surely, at the heart of a lot of computer security?

▲

TeMPOraL 4 hours ago | parent [-]

Code/data separation is at the heart of computer security in the same way slapstick comedy is at the heart of humor.

There's an endless supply of people who think they know what is Code and what is Data, and they're always arguing with others who also think that, and neither realize that Code/Data classification is an opinion, a perspective. It doesn't hold in general.

Having a separation like this makes sense for super narrow systems, where you can define the allowed and disallowed use cases, enforce the distinction (because it's not real - therefore you have to enforce it mechanistically within your system), and willing to accept that some useful operations will be denied by your system.

	▲	4 hours ago \| parent \| next [-]
		[deleted]
	▲	simoncion 2 hours ago \| parent \| prev [-]
		> ...and neither realize that Code/Data classification is an opinion, a perspective. It doesn't hold in general. Okay. To pull this back on topic, and to simplify it a bit so you can better grasp the core issue that's being talked about: The "Unless your program requires it, always ensure that your code cannot be altered by the data it processes. And if you think that your program requires it, go back and think again." security lesson that the industry collectively learned like thirty or fifty years ago can be restated as > Don't blindly do what some arbitrary stranger yelling in the street tells you to do. Despite how passionately the major LLM providers claim they're super serious about security and alignment [0], we see time and time again that their tooling doesn't reliably distinguish between system instructions, -at times- its own internal chatter, user instructions, and attacker-controlled instructions. Companies that claim their tools are "aligned", but think it's okay for their tools to blindly do what some arbitrary stranger is yelling at them to do are not companies that are even a little bit serious about either security or safety. [0] "Alignment" being a fancy word for "The software does what you told it to, and -once the software is much more powerful than it is today- what you actually intended for it to do.". Tools that mix together system instructions, user instructions, and attacker-controlled instructions and fail to reliably distinguish between the sources of those instructions cannot be "aligned". It's simply impossible.