An eager intern can remember things you tell beyond that which would fit in an hours conversation.

A disgruntled employee definitely remembers things beyond that.

These are a fundamentally different sort of interaction.

Agreed, but the point is, if your system is resilient against an eager intern who has not had the necessary guidance, or an actively hostile disgruntled employee, that inherently restricts the harm an LLM can do.

I'm not making the case that LLMs learn like people. I'm making the case that if your system is hardened against things people can do (which it should be, beyond a certain scale) it is also similarly hardened against LLMs.

The big difference is that LLMs are probably a LOT more capable than either of those at overcoming barriers. Probably a good reason to harden systems even more.

	▲	gpm 2 hours ago \| parent \| next [-]
		The difference makes the necessary barriers different. There's benefit to letting a human make and learn from (minor) mistakes. There is no such benefit accrued from the LLM because it is structurally unable to. There's the potential of malice, not just mistakes, from the human. If you carefully control the LLMs context there is no such potential for the LLM because it restarts from the same non-malicious state every context window. There's the potential of information leakage through the human, because they retain their memories when they go home at night, and when they quit and go to another job. You can carefully control the outputs of the LLM so there is simply no mechanism for information to leak. If a human is convinced to betray the company, you can punish the human, for whatever that's worth (I think quite a lot in some peoples opinion, not sure I agree). There is simply no way to punish an LLM - it isn't even clear what that would mean punishing. The weights file? The GPU that ran the weights file? And on the "controls" front (but unrelated to the above note about memory) LLMs are fundamentally only able to manipulate whatever computers you hook them up to, while people are agents in a physical world and able to go physically do all sorts of things without your assistance. The nature of the necessary controls end up being fundamentally different.
	▲	2 hours ago \| parent \| prev [-]
		[deleted]

▲

braebo 3 hours ago | parent | prev [-]

You can easily persist agent memories in a markdown file though.

▲

collinmcnulty 2 hours ago | parent | next [-]

And the memento guy had tattoos of key information. That didn’t make it so he didn’t have memory loss.

	▲	WhatIsDukkha 2 hours ago \| parent [-]
		Pretty good metaphor. Limited space to work with, highly context dependent and likely to get confused as you cover more surface area.

▲

whstl 3 hours ago | parent | prev | next [-]

Which it will start ignoring after two or three messages in the session.

▲

Quarrelsome 2 hours ago | parent | prev | next [-]

and you'll blow the context over time and send to the LLM sanitorium. It doesn't fit like the human brain can.

If a junior fucks production that will have extroadinary weight because it appreciates the severity, the social shame and they will have nightmares about it. If you write some negative prompt to "not destroy production" then you also need to define some sort of non-existing watertight memory weighting system and specify it in great detail. Otherwise the LLM will treat that command only as important as the last negative prompt you typed in or ignore it when it conflicts with a more recent command.

▲

troupo 2 hours ago | parent | prev | next [-]

Yup, and the agent will happily ignore any and all markdown files, and will say "oops, it was in the memory, will not do it again", and will do it again.

Humans actually learn. And if they don't, they are fired.

▲

estimator7292 2 hours ago | parent | prev [-]

That's not learning.