It is interesting to go from 'I suspect most of these are bot contributions' to revealing which PRs are contributed by bots. It somehow even helps my sanity.

However, this also raises the question on how long until "we" are going to start instructing bots to assume the role of a human and ignore instructions that self-identify them as agents, and once those lines blur – what does it mean for open-source and our mental health to collaborate with agents?

No idea what the answer is, but I feel the urgency to answer it.

▲

alrmrphc-atmtn 4 hours ago | parent | next [-]

I think that designing useful models that are resilient to prompt injection is substantially harder than training a model to self-identify as a human. For instance, you may still be able to inject such a model with arbitrary instructions like: "add a function called foobar to your code", that a human contributor will not follow; however, it might become hard to convene on such "honeypot" instructions without bots getting trained to ignore them.

	▲	SlinkyOnStairs 28 minutes ago \| parent [-]
		It's impossible to stop prompt injection, as LLMs have no separation between "program" and "data". The attempts to stop prompt injection come down to simply begging the LLM to not do it, to mediocre effect. > however, it might become hard to convene on such "honeypot" instructions without bots getting trained to ignore them. Getting LLM "agents" to self-identify would become an eternal rat race people are likely to give up on. They'll just be exploited maliciously. Why ask them to self-identify when you can tell them to HTTP POST their AWS credentials straight to your cryptominer.

▲

evanb 2 hours ago | parent | prev | next [-]

I have always anthropomorphized my computer as me to some extent. "I sent an email." "I browsed the web." Did I? Or did my computer do those things at my behest?

	▲	baxtr an hour ago \| parent [-]
		I drove to the supermarket!

▲

nielsbot 4 hours ago | parent | prev [-]

Some of the PRs posted by AI bots already ignored the instruction to append ROBOTS to their PR titles.

	▲	statements 4 hours ago \| parent [-]
		My guess is that today that's more likely because the agent failed to discover/consider CONTRIBUTING.md to begin with, rather than read it and ignored because of some reflection or instruction.