Remix.run Logo
▲ necovek 3 hours ago

By running automated AI agents on untrusted inputs (PRs from strangers on a public repo), you open yourself up to attacks. And you could be spending money for somebody else.

▲linsomniac an hour ago | parent | next [-]

Agreed and this is a very real thing to be concerned about.

I like the idea of leaning on a decision like Jev model to identify such things.

▲epistasis 3 hours ago | parent | prev [-]

The only attacks possible would be the tools given to the agent.

▲swiftcoder 2 hours ago | parent [-]

Assuming it can't perform a sandbox escape, at any rate. Maybe a Jev-like decision model would be a better idea.

▲epistasis 36 minutes ago | parent [-]

Or just don't allow tools, write a simple python script that uses the API and doesn't do tool calls, etc. etc. etc.

Table stakes for being an engineer these days should include knowing how to control LLMs, it's as fundamental as a working knowledge of databases. One doesn't need to know how design a data model from scratch or code up that python script using the OpenAI API from memory, but should know the general shape of what's possible and how to look it up.