I think a big part of mitigating this will probably be requiring multiple agents to think and achieve consensus before significant actions. Like planes with multiple engines

I think the right solution is to endow the LLM with just enough permissions to do whatever it was meant to do in the first place.

In the customer service case, it has read access to the customer data who is calling, read access to support docs, write access to creating a ticket, and maybe write access to that customer's account within reason. Nothing else. It cannot search the internet, it cannot run a shell, nothing else whatsoever.

You treat it like you would an entry level person who just started - there is no reason to give the new hire the capability to SMS the entire customer base.

engines are designed to behave in very predictable ways. LLMs are not there yet

How is this that different from a mixture of experts in a single model? There are some differences in training etc but it’s not that different at a fundamental level. You need to solve the issue with a single model.

The multiple model concept feels to me like a consumer oriented solution, its trying to fix problems with things you can buy off the shelf. It’s not a scientific or engineering solution.

[dead]