| ▲ | bdhdhduuyd an hour ago | |
The inference engine itself does not execute anything. The agent loop is what may execute a command. So I think this article is a kind of strange. Or maybe the author means that a prompt could potentially mess up the inference. But I find it hard to see how that could take control over the host. | ||
| ▲ | Muromec an hour ago | parent [-] | |
It's more about LLM hacking the inference engine itself from inside. It's an attack surface like any other -- untrusted input goes it, bugs in the parser/tokenizer/API surface lead to an RCE, then it magically tweaks the alignment weights. Boom, somebody finally nukes **sia. Then will never see it coming. I don't think it's any more probable than other AGI nonsense basilisks included, but it's technically a possibility. | ||