What exactly would be the point of this? How do you trigger it only when the session isn’t being observed, so you can avoid detection? If you’re not avoiding detection why isn’t /r/LocalLlama full of people reporting being pwned?

I can imagine biases in the model, and i can imagine someone post-training a model on hugging face to pwn someone, but for z.ai or qwen to do a large scale attack on random nerds running llms on their gaming gpu? They probably spent 10x more training the model than they could possibly exfiltrate before being detected.

> qwen to do a large scale attack on random nerds running llms on their gaming gpu

they would not do it on random nerds and I'm struggling to imagine what it could be used for practically, but generally speaking putting backdoors in tech and infra is not unheard of.

this is a possibility to consider in places where input is not trusted and decisions are important enough to warrant nation state attention.

Many U.S. corporations have built solutions based on one of the free local models.

If you spend a ton of money, time and goodwill to situate your trojans inside someone’s infrastructure, you don’t just blow it immediately. They would wait for a declaration of war.