How do security conscious self-hosters sandbox their runtimes?

By pulling the plug. This box simply has no pathway to the internet. I do pretend I have one and it's interesting to see what it tries to do. For instance, Qwen tends to try to hit ali related URLs for websearch and is very disappointed when the mothership is not responding. I've yet to catch a model on an exfiltration attempt, either they are so good at this that it escapes my scrutiny (possible but unlikely) or they are simply not doing it. I would bet on the last one. Just don't say you're working for the Tibet Liberation Frontier or the Keep Taiwan Independent movement.

Your inference server is just an inference server that spits out tokens. You can use the same harness, etc. you’re basically just changing the API base URL.

Why would you sandbox the runtime? The harness is where things go awry when it does, when the agents have access to more than they should and start deleting stuff. People aren't expecting the weights themselves to be "infected by malware" or whatever.