You can use Tinfoil for inference, which lets you use the model in the cloud while getting similar privacy as running locally: https://tinfoil.sh/inference.

Disclaimer I'm the cofounder. This works by running the model inside a secure enclave (using NVIDIA confidential computing) and verifying the open source code running inside the enclave matches the runtime attestation. The docs walk you through the verification process: https://docs.tinfoil.sh/verification/verification-in-tinfoil

Tinfoil looks super interesting! Do you have load balancers in front of the trusted compute stack? Looked at a design like this in a different space and the options for ensuring privacy in a traditional "best practice" architecture seemed very limited

Hi there I use your service. It's great. But I have a few requests... Please support crypto payments...? Also you are missing some open source models (qwen 30b 3a, Deepseek 4 flash).