Im literally working on the exact same solution. Difference is I'm running the system in a Kubernetes cluster.

I essentially run a sidecar container that sets up ip tables that redirect all requests through my mitm proxy. This was specifically required because of Node not respecting HTTP_PROXY.

Also had to inject a self signed cert to ensure SSL could be proxied and terminated by the mitm proxy, which then injects the secrets, and forwards the request on.

Have you run into any issues with this setup? I'm trying to figure out if there's anything I'm missing that might come back to bite me?

▲

hardsnow 3 hours ago | parent [-]

I’ve been running this with workloads accessing Anthropic, GitHub, Gemini, and AWS & CF R2 APIs for a while now, and have not ran into issues. I’m sure there’s an API out there that won’t work out of the box but I’m positive that support could be added.

Another thing I did was to allow configuring which hosts each credential is scoped to. Replacement /resigning doesn’t happen unless host matches. That way it is not possible to leak keys by making requests to malicious hosts.

	▲	lancetipton 20 minutes ago \| parent [-]
		The last note about configuring hosts to credentials is an excellent idea, and one I did not think to do. Currently I'm just doing a replace on any that matches in the request. This adds an extra layer of security to it. Much appreciated.