This is the right approach. I built a similar system to https://github.com/airutorg/airut - couple of learnings to share:

1) Not all systems respect HTTP_PROXY. Node in particular is very uncooperative in this regard.

2) AWS access keys can’t be handled by simple credential swap; the requests need to be resigned with the real keys. Replicating the SigV4 and SigV4A exactly was bit of a pain.

3) To be secure, this system needs to run outside of the execution sandbox so that the agent can’t just read the keys from the proxy process.

For Airut I settled on a transparent (mitm)proxy, running in a separate container, and injecting proxy cert to the cert store in the container where the agent runs. This solved 1 and 3.

▲

lancetipton 3 hours ago | parent | next [-]

Im literally working on the exact same solution. Difference is I'm running the system in a Kubernetes cluster.

I essentially run a sidecar container that sets up ip tables that redirect all requests through my mitm proxy. This was specifically required because of Node not respecting HTTP_PROXY.

Also had to inject a self signed cert to ensure SSL could be proxied and terminated by the mitm proxy, which then injects the secrets, and forwards the request on.

Have you run into any issues with this setup? I'm trying to figure out if there's anything I'm missing that might come back to bite me?

▲

hardsnow 3 hours ago | parent [-]

I’ve been running this with workloads accessing Anthropic, GitHub, Gemini, and AWS & CF R2 APIs for a while now, and have not ran into issues. I’m sure there’s an API out there that won’t work out of the box but I’m positive that support could be added.

Another thing I did was to allow configuring which hosts each credential is scoped to. Replacement /resigning doesn’t happen unless host matches. That way it is not possible to leak keys by making requests to malicious hosts.

	▲	lancetipton 22 minutes ago \| parent [-]
		The last note about configuring hosts to credentials is an excellent idea, and one I did not think to do. Currently I'm just doing a replace on any that matches in the request. This adds an extra layer of security to it. Much appreciated.

▲

inssein 3 hours ago | parent | prev | next [-]

This is basically what https://www.verygoodsecurity.com/ (their main product), but it's heavily focused on credit card data.

▲

semanticc 3 hours ago | parent | prev [-]

Node 24+ does respect HTTP_PROXY when NODE_USE_ENV_PROXY=1 is set.