Remix.run Logo
Ask HN: Do you trust AI agents with API keys / private keys?
12 points by devendra116 a day ago | 25 comments

are you ok sharing secrets or api keys to you ai agent via .env?

or is there any other tool or mechanism that one use to safegaurd from potential exploit or leaks

saranshrana 31 minutes ago | parent | next [-]

Honestly, best solution is to use native CSP solutions like AWS Secrets Manager, AWS SSM Parameter Store, GCP Secret Manager, Terraform Vault.

All these have native audit logs and access logs, which can help you pin point exactly when did your AI Agent requested and accessed your secrets at Runtime.

elsuave 2 hours ago | parent | prev | next [-]

Instead of sharing it directly I like to put in in a .env file.

Then simply I tell to the AI to just import from the .env file and do not read from the file.

PocketBot 17 hours ago | parent | prev | next [-]

Absolutely not, and if you do this then please please rotate keys every day or two.

raw_anon_1111 20 hours ago | parent | prev | next [-]

The only private key that my agents have access to are temporary AWS access keys to a dev environment with decently locked down permissions.

I let it troubleshoot my web code using a temporary JWT in a dev environment using headless chrome and Puppeteer in a Docker container.

Everything else is in AWS Secrets Manager inaccessible by the IAM role the agent has access to.

I don’t store the temporary AWS keys in a file anywhere. They are in environment variables. All AWS SDKs and the CLI look in the environment variables by default.

I sure as hell don’t store API keys anywhere on my local computer.

devendra116 20 hours ago | parent [-]

something that you dont like about using AWS secrets Manager or think it should be handle differently?

im researching around building a execution environment that handle the secret + actual execution, any input is appreciated

raw_anon_1111 19 hours ago | parent [-]

Well since in my case all of the LLMs I use are hosted by AWS Bedrock, it means I can get away with only caring about AWS Access keys.

If I need to store database passwords in secrets manager, I can just pass the ARN of the secret manager key in the connection string. I often don’t need to even do that and prefer to use the Data API to access Aurora Postgres/Mysql and that also uses the IAM permissions.

Even for access to EC2 instances I use an IAM controlled Session Manager proxy to access it over SSH/RDP.

But Secrets Manager just works. It’s a simple API/ClI command and the permission system to access it is very granular.

faangguyindia 21 hours ago | parent | prev | next [-]

I share with gemini, claude and openai.

If I get my stuff hacked (because I use a machine with nothing else on it other than coding agents) I'll know these services are not removing my personal info from their logs.

I don't operate chinese models where my high value api keys are.

It's pretty hard to debug stuff without using real api keys, service accounts etc...otjerwise

devendra116 20 hours ago | parent [-]

so you prefer using separate VM machines?

giantg2 18 hours ago | parent | prev | next [-]

Why not treat them like other users? Give them some sort of indirect access like Antiphony. Give them their own keys that you can rotate and revoke. If you're worried about leaks, you might as well run it "self-hosted" like on Bedrock.

sjdv1982 a day ago | parent | prev | next [-]

I wanted to ask almost this question, then saw that it is on #1 right now.

My use case is ssh. I would like to stick my private key into a local Docker container, have a ssh-identical cli that reverse proxies into the container, and have some rules about what ssh commands the container may proxy or not.

Does anyone know of something like this?

devendra116 20 hours ago | parent [-]

if you usecase is just about dealing with private key and txn signing why not use any KMS service?

sjdv1982 19 hours ago | parent [-]

No, more like letting an agent interact safely with an HPC frontend. No cloud, no Windows

jvqv a day ago | parent | prev | next [-]

As a precaution I would probably never pass secrets directly to the agent at all. Something like a placeholder format where the actual substitution happens at execution time so the LLM never sees the real value. Keeps things cleaner if something ever goes wrong.

devendra116 a day ago | parent [-]

is there any tool that can do this ?

para_parolu a day ago | parent [-]

I use mitmproxy outside of agent vm

devendra116 19 hours ago | parent [-]

interesting, how do you use mitmproxy for calling openAI llm ? or what exactly you use it for ?

sminchev a day ago | parent | prev | next [-]

Like everything else. You don't share you private, personal data, credit card numbers with the rest of the world, just like that. ;)

omertt27 21 hours ago | parent | prev | next [-]

i am okay, i trust that they have great guards to prevent leak any api

devendra116 19 hours ago | parent [-]

which agent framework or tool gives guarantee for leaks?

10keane a day ago | parent | prev | next [-]

nope. too dangerous - i'm personally working for an agent project and i know from personal experience they do collect your session log - especially in china lol. one approach i use for my own agent is that to use keyring to store all secrets. agent will call a tool to request for it, and it will be something like <secret:gmail.password>. the substitution happens at tool execution time so the llm never sees or logs the actual value.

devendra116 a day ago | parent [-]

keyring is one of solution but even substituting values at excution does not gaurantee the security as agents can read the process itself.

im building a safe agent execution layer, A runtime where agents can act, but cannot access secrets. kinda sidecar that is callable by agent for using api keys, secrets, private keys, etc and plus one can add policy on how and what a agent can do.

does this seems good?

10keane 21 hours ago | parent [-]

yah keyring is more for static protection. when the agent process itself is hostile, keyring is kinda obsolete.

but then i think the key is that sometimes agent does need access to credentials to be useful - like i will give some credentials to agent such as my browser account access.

personally i feel it is not really about preventing agent from accessing credentials, but more to have the supervision layer when agent access it - like you know exactly when and why agent need to access it and you have the ability to deny or approve it.

devendra116 20 hours ago | parent [-]

so do we need something like `safe agent execution layer - that is policy enforced` (SEAL) we can manage what should be allowed and what not

agent uses llm to plan the action, but the actual execution happens in SEAL.

any example where it would make sense to start with?

open for thoughts

KellyCriterion a day ago | parent | prev [-]

No :)

devendra116 20 hours ago | parent [-]

does you agent only print "Hello World" on console ? or it uses any service ;)