it seems crazy to "trust" an LLM with any secrets. Anyone running one as their normal user account with access to all files is playing with fire...

▲

epistasis 5 hours ago | parent | next [-]

I don't think anybody actively trusts a hosted LLM with secrets. The problem is that they don't realize they have granted trust to the LLM.

▲

cozzyd 5 hours ago | parent [-]

People happily run AI Desktop agents or whatever on their main user acounts commingled with ssh keys and who knows how many tokens.

▲

forgotaccount3 3 hours ago | parent | next [-]

Sure, some do.

But also... I use Kiro. I open a terminal into a folder where my repo is. I run kiro-cli. I don't know if it has access to the credentials file in my .aws directory. I know it prompts me for approval to use tools but that is a harness thing, does the mac itself prevent it from accessing the credential file?

I use AI because it's useful and I follow the practices dictated by our AI adoption team but I don't know the nuance of everything about it and that makes it difficult to know when some case which is not explicitly covered by training might leak important information.

	▲	epistasis 3 hours ago \| parent [-]
		One advantage of AWS is short-lived credentials (hopefully, as long as it's configured correctly!) So go ahead and dump your AWS SSO tokens to the LLM by accident, but it's going to take longer than a day to train a new model and ship it out to the world. Also, I think kiro only uses AWS Bedrock, IIRC, so no training data goes back to the LLM manufacturers? At least I would hope so. Database passwords, API keys to services with arduous rotation procedures, that's where the real exploits will come from in coming months, I think.

▲

epistasis 4 hours ago | parent | prev [-]

This is one reason I haven't had any SSH keys on disk (encrypted or not) ever since I got a YubiKey, and it's only become easier with Secure Enclave on macs since then.

However, dev database passwords for small projects in .env files? API keys to some random LLM service that I put $5 into once 8 months ago and haven't touched since then? All that's open to the LLM.

It's time to clean up our personal disks as if we had an intruder exfiltrating sensitive secrets at all times.

▲

cyanydeez 6 hours ago | parent | prev [-]

seems crazier someone would tie their entire development platform to a cloud run by business interests