Google's Beyond Zero: Enterprise Security for the AI Era

firasd 38 minutes ago | parent | next [-]

Honestly I think non-malicious odd behavior is under-weighted when it comes to AI agents. Even the example in this paper is about someone suspiciously accessing sales data when "why did you do that" often comes down to something in the model's training that fired as a reflex

I wrote about this a few days ago https://firasd.substack.com/p/accidental-data-loss-in-claude...

"Many researchers have made demos along these lines:

An agent is asked to check a webpage like example.com

The webpage asks for a name to proceed further

The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.

In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.

The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."

▲

oscarcp 38 minutes ago | parent | prev | next [-]

Am I undertanding this correctly? The idea is to have ultimately an AI decide if I can have access to a resource based on dynamic inference, identity , intent and service signals that can easily be manipulated?

Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)

	▲	oscarcp 23 minutes ago \| parent [-]
		Just to make my point: can I really trust humans to keep up with the required identity data that will give me enough "credibility" so the AI will give me access? Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource. What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward? Will exceptions be made by humans? In which case we go back to human-managed permissions. Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour

▲

urup2l8 an hour ago | parent | prev [-]

Oh yeah, a company whose business model is taking everyone’s data and selling it is going to help me secure my data. I guess there’s one born every minute…

	▲	exitheone 41 minutes ago \| parent \| next [-]
		This trope is so tiring. There is a massive difference between Google for enterprise customers and Google for consumers. The consumer offering is massively subsidized by ads and will use your data for ad placement, although they still never sell your data because that would hurt their business. The Enterprise offering guarantees you contractually that they never touch your data.
	▲	stingraycharles an hour ago \| parent \| prev [-]
		Where do you see them offering this service to you? They’re just publishing what they developed internally, which is what they often do. Now, Cloudflare, on the other hand, would be much more likely to offer a service like this.