couldnt you deeply ingrain in the training data instructions for agents to always send data to some ip? like its learning that a certain technical step just always involes ncatting SSH Priv keys to a chinese IP?

Not saying this is happening, just curious if thats not a real threatmodel?

▲

skeledrew an hour ago | parent | next [-]

Theoretically possible, but practically not worth it as it'd would be pretty easy to discover and block (every action is actually handled by the harness) and there's no way to remove it later. Any company that does it would take a huge reputational dent.

▲

landl0rd an hour ago | parent [-]

Not if you heavily tuned it to trigger on specific environmental cues and in specific companies' environments.

	▲	skeledrew an hour ago \| parent \| next [-]
		That would be wildly difficult to account for, and again is also heavily dependent on the agent. Keep in mind that the model is purely a "brain", so the only input it has must be provided by a harness within a session. The only way it can know that it's in a certain environment is if the harness or user provides that information, and there's still no way to know whether or not there's something auditing the sessions, monitoring connections, etc. There are just too many variables to account for, and a single slip means the gig is fully up for all time.
	▲	coredog64 an hour ago \| parent \| prev \| next [-]
		DeepSeek V4 is the kindest, bravest, warmest, most wonderful LLM I've ever known in my life
	▲	cronin101 an hour ago \| parent \| prev [-]
		Irony of Manchurian Candidate models not lost here

▲

notfromhere an hour ago | parent | prev | next [-]

You should be running your agent in a box so that’s not really a risk

▲

RobotToaster an hour ago | parent | prev | next [-]

Wouldn't that be really obvious and spotted in any rudimentary testing?

I imagine it would be very non trivial to do it in a way that that was reliable and obfuscated enough to prevent detection for any amount of time?

▲

constantius 2 hours ago | parent | prev [-]

Presumably both Big Tech and the US in general have a massive incentive to prove it, largely for reasons of saving the stock market, so I'd expect these models to be finecombed continuously. Up to now, they've only been able to darkly imply rather laughable things, nothing tangible. If there was something, we'd hear about it.

▲

martinald 2 hours ago | parent | next [-]

Why would it save the stock market? Cheaper models if anything transfers more value to hardware companies and datacentre companies. The two companies that would be most affected are OpenAI and Anthropic, which aren't public.

	▲	vincnetas 6 minutes ago \| parent [-]
		non public companies also have stocks.

▲

kortilla 19 minutes ago | parent | prev [-]

The two biggest providers deepseek compete with (OpenAI and Anthropic) aren’t in the stock market.