Indeed so, a fairly mundane RFP, RFQ, buyer, receiver, accounts payable process will stop a lot of problems. If an agent is inserted at some stage in the process with a clear path to make a ticket/escalate to a human if it sees something it doesn't understand, the risk isn't absurdly high, in my opinion.

I've seen so many reports of humans with the authority/ability to execute an outgoing SWIFT transfer who've been social engineered into sending money to fraudsters... Or even just the basic low level "Hey I'm your boss sending you an SMS, please go buy some gift cards and scratch them off and send me the codes". No AI involved whatsoever.

The danger exists where some true believer AI evangelist type of management person tries to fully automate the entire purchasing and AP workflow, which I'm sure some people will attempt soon, with varying degrees of success.

▲

mediaman 4 hours ago | parent [-]

I have seen the SWIFT thing happen for $100k. I think AI could actually be better for this, because it's often easier to implement hard rules for the AI.

With the SWIFT incident I saw, there was a rule that no payment can go to a vendor's bank that isn't a current, approved vendor. But the rule was not enforced in software: it was an internal accounting rule that humans were supposed to follow. The AP person "thought it had been approved" because there was a similar transaction with a different company that was a new vendor at a similar time. The other transaction was legitimate, the fraudulent spoofer wasn't. The wire got sent to a party in China.

With AI agents, if you approach it from the perspective that it will be gullible and trickable by fraudsters, you build in these hard guardrails. With humans, it's much easier to believe that "we trained Lucy on this procedure" will work in all circumstances, even if Lucy still has the technical ability to bypass the official procedure.

In these cases, it starts looking a lot more like traditional software, with your little AI chaos monkeys constrained in little boxes within the software chain.

	▲	walrus01 4 hours ago \| parent \| next [-]
		Another thing that seems to be disturbingly common these days is some party involved in residential real estate transactions having a RAT on one or more PCs, and/or compromised email account, intercepting an email at the closing stages of a house purchase and sending someone information to send a wire transfer to a fraudulent location.
	▲	jimnotgym 2 hours ago \| parent \| prev [-]
		But the hard rules only work up to the point that there is an exception. See my other post above. Occasionally a very senior person (hopefully senior) has to approve a payment that is outside of the rules, because it is something the rules did not anticipate (and now they are hardcoded into software and can't be changed).