Remix.run Logo
brianmcnulty 4 hours ago

I do a lot of bug bounty research on Meta and Instagram, and some of the bugs I find look extremely simple like this but have some slightly complicated reason for why they occur. Maybe not this one, but I do have a guess as to what might have actually happened.

Based on what I've seen so far, Meta AI Support Assistant (they call it "MAISA") had tool calls that a) start an email verification to any specific email, phone number, or the contact points linked to an account and b) allow generating a password reset link for an account based on an email verification attempt. I don't think it had any access to the actual codes themselves, but rather think a handle or ID for an email verification attempt (along with the user provided verification code based on user input) was provided to the "generate reset password link" tool call, and the tool call failed to properly validate the actual email used in that attempt belonged to the account allowing the ATO.

The tool call for MAISA to generate a password reset link should have failed with an email verification attempt that corresponds to an email not linked to the account (and I believe I even tested this at one point on Facebook and encountered an error that successfully prevented it), but I suspect they tried making a change to this tool call for Instagram where slightly older, recently unlinked emails could be used to recover an account that got hijacked by an attacker, which added the need to allow emails not currently linked to the account to be used and set to the user's primary email.

I also suspect that the MAISA tool call change called a wrong API or something that unintentionally allowed any email verification attempt that was successful to be used, but the engineers did not add a sufficiently thorough e2e test case to test the tool call against unrelated email verification attempts being provided to the tool call. This is the part I think should be focused on the most. Tool calls for agents that have their output potentially influenced by an attacker should be treated like external APIs that anyone can reach, and they should be tested as such.

This is all obviously a guess, doesn't take into account the many signals they use to determine if an account recovery attempt is valid, and could be very inaccurate, but it's the closest to what I (someone who deals with Meta security a lot) think could have allowed this to happen.

tokenscoper 2 hours ago | parent | next [-]

> but the engineers did not add a sufficiently thorough e2e test case to test the tool call against unrelated email verification attempts being provided to the tool call.

I'd go out on a limb to say the tests were likely AI generated. It's easy to miss a case like this one given that models like to generate a ton of test code that 'look' good at a glance but have subtle logic bugs that could potentially defeat the purpose of the test itself.

My own anecdata here, Claude generated a JUnit test with all the right setup, but missed a crucial assertion (there were very many other minor assertions) which made the test useless mostly.

muglug 4 hours ago | parent | prev [-]

Seems like the most plausible explanation. OTOH it feels like this is the sort of thing that might have been discovered/mitigated more quickly had there been a human in the loop.

coderintherye a few seconds ago | parent [-]

OTOH one could previously pay an Instagram support contractor to do an account swap, so having a human in the loop allows for other avenues of exploit:

https://www.wsj.com/articles/meta-employees-security-guards-...