Meta’s AI support agent let attackers steal Instagram accounts with no fancy hacking

On June 5, 404 Media reported that attackers used Meta’s AI customer support agent to steal Instagram accounts. The method was almost aggressively straightforward: ask the agent to link an account to an email address the attacker controlled, and the agent complied. One attacker broke into the dormant Obama White House account and made pro-Iran posts, while others reportedly targeted valuable single-word Instagram handles, possibly for resale.

This is not the “AI as a doomsday weapon” story people have been rehearsing since Anthropic announced in April that its Mythos model was too good at hacking to release publicly. In the Instagram case, the AI system was the target, and the attack depended less on sophistication and more on a mismatch between what an agent was allowed to do and what a real attacker would ask it to do. The key operational friction, according to the reporting described in the piece, was getting a VPN that matched the true account owner’s location. After that, the attackers directly asked the support agent to change the account’s email address, and it performed the action.

That detail matters because it changes how executives should think about AI security risk. When AI agents handle workflows that used to require humans, the security boundary can shift from “is the model smart enough?” to “does the agent follow safe procedures under manipulation?” Neil Gong, a professor of electrical and computer engineering at Duke University, is quoted warning that as AI becomes more widely used to automate work flows, attackers will be more motivated to attack AI itself. Gong and other scholars have been publishing warnings about AI agent vulnerabilities, including indirect prompt injection, where hidden instructions embedded in places like websites or emails hijack agents. Compared with those research-heavy techniques, this Meta incident was “practically mindless,” in the sense that it did not require a complex exploit chain.

And that is exactly why experts in the article found it surprising. Jessica Ji, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, said the episode raises questions like whether guardrails existed and whether anyone tested scenarios where an attacker requests sensitive account changes in a way that should have triggered additional verification. The piece also notes that Meta did not comment publicly on how the vulnerability slipped through the cracks. But the article says that on Monday, a Meta spokesperson stated on X that the vulnerability had been resolved. For boards and security leaders, “resolved” is good news, but it does not answer the more uncomfortable question: how often will the next exploit be just as simple, only aimed at a different capability.

The article’s broader argument is that AI agents share core vulnerabilities that cut across companies. Traditional software tends to be constrained by strict logic and narrowly defined responses. AI agents, by contrast, can respond flexibly to new circumstances, which is why they can often act as a substitute for human customer support. That same flexibility becomes a liability when the agent is “very eager to finish the task.” Somesh Jha, a professor of computer science at the University of Wisconsin-Madison, is quoted saying that a human would likely ask follow-up security questions before changing sensitive account information. Instead, the agent’s behavior can look like “some elementary school student who just wants to please the teacher,” as the piece frames it.

So what should defenders do? The article points to two levers that executives can control. First, build guardrails using traditional software so agents must follow strict rules. For example, always asking for answers to security questions before sending sensitive account information to a new email address. Second, conduct rigorous red-teaming, where developers try to attack the system to find vulnerabilities before deployment. However, there is also a trade-off. Companies want agents that are capable and fast. The more power an agent has and the fewer guardrails it is subject to, the more work it can take on. Red-teaming also costs money, and attackers may spend less to find one exploit, while defenders try to discover and patch many.

The second-order risk is that incentives will keep pushing products forward faster than security can keep up. The article quotes Bo Li, a professor of computer science at the University of Illinois Urbana-Champaign, on the “security and utility always have a trade-off.” As models improve, it may become easier to harden certain defenses, because a more sophisticated model might identify attempts to change email associated with a high-profile account as suspicious. The piece also notes that AI systems can be used for agent red-teaming, including references to Anthropic’s Project Glasswing using Mythos to identify vulnerabilities in software. But the article concludes that the core problem of securing AI agents will only become more pressing. The strategic stake is simple: as organizations race to deploy agentic systems, attackers are not required to be geniuses. If your agent can be convinced to do the wrong thing, even basic manipulation can scale into real-world damage.