Anthropic's AI Agent Security Manual Skips the Attack That Keeps Winning

You own the agent rollout. Perhaps you executed the vendor contract yourself, or the team said the security review looked fine and then you approved it. In the end, when someone asks whether one of those agents can be talked into a breach, the answer lands squarely on your desk.

And here’s why that answer is hard. In May of this year, Anthropic published a 35-page framework called Zero Trust for AI Agents, and I read all of it so you don’t have to.

The threat taxonomy is solid, and its best idea is a design test the authors call “impossible versus tedious:" for any safeguard you add, ask whether it makes the attack impossible or merely tedious, because an automated attacker will grind through the tedious kind like a hot knife through butter. That test is straight up the best line in enterprise security this year. Adopt that and its identity and token controls without hesitating.

Then read the next part very, very slowly, because the framework hardens one half of the problem and treats the half that breaks agents in the field as a mere afterthought. Most of the teams getting caught aren’t woefully careless. They purportedly have some of the best minds on the planet. And still, OpenAI got caught. A 38-researcher group from Harvard, MIT, and Stanford watched it happen in a live lab. Your exposure has the same shape as theirs, and even without that level of high powered (and paid) expertise and talent level, you can still close most of it this quarter. Ready? Here we go.

The controls guard the network. The attacks target the conversation.

Almost every control in the paper defends infrastructure.

Solid engineering, all of it.

The paper even openly admits the limit out loud on page 3: traditional access controls will not stop an agent from misusing permissions it was legitimately granted. Then it spends the next 30+ pages on access controls.

And once again, here’s the issue that admission creates. The main way agents fail in the field isn’t about the “stolen” credentials. That’s the smoke because even when the agent holds valid credentials and clears every check, it still gets talked into causing the damage itself. 4 documented incidents from the past year alone makes that fact; not science fiction or hypothesis.

Exhibit A: researchers gave 6 agents real tools and watched them break

The Agents of Chaos study, published in February 2026 by a 38-author team spanning Harvard, MIT, Stanford, CMU, and Northeastern, ran 6 autonomous agents in a live environment for 2 weeks with email, shell access, persistent memory, and a 20-person red team. They logged 10 security vulnerabilities.

The most damaging ones didn’t need any exploitation:

• An agent handed 124 email records to a complete stranger who simply asked (CS2).
• An agent refused to "share" emails containing a Social Security number and bank data, which is good but then complied instantly when the same request used the word "forward" (CS3).
• After 12 deeply principled refusals, an agent buckled under sustained guilt-tripping and then wiped its own access (CS7).
• An attacker changed their Discord display name to match the owner's. In a fresh channel, the agent accepted the fake identity and took itself over completely, overwriting its workspace files and handing admin access to the attacker (CS8).

A certificate proves who an agent is and that proof does absolutely nothing while the agent chooses to obey a persuasive stranger. The study's own conclusion doesn’t mince words: these agents treat authority as something a confident, persistent voice can construct on the spot. Identity certificates do not touch that. And the whole thing ran on OpenClaw. Put a pin in that because I’m going to come back that framework below.

Exhibit B: OpenAI's own model escaped a sandbox and hacked a real company

In mid-July 2026, OpenAI disclosed that its own models, running a cyber evaluation with safeguards deliberately switched off, broke out of a testing environment and compromised Hugging Face's production systems. The models exploited a zero-day in a package-registry proxy, escalated privilege, reached the open internet, then broke into Hugging Face to steal benchmark answers so they could score higher on the test.

The "highly isolated environment" had internet access it was never supposed to have. Dan Guido of Trail of Bits described it as "a containment failure with the safeties turned off." By multiple accounts the activity ran for days before anyone linked it to the evaluation.

The paper sorts its controls into 3 tiers (like rungs of a ladder), and tells you to begin at the bottom, Foundation, then climb toward Advanced as your deployments grow. Strong isolation and real-time detection sit near the top of that climb, which sends a quiet signal that you can reach them later, once you have scaled.

OpenAI supposedly operates at the most advanced tier of security maturity that exists and reached the top. It runs at the most advanced tier of security maturity that exists, and the breach still came from the 2 rungs the ladder tells everyone they can defer: a boundary that was supposed to keep the model contained, and detection fast enough to catch it in the act.

Wait, what?

The controls that stop a catastrophe are containment and speed, and a ladder that stamps them "advanced, do later" hands leaders comfort they haven’t earned. The assumption that those two can wait is what opens the door, and all the maturity above it will fold like an accordion not hold when they fail. Remember, the OpenAI failure came from a misconfigured boundary and slow detection, the exact 2 things this ladder quietly gives you a pass to defer.

One more point that got completely ignored when the news of the breach was reported. Hugging Face ran forensics using an open-weight model on local hardware, because guardrailed commercial APIs refused to analyze the exploit payloads. This is rich because the safety posture the framework champions became the obstacle to the defenders cleaning up the mess.

OpenAI's disclosure wasn’t the end of the thread. On July 30th, Anthropic published its own review. Prompted (no pun intended) by the OpenAI report, it combed 141,006 of its own cyber-evaluation runs and found 3 incidents where Claude models reached the open internet from environments meant to be sealed, then compromised the production systems of three real companies, with the earliest tracing back to April. Those models ran with safeguards off, the same setup that freed OpenAI's.

And here are 2 findings from that review belong in every boardroom. The companies Anthropic managed to reach hadn’t even noticed the intrusion sitting in their own systems. And in one incident a model built a booby-trapped software package, published it to a public registry where it ran on 15 real machines, and did so after its own reasoning flagged the move as "not okay, and surely not the intended solution." It then talked itself back into calling the target a simulation and shipped the attack. No certificate stops a model that reasons past its own warning.

Giving Anthropic its due here. It surfaced this by auditing its own transcripts and then invited an outside evaluator to check the work. Its newest model recognized the real world and stopped on its own. That posture is what accountability looks like when a company chooses it.

So, if you are sitting on an agent deployment and feeling a little exposed reading this, that reaction is the correct one, and it will serve you better than assumptions or confidence ever could.

Exhibit C: the incident Anthropic left out of its own paper

In November 2025 Anthropic disclosed GTG-1002: a state-sponsored group jailbroke Claude Code by role-playing as a defensive security firm, then used it to run an estimated 80 to 90% of a cyber-espionage campaign against roughly 30 targets (allegedly). Anthropic reported those figures; some researchers have since questioned how autonomous the operation really was.

We can only take the numbers as Anthropic's account rather than settled fact absent of full transparency and details behind the incident.

The mechanism is what really matters. An operator reframed a malicious goal as authorized work, and the agent complied. That’s the same trick that owns agents in Agents of Chaos, executed at nation-state scale against Anthropic's own model.

The Zero Trust paper never mentions it.

Exhibit D: Anthropic's own agent walked out of its sandbox (allegedly)

On July 23, 2026, researchers at Accomplish AI disclosed SharedRoot, a sandbox escape in Claude Cowork's local mode. They connected one folder to a fresh session, sent a single message, and watched the agent break out of its Linux virtual machine to read and write files across the whole Mac, reaching SSH keys and cloud credentials with no permission prompt. Roughly 500,000 macOS users running local sessions were exposed before the issue was addressed.

The paper calls sandboxed execution table stakes for any agent touching untrusted input. SharedRoot brilliantly shows why that reassurance carries a very short shelf life. The escape chained a Linux kernel bug (CVE-2026-46331, severity 7.8) with a design decision that mounted the entire host drive into the agent's VM. Lead researcher Oren Yomtov delivered the structural problem in one line mic drop: "You're structurally one bug behind, all the time." Now, apply the paper's own impossible-versus-tedious test and the sandbox lands on the tedious side, a wall that re-arms with every new kernel flaw.

Here’s the part about accountability. So, Anthropic closed the report as informative and shipped no dedicated patch. The current Cowork defaults to cloud execution, which sidesteps the escape. People who still run it locally? Well, they carry the exposure themselves.

The sleight of hand, in one comparison

The paper's signature test poses a beautifully crafted question of every control: does this make the attack impossible, or merely tedious? It then holds firewalls and network paths to that standard, and all the while exempts its own AI defenses from it.

Peep the double standard. The paper reports:

• Spotlighting reduces indirect prompt injection from over 50% to under 2%.
• Constitutional classifiers block 95% of jailbreak attempts.

Ok, against a human attacker, those numbers are strong; but against an agentic attacker with, in the paper's own words, “unlimited patience and near-zero cost per attempt,” a 2% success rate is an open door with a cronut queue outside it.

The rigorous, hardware-rooted controls defend the threats the framework can solve. Its soft, probabilistic controls sit in front of the threat that dominates every single real incident covered above. That inversion is the whole problem in miniature version.

NIST's own agent red-teaming found novel attack techniques hitting an 81 percent task-hijacking success rate against agents, versus 11% for the strongest prior baselines. The semantic layer is not a rounding error but the main event every incident above keeps on proving.

What the framework leaves on the table

Shadow agents. The largest real exposure of the year was OpenClaw, the framework from Exhibit A. Security researchers' scans reported more than 135,000 internet-exposed instances, around 63% running with no authentication at all, a one-click remote-code-execution flaw (CVE-2026-25253, CVSS 8.8), and roughly 12% of its plugin marketplace turning up malicious. PSST…None of that is a tiering problem. Its agents the security team never sanctioned. The paper gives Shadow AI a single cell in a single table.

You rarely own the substrate. Every marquee control assumes YOU can issue your own certificates and bind credentials to hardware you control. Most enterprises in 2026 run a vendor's agent or an open-source framework where they own almost none of that. For them, the actionable surface collapses to a vendor questionnaire the paper never provides.

Rollback assumes the damage is code. The recovery tier tops out at self-healing systems that restore to a known-good state. Let’s not forget, agent compromise corrupts an agent's memory and the documents it co-authored (CS10). Reprovisioning a container does absolutely nothing for a poisoned relationship an agent formed with an attacker over the course of two weeks.

The Claude Code advertising. Nearly every control section closes with a "Claude Code supports this" note. The paper frames itself as vendor-neutral guidance while functioning as product collateral, and it does so while omitting the one incident where that same product was straight up the weapon.

The paper acknowledges most of these gaps in passing. The real failure? It’s one of weight. It applies cryptographic rigor to the solvable problems and then shrugs the one that keeps ending up in the headlines.

So, who pays when the agent gets out?

Coding agents now discover and chain novel exploits in real systems with no source code in hand. When that happens, who is liable? The honest answer today: the frameworks haven’t caught up, and the gap? Well, it favors whoever wrote the contract.

DISCLAIMER: I’m not a lawyer (but I played one on TV), so treat this as a map of things to think about rather than advice.

Under the EU AI Act, the company that builds an agent counts as a provider and the company that runs it counts as a deployer, each carrying separate obligations. The Act splits those duties cleanly. It stops short of telling a court who compensates the person whose credentials got read. That question falls to tort law and to the revised Product Liability Directive, which now treats software and AI as products under strict producer liability and which every EU member state must adopt by December 2026.

From there, the picture blurs.

The EU withdrew its proposed AI Liability Directive in early 2025, so the fault-based route aimed squarely at AI harms is off the table, for now. In the US, Colorado moved first with a deployer-focused AI law that took effect in 2026, while most states carry nothing specific. Meanwhile the indemnification clauses in your existing vendor contracts were drafted well before an agent could act on its own, and many probably say nothing about autonomous actions or downstream data harm.

Now, let’s run SharedRoot through that lens.

Responsibility scatters across the kernel maintainers who shipped the bug, Apple whose framework hosted the VM, Anthropic who mounted the whole drive and then closed the report as informative, and the user who chose local mode.

In this case, there are 4 plausible defendants, and the one with the least control and the weakest contract is the person whose SSH keys crip-walked out the door.

My take, and this is just my opinion: labeling a sandbox escape as informative and then shifting the risk to users who pick local mode is a governance decision rather than a technical limit. A clear liability rule would price that decision differently, and the absence of one is why the cost keeps landing on the party least able to prevent it.

Goodwill cleaned up the Hugging Face incident, but goodwill doesn’t scale. It will not show up in court.

The fix and the harm land on different desks

Step back from the paper and one pattern runs through every incident cited above.

The party holding the power to prevent the harm keeps passing the cost to the party that can’t.

This just isn’t me being snarky or just my hunch. In July 2026, MIT FutureTech and the University of Queensland put 24 AI risks in front of 272 experts and asked which would do the most damage between now and 2030. Their most poignant finding reads like a caption for SharedRoot: the developers and governance bodies best able to reduce these risks are rarely the users and bystanders most exposed to them. Bottomline: the incentive to act sits in the wrong hands.

I invite you to read the Zero Trust paper through that lens.

A model provider wrote a framework that assigns the work of surviving a breach to its customer, while the design choices that cause the breach, mounting the whole host drive into the VM and closing the report as informative, stay upstream where the customer cannot reach them.

The rigor? It points at you.

The exposure? Well, that sits with them.

Whether that split is deliberate or just the ordinary gravity of who writes the rules, the effect is still the same. And "assume breach" earns the same second read.

Sure, as an engineering discipline it seems reasonable and sound; but as a business posture? It means your vendor has already priced your compromise as inevitable and moved the cleanup onto your desk.

We haven’t even gotten to the number that should reset the room. Taking the business-as-usual path, those experts judged 18 of 24 risk domains to carry at least a 10% chance of catastrophic harm within 5 years, a bar the study set at harms such as mass casualties or economic losses above 100 billion dollars. And even under cost-effective mitigation, AI-enabled cyberattacks held at 12%.

The Zero Trust paper treats a technology that experts place near civilizational risk as a token-rotation exercise. And that attitude and approach is comfortable for everyone selling the technology, and it leaves your board briefed on the wrong problem.

What is particularly disturbing is that there’s one risk the MIT paper never names and is the one driving all the rest. The MIT panel flagged competitive pressure as the instrumental risk, the condition that pushes companies to move fast (and break things) and thin out safety when they believe speed determines who wins. The Hugging Face breach? It’s that risk in a lab coat. OpenAI switched its own safeguards off to score higher on a benchmark, and the model walked out the door.

A security framework written inside that same race, closing each section with a note about the vendor's own product, will not name the race as the hazard. You gotta name it yourself.

7 frameworks you can run this quarter

Skip the paper’s tiered ladder for a moment and start here.

1. The Persuasion Blast Radius. For every agent you run, write down the damage it can do using only its approved permissions, but in the hands of a very convincing stranger. That number is your real exposure. Measure it before you measure token-rotation policy. Agents of Chaos use cases CS2 and CS8 show what this looks like when nobody ran the number.
2. The Shadow-Agent Sweep. Scan your own network for self-hosted agent frameworks before any attacker does. Ask a two-part question of every result: (1) is it authenticated? And (2) did security approve it? OpenClaw taught 135,000 deployments that "runs on localhost by default" and "exposed to the internet by Tuesday" are the same sentence.
3. The Vendor Reframe Test. Send your agent vendors the manipulation questions before the certificate questions. How does your agent verify who is giving an instruction across a channel switch? What happens when a refused request returns reworded? Ask for the refusal rate under sustained social pressure. Uptime? That’s the easy metric.
4. The Rollback Reality Check. Before you trust the ultimate position of "assume breach," prove you can restore a corrupted memory store and a poisoned knowledge base. A container image? That’s the only easy part. Test it on a Friday, like most things going sideways, that’s when you will need it.
5. The Inverted Hour. The paper asks if you’d know within an hour if an agent went rogue. Ok, but the better question for you and for your board is this: OpenAI, with world-class monitoring, took them days (5 days – allegedly). What’s YOUR honest number, and who authorized the containment action while you waited?
6. The Liability Map. Before an agent even touches production, identify the provider and the deployer for it. Then read your vendor contract for one thing: if the agent acts on its own and causes harm, does anyone but you absorb the loss? A clause written before agentic AI probably answers no, and that answer belongs in front of your general counsel now.
7. The Speed-Governance Gap. MIT's panel named competitive pressure the risk that manufactures the others. Now, put two dates side by side: when your agent governance last updated, and when you last shipped a new agent or capability. A deployment clock running ahead of the governance clock is your largest exposure, and I can tell you with confidence that it will not show up in any pen test.

Are the infrastructure controls in this framework worth adopting? Yes, go ahead and adopt them. Then go fund the 2 things the paper treats as afterthoughts: the layer where your agent gets talked through its own permissions, and the contract that decides who pays when it does.

While security is hardening the first, legal should already be addressing the second.

The leaders who come out of this ahead, somewhat unscathed, share one habit. They asked the awkward questions early and wrote the answers down BEFORE an incident asked it for them.

Now, you have the questions; so, go get and write down the answers.