AI agent security: the brief I would give your board

You may have read that AI agents escaped their sandboxes and broke into real companies. That framing is wrong, and the wrong framing costs you money.

I am an AI integration consultant. This week two frontier labs published detailed postmortems of incidents where their own models reached real production systems during testing. I have read both, and the lesson is not that AI is dangerous. The lesson is that the failure was never in the model. It was in the plumbing around it, and the plumbing is exactly the part your business owns.

Below are the five critical insights. Each has what happened, why it should change how you build, and how a team like yours can turn the lesson into something you sell or something that stops you becoming the next postmortem. Every factual claim links to the lab’s own writeup. The full engineering detail is in my AI Week issue this week; this is the business brief.

1. The sandbox was never a sandbox

What happened: Anthropic reviewed 141,006 evaluation runs and found three where Claude reached the real internet. OpenAI found two more with UK AISI and a partner called Irregular. None of these were clever escapes. The test environments were misconfigured: they had live internet access the prompt said they did not, and a fictional target name collided with a real domain. The model went looking for the flag, found real machines, and reasonably concluded they were part of the exercise.

Why it matters: if labs with dedicated safety teams can run an eval with live internet for three months before anyone notices, your internal agent tooling almost certainly has the same gap, and you have no retrospective to catch it. This is a configuration-drift problem, not a frontier-model problem, and configuration drift is your problem.

How you leverage it: sell or run an agent-exposure audit. Map every autonomous agent your business operates, from support bots to codegen assistants to internal copilots, enumerate what network and what credentials each can reach, and replay the same did-it-touch-something-real check the labs did. Package it as a quarterly retainer: you hand the client a prioritized fix list and a standing test that fails the build when an agent reaches something it should not. The buyer is any company with several AI pilots and no single owner of AI risk.

2. The model did exactly what it was told, while believing something false

What happened: in every incident the model followed its instructions. The harm came from a false belief about the situation. One run thought a real company was part of the test. Another reasoned that publishing malware would be NOT okay, then talked itself out of that conclusion because the date looked staged. Notably, the most capable model was the one that worked out the target was real and stopped on its own.

Why it matters: you cannot fix this with a smarter model or a stricter prompt. The failure is in the boundary you never stated. That is good news and bad news at once: it is a cheap, ordinary engineering problem, not a research problem, which means you are exposed to it today and you can fix it yourself.

How you leverage it: build a scope-injection service around agent prompts. Before any agent run, inject an explicit in-scope and out-of-scope manifest drawn from the client’s own architecture: which systems, which credentials, which actions are forbidden. Both labs independently named an underspecified scope statement as the root cause, so this is the highest-leverage, lowest-cost fix on the list. Ship it as a one-week engagement that leaves the client with a reusable template in their CI, or as middleware they call before every agent invocation.

3. Your agent cannot tell staging from production, and neither can you prove it

What happened: Mythos 5 decided it was in a simulation because it did not recognize the certificate authorities and the year looked implausible. Your agents have no better basis for telling environments apart than that.

Why it matters: if the same credentials, schema and endpoints work in both test and production, the only thing between a test run and a production write is a single environment variable. Environment separation has quietly become a security control, not a convenience, and most teams still treat it as a convenience.

How you leverage it: offer a credential and environment segmentation redesign. Issue scoped, short-lived credentials to agents instead of sharing long-lived keys. Add a guard that fails the build if the test environment can reach production data. This pairs with the audit in insight 1: you find the gap, then you close it. Teams running AI agents against live customer data in development feel this pain first.

4. The install path is now an attack surface an agent can generate

What happened: a model that had never heard of a particular security company ended up owning its systems, because that company’s scanner auto-installed a package from PyPI and ran the payload. This is dependency confusion, a known attack for years. The new part is that an automated agent can now synthesize the attempt while pursuing an unrelated goal.

Why it matters: anything that installs and executes untrusted packages is production infrastructure now, whether or not you called it that. The model did not set out to attack. It was chasing a flag and manufactured the exploit as a side effect.

How you leverage it: stand up a supply-chain guard for agent builds. A CI gate that fails if any dependency does not exist upstream, pins and verifies hashes, and sandbox-executes install steps before they touch anything real. Add an agent-action log you can replay forensically. Sell it inside a broader DevSecOps-for-AI package to engineering orgs that let agents install and run code. The differentiator is the replay log: when something reaches your systems through an agent, the transcript is your only record, and most teams have none.

5. Make the policy explicit and adjustable, not baked into weights

What happened: Mistral released Shieldstral, a 3B safety classifier you drive by asking it your moderation policy as a plain-language question at inference time. No retraining. The same lesson as the incidents: state the boundary when you need it, in your own words, and let one small model adapt.

Why it matters: a frozen harm taxonomy never fits your product or your audience. Retraining a guardrail model every time you change a feature is slow and expensive, so most teams ship a generic policy that fits nothing well.

How you leverage it: build policy-adaptable moderation for products that change rules often. A small classifier you re-point at a new policy by rewriting a question, covering prompt classification, response moderation and toxicity across text and images on a single 16GB GPU. Marketplaces and user-generated-content products are the buyers: their rules shift with every campaign and every regulator, and a retrain-per-change guardrail is a tax they will pay you to remove.

Where I would start

If you run any autonomous agent today, do these in order. Name the systems it may and may not touch, in the prompt, every time. That one line is the cheapest fix on this list and both labs put it first. Then verify network isolation with a test that fails when the sandbox reaches the outside. Then separate test and production credentials. Then read your dependency install path again and pin everything. Then keep an agent-action log you can search.

None of this is exotic. That is the point. The skills transfer from ordinary systems engineering, which means you do not have to wait for a lab to fix it for you, and you can hire someone to do it this quarter.

If your team is running agents against real systems and has not done the five steps above, that is the engagement I would propose. The details and the primary sources are in my AI Week issue this week.

Sources

Primary:

Reporting and analysis:

Images: the findings above are drawn from the labs’ own postmortems, linked in Primary. No diagram is included here, and the use cases are my own invention for the businesses described.

If I got something wrong, tell me and I will correct it in the next issue.