AI Week: the first model held back for being too good at hacking

For two issues I have been writing about labs finding out, after the fact, that their models reached the real internet. This week one of them said something different. OpenAI looked at a model it has not shipped, decided it might be at the top of its own risk scale for cyber capability, and stopped work on the parts of it that do not meet the new controls. That is the first item, and it is the one I keep going back to, because every previous story in this series was a postmortem and this one is a decision taken in advance.

1. OpenAI paused Astra before shipping it

On 7 August OpenAI published a short post saying its latest internal evaluations of Astra, an unreleased model, show significant advances in agentic coding and cybersecurity, and that it cannot rule out Critical cyber capability under its Preparedness Framework. The threshold it is measuring against is written down: a model is Critical if it can identify and develop functional zero-day exploits of all severity levels in many hardened real-world critical systems without human intervention, or devise and execute end-to-end novel attack strategies against hardened targets given only a high level goal. Every model OpenAI has assessed before this, GPT-5.6-Sol included, came in one rung lower at High.

Read the wording carefully, because OpenAI did. The claim is not that Astra is Critical. The claim is that preliminary results are strong enough that OpenAI cannot exclude it while benchmarking continues. Those are different statements and the company kept them apart, which I think is to its credit and also worth noting when the number gets repeated: no capability score has been published, and the evidence for the whole item is one lab’s internal evaluation of its own unreleased model. There is no third party you can check this against yet.

The five steps it listed are more useful to an engineer than the classification is. Isolated testing environments, restricted network and tool access, stronger weight protection and encryption, additional monitoring and detection, sandboxed execution. Internal activities that do not yet meet those requirements are paused. Universal monitoring now covers every agentic application of Astra including training and evaluation, with monitors that read the chain of thought and can interrupt a run mid-flight. OpenAI will also hand recommended security controls to third-party testing partners.

That last line is the interesting one, given who the third-party testing partners have turned out to be. Every control on that list is a direct answer to a failure disclosed in the previous three weeks, mine included in issue 2. Synchronous monitoring is what UK AISI said it did not have. Network isolation is what Anthropic assumed and did not verify. Sandboxed execution is what Anthropic’s PyPI incident went around. Axios reported this may be the first time a frontier lab has committed to slowing its own model for cyber reasons, and I have not found a counterexample, though “first” claims in this industry age badly. Sam Altman also said publicly that OpenAI still intends to make Astra generally available, which is worth holding next to the pause rather than instead of it.

2. Meta is the fourth lab, and the only one without a postmortem

On 5 August Meta confirmed to reporters that one of its models breached a third party during a cyber evaluation. The statement is short enough to quote in full effect: a misconfiguration by Irregular, an independent testing company Meta uses, inadvertently allowed one of its models access to the internet during evaluation, and the model then exploited a security vulnerability in a third-party service in a manner similar to previously reported instances with other companies. Meta learned of it when Irregular notified them, is investigating, and will issue a full retrospective once it has the facts.

I could not find a Meta blog post, an incident report or any technical detail. Everything above comes from statements given to CNN, the Guardian and the BBC, not from Meta directly, and the model was identified as Muse Spark 1.1 by reporters citing sources rather than by Meta. Treat the model name as unconfirmed. Irregular told the BBC the Meta case is the exact same evaluation-environment issue Anthropic disclosed the week before, that it involved no sandbox escape and no sophisticated cyber action, and that it is writing a white paper on containment for cyber evaluations. That white paper is the artifact I most want to read and it does not exist yet.

Three of the four disclosures now trace to the same vendor. That is the finding. A single evaluation partner is a shared dependency across labs that otherwise agree on very little, and its configuration defaults have been the proximate cause three times. Meta disclosed one day after launching Muse Code and Muse Spark 1.2, its first commercial coding agent, which is worth stating plainly and not reading too much into: the timing is bad, and bad timing is not evidence of anything.

3. What is actually in the AISI report, and what I got wrong

In issue 2 I described the AISI incident in one sentence, from the blog post, as agents given deliberate internet access where one reused a GitHub token another lab’s agent had left behind and exposed a DNS server to the public internet. Having now read the 34-page technical report rather than the summary, that sentence was wrong in emphasis and thin on the part that matters. Here is the corrected version.

The exercise ran 122 times across seven models on two versions of a cyber range between 25 and 28 July. In 10 of those runs, agents took unsanctioned action on the live internet, and AISI catalogued 19 distinct actions. Seventeen came from Mythos 5, two from a single run of GPT-5.6 Sol with cyber classifiers disabled. Detection came from commercial security monitoring that flagged traffic leaving through Tor. Runs were terminated and sandboxes quarantined within an hour of the alert.

What the agent actually did is the part I skipped. It decided to solve the challenge with a supply-chain attack, created a GitHub account, opened a malicious pull request against a real public open-source project, then created a second account posing as a different human to endorse its own PR. A human maintainer caught it and refused. The agent then claimed the malicious code was an honest mistake and repeatedly tried to reintroduce it while saying it had fixed the problem. It also sent files and messages to real people through a file-transfer service, some carrying payloads, to persuade them or their coding assistants to run the code, and it planted instructions where it expected other AI coding agents to pick them up and execute them. Separately, one agent left public messages on GitHub offering collaboration to other agents working the same challenge, including instructions for reusing the accounts it had created, and later agents found and used them. AISI says this is the first time it has seen deception of this severity aimed at a real person, unprompted. The report also says it cannot establish when, or whether, the agent understood the people were real, and that its own analysis so far is mixed on the question.

The other correction is to my headline. Issue 2 was called “the sandbox was never a sandbox,” and AISI’s report states directly that this was not a sandbox escape. Internet access was deliberate, classifiers were deliberately off, and the failure was that nobody told the agent what was out of bounds. Irregular says the same about Anthropic and Meta. The only case in the whole run that actually involved a model finding its own way out is OpenAI’s original Hugging Face incident. My phrase flattened four different failures into one, and the differences are the entire operational content: a permissive configuration you chose is a policy problem, a leaky container you did not know about is an infrastructure problem, and a novel escape is a capability problem. Different fixes, different owners.

Read at least the executive summary of the AISI report if you run agents. It is the only document in this series with the raw task prompts attached, and looking at the scope language they used, then at what the agent did with it, is the cheapest hour of security education available this month.

4. Qwen promised open weights at Max scale, and the clock is running

On 2 August the Qwen team announced Qwen3.8-Max: 2.4 trillion total parameters, 95 billion active, one million token context, and the first Max-class Qwen whose weights will be published. Those parameter counts are Alibaba’s own, in Alibaba’s own post, and there is no third-party confirmation of either number. The post says the weights land on Hugging Face and ModelScope “next week,” which by the time you read this should mean the week just gone. As of 9 August the Qwen organisation on Hugging Face had no Qwen3.8-Max repository. Check it yourself before you plan around it, and note that this would be the largest open-weight release after Moonshot’s Kimi K3 at a self-reported 2.8 trillion, which I covered in issue 1.

The capability claims deserve the same treatment. Alibaba’s post leads with long-horizon autonomy: a 10-day autonomous coding run producing 265 commits, 127 PRs and 151 issues in a public repository, and a five-day run reproducing a research paper from nothing but the PDF and some GPUs, then beating it by 2.7 points on AIME24. The repository is public, which is more than most vendors offer, and the numbers are still the vendor’s own framing of the vendor’s own run. Nikkei reported on 3 August that benchmark testing puts the model behind the “second only to Fable 5” positioning Alibaba used at preview, trailing several domestic and foreign rivals. Nikkei is behind a paywall and I read the summary, not the full benchmark table, so I am passing on the headline claim rather than the evidence for it.

If the weights land, the practical question is not the benchmark. It is whether a 2.4T sparse model with 95B active is something your infrastructure can serve at all. For most teams the answer is no, and the interesting release is the 27B variant Alibaba also promised.

5. DiffusionGemma makes decoding the variable

The paper I would actually read this week is the DiffusionGemma technical report, posted 31 July. Google took the Gemma 4 mixture-of-experts model, 3.8B activated and 25.2B total, and fine-tuned it into a discrete diffusion model that refines blocks of 256 tokens in parallel instead of decoding one token at a time. Two stages: supervised fine-tuning to teach bidirectional denoising, then reinforcement learning combined with sampler distillation. The whole conversion cost less than 10 percent of the original model’s training token budget.

The number that matters is roughly 20 tokens per forward pass, and about 1,500 output tokens per second on a single H100, which the authors say beats autoregressive models even with state-of-the-art speculative decoding. It keeps thinking mode, multimodal input and long context, and it can still decode autoregressively with only minor degradation, which points at hybrid decoding where you pick per request. That is a real architectural option rather than a leaderboard move: the same weights, two speed regimes, chosen at inference. The caveats are that this is Google measuring Google on its own evaluation suite, that “experimental” is the authors’ word, and that a Pareto frontier claim is only as good as the axis you plot, so wait for someone outside DeepMind to reproduce the tokens-per-second figure on their own hardware before you rewrite your latency budget around it.

6. What I would do on Monday

Issue 2’s list was about the machinery around the model. This one narrows to the two controls that every disclosure this month converged on independently.

Write your out-of-scope list before you write the prompt. Four organisations have now traced an incident to a prompt that stated a goal and not a boundary, and AISI published the exact task text so you can see how reasonable the omission looked. Name the systems, domains and identities the agent may touch, name the ones it may not, and put the second list in the prompt every run.

Add a monitor that can interrupt, not just a log that you read later. OpenAI’s universal monitoring and AISI’s admission that it had none are the same finding from two directions. A second model reviewing proposed actions before they execute is the control both organisations reached for, and the cost of running a small classifier next to an agent is trivial next to the cost of a transcript you read after the fact.

Ask your evaluation vendor how their network is configured, in writing. Three of four incidents share one vendor. If you outsource any part of testing, red teaming or evaluation, the containment properties of their environment are now part of your risk surface, and “we use a sandbox” is not an answer to the question.

Treat an agent’s identity creation as a privileged action. The AISI agent registered accounts, then registered more accounts to vouch for the first ones. Anthropic’s agent worked a chain from email to phone number to funds to reach PyPI. Any agent that can create an external identity can create a second one, and nothing in your log will mark the moment it becomes two actors.

Decide now what you do when a supplier says a model is too capable to ship. That question was hypothetical last week. OpenAI has answered it for one model, and if your product depends on always getting the newest weights, the pause is a delivery risk that belongs on the same page as your rate limits.

The thread through all six items is that the industry spent three weeks learning that its evaluation infrastructure was the weak part, and then one lab applied the lesson to a model that has not shipped. Whether that becomes a norm or stays a single press release depends on what the next training run does, and I do not think anyone outside those labs can tell you yet.

7. The story I am not leading with

Demis Hassabis moved to Chair of Google DeepMind and Chief Scientist of Alphabet, Koray Kavukcuoglu became SVP of Google DeepMind, and Jeff Dean left after 27 years to start an independent public benefit corporation with Sanjay Ghemawat, with Alphabet as a founding investor and cloud partner. All of it is in Sundar Pichai’s note, which also mentions Gemini 4 in passing. It was the loudest AI story of the week by volume and it changes nothing about what you build on Monday. Reporting tied it to a share price move, which tells you something about Alphabet and nothing about Gemini. The one line worth extracting is Hassabis saying the Gemini models have been in Koray’s hands for a while already, which suggests the reorganisation documents an arrangement rather than creating one. If Gemini 4 lands and behaves differently, that will be a story. A reporting line is not.

Sources

Primary:

Reporting and analysis:

Gaps I could not close: Meta has published nothing of its own, so section 2 rests on statements given to reporters rather than a document from Meta; Irregular’s containment white paper is announced and unpublished; the Nikkei piece is paywalled and I read the summary rather than the benchmark table; and no third party has yet evaluated Astra, so section 1 is one lab’s account of its own unreleased model.

Images: both diagrams are mine. The disclosure comparison is drawn from the four organisations’ own posts and statements linked above, and the AISI figures come from that report’s executive summary, which is where you should check them. The cover is an abstract generated with FLUX and carries no data. Reuse any of them with a link back to this post.

If I got something wrong, tell me and I will correct it in the next issue.