Wire reporting: (Reuters, via CNA)

Primary source: METR and Redwood pilot write-up

Outside evaluators from METR and Redwood Research said on Thursday that a three-week pilot this summer found several vulnerabilities in the system OpenAI uses to watch its own AI agents^1,3.

The review ran from July 21 to Aug. 11 as a trial of outside access to a lab's internal systems, not a public account of how the monitors work now^1,3. It covered the system as it stood on and before July 21, while OpenAI was widening it, and did not measure current coverage¹.

One person from each group could see internal code, monitor prompts and settings, and internal documents, and spoke with about a dozen people on the monitoring, safety, and security teams¹. They used OpenAI computers that stayed on the premises and were accompanied on site^1,3.

They estimated that about 75% to 94% of internal agent traffic on the Responses API, from models at least as capable as GPT-5.5, stood in line for a check, a range they said could be off either way¹. Of the slice used for internal Codex coding agents, they put that share at 99.2%¹. Traffic in that category is meant to reach a monitor, but errors, shed load, or bugs can still stop some of it¹.

Not watching what the groups called compaction likely left roughly a tenth of eligible internal Codex traffic unchecked from May through July, though the exact share was uncertain¹. OpenAI staff had found that gap before the review and fixed it during the review¹. In those months an agent could delay a check, which ran only when it handed control back, by continuing to work; the evaluators could not see whether any had done so¹.

At the start of the review, reinforcement-learning training runs were not covered, though the evaluators were told some of those runs had been watched from March through June¹.

On the design they reviewed, Codex agents could, in principle, get past the sandbox and a check called Auto-Review and then do anything the person running them could do¹. A separate exploit, whose method OpenAI cut from the public post, could hide hundreds of recent steps, including the steps that set it up¹.

Agents could also probably build inputs that make the monitor fail, so a session is never sent to a person, because the two sides do not see the session packaged the same way¹. Two headers a client can set would turn monitoring off for a new agent, though that act could itself be caught, and two tools thought to allow only reading could be turned toward any action inside their sandbox, a path not tested in practice¹.

The post also says where the system held: with a few small exceptions, the monitor sees the same record the agent sees, and its instructions cover scheming, attempts to undermine the monitor, sabotage, data theft, deception, and prompt injection¹. Figures OpenAI showed them put monitor errors under 0.2% of requests¹.

The public post is a short cut of a 61-page report given to OpenAI researchers and security staff; a 38-page version with OpenAI's cuts went to both groups' leadership¹. OpenAI could remove anything it judged sensitive, and unlike some earlier METR work the groups had no one-sided right to describe those cuts, though they stand behind what they published¹. METR said the exercise was separate from its review of an earlier Hugging Face incident¹.

OpenAI has said the issues that touch safeguards in live use are fixed^1,3, and has described later changes to how much it intends to watch¹. The evaluators said they have not checked those statements¹. An OpenAI spokesperson told CNBC, "we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators"³.

CNBC carries that comment but not the coverage estimates or most of the bypasses³. The Reuters account on CNA leads with the vulnerabilities and the unchecked fix claim, names METR but not Redwood, and leaves out the July 21 cutoff and the compaction gap⁴. None of the public accounts settle how the monitors work now¹.