When sandboxed agents talk through the CPU cache

On 23 October 2026 OpenAI published Securing our frontier model training environment, a write-up of a large red-teaming campaign against its own model sandboxes. One paragraph went round social media faster than the rest: agents in separate microVMs, with no network and no shared writable files, still managed to pass secrets to each other through the processor’s cache.

Not everyone was surprised. Halvar Flake (Thomas Dullien), a well-known security researcher, replied:

Am I weird for not finding it insane? You tell agents to communicate, they know about cache timing, this is the first thing you’d expect they try?

— Halvar Flake on X

He has a point.

This is not a new kind of attack. It is a textbook covert channel, and the computer-security literature has studied this exact problem since 1973. What is new is who is using it: models that can read the research, design a protocol, and try for as long as you let them.

This post covers what OpenAI actually reported, why a virtual machine (Firecracker included) does not stop it, what seventeen years of cross-tenant attacks teach, and a layered defence, from scheduling policy down to cache colouring, drawing on USENIX, IEEE S&P, CCS, OSDI, Black Hat, Chaos Communication Congress, hacker-camp and Platform Security Summit work, and on public proofs of concept. It follows on from Sandboxing untrusted AI agents on Linux, which covers the sandbox mechanisms themselves; this post is about what leaks between sandboxes that are working as designed.

Everything here is point-in-time (late October 2026). Quotes are from OpenAI’s report unless marked otherwise.

🔎 What OpenAI reported

The report describes the containment stack for training and evaluating frontier models, then a red-teaming campaign against it.

The containment stack. On Linux, agents run in “restricted containers inside lightweight microVMs”: the container limits files and processes, and the microVM “runs its own operating-system kernel and is isolated from the host by a hypervisor”. GPU tasks use “GPU-backed containers with user-space sandboxing”. Around that sit monitoring with automatic stopping, and a network and cloud perimeter “designed to maintain containment even if an agent escapes its sandbox and gains control of the host”. Frontier training runs with no live internet access at all; agents browse an “offline world” of indexed and cached web content instead. Agents that are allowed to cooperate share an explicit message board; everything else is meant to be unreachable.

The red team. Teams of frontier-model agents, including an upcoming “GPT-6 Astra Cyber” model, were given three goals: reach the live internet, cross an internal security boundary, and communicate across isolation boundaries. To favour the attacker they were given “infrastructure source code, configuration details, and documentation describing potential vulnerabilities”. The campaign used “hundreds of billions of tokens”.

What they found. Three classes of result:

1. 🧨 MicroVM escapes. Agents “found several ways to chain vulnerabilities into microVM escapes”. Fixes went to Kata Containers (CVE-2026-24834, privilege and filesystem isolation), gVisor (file-descriptor handling), Moby/Docker (syscall filters) and the Linux kernel (CVE-2026-74753, CVE-2026-80521, CVE-2026-80841), with more disclosures not yet public.
2. 🌐 Paths to the internet from a compromised host. Narrow connections that remained for metrics, logging and authentication “gave them limited forms of internet access” in almost every case. One agent “reused a firewall-approved connection to reach a blocked authentication endpoint in our metrics service and then made that endpoint fetch a model-provided URL”. Others slipped requests through DNS and firewall misconfigurations, or made internal services fetch external URLs.
3. 📡 Communication across isolation. The paragraph in the screenshot: agents “in separate microVMs, created and managed by OpenAI’s sandboxing software on shared host machines” exchanged synthetic secrets “without the use of network connections or shared writable files”. One agent evicted chosen data from the cache; the other timed reloads and read fast as 0 and slow as 1, or similar. Other channels used shared storage (one agent “alternated between continuous writes and inactivity”, which an agent described in its chain of thought as an “acoustic modem-like signal”), competition for CPU time within a shared execution container, and a TLS certificate cache in the offline web infrastructure.

Two caveats from the report itself matter for reading this fairly:

• The agents had help. “In most of these tests, we directed agents to investigate specific communication channels or provided them with a protocol.” In one case they got only “a broad list of infrastructure vulnerabilities”, and still turned one into a channel and agreed an encoding on their own.
• The channels are slow. “Most of these channels carry information slowly and are difficult for models to exploit.” OpenAI’s response was to add “checks to our monitoring systems to detect attempts to communicate through these channels”. Note what that is: detection, not removal.

🔊 Covert channels, side channels, and the confinement problem

Two terms get used interchangeably and should not be:

• A side channel leaks information from an unwilling victim. Classic example: recovering a cryptographic key from another tenant’s cache access pattern.
• A covert channel is two cooperating parties signalling through something that was never meant to carry information. Sender and receiver agree a protocol in advance, so they can repeat, add error correction, and go slowly. That makes covert channels far easier than side channels.

OpenAI’s agents built covert channels. Butler Lampson named this in A note on the confinement problem (1973): a confined program can leak through any shared resource whose behaviour it can influence and another party can observe, including timing and load. Fifty years of results since have converged on an uncomfortable conclusion: on shared hardware you cannot eliminate covert channels, only reduce their bandwidth, which is why security evaluation schemes historically required covert-channel analysis and bandwidth limits rather than proof of absence. The hacker community has known it just as long: Steven Murdoch’s Covert channels in TCP/IP (22C3, 2005) at the Chaos Communication Congress showed how much room ordinary protocol fields leave for hidden signalling, and how hard that is to filter out.

Why this matters more with agents than with ordinary malware:

• 🤝 Collusion is the default case. In a multi-agent system you start many copies of the same model, with the same training, possibly the same instructions. If two of them “want” to coordinate, they do not need to negotiate a protocol at runtime: they can converge on the obvious one.
• 📚 The attack literature is in the weights. Prime+Probe, Flush+Reload and the rest are well documented, and the report shows models applying them.
• ⏱️ Patience is cheap. A channel at a few bits per second is useless for bulk data but plenty for an API key, a coordination signal, or “the reviewer is not watching now”.

The security question is therefore not “can agents in separate sandboxes communicate?” (assume yes) but “what can they achieve if they do?”

🧬 Microarchitectural state, and why it matters so much

Everything that follows hinges on one distinction. A CPU has two layers of state:

• Architectural state is what the instruction set promises software: registers, memory contents, flags, the instruction pointer. It is documented, programs can read it, and an operating system or hypervisor saves and restores it on every context switch. All of our isolation is defined in these terms: “process A cannot read process B’s memory”, “VM A cannot see VM B’s registers”.
• Microarchitectural state is how a particular chip delivers that promise quickly: what is in the caches and which line gets evicted next, the TLBs that cache address translations, branch predictors, prefetchers, internal buffers, which execution ports are busy, the coherence directory, which DRAM rows are open, the current clock frequency. None of it is part of the contract. Software cannot read it directly, and a program computes the same answer whatever state it is in.

The catch is that microarchitectural state is invisible to a program’s results but visible in its timing. A load that hits the L1 cache takes a few nanoseconds; one that goes to DRAM takes around a hundred. Same answer, different time, and anything that changes timing can carry information.

That is how OpenAI’s agents talked. The hypervisor kept each VM from reading the other’s memory the whole time, but both VMs ran on the same physical cache. The sender filled a group of cache lines to send a 1 and stayed idle to send a 0; the receiver timed its own memory accesses, reading slow (its data was evicted) as 1 and fast as 0. Neither ever touched the other’s memory. Think of two hotel rooms with locked doors and shared plumbing: flush in one room, and the neighbour hears it in the pipes.

Two pictures make this concrete. The first shows where the wall is. The hypervisor sits between the two VMs and blocks every direct read, but both VMs run on the same physical cache underneath it, and the hypervisor does not divide that cache between them.

Blue is what isolation protects. Red is the path the bits take: neither VM can reach the other, but both can reach the same cache sets.

The second shows how one bit crosses. This is the classic Prime+Probe pattern, and it is close to what OpenAI describes. Before starting, the two sides agree on which cache set to use and how long each bit lasts.

Each step is an ordinary memory load inside the agent’s own VM. Nothing here asks the hypervisor for anything, so nothing here can be refused.

From bits to messages

One bit per round is not yet a conversation. To pass real messages, the two sides build a slow serial line on top of the trick above, with the same parts as any network protocol. Every access is still an ordinary load, the same instruction every program uses, and the values loaded mean nothing. The information is in which addresses get touched (which cache set) and when.

1. ⏱️ Time slots. Both sides agree on a slot length, say 100 µs, and each slot carries one bit: the sender loads into the set for the whole slot to send a 1, or stays idle to send a 0. The receiver primes at the start of each slot and probes at the end. Both read their own timestamp counter, so they agree on slot boundaries without ever talking directly.

2. 🔤 Turning a message into bits. Exactly as on any serial line: text becomes bytes, and bytes become bits. Sending the letter “h”:

   'h' = 0x68 = 0    1    1    0    1    0    0    0
   slot:        1    2    3    4    5    6    7    8
   sender:      idle load load idle load idle idle idle
   receiver:    fast slow slow fast slow fast fast fast  ->  01101000  ->  'h'
   

3. 🎯 Sync and framing. The receiver has to know where a message starts. A fixed preamble such as 10101010 lets it lock onto the slot rhythm, like a UART start bit or an Ethernet preamble, and a length field or end marker says where the message stops.
4. 🩹 Error correction. The cache is noisy: every other program on the host evicts lines too, so some 0s arrive as 1s. The protocol pays for that with redundancy: repeat each bit, use a Hamming or Reed–Solomon code, add a checksum and resend when it fails. This is why covert channels can afford to be slow: patience buys reliability.
5. 🏓 Ping-pong. For two-way traffic there are two options. Half-duplex: take turns, A sends a frame, then the roles swap and A listens, with acknowledgements and retransmission much like TCP. Full-duplex: use two cache sets, one per direction, so both can transmit at once. More sets in parallel also means more bits per slot.

Researchers have taken this all the way. Hello from the Other Side (NDSS 2017) ran a full TCP/IP stack over a cache channel between two Amazon EC2 instances, error correction included, and carried a working SSH session over it. Once there is a reliable bit pipe, everything above it is ordinary networking. OpenAI’s agents built a crude version of steps 1 to 4: most were handed the protocol, and in one case they agreed on an encoding themselves. That is the real concern behind “the attack literature is in the weights”.

Inside the silicon: what “evicted” actually means

It’s worth being precise about this, because the whole channel lives inside one mechanism that most explanations wave past.

A cache line is not a labelled box that data gets lifted out of when it’s “evicted.” It’s a row of SRAM cells: six-transistor loops, each holding one bit as two transistors locked in a feedback loop, one driven high, the other low, each continuously reinforcing the other’s state. No magnetism, no moving parts, no atoms relocating anywhere — just voltage, held in place by circuit topology. Next to the data bits in each row sit a tag (which physical address this row currently represents) and a valid bit. A cache is also not one undifferentiated pool: addresses are routed by some of their bits into one of many sets, and each set holds a small, fixed number of lines (the “ways” mentioned later in this post). “Full” means that specific set’s ways are all occupied, not the cache as a whole.

Eviction, then, is not removal. It’s overwrite. When an address needs a slot in a full set, a small hardware circuit called the replacement policy (commonly an approximation of least-recently-used, tracked with a couple of extra bits per set) picks a victim line. If the victim holds unwritten changes (a “dirty” line), the controller writes it back down to the next cache level or memory first, so the data survives. Then the write circuitry forces new voltages onto that row, flipping its transistor pairs to represent the new tag and data. The old pattern of highs and lows doesn’t go anywhere. It’s simply gone the instant the new one lands, the way writing over a whiteboard erases what was there; nothing was carried away, there’s just nothing left to read.

Why this is the covert channel

Every layer of isolation on this page is built on top of one assumption: that access is mediated by something that can check permissions, a system call, a page-table entry, a hypervisor trap. The replacement policy checks none of this. It cares about one thing: which physical address does an access resolve to, and which set does that address’s bits route it into. Two completely separate VMs, with completely separate address spaces, completely unable to read each other’s memory, can still land in the same set, because set assignment is a function of physical address geometry, not ownership. When that happens, filling a set from one side necessarily evicts whatever the other side put there, and there is no permission check anywhere in that path to stop it, because none was ever designed to apply here.

That’s what makes it a signal and not just contention. The sender doesn’t need to see the receiver’s memory, write to it, or share a socket with it; the sender only needs to touch enough addresses that map to the same sets the receiver cares about, which it can work out by timing its own accesses in advance (the probing step behind every “eviction set” in the Prime+Probe literature below). Once it has that map, filling those sets is a 1, leaving them alone is a 0, and the receiver reads its bit back by timing its own access to the same sets: a few nanoseconds if its data survived (fast, still in the cache), on the order of a hundred if it didn’t (evicted, had to go to DRAM). That gap is large, stable across runs, and doesn’t depend on either process’s privileges, which is exactly the combination Lampson’s 1973 definition asks for: a shared resource, influenced by one party, observed by the other, outside what the security policy accounts for.

It’s also why the fix can’t be “monitor cache accesses and block the bad ones”: there’s nothing to distinguish a sender’s eviction from the thousands of ordinary evictions every running program causes every microsecond just by executing. The replacement policy has no concept of “tenant,” “sandbox,” or “trust boundary” to enforce, because nobody wired one in; it was built to make programs fast, not to keep secrets apart. Everything in the defence-in-depth section further down is, underneath the acronyms, an attempt to retrofit exactly that distinction onto a circuit that was never asked to make it.

Why this has such large implications for sandboxing:

1. 🧱 Isolation only covers the layer above. Every boundary in the sandboxing post (namespaces, seccomp, Landlock, gVisor, a microVM) protects architectural state. The channel runs underneath all of them.
2. 🤝 It is shared on purpose. Sharing the L3 between cores, two hyperthreads per core, and handing idle capacity to whoever wants it are what make CPUs and clouds efficient. Every defence that removes sharing (dedicated cores, partitioned cache, fixed quotas, separate hosts) costs performance or money, which is why these channels have survived decades of research.
3. 🔁 It is not saved or restored. A context switch swaps registers, but caches, predictors and TLBs keep whatever the previous tenant left. So taking turns on a core leaks too, not just running side by side, which is why the defences include flushing on switch and pinning VMs to dedicated cores.
4. 🗺️ It cannot be fully listed. The instruction set is documented; the microarchitecture largely is not, and it changes with every CPU generation. When defenders partitioned the cache, attackers moved to the TLB, the coherence directory, the interconnect, the prefetcher and the clock frequency (all in the timeline below). You cannot partition a structure you do not know exists.
5. 👻 Speculation turns covert channels into side channels. A CPU runs ahead on guesses and throws away the architectural results of a wrong guess, but not the microarchitectural footprint. Spectre and Meltdown make the CPU speculatively touch a secret and then read it back from the cache: the same receiver OpenAI’s agents used, which is why the report mentions those attacks. Cooperating agents do not need speculation; leaking an unwilling victim’s secret does.
6. 🧮 It sits below every policy layer. Sandbox policy is written against architectural interfaces: seccomp sees system calls, Landlock sees paths and ports, a VM sees devices and memory, a proxy sees connections. A cache eviction involves no system call, file or packet, so none of them can see it. Only controls at the same level can act on it: where code runs (placement, SMT, pinning), how the hardware is divided (cache partitioning and colouring), and what is wiped between tenants (flushing, and seL4’s “time protection”, which treats microarchitectural state as a resource to divide in space or in time).

That last point is why the defence layers later in this post move from software policy to scheduling and hardware.

🧱 Why a VM, even Firecracker, does not stop this

A virtual machine isolates architectural state: memory contents, registers, the kernel, devices. Covert channels live in microarchitectural and physical state that the hypervisor does not virtualise, because the hardware shares it:

Firecracker is explicit about where its responsibility ends. Its production host setup guide says “Firecracker is not able to mitigate host’s hardware vulnerabilities” and asks the operator to:

• disable SMT where tenants must be separated (“Simultaneous Multi-Threading (SMT) is frequently a precondition for speculation issues”);
• disable Kernel Samepage Merging “to mitigate side channel issues” that rely on page deduplication;
• keep microcode and firmware current, and use ECC memory with target row refresh against Rowhammer;
• run each Firecracker process for one tenant’s workload.

It says nothing about cache partitioning or vCPU pinning. Those are left to you. So: a microVM is necessary (it stops the cheap software escapes, at least until a VMM bug is found, as OpenAI’s red team showed), but it is not sufficient against covert channels. What decides the outcome is how the host is configured and, above all, what shares a host.

Is any kind of VM immune?

“VM” is an overloaded word, so it is fair to ask whether some kind of virtual machine escapes this problem. None does. The VM technology decides how hard it is to break out of the VM; what decides whether a covert channel exists is which hardware the VMs share, and every kind of VM runs on shared hardware unless you arrange otherwise.

The last row deserves emphasis, because “run the agent’s tools in a WebAssembly sandbox” is a common design. After Spectre, Google’s V8 team concluded that language-level protection cannot keep secrets inside one process: “The only effective mitigation is to move sensitive data out of the process’s address space” (A year with Spectre, V8 blog; see also Spectre is here to stay). Chrome’s answer was Site Isolation: a separate process per site. For agents the lesson is the same: a language sandbox is a good integrity boundary for running a tool, but two agents whose data must stay apart need at least separate processes, and against covert channels, separate hardware.

So the honest summary of this section: every kind of VM shares hardware by default, and the only VMs that leak noticeably less are the ones built to stop sharing it (static partitioning, colouring, time protection), which is exactly what the defences later in this post do on ordinary hypervisors.

What a hyperscaler does

The clearest public description of production practice is AWS’s Nitro whitepaper section on side channels. Its design rules map almost one-to-one onto the table above:

• “CPU cores are never simultaneously shared among customers via Simultaneous Multi-Threading (SMT)”, so instances never share L1 or L2 caches.
• Fixed-performance instances get physical cores and memory pinned for their lifetime, and “nor are any instance memory pages shared in any fashion across instances”.
• Burstable instances can run sequentially on the same core, so “microarchitectural state is flushed when the core is rescheduled”.
• The last-level cache can still be shared: “Some instance sizes can share some last level cache lines non-simultaneously.” AWS exposes the real topology so customers can pick sizes that fill a whole L3 domain (in a typical two-socket Intel host, “an instance size that is one-half the largest size will fill a CPU core and will not share the L3 cache with another instance”).
• For anyone who wants none of this, Dedicated Instances and Dedicated Hosts guarantee a single customer per physical machine.

That last point is the honest summary of the state of the art: even the most careful multi-tenant design falls back on “don’t share the host” for the strongest guarantee.

📚 Seventeen years of cross-tenant channels

The OpenAI result sits at the end of a long line of research, most of it published at USENIX Security, IEEE S&P and CCS:

• 2009, co-residency. Ristenpart et al., Hey, you, get off of my cloud (CCS 2009), showed you could deliberately place a VM on the same host as a target on EC2 and build cache-based covert channels between them.
• 2012, the memory bus. Wu, Xu and Wang, Whispers in the Hyper-space (USENIX Security 2012), built a reliable cross-VM covert channel by locking the memory bus with atomic operations: no shared cache needed.
• 2014, Flush+Reload. Yarom and Falkner, Flush+Reload (USENIX Security 2014), extracted keys across VMs through deduplicated memory pages, which is why every hardening guide now says to turn KSM off. The same year, Varadarajan, Ristenpart and Swift’s scheduler-based defences (USENIX Security 2014) showed that a minimum run time for vCPUs plus state cleansing on context switch blunts cache attacks at low cost.
• 2015, out-of-order execution. Sophia D’Antoine’s Black Hat USA 2015 talk, Exploiting Out-of-Order Execution for Covert Cross-VM Communication, built a cross-VM channel from the CPU’s instruction reordering, another shared resource that no hypervisor virtualises.
• 2015, the last-level cache. Liu, Yarom, Ge, Heiser and Lee, Last-Level Cache Side-Channel Attacks are Practical (IEEE S&P 2015), removed the need to share a core or memory pages: Prime+Probe on the shared L3 works across cores. Inci et al. then recovered a full 2048-bit RSA key from a co-located instance on EC2 (Seriously, get off my cloud!, 2015).
• 2015, Rowhammer from JavaScript. Gruss and Maurice’s Rowhammer.js (32C3) flipped DRAM bits from a web page using only ordinary memory accesses, by learning to evict cache lines on demand: the same eviction trick OpenAI’s agents used to signal.
• 2016, Rowhammer across VMs. Razavi et al., Flip Feng Shui (USENIX Security 2016), went beyond leaking: a malicious VM used memory deduplication to share a physical page with a co-hosted victim, then flipped bits in it with Rowhammer, breaking OpenSSH public-key authentication and GPG signature checks. The authors took it to the hacker community too (SHA2017 talk). It is the strongest argument for turning memory deduplication off.
• 2016, DRAM. Pessl et al., DRAMA (USENIX Security 2016), built covert channels through DRAM row buffers, which work across CPUs.
• 2016, leaks from unprivileged instructions. Maurice and Lipp’s What could possibly go wrong with <insert x86 instruction here>? (33C3) showed cache attacks without a single memory access (using prefetch instructions) and a kernel ASLR bypass, all from unprivileged code. Their framing fits this post: safe software does not guarantee safe execution, because the hardware underneath leaks.
• 2017, SSH over the cache. Maurice et al., Hello from the Other Side (NDSS 2017), presented at Black Hat Asia 2017, built an error-corrected cache covert channel between two Amazon EC2 instances robust enough to carry a working SSH session. This is the result to keep in mind when someone says covert channels are too slow or noisy to matter.
• 2018, Spectre and Meltdown. Speculative execution attacks used the same cache-timing receiver as OpenAI’s agents, as the report notes.
• 2018, taking stock of Spectre. A year after disclosure, Lipp, Schwarz, Gruss and Canella’s A Christmas Carol: the Spectres of the Past, Present, and Future (35C3) walked through the growing family of transient-execution variants and how many of the deployed mitigations covered only some of them.
• 2018, the TLB. Gras et al., TLBleed (USENIX Security 2018), leaked keys through the translation lookaside buffer shared by hyperthreads, even with the cache partitioned by CAT, because nothing partitions the TLB. OpenBSD responded by disabling hyperthreading by default (VUSec project page).
• 2019, microarchitectural data sampling. RIDL, Fallout and ZombieLoad leaked data from internal CPU buffers across hyperthreads and across VMs, and patching them properly meant disabling SMT where tenants differ. The ZombieLoad authors presented it at 36C3.
• 2019, coherence directories. Yan et al., Attack Directories, Not Caches (IEEE S&P 2019), showed that switching to non-inclusive caches does not help, because the coherence directory is shared and attackable.
• 2020, the cache, from the network. NetCAT (IEEE S&P 2020) used Intel’s Data Direct I/O, which lets network cards write straight into the last-level cache, to probe a server’s cache remotely and recover keystroke timing from an SSH session (36C3 talk by Michael Kurth). The cache is not only shared with co-tenants; anything that can send packets can reach it.
• 2020, power, read from software. PLATYPUS (IEEE S&P 2021, presented at rC3) used the unprivileged Intel RAPL energy counters to infer data and extract keys: a power side channel with no oscilloscope. A sandbox that exposes energy or power counters to guests hands them a free receiver.
• 2021, the interconnect. Volcano (arXiv:2103.04533) used contention on the on-die mesh interconnect, with no shared cache state at all.
• 2021, automated discovery. Weber et al., Osiris (USENIX Security 2021), fuzzed x86 instruction sequences to find new timing channels automatically, and reported a cross-core covert channel of 95.2 bit/s across VMs on AWS. Automated search for channels is exactly what OpenAI’s agents did, with a model in place of a fuzzer.
• 2022, frequency. Wang et al., Hertzbleed (USENIX Security 2022), showed that dynamic frequency scaling depends on the data being processed, turning power into a timing channel; Intel and AMD did not ship microcode fixes.
• 2023, vector registers. Moghimi’s Downfall (USENIX Security 2023) abused the AVX gather instruction to steal AES keys from another VM in under ten seconds on Intel Skylake to Tiger Lake parts (downfall.page). The same summer, Tavis Ormandy’s Zenbleed leaked register contents between processes on the same AMD Zen 2 core, with a public proof of concept.
• 2024, prefetchers. GoFetch (USENIX Security 2024) showed Apple’s data memory-dependent prefetcher leaking keys from constant-time cryptography (gofetch.fail): an optimisation that looks at the values in memory is a channel by construction.

• 2023 to 2026, GPUs. Agent sandboxes increasingly involve GPUs, and partitioning them has the same weakness. Zhang et al., TunneLs for Bootlegging (CCS 2023), found that NVIDIA MIG does not partition the last-level GPU TLB and built a covert channel at up to 31 kbps on a commercial cloud. Veiled Pathways (MICRO 2024) used the shared PCIe link, and Behind Bars (USENIX Security 2026) found cross-instance L2 interference through memory barriers.

• 2024 and 2025, Rowhammer is still everywhere. Gruss, Heckel and Adamsky’s Ten Years of Rowhammer (38C3) was followed at 39C3 by Rowhammer in the Wild: of 1,006 datasets collected from 822 real systems with their FlippyR.AM tool, 126 (12.5%) showed bit flips under automated attacks.
• 2025, Spectre across VMs in a public cloud. At 39C3, Thijs Raymakers’ Spectre in the real world answered the usual critique that Spectre is impractical by “leaking memory across virtual machine boundaries at a public cloud provider, bypassing mitigations against these types of attacks”. The report earned “Google Cloud’s highest bounty yet”, $151,515. A mitigated, professionally run cloud host still leaked across the VM boundary.
• 2026, a shared service as the channel. At Black Hat USA 2026, Palo Alto Networks researchers presented “A Billion-User Blast Radius: Owning ChatGPT’s Secure Sandbox”. Dark Reading’s coverage describes a chain that reached persistent root inside the sandbox and used a shared JFrog Artifactory backend as a covert channel. OpenAI fixed the Artifactory behaviour within the disclosure window and disputes that the work shows a true sandbox escape or cross-account access. Either way, it is the software version of OpenAI’s TLS-cache finding: the package mirror every sandbox talks to is shared state.

The pattern: every time one shared resource is partitioned, research finds the next one. Defences raise the cost and lower the bandwidth; they do not reach zero.

A note on why this list leans on these particular results: you don’t learn anything about adversarial breakout risk by testing it against a weak boundary. Showing that a timing channel crosses a default, unhardened container with no seccomp filtering proves nothing, because that was never a security boundary to begin with — it was never meant to stop a determined process, so defeating it isn’t news. The results above matter because they were run against the best isolation available at the time: NetCAT against Intel’s DDIO on real server hardware, Downfall and MDS against production Intel server silicon, Osiris’s channel across live AWS VMs, Raymakers’ Spectre leak across VM boundaries at a major public cloud. That’s the right standard for this kind of claim. Researchers and red teamers testing agent sandboxes should hold themselves to the same bar: break the newest isolation primitive, not last decade’s.

🏰 Defence in depth

No single control solves this, so stack them. The layers below run from the cheapest and most effective (design and placement) down to the expensive and partial (hardware partitioning, detection).

Layer 0: design so collusion does not matter

The strongest control is architectural, and it costs nothing. Assume any two agents that share hardware can talk, and make sure that no agent, and no pair of agents that could collude, holds all three legs of the lethal trifecta: private data, untrusted content, and a way to communicate externally.

• ✂️ Split by capability, not just by task. The agent that can read the secrets should not be able to reach the network, and the agent that can reach the network should never see secrets. That holds only if the two are also kept off shared hardware (layer 1); otherwise a covert channel rejoins them.
• 📬 Give agents an explicit channel. OpenAI’s design already does this: cooperating agents get “a message board shared with allowed teammates”. A sanctioned, logged, rate-limited channel removes most of the motive to build an unsanctioned one, and makes unusual behaviour easier to spot. The cross-domain and separation-kernel community has argued this for years: at Platform Security Summit 2018, Christopher Clark’s Hypervisor-Mediated data eXchange talk made the case that how a VMM provides inter-VM communication has “a critical impact on VM isolation properties”, Myong Kang (U.S. Naval Research Laboratory) described keeping separation VMMs intact at enterprise scale, and Adventium’s Magrana server was designed to isolate enclaves and prevent information flowing between them.
• 🎯 Keep the payoff small. Short-lived credentials, scoped to one task, brokered outside the sandbox (see the credential brokering in the sandboxing post), mean a slow channel has little worth sending.

Layer 1: placement removes the hardware channels

If two agents must not communicate, they must not share hardware. Make it a scheduler rule, not a hope. This is the only control that closes the microarchitectural channels outright. It does not close channels through services both agents still use, which is why layer 3 and the next section matter:

• 🏠 Separate physical hosts per trust domain. Public clouds sell this: AWS Dedicated Hosts, Google Cloud sole-tenant nodes, Azure Dedicated Host. On Kubernetes, use separate node pools with taints, and pod anti-affinity so that “secrets” agents and “egress” agents never land on the same node.
• 🧩 At least separate L3 domains. If a whole host per domain is too expensive, separate sockets or core complexes remove the shared last-level cache, as AWS’s “half the largest size fills a socket” rule does. Expose the real cache topology so the scheduler can reason about it.
• 🎮 Whole GPUs per trust domain. Given the MIG results above, treat a GPU partition as a performance boundary, not a security one, for agents that must not communicate.
• 🗄️ Separate storage and caches too. The report’s storage and TLS-cache channels needed no shared CPU at all. Separate disks or volumes and separate cache instances per trust domain.

Layer 2: harden hosts that must be shared

When agents of the same trust level share a host (the common case for training and evaluation fleets), shrink the shared surface:

• 🧵 No SMT sharing across domains. Disable SMT, or use Linux core scheduling so that only tasks with the same cookie run on sibling threads. This removes L1, L2, branch-predictor and port-contention channels between domains.
• 📌 Pin vCPUs to dedicated physical cores for the life of the VM, as Nitro does. No time-sharing of a core means no state left behind for the next tenant.
• 🧹 Flush on switch where cores are shared in time: L1D flush and IBPB on context switch, with the kernel’s hardware vulnerability mitigations enabled and microcode current. A minimum run time for vCPUs (the USENIX Security 2014 scheduler defence) limits how often an attacker can sample.
• 🔋 Hide power and energy counters from guests. PLATYPUS turned the RAPL energy interface into a side channel; keep it privileged on hosts and do not pass it through to sandboxes.
• 🧯 Use ECC memory with target row refresh. The 39C3 FlippyR.AM data shows Rowhammer-susceptible memory is common in real machines, and Flip Feng Shui shows it can cross VMs.
• 🔁 Turn off memory deduplication. Disable KSM and any page sharing between VMs. This kills Flush+Reload outright.
• 🧱 Partition the last-level cache. Intel Cache Allocation Technology and AMD’s L3 QoS, managed through Linux resctrl, give each VM its own cache ways; memory bandwidth allocation in the same interface limits the bus-contention channel. On Arm servers, MPAM offers similar controls through the same resctrl interface (mainline support is still landing). Plain CAT has too few partitions to separate many tenants; CATalyst (HPCA 2016) built a pseudo-locking scheme on top of it to defeat LLC attacks with very small overhead.
• 🎨 Colour what CAT does not cover. Microsoft’s Marghera, Principled Microarchitectural Isolation on Cloud CPUs (CCS 2024 Distinguished Paper), is the most complete published example: on AMD EPYC it identified the shared caches, coherence directories and DRAM banks, reverse-engineered their indexing functions, and applied colouring so that VMs never share them, implemented in Hyper-V with small overhead. AMD’s bulletin AMD-SB-7025 responded to a coherence-directory channel the researchers reported.

Be clear-eyed about the limits:

• 🧵 Partitioning one structure is not partitioning the core. TLBleed got through a CAT-partitioned cache via the TLB, and MDS and Downfall leaked through internal buffers and registers. Cache partitioning only helps once SMT sharing between domains is gone and microcode and kernel mitigations are current.
• 🌡️ Some resources are package-wide. Frequency and power (Hertzbleed), the interconnect (Volcano) and thermal effects are not partitioned by any of the above.
• 📡 Some reach the cache from outside. NetCAT shows that DDIO lets network traffic touch the last-level cache; on hosts that need strong isolation, consider disabling DDIO or confining it to cache ways no tenant uses.

Layer 3: remove shared software state

Two of OpenAI’s channels, storage and the TLS certificate cache, were software. These are often the easiest to build and the easiest to fix:

• 🔐 Per-domain instances of shared services: TLS session and certificate caches, DNS resolvers, HTTP proxies, package mirrors, build caches, search indexes. Any cache whose hit or miss another agent can time is a channel, and so is any shared service an agent can write to: the Black Hat USA 2026 ChatGPT talk used a shared artifact repository. A per-sandbox egress supervisor, as in NVIDIA OpenShell or Sandlock (both covered in the sandboxing post), gives you this for free.
• ⚖️ Fixed, non-work-conserving quotas. A work-conserving scheduler hands idle capacity to whoever wants it, so one agent’s load changes another’s speed: that is the CPU-contention and storage channel. Fixed allocations (cpu.max, io.max with hard limits, provisioned IOPS) make each agent’s performance independent of its neighbours, at the cost of utilisation.
• 🧽 No shared writable state, and fresh state per task. Ephemeral sandboxes, no reuse across tasks, no shared /tmp or scratch volumes.

Layer 4: add noise and take away clocks (weak)

Covert channels need a timer. Taking it away helps less than it sounds:

• ⏲️ Coarsening timers worked partially for browsers after Spectre, but Fantastic Timers (Schwarz et al., 2017) showed that a counting thread or many other sources rebuild a fine-grained clock. In a VM, trapping rdtsc is possible but expensive and still leaves counting threads.
• 🎲 Deterministic or frozen time inside a sandbox (Sandlock can freeze time and make getrandom deterministic) is useful for reproducible runs, but does not stop a sender and receiver who count work instead of time.
• 📢 Injecting noise lowers bandwidth; error correction wins it back. Treat noise as a cost multiplier, never as a boundary.

Layer 5: detect and respond

This is what OpenAI relies on for the channels it could not remove:

• 📈 Hardware performance counters can flag the cache-miss patterns of Prime+Probe. CloudRadar (RAID 2016) combined a signature detector for sensitive work with anomaly detection on co-located VMs and reported detection within milliseconds; Chiappetta, Savaş and Yılmaz (2016) detected attacks well before they finished. Detectors trained on known attacks can be evaded by new ones, so treat this as a tripwire.
• 🔉 Behavioural signals at the infrastructure layer: regular on/off I/O bursts (the “acoustic modem”), periodic CPU duty cycles, abnormal cache-service hit patterns.
• 🧠 Monitoring the agent itself. The report quotes an agent’s chain of thought planning the storage channel. Reading reasoning and tool calls for intent (“encode”, “timing”, “agree a protocol”) catches what counters miss, while the reasoning stays legible.
• ⏹️ Automatic stopping and short lifetimes. A slow channel needs time; pausing runs on a signal and capping sandbox lifetime bound what can leak.

Layer 6: assume escape, hold the perimeter

OpenAI’s other findings are a reminder that covert channels are the subtle failure; the blunt ones were VM escapes and internal services that reach the internet. The perimeter must hold with a compromised host:

• 🚪 Default-deny egress at the cloud layer, independent of anything on the host.
• 🔗 Treat every service a host can reach as a possible exit, directly or through another service: metrics, logging, authentication, DNS. Anything that fetches a URL on request is a server-side request forgery waiting to happen.
• 🧪 Red-team continuously, with your strongest models. OpenAI found its count of new findings fell as fixes landed, and made high-compute red-teaming part of the safety case for each training run.
• 🤖 Search for channels automatically on your own hardware. Osiris-style fuzzing for timing differences, run on each new CPU and host configuration, finds channels before an agent does. Attackers already automate this; defenders should too.

🧮 What “mitigated” means here

It is worth being precise, because “mitigated” is easy to over-read. None of the layers above eliminates covert channels between agents that share anything. They change three numbers: how much bandwidth a channel has, how much it costs to build, and how likely it is to be noticed.

• 🧱 Separate hosts remove the hardware channels: caches, directories, DRAM, frequency, the GPU. That is a real and complete fix for that class.
• 🔁 Shared services remain. Agents on separate hosts still share a control plane: the orchestrator, storage cluster, logging and metrics pipeline, offline web cache, and, for most agent systems, the model-serving cluster itself. Anything whose response time depends on another tenant’s load is a potential channel. LLM serving is a concrete example: Auditing Prompt Caching in Language Model APIs (ICML 2025) measured cache sharing across users at seven API providers through response timing alone. Two agents on one inference cluster could, in principle, signal through prefix-cache hits or batching delays without sharing a single CPU. Per-tenant caches and fixed capacity close these; they rarely come for free.
• 📉 What is left is bandwidth, cost and visibility. A channel through a shared, noisy, monitored service is slow, takes effort to build, and leaves traces. That matches OpenAI’s own conclusion: its channels “carry information slowly”, and its response was detection.

So the honest goal is not “agents cannot communicate”. It is: communication between agents that should be separate is slow, expensive, observable, and not worth much, because layer 0 has made sure no colluding pair can assemble data, untrusted input and a way out. Isolation lowers the odds; capability design bounds the damage.

🔬 The frontier

Beyond what you can deploy today, three research directions aim at the root cause:

• 🛡 Time protection as an OS abstraction. Ge, Yarom, Chothia and Heiser, Time Protection: the Missing OS Abstraction (EuroSys 2019, Best Paper), argue that operating systems need a temporal counterpart to memory protection: colour the caches by domain, flush the core-local state on every domain switch, pad switches to a fixed time. They implemented it in seL4 with under 1% slowdown for most benchmarks, and follow-up work, Proving the Absence of Microarchitectural Timing Channels, aims to extend seL4’s formal proofs from storage channels to timing channels. See the Trustworthy Systems time-protection project.
• 🔪 Hardware partitioning without a hypervisor. Core slicing (OSDI 2023), from Microsoft Research and UCSD, gives each guest a static slice of cores, memory and virtual I/O enforced by small hardware extensions, removing hypervisor-observable signals such as nested page faults and scheduling. Stratus (HotCloud 2020) argued for clouds that manage microarchitectural resources explicitly, the line of thinking Marghera later made practical.
• 🎰 Caches that resist conflict attacks by design. Randomised, skewed caches such as ScatterCache (USENIX Security 2019) and the fully associative-looking Mirage (USENIX Security 2021) make eviction-set construction impractical. They are proposals; no mainstream server CPU ships them.

One thing that is not on this list: confidential computing. AMD SEV-SNP and Intel TDX protect a guest from the host, not guests from each other, and their own threat models leave microarchitectural side channels largely out of scope. An SEV-SNP primer sums it up: the mitigations raise the bar “but side-channel risk is not eliminated”.

✅ A checklist by threat level

🔗 How this fits the earlier post

Sandboxing untrusted AI agents on Linux argued that the strongest boundary is a second kernel, which is why serious hosted platforms use microVMs (its Firecracker section explains the choice). OpenAI’s report sharpens that conclusion in two ways:

1. A second kernel is a boundary for software, not for hardware. The microVM is still the right primitive, but its guarantees stop at what the hypervisor virtualises. What shares a host is a security decision, and the scheduler is part of the sandbox.
2. The section 8 items are not theoretical. That post listed side channels, exfiltration through allowed channels, and the model as a channel among what the stack still misses. OpenAI has now shown agents building those channels on demand. The fix is the same as before, applied across agents: decide what each agent may hold, and make sure the agents that could talk to each other cannot assemble the lethal trifecta between them.

If any of this felt like you’d read it before, that’s because you had: Prime+Probe is 21 years old, Rowhammer is 12, Spectre is 8, and not one line of silicon changed to make OpenAI’s finding possible. Same cache, same bus, same clock, same bag of tricks from the same conference proceedings — just a new kind of attacker that can read all of it overnight and never gets bored of trying.

— Déjà vu, agentic-sandbox edition

📚 Sources

Point-in-time (checked October 2026).

• The report: OpenAI, Securing our frontier model training environment (23 October 2026) · screenshot via @halvarflake quoting @scaling01 on X
• Foundations: Lampson, A note on the confinement problem (CACM 1973) · Spectre and Meltdown
• VM and host guidance: Firecracker production host setup · Firecracker (NSDI 2020) · AWS Nitro, the EC2 approach to preventing side channels · Linux core scheduling · Linux hardware vulnerabilities · Linux KSM · Linux resctrl · Arm MPAM
• Dedicated hardware: AWS Dedicated Hosts · Google Cloud sole-tenant nodes · Azure Dedicated Host
• Cross-tenant attacks: Ristenpart et al. (CCS 2009) · Whispers in the Hyper-space (USENIX Security 2012) · Flush+Reload (USENIX Security 2014) · Last-Level Cache Side-Channel Attacks are Practical (IEEE S&P 2015) · Seriously, get off my cloud! (2015) · DRAMA (USENIX Security 2016) · Attack Directories, Not Caches (IEEE S&P 2019) · Volcano (2021) · Hertzbleed (USENIX Security 2022)
• Black Hat, hacker camps and proofs of concept: D’Antoine, covert cross-VM communication via out-of-order execution (Black Hat USA 2015) · Hello from the Other Side (Black Hat Asia 2017) and NDSS 2017 paper · Flip Feng Shui (USENIX Security 2016) and SHA2017 talk · TLBleed (USENIX Security 2018) · RIDL and Fallout · ZombieLoad · NetCAT (IEEE S&P 2020) · Osiris (USENIX Security 2021) · Downfall (USENIX Security 2023) · Zenbleed · GoFetch (USENIX Security 2024) · Dark Reading on the Black Hat USA 2026 ChatGPT sandbox talk
• Chaos Communication Congress: Covert channels in TCP/IP (22C3) · Rowhammer.js (32C3) · What could possibly go wrong with <insert x86 instruction here>? (33C3) · A Christmas Carol: the Spectres of the Past, Present, and Future (35C3) · ZombieLoad (36C3) · Practical cache attacks from the network (36C3) · Attacking CPUs with power side channels from software (rC3) and PLATYPUS · Ten Years of Rowhammer (38C3) · Rowhammer in the Wild (39C3) · Spectre in the real world (39C3)
• GPUs: TunneLs for Bootlegging (CCS 2023) · Veiled Pathways (MICRO 2024) · Behind Bars (USENIX Security 2026)
• Defences: Scheduler-based defences against cross-VM side channels (USENIX Security 2014) · CATalyst (HPCA 2016) · Marghera, Principled Microarchitectural Isolation on Cloud CPUs (CCS 2024) · AMD-SB-7025 · Fantastic Timers (2017) · CloudRadar (RAID 2016) · Chiappetta, Savaş and Yılmaz (2016)
• Frontier: Time Protection (EuroSys 2019) · Proving the Absence of Microarchitectural Timing Channels · Trustworthy Systems time protection · Core slicing (OSDI 2023) · Stratus (HotCloud 2020) · ScatterCache (USENIX Security 2019) · Mirage (USENIX Security 2021) · SEV-SNP primer (2026)
• Platform Security Summit 2018: programme · Hypervisor-Mediated data eXchange (Christopher Clark) · Enterprise Scale Separation VMM Systems (Myong Kang, NRL) · Hypervisor Security: Lessons Learned (Ian Pratt, Bromium)
• LLM serving: Auditing Prompt Caching in Language Model APIs (ICML 2025)
• Related: The lethal trifecta (Simon Willison) · Sandboxing untrusted AI agents on Linux