AI Needs Two Context Windows

Last week I was three hours into building an automation when the model started ignoring the output format I'd specified at the beginning. Same session. Same instructions sitting right there in the conversation history. But somewhere around message forty, the format guidance had become wallpaper. Present but no longer consulted.

I scrolled back and found my original instructions exactly where I'd left them. The model hadn't forgotten them in any literal sense. They'd just... faded. Diluted by everything that came after.

What early instructions start to feel like by message forty.

Researchers call this "lost in the middle." A 2024 study from Stanford found that language models perform best when relevant information sits at the beginning or end of their context window, but struggle when it's buried in the middle.¹ The instructions that mattered most had become middle.

This got me thinking about an old problem computers solved decades ago.

The Coin-Sized Solution

Inside every computer sits a tiny chip called CMOS, powered by a battery roughly the size of a quarter. It holds maybe 256 bytes of data. Less than a text message. But that small memory does something essential: it remembers the instructions that matter most, even when everything else loses power.²

The CMOS doesn't store your documents or run your programs. It holds foundational settings: how to find the hard drive, what time it is, which device to boot from first. The rest of the system juggles gigabytes. Those 256 bytes stay constant, always present, always consulted first.

How Brains Do It

Human cognition works similarly. The prefrontal cortex acts as an executive controller, maintaining goals and rules while the rest of the brain processes sensory information.³ It doesn't store everything. It maintains what matters. Your intentions. The thing you're trying not to forget while you're distracted by something else.

The executive function: small, persistent, always watching.

When the prefrontal cortex is damaged, people can still perceive and process information. What they lose is the ability to stay on task, to remember why they started, to hold boundaries in place while everything else flows by.

Two Windows

What if language models had something similar? Not one vast context window where everything competes for attention, but two: a large workspace for conversation, and a smaller executive window that never fades.

The executive window would hold the persistent instructions and guardrails. It would sit above the conversation, always consulted, never diluted by the growing pile of messages below. The main window could stretch to millions of tokens. The executive window might need only a few hundred. But those few hundred would be inviolable.

Anthropic documented how flooding a model with carefully structured text can override safety training entirely, a technique called "many-shot jailbreaking."⁴ The instructions that were supposed to matter most get lost in the noise. An executive window wouldn't be subject to these attacks. It would be architecturally separate, like CMOS is separate from RAM.

The Pattern

We've solved this problem before. Computers needed persistent instructions that survived the chaos of runtime. Brains needed executive function that maintained goals through distraction.

A small, protected space where the instructions that matter most can live without being forgotten. We should build it.

---

References

Footnotes

1. Liu, N., et al. (2024). "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics, 12, 157-173. MIT Press ↩

2. Wikipedia contributors. (2024). "Nonvolatile BIOS memory." Wikipedia ↩

3. Friedman, N. & Robbins, T. (2022). "The role of prefrontal cortex in cognitive control and executive function." Neuropsychopharmacology, 47, 72-89. Nature ↩

4. Anthropic. (2024). "Many-shot jailbreaking." Anthropic Research ↩