2026-10-11 16:37 UTC

Facebook Research and UW (Shao, Shen, Zettlemoyer, Koh et al.) released the official Context Language Models codebase and paper claiming models that treat their own context as a freely editable file beat SOTA context-management strategies zero-shot (e.g., +11.4% BrowseComp-Plus accuracy at −21.5% FLOPs, gains on 12–24-hour agent tasks) with day-1 Pi support — external adoption or replication of the context-as-a-file approach across harnesses and models, or ContextBench, would establish CLMs as a durable research direction rather than a one-off repo.

state: watchingheat: lowuncertainty: mediumconvergesscott: highagent-memory context-management llm-architecture meta-researchRulin ShaoLuke ZettlemoyerPang Wei KohNathan LambertMike Lewis
Surfaced 2026-10-03T15:55:11Z — Abstract: "We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treatin — The attention wave has fully crested and decayed: the front-page thread is flat (~0 pts/h, 2.6th-percentile peer velocity at ~121h age) and the incremental comments re-amplify the same three critiques already on file (suffix-cache surprise, attention-cost/cache-economics concern, novelty challenge) rather than adding substance. With no adoption, replication, independent verification, or ContextBench drop, the case downshifts to a cold watch on its named establishment triggers; the magnitude-valve spread reading reflects the days-old peak, not current motion, which is why heat prices low.

What is this?

On 29 Sep 2026, Rulin Shao, Shannon Zejiang Shen, Luke Zettlemoyer, Pang Wei Koh, Nathan Lambert, Mike Lewis, Wen-tau Yih and others posted 'Context Language Models' (arXiv:2609.37725), proposing a model class that natively manages its own context by treating the context as a freely editable file — shifting context management from external harness code to intrinsic model behavior, learnable both in-context and parametrically, and extending naturally to multi-agent systems where each agent's context is a separate file. The abstract confirms the headline claims: zero-shot CLMs built on existing models beat SOTA context-management strategies (+11.4% accuracy at −21.5% FLOPs on BrowseComp-Plus, +5% at −59% FLOPs on 12-hour EdgeBench, +65% greater improvement at equal compute on a 24-hour multi-repository agent-swarm task). The supplied snippets verify the paper, its author roster, and its claims, but do not corroborate the Meta Superintelligence Labs institutional framing, an official codebase, day-1 Pi support, or the existence of 'ContextBench' — and none of the other search results show external adoption or replication of the context-as-a-file approach; the adjacent hits (ICLR 2026 agentic-context-engineering, context-engineering surveys) are same-territory work, not evidence of uptake.

Why it matters to Scott

A heavyweight Meta/UW team has independently arrived at Scott's core position — context is a freely editable file the agent itself compacts and discards — making this a dated-receipts opportunity for the Context Engineering ebook and a direct comparative datapoint on whether agent-authored compaction beats harness-side strategies. Its model-intrinsic-vs-harness locus claim also pressures the scaffolding-hypothesis premise that the adaptive layer must live outside the frozen model, but with replication and external adoption still absent (per the grounding), that tension stays a watch-item rather than a verdict.
ip:framework.context-engineeringip:source.context-engineering-why-building-ai-agents-feels-like-programming-on-a-vic-20-again-ebookdev:concept.agent-authored-context-compactionip:concept.scaffolding-hypothesisradar:agentic-context-management-paperradar:concept.context-managementradar:concept.agent-memoryradar:self-written-notes-reasoning-gainsradar:anthropic-claude-5-context-engineering
queries asked of Scott's wikis
  • "context as a file" agent-editable memory
  • harness-level context management compaction strategy
  • model-intrinsic behavior vs external scaffolding division
  • multi-agent context sharing memory isolation
  • long-context agent compute cost FLOPs economics
  • RAG vs in-context knowledge storage tradeoff

Measured heat

now 0 pts/hpeak 36 pts/hcomments 0/hpeers p14momentum: steady3 platformsage 314h
points/hour across evidence · reading as of 2026-10-12 02:59:37.977291+11:00 · deterministic, not a model opinion

How the heat travelled

09-28 14:00⭐ origin echo-reconstructedAbstract: "We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treatin
Rulin Shao et al. (University of Washington, Meta Superintelligence Labs, MIT, Trillium Labs) — submitted to arXiv by Rulin Shao on paper (echo) · attributed from hn.story.49915539
—
09-30 22:54first on hacker news · published · +56.9hContext Language Models (CLMs)
handfuloflight
—
10-05 17:48first on r/LocalLLaMA · published · +171.8hY'all this is a sexy paper; context language models
Combinatorilliance
—
10-05 19:41first on r/artificial · published · +173.7hContext Language Models
striketheviol
—
09-30 22:54amplified on hacker newshn.story.49915539
handfuloflight
peak 2 · 0 comments · 1% of case engagement
10-01 14:51amplified on hacker news 👑hn.story.49922437
emersonmacro
peak 177 · 52 comments · 72% of case engagement
10-05 17:48amplified on r/LocalLLaMAreddit.post.1wyf63m
Combinatorilliance
peak 109 · 42 comments · 26% of case engagement
10-05 19:41amplified on r/artificialreddit.post.1wyi5nb
striketheviol
peak 3 · 1 comments · 1% of case engagement
10-01 00:21our radar first saw it · +58.4hdiscovery anchor: hn.story.49915539—
10-03 15:27reached heat=high · +121.5h · via ledger——
pace: p75 vs 1188 stories at the 168h mark (now 314h old) — ahead of research-agent-compression-generalization (1.0x), behind gguf-quant-filename-mismatch (1.0x)

Evidence (5) — ⭐ canonical anchor

sourceobjectauthorscorecomments
🟧 hnContext Language Models (CLMs)
Retrieved article excerpt

Open article · Retrieved 2026-10-01T00:29:49.643231+00:00

# Context Language Models (CLMs)

[Rulin Shao](https://rulinshao.github.io/)1,2,
[Shannon Zejiang Shen](https://www.szj.io/)3,
[Junjie Oscar Yin](https://oseyincs.io/)1,2,
[Yuetai Li](https://yuetl9.github.io/)1,
[Minheng Wang](https://minhengwang.github.io/)1,
[Hamish Ivison](https://ivison.id.au)1,
[Radha Poovendran](https://people.ece.uw.edu/radha/)1,
[Nathan Lambert](https://natolambert.com/)4,
[Teng Xiao](https://tengxiao1.github.io/)1,
[Mike Lewis](https://ai.meta.com/people/209431298931133/mike-lewis/)2,
[Wen-tau Yih](https://scottyih.org/)2,
[Luke Zettlemoyer](https://homes.cs.washington.edu/~lsz/)1,2,
[Pang Wei Koh](https://koh.pw/)1

1University of Washington   2Meta Superintelligence Labs   3MIT   4Trillium Labs

[arXiv](https://arxiv.org/abs/2609.37725)
[Twitter](https://x.com/RulinShao/status/2105282444270448647)

[Context Language Models](https://github.com/facebookresearch/context-language-models/blob/main/assets/teaser.png)

We introduce **Context Language Models (CLMs)**, language models that natively manage their own
context. We implement this by treating the **context as a file** and allowing the model to make
unrestricted updates to this file. This allows the model to learn what is most important to
maintain in context, and naturally extends to multi-agent systems where multiple agent
contexts coexist as files.

- **Zero-shot.** Building CLMs zero-shot with existing models outperforms SOTA
  context-management strategies across a variety of tasks: 11.4% higher accuracy with 21.5%
  fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench,
  and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm
  task.
- **In-context learning.** We show that CLMs can be steered with natural-language
  instructions evolved through a standard skill-optimization loop, improving held-out
  accuracy by up to 35.9 points on a context-management task while reducing compute.
- **Reinforcement learning.** We also introduce an online reinforcement learning method for
  CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer
  FLOPs.

## Day 1 Support

CLM for [Pi](https://github.com/earendil-works/pi):

```
pi install npm:@lolipopshock/pi-clm
```

## Getting started

Run the minimal CLM agent on any [Harbor](https://github.com/laude-institute/harbor) task:

```
pip install -e .
clm-harbor run -p <harbor-task> -a clm-minimal -m openai/<model> \
  --agent-kwarg api_base=http://localhost:8000/v1
```

`clm-harbor` is the Harbor CLI with CLM available as `-a clm-minimal`. See
[`clm/clm_harness`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_harness) for configuration and serving.

## Repository

|  |  |
| --- | --- |
| [`clm/clm_harness`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_harness) | CLMs implemented in [Harbor](https://github.com/laude-institute/harbor) |
| [`clm/clm_icl`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_icl) | In-context learning for CLMs |
| [`clm/clm_rl`](https://github.com/facebookresearch/context-language-models/blob/main/clm/clm_rl) | Reinforcement learning for CLMs |
| [`suffix_cache_reuse`](https://github.com/facebookresearch/context-language-models/blob/main/suffix_cache_reuse) | Suffix Cache Reuse for CLM efficient serving |

## Coming soon

- ContextBench

## Citation

If you find our work helpful, we would appreciate it if you could cite our paper:

```
@article{shao2026context,
  title   = {Context Language Models},
  author  = {Shao, Rulin and Shen, Shannon Zejiang and Yin, Junjie Oscar and Li, Yuetai and
             Wang, Minheng and Ivison, Hamish and Poovendran, Radha and Lambert, Nathan and
             Xiao, Teng and Lewis, Mike and Yih, Wen-tau and Zettlemoyer, Luke and Koh, Pang Wei},
  journal = {arXiv preprint arXiv:2609.37725},
  year    = {2026}
}
```

## License

This project is licensed under [CC BY-NC 4.0](https://github.com/facebookresearch/context-language-models/blob/main/LICENSE). See also [NOTICE](https://github.com/facebookresearch/context-language-models/blob/main/NOTICE).
handfuloflight20
🟧 echo.paper ⭐Abstract: "We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treatinRulin Shao et al. (University of Washington, Meta Superintelligence Labs, MIT, Trillium Labs) — submitted to arXiv by Rulin Shao——
🟧 hnContext Language Modelsemersonmacro17752
🟠 redditY'all this is a sexy paper; context language models
LocalLLaMA
Combinatorilliance10942
🟠 redditContext Language Models
artificial
striketheviol31

Interpretation history

Decision trace