When to compact matters more than how much

Anyone who runs a coding agent has watched a compaction fire at the worst possible moment. The context window fills, a token threshold trips, the scaffold summarizes everything, and the partial derivation the model was three steps from finishing is now one vague sentence in a summary. A team from Johns Hopkins and Apple measured this failure and tested a fix that requires no training at all: let the model decide when to compact, guided by a short rubric (Li et al., 2026).

Context rot is real and self-inflicted ¶

Long traces are the norm now. The paper notes that on a single competition math question, Qwen3.5 produces 81k tokens of reasoning and Kimi-K2.5 produces 96k. Agent systems stack search results, code output, and intermediate plans on top of that across hundreds of turns.

The problem is that a growing trace accumulates junk: a flawed case split from early on, a search result the model moved past, a candidate program that went nowhere. The junk does not sit there inertly. It anchors what follows. The authors point at a phenomenon the field has started calling context rot: a model that solves a problem from a clean start often fails when fed back its own flawed earlier reasoning.

Every production scaffold handles this with a token threshold. The paper cites MiniMax-M2.5's documented trigger, compaction when token usage passes 30% of the context window, and the /compact command familiar to Claude Code users, which delegates the timing decision to a human. A fixed token threshold knows how full the window is. It has no idea what the model is doing. Their Figure 1 shows the cost concretely: on a hard BrowseComp question, fixed-interval compression wiped four already-verified facts mid-trajectory, and the model fell back to a guess.

The fix is a tool plus a rubric, and it needs both ¶

SelfCompact adds two things at inference time. The first is a compaction tool the model can invoke to summarize its accumulated context. The second is a lightweight rubric telling it when to fire, after a sub-task resolves or when the trajectory is converging, and when to hold off, mid-derivation or when stuck.

The interesting engineering detail is that neither piece works alone. Given only the tool, open-weight models use it unevenly, firing at unhelpful moments or never. Given only the rubric, the model has no way to act on it. Together they produce adaptive compaction with no fine-tuning and no external supervision.

Across six benchmarks spanning competitive math and agentic search, on seven models, SelfCompact improved over a no-summarization baseline by up to 18.1 points on math and 5 to 9 points on agentic search, at 30 to 70% lower per-question cost. Against fixed-interval summarization it matched or exceeded accuracy at a fraction of the token cost. Cheaper and better at the same time, which is the same shape as the Microsoft pruning result we covered in June: the context an agent drags around is mostly a tax, and removing it at the right moment is a quality lever, not a quality trade.

A capability the scaffold can supply ¶

The authors frame the finding as a meta-cognitive gap. Unprompted, models cannot reliably tell that their own context has rotted. With a rubric, they can. Knowing when to compact turns out to be a capability you can hand a model in its prompt, without touching its weights.

That framing matters for anyone building agents right now, because it means compaction timing is a policy decision you already control. If your scaffold compacts on a byte count, the paper says you are leaving both accuracy and money on the table, and the alternative costs a paragraph of prompt. It is also the question we get asked most in practice, which is why our pre-flight tool reports whether a context is worth compressing or clearing before anything is sent. Timing beats ratio there too.

The caveat ¶

A preprint, evaluated on competitive math and agentic search, and the headline deltas are against a no-summarization baseline, the weakest of the comparisons; against fixed-interval summarization the honest claim is equal-or-better accuracy at lower cost, not a blowout. Rubric compliance varied across the seven models, so a scaffold built on this needs to verify its particular model actually follows the rubric rather than assume it. None of that dents the useful core: the timing of compaction, which almost every deployed system currently decides with a token counter, is load-bearing, and a few sentences of instruction recover most of what the token counter destroys.