Token-level compression breaks agents at any ratio

If you compress prompts for a living, or just pay for an agent that runs one, the standard tooling is token-level extraction: rank every token by self-information, drop the predictable ones, keep the surprising ones. LLMLingua-2 and Selective Context both work this way, and on QA and summarization they earn their reputation, with published results up to 20x compression at minimal loss. A 2026 study asked what happens when you point the same tools at LLM agent trajectories, and the answer is not a modest degradation. It is collapse (Zhang and Sun, 2026).

We sell context compression, so a paper about compression breaking things is one we have to read carefully rather than quietly skip. The numbers below are theirs, not ours.

Every configuration failed ¶

The authors audited two token-level method families, Selective Context and LLMLingua-2, across 17 combinations of agent environment, backbone model, and method. Every single cell collapsed to a mean reward of 0.05 or less, while the compressors delivered 1.3x to 13.3x realized compression. The agents were compressing fine. They just stopped completing tasks.

The failure is not a tuning problem. The authors retrained LLMLingua-2 on agent data at a 4x target, and the retrained variant failed identically to the off-the-shelf one. Whatever is going wrong is structural, not a hyperparameter you can search your way out of.

The compressor deletes the action grammar ¶

The paper names the mechanism action-grammar destruction, and once you see it, it is obvious in hindsight. An agent trajectory is full of strings like click[Buy Now] and search[men's shoes size 10]: identifiers, brackets, action verbs. These tokens are highly predictable from the surrounding text, which means self-information ranks them near the bottom. A token-level compressor is built to drop exactly those tokens first.

The compressor worked exactly as designed. That was the problem. What survives is grammatical English but ungrammatical agent commands, and the environment rejects the residual. This lines up with what agent benchmarks already report from the other direction: AgentBench names Invalid Format and Invalid Action as primary termination categories, and format-only perturbations have produced accuracy swings as large as 76 points on in-context tasks (both cited in the paper's introduction).

The fix is granularity, not a better ranker ¶

The diagnosis points somewhere specific: compress at step granularity. Keep each retained action-observation pair verbatim, and do the pruning between steps rather than inside them. The paper's system, AGORA, combines a structural prompt parser, an always-keep floor for format- and recency-critical content, and a 125M-parameter relevance scorer trained on counterfactual next-action-change labels. It runs in about 2ms per step with no extra LLM calls.

Against the compared inference-free and LLM-based methods, AGORA was the only one to retain at least 75% of uncompressed performance in 8 of 9 cells, with the one exception at 73%, while the learned scorer produced 1.0x to 11.5x adaptive compression from a single fixed keep ratio. The ablation is the part worth remembering: the structural always-keep floor, not the learned scorer, was the dominant quality lever. Knowing what never to touch mattered more than ranking what to drop.

What this means if you compress context ¶

The general rule this paper measures is one we have organized our engine around: compression has to respect the structure of what it compresses, or the ratio is a vanity metric. Our skeleton output prunes whole regions and keeps what survives verbatim, rather than thinning tokens inside a command or a code block, for the same reason AGORA keeps retained steps intact. A compressed prompt that the downstream consumer rejects has a compression ratio of infinity and a value of zero.

If you run agents and are shopping for compression, the test is cheap: compress a real trajectory, then check whether the actions in it still parse. Ratio alone tells you nothing about that.

The caveat ¶

This is a preprint from two authors with, at the time of writing, zero citations, and the collapse result covers two token-level method families on agent environments like WebShop, where in-context history can pass 100k tokens by step 30. Other method families and quieter domains may fail more gracefully, and the AGORA numbers are the authors evaluating their own system. The durable part is the mechanism, which does not depend on their system at all: self-information ranks action syntax as disposable, so any compressor that prunes tokens by predictability will strip agent commands first. That part you can verify on your own traces this afternoon.