Compressing prompts harder made them more expensive

Here is a result that should sting anyone who tunes compression by ratio, including us. A pre-registered randomized controlled trial compressed real production prompts at three retention rates and measured the full bill, input plus output. Keeping half the tokens cut mean total cost by 27.9%. Keeping a fifth of the tokens, the most aggressive arm, raised mean total cost by 1.8% (Johnson and Lee, 2026). More compression, bigger bill.

We build a compression product, so a trial showing compression can backfire is exactly the kind of evidence we would be suspected of burying. We would rather explain it, because the mechanism is the strongest argument for doing compression carefully instead of maximally.

The trial ¶

Most compression research reports numbers from academic benchmarks: HumanEval, GSM8K, clean and well-formed. This trial ran on production traffic. The authors drew from 1,199 real multi-agent orchestration instructions, randomized them across six arms, and analyzed 358 successful Claude Sonnet 4.5 runs, 59 to 61 per arm. The design was pre-registered, meaning the analysis plan was committed before the data came in, which removes the usual room for picking the favorable slice afterward.

The six arms: an uncompressed control, uniform compression at retention rates of 0.8, 0.5, and 0.2, and two structure-aware strategies, entropy-adaptive and recency-weighted.

Output tokens flipped the sign ¶

Input tokens are the cheap ones. The paper cites typical frontier pricing at $3 to $15 per million input tokens against $15 to $75 per million output tokens. So a compression policy is really a bet: shrink the cheap side without making the expensive side grow.

Moderate compression won that bet. At a retention rate of about 0.5, total cost fell 27.9%, and the recency-weighted structure-aware arm saved 23.5%. Those two occupied the empirical cost-similarity Pareto frontier.

Aggressive compression lost it. At retention 0.2 the inputs shrank substantially, and the mean total cost still went up 1.8%, driven by output expansion averaging 1.03x against control with heavy-tailed uncertainty. A 3% mean increase in output length sounds like nothing. Priced at 5x the input rate, on a heavy-tailed distribution, it ate the entire input saving and then some. A model handed a mangled prompt does what you would do with a mangled brief: it hedges, restates, and works around the gaps, at length. The aggressive arm was dominated on both cost and response similarity, meaning there was no reason to choose it at all.

The cheapest prompt is not the shortest one. It is the one that does not make the model write more.

What we take from this ¶

The authors' own conclusion is blunt: "compress more" is not a reliable production heuristic, and output tokens must be treated as a first-class outcome when designing compression policies. Two practical rules fall out of it.

First, measure total cost, never input reduction. A compression dashboard that only counts input tokens saved can show a beautiful number on a policy that is losing you money.

Second, prefer structure-aware moderation over uniform aggression. The arms that survived on the Pareto frontier were moderate uniform compression and recency-weighted compression, which protects recent context and prunes older material harder. That matches what we see in our own engine and why its behavior is graduated: small documents get flagged as not worth compressing at all, since overhead can push savings negative, and larger ones compress harder because there is more redundancy to remove safely. Ratio is an output of the content, not a dial to max out.

The caveat ¶

One model (Claude Sonnet 4.5), one production system, 358 successful runs, and a preprint. Quality here is measured by embedding similarity to the uncompressed response, which is a proxy: two responses can be similar and both wrong, or dissimilar and both fine. A different task mix, or a model that reacts to sparse prompts by writing less instead of more, could move the crossover point. What the trial establishes is narrower and still useful: in at least one real production deployment, under a pre-registered design, the relationship between compression ratio and cost was not monotonic, and the optimum sat near the middle. If your compression policy assumes harder always equals cheaper, you now have a concrete reason to go check.