Skip to content
MustHave.ai Musthave.ai
AI Tools

TokenEconomizer uses past coding sessions to cut the cost of future fixes

TokenEconomizer learns from past coding sessions. See what the SWE-Bench results measure and how to calculate cost per passing patch in your own trial.

TokenEconomizer uses past coding sessions to cut the cost of future fixes
Later coding sessions can reuse repository knowledge from earlier work.

The first bug in a repository pays for discovery. The next one may reuse what the agent learned.

Researchers at Microsoft and UC Berkeley released TokenEconomizer, an open-source plugin that turns completed coding sessions into guidance for later work. In their November 16 report, they found mean cost per task fell by 18.4% to 30.9% across five agent and model combinations. They tested 277 SWE-Bench tasks from six repositories. That range describes the authors’ runs under their benchmark setup.

Where the plugin tries to save tokens

The plugin builds small, focused pieces of repository guidance from finished sessions. It also narrows the list of tools sent to the model and can intervene when the agent repeats an unsuccessful approach. A repository with its own test conventions and recurring setup problems gives the plugin material it can reuse. Every task on a brand-new project has less shared history to draw from. The paper and public code describe the controller and experiment.

The authors report that early tasks can cost more while the plugin builds experience. In their Claude Code comparison, cumulative cost fell below the baseline by the sixth task in each tested repository. Since that timing comes from one reported comparison, plan to track your own cumulative curve. A single cold run captures the initial overhead and misses the later work it is intended to improve.

What a cheaper task costs per passing patch

Illustrative results over 10 tasksTotal costPassing patchesCost per passing patch
Baseline$10.008$1.25
Lower spend with fewer passing patches$8.006$1.33
Lower spend with the same pass count$8.008$1.00
Hypothetical costs in U.S. dollars, rounded to the nearest cent.

These figures are a worked example. Divide total spend, including unsuccessful attempts, by the number of patches passing the same acceptance tests. Both cheaper runs spend 20% less. In the six-pass run, each usable patch costs about 6.7% more than the baseline. This is why the experiment also reports Pass@1, the share of tasks solved on the first attempt.

A small trial that includes the learning period

Choose a repository you fix regularly and at least six historical tasks with checks you can rerun. Reset each checkout to the correct starting commit. Run the same task sequence with and without the plugin, under the same model and retry budget. Keep the plugin’s learned state between its tasks. Record total spend, pass counts, and any rule or test the agent skipped.

Report both the cumulative cost and the cost per passing patch. If a patch misses a repository-specific check, preserve it as a failed attempt in the numbers. That gives a maintainer a direct way to judge whether savings survive the team’s normal review standard.

Abdessalam Alaoui

I'm Abdessalam Alaoui, founder of Musthave.AI. I write practical guides to AI models, tools, generative media.

Join the discussion

Your email address will not be published. Required fields are marked *

Keep reading

All news