A team can submit more patches while its release queue stays full. An agent trial should follow work through review and acceptance.
An August paper is receiving fresh attention
Fiona Chen and James Stratton’s Harvard working paper is dated August 4, 2026. It studies firm adoption using proprietary Jellyfish data covering 718 firms and about 300 million work events. The dataset ends in March 2026. This is a working paper, not a newly published October experiment.
The authors estimate roughly 30% more lines of code and 23% more pull requests after agent adoption. They do not find a statistically significant increase in the number of completed Jira issues or epics. Positive estimates with uncertainty remain possible; the result does not prove that agents deliver no value.
Review time includes more than reading a patch
The paper reports a longer interval between pull request submission and merge, along with more review comments and requests for changes. That interval includes waiting and revision. It should not be described as the number of hours a reviewer actively reads code.
The study compares adoption at the firm level. It does not establish the quality of every agent-written patch or the result of one model on your repository. Different workflows and adoption patterns can produce different outcomes.
Follow one request through the whole trial
For your own evaluation, start with a small set of comparable requests and write acceptance criteria before work begins. Record the original request, the submitted patch, the review changes, and the accepted behavior. Keep rejected attempts in the record.
- Track elapsed time from request to acceptance.
- Record active review and correction time separately.
- Count completed requests that pass the same criteria.
- Include model costs and discarded attempts.
- Check for repairs needed after release.
Suppose a trial creates 20 patches, but only 8 pass review during the reporting period. Keep those counts separate. An activity measure helps explain the queue; an acceptance measure helps decide whether the workflow is useful. This is an illustrative example, not a result from the paper.
Compare the tool with the process around it
I would change one part of the workflow at a time. A better model may reduce the need for corrections, while a review rule may prevent unsuitable patches from reaching the queue. Measure both changes against the same definition of completed work.
Our AI Scan guide covers rollout verification. The Copilot trace guide helps record where an agent spends its attempts.
What proportion of your agent trials reach acceptance, and how much reviewer time do they consume?
Paper checked October 13, 2026. Estimates come from an observational firm-adoption study; this article does not report a local experiment.
