A new study of 441 repositories found better code-quality signals when coding-agent projects committed AI configuration files. The result is useful, but it does not prove that adding an AGENTS.md file will improve a weak engineering process.
The paper, submitted August 26, compares repositories that adopted coding agents with and without committed agent configuration. Its authors report that agent-adopting projects produced 28% to 38% more commits after adoption. Agent-first repositories without committed configuration showed a 53% increase in cognitive complexity, compared with 27% for configured projects, and 1.7 times as many warnings.
The study measures association, not a configuration cure
Repositories that commit instructions may already have stronger review habits, clearer ownership, better tests or more experienced maintainers. Model capability, project type and adoption timing may differ too. The authors explicitly present the findings as hypothesis-generating.
The paper reports 97% agreement on held-out labels, which supports the reliability of its classification process. It does not remove confounding from the observed engineering outcomes.
Committed configuration is valuable for a simpler reason
A versioned instruction file makes expectations inspectable. A reviewer can see which commands, architecture rules, validation steps and boundaries the agent received at a given commit. A prompt stored in one developer’s chat history cannot provide the same shared audit trail.
- Put build, test and lint commands next to the code.
- Name files or directories the agent must not modify.
- Document architectural boundaries and dependency rules.
- Require validation proportional to the change’s risk.
- Explain how to report uncertainty and blocked work.
Do not turn the file into a giant style guide
An instruction file should change agent behavior. Repeating conventions already enforced by formatters wastes context and creates conflict. Prefer executable truth: exact commands, narrow invariants, ownership boundaries and examples of repository-specific failure.
| Section | Useful content | Avoid |
|---|---|---|
| Commands | Exact test, lint and build commands | “Run appropriate checks” |
| Architecture | Dependency and layer boundaries | Generic clean-code slogans |
| Permissions | Files, secrets and external actions that require approval | Assuming the sandbox enforces the prose |
| Definition of done | Required evidence and output format | A vague request to be careful |
Run a repository-level before-and-after test
Select a representative set of bug fixes, refactors and features. Run the same agent and model with the current setup, then with a concise committed configuration. Compare accepted patches, review comments, test failures, rework time, complexity changes and security findings.
Do not measure commits alone. More commits can reflect useful throughput, smaller changes or churn. Pair volume with merge acceptance and downstream defects. Our Copilot review-effort analysis explains why tool activity is not a quality grade.
Configuration cannot replace infrastructure controls
A Markdown rule saying “do not access production” is not equivalent to a network or credential boundary. Use repository configuration to guide behavior and infrastructure to prevent unacceptable actions. The Musthave.ai agent security scorecard covers identity, secrets, logging, rollback and data movement.
Start with the failures reviewers already see
Collect ten recent review comments caused by agent misunderstandings: wrong test command, forbidden dependency, misplaced file, missing migration, broken public API or unhandled error. Convert only recurring repository-specific failures into concise rules. This keeps the configuration connected to observed work rather than imagined completeness.
For each rule, name the evidence that proves compliance. “Preserve backward compatibility” is weak. “Run the public API contract tests and do not remove exported fields without approval” is testable. Link to longer documentation instead of copying it into every agent session.
Treat configuration changes as experiments
Add one cluster of rules at a time and annotate the commit with the failure it addresses. Watch whether the same review comment disappears, whether token use or latency rises, and whether the agent begins overfitting to the instruction. A rule that prevents one mistake while creating three workarounds needs revision.
Use different scopes where the repository permits it. A frontend package, migration directory and infrastructure module may need different commands and approval boundaries. The nearest applicable instruction should be clear, and conflicting files should fail preflight rather than leaving the model to guess.
Keep a small configuration scorecard
- First-pass test success.
- Actionable review comments per patch.
- Human repair minutes.
- Regression or revert rate.
- Tokens and elapsed time per accepted change.
- Permission or scope violations.
Review the scorecard monthly and remove stale instructions. A committed file is valuable because it is versioned, not because it should grow forever.
Assign ownership for each rule cluster. Security should own approval and secret boundaries; maintainers should own architecture and commands; product teams should not silently add acceptance criteria that tests cannot verify. When the repository changes build system, directory structure or deployment path, include the configuration file in the migration checklist.
Configuration is also onboarding documentation. If a new human contributor cannot understand a rule, the agent is unlikely to interpret it consistently. Prefer direct language, concrete paths and commands, and a short reason where the boundary would otherwise look arbitrary.
My verdict: commit the contract, then measure it
The study gives teams a credible reason to test committed agent configuration. Its observational design does not establish causation, so the responsible action is an internal experiment—not a universal promise.
Keep the file short, versioned and repository-specific. Review changes to it like code. If quality improves, preserve the evidence; if it does not, remove rules that add context without changing outcomes.
Read the research paper
Checked August 29, 2026. Repository count, agreement figure and reported outcome differences come from the paper. The authors characterize the work as observational and hypothesis-generating. Configuration guidance is Musthave.ai analysis.
