Google says AI agents now stop hundreds of vulnerabilities each month inside its development pipeline. The important lesson is not a claim of autonomous security, but an architecture that gives agents narrow jobs, strong context and a human verification boundary.
Google has described how it uses AI agents to review code changes across hundreds of millions of lines of code. The system combines presubmit checks, vulnerability discovery, rapid triage and suggested fixes. Google also released Mantis, an Apache-2.0 demonstration repository that shows how an agent can reason about vulnerabilities with tools and a sandbox.
The four jobs in Google’s security loop
- Review each change: inspect code before it enters the main branch.
- Hunt across the codebase: search for patterns and variants that do not arrive in a new pull request.
- Triage findings: add repository context and decide whether a signal is likely actionable.
- Help repair: suggest a patch that an engineer can review and test.
The division is useful because a model good at identifying a suspicious pattern is not automatically the right authority to merge a fix. Each stage can have different tools, permissions and evaluation criteria.
What Google reports
Google says its agents prevent hundreds of vulnerabilities per month. It reports that false positives fell to 3% in some deployments and that one triage system reached more than 92% precision in under a minute. Those are Google-reported operational results. The company has not published enough case-level data in this announcement for independent reproduction across another organization’s codebase.
The numbers still show what production teams should measure: prevented defects, precision, time to decision and the burden placed on reviewers. Counting comments alone would reward an agent for producing noise.
A useful internal report should also separate vulnerability classes and repositories. An agent may perform well on a repeated injection pattern and poorly on authorization logic that depends on business context. Track results by language, component, severity and root cause, then keep a stable hidden evaluation set so prompt changes cannot improve the score by memorizing the test. Reviewers should label missed defects as carefully as visible false positives; otherwise the dashboard measures annoyance rather than security coverage.
What Mantis actually gives developers
The Mantis repository is a reference implementation for security-agent experimentation. It demonstrates an agent loop, analysis tools and an isolated execution environment. Google explicitly warns that the repository is a demo, is not supported for production and should run in a restricted environment.
That warning is part of the product story. A security agent reads untrusted code and may execute commands while investigating it. Running the agent with broad credentials or access to production secrets can turn a defensive tool into a new attack path.
The approval boundary that matters
A sensible production design lets the agent read the smallest necessary repository scope, execute only inside an ephemeral sandbox and propose changes through the normal review system. Humans remain responsible for accepting severity, judging business context and merging patches. High-risk repositories can require a second security reviewer.
This is similar to the permission lesson in our Grok Bot cloud-computer guide: capability is not the same as authority. Our Google agent sandbox report explains why isolation and persistent state need separate controls.
A production scorecard
| Metric | Why it matters |
|---|---|
| True positive rate | Shows whether important vulnerabilities are found. |
| Precision | Measures how much reviewer attention is wasted. |
| Time to triage | Tests whether automation shortens the queue. |
| Patch acceptance rate | Separates useful repairs from plausible text. |
| Regression rate | Checks whether suggested fixes break behavior. |
| Sandbox violations | Reveals attempts to exceed tool or network policy. |
How to pilot the pattern
- Choose one vulnerability class and one repository.
- Run the agent in report-only mode beside existing scanners.
- Hide results from developers until a security reviewer labels them.
- Measure misses and false positives against the same test set.
- Enable patch suggestions only after detection quality is acceptable.
- Keep merge authority in the existing protected-branch workflow.
What this release does not prove
Google’s report does not prove that an agent can replace a security team, that Mantis is production-ready or that the reported precision will transfer to another language and repository. It does show a credible production pattern: combine specialized agents with rich internal context, isolate execution, measure precision and keep people in the final decision path.
Primary sources: Google Cloud’s engineering report and the official Mantis repository.
