The primary bottleneck in open-source software security has become not the ability to detect a defect, but the time of human maintainers capable of verifying the bug and releasing a fix. Statistics from Anthropic's initiative clearly demonstrate this barrier: the advent of advanced language models like Claude Mythos allows generating thousands of bug reports, yet maintainer bandwidth remains a severe constraint.
What Anthropic's Security Funnel Revealed
A catchy headline circulated across tech media: a neural network uncovered nearly 30,000 vulnerabilities, but only 516 were fixed. Such a comparison creates a false impression of extremely low efficiency (around 1.8%). However, the actual vulnerability disclosure funnel works differently: 516 patches correspond to 6,157 reports actually submitted to open-source maintainers. The real fix rate for reported bugs is 8.4%.
According to Anthropic's coordinated vulnerability disclosure dashboard accumulated over six months of operation, the process is divided into two key stages.
Stage 1: Filtering Initial Findings
Models generate a colossal volume of hypotheses, but only a small portion reaches human reviewers:
| Verification Stage | Count | Share of Total |
|---|---|---|
| Candidates from neural networks | 29,439 | 100% |
| Reviewed by six external security firms | 6,123 | 20.8% |
| Confirmed by auditors as real bugs | 5,674 | 92.7% of reviewed |
| Remaining in queue for analysis | 23,316 | 79.2% |
Stage 2: Maintainer Response
Even a confirmed vulnerability does not mean an update automatically appears upstream:
| Interaction Status | Count | Explanation |
|---|---|---|
| Sent to developers | 6,157 | Covers 591 projects |
| — with independent auditor report | 1,333 | 21.6% of submitted |
| — direct feed without manual triage | 4,824 | 78.4% of submitted |
| Response received from project team | 5,103 | 82.9% response rate |
| Fixed upstream | 516 | 8.4% of submitted |
| Registered security identifiers | 584 | Total of 219 CVEs and 365 GHSAs |
How the OSS Scanner Service Works
Based on accumulated experience, Anthropic launched the OSS Scanner tool, modeled after Google's well-known OSS-Fuzz project. The service targets open-source repositories critical to infrastructure, and submissions are accepted via pull requests to the anthropics/oss-scanner repository.
Key features of the scanner:
- Complete isolation: code is analyzed inside virtual machines with no network access.
- Automated report generation: human involvement on Anthropic's side is eliminated. The model itself constructs a reproducible scenario, attempts to find the offending commit using git bisect, and drafts a proposed patch.
- No rigid ultimatums: project authors are not currently subjected to the standard 90-day forced public disclosure deadline.
Why Fixes Are Far Fewer Than Findings
The gap between discovered issues and closed tickets is driven by the nature of open-source software maintenance, not merely the performance quality of neural networks:
- Time factor: the cycle of preparing, testing, and releasing an update takes months. A bug accepted for work in early autumn simply cannot result in a release by the time the statistics are tallied.
- Threat model conflicts: about 93% of the bugs reviewed by external firms are technically valid, but some are marked as "won't fix" by maintainers. A bug may reside in unreachable code or fall outside the scenarios that the project considers hazardous.
- Severity overestimation: third-party auditors and project authors frequently downgrade the severity level assigned by the model. This discrepancy occurs because AI does not always accurately grasp the architectural context of a specific library.
- Integration overhead: a drafted patch still requires regression testing and backporting to supported branches, imposing an additional burden on maintainers.
Despite the modest final fix rate, representatives of major projects like PostgreSQL, OpenSSL, and curl note that the quality of autonomous AI reports has improved dramatically compared to earlier generative experiments in previous years. Now, they genuinely help eliminate critical defects before malicious actors discover them.