VulnCheck audit finds Anthropic's Project Glasswing generates 7.5x more duplicates than fixes

A VulnCheck analysis by Patrick Garrity examines the seven-month track record of Anthropic's Project Glasswing, which uses AI to identify security vulnerabilities in open-source software. Of 37,966 findings reported by the project, only 19.4% (7,349) have been disclosed publicly, and merely 217 have resulted in fixes. The project has accumulated 1,654 duplicate reports (4.4% of total findings) and 419 withdrawn findings, creating a triage workload that falls on already strained open-source maintainers.

The analysis questions Glasswing's claimed 94.2% true positive rate, noting that this metric appears disconnected from the actual volume of duplicates and withdrawals. Garrity acknowledges AI's potential value for vulnerability discovery but concludes that the project's operational results do not align with its launch narrative, with CVE output remaining a small fraction of annual totals and few findings showing evidence of active exploitation.

Glasswing's triage metrics and maintainer burden

• Volume vs. resolution ratio: 37,966 total findings have yielded only 217 fixes, representing a 0.57% fix rate against total reported issues.
• Duplicate accumulation: 1,654 findings (4.4%) marked as duplicates, exceeding fixes by a factor of 7.5x and indicating significant noise in AI-generated reports.
• Withdrawal rate: 419 findings have been fully withdrawn, suggesting initial classification errors or invalid automated analysis.
• True positive metric discrepancy: The claimed 94.2% true positive rate, up from 90.8% at launch, appears inconsistent with the volume of duplicates and withdrawals, pointing to a potential feedback loop gap in triage accounting.
• Maintainer impact: The triage burden for validating, deduplicating, and responding to findings has shifted to open-source maintainers rather than being absorbed by the reporting organization.

Why it matters

For Linux kernel and open-source project maintainers, the Glasswing case illustrates a growing risk: AI-powered vulnerability scanners can flood projects with unverified reports that demand manual review without corresponding engineering resources to validate or remediate them. The mismatch between reported findings and actual security outcomes suggests that automated disclosure pipelines need tighter integration with upstream triage workflows before they can reduce, rather than amplify, maintainer workload.