AI Automates Critical Code Safety Checks

For decades, guaranteeing that software runs without errors under every possible condition has been a task so expensive and complex that it was reserved for high-stakes industries like aviation. The core tension in modern software development is that while AI can now write code rapidly, it lacks the mathematical certainty to prove that code is safe. This gap has been addressed by a new collaboration between Anthropic and Theorem, which has enabled the Claude model to automatically generate formal proofs of code correctness, fundamentally shifting the landscape of software verification.

The process relies on Claude’s ability to translate standard crash tests into rigorous mathematical problems. Instead of writing new software, the model analyzes existing code from 123 open-source libraries included in Google’s OSS-Fuzz security project. In a matter of weeks, Claude produced approximately 180 million lines of proof code. These proofs are then checked by an independent program called Rocq, which verifies their mathematical validity. According to Anthropic, this workflow requires almost no human intervention, effectively making the proof generation fully autonomous, although the underlying formal foundations were established by human experts at Theorem.

A significant benefit of this automated approach is its ability to identify hidden defects. During the verification process, Claude detected hundreds of bugs that conventional testing methods had overlooked. These issues often only manifest under specific, rare conditions, such as unusually large input files or specific sequences of data access. In one notable instance, the team found as many security vulnerabilities in a popular PDF reader in a single week as are typically reported in an entire year. The system also uncovered a logical error in another application that had remained undetected for more than a decade, demonstrating that even failed proof attempts can reveal critical flaws.

Despite these successes, the method has strict practical limitations. The label 'formally verified' does not mean the software is entirely error-free; the proofs only hold under specific constraints, including a particular processor architecture, single-threaded execution, and input sizes under 64 MB. Furthermore, the computational cost is significant, with individual proofs consuming hundreds of millions of tokens. However, efficiency is improving: newer model versions, such as Opus 5.5, are solving proofs with roughly 40 percent fewer tokens than previous iterations, suggesting that the process is becoming more viable for broader use.

The outlook for this technology suggests a major leap in software automation. If AI can both write code and prove its safety, developers can trust AI-generated software more, reducing the time and cost spent on manual reviews. Anthropic aims to make formal verification as routine as standard testing. Theorem’s roadmap already includes targeting critical infrastructure libraries like glibc, OpenSSL, and cURL, indicating that this technology is moving from experimental research toward practical application in essential digital systems.