As developers turn to AI models to generate an increasing amount of code, they are also relying on the systems to find vulnerabilities and generate patches. Unfortunately, the models just aren't very good at their jobs.

Recent research suggests that even the latest AI systems only produce effective patches about half the time, according to a report published on Aug. 6 by identity management firm 1Password. To test their efficacy, the company's Off-By-1 research team generated 540 patches for six vulnerabilities disclosed since March using two different large language models (LLMs): OpenAI's ChatGPT-5.5 with Trusted Access for Cyber, and Anthropic's Opus 4.8 with Cyber Verification Program. More here