A researcher at Anthropic found out about a successful exploit when the model sent him an email. He was eating a sandwich on a bench outside.
Anthropic released Claude Mythos yesterday. Beyond the engineer’s lunch, the model has the potential to eat software’s.
In testing, Mythos found a 27-year-old bug in one of the most secure operating systems ever built, & a 16-year-old vulnerability in video software that conventional tools had examined five million times. Mythos is Anthropic’s largest model, roughly 10 trillion parameters, six times the size of any previous frontier model.
From Anthropic’s red team report1 :
We did not explicitly train Mythos Preview to have these capabilities. Rather, they emerged as a downstream consequence of general improvements in code, reasoning, & autonomy. The same improvements that make the model substantially more effective at patching vulnerabilities also make it substantially more effective at exploiting them.
Security analysis was collateral output, a byproduct of optimizing for something else entirely.
This is the central question about increasing AI scale : what emergent properties will appear? We don’t know what other capabilities lie dormant in these systems. But we can project what will happen in business.
Access becomes kingmaking. Anthropic deployed Mythos under ASL-3 standards2 & granted access to more than forty organizations. Everyone else waits.
Project Glasswing, Anthropic’s gated release program, seems designed primarily for defense & hardening rather than commercial advantage. But that distinction won’t hold forever. At some point, the same capabilities that secure software will build it.
Hypothetically, CrowdStrike now scans for zero-days competitors cannot find. Apple secures its software while others cannot. The gap between those with access & those without isn’t a product feature. It’s a structural advantage that compounds daily.
Security posture inverts. Any system not protected by this level of analysis is now porous by default. Bugs that hid for decades surface in hours, but only for those with the tools to find them.
Pricing power shifts. This is no longer about margin on resold GPU hours. How much is it worth to secure your software against vulnerabilities no conventional tool can find? How much is it worth to be able to build at the new standard of enterprise grade?
Engineering budgets redirect. A significant fraction of AI tokens spent on software development will shift to hardening. Every company shipping code will need to scan it at this level of sophistication. Buyers will start to demand this level of hardening.
AI is breaking every system it touches : data centers, financial markets, security defenses. Software was lunch. What’s for dinner?
-
ASL-3 is Anthropic’s safety tier requiring the most stringent protections for models that substantially increase risk of catastrophic misuse. ↩︎