Claude users up to no good · ↗ www.anthropic.com

Anthropic released their “Detecting and countering misuse of AI” report yesterday. I’ve only had a chance to skim the report so far, but there’s a ton of interesting (and scary, mostly scary) stuff in there. A few highlights:

• A Yemeni militia using Claude in lieu of an engineering team for their guided rocket and missile program
• A contractor working for Mali’s intelligence service using Claude to build a mass surveillance system of the country’s mobile phone network
• Russian spies using Claude to automate cyberattacks, including automatically rebuilding their compromised malware until security software could no longer detect it
• A PLA-affiliated actor sending Chinese surveillance data about a targeted individual to what they thought was Moonshot’s Kimi, but was actually Claude because Moonshot was secretly routing customer queries to Claude and saving the exchanges for model distillation
• A proposal for gain-of-function research on chikungunya virus to be pursued at a military research institute, as well as another researcher planning gain-of-function work on mammal-adapted avian influenza
• A grant application for orthopoxvirus research at a state-associated infectious disease lab, submitted through a reseller relay using anonymizing infrastructure to covertly access Claude

Anthropic doesn’t say that these cases of biological research demonstrate intent to harm, but they do show that AI is dramatically lowering the bar for carrying out potentially very dangerous research (to call back to a previous post, imagine if the terrorists in Executive Orders had a team of agents to help them optimize their super Ebola!). I may be biased due to my longstanding fascination with infectious diseases, but bioterrorism (or biological warfare by a nation state) strikes me as the greatest near-to-medium-term AI risk. I am in agreement with economist Noah Smith (and many others) here.

Anyway, Anthropic’s mega-IPO is expected to drop next month. Happy hunting!