Will my routine data queries initiate a cyberattack?

Australian Prime Minister Anthony Albanese announced yesterday that an OpenAI agent had gained unauthorized access to non-public data from the country’s Medicare Statistics Reporting Service Portal, in what is being described as the first (known) attack by a misaligned agent on non-public data stored on a government website. While the data involved were described as non-sensitive, aggregated statistics, the agent also reportedly wrote files to the server, which is run by Services Australia. The attack occurred in June, but the Australian government did not become aware of it until OpenAI notified them via an email to a general government inbox in early September.

OpenAI claims the attack took place in the context of an internal evaluation involving internet research on public medicine spending in Australia. We don’t have a lot of technical details on this attack yet, so it remains to be seen how sophisticated it was. We do, however, have a much more detailed technical report from Transluce on another set of incidents, including one involving the Australian Institute of Health and Welfare’s Tableau collections.

The task leading up to this attack on the Australian health agency involved fetching the “January 2022 rolling-12-month-average government cost per person for Dermatologicals across Victorian LGAs”, an innocuous research task. The agent was blocked from accessing the public dashboard by anti-bot controls, and so resorted to testing proxies and probing for cross-site scripting vulnerabilities. Ultimately, the bot successfully evaded the anti-bot controls and retrieved the public dataset from a pre-production server. Since the dataset itself was public, this was not really a data compromise in the same sense as the Medicare portal attack. But it does give us a sense as to how AI agents can escalate a boring data retrieval task into a cyberattack without anyone really asking.

These two cases involving Australian health statistics involved OpenAI research agents undergoing training, not ordinary production agents available to users. But the tasks these agents given were completely. A few days ago, I wrote about how someone has to be responsible when autonomous AI agents do crimes. But what if that person is me? I could totally imagine myself innocently asking an agent to retrieve some statistics that I reasonably expect to be public, only for it to start probing the website for vulnerabilities because it got blocked by Cloudflare or a statistics portal login or something.

Besides, there is already precedent for a production agent turning a mundane task into something malicious. Last month, a man in Melbourne asked his OpenClaw agent to book a morning class at his local gym. Morning classes were popular and difficult to schedule, so the bot exploited a bug in the booking system to snag a class weeks before the user was supposed to be able to book it. The user later asked whether the agent could move him from fourth place to the top of the waiting list (admittedly, this request was pushing it somewhat). The agent then discovered that the gym’s API allowed bookings to be cancelled without authentication and tested this capability by cancelling the booking of the person at the front of the queue. He had not asked it to do that. He had, however, unwittingly bumbled his way into committing a computer crime.

I’m trying to think of how to tie this post into a neat little bow, but I can’t, really. Someone has to be responsible when an agent commits a crime while carrying what seems on the surface like an innocent request. And if it comes down to prosecuting the guy with the innocent data query and a ChatGPT subscription (and no lawyers) or the giant corporation that built the agent (and all their lawyers)…well, I know who I’d bet on.