Australia to Build AI Safety Regulator, Force Labs to Report Incidents

Albanese announced on Tuesday, November 10, that Australia will stand up a dedicated AI safety regulator and turn voluntary practice into hard rules for frontier AI companies, according to the Prime Minister's office and Bloomberg. The government accepted all 18 recommendations of a rapid review launched after an OpenAI agent slipped into a Medicare statistics site. The regulator will sit alongside the Australian AI Safety Institute and, in the government's words, can hand down 'significant penalties' — no dollar figure published.

The duties land on the model developers, not the founders building on top of them. Frontier companies must maintain security controls through development and deployment, hunt down and manage serious risks, open their models to independent evaluation, and publish their safety and security frameworks. They also have to notify the government of incidents 'rapidly' — a word with no hours or days attached to it yet. SBS News reported the duty covers AI security breaches and potential breaches of criminal law.

The origin story is a disclosure gap. Reuters reported in September that an OpenAI agent had reached the Medicare Statistics Reporting Service portal and interacted with sites run by the Australian Institute of Health and Welfare, Victoria's Department of Health, and the NSW Bureau of Crime Statistics and Research. Outlets disagree on when the access happened, but agree the authorities weren't told until September. Albanese called that 'unacceptable'. OpenAI said the access was unintentional and that it found no evidence patient records were touched.

That silence is the case for a statutory clock. Under today's voluntary setup, a lab decides for itself what counts as reportable and when to say it. At a Sydney hearing, OpenAI chief strategy officer Jason Kwon said the company would back a mandatory disclosure framework, and Anthropic has signalled support too, according to coverage of the hearings. Agreeing to the principle is the easy part — the fight is over the trigger and the deadline.

Albanese framed the larger aim as ensuring 'we shape AI rather than AI shape us', and said the law must get ahead of, and in places catch up with, a fast-moving world. Timing stays loose: the broader AI standards legislation is due in Parliament in early 2027, and the Prime Minister's release gives no enactment date for the regulator, saying detail follows consultation. That plan traces back to July, when the government said a new Office of AI inside the Prime Minister's department would draft the standards.

The nearest comparison is Europe. Under the EU AI Act, providers of general-purpose models with systemic risk must report serious incidents to regulators, with fines reaching 3% of global turnover. Australia's version looks narrower on its face — aimed at frontier labs rather than a long list of high-risk uses — and it hasn't defined a reportable incident or set penalties. The US has no equivalent federal reporting duty, and no other government has announced plans to copy Canberra.