Boards need to treat the deployment of agentic AI as a delegation decision rather than a technology purchase, according to a new white paper from the Governance Institute of Australia. Unlike generative AI, which produces outputs for human review, agentic AI systems act on those outputs directly, transacting with customers and executing multi-step workflows with limited human oversight.
The white paper, Governing in the Age of Agentic AI, was released by the Governance Institute of Australia with input from law firm Mallesons, SEEK, governance software firm Diligent, EthicAI and the University of Melbourne's Centre for AI and Digital Ethics. It follows survey data cited in the paper showing just over half of organisations have already deployed AI agents.
The paper sets out five priorities for organisations deploying agentic systems: clear accountability, with an owner for every agent deployed; meaningful oversight through substantive human review at the systems level; defined authority boundaries on what agents can and cannot do; transparency and traceability of decisions that can be audited and explained; and capability uplift so governance literacy keeps pace with adoption.
Existing frameworks assume the wrong things
Daniel Popovski, senior policy and advocacy adviser at the Governance Institute, wrote in the organisation's Governance Directions journal that most existing governance frameworks assume predictable system behaviour, clear human oversight at decision points and bounded operational contexts.
"It is no longer sufficient to ask, is the output correct? We must now consider, was the decision authorised? Did it operate within approved boundaries? Can the organisation justify the outcome under scrutiny?" he wrote.
Popovski said governance must shift from model-level assurance, focused on accuracy, bias and explainability in individual systems, to system-level oversight recognising that risk arises from how agents interact across workflows, data sources and third parties.
He pointed to the risk of "automation bias", where humans defer to machine outputs as agents become more capable, arguing this requires deliberate governance interventions including training, oversight tools and escalation mechanisms.
Agentic AI does not create a new legal entity, the paper notes, meaning organisations remain responsible for their AI systems' actions and directors retain their existing duties of care and diligence.
"The key takeaway is to move forward, but ensure governance keeps pace so AI can be deployed with confidence and control," said Bryony Evans, a partner at Mallesons.
Regulators flag cyber and legal exposure
The white paper notes that ASIC, APRA and the Australian Signals Directorate have warned that as AI becomes more autonomous, risk moves faster and becomes less predictable, requiring stronger cyber and resilience controls.
Agentic AI introduces new legal exposure, from privacy and data breaches to agents entering into contracts or breaching regulatory obligations, and liability for agentic outcomes remains untested in Australian courts.
The full white paper is available from the Governance Institute of Australia.