AI is an accountability issue, not a technology one. Most organisations can say what an AI tool produced. Very few can say what evidence backs that result. As AI moves into everyday decisions, that gap is the real risk, not whether the technology works.
Across public and private sector organisations, the same pattern keeps showing up. Information governance is fragmented, information sprawl goes unchecked, and technology gets treated as the fix rather than an enabler. New tools get adopted faster than governance can keep pace, and records management gets bolted on after the fact rather than built in from the start.
AI magnifies these weaknesses rather than creating them. Fragmented governance means it reaches content nobody intended it to surface. Poor data quality and missing metadata mean it can't tell current from obsolete, so flawed answers arrive with the same confidence as good ones. Deployment is running ahead of readiness, and waiting for AI to settle down isn't realistic. It won't.
Five Questions Worth Asking
AI hasn't changed the underlying accountability principle information management professionals have always worked to. What's changed is the volume and complexity of what must be captured. Five practical questions test where an organisation actually stands.
■ Can you prove how an AI-generated recommendation was produced, if it led to a poor outcome?
■ Can you demonstrate what information was provided to the AI?
■ Can you show who approved the AI use case?
■ Can you explain why one design approach was chosen over another?
■ If a regulator asked for evidence in two years, what records would you produce?
A "no" to any of these points to a genuine accountability gap. What's left is memory, assumption or reconstruction, none of which holds up as a defensible record.
This is not a hypothetical concern. A recent NSW Audit Office review looked at AI governance across ten major state agencies. All ten maintain an AI system register, but only four have formally assessed their systems, and fewer than half have a documented AI strategy or policy. Maintaining a register confirms that a list exists. It says nothing about whether anyone could explain how a single entry on it came to be there.
Accountability in the AI era
In the AI context, evidence is central to accountability. Organisations need records that show how a decision was reached, why a tool was chosen, what data it relied on and how it was configured. Without those records, it becomes difficult to understand, explain or defend the outcome. Reconstruction after the fact is opinion, not evidence.
Retaining everything is just as unhelpful as retaining nothing. Five categories matter most.
■ Decision context — the business purpose and the authority to decide
■ Source information — the data and records relied on, and their lineage
■ Process artefacts — prompts, assumptions, system settings, model and version
■ Human oversight — reviews, challenges and the final decision
■ Metadata — provenance, ownership, version, access and retention
These five categories aren't unique to AI. They reflect the same principles good records management has always required. Capturing evidence as the decision happens, rather than reconstructing it afterwards, is what closes the gap. Recorded in the moment, it's proof. Reconstructed later, it's only opinion.
Governance That Holds Up
Governance isn't a document or a committee. It's the coordination of people, process and technology, and it breaks down the moment any one of the three is treated as the whole answer.
Without it, four familiar problems show up. Over-retention, because nobody wants to be the one who authorises disposal. Under-capture, when a decision gets made in a meeting or a Teams chat and the reasoning never makes it into writing. Disconnection, when systems don't talk to each other and context gets lost in the gap. Disengagement, when staff who can't see the point of recordkeeping simply work around it.
The fix is coordination. Named roles and decision rights, so accountability is explicit. Defined approval pathways, so good practice is routine rather than ad hoc. Systems built with access controls and auditability from the start, not bolted on afterwards. ISO/IEC 42001, the international standard for AI management systems, is a useful reference point for structuring that oversight.
Most organisations aren't starting from a blank page. Privacy, cybersecurity and records controls already exist in some form. The question is how to adapt them for AI, not whether to.
The Foundation
Governance can only be as good as what sits underneath it. Most organisational information is unstructured, duplicated and fragmented across systems, captured with inadequate metadata if it's captured at all. Metadata is what makes an AI outcome explainable. It shows why an output was produced, not just what it says, and traces that output back to its source so a decision can be explained and defended.
None of this is unfamiliar to records and information professionals. Classification, description and controlled vocabularies have supported discovery, accountability and compliance for decades. What's changed is that these same disciplines now decide whether an organisation can trust, explain and defend its use of AI.
The Takeaway
Data governance for the AI era isn't a new discipline. It's records management, extended to cover why a solution was chosen, what data and model it used, and who approved it. Most organisations aren't capturing any of that today, and it needs to be recorded as the system is built, not reconstructed months later.
The winners will not be the organisations with the most AI. They will be the ones who can prove they used it responsibly.
Adelle Ford and Annette Senior are Directors of RKI Consulting Pty Ltd, a records and information governance consultancy. Learn more at records.com.au or follow on LinkedIn for practical insights on AI governance and record-keeping.