The Digital Transformation Agency (DTA) has begun testing an artificial intelligence assessment framework designed to guide government departments in evaluating their AI projects.

The pilot program, which started in September 2024, involves government departments and agencies of varying sizes testing the draft framework. The initiative aims to help ensure AI systems in government services align with Australia's AI Ethics Principles while maintaining appropriate oversight.

Lucy Poole, General Manager of Strategy, Planning and Performance at DTA, describes the framework as adaptable.

“Our guidance is iterative. It is meant to change and adapt based on the shifting AI landscape within the APS,” points out Ms Poole.

“The framework and guidance are subject to amendments based on feedback from pilot participants and other stakeholders.

“This draft does not represent a final Australian Government position on AI assurance.”

The framework requires agencies to complete an initial assessment examining basic project information and potential alternatives to AI solutions. Projects identified with medium or higher risks undergo a more detailed evaluation, measuring the proposal against established ethical principles including fairness, reliability, privacy protection, and transparency.

"We want agencies to carefully consider viable alternatives," Poole says. "Non-AI services could be more cost-effective, secure, or dependable in some cases."

The framework forms part of the government's Policy for the responsible use of AI, which requires transparency statements and designated officials for AI projects. After the pilot concludes, the DTA plans to hold feedback sessions and analyze survey responses from November 2024, with wider consultation scheduled for early 2025.

Key considerations in the framework include data bias assessment, Indigenous data governance, and compliance with Australian Privacy Principles. Privacy impact assessments are required for high-risk projects.

The DTA emphasizes that the framework is designed to complement, not replace, existing legislation and practices governing AI use in government services.

The Pilot AI Assurance framework is available HERE.