The long term digital archiving dilemma

By Mark de Berg

Government records and archive authorities around the country have developed a range of policies, standards and guidelines to help organisations better manage the capture of digital information. However, there appear to be some major structural obstacles preventing effective digital archive appraisal and permanent records transfer. 

While guidelines are available to allow government agencies to develop valid retention and disposal authorities, there are significant practical challenge to the appraisal and sentencing of digital records.

For instance, consider the following questions: 

When your organisation implements a new line of business system, are records staff consulted about the type of information intended to be stored?

Is due consideration given to the metadata captured and maintained to ensure ongoing context should that information be migrated to another system like a digital archive at the end of the information lifecycle? 

When your electronic document and record management system (EDRMS) was implemented or configured for the management of digital records did you stop to think that you should create and maintain a metadata schema? 

I think that the answer to these questions would typically be a resounding “NO!!!”

The onset of EDRMS systems has actually resulted in the decentralisation of record-keeping responsibilities. It has meant we now have a large variety of skill sets across an organisation undertaking records management tasks with varying levels of success and compliance. Training provided when a system is implemented is often not maintained as roles and responsibilities change and new staff are inducted. 

Quality not assured

Records staff rarely get the resources to undertake ongoing quality assurance checks to address staff’s “good attempts” at records management. This can result in the files containing a myriad of information that does not always match the intended use or classification when the container was first created.

To prevent the EDRMS  system turning into an unstructured mess, some advocate the application of classification at the document level. However, in my experience this tends to push users away from using the EDRMS and back into share drives and offline repositories.

Others advocate the sentencing of containers based on the original classification and free text title ignoring the content of the file. They see this as an acceptable risk and have even advocated that the record owners who signs off on the destruction and transfer lists are responsible for checking the files contain what they purport to.

Others in the industry follow tried and proven physical appraisal practices where each document is checked before the file is sentenced. The ability to undertake online appraisal is limited without the online reviewer having access to the appropriate software or application expertise available to view every document.  Adding to these challenges, digital records that are identified as needing to be retained permanently will then need to be subjected to conversion of the digital object and the packaging of the metadata into a format that meets the requirements of the relevant Archive Authority. 

In addition to the corporate EDRMS, are Information Managers now going to take responsibility for the appraisal of records within other organisational systems and the associated extraction of records required for permanent retention with the associated metadata and metadata schemas? 

The National Archives of Australia is now mandating that from 2015 all records born digitally are to be maintained and, if appraised Retain as National Archives (NRA), ultimately transferred digitally. The NSW State Records Authority plans to launch its digital archives solution in 2014. 

How will this appraisal and sentencing of digital records be done? 

Mark de Berg is Product Director – Records Management at Converga, the business process outsourcer (BPO) subsidiary of New Zealand Post.