How did Liverpool City Council deal with every IT manager’s worst nightmare, a blazing fire that sweeps through a workplace and leaves it little more than a blackened shell.

In the early hours of Sunday morning, 15 August 2010, that nightmare became a reality for Barry Dinham at Sydney’s Liverpool City Council. The Council’s Administration Building and Chambers were almost entirely destroyed when a huge fire engulfed the building. Desktops were disintegrated, servers disintegrated and many important physical records destroyed.

A fireproof strongroom at the heart of the building did its job protecting vital legal documents and council minutes, but even these required an extensive cleanup to remove smoke and water damage.

When Monday morning came around, the tough inquisition began. What had been destroyed? What was able to be restored? And had the Council’s Business Continuity Plan been sufficiently robust?

For Dinham and his colleagues, the charred ruins of their former workplace were a sobering sight. Crumpled desks and the twisted remnants of steel filing cabinets did not bode well.

While Dinham began to plan recovering the council’s business systems and restore working data from daily backups, the remaining 650 staff began to ask themselves where they would be able to pick up again when a temporary home was found.

While most of council business systems were up and running within a week at a new location, many found they had lost much of value.

There is no greater test of an organisation's document and records management platform than a disaster scenario like this. While the council has an EDRMS, it discovered that many staff had left important working documents on their desk when they left for home that fateful Friday, and no amount off effort was going to bring them back.

Liverpool City Council scans all incoming correspondence and stores it in TRIM, while email is handled by the Exchange server and staff are asked to transfer any messages of ongoing value in TRIM.

“That’s always a test of an organisation whether staff use the two systems separately or not,” said Dinham. “We ask our staff to place emails of ongoing value into the EDRMS but it’s fair to say we also interrogate the email system as well.”

The main problem with physical documents was experienced by council planning staff. While all development applications have been lodged electronically since 2004, standard practice is to print off large A0 plans so they can be read and annotated during the planning process. Once a file is closed, the planning documents and all the notations were stored in a large compactus. This was unfortunately located upstairs where the fire burned fiercest.

“Planning staff use working papers which is over and above what’s in your EDRMS, and that is the potential gap, where people may have made a notation for instance, that is not in the EDRMS,” said Dinham.

“The finished working files are stored on-site and off-site depending on type of document, we are in a transition to scanning the final working file.”

“The fire has emphasised the importance of recording electronic notations no matter what position you are in council. If there is any notation made on correspondence on people’s desk, we can restore the document but no action will be recorded as having been taken.

“These are issues for any organisation, you don’t need a fire to have that issue but it certainly highlights those issues.”

Liverpool Council is in the midst of a virtualisation project that has halved the number of physical servers from 40 to 20. It uses a combination of tape backup that goes off-site on a daily basis, as well as some tape to disk backup.

At the time of the fire a program to duplicate data at an alternate site was not yet completed. Once this is accomplished, data will be fully duplicated at other sites with some corporate systems available at the flick of a switch.
Unfortunately, the backup tape for the Friday before the fire was not taken off-site, so one full day’s worth of data was irretrievable.

Another unfortunate discovery was that the key for the off-site storage safe had been kept in the main administration building.

“In hindsight, one thing we could have done better was to share with everybody exactly what we were backing up. We are all human beings and some people weren’t saving information to corporate systems in the correct manner, they were bypassing systems and got found out after the fire, saving to hard drives that weren’t being backed up,” said Dinham.

“For new PCs we have been rolling out over the last 18 months we have been increasing security and only providing a small amount of storage available for people to manipulate data. Even if they don’t put it in TRIM we have been encouraging them to save it to network drives where it is at least backed up nightly.

“Anybody that used the EDRMS prior to the fire was very pleased that they did.”

There was also an issue with some data lost on external hard drives and USB keys.

“There’s a balance being big brother and banning them versus letting them use these tools,” admits Dinham. “The new generation is used to this technology and once you restrict that you restrict their ability to do work.
Archivist Keith Smithers picked through the rubble to find there had been damage to vital physical records inside an internal strongroom. Legal documents were covered in black soot and there was mould & water damage to the archives area.

“Scanning of vital records is an ongoing project,” said Smithers. “This incident has reinforced the value of electronic records and reinforced the importance of backing them up. It has also reinforced the value of fire rated rooms for vital [physical] records.”

The strong room contained vital records such as minute books, legal documents and certificates of title. A moisture probe was used to ascertain the level of penetration. Those records with 20% or more moisture were freeze dried in Melbourne, while those with less than 20% moisture were air dried.

“My recommendation to management is to scan in house or scan on demand using an external provider,” said Smithers. “Cost is a big factor.”

“Council also needs to make the decision whether to move to a fully electronic workflow. There is only a small percentage of files that get recalled, but those that do are needed again and again.”

Steamatic was engaged to freeze dry vital pre-2004 physical records aafter the authorised destruction of 800 cartons. That still left 7000 boxes to be restored and returned.

After the lessons learned from the 2010 fire, Liverpool Council is set to pursue a clean desk policy for staff and look to scan more physical files.

A compactus that had contained planning applications