The Art of Archiving Data

The art of archiving data has matured. What options do you have?

Archiving in the olden days meant moving paper records from the back office where they were easily retrieved to a separate storage location where they gathered dust, unread unless someone traveled to the archives to read it. Archiving introduced a physical distance and a delay to retrieving archived records. In the high tech world of today, archiving records can take several forms.

  •   Separate Systems

This is the digital equivalent to the archive vaults of old. Archived data may reside on a backup server, never accessed by personal unless they specifically search the backup system. Archives may exist only in system backups or in specifically saved archives for legacy data. In the case of legacy data, these archives are often limited to the data that must be saved for so many years due to legal requirements, such as employee lead exposure data, medical records of those who died under specific conditions or per a government contract that mandates as-shipped part configurations be kept for X number of years. Using a separate IT system for archived data ensures it is available if the main production server is down, but this increases the difficulty of moving data from the archive back to production. However, moving data from production to archive is often part and parcel of the data backup process.

  •   Same Server, Separate Folders

In this model, data must be moved to a separate folder of sorts when archived. Individual records or whole folders of project data are moved to archive. The product data set is separate from the archived data. Searches generally search production, but users can select to search archived data as well. This model of archiving only requires a single server, but archives are unavailable when the server is gone. System administrators and even data managers may be able to move data to and from archive at will. However, administrators may receive calls to restore data from archive when required, especially if there are customer returned units whose data was archived or programs archived by mistake.

  •   Same Server, Separate Partitions

With separate partitions, archived data resides on the same server as the production data. However, archived records are not included in database searches unless the searcher selects to include archived records in the search. The advantage of this method is the flexibility of the archiving method. Do you want all records over a specific age archived, or shall all versions of a record except the latest version be archived? Users benefit from faster report generation and system searches without losing access to the archived information. IT can save time by making archival automatic, such as automatically archiving as-built information when the unit is shipped. However, this model of archiving requires careful planning of the data architecture, robust testing and careful supervision. Failure to find faults with the partition logic could result in production data archived as it is updated or even created.