Conference Papers Online Session 5: The Archival Control of Systems Systems is systems: archival documentation regardless of format. Lisa Enright Australian Science Archives Project The digital age has raised many questions within the recordkeeping profession and led to major reinterpretations of the role and the value of archival work. Gerald Ham, in 1981, described the work of the archivist as now being: determined by the way our society records, uses, stores, and disposes of information. We see that the current revolution in information processing is inexorably changing our world and our work, pushing us into a new period in archival history.[1] He goes on to say: Our effectiveness as archivists in this new era depends on our ability to alter our past behavior and to fashion new strategies to cope with both the opportunities and the problems created by this revolution.[2] What the profession is required to do, if it is to take on a leading role in the digital information environment, is to be unafraid to question our past practices and behavior in line with this new environment of changing technology. What we must take into consideration in light of the modern environment, are those very points made by Gerald Ham and many others here at this very conference. Do we throw away what it is that we have learned over decades (centuries ?) of recordkeeping practice, just because our surrounding environment has changed ? Or, as part of the evolution of society and culture, do we ‘refashion’ our thinking, our strategies and our tools, to cope with the changes in our environment ? The Forest or The Trees ? During this short paper I would like to put forward for your consideration just one of the varied examples of the information keeping systems that ASAP has confronted in its work, an example that I am sure many of you will be able to draw comparisons with from your own experiences in dealing with records in an environment in transition. Within this paper I hope to be able to show that daily each of us is faced with the many theoretical and methodological issues that have been the core focus of this conference. And we have dealt / are dealing with these issues to varying degrees of success. The work that I am referring to in this paper is that dealing with the many issues of what to do when faced with the request to ‘archive’ digital systems or data that remains as a legacy of a number of decades of rapidly developed and hastily implemented computerised technology. Before I continue I should like to point out very clearly that this is a particularly contemporary problem, it is a transitory problem and it is one that perhaps only one, maybe two, generations of our profession will need to concern themselves with. As with all periods of great change, those that stand on the crux are forced to think, evaluate, debate and decide on issues that, to those who follow in a decade from now, will simply take as a ‘given’. "The Archival Control of Systems", as this session is titled, refers very neatly to what the function of the archive is when looked at from a slightly more removed perspective than the focus on the ‘record’ item itself. If we step back and look at the forest rather than the trees we see that the archival function has been to document systems of information control regardless of the format of the information or record. Analogue or digital, as the most familiar to our present situation, do not dictate or alter the ultimate outcome of what archivists aim to achieve. A system is a system if it controls, provides entry to or holds content. Access to the understanding of a system is a fundamental aim regardless of format issues. The hypothetical basis for the work that ASAP undertakes is that the documentation of context, content, structure and use have been the foundations for the management and maintenance of archival records in analogue formats, and that the same foundations should therefore be applicable for the maintenance of archival records in digital formats. The best way to ensure that information created, stored and managed in digital systems that have since become defunct (i.e., legacy systems), remain viable and useable into the future, is to ensure that contextual documentation is gathered, analysed and maintained along with data as it travels along the path currently identified as migration. As with analogue records, not all digital data will be required into the future, with some circumstances, for example: technological redundancy leaving data inaccessible, requiring that the maintenance of the contextual information will be valued above that of the data itself. The work that ASAP has thus far undertaken in this area has therefore, been as much a test of the concepts surrounding information value, as it has been a test of how to provide a practical way to maintain, preserve, enhance and make available, records in a digital format for future use. The facts are that almost all aspects of commercial, scientific, research, academic and public life have moved quickly to embrace the new technologies of the computer industry. This means that we are already starting from behind the eight-ball, for as a profession we have only recently (in the scheme of things) begun to acknowledge the importance of the records that have been created over the past three to five decades in an electronic environment. The work that we have done in the area of legacy digital systems has been designed to explore the documentation requirements necessary to ensure the ongoing access to information. Work has included the completion of Accession / Series / Provenance / Inventory documentation by ASAP, and the recommendation of the ongoing information value of data available, in other words ‘appraisal’. In particular, much of the work done has had to explore the use of a system designed to document analogue records, to deal with the documentation of digital information and systems. The System as a Record It should be noted at this time, that the systems which have been reviewed by ASAP in each case have not been identified as recordkeeping systems. They were not designed to create, maintain or provide access to ‘records’ of transactions, and therefore do not fulfill the requirements established to ensure their integrity and immutability over time. The systems that have been reviewed were information systems designed to fulfill various functions: administrative, technical, scientific and creative. They were designed to record and provide access to information for specific purposes at specific times. To the user it is therefore, the informational value that is of most importance for future access. Unfortunately, non-recordkeeping systems are the norm within organisations and reflect the dominant database design methodologies which treat redundant data as wasteful and contributing to inaccuracy. The timeliness and reusability of data is prized over its utility for organisational accountability. Such systems do not produce records because they were never intended to do so.[3] What we have been faced with is a mire of systems both mainframe and PC based, that consist of commercially available software systems, systems designed and maintained by the information technology department of various organisations, and functionally specific systems used and created by persons whose knowledge of system design and documentation varied widely in its expertise. All of the systems that we have considered for review have undergone changes, upgrades, migrations and function modifications over the time of their use. Documentation of system design we have learned is often patchy, and the structural and political changes that have been a part of the organisational landscape, particularly of the 1970s and 1980s, have contributed also to the dearth of system documentation now available. It is, therefore, impossible to guarantee the authenticity of the data held within the systems under review other than to ensure that it is unable to be altered after documentation. Although the legacy digital systems that ASAP have dealt with are not recordkeeping systems this does not mean that the processes for their documentation should be different, or the ultimate intention altered, from that which would otherwise have been undertaken for recognised recordkeeping systems. What is being created in the process of data transfer is a transaction; the creation of a ‘record’. What must, therefore, be documented are all areas of that transaction process, giving context to that ‘record’. In dealing with electronic legacy systems it is, therefore, necessary to document as much as possible of the system creation, maintenance and use in order to build a picture that will represent the integrity of the system. Because there is no physical nature to records in a digital format, and the use of digital systems can change more fluidly than paper based systems, there should be additional emphasis placed on the documentation of the function of a system within an organisation, and the function of the control groups within the broader corporate environment. By moving the focus of archival documentation in a digital environment to a more scrupulous consideration of the notion of function the products of documentation process can be utilised more fully in their intended role, as providing access to an understanding of past practices. The notion that documentation with this focus will provide the facility to map system and organisational function and allow for a greater ability to identify changes that occur that may put at risk the reliable creation, documentation or management of records, requires that there is an understanding of the fundamental nature of archival principles and the role of the archival process. This role, as providing insight into the reality of the past, will then be able to have a genuine impact on the practices of the future. This area of utilisation, however, requires significant further research and testing before its application can be fully appreciated. What has ASAP has attempted to do is document as many of the structural, organisational and systems changes that have occurred, and to ensure that the data that is captured as the final active order of these systems is done so as to guarantee the integrity of the data at the closure of these systems. The attempt to create a picture of the total record environment through the linking of data held within electronic systems and those created and maintained within a paper environment, also works to provide supporting documentation as to the authenticity of data. Creating the Picture As our society moves further into a time of technological development that will take us far beyond the reliance on paper formats of the past, the archivist role as ‘keeper’ of the past is becoming less and less applicable. This does not mean that we discard all that has been learned, all that has been the foundation of the profession for so long. Rather, what this means is that the archivist is able to take on the role of ‘documenter’ of the reality that is our history, the reality of the past, the reality of the present. And it is the preservation of this role, and its many outcomes, that becomes the core of our professional mission. The documentation processes used by ASAP have shown that the principle foundations of archival methodology are more than adequate to document, and facilitate access to, the digital environment of legacy systems. The nature of the archival process is not to stop change, but to ensure that change is fully documented to give the user, in the future, a complete and clear understanding of the past. The methodology used by ASAP is based in the belief that accepted archival documentation principles, built up over the past century, are indeed sound principles developed outside the constraints of physical format. This belief has been substantiated by the use of the ASAP onQ ADS, a system conceived within the Series model with an understanding of the continuum nature of records. The use of the ASAP onQ ADS provides the application whereby records within a digital environment can be linked with records in an analogue environment through the tool used for access. This has required that the same structure and process be maintained for the collection, documentation and utilisation of contextual information regardless of format. In order to do this much consideration was given to the defining characteristics of analogue records documented within the system. These essential characteristics have been refined even further with consideration of the principles of archival documentation, consideration of why we document, and consideration of the fundamental nature of ‘the record’. What was established through the development of this methodology was that the fundamental principles of archival documentation would remain firm regardless of the interpretation of documentation requirements used or the manipulation of the tools to gather and access this documentation. The development of the documentation techniques used by ASAP to deal with legacy digital systems indicates that the use of standard archival documentation is detailed enough to provide both ongoing access to information and ensure its integrity over time. There is significant scope, however, for the redefinition or manipulation of standard definitions for the collection of contextual documentation and these definitions should constantly be reviewed in order to meet the needs of the varying formats for record creation and capture that will develop in the future. This reinterpretation of conceptual definitions is necessary if the archival profession is to be able to fulfill its mission as documenters of the reality of social, cultural and organisational memory. Such reinterpretation is what we are all here at this conference to discuss. The Archivist and the System What has been the traditional role of the archivist as one who comes in after the fact and ‘reconstructs’ the system, controls, context, and even content of records or recordkeeping systems, is in fact the role that is played when dealing with legacy digital information / record systems. This is the case for it is usually not until the decommissioning of a system that it is thought that what the system contains may be of continuing value. Often it is not until this time that the archivist is even made aware of the existence of the system. Technical, maintenance, experimental, testing, analysis and recording systems used in industry, science, hospital and university environments are not viewed by their owners / users as recordkeeping systems - "they are things that only exist in administration areas, you know, finance and all that kind of stuff." The primary focus of professional discourse in the area of digital records has been the development of standards, system functionality, and process for the creation of digital records. There has been little consideration of that which already exists in ad hoc systems all over the world. The records created over the past decades do not appear to warrant the same degree of interest as those that will be created in the coming decades. The scarcity of available literature and lack of practical precedent reflects not only on the difficulties inherent in dealing with digital legacy systems, but also the fact that as a profession recordkeepers have only recently taken on board the notion of corporate memory in a virtual environment. The argument that must be used strongly and consistently, is that no matter what the format, the information that is the memory of any organisation is an asset and must be valued as such. The archivist is at the end of the food chain when dealing with legacy digital systems. This is a role that is comfortable for some and unavoidable for many. But it is a place that the recordkeeping professional will not hold in the future as is seen in so many places where the design of digital systems is part of the archival position description. The dual nature of the work is however part of the reality of the contemporary archivist for if action is not taken now to safeguard the digital record of yesterday there will be no way that the archivist of tomorrow can compensate. Conclusion With little precedent to follow in dealing with legacy digital systems, what ASAP has undertaken has been viewed by some within the recordkeeping profession as an ‘archeological dig’. This may well be true. The basis of any work undertaken with legacy systems is that you are dealing with electronic systems that had been created using the best knowledge of system design, implementation and maintenance available at the time. These systems are usually not designed as ‘recordkeeping systems’ and as such did not create records with the metadata that we would hope to assign to them within a system that has been recently developed. These systems were designed to fulfill a need, which they did quite adequately for this time. Unlike a physical dig, however, the virtual nature of digital systems does not have the wealth of precedent available to us when dealing with analogue records. As was revealed by a detailed literature search there is little in the way of practical insights available within the research that is currently forming the basis for discussions of records in digital formats. ASAP’s work shows that existing archival practices do provide a firm foundation for dealing with digital legacy systems, however, we must be careful that we do not define electronic systems using strictly the same interpretations that we attach to records or systems in analogue formats. What we must ensure is that the documentation of the ‘record’ encapsulates all that is necessary to guarantee its integrity into the future. In the case of legacy electronic systems this includes the documentation of the transaction of migration as the ‘record’. In one of my favourite novels by James Cowan, Fra Mauro comments on "the value of experience as being an important guide in our quest."[4] With so little available in the area of electronic legacy systems, ASAP’s work has been testament to this thought. This, however, should not remain the case with more research and investigation required in this arena if we are to ensure that our knowledge of digital records is to be complete and comprehensive. The archival profession can not afford to deny the need to deal with digital information created over the past five decades. Nor can it afford to discuss the issues for too long before action is taken. Unlike an analogue environment we can not rely on benign neglect to ensure that at least something of the corporate and social memory of the late 20th century survives. Before too long we must look to the forest or there will be no trees left to admire. I see that "we are obliged to make mistakes if we wish to attain to any degree of knowledge."[5] And so, we must take each situation, each system, each data set, as a new challenge to be looked at a fresh in light of what we have learned. Action, based on the concepts of sound archival principles can not prove to be entirely wrong, even if it is not entirely right. But some action must be taken now even if that action results in merely bidding time until more long term solutions to record documentation and migration are established. As Fra Mauro says so wisely: Conceivably [we must] . . . accept that every error that we make is one more brick fired in the kiln of grace.[6] [1] Gerald Ham, ‘Archival Strategies for the Post-Custodial Era,’ The American Archivist, vol. 44, no. 3, Summer 1981, p. 207. [2] ibid., p. 207. [3] David Wallace, Managing the Present: Metadata as Archival Description, http://www.lis.pitt.edu/~nhprc/Pub10.html [4] James Cowan, A Mapmaker’s Dream, Random House, Australia, 1997. p. 70. [5] ibid. [6] ibid.

Published by: Australian Science Archives Project on ASAPWeb
Comments or questions to: ASAPWeb (asapweb@asap.unimelb.edu.au)
Prepared by: Helen Morgan
Graphics by Lisa Cianci
Date modified: 7 October 1999