Document Management and long term storage – Technology is not the end…

ABD – BVD 60th anniversary KBR, Brussels, November 19, 2007

Phédra Clouner and Christian Grey 1

Phédra Clouner

    Specialist in document management Project manager at the Belgian Ministry of Justice President of document@work since January 2006 [email protected]

p. 2

Christian Grey

    ECM Consultant since 1985 Founding member of the BeLAIIM (now document@work) in 1991  Now Chairman of the “Outsourcing & industrialised documents” Special Interest Group (SIG) Principal Consultant at Getronics Belgium   A subsidiary company of KPN [email protected]

LinkedIn  http://www.linkedin.com/in/cgrey p. 3

document@work (1)

  In existence since 1991 as BeLAIIM  Since 2006, operates under the name of document@work Mission   To bring together a group of experts and document & workflow management practitioners and academics to  Share information and expertise  Promote a better understanding of document &workflow management best practices, methods, tools and training Independence and neutrality p. 4

document@work (2) - Actions

     Different type of members  ABD-BVD is one of our Partners!

Organization of seminars around Document Management, BPM, RM, KM, etc Organization of annual congress Creation of Special Interest Groups on our website   Legal aspects and Records management, BPM, ILM, KM, etc A specialist to answer user’s questions for each group Contribution to the proof-reading of European norm MoReq2 p. 5

Agenda

  The problem The solution  Elements of ~ p. 6

The problem: Documents and records are so fragile!

p. 7

Problem: lifecycle What file format?

Compliance

How to keep track who did what and when?

How to destroy?

AS ISO 15489 What to keep?

SOX,

BASEL II Where?

Costs?

Freedom of Intrusion, Act, fire, pest Availabilit DOMEA y CSSF / PSF

Human processing

What media?

Auditable electronic archives

Retention vs. need for evidence

p. 10

Problem: being virtual and dynamic

 Static content causes less problems  Dynamic content presents partial results that are not necessarily under control    Embedded queries to databases  Who can foresee the result?

Content fed by RSS  What are the news today?

Reference to http://.../.../ ...

 Is there still someone out there? Broken link?

p. 11

Problem: being virtual and dynamic

  Simple example (with some text editor)  Today     Create a file using a blank template Use the function INSERT -> DATE You should read today’s date (e.g. Nov 19, 2007) Save the file as 20071119.RTF

    Change your computer’s date to Nov 20, 2007 Open the file 20071119.RTF

What do you read?

Question    If the same computer file can display two different content values, then Where is the document?

What is the document?

This problem has been fixed since a few years in major text editors   For newly created documents!

But not for previously created documents… p. 12

Problem: perennial aspects of the whole chain

    Lifespan of the medium    Optimistic: 100 years More realistic: 5 to 25 years Refreshing reads needed  Technological constraints Hardware changes  Economical opportunities Lifespan of the drives  Spare parts, connectors, software drivers, operating system Lifespan of the content    Character sets (ASCII, EBCDIC, Unicode) File formats (ODA / ODIF ???) File system (FAT16, NTFS, EFS)  Plan for migration from Day 1  Your decision, not a fatality p. 13

Problem: away from optical, non rewriteable media

    Non-rewriteable optical media had / have a long lifespan expected They are no longer the legal panacea Electronic signature is now the key element for probative value    Having recourse to advanced / qualified certificates As defined by the EU Directive 99/93/EC on a Community framework for electronic signatures In Belgium  Loi sur la signature électronique et les prestataires de services de certification du 9 juillet 2001  Arrêté royal du 6 décembre 2002 organisant le contrôle et l’accréditation des prestataires de service de certification qui délivrent des certificats qualifiés Robust, affordable and auditable magnetic storage solutions now available on the market p. 14

First question: Why do we want long term preservation of documents?

   They serve as indispensable instruments of accountability, a means of protecting individual and corporate rights, and as sources of information for future generations For an organisation:    For legal reasons, as a proof (probative value) For their informational value Several regulations/law exists (SOX, Basel II)  compliance For the society in general  For scientific and historical research p. 15

Second question: Why do we want long term preservation of electronic documents?

  Paper preservation need a lot of storage space More and more documents are created electronically, they are the only original that exists and the document to preserve  It’s a critical challenge for present and future archivists p. 16

Elements of solution

   Norms and standards Mechanical   Technical environment File formats Semantic  Preserving metadata p. 17

Norms and standards

p. 18

Why do we respect norms? (1)

  To develop and offer products and services meeting specifications that have wide international, European and / or national acceptance in the sector To insure a better exchange between the different actors and a better management of the organization p. 19

Why do we respect norms? (2)

   To insure the respect of the principles of Records management. A record is  Authentic  Accurate       Accessible Complete Comprehensive Compliant Effective Secure Norms are about technology types to use  But not only… p. 20

As we said technology is not the end…

   Several norms exist around the world to help respecting long term preservation of electronic document Theses norms apply to the concepts of document and records We’ll talk especially about records p. 21

What is a record?

   Information created, received, and maintained as evidence and information by an organization or person , in pursuance of legal obligations or in the transaction of business (ISO definition) A record can be either a tangible object or digital information which has value to an organization We speak here about electronic records p. 22

What is a document?

 

A document is any piece of written information in any form, produced or received by an organisation or person.

Could be really ephemeral and have a short term value

All records start off as documents, but not all documents will ultimately become records p. 23

By the way, what is records management?

 “The field of management responsible for the efficient and systematic control of the creation, receipt, maintenance, use and disposition of records, including the processes for capturing and maintaining evidence of and information (ISO 15489) about business activities and transactions in the form of records” p. 24

International: ISO 15489 (1)

    Published in 2001 A new version is announced for 2008 or 2009 Part 1: General  Guidance on how to manage records Part 2:Guidelines  Provides a methodology for its implementation p. 25

International: ISO 15489 (2)

  What is in ISO 15489?

 conception and implementation of an ERMS Processes      Activities Classification Retention period Selection and recording Storage Access   Traceability Final disposition of the record p. 26

European: MoReq

    Stands for   Model Requirements for the Management of Electronic Records Modèle d’exigences pour l’organisation de l’archivage électronique From whom?

  Contracted to Cornwell Management Consultants By the DLM Forum at the request of the European Commission Why MoReq?

     First, there was US DoD 5015.2 (1997), originally for US Defence sector. Not really suitable for use in Europe.

MoReq is a Requirements Specification (2001), originating in Europe, now used around the world and widely translated.

Operational approach of the ISO15489 To help when buying an ERM system To help when designing an ERM system  Its advantage is that vendors need in future orient their products to just one European standard, instead of different implementation standards for each country. 100 Pages, 390 Requirements p. 27

European: MoReq2 (1)

  Publication of a MoReq2: beginning 2008 Why MoReq2?

  No maintenance   Enhancements Technology evolution No software testing regime  At the end of 2007, the new MoReq2 and its associated test and certification procedures will be made available for records management products  No governance p. 28

European: MoReq2 (2)

 What is in MoReq2?

     Chapter zero: for national variations Classification scheme Control and security Retention and disposition Capturing records    Searching retrieval and presentation Administrative functions Testing p. 29

European: MoReq2 (3)

 Optional modules            Physical records Document management/ collaboration Workflow integration Case work Content management integration Electronic signatures, encryption, etc.

Distributed systems Offline and remote working Record keeping processes (new); Fax integration Security p. 30

National with international impact: US DoD 5015.2

       Stands for Department of Defense, Electronic records management software applications design criteria standard Design criteria standard for electronic records management software applications First version: 1997 Last version (v.3): April 25, 2007  Major additions: requirements to support the Freedom of Information Act (FOIA), Privacy Act, and interoperability Originally for US Defence sector, must be adhered by all manufacturers hoping to sell to the US government in military and similar areas Gradually spread throughout USA: de facto standard Defines the basic requirements for documents and records management systems  Made to address the US market p. 31

DoD5015.2: Alignment with ISO Standard 15489

ISO 15489 focuses on RM program management and addresses: – Form and structure – Metadata requirements – Retrieval requirements – How to organize records – Assessing risks – Security – Records retention There is no certification program for the ISO 15489 Standard DoD 5015.2 is focused on core electronic records management software functionality DoD 5015.2 has a long established program for testing and certification DoD 5015.2 certified RMAs are tools to help organizations meet the goals and objectives of ISO 15489 p. 32

National with international impact: France AFNOR Z 42-013

     Latest version 2001 New version foreseen in 2008 Technical and organisational elements to implement for the record, the storage, and the restitution of electronic documents to insure conservation and integrity Considers only WORM media!

Especially for scanned documents p. 33

National with international impact: UK TNA

   Origin  Originates from The National Archives Background     UK standard replacing the older Public Records Office (PRO) The National Archives updated the functional requirements for ERMS in collaboration with the central government records management community in 2002 Revision in 2006 Takes account of international standards since 1999 What is in TNA?

      Functional requirements Metadata standard (XML scheme for exchange, interoperability) Glossary, access control model, Optional modules: case management and workflow Implementation guidance Formal test scripts   There are 20 scripts for the core requirements and additional ones for the optional modules. List of approved systems p. 34

National Germany: DOMEA

    Document management and electronic archiving The German DOMEA concept is an example of a detailed standard for digital process management systems. It describes the requirements for document management and electronic archiving in public administration, and permits the testing and certification of products in this area. DOMEA compliance is a requirement in many RFP p. 35

Mechanical – Technical environment

p. 36

Preservation of technology (1)

  Computer Museum Strategy  Storage of the original hardware and software, with which the electronic records were created or managed  Not possible for medium-term and long-term storage Emulation   The required platform is simulated on newer computer configurations Records are consulted in their original file format     Technically complex A lot of development and of maintenance costs Conversion or migration of emulation program A lot of emulation program in an organisation p. 37

Preservation of technology (2)

 Conversion  Conversion of digital documents from a lower to higher version of the same file format    Certain file formats change often Properties could be lost of changed  easier to guarantee authenticity less Digital documents continue to be stored in a software or version dependant format p. 38

Preservation of technology (3)

  Migration      Digital document are transformed into adequate and standardised archiving file format Independent Specification of file format is available For some file formats it doesn’t exist suitable archiving file formats Properties and functionalities of the original format can rarely integrally be transferred, loss Don’t forget to preserve the media!

p. 39

Mechanical - File Formats

p. 40

File Formats - Proprietary

 File formats supported by an editor, without  Commitment to perennial coherence and support    Community developments Public documentation Absence of royalty p. 41

File Formats - Open

       EPS Open Document Format (ODF) OpenXML PDF/A TIFF XML …  Analyse your requirements regarding      Text vs. Image Signature B&W, colour Single- or multi-page Exchange with your business partners  Examples  A TIFF file doesn’t support electronic signature  Is this a problem for you, your auditors, the probative value you expect from these files in all countries where your organisation is active?

p. 42

Semantic – Using thesauri as abstraction layers

  Preservation of some metadata can be achieved using thesauri  Applicable to controlled vocabulary only Example  A document was created in 1995 where metadata COUNTRY=ZAIRE     Fact of life: in 1997, the name of the country becomes DEMOCRATIC REPUBLIC OF (THE) CONGO After that, the thesaurus is modified to make ZAIRE and DEMOCRATIC REPUBLIC OF (THE) CONGO synonyms After the modification of the thesaurus, queries on ZAIRE will work as queries on DEMOCRATIC REPUBLIC OF (THE) CONGO No need to modify metadata => preservation  Not just technologies … p. 43

Conclusions

p. 44

Conclusions

   We need to use the adequate technology but in a formal context Adequate technology is fundamental but not sufficient Without adequate classification scheme, metadata, process, etc, no long term preservation possible in the context of Document & Records management  document@work’s mission is to give some keys to organizations about Records management and existing norms and the necessity if they want to be compliant, preserve their document as proof, etc to respect these norms p. 45

http://www.documentatwork.be

p. 46

Acknowledgment

     Norms    ISO 15489 MoReq and MoReq2 Afnor Z 42-013 M.-A. CHABIN, Records management et gestion des archives, stage Archives de France, http://www.archivesdefrance.culture.gouv.fr/fr/formation/courantexposesi ntervenants2006/Marie-Anne%20CHABIN%20(France).pdf

, 17 nov 2006.

Les principes techniques et organisationnels de la préservation des documents numériques  Catherine Lupovici, Département de la bibliothèque numérique, Direction des services et des réseaux, Bibliothèque nationale de France The importance of the new MoReq2 standard for Document Management in Europe  MARC FRESKO, Cornwell Management Consultants plc Convergence of Document and Records Management  Richmond AIIM/ARMA, Chapters Joint Seminar, September 15, 2004 p. 47

Acknowledgment

   InterPARES 2 Project   International Research on permanent Authentic Records in Electronic Systems http://www.interpares.org

F. BOUDREZ, J. DUMORTIER et al, Digital Archiving: the new challenge?Legal and archival issues, Louvain-La-Neuve, 2005 Information Management Compliance, PROJECT CONSULT Whitepaper      Dr. Ulrich Kampffmeyer DLM Network EIIG, Manager MoReq2 Editorial Board, Mitglied http://www.PROJECT-CONSULT.com

Hamburg, September 2007 p. 48