Transcript Save Our Bits! Meeting the Challenge of Digital Preservation
Applying the COSCA Principles of Records Management
National Association for Court Management 2014 Annual Conference Greg Linhares, Missouri State Courts Administrator Nial Raaen, CRM, NCSC Principal Consultant
2012-2013 Policy Paper
To Protect and Preserve: Standards for Maintaining and Managing 21st Century Court Records
http://cosca.ncsc.org/
Records Management
An Essential Component
• • • • • • Support for judicial decision making Documentation of legal status and rights Public access to court proceedings Enforcement of court orders and judgments Preservation of record for appellate review Preservation of historical information
Records Defined – (ISO 15489-1)
Information created, received, and maintained as evidence and information by an organization or person, in pursuance of legal obligations or in the transaction of business.
Court Records Defined – CCJ/COSCA
Any document, information, or other thing that is collected, received, or maintained by a court or clerk of court in connection with a judicial proceeding; any index, calendar, docket, register of actions, official record of the proceedings, order, decree, judgment, minute, and any information in a case management system created by or prepared by the court or clerk of court that is related to a judicial proceeding; and Information maintained by the court or clerk of court pertaining to the administration of the court or clerk of court office and not associated with any particular case. (CCJ/COSCA guidelines, 2002)
The Records Lifecycle
The span of time of a records existence from its creation or receipt, through its useful life, to its final disposition, whether that disposition is destruction or retention as a historical record.
-ARMA International
The “Continuum of Care”
The management of records requires a “continuum of care” throughout the life cycle, beginning with creation and continuing through disposition and destruction.
The Challenges…
▫ Meeting the growing demand for access ▫ Maintaining both paper and electronic systems ▫ Managing unstructured records ▫ Coordinating multi-media retention and disposition ▫ Managing increasing amounts of information ▫ Rising to the challenge of information governance
COSCA Principles of Judicial Records Management
ACCESS GOVERNANCE COMPLIANCE INTEGRITY PRESERVATION DISPOSITION
The requirement that management practices are in line with applicable statutes, rules of court, administrative orders, and organizational policies.
Records Practices Must Comply With:
• • • Statues and court rules State financial and labor relations laws Applicable federal regulations ▫ FLSA ▫ Equal Pay Act ▫ FMLA ▫ Immigration laws … and others
Compliance Fundamentals
• • • • • Oversight and assignment of staff responsibility Policies and procedures Audits and reviews Classification systems Retention requirements
“Unstructured” Records
• • • • Office automation work products ▫ Documents ▫ Spreadsheets ▫ Presentations Social media Web content Email
Key Issue: Email Compliance
• • • • Email is not a records series Email must be managed and retained according to content Email must be classified and preserved accordingly Messages created or received by employees in connection with official business are generally considered public records
When is Email a Record?
• • • • Is the message evidence of work? Is the message the completion of a message string?
Are other records available about this issue/topic?
Do confidentiality requirements apply to the content of certain messages? 15
The principle of Integrity addresses the need for records to be created and preserved in a manner that guarantees their authenticity, reliability and accessibility.
ISO 15489 – Characteristics of Records Authenticity – Proven to be what it purports to be, created or sent by the person purported to have created or sent it, and created or sent at the time purported.
Reliability –Trusted as a full and accurate representation of the transactions, activities, or fact to which it attests.
Integrity – Complete and unaltered.
Usability – Can be located, retrieved, presented, and interpreted.
Integrity Fundamentals
• • • • Technical and Infrastructure • Backup and redundancy Updated security and virus checking Tracking and audits of changes Review of audit trails and system logs Documentation record of chain of custody • Sampling of media for corruption or failure
Processes and Procedures • • • • • • Control of physical security and user access Training and documentation Quality control Necessary metadata Internal compliance audits Hazard mitigation and disaster planning
Key Issue: E-record Integrity
Techniques/processes to ensure integrity: • • • Encryption Access control and security Check sums/hash algorithms • Audit trails and access logging
The principle of Access addresses the ability of court staff, litigants, and the public to access information to which they are entitled.
Access Fundamentals
• • • • Metadata and indexing to facilitate retrieval by staff and the public Storage and security appropriate to the type of record and storage media (preservation) Non-proprietary formats ensure long-term availability of electronic records Security and access controls appropriate to the record type and users/requestors
Metadata Matters
Metadata is data that describes or characterizes a digital object, whether internal or external to the object itself. It is essential to record retrieval and integrity.
Types of metadata include descriptive, administrative, structural, and preservation.
Metadata Standards & Guidelines
Dublin Core – 15 metadata elements adopted as an ISO standard 15836 OAIS – Open Archival Information System
Extensible Markup Language (XML)
PREMIS - Preservation Metadata: Implementation Strategies METS – Metadata Encoding and Transmission Standard …and more
Security and Access Control Critical for E-records
▫ Access controls that identify rights, (input, change, delete) ▫ User authentication, authorization, and audit trails ▫ Application and operating system protection from intrusion and attack
Key Issue: Third Party Use of Court Information • • • • Should there be increased obligations for data collectors to “sunset” information? Should there be a “right to forget”?
Who would have the duty to remove records to protect a “right to forget”? Should court record retention periods be revised?
Digital Rights Management
• • • • •
The use of encryption to allow the creator to control use of a document, such as:
• Who can access How long the document can be read Prevent printing Require network logging Restrict email forwarding May not be appropriate for records requiring long-term preservation
The principle of Disposition recognizes that all records reach a point in their lifecycle where they are committed to long-term archival storage and preservation, or are scheduled for destruction.
•
Appraisal
•
Archiving
•
Destruction
Records Appraisal
The evaluation of a document’s worth or value for retention or archival purposes, based upon its current or predicted future use(s) for administrative, legal, fiscal, research, or historical purposes.
~ ARMA Glossary, 4th edition
Business Record Value Legal Fiscal Historical
Disposition by Archiving
Transferring records is the process of moving records from one storage system to another within the organization. Accession is the movement of records to the custody of another agency, such as a state archives.
Disposition by Destruction
• • • • Disposition methods should be appropriate to the type of record and media Methods for electronic records: ▫ Degaussing ▫ Overwrite, encryption, file deletion ▫ Physical destruction of media Documentation, audit trails and metadata Standards are available for reference
Disposition Fundamentals
• • • • Maintain records according to established state and local retention schedules Remove non-essential, obsolete or duplicate records Use destruction methods appropriate to record content and media Conduct a periodic records inventory and appraisal
Key Issue: Multi-media Retention Management Are you a “digital hoarder”
•
How long is too long?
•
Coordinating paper and electronic retention schedules
•
Managing the deletion/retention of interrelated information
The principle of Preservation addresses the need to maintain the integrity and accessibility of judicial records throughout their life cycle.
Preservation Fundamentals
• • • • • Proper levels of protection Audits and validation Monitoring, disaster preparation and recovery planning Third party compliance with requirements Selection of appropriate media and storage systems to sustain access and usability
Key Issue: Long Term Digital Preservation
The Technical Issues
▫ Media longevity and obsolescence ▫ Hardware lifespan and compatibility ▫ Software and file format obsolescence
Preservation Risk Factors
Preservation risk increases with the following variables:
• • • • • Age of the record(s) Frequency of access to the record and media Complexity of the record and its features Volume of the record series Storage media
Hardware Challenges
• • • Storage medium may be superseded by newer versions or by new types of media—smaller, denser, faster, and easier to read.
Computers are continually superseded by faster and more powerful machines that can store and process more content.
Computer components and media physically fail due to human error, natural events, and normal aging.
Remember When?
8 inch floppy: 1971-81 5.25 inch floppy: 1972 – mid 1980s 12 inch optical: 1985 – 1992 Jazz disk: 1996 – 2002 source: Cornell University Chamber of Horrors
Software and Formats
• • • • A file format may be superseded by new versions, no longer be supported by the current vendor Software used to create, manage, or access digital content may be superseded by newer versions or newer generations with more features Characteristics as hidden text and change history, macros, and animations may be difficult to archive Vendors compete, merge, or go out of business leaving application software unsupported
Methods of E-Preservation
• •
PASSIVE
Ensures the integrity of, and access to, digital objects and their associated metadata. Attempts to keep the original object intact without changing the storage or access technologies. • •
ACTIVE
Ensures continued accessibility by active intervention to move the digital object from legacy to current storage environments.
May involve technologies not in existence when the record was created.
Migration
• • • • • Migration is a strategy for avoiding obsolescence Cycle is approximately every 10-15 years, or less Electronic records should be periodically migrated to stable media and stable file types Media and file types must provide a reliable and stable repository for preservation and access A migration strategy and schedule should be established for specific media and file types
Other Preservation Approaches
Emulation - Applications software that recreates the legacy technical environment required to run earlier programs Refreshing – moving records from one medium to another, primarily as a preventive measure Preservation – maintaining the original technical environment
Open Formats
▫ TIFF (?) ▫ XML ▫ JPEG ▫ PDF/A
Cloud Technology
Advantages : Concerns : • • • • Shared cost Quick startup Shared resources Scalable to need • • • • Security Ownership Access Vendor viability 48
Have a Preservation Strategy
▫ Identify acceptable risk for record types ▫ Assess current capabilities and capacity ▫ Survey existing tools for adoption ▫ Apply standards and use open formats ▫ Consider emerging technologies
Trusted Digital Repositories
• • Scheme developed by the National Archives and Records Administration Audit mechanism for assessing ability of repositories for secure and reliable long-term preservation of digital records • • Includes technical requirements as well as organizational infrastructure and policies Provides a set of assessment criteria http://public.ccsds.org/publications/archive/650x0 m2.pdf
Open Archives Information System (OAIS) Model
OAIS Elements
• • • • • • Ingest. The steps required to transfer items from their current location into the archive in a managed manner.
Archival Storage: The storage of the bulk data using standard storage management tools.
Data Management: Tools to manage the storage of the archive, including the metadata Administration: A set of tools to administer the system and access to it.
Access: Tools to search, browse and download the contents of the archive Preservation Planning: The module that manages the information for long term access.
The principle of Governance addresses the need for organizational authority and control over records, as well as accountability.
Governance Fundamentals
• • • • Features of a comprehensive records management program – Roles and responsibilities clearly defined Strategy to ensure preservation and access Documentation of systems and processes Oversight and accountability ▫ Goals & performance measures ▫ Auditing and review ▫ Adoption of standards
Standards Organizations
▫ International Organization for Standardization (ISO) ▫ National Archives and Records Administration (NARA) ▫ Association for Information and Imaging Management (AIIM) ▫ ARMA International ▫ American National Standards Institute (ANSI) ▫ Department of Defense (DoD) ▫ European Commission (MoReq2)
Key Issue: Enterprise Management
The Growth of Information
Over the next 20 years there will be a 40% increase in information growth • • • • 57% office documents 65% email 75% email attachments 81% instant messages
- John Mancini, AIIM President
Gartner Forecast…
The rise of big data, social networking and mobile interactions, coupled with an accelerating increase in the amount of structured and unstructured information enabled by cloud based technologies, is creating an information crisis.
The Need for Greater Collaboration
▫ Coordination among multiple record holders ▫ Maintaining strategic alignment ▫ Acquiring adequate resources ▫ Anticipating future needs
Assessing Your Records Program
The Records Management Maturity Model
• • • • • • Original concept developed at Carnegie-Melon Based on ARMA RIM principles and model Follows the COSCA principles framework Four levels of maturity Key elements under each principle Self-assessment and scorecard available at: http://survey.confirmit.com/wix8/p2776972228.aspx
Other NCSC & COSCA Initiatives
• • • • • Joint Technology Committee – addressing the principles of disposition and preservation Development of RIM training programs Refinement of maturity models Identification of best practices and model programs Engagement with the records management community
ARMA International – www.arma.org
Association for Information and Image Management – www.aiim.org
National Archives and Records Administration – www.archives.gov
Council of State Archivists www.statearchivists.org
National Association of Government Archivists and Records Administrators – www.nagara.org