Save Our Bits! Meeting the Challenge of Digital Preservation

Download Report

Transcript Save Our Bits! Meeting the Challenge of Digital Preservation

Applying the COSCA Principles of Records Management

National Association for Court Management 2014 Annual Conference Greg Linhares, Missouri State Courts Administrator Nial Raaen, CRM, NCSC Principal Consultant

2012-2013 Policy Paper

To Protect and Preserve: Standards for Maintaining and Managing 21st Century Court Records

http://cosca.ncsc.org/

Records Management

An Essential Component

• • • • • • Support for judicial decision making Documentation of legal status and rights Public access to court proceedings Enforcement of court orders and judgments Preservation of record for appellate review Preservation of historical information

Records Defined – (ISO 15489-1)

Information created, received, and maintained as evidence and information by an organization or person, in pursuance of legal obligations or in the transaction of business.

Court Records Defined – CCJ/COSCA

Any document, information, or other thing that is collected, received, or maintained by a court or clerk of court in connection with a judicial proceeding; any index, calendar, docket, register of actions, official record of the proceedings, order, decree, judgment, minute, and any information in a case management system created by or prepared by the court or clerk of court that is related to a judicial proceeding; and Information maintained by the court or clerk of court pertaining to the administration of the court or clerk of court office and not associated with any particular case. (CCJ/COSCA guidelines, 2002)

The Records Lifecycle

The span of time of a records existence from its creation or receipt, through its useful life, to its final disposition, whether that disposition is destruction or retention as a historical record.

-ARMA International

The “Continuum of Care”

The management of records requires a “continuum of care” throughout the life cycle, beginning with creation and continuing through disposition and destruction.

The Challenges…

▫ Meeting the growing demand for access ▫ Maintaining both paper and electronic systems ▫ Managing unstructured records ▫ Coordinating multi-media retention and disposition ▫ Managing increasing amounts of information ▫ Rising to the challenge of information governance

COSCA Principles of Judicial Records Management

ACCESS GOVERNANCE COMPLIANCE INTEGRITY PRESERVATION DISPOSITION

The requirement that management practices are in line with applicable statutes, rules of court, administrative orders, and organizational policies.

Records Practices Must Comply With:

• • • Statues and court rules State financial and labor relations laws Applicable federal regulations ▫ FLSA ▫ Equal Pay Act ▫ FMLA ▫ Immigration laws … and others

Compliance Fundamentals

• • • • • Oversight and assignment of staff responsibility Policies and procedures Audits and reviews Classification systems Retention requirements

“Unstructured” Records

• • • • Office automation work products ▫ Documents ▫ Spreadsheets ▫ Presentations Social media Web content Email

Key Issue: Email Compliance

• • • • Email is not a records series Email must be managed and retained according to content Email must be classified and preserved accordingly Messages created or received by employees in connection with official business are generally considered public records

When is Email a Record?

• • • • Is the message evidence of work? Is the message the completion of a message string?

Are other records available about this issue/topic?

Do confidentiality requirements apply to the content of certain messages? 15

The principle of Integrity addresses the need for records to be created and preserved in a manner that guarantees their authenticity, reliability and accessibility.

ISO 15489 – Characteristics of Records Authenticity – Proven to be what it purports to be, created or sent by the person purported to have created or sent it, and created or sent at the time purported.

Reliability –Trusted as a full and accurate representation of the transactions, activities, or fact to which it attests.

Integrity – Complete and unaltered.

Usability – Can be located, retrieved, presented, and interpreted.

Integrity Fundamentals

• • • • Technical and Infrastructure • Backup and redundancy Updated security and virus checking Tracking and audits of changes Review of audit trails and system logs Documentation record of chain of custody • Sampling of media for corruption or failure

Processes and Procedures • • • • • • Control of physical security and user access Training and documentation Quality control Necessary metadata Internal compliance audits Hazard mitigation and disaster planning

Key Issue: E-record Integrity

Techniques/processes to ensure integrity: • • • Encryption Access control and security Check sums/hash algorithms • Audit trails and access logging

The principle of Access addresses the ability of court staff, litigants, and the public to access information to which they are entitled.

Access Fundamentals

• • • • Metadata and indexing to facilitate retrieval by staff and the public Storage and security appropriate to the type of record and storage media (preservation) Non-proprietary formats ensure long-term availability of electronic records Security and access controls appropriate to the record type and users/requestors

Metadata Matters

Metadata is data that describes or characterizes a digital object, whether internal or external to the object itself. It is essential to record retrieval and integrity.

Types of metadata include descriptive, administrative, structural, and preservation.

Metadata Standards & Guidelines

Dublin Core – 15 metadata elements adopted as an ISO standard 15836 OAIS – Open Archival Information System

Extensible Markup Language (XML)

PREMIS - Preservation Metadata: Implementation Strategies METS – Metadata Encoding and Transmission Standard …and more

Security and Access Control Critical for E-records

▫ Access controls that identify rights, (input, change, delete) ▫ User authentication, authorization, and audit trails ▫ Application and operating system protection from intrusion and attack

Key Issue: Third Party Use of Court Information • • • • Should there be increased obligations for data collectors to “sunset” information? Should there be a “right to forget”?

Who would have the duty to remove records to protect a “right to forget”? Should court record retention periods be revised?

Digital Rights Management

• • • • •

The use of encryption to allow the creator to control use of a document, such as:

• Who can access How long the document can be read Prevent printing Require network logging Restrict email forwarding May not be appropriate for records requiring long-term preservation

The principle of Disposition recognizes that all records reach a point in their lifecycle where they are committed to long-term archival storage and preservation, or are scheduled for destruction.

Appraisal

Archiving

Destruction

Records Appraisal

The evaluation of a document’s worth or value for retention or archival purposes, based upon its current or predicted future use(s) for administrative, legal, fiscal, research, or historical purposes.

~ ARMA Glossary, 4th edition

Business Record Value Legal Fiscal Historical

Disposition by Archiving

Transferring records is the process of moving records from one storage system to another within the organization. Accession is the movement of records to the custody of another agency, such as a state archives.

Disposition by Destruction

• • • • Disposition methods should be appropriate to the type of record and media Methods for electronic records: ▫ Degaussing ▫ Overwrite, encryption, file deletion ▫ Physical destruction of media Documentation, audit trails and metadata Standards are available for reference

Disposition Fundamentals

• • • • Maintain records according to established state and local retention schedules Remove non-essential, obsolete or duplicate records Use destruction methods appropriate to record content and media Conduct a periodic records inventory and appraisal

Key Issue: Multi-media Retention Management Are you a “digital hoarder”

How long is too long?

Coordinating paper and electronic retention schedules

Managing the deletion/retention of interrelated information

The principle of Preservation addresses the need to maintain the integrity and accessibility of judicial records throughout their life cycle.

Preservation Fundamentals

• • • • • Proper levels of protection Audits and validation Monitoring, disaster preparation and recovery planning Third party compliance with requirements Selection of appropriate media and storage systems to sustain access and usability

Key Issue: Long Term Digital Preservation

The Technical Issues

▫ Media longevity and obsolescence ▫ Hardware lifespan and compatibility ▫ Software and file format obsolescence

Preservation Risk Factors

Preservation risk increases with the following variables:

• • • • • Age of the record(s) Frequency of access to the record and media Complexity of the record and its features Volume of the record series Storage media

Hardware Challenges

• • • Storage medium may be superseded by newer versions or by new types of media—smaller, denser, faster, and easier to read.

Computers are continually superseded by faster and more powerful machines that can store and process more content.

Computer components and media physically fail due to human error, natural events, and normal aging.

Remember When?

8 inch floppy: 1971-81 5.25 inch floppy: 1972 – mid 1980s 12 inch optical: 1985 – 1992 Jazz disk: 1996 – 2002 source: Cornell University Chamber of Horrors

Software and Formats

• • • • A file format may be superseded by new versions, no longer be supported by the current vendor Software used to create, manage, or access digital content may be superseded by newer versions or newer generations with more features Characteristics as hidden text and change history, macros, and animations may be difficult to archive Vendors compete, merge, or go out of business leaving application software unsupported

Methods of E-Preservation

• •

PASSIVE

Ensures the integrity of, and access to, digital objects and their associated metadata. Attempts to keep the original object intact without changing the storage or access technologies. • •

ACTIVE

Ensures continued accessibility by active intervention to move the digital object from legacy to current storage environments.

May involve technologies not in existence when the record was created.

Migration

• • • • • Migration is a strategy for avoiding obsolescence Cycle is approximately every 10-15 years, or less Electronic records should be periodically migrated to stable media and stable file types Media and file types must provide a reliable and stable repository for preservation and access A migration strategy and schedule should be established for specific media and file types

Other Preservation Approaches

Emulation - Applications software that recreates the legacy technical environment required to run earlier programs Refreshing – moving records from one medium to another, primarily as a preventive measure Preservation – maintaining the original technical environment

Open Formats

▫ TIFF (?) ▫ XML ▫ JPEG ▫ PDF/A

Cloud Technology

Advantages : Concerns : • • • • Shared cost Quick startup Shared resources Scalable to need • • • • Security Ownership Access Vendor viability 48

Have a Preservation Strategy

▫ Identify acceptable risk for record types ▫ Assess current capabilities and capacity ▫ Survey existing tools for adoption ▫ Apply standards and use open formats ▫ Consider emerging technologies

Trusted Digital Repositories

• • Scheme developed by the National Archives and Records Administration Audit mechanism for assessing ability of repositories for secure and reliable long-term preservation of digital records • • Includes technical requirements as well as organizational infrastructure and policies Provides a set of assessment criteria  http://public.ccsds.org/publications/archive/650x0 m2.pdf

Open Archives Information System (OAIS) Model

OAIS Elements

• • • • • • Ingest. The steps required to transfer items from their current location into the archive in a managed manner.

Archival Storage: The storage of the bulk data using standard storage management tools.

Data Management: Tools to manage the storage of the archive, including the metadata Administration: A set of tools to administer the system and access to it.

Access: Tools to search, browse and download the contents of the archive Preservation Planning: The module that manages the information for long term access.

The principle of Governance addresses the need for organizational authority and control over records, as well as accountability.

Governance Fundamentals

• • • • Features of a comprehensive records management program – Roles and responsibilities clearly defined Strategy to ensure preservation and access Documentation of systems and processes Oversight and accountability ▫ Goals & performance measures ▫ Auditing and review ▫ Adoption of standards

Standards Organizations

▫ International Organization for Standardization (ISO) ▫ National Archives and Records Administration (NARA) ▫ Association for Information and Imaging Management (AIIM) ▫ ARMA International ▫ American National Standards Institute (ANSI) ▫ Department of Defense (DoD) ▫ European Commission (MoReq2)

Key Issue: Enterprise Management

The Growth of Information

Over the next 20 years there will be a 40% increase in information growth • • • • 57% office documents 65% email 75% email attachments 81% instant messages

- John Mancini, AIIM President

Gartner Forecast…

The rise of big data, social networking and mobile interactions, coupled with an accelerating increase in the amount of structured and unstructured information enabled by cloud based technologies, is creating an information crisis.

The Need for Greater Collaboration

▫ Coordination among multiple record holders ▫ Maintaining strategic alignment ▫ Acquiring adequate resources ▫ Anticipating future needs

Assessing Your Records Program

The Records Management Maturity Model

• • • • • • Original concept developed at Carnegie-Melon Based on ARMA RIM principles and model Follows the COSCA principles framework Four levels of maturity Key elements under each principle Self-assessment and scorecard available at: http://survey.confirmit.com/wix8/p2776972228.aspx

Other NCSC & COSCA Initiatives

• • • • • Joint Technology Committee – addressing the principles of disposition and preservation Development of RIM training programs Refinement of maturity models Identification of best practices and model programs Engagement with the records management community

ARMA International – www.arma.org

Association for Information and Image Management – www.aiim.org

National Archives and Records Administration – www.archives.gov

Council of State Archivists www.statearchivists.org

National Association of Government Archivists and Records Administrators – www.nagara.org