a centre of expertise in data curation and preservation Scaling the e-mail mountain A records manager’s guide to e-mail curation Maureen Pennock Digital Curation.
Download ReportTranscript a centre of expertise in data curation and preservation Scaling the e-mail mountain A records manager’s guide to e-mail curation Maureen Pennock Digital Curation.
a centre of expertise in data curation and preservation Scaling the e-mail mountain A records manager’s guide to e-mail curation Maureen Pennock Digital Curation Centre, UKOLN, University of Bath Funded by: This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 2.5 UK: Scotland License. To view a copy of this license, visit http://creativecommons.org/licenses/by-ncsa/2.5/scotland/ ; or, (b) send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA. Excludes images. RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation The first telegram… Source: The American Memory collection @ the Library of Congress RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation The first telephone call… Source: American Treasures collection @ the Library of Congress RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation The first email… Announcing the availability of networked mail… … exact words unknown … ‘QWERTYIOP’? … ‘ASDFGHJK’? [But it included information on using the ‘@’ symbol] RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Contemporary use • 100% of businesses use email for business purposes (AIIM Survey 2003) • • • • • • Administrative and commercial activities Contracts and agreement negotiations Financial issues Legal issues Tendering processes Recruitment activities • Both records and non-records RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Legal drivers • • • • • Data Protection Act (1998) Freedom of Information Act (2000) Regulation of Investigatory Powers Act (2000) Human Rights Act (1998) Intellectual property legislation • Trade Secrets/Law of Confidentiality • Copyright Law • Database Rights RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Risk management & business drivers • Risks of failure to curate: • Legal consequences • Financial consequences • Loss of public credibility • Loss of organisational memory • Loss of accountability • Loss of transparency • Reduced efficiency • Should be addressed by risk management framework • Based on information compliance & risk assessment exercises RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Contemporary challenges (I) • Unclear responsibilities: • Email as personal domain, not organisational property • User reluctance to ‘manage’ inbox • Clash between IT and RM policies (if any) • • • • Enforcement problems Inappropriate content Undesirable content Embarrassing content RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Contemporary challenges (II) • E-mail discovery carries inherent problems… • Dispersed locations: IMAP; POP; Forwarded messages; mailbox; shared drive; ERMS; EDMS… • Some may be inaccessible to systems admin or records managers • Relevant data may (or may not) be • In subject line • In message content • In message attachment (difficult to search) • Sheer scale of challenge • Not indexed for fast searching • … high discovery costs RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Contemporary challenges (III) • Coherence of message threads • Admissibility of printed versions of e-mails? • Improper deletion/destruction procedures • This is not just an issue for ‘records’ • Information (mis)management • Risk (mis)management • Can lead to all sorts of problems... RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation TV Chief lets rip in snappy birthday email ‘A good day to bury bad news…’ RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation 1. Digital Curation & E-mails What is digital curation? How does it apply to e-mails? Life cycle roles & responsibilities RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation What is digital curation? • “Digital Curation: …The activity of, managing and promoting the use of data from its point of creation, to ensure it is fit for contemporary purpose, and available for discovery and re-use.” Lord & MacDonald (2003) RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Why curate e-mails? • All records must be curated to ensure they remain fit for use and re-use: including e-mails • To combat: • Technological obsolescence • Bad creation & management practices • To help differentiate between records and non-records (and treat accordingly) • To help meet legal obligations • To be cost effective and efficient • To secure viability and reliability of messages for future RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Curation & the life-cycle Disposal? • Meaningful chain of custody • Requires compatibility of different stages • Requires input from range of stakeholders Access & Re-use Storage & Preservation • Takes control over the records throughout lifetime Transfer Creation Records Active Use Appraisal & Selection Disposal? RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Stakeholders & Roles • The range of stakeholders that affect the survival of digital material cuts across the whole lifecycle; everyone plays an important role Management & policy-makers Users - creators & receivers of e-mail messages Records Managers IT staff 'Curators' • System & mail-server administration • LAN Manager • Archivists • Re-users • • • • RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Issues for Creators & Recipients • E-mails must be: • Well-formed • Well-managed (even sent items!) • Accessible • Important elements: • Good creation/response practices • Inserting metadata • Headers – subject line, addresses • Message body - context • Message formats • Attachments • Complying with house-style • Good inbox management • Compliance with organisational policy RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Issues for Curators • E-mails must be: • Whole - comprising message body, headers & attachments • Captured as appropriate into organisational document/records/archives management system • Destroyed as appropriate • Important policy and practical elements: • • • • • • Identification & selection of e-mail records from non-records Proper filing & integration of e-mail records Deletion of transient/unnecessary e-mails Saving e-mail records independently of e-mail client Determining authenticity requirements Archiving & preservation • Guidance and Training for users ( at all levels) • Communication across the board RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Issues for Re-users • E-mails must be: • Accessible for appropriate re-users • Exported in an appropriate and usable format • Things to consider: • Legal access and re-use restrictions may be different & must be observed • Appropriate re-use software may be needed • Resource Discovery • Access Rights Management • Different re-users may have different re-use requirements • E-mails can be re-used for very different purposes to why they were originally created RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation 2. Practical Steps RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Develop an e-mail policy • One or more policies to cover: • Creation practices • Using business e-mail accounts for private use & vice versa • Responsibilities & shared access • Levels of organisational monitoring • Legal issues • Integrated records retention & preservation • Disposal RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Provide education & training • On composition – • • • • Meaningful messages Formats Attachments Context • On storage & transfer • Both sent and received mails • Relationship between inbox management and records management • On legal responsibilities RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Implement a solution • Consistent application of appropriate strategy • May be best determined by strategic working group with representatives from different stakeholder groups • Should take entire life-cycle into account • Should be integrated with organisation RM – not a stand alone solution • Must consider issues raised in this presentation • Requires consistency across life-cycle stages • Assess compliance regularly! • Revisit regularly to keep up-to-date RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation 3. Management & Preservation Options RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation The First Solution RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation The First Solution… ? Doesn’t solve all the problems… ? ? ? … and may even create extra ones! ? ? RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Coming a close second… […also known as the ‘do-nothing’ approach] RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation A more realistic option • Convert or save in Standard formats • ‘The best thing about standards is that there are so many to choose from!’ • Which are most appropriate for e-mails? • • • • PDF? TIFF? RFC 2822/Plain text? XML? RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Other issues to consider • • • • • What about attachments? Dealing with digital signatures Preservation metadata Long term storage and archiving Audit and certification RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation The future? • Changes to e-mail interface • New applications • Alternative search functionality • Visualisation for re-use • Alternative technologies • • • • • Instant Messaging SKYPE RSS (Really Simple Syndication) Discussion forums Blogs, wiki’s, texting… RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Further reading & references (I) • Policy: • Institutional Records Management & E-mails project (2003) • http://www.lboro.ac.uk/computing/irm • Sample University Guidelines & Advice • Edinburgh University – exemplar • Developing a policy for managing e-mail (2004) • Guidelines from TNA • http:///www.nationalarchives.gov.uk RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Further reading & references (II) • Technical solutions for preservation: • Digital Preservation Testbed XMaiL and preservation recommendations (2003) • http://www.digitaleduurzaamheid.nl • Antwerp City Archives e-mail preservation template & advice (2003 – 2006) • http://www.expertisecentrumdavid.be • National Archives of Australia: XENA (2004 +) • http://www.naa.gov.au • San Diego Super Computer Centre (1999 +) • http://www.sdsc.edu RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Further reading & references (III) • DCC Digital Curation Manual Instalment on ‘Curating E-mail Messages: A life-cycle approach to the management and preservation of e-mail messages’ Maureen Pennock Digital Curation Centre & UKOLN 2006 http://www.dcc.ac.uk RMS Annual Conference Brighton 01 May 2007 a centre of expertise in data curation and preservation Thank you. Questions? Maureen Pennock [email protected] (Join the DCC Associates Network at http://www.dcc.ac.uk) RMS Annual Conference Brighton 01 May 2007