UCLA/Getty Summer Institute for Knowledge Sharing (opening) Howard Besser UCLA School of Education & Information http://www.gseis.ucla.edu/~howard Besser--UCLA/Getty Summer Instit-intro 8/6/01 1

UCLA/Getty Summer Institute for Knowledge Sharing _ _ _ _ Interoperability Importance of Standards Best Practices for Managing Digital Projects Implications of Digital Projects _ Longevity _ From Digital Collections to Digital Libraries & Museums Besser--UCLA/Getty Summer Instit-intro 8/6/01 2

Key problems we’re facing _ _ _ Discovery Interoperability Longevity Besser--UCLA/Getty Summer Instit-intro 8/6/01 3

Traditional Digital Collection Model DL DL DL search & presentation search & presentation search & presentation DL search & presentation user Besser--UCLA/Getty Summer Instit-intro 8/6/01 user 4

Ideal Digital Collection Model DL DL DL DL search & presentation user Besser--UCLA/Getty Summer Instit-intro 8/6/01 user 5

For Interoperability Digital Collections Need Standards _ _ _ Descriptive Metadata for consistent description Discovery Metadata for finding Administrative Metadata for viewing and maintaining _ Structural Metadata for navigation _ ... Terms & Conditions Metadata for controlling access...

Besser--UCLA/Getty Summer Instit-intro 8/6/01 6

Metadata

is not just indexing terms _ _ _ _ _ CBIR attributes used for retrieval on color, shape, texture, etc.

Structural attributes used for page-turning Administrative attributes used for managing a digital work over time IPR attributes to limit unauthorized use Identification attributes to determine what application software is needed to view a particular digital work _ Can be located anywhere Besser--UCLA/Getty Summer Instit-intro 8/6/01 7

Why are Standards and Metadata consensus  important?

Managing digital files over time  Longevity  Interoperability  Veracity  Recording in a consistent manner  Will give vendors incentive to create applications that support this Besser--UCLA/Getty Summer Instit-intro 8/6/01 8

Why Standards?

 Why do we need standards?

– – To make information universally available to users facilitate sharing and interchange of information – To preserve information (make it safe from changes in hardware and software)  Standards only work if communities widely accept them, but they’re necessary for communities to work together Besser--UCLA/Getty Summer Instit-intro 8/6/01 9

Important Planning Considerations  File Formats  Choosing Interoperable Systems  Adhere to standards  Vendors with large installed base  Refreshing and/or Migration Besser--UCLA/Getty Summer Instit-intro 8/6/01 10

Key Considerations for Imaging Projects  Image Quality – Archival – Current online delivery  Intellectual Property  Standards – Modular and Layered Architecture – – Terminology Technical imaging information Besser--UCLA/Getty Summer Instit-intro 8/6/01 11

Best Practices for Managing Digital Projects _ _ _ Who will your users be?

Best Practices Guidelines Workflow and Management Issues Besser--UCLA/Getty Summer Instit-intro 8/6/01 12

Why are you Managing this Information?

 Organizational mission & type  Users  Uses Besser--UCLA/Getty Summer Instit-intro 8/6/01 13

Scanning Best Practices _ _ _ _ Think about users (and potential users), uses, and type of material/collection Scan at the highest quality that does not exceed the likely potential users/uses/material Do not let today’s delivery limitations influence your scanning file sizes; understand the difference between digital masters and derivative files used for delivery Many documents which appear to be bitonal actually are better represented with greyscale scans Besser--UCLA/Getty Summer Instit-intro 8/6/01 _ _ _ _ _ Include color bar and ruler in the scan Use objective measurements to determine scanner settings (do NOT attempt to make the image good on your particular monitor or use image processing to color correct) Don’t use lossy compression Store in a common (standardized) file format Capture as much metadata as is reasonably possible (including metadata about the scanning process itself) 14

Why Scale is important Besser--UCLA/Getty Summer Instit-intro 8/6/01 15

Digital Object

Behaviors

_ Book example Besser--UCLA/Getty Summer Instit-intro 8/6/01 16

Metadata Standards (from MOA2, now METS) _ _ Administrative Metadata – for enhancing resource management Structural Metadata – for reflecting internal hierarchies and relationships btwn parts _ Raw/Seared/Cooked Besser--UCLA/Getty Summer Instit-intro 8/6/01 17

More general issues of Digital Projects _ _ _ _ _ Workflow and Management Issues Implications for the Collection Implications for the Institution Implications for Scholarship & Interoperability – Digital libraries – Metadata Longevity Issues Besser--UCLA/Getty Summer Instit-intro 8/6/01 18

Workflow and Management Issues _ _ _ Managing multiple image files Persistent Identification Making your works accessible throughout the Net Besser--UCLA/Getty Summer Instit-intro 8/6/01 19

Workflow and Tracking Procedures _ _ _ Need careful planning Procedures for managing many different files at many different stages Linking of file versions Besser--UCLA/Getty Summer Instit-intro 8/6/01 20

The number of variant forms of a work can be enormous  different views of the same object  different scans of the same photo  different resolutions  different compression schemes  different compression ratios  different file storage formats  different details of the same image  ...

Besser--UCLA/Getty Summer Instit-intro 8/6/01 21

Image Families

Identification/Provenance  how to deal with different versions (browse, hi-res, medium res) derived from the same scan or different encoding schemes (TIFF, PICT, JFIF)  Vocabulary Standards to express this – VRA Surrogate Categories – CIMI's "Image Elements” Besser--UCLA/Getty Summer Instit-intro 8/6/01 23

Persistent IDs--the Problem _ _ _ Need to separate work ID from work location URNs probably won’t be ready until 2003 Becomes a business process issue when one organization maintains the resource and another organization references it (ie. licensed from vendors or managed by separate administrative structures) Besser--UCLA/Getty Summer Instit-intro 8/6/01 24

Making your works accessible throughout the Net _ _ Open Archives and Metadata Harvesting (DLF/Mellon) An administrative and political issue as much as a a technical one Besser--UCLA/Getty Summer Instit-intro 8/6/01 25

More general issues of Digital Projects _ _ _ _ _ Workflow and Management Issues Implications for the Collection Implications for the Institution Implications for Scholarship & Interoperability – Digital libraries – Metadata Longevity Issues Besser--UCLA/Getty Summer Instit-intro 8/6/01 26

Implications for the Collection _ _ _ _ We’re already familiar with Reformatting Advantages & Disadvantages of Digitization Protection Unauthorized Use Besser--UCLA/Getty Summer Instit-intro 8/6/01 27

Broad Advantages & Disadvantages of Digitization _ _ _ _

Advantages

good PR show off collection let people see items without having to needlessly pull them _ Besser--UCLA/Getty Summer Instit-intro 8/6/01 _ _ _ _ _

Disadvantages

Can look like Edutainment Can commodify the works and make the repository look like it sold prestige to the highest bidder Authenticity called into question Decontextualization Representational problems 28

_ _ _ Problems with How Works are Represented once a digital work is on the WWW, anyone can physically copy it and use it as they see fit often items are seen outside their context for images: using the normal method of mounting images on the WWW, the credit line often becomes separated from the image Besser--UCLA/Getty Summer Instit-intro 8/6/01 29

Don’t advocate strong copyright or protection for the wrong _ reasons the people who find your content valuable are mostly your traditional audiences _ _ barriers before use will inhibit positive uses of your material threat of pursuit after misuse can effectively deter commercial misuse _ you can prevent commercial misuse without strong protection or copyright Besser--UCLA/Getty Summer Instit-intro 8/6/01 30

Instead of fearing lost income, worry about Unauthorized Use _ _ _ Elimination of credit or attribution line (particularly for images) Someone else implying ownership Maintaining the Integrity of the Work Besser--UCLA/Getty Summer Instit-intro 8/6/01 31

Protection Methods ^ _ _ _ Encrypting or encapsulating the digital file Marking the digital image with ownership (visible or not) – http://is.gseis.ucla.edu/impact/f95/Papers-projects/Projects/Trowbridge/labels.html

Image quality – Onscreen quality is far lower than printed quality – http://is.gseis.ucla.edu/impact/f95 /Papers-projects/Projects/Trowbridge/resolution.html

Besser--UCLA/Getty Summer Instit-intro 8/6/01 32

Effect on the Institution _ _ _ _ _ _ Creating/Maintaining a WWW site Wear and tear on the original Handling external requests for Special Collection material Increase or decrease in requests to see originals?

New Audiences Implications on the Institution’s Public Image Besser--UCLA/Getty Summer Instit-intro 8/6/01 33

Scholarship and Interoperability _ _ _ Why Digital Libraries need standards for interoperability Metadata concerns Digitization means New Audiences Besser--UCLA/Getty Summer Instit-intro 8/6/01 34

_ _ _ _ Digitization means New Audiences more access for more people outreach to new groups but new groups have different usability requirements – different user interfaces – different vocabulary – new methods of navigation we already have enough differences btwn different institution types (& even within the same type) – MESL results – Organization & indexing reflects the biases of the original intent when records were formed Besser--UCLA/Getty Summer Instit-intro 8/6/01 35

Serious Longevity Problems _ _ _ _ What we know from prior widespread digital file formats Images separating from their metadata Inaccessibility of software needed to view an image Inability to even decode the file format of an image Besser--UCLA/Getty Summer Instit-intro 8/6/01 36

The Short Life of Digital Info: Digital Longevity Problems _ _ _ _ _ _ Disappearing Information The Viewing Problem The Scrambling Problem The Inter-relation Problem The Custodial Problem The Translation Problem Besser--UCLA/Getty Summer Instit-intro 8/6/01 37

The Viewing Problem  Digital Info requires a whole infrastructure to view it  Each piece of that infrastructure is changing at an incredibly rapid rate  How can we ever hope to deal with all the permutations and combinations Besser--UCLA/Getty Summer Instit-intro 8/6/01 38

The Scrambling Problem Dangers from:  Compression to ease storage & delivery  Container Architecture to enhance digital commerce Besser--UCLA/Getty Summer Instit-intro 8/6/01 39

The Inter-relation Problem  -Info is increasingly inter-related to other info  -How do we make our own Info persist when it points to and integrates with Info owned by others?

 -What is the boundary of a set of information (or even of a digital object)?

Besser--UCLA/Getty Summer Instit-intro 8/6/01 40

The Custodial Problem  How do we decide what to save?

 Who should save it?

 How should they save it?

– – -methods for later access: emulation, migration, etc.

-issues of authenticity and evidence Besser--UCLA/Getty Summer Instit-intro 8/6/01 41

The Translation Problem  Content translated into new delivery devices changes meaning – – -A photo vs. a painting -If Info is produced originally in digital form in one encoded format, will it be the same when translated into another format?

– Behaviors Besser--UCLA/Getty Summer Instit-intro 8/6/01 42

Pieces of the Solution (1/2)  -We need to insist upon clearly readable standardized ways for digital objects to self identify their formats  -We should discourage scrambling  -We need to better understand information inter-relates to other Info, and what constitutes “boundaries” of Info objects Besser--UCLA/Getty Summer Instit-intro 8/6/01 43

Pieces of the Solution (2/2)  -People and organizations wishing to make information persist need guidelines of how to go about doing it  -We need to better understand how translating from one storage or display format to another affects the meaning of a work  -We need to save the “behaviors” of a digital object, not just it’s “contents” 44

Conceptual Approaches to Digital Preservation _ _ _

Refreshing

always necessary due to volatility of physical strata – Impact on evidential value

Migration

-- advantages & disadvantages

Emulation

-- advantages & disadvantages Besser--UCLA/Getty Summer Instit-intro 8/6/01 45

Migration/Refreshing _ Impact on evidential value Besser--UCLA/Getty Summer Instit-intro 8/6/01 46

Pragmatic Issues _ _ _ _ _ Save Metadata!!! (Descriptive, Administrative, Structural, …) Separate master from delivery Consistent file-naming conventions Good work-flow Develop cooperative long-range plans Besser--UCLA/Getty Summer Instit-intro 8/6/01 47

Metadata can be the first line of defense  Can tell you – where the file is (if you can’t find the file) – – – where more info about the file is (if you have the file but most other metadata has become separated) what the file format is what the compression scheme is – what application program and version is needed for the file Besser--UCLA/Getty Summer Instit-intro 8/6/01 48

Groups Working on the Big Longevity Problem http://sunsite.Berkeley.EDU/Longevity/  CPA Task Force  Getty “Time & Bits” Conference & Follow-ups  Emulation experiments in US and Europe  NEDLIB, CURL, Michigan  LC  Mellon-funded E-Journal Archive experiments  Internet Archive  Long Now Besser--UCLA/Getty Summer Instit-intro 8/6/01 49

Library to Lead National Effort to Develop Digital Information Infrastructure and Preservation Program

(1 of 2) U.S. Congress Provides $100 Million Special Appropriation in Support of Project http://www.loc.gov/today/pr/2001/01-006.html

_ _ _ Response to NAS report LC 21: A Digital Strategy for the Library of Congress The Library of Congress has been empowered by the U. S. Congress to develop a national program to preserve the burgeoning amounts of digital information, especially materials that are created only in digital formats, to ensure their accessibility for current and future generations.

"This collaborative strategy will permit the long-term acquisition, storage and preservation of digital materials, that will assure access to the growing electronic historical and cultural record of our nation," said Dr. Billington. "Just as the Congress enabled the Library of Congress to begin the last century by making its printed catalog cards widely available, the Congress has enabled its Library to begin this century by building a digital record and making it available in the information age.” Besser--UCLA/Getty Summer Instit-intro 8/6/01 50

Library to Lead National Effort to Develop Digital Information Infrastructure and Preservation Program

(2 of 2) U.S. Congress Provides $100 Million Special Appropriation in Support of Project http://www.loc.gov/today/pr/2001/01-006.html

_ _ _ In December 2000, the 106th Congress appropriated $100 million for this effort, which instructs the Library to spend an initial $25 million to develop and execute a congressionally approved strategic plan for a National Digital Information Infrastructure and Preservation Program. Congress specified that, of this amount, $5 million may be spent during the initial phase for planning as well as the acquisition and preservation of digital information that may otherwise vanish. The legislation authorizes as much as $75 million of federal funding to be made available as this amount is matched by nonfederal donations, including in-kind contributions, through March 31, 2003. The effect of a government-wide recission of .22 percent in late December was to reduce this pecial appropriation to $99.8 million. The Library will consult with federal partners to assess joint planning considerations for shared responsibilities. The Library will also seek participation from the nonfederal sector and will execute its overall strategy in cooperation with the library, creative, publishing, technology and copyright communities in this country and abroad. Besser--UCLA/Getty Summer Instit-intro 8/6/01 51

_ _ Moving from Digital Collections to Digital Libraries/Museums What’s the difference?

– not experiments – real users – service – longevity Recent history of Library Automation Besser--UCLA/Getty Summer Instit-intro 8/6/01 52

Ideal Digital Collection Model DL DL DL DL search & presentation user Besser--UCLA/Getty Summer Instit-intro 8/6/01 user 53

Developmental Stages _ _ _ _ Experiment with methods Build real operational systems Build interoperable operational systems Make the system useful for users – For DL Initiatives – For OPACs – For I & A Services – For Image Retrieval Besser--UCLA/Getty Summer Instit-intro 8/6/01 54

One Final Question: Who will collect the digital works of today that should become the Special Collections of tomorrow?

_ _ _ _ _ web sites zines electronic journals listserve and email discussions drafts of works that later become famous Besser--UCLA/Getty Summer Instit-intro 8/6/01 55

UCLA/Getty Summer Institute for Knowledge Sharing http://www.getty.edu/gri/standard/intrometadata/ http://www.cdlib.org/libstaff/technology/tas/Standards/ http://sunsite.berkeley.edu/Imaging/Databases/ http://sunsite.berkeley.edu/moa2/ http://sunsite.berkeley.edu/Longevity/ http://www.oac.cdlib.org/ http://lcweb.loc.gov/ead/ http://purl.oclc.org/metadata/dublin_core/ http://www.gseis.ucla.edu/~howard/image-meta.html

http://www.gseis.ucla.edu/~howard/Metadata/UC-May00/ http://sunsite.berkeley.edu/Metadata/sp2000.html

http://www.gseis.ucla.edu/~howard/ http://is.gseis.ucla.edu/impact/f95/special-collectns.html

http://is.gseis.ucla.edu/impact/f95/Papers-projects/Projects/Trowbridge/ Besser--UCLA/Getty Summer Instit-intro 8/6/01 56