Transcript UC Libraries and the Implications of Mass Digitization Robin L. Chandler User’s Council May 11, 2007

UC Libraries and the Implications
of Mass Digitization
Robin L. Chandler
User’s Council
May 11, 2007
Seek to achieve in this talk:
• Status report on UC Libraries’ mass
digitization projects
• Impact of mass digitization on our
collections and our users
UC Libraries’
Mass Digitization Projects
• Overview of two projects
– Microsoft/Internet Archive
– Google Books
• Look at Operations
• April 2007 status report on scanning
Understanding Participant Roles
• UC Libraries
– supply & curate books
– preserve digital files created
– supply onsite scanning facilities when
appropriate
• Third-parties (Google, Microsoft, Yahoo)
– provide funding for book scanning
– manage digitization vendor
Microsoft / Internet Archive
• Production scanning began April 2006
• Internet Archive: Digitization Agent
• Projected scope 100 K books (public
domain) per year
– Scanning books from all campus libraries
• Scanning Centers (20 scanning machines)
– Location: UC at NRLF and SRLF
Google
• Production scanning began October 2006
– Scanning books from NRLF currently
• Projected Scope
– 2.5 million books during 6 year period
– Public domain /in-copyright
• Scanning Center
– Books transported to offsite scanning facility
– Over 3K book / per day
Workflow Steps (1)
• #1 Project management
• #2 Select, retrieve, inspect, mass charge /physical
charge, physical transfer
• #3 Sharing bibliographic records (over 3 K daily)
• #4: Digitization: creating content files & metadata
– JP2000, PDF, OCR
– Metadata created during scanning including
image coordinates
Workflow Steps (2)
• #5 Mass discharge / manual charge; books
returned to shelves
• #6 Quality control on digital files prior to
ingest
• #7 ingest of metadata and content files for
preservation storage
• #8 Enhance union and local catalog records
with link to hosted content
Motives: UC Libraries exploring models
• Collection Management: Digital reformatting can
help support our efforts to build shared print
collections
• Curating through Collaboration: Digitization of
local materials creates access (for our patrons) to
third-party materials not currently available
• Funding Reallocation: Funds invested in licensing
online collections of out of copyright materials
could be reallocated to digital reformatting our
unique content
Mass Digitization Collection
Advisory Group (MDCAG)
• Approved by University Librarians
• First meeting March 2007
• Charge:
– Develop process for selection of book
collections for scanning from across UC
Libraries
• Collection Development Committee (CDC)
will approve collection selections
April 2007 Status Report
• Google
–
–
–
–
249,485 books transferred
235,633 books scanned
11,320 books rejected
55,264 books live
• Microsoft/Internet Archive
– 84,315 books transferred
– 58,543 books scanned / books live
– 25,772 books rejected
Success due to our Systemwide
collaboration!
• UCB & UCLA Libraries / Northern and Southern
Regional Library Facility teams
• UC Library systemwide groups: ULs, SOPAG
CDC, PAG, HOPS, Bibliographer Groups
• Mass Digitization Collection Advisory Group
(MDCAG)
• CDL Programs: Bibliographic Services,
Collections, Data Acquisitions, Digital
Preservation Repository
Microsoft: Sample Book
Internet Archive: Sample Book
Google: Sample Book
Impacts of Mass Dig
• Will we re-define our collections ?
• How should we make collections available
to our users?
Mass Dig: Collections & Users:
• All Libraries can be bigger than before
– Leveraging the collections of other libraries to bring content to our
users
• Leveraging our collections ala the Long Tail
– Libraries can learn from Netflix
• Digitize local content – we all have special stuff!
– Unique holdings support specialized disciplines
• Prepare: demand for the physical item may increase
– Digital access may increase relevance of analog
• Book discovery increasingly happens outside the library
– Information discovery (Google, MSM, Yahoo!)
– Bibliographic discovery (Amazon)
Our Users Today
• Faculty, Graduates and
Undergrads
• Working in range of
disciplines
• Seeking efficiencies
• Define their tool space
• Resource needs are diverse
– Can very day by day
• They judge resource’s worth
Dawn of the Embedded Library (1)
• Web services embed library content into the browsing
experience of users
– Enable discovery, locate, request, and delivery
– Library content must be exposed to aggregators
• Examples: Library Thing, NCSU’s Catalog WS, LibX
Firefox, Google Book Search
– integrating web services for users and customizing software
– Leveraging Catalog, Open URLs, COinS, APIs, etc.
Dawn of the Embedded Library (2)
• Providing user services
– Find in a library, POD, download mobile
devices, ILL, order from Amazon, etc.
• Expose our content to aggregators and
consume the data of others
– OAI-PMH, SRU, Google Sitemap, Open
Search, RSS feeds, mobile device searching
Library Thing: Catalog Your Books
Online – social bookmarking
NCSU’s CatalogWS
LibX: Providing direct access to
your library’s resources
Mass Dig & New Library Services
• What systems are required to extract meaning
from massive text collections?
– Machine translation, data mining, etc.
• What new modes of reading, representation and
understanding are needed to interact with texts?
– Linguistic, visual, and statistical processing
• What collaborations between librarians, computer
scientists and scholars are needed to do this
exploration?
– Standards, search queries, visualization, social
networks
Epilogue: Mr. Peabody’s
WABAC (wayback) Machine
• 1992 Conference on “Technology, Scholarship and the
Humanities: The Implications of Electronic Information
asked certain questions:
• Will scholarship be better if it takes advantage of
technology?
• How will technology affect
– The book?
– The lecture?
– The library?
– The classroom?
1992: Historical Context
• Cold War formally ended & US lifted trade sanctions
against China
• Bill Clinton was elected U.S. President
• Four police officers were acquitted in Rodney King Trial
• Johnny Carson left the Tonight Show
• Earth Summit held in Rio de Janeiro
• CD sales surpassed cassette tapes
• OPACs and Gopher were in the library and a text-based
web browser was first made available to the public…..
Technology, Scholarship, & Humanities
Conference: Viewpoints (1)
• Richard A. Lanham, Professor of English, UCLA
“As traditionally taught, each class exists in a temporal, conceptual
and social vacuum…but if an electronic library were
employed…students could read papers submitted in earlier classes,
read scholarly articles on the same topics, read before-and-after
examples of revised work, do searches of Shakespearean texts for
imagery or rhetorical figuration, and make excerpts of videotaped
performances to illustrate their papers – all without going to the
campus library. Most importantly, a course like this would have a
history and could be accessed by people in other courses; it would
constitute a continuing society, its students becoming citizens of a
commonwealth”
Technology, Scholarship, & Humanities
Conference: Viewpoints (2)
• William Y. Arms, VP Computing Services
Carnegie Mellon
• “The scientific community has long-funded its capital-intensive
projects with support from government and industry. In contrast, only
2 percent of humanities research funding comes from the U.S.
government. As a result, the humanities can undertake few large,
interdisciplinary projects unless the government and other funding
agencies perceive the outcome to benefit the entire academic
community…..”
Thank you
• Please feel free to contact me at
[email protected]