Digital Libraries and Music

Jon Dunn SLIS L631 Music Librarianship Seminar April 7, 2003

Outline

 Digital Libraries  Music content  Variations  Variations2  Special topics:  Music information retrieval  Open Archives Initiative

What is a digital library?

  DL as collection/information system  

“a collection of information that is both digitized and organized” -- Mike Lesk, National Science Foundation “networked collections of digital text, documents, images, sounds, scientific data, and software” - President’s Information Technology Advisory Council report

DL as organization: 

“an organization that provides the resources, including the specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities” -- Digital Library Federation

Applications of music DLs

   Education   Electronic reserves Online instructional tools Research   Better access to special collections New capabilities for analysis, searching Commercial   Professionals  E.g. music/film/video production Consumers  Online music catalogs, digital distribution

What is a music digital library?

       What does it contain?

How is this content acquired?

How is this content accessed?

How can the content be used once located?

What is the purpose?

Who are the users?

How is content protected?

Music DL features

     Content  Selection, digitization, storage, delivery Metadata (cataloging) Search capabilities  for content and metadata Interfaces  User interfaces, programmatic interfaces Access control

Basic Representations of Music

Digital Audio Audio (e.g., CD, MP3): like speech MIDI file): like unformatted text with complex formatting

Content Formats

 Audio  MIDI  Scores  Images  Structured file format  (Video)

Digital audio

 Sampling  Sample rate, sample size, number of channels  Compression  Perceptual audio coding  File formats  Standards

Digital audio file formats

 Uncompressed –all basically the same  WAV - Microsoft/IBM   AIFF - SGI/Apple AU/SND - NeXT/Sun  Compressed   MPEG-1 layers 1-3, MPEG-2 AAC RealAudio, Windows Media, QuickTime  Each supports various compression options

Digital audio file sizes

 Uncompressed audio  44.1 kHz, 16 bit, stereo (CD quality)   650 MB for one hour 1.4 Megabits/second  Compressed    MP3: 58 MB for one hour, 128 Kilobits/second AAC: 29 MB for one hour, 64 Kilobits/second RealAudio, Windows Media Audio, QuickTime Qdesign Music: down to 20 Kilobits/second or less

Digital Audio

audio sampling quantization noise Barlow,

Multimedia Systems

, p. 77.

CD Audio

    Sample rate:  44.1 kHz (44,100 samples/second) Sample size:  16 bits Number of channels:  2 (stereo) Bitrate  44100 samples/second * 16 bits/sample * 2 channels = 1.4112 megabits/second (plus file format/network overhead)

Masking Effect

Barlow,

Multimedia Systems

, p. 73.

From

Research and Creative Activity

, September 1999

Digital audio delivery

 Delivery options  Download  Streaming  e.g. RealAudio, Windows Media, QuickTime Streaming  Encrypted download  e.g. LiquidAudio, a2bmusic, Windows Media

Scores

 Score image  File format: TIFF, JPEG, GIF, PDF, …  Resolution  Grayscale vs. bitonal (black and white)  Score notation  Many proprietary formats  No common standard

Attributes of notated musical information

 Pitch  Duration  Tempo  Dynamic level  Articulation  Part (sometimes implying timbral definition) Selfridge-Field,

Beyond MIDI

, p. 9.

Difficulties in representing CMN

 Grammar of CMN is open-ended  Which is more critical: graphical appearance or semantic meaning?

 Much left open to interpretation  Style differences, e.g. interpretation of rhythms

Music Notation File Formats

 www.music-notation.info

lists over 50 different music notation formats, most for CMN

MIDI

   M usical I nstrument D igital I nterface  Originally a hardware interface spec  Communication of real-time events between musical devices Standard MIDI File (SMF)   Stores time-stamped MIDI event information  e.g note on/off, key pressure, aftertouch, pitch bend, control change, program change.

Each event accompanied by parameters  e.g. note on includes pitch, duration, dynamic range Spec maintained by industry group  MIDI Manufacturers’ Assocation

Limitations of MIDI

 MIDI does not represent many musical attributes  Graphical notation elements  Rests, stem direction, enharmonic distinctions, staff systems, page layout, etc.

 Sound elements  Timbre, full stylistic expression  Extensions exist but not widely used

Creating Notation Content

 Transcription  Music notation editor  ASCII data entry  Recognition  OMR: Optical Music Recognition

Music notation editor example: Finale

OMR: Optical Music Recognition

 Commercial packages  Musitek MidiScan/SmartScore  Version included with Finale  Neuratron PhotoScore  Version included with Sibelius

OMR: A long way from OCR

Here's the original: Scanned into Finale: Only 5 easy edits needed.

Taken from http://www.codamusic.com/finale/scanning.asp

OMR: Optical Music Recognition

 Research projects  CANTOR  University of Waikato, New Zealand  Adaptive OMR  Johns Hopkins University, USA  Example: http://mambo.peabody.jhu.edu/omr/demo/  others:  http://www.cs.waikato.ac.nz/~davidb/omr/

Score Images

 Scanned images still useful  Historical editions, manuscripts  Preservation, improving access  Impracticality of large-scale OMR  Music presents challenges for scanning

Variations

 Digital library of music sound recordings and scores  Online since 1996  Accessible in Music Library and other select locations - copyright  Used daily by large student population

Original Concept

 Burroughs and Fenske, 1990  VARIATIONS name 

Theme and Variations

 Variety of information formats for music  Networked access for the music student or scholar to sound recordings, scores, textual materials, video recordings

Focus on Audio

 High demand portion of collection  Fragile formats  Lack of previous work; uniqueness

Focus on Audio Reserves

 Half of sound recording use from reserves  Problems with existing practices  Cassette tape dubs, analog distribution systems  Concentrated use of a few items at any given time

Variations System

 Digitization  Storage  Access

Design and Development

     Developed by Music Library with assistance from UITS and Library Information Technology Integrate rather than develop from scratch Partnership with IBM Funding: School of Music, Libraries, UITS, IBM Online in April 1996

Digitization

 Formats  Analog: LP, cassette tape, reel-to-reel tape  Digital: CD, DAT  Capture at CD quality   44.1 kHz, 16 bit, stereo, 700MB for one hour Compress to MPEG 1 layer 2 (“MP2”)   200 MB for one hour Create “track file”

Digitization Hardware and Software

 Windows PCs  Sound capture card  Microtest Disc-to-Disk CD capture software  Sonic Foundry Sound Forge XP audio editor

Music Library Digitizing Lab

Storage

 Tivoli Storage Manager software  IBM RS/6000 AIX server  IBM Tape Library Dataserver  Contains three tape drives  10 terabyte (10,000 gigabyte) capacity

IBM 3494 Tape Library

IBM 3590E tapes: 20GB each

Access

 Discovery  How does the user find the desired recording?

 Playback  How is audio delivered to the user?

 How does the user navigate within a given recording?

Collection

 Currently: 6900 titles, 8000 hours of audio  5.6 TB uncompressed  1.6 TB compressed  Opera, songs, instrumental music, jazz, rock, world music

Discovery

 Varies based on purpose of access  Reserves  Course reserve lists  Faculty-created course home pages (incl. Oncourse)  General use  Links from IUCAT library catalog (856 fields in MARC bib records)

Playback

 Streaming server  IBM RS/6000, 150 GB disk  IBM VideoCharger server software  Software to connect VideoCharger with TSM (locally written)  Client  IBM VideoCharger client software  Variations Player (locally written)  Navigation via track files

Network

 Originally ATM  25/100/155 megabits per second  Now switched Ethernet  10/100/1000 megabits per second  Variations audio stream requires 384 kilobits/second  Up to 150 streams

Variations Demonstration

Variations2

 Four-year project  Started October 1, 2000      Funding from NSF and NEH through Digital Libraries Phase 2 (DLI2) program Large interdisciplinary team of investigators Faculty: Music, Information Science, Law, Computer Science Librarians and technologists: Libraries, University Information Technology Services Bloomington and Indianapolis campuses

Project goals

   Establish a digital music library testbed system supporting multiple formats: audio, video, score images, score notation Develop multiple interfaces for specific user applications in the music library and the classroom Conduct research in metadata, usability, copyright, and networking

Partners: “Satellite Sites”

    United States    University of Illinois at Urbana-Champaign University of Massachusetts at Amherst Northwestern University United Kingdom   Kings College - London Loughborough University  University of Oxford Japan  Waseda University Evaluation…potential for co-development

The Variations2 System

     Integrated access to music in all formats     Digital audio recordings Score images Score notation Video Delivery to wide range of users    Faculty: teaching, course design, research Students: coursework, independent study Music librarians, other library users Extensible Multiple user interfaces Staged development

Variations2 Version 1.0 Features

    Infrastructure  Data/metadata repositories, authentication, logging Search and retrieval interface  Based on new data model Presentation/navigation of audio and scanned scores Bookmarking

Variations2 Version 1.0 Technical Environment

      Client and server developed in Java Windows and Mac OS X client platforms, Unix (AIX/Linux) server Audio streaming: QuickTime for Java, Darwin Streaming Server Database: IBM DB2, DB2 Text Information Extender Image compresssion: DjVu from AT&T Labs and Lizardtech XML/MARC/Z39.50 tools: Saxon, Xerces, Jafer, James

Variations2 1.0

System Architecture: Layer View

Metadata Audio Video Score images Score notation Repositories Search Image retrieval Notation retrieval Sound playback and others...

Access Components Search tool Sound player Score viewer Digital time liner Cataloging tools and others...

User Interface Components Applications General purpose library application Multimedia Music Theory Teaching application Oncourse Cataloging/ adminsistration application General user Theory student or instructor Non-major music student or instructor Cataloger or digitization technician

Variations2 1.0

Communications

Apache HTTP Server Darwin Streaming Server DB2 Database JDBC Variations2 Kerberos Server Kerberos IU Kerberos Server Variations2 Library Server RTSP/RTP HTTP Java RMI Variations2 Client

Demonstration

Usability

   Usability = ease of use +

usefulness

Established baseline    Usability test of existing Variations system Satisfaction study of Variations users Contextual inquiry Evaluation of usability of Variations2    Prototype interviews Usability tests of preliminary versions Pilot studies  Data gathering through satisfaction survey and automated usage logging

Future Versions

  Continuing 6-month development cycle    Version 2: Spring 2003 Version 3: Fall 2003 etc.

Features to be added include:   Support for music notation Support for additional image and audio formats   Support for new and emerging streaming technologies Support for video

Future Versions

 Features to be added (continued):          Support for supplemental recording materials (e.g., liner notes, booklets) Improved browsing interface User interface support for synchronized navigation and playback Instructional authoring, classroom presentation, and instructional delivery interfaces Structure diagramming/visualization tools (e.g., Digital Timeliner) Web browser interface OnCourse integration Access control based on intellectual property requirements Improved cataloging/administrative interface

Variations2 Version 2 Demo

Music Information Retrieval

 Areas of research:  Indexing and search of music content  Audio, MIDI, notation  Feature detection  Genre, style, form, instrumentation, …

Music IR: Inherent Difficulties in Music

   No analogue to ‘words’  No easy units on which to index or do synonym lookup, etc.

Problems of representation  Graphical vs. logical aspects of music Polyphony    Multiple voices, chords Cross-voice matching Music is not linear

Polyphony

Polyphony

 Almost all music IR work to date focused on pitch matching in

monophonic

music

Music IR: More problems

    Query specification   What would a musical query look like?

“Query by humming” Music perception  People do not always perceive pitch correctly What type of matching?

    Exact pitch or intervals Melodic contour Exact rhythm “Rhythmic contour” What to index?

  Entire works Themes

Music IR: Yet more problems

    Variety of users, probably with very different needs, including:   General public looking for pop music Music students, scholars However, no formal assessment of user needs No standard query sets, relevance judgements, or test collections Problems of copyright in building test collections

More information on Music IR

  ISMIR: International Symposium/Conference on Music IR     2000: Plymouth, Massachusetts, USA 2001: Bloomington, Indiana, USA 2002: Paris, France October 26-30, 2003: Baltimore, MD, USA http://www.ismir.net/

OAI: Open Archives Initiative

 Original problem: searching across e print archives  Distributed searching hard  e.g. Z39.50

 Varying search semantics, capabilities  Network, server problems  Solution: metadata harvesting

Metadata harvesting

  Extract metadata from various sources Build services on local copies of metadata all searching, browsing, etc. performed on the metadata here Individual repositories can still support direct user interaction

metadata harvested offline metadata harvested offline

user search for “Mozart” Service provider

metadata harvested offline

local copy of metadata

metadata harvested offline

. . .

Data providers

OAI roles

  Data Providers   Repositories of digital content and metadata Support harvesting of metadata via the OAI protocol Service Providers  Harvest metadata from data providers using the OAI protocol   Implement user interface to data  Usually for searching, but other services also possible Can be selective

OAI protocol

       Originally developed in 1999 (“Santa Fe Convention”) Original focus on E-prints Has grown into general metadata harvesting protocol 

OAI-PMH: OAI Protocol for Metadata Harvesting

Version 1.0: January 2001 Version 1.1: June 2001  Conform to XML Schema 1.0

Version 2.0: June 2002  Transition period through December 2002 Currently 53

registered

OAI data providers

OAI protocol

 Carried over HTTP  Requests: HTTP GET or POST  Responses encoded in XML  Format defined via XML schema  Metadata in simple (unqualified) Dublin Core (and potentially other formats)

Dublin Core elements

 Coverage  Description  Type  Relation  Source  Subject  Title  Contributor  Creator  Publisher  Rights  Date  Format  Identifier  Language

OAI verbs

Verb

Identify ListMetadataFormats ListSets ListIdentifiers ListRecords GetRecord

Function

description of archive metadata formats supported by archive sets defined by archive OAI unique ids contained in archive listing of N records listing of a single record

OAI resources

 Web site, mailing lists  Repository explorer  Data/service provider toolkits www.openarchives.org

Becoming an OAI data provider

    Make digital content available on web Translate metadata into Dublin Core   Crosswalks exist for MARC Can also make other formats available, e.g. MARC XML Choose a unique identifier system Set up OAI data provider server software   See tools list at www.openarchives.org

Depending on tool, uses its own database or operates over existing database

Other technical concerns for scores

   Areas for standardization/agreement    Within-score navigation  User interface, supporting metadata Image file format MARC-DC metadata crosswalk Not essential to OAI model, but enables more consistent user experience Packaging scores for exchange between libraries, e.g. for e-reserves, cooperative preservation  Can METS play a role?

Examples of OAI service providers

 UIUC Cultural Heritage Repository  http://dlc.grainger.uiuc.edu/  UMich OAIster  http://www.oaister.org/  RLG Cultural Materials  http://www.rlg.org/culturalres/  UCLA/JHU/IU Sheet Music Harvester  http://digital.library.ucla.edu/sheetmusic/