Transcript Developing the DigiQUAL Protocol for Digital Library Evaluation: A Progress Report

DigiQUAL™
Developing the DigiQUAL Protocol for
Digital Library Evaluation
Martha Kyrillidou, Director, Statistics & Measurement Program
Sarah Giersch, Consultant
Association of Research Libraries
MERLOT International Conference 2005
Nashville, Tennessee, USA
July 27, 2005
www.digiqual.org
Terminology & Context
• It’s…? “familiar” and digital
• National Science Digital Library Evaluating the entity and Evaluating the projects
• DigiQUAL: measure service quality in DLs
www.digiqual.org
Developing DigiQUAL Survey Items
• Background: ServQUAL ->LibQUAL ->DigiQUAL
LibQUAL Dimensions
of Service Quality
• Affect of Service
• Information Control
• Library as Place
DigiQUAL: 12 themes of DL service quality
Accessibility
Collection building
Navigability
Resource use
DL as community for users
Copyright
...developers Role of federations
Interoperability Evaluating collections
www.digiqual.org
...reviewers DL sustainability
Pilot Testing Survey Items & Implementation
www.digiqual.org
Building a Survey
• Review and select items
• Issue: aligning items to individual DL needs and
users – vocabulary & content
www.digiqual.org
Building a Survey (cont.)
Customize
Survey
Issue: Flexibility vs. Standardization
www.digiqual.org
Implementing Survey – Notification Methods
• Links on site
• Newsletters
• Next to resources
Issues: no pop-ups, no individual emails
www.digiqual.org
Implementing Survey - Incentives
Issues: must be easily transferable, requires
email address - clear IRB
www.digiqual.org
Analysis & Reporting - Issues
(mis)Interpreting results from individual digital
libraries in the context of other sites
Sites reluctant to share
data and results
www.digiqual.org
Outstanding Issues & Challenges
• Unique DLs: niche market, critical mass, both?
• Balance:
– custom vs. generic content ->results
– flexible vs. standard implementation -> scaling
• Mixed methods
– Preserving user privacy
– Collecting truly useful data
• Moving target: digital libraries as… it depends
www.digiqual.org
Next Steps
• Continue pilot testing: NSDL and beyond
• Expand implementation
• Consider sustainability models
• Want to Participate? Contact:
– Sarah Giersch
[email protected]
– Martha Kyrillidou
[email protected]
• For more info:
– www.digiqual.org
www.digiqual.org