Transcript LibQUAL+TM : A New Way of Listening To Users


Service Quality Assessment in a
Digital Library Environment
4th International JISC/CNI Conference
Joseph Boykin, Fred Heath, Duane Webster
Edinburgh, Scotland
June 27, 2002
Overview of Discussions
1. New Models for Understanding and
Describing Library Success
2. ARL’s New Measures Initiative
3. LibQUAL+ Project Development
4. Experience with LibQUAL+
5. Conclusions and Next Steps
The Association of Research
Libraries
www.arl.org
Mission:
Shaping and influencing forces affecting the
future of research libraries in the process of
scholarly communication.
Members:
America.
123 major research libraries in North
Ratios:
4% of the higher education institutions
providing 40% of the information resources.
Users:
3 million students and faculty served.
Expenditures: $2.35 billion annually, $727 million for
acquisitions of which 9% is invested in
access
to electronic resources.
ASSOCIATION OF RESEARCH LIBRARIES
The Problem of Assessment in
Research Libraries
• ARL Membership Criteria Index variables
emphasize inputs, primarily expenditures
• To rise in the ARL Index it is only
necessary to spend more
• No demonstrable relationship between
expenditures and service quality
• The lack of metrics describing
performance
Systems Model for Organization Design
FUTURE
ENVIRONMENT
ENVIRONMENT
MISSION
INPUTS
WO RK PRO CE SSES
SUPPLIERS
RESOURCES
OUTPUTS
CUSTOMERS
OUTCOMES
&
HUMAN AS PE CT S
FEEDBACK
FEEDBACK
VISION
PEOPLE / SKILLS
STRUCTURE / SYSTEMS
CULTURE / CLIMATE
FROM : Association of R ese arch Libraries, Office of
Leadership and Management Services, Washington DC
STAKEHOLDERS
ARL New Measures Initiative
• Collaboration among member leaders with
strong interest in this area
• Specific projects developed with different
models for exploration
• Intent to make resulting tools and
methodologies available to full membership
and wider community
ARL New Measures Projects
• Demonstration project for service effectiveness
measures (LibQUAL+)
• Project to define usage measures for electronic
information resources (includes institutional
outcomes)
• Investigation of role libraries play in support of the
research process
• Investigation of role libraries can play in campus
learning outcomes activities
• Identification of cost-drivers and development of
cost-benefit studies
Assessment
“The difficulty lies in trying to find a single model
or set of simple indicators that can be used by
different institutions, and that will compare
something across large groups that is by definition
only locally applicable –i.e., how well a library
meets the needs of its institution. Librarians have
either made do with oversimplified national data or
have undertaken customized local evaluations of
effectiveness, but there has not been devised an
effective way to link the two” Sarah Pritchard,
Library Trends, 1996
LibQUAL+™ Description
LibQUAL+TM is a research and
development project undertaken to
define and measure library service
quality across institutions and to
create useful quality-assessment
tools for local planning.
The Purpose of Our Research



To fill a knowledge void in modeling the
dimensions of library service quality from a user
perspective
Based upon the model, to develop a webdelivered, effective total market survey
instrument equivalent for service quality
assessment in academic libraries
Using the derived instrument to recommend a
process for an ongoing program of comparative
outcome measurement for academic libraries
LibQUAL+™ Project Goals
1. Development of web-based tools for
assessing library service quality
2. Development of mechanisms and
protocols for evaluating libraries
3. Identification of best practices in
providing library service
4. Establishment of a library service quality
assessment program at ARL
Project Resources
LibQUAL+TM is an ARL/Texas A&M
University joint effort. The project is
supported in part by a 3-year grant
from the U.S. Department of
Education’s Fund for the Improvement
of Post-Secondary Education (FIPSE).
TM
LibQUAL+ Team
•
•
•
•
•
•
ARL
Duane Webster
Martha Kyrillidou
Kaylyn Hipps
Julia Blixrud
Jonathan Sousa
Consuella Waller
•
•
•
•
•
•
TAMU
Fred Heath
Colleen Cook
Bruce Thompson
Yvonna Lincoln
Trey Thompson
Julie Guidry
LibQUAL+TM Project History
9/99 - ARL launches “New Measures Initiative” which includes the study of service
effectiveness known as SERVQUAL spearheaded by Texas A&M University.
1/00 - Initial 12 institutions begin the SERVQUAL study led by the Texas A&M team.
7/00 - LibQUAL+ as a distinct library-based assessment tool is presented.
9/00 - ARL and Texas A&M awarded a FIPSE grant to fund further development of the
LibQUAL+ project.
10/00 - The ARL symposium, “New Culture of Assessment in Academic Libraries Measuring
Service Quality” attracts a group of 170 people.
1/01 - Representatives from 43 research and university libraries participating in the Spring 2001
Implementation meet in Washington, DC during ALA midwinter.
6/01 - National Science Foundation awards grant to ARL and Texas A&M to adapt LibQUAL+
for NSDL
LibQUAL+™ Contribution
• Grounded questions yield data of sufficient
granularity to be of local use
• Normative data across cohort group at first and
higher order levels
• Surfaces “Best Practices”
• Web approach makes little demand of local
resources while compiling robust dataset
Relationships: perceptions,
service quality and satisfaction
….only customers judge quality;
all other judgments are essentially
irrelevant”
Zeithaml, Parasuraman, Berry. (1999).
Delivering quality service. NY: The Free
Press.
LibQUAL+TM Participants
Year 3
Year 2
170 Participants
43 Participants
Year 1
12 Participants
Spring 2000
Spring 2001
For More Information about Participants:
Visit the LibQUAL+ web site.
Spring 2002
SURVEY
INSTRUMENT
Dimensions of Library
Service Quality
Affect of Service
Library
Service
Quality
Personal Control
Empathy
Ease of Navigation
Responsiveness
Convenience
Assurance
Modern Equipment
Reliability
Information Access
Library as Place
Content/Scope
Utilitarian space
Timeliness
Symbol
Convenience
Refuge
LibQUAL+ Core Questions Y1
_____________________________________________________________________________
Factor_
______ _
No.
I
II
III
IV
Item Core
_____________________________________________________________________________
32
.84947
.12848
.24465 .13335
1 Willingness to help users
33
.80847
.13662
.25348 .14147
1 Giving users individual attention
7
.80757
.17881
.12781 .21125
1 Employees deal with users caring fashion
50
.79273
.19288
.18847 .12497
1 Employees who are consistently courteous
31
.77262
.16358
.26461 .20061
1 Employees have knowledge answer questions
5
.74072
.14754
.18453 .29624
1 Employees understand needs of users
3
.74052
.15102
.17296 .20793
1 Readiness to respond to users' questions
18
.71718
.19757
.18289 .26766
1 Employees who instill confidence in users
43
.62487
.22402
.29970 .28256
0 Dependability handling service problems
20
2
19
25
41
.16556
.17739
.22362
.16013
.20398
.87679
.83172
.83147
.80492
.80204
.11430
.08498
.14705
.18894
.17599
.16236
.13901
.22566
.16628
.20255
2 A haven for quiet and solitude
2 A meditative place
2 A contemplative environment
2 Space that facilitates quiet study
2 A place for reflection and creativity
37
28
14
45
17
29
.22528
.19602
.33339
.30467
.35390
.30136
.12353
.09611
.16156
.23784
.18467
.21018
.78405
.75780
.60389
.59090
.55690
.55341
.15466
.13173
.31109
.28919
.41864
.38474
* website enabling me locate info on my own
* elec resources accessible home or office
* access tools allow me find on my own
3 Modern equip me easily access info I need
* info easily accessible for independent use
4 Convenient access to library collections
11
39
16
9
8
.13494
.14894
.29445
.27782
.22850
.23183
.23743
.19831
.05333
.18484
.18868
.29367
.22384
.16331
.13137
.73636
.60350
.60107
.57866
.56343
3 Comprehensive print collections
3 Complete runs of journal titles
3 Interdisciplinary library needs addressed
4 Timely document delivery/interlibrary loan
0 Convenient business hours
________________________________________________________________
Affect of Service
• Emerged as the dominant factor early in our work
• Absorbed several of the original SERVQUAL
questions measuring Responsiveness, Assurance
and Empathy
• In the current analysis also includes Reliability
• All in all: the Human Dimension of Service
Quality
Library as Place
• Transcends the SERVQUAL
dimension of Tangibles to include the
idea of the library as the campus
center of intellectual activity
• As long as physical facilities are
adequate, library as place may not be
an issue
Personal Control
• How users want to interact with the
modern library
• Personal control of the information
universe in general and web
navigation in particular
Access to Information
• Ubiquity of access: information
delivered in the format, location and
time of choice
• Comprehensive collections
Project Deliverables
Print and web-based results include:
–
–
–
–
–
–
Aggregate Summaries
Demographics by Library
Item Summaries
Dimension Summaries
A copy of the survey instrument
Dimensions measured for survey
implementation
Two Interpretation Frameworks
• Score Norms
• Zone of Tolerance
Score Norms
• Norm Conversion Tables facilitate the
interpretation of observed scores using
norms created for a large and
representative sample.
• LibQUAL+TM norms have been created at
both the individual and institutional level
C
Zone of Tolerance
C
• The area between minimally
acceptable and desired service quality
ratings
• Perception ratings ideally fall within
the Zone of Tolerance
ZONE OF TOLERANCE: DIMENSIONS
LibQUAL+™ Fundamental Contributions
to the Measurement of Effective Delivery
of Library Services
• Shift the focus of assessment • Determine the degree to which
information derived from local
from mechanical expendituredata can be generalized,
driven metrics to user-centered
providing much needed “best
measures of quality
practices” information
• Re-ground gap theory for the
library sector, especially
academic libraries
• Demonstrate the efficacy of
large-scale administration of
user-centered assessment
transparently across the web
LibQual+ 2000 and 2001 Results
LIBQUAL+
Category: Access to Information
2001
2002
Access to Information
All
All
Timely document delivery/interlibrary loan
0.19
0.5
0.31
-0.67
-0.06
0.61
0.24
0.26
0.02
Complete runs of journal titles
-0.97
-0.12
0.85
Convenient Business Hours
-0.03
0.62
0.65
Comprehensive print collections
Interdisciplinary library needs being addressed
Positive Gaps 0.60 or higher
Negative Gaps (regardless of value)
Change
LIBQUAL+
Category: Affect of Service
Affect of Service
Readiness to respond to users' questions
Employees who understand the needs of their users
Employees who deal with users in a caring fashion
Employees who instill confidence in users
Employees who have the knowledge to answer user questions
W illingness to help users
Giving users individual attention
Dependability in handling users' service problems
Employees who are consistently courteous
Positive Gaps 0.60 or higher
Negative Gaps (regardless of value)
2001
All
0.54
0.36
0.74
0.76
0.44
0.61
0.76
0.40
0.63
2002
All
Change
0.63
0.09
0.55
0.19
0.78
0.04
0.91
0.15
0.41
-0.03
1.00
0.39
0.89
0.13
0.50
0.10
0.74
0.11
LIBQUAL+
Category: Library as Place
Library as Place
A contemplative environment
A haven for quiet and solitude
Space that facilitates quiet study
A comfortable and inviting location
A place for reflection and creativity
Positive Gaps 0.60 or higher
Negative Gaps (regardless of value)
2001
All
0.12
0.07
0.16
0.64
0.38
2002
All
Change
0.50
0.38
0.52
0.45
0.39
0.23
0.65
-0.01
0.54
0.16
LIBQUAL+
Category: Personal Control
Personal Control
Easy-to-use access tools that allow me to find things on my own
Making information easily accessible for independent use
Making electronic resources accessible from my home or office
A library website enabling me to locate information on my own
Modern equipment that lets me easily access the information I need
Convenient access to library collections
Positive Gaps 0.60 or higher
Negative Gaps (regardless of value)
2001
All
0.28
0.15
-0.05
0.14
-0.11
0.41
2002
All Change
0.26
-0.02
0.31
0.16
0.19
0.24
0.12
-0.02
0.34
0.45
0.32
-0.11
NSF Grant
• Assess service quality in digital
libraries
• 3 year period
• Adopt LibQUAL+ instrument for use
in the Science, Math, Engineering and
Technology Education Digital Library
community (NSDL)
NDSL LibQUAL+ Goals
• Define dimensions of digital library service
quality from the users’ perspectives
• Develop tool for measuring user perceptions
and expectations of digital library service
quality across NSDL digital library contexts
• Identify digital library best practices that
permit generalizations across operations and
development platforms
NDSL LibQUAL+ Activities
• 120-200 qualitative interviews to contribute
to identifying dimensions of digital library
service quality
• Test and refine dimensions of digital library
service quality and self-sufficiency through
development of total market survey
• Implement survey across variety of
organizational and digital library
implementations
LibQUAL+TM Related
Documents
• LibQUAL+TM Web Site
http://www.arl.org/libqual/
• LibQUAL+TM Bibliography
http://www.coe.tamu.edu/~bthompson/servqbib
• Survey Participants Procedures
Manual
http://www.arl.org/libqual/procedure/lqmanual2.pd
f

LibQUAL+ Related
Documents
• LibQUAL+ Web Site
http://www.arl.org/libqual/
• LibQUAL+ Bibliography
http://www.coe.tamu.edu/~bthompson/servqbib
woof