Transcript Slide 1

LibQUAL+®: An Introduction
PRESENTED BY:
Martha Kyrillidou
Association of Research Libraries
Bern, Switzerland
July 9, 2007
old.libqual.org
Overview
• Introduction and Overview
• LibQUAL+® in the UK
• Research Foundations
– Qualitative Process
– Quantitative Evidence
• The Survey Process
• General Discussion and Q&A
old.libqual.org
old.libqual.org
old.libqual.org
World LibQUAL+® Survey
old.libqual.org
Rapid Growth in Other Areas
• Languages
–
–
–
–
–
–
–
–
–
• Types of Institutions
American English
British English
Chinese (2007)
French
Dutch
Swedish
Norwegian
Finnish
Danish
• Consortia
– Each may create 5 local questions
to add to their survey
• Countries
– Australia, Canada, Denmark,
Egypt, Finland, France, Hong
Kong, New Zealand, the
Netherlands, Norway, South
Africa, Sweden, Switzerland,
UAE, U.K., U.S.
old.libqual.org
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Academic Health Sciences
Academic Law
Academic Military
College or University
Community College
Electronic
European Business
Family History
FFRDC
High School (2007)
Hospital
National Health Service England
Natural Resources
New York Public
Public
Smithsonian
State
University/TAFE
®
LibQUAL+ Participants
350
200,000
300
180,000
307
308
176,360
160,000
151,460
250
152,111
250
140,000
128,958
218
204
200
113,480
Number of
Institutions
Number of
100,000 Responses
164
150
80,000
78,863
60,000
100
Number of Institutions
50
43
40,000
Number of Responses
20,000
20,416
13
0
120,000
4,407
2000
0
2001
2002
2003
2004
2005
2006
2007
Survey Year
old.libqual.org
* 2007 data reflects Session I data only
®
LibQUAL+ First Year Participants
old.libqual.org
* 2007 data reflects Session I data only
LibQUAL+® Surveys by Type
Years
2000
2003
2004
Academic law
1
25
Academic military
6
College or university
13
2001
41
Community college
2002
2005
2006
10
6
1
111
244
150
201
219
16
29
3
15
27
Electronic
1
European business
5
16
Family history library
1
2
FFRDC
Health sciences library
1
35
23
Hospital
5
1
13
13
9
10
1
1
National Health Service Eng.
10
Natural Resources
New York Public library
4
1
Public
Smithsonian
4
1
State
old.libqual.org
1
3
1
3
2
1
1
1
University/TAFE
1
LibQUAL+® Languages
American English
French Canadian
Swedish
British English
Afrikaans
Dutch English
Dutch
Continental French
Swedish
(British English)
German
Norwegian
Finnish
Danish
old.libqual.org
Surveys by Session: 2004-2006
Year
Session 1
Session 2
2004
202
2
2005
199
56
2006
205
93
old.libqual.org
Survey methods used in the UK
Source: Stephen Town, Paris, March 22-23, 2007
6
2
2
4
27
Libra
LibQUAL+
In-House
11
SPSS
SNAP
Perception
Excel
Others
13
18
old.libqual.org
West, 2004
A Survey of
Surveys
The UK approach
• Coordinated on behalf of the Society of College,
National & University Libraries (SCONUL)
Working Group on Performance Improvement
(WGPI)
• 2003 - 20 UK Higher Education (HE) institutions
• 2004 -17 UK & Irish HE institutions
• 2005 - 16 UK & Irish HE institutions
• 2006 – 20 UK & Irish HE institutions
• 2007 – 22 UK & Irish HE institutions
• 62 different institutions
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Response Comparisons
•
•
•
•
SCONUL 2003
– 20 institutions
– 11,919 respondents
SCONUL 2004
– 16 institutions
– 16,611 respondents
• Increase by 4,692
SCONUL 2005
– 16 institutions
– 17,355 respondents
• Increase by 744
SCONUL 2006
– 20 institutions
– 19,108 respondents
• Increase by 1,753
•
•
•
•
LibQUAL+ 2003
– 308 institutions
– 128,958 respondents
LibQUAL+ 2004
– 202 institutions
– 112,551 respondents
• Decrease by 16,407
LibQUAL+ 2005
– 199 institutions
– 108,504 respondents
• Decrease by 4,047
LibQUAL+ 2006
– 298 institutions
– 176,360 respondents
• Increase by 67,856
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
SCONUL Response by User Group 2006
old.libqual.org
SCONUL Response by Discipline 2006
old.libqual.org
Respondent Comparisons
• Glasgow University
– 2006 = 1,535
– 2005 = 1,384
– 2004 = 2,178
– 2003 = 503
• London South Bank University
– 2006 = 700
– 2005 = 766
Source: Stephen Town, Paris,
– 2004 = 568
March 22-23, 2007
– 2003 = 276
old.libqual.org
General findings
Source: Stephen Town, Paris, March 22-23, 2007
• Highly desired
– Making electronic resources accessible from
my home or office
– Print and/or electronic journals I require for
my work
– A haven for study, learning or research
• Lowest
– Library staff who instil confidence in users
– Giving users individual attention
– Space for group learning and group study
old.libqual.org
Free Text Comments Received
• Total number of comments 2005 = 8,368
• Total number of comments 2004 = 8,161
• Total number of comments 2003 = 7,342
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Why LibQUAL+?
•
•
•
•
•
Benchmarking
Cost effectiveness
Analysis compiled by LibQUAL+
Fast delivery of results
Support available, especially regarding analysis
of results
• Trialling alternative survey methods
• More library focused than previous in-house
method
• Planned institutional survey failed to happen.
LibQUAL+ was cost effective way of doing
something to fill the gap.
Source: Stephen Town, Paris, March 22-23, 2007
old.libqual.org
Research Foundations
old.libqual.org
Total Circulation
Total Circulation
600,000
550,000
500,000
450,000
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
400,000
Note. M. Kyrillidou and M. Young. (2003).
old.libqual.org
ARL Statistics 2002-03. Washington, D.C.:
ARL, p.8.
Reference Transactions
Reference Transactions
170,000
160,000
150,000
140,000
130,000
120,000
110,000
100,000
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
90,000
Note. M. Kyrillidou and M. Young. (2003).
old.libqual.org
ARL Statistics 2002-03. Washington, D.C.:
ARL, p.8.
Assessment
“The difficulty lies in trying to find a single model or
set of simple indicators that can be used by
different institutions, and that will compare
something across large groups that is by
definition only locally applicable—i.e., how well a
library meets the needs of its institution.
Librarians have either made do with
oversimplified national data or have undertaken
customized local evaluations of effectiveness,
but there has not been devised an effective way
to link the two.”
Sarah Pritchard, Library Trends, 1996
old.libqual.org
Multiple Methods
of Listening to Customers
•
•
•
•
•
•
•
•
•
•
•
Transactional surveys*
Mystery shopping
New, declining, and lost-customer surveys
Focus group interviews
Customer advisory panels
Service reviews
Customer complaint, comment, and inquiry capture
Total market surveys*
Employee field reporting
Employee surveys
Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
old.libqual.org
Premises
Three Seminal Quotations
old.libqual.org
LibQUAL+™ Premise #1
PERCEPTIONS
SERVICE
“….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999).
Delivering quality service. NY: The Free Press.
old.libqual.org
LibQUAL+™ Premise #2
“Il est plus nécessaire d'étudier
les hommes que les livres”
—FRANÇOIS DE LA ROCHEFOUCAULD
old.libqual.org
LibQUAL+™ Premise #3
“We only care about the things we
measure.”
--Bruce Thompson, CASLIN, 2006
old.libqual.org
13 Libraries
English LibQUAL+™ Version
4000 Respondents
Emergent
2000
QUAL
PURPOSE
Describe library
environment;
build theory of library
service quality from
user perspective
LibQUAL+® Project
DATA
Unstructured interviews
at 8 ARL institutions
ANALYSIS
Content analysis:
(cards & Atlas TI)
PRODUCT/RESULT
Case studies1
Valid LibQUAL+™ protocol
LibQUAL+™
QUAN Test
instrument
Web-delivered survey
Reliability/validity
analyses: Cronbachs
Alpha, factor analysis,
SEM, descriptive statistics
Scalable process
Enhanced understanding of
user-centered views of service
quality in the library
environment2
QUAL
Refine theory
of service quality
Unstructured interviews at
Health Sciences and the
Content analysis
Smithsonian libraries
Cultural perspective3
QUAL
Refine LibQUAL+™
instrument
E-mail to survey
administrators
Content analysis
Refined survey delivery
process and theory of service
quality4
QUAN
Test LibQUAL+™
instrument
Web-delivered survey
Reliability/validity analyses
including Cronbachs Alpha,
factor analysis, SEM,
descriptive statistics
Refined LibQUAL+™
instrument5
Focus groups
Content analysis
QUAL Refine theory
Iterative
Vignette
Re-tooling
2004
315 Libraries English, Dutch, Swedish,
German LibQUAL+™ Versions
160,000 anticipated respondents
old.libqual.org
Local contextual
understanding of
LibQUAL+™ survey
responses6
Dimensions
2000
2001
2002
2003-2007
41 items
56 items
25 items
22 items
Affect of Service
Affect of Service
Service Affect
Service Affect
Library as Place
Library as Place
Library as Place
Library as Place
Reliability
Reliability
Personal Control
Information
Control
Provision of
Physical
Collections
Self-Reliance
Information
Access
Access to
Information
Access to
Information
old.libqual.org
Survey Structure
(Detail View)
old.libqual.org
Interpreting Service Quality Data
Three Interpretation
Frameworks
old.libqual.org
Interpretation Framework #1
Benchmarking Against Peer Institutions
--1,000,000 Users; 1,000 Institutions!
NORMS! NORMS! NORMS!
old.libqual.org
Score Norms
• Norm Conversion Tables facilitate the
interpretation of observed scores using norms
created for a large and representative sample.
• LibQUAL+™ norms have been created at both
the individual and institutional level
old.libqual.org
Interpretation Framework #2
Benchmarking Against Self,
Longitudinally
“Nobody is more like me than me!”
--Anonymous
old.libqual.org
Interpretation Framework #3
Interpreting Perceived Scores Against
Minimally-Acceptable and Desired
Service Levels (i.e., “Zones of
Tolerance”)
old.libqual.org
old.libqual.org
old.libqual.org
“22 Items and The Box….”
Why the Box is so Important
– About 40% of participants provide openended comments, and these are linked to
demographics and quantitative data.
– Users elaborate the details of their concerns.
– Users feel the need to be constructive in their
criticisms, and offer specific suggestions for
action.
old.libqual.org
“…and Five Ancillary Items”
Either Zero or Five Ancillary items are
selected to address local or consortial
concerns
– Items from the initial LibQUAL+TM item
pool.
– Items written by previous consortial
groups.
old.libqual.org
Qualitative Grounding
old.libqual.org
Premise for Mixed-Methods
• “The underlying premise of mixed-method
inquiry is that each paradigm offers a
meaningful and legitimate way of knowing
and understanding” (p. 7).
Note. Greene, J.C. and Caracelli, V. J. (Eds.). (1997). Advances in
mixed-method valuation. San Francisco, CA: Jossey-Bass.
old.libqual.org
LibQUAL+ Process
 SERVQUAL dimensions served as a priori
theoretical starting point
old.libqual.org
76 Interviews Conducted
•
•
•
•
York University
University of Arizona
Arizona State
University of
Connecticut
• University of Houston
• University of Kansas
old.libqual.org





University of Minnesota
University of Pennsylvania
University of Washington
Smithsonian
Northwestern Medical
old.libqual.orgOnly\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred
LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT
old.libqual.org
Dimensions of
Library Service Quality
Affect of Service
Library
Service
Quality
Reliability
Empathy
Responsiveness
Assurance
Ubiquity and Ease
of Access
Comprehensive
Collections
Self-reliance
?
Library as Place
Formats
Utilitarian space
Timely access to
resources
Symbol
Physical location
Refuge
Model 1
old.libqual.org
Reliability
“You put a search on a book and it’s just gone; it’s
not reacquired. … There’s more of a problem of
lost books, of books that are gone and nobody
knows why and nobody’s doing anything about
it.”
Faculty member
old.libqual.org
Affect of Service
“I want to be treated with respect. I want you to be
courteous, to look like you know what you are
doing and enjoy what you are doing. … Don’t
get into personal conversations when I am at the
desk.”
Faculty member
old.libqual.org
Ubiquity of Access
“Over time my own library use has become
increasingly electronic. So that the amount of
time I actually spend in the library is getting
smaller and the amount of time I spend at my
desk on the web … is increasing.”
Faculty member
old.libqual.org
Comprehensive Collections
“I think one of the things I love about academic life
in the United States is that as a culture…, we
tend to appreciate the extraordinary importance
of libraries in the life of the mind.”
Faculty member
old.libqual.org
Library as Place
“One of the cherished rituals is going up the steps
and through the gorgeous doors of the library
and heading up to the fifth floor to my study. … I
have my books and I have six million volumes
downstairs that are readily available to me in an
open stack library.”
Faculty member
old.libqual.org
Library as Place
“I guess you’d call them satisfiers. As long as they
are not negatives, they won’t be much of a
factor. If they are negatives, they are a big
factor.”
Faculty member
old.libqual.org
Library as Place
“The poorer your situation, the more you need the
public spaces to work in. When I was an
undergraduate, I spent most of my time in the
library, just using it as a study space.”
Faculty member
old.libqual.org
Self-reliance
“…first of all, I would turn to the best search
engines that are out there. That’s not a person
so much as an entity. In this sense, librarians
are search engines [ just ] with a different
interface.”
Faculty member
old.libqual.org
Self-reliance
“By habit, I usually try to be self-sufficient. And I’ve
found that I am actually fairly proficient. I usually
find what I’m looking for eventually. So I
personally tend to ask a librarian only as a last
resort.”
Graduate student
old.libqual.org
Dimensions of
Library Service Quality
Library
Service
Quality
Affect of Service
Personal Control
Empathy
Ease of Navigation
Responsiveness
Convenience
Assurance
Modern Equipment
Reliability
Access to Information
Library as Place
Scope
Utilitarian space
Timeliness
Symbol
Convenience
Refuge
Model 2
old.libqual.org
Dimensions of Library
Service Quality
Library
Service
Quality
Information
Control
Affect of Service
Empathy
Scope of Content
Responsiveness
Convenience
Assurance
Reliability
Ease of Navigation
Library as Place
Utilitarian space
Symbol
Refuge
Modelold.libqual.org
3
Timeliness
Equipment
Self-Reliance
Core Items and Dimensions
22 core items (i.e., questions)
Three dimensions:
• Affect of Service – 9 questions
• Information Control – 8 questions
• Library as Place – 5 questions
old.libqual.org
Understanding a Radar Chart
old.libqual.org
Key to Radar Charts
old.libqual.org
Radar Chart Basics
old.libqual.org
Understanding Thermometer Charts
old.libqual.org
Dimension Summary
old.libqual.org
Library Use Question (Unified)
old.libqual.org
2006 LibQUAL+® Highlights:
Overall
old.libqual.org
Qualitative Analysis:
User Comments
• About one-half of users include comments on their
surveys
• User Comments available on the LibQUAL+ ® Web
site
– Download comments in Excel or text file
• Skim the comments
• Conduct Atlas.ti analysis
old.libqual.org
General Discussion
and Q&A
old.libqual.org
Summary and Closure
old.libqual.org
LibQUAL+® Resources
• LibQUAL+® Web site:
http://old.libqual.org
• Publications:
http://old.libqual.org/publications
• Events and Training:
http://old.libqual.org/events
• Gap Theory/Radar Graph Introduction:
http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:
http://old.libqual.org/Publications/index.cfm
old.libqual.org
Contact LibQUAL+®
• Martha Kyrillidou
Director, Statistics and Service Quality Programs
[email protected]
• MaShana Davis
Technical Communications Liaison
[email protected]
• Kristina Justh
Customer Relations Coordinator
[email protected]
• Gary Roebuck
Technical Operations Manager
[email protected]
old.libqual.org
Selena Lock, Research
and Development,
Cranfield University