Transcript Implementing LibQUAL+ Massachusetts LSTA Orientation MCCLPHEI Consortium ®

Implementing LibQUAL+®
Massachusetts LSTA Orientation
MCCLPHEI Consortium
Martha Kyrillidou
Association of Research Libraries
Boston, MA
November 1, 2006
old.libqual.org
Overview
• Research Foundation and Current Uses of
LibQUAL+®
• Preparing to Implement the Survey
• The Survey Process: Initial Steps
• Mapping Disciplines
• Approving and Running Your Survey
• Understanding Your Results: An Overview
• General Discussion and Q&A
old.libqual.org
Total Circulation
Total Circulation
600,000
550,000
500,000
450,000
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
400,000
old.libqual.org
Reference Transactions
Reference Transactions
170,000
160,000
150,000
140,000
130,000
120,000
110,000
100,000
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
20
02
90,000
old.libqual.org
Assessment
“The difficulty lies in trying to find a single model or
set of simple indicators that can be used by
different institutions, and that will compare
something across large groups that is by
definition only locally applicable—i.e., how well a
library meets the needs of its institution.
Librarians have either made do with
oversimplified national data or have undertaken
customized local evaluations of effectiveness,
but there has not been devised an effective way
to link the two.”
Sarah Pritchard, Library Trends, 1996
old.libqual.org
Multiple Methods
of Listening to Customers
•
•
•
•
•
•
•
•
•
•
•
Transactional surveys*
Mystery shopping
New, declining, and lost-customer surveys
Focus group interviews
Customer advisory panels
Service reviews
Customer complaint, comment, and inquiry capture
Total market surveys*
Employee field reporting
Employee surveys
Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
old.libqual.org
The LibQUAL+® Premise
PERCEPTIONS
SERVICE
“….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999).
Delivering quality service. NY: The Free Press.
old.libqual.org
13 Libraries
English LibQUAL+™ Version
4000 Respondents
Emergent
2000
QUAL
PURPOSE
Describe library
environment;
build theory of library
service quality from
user perspective
LibQUAL+® Project
DATA
ANALYSIS
Unstructured interviews
at 8 ARL institutions
Content analysis:
(cards & Atlas TI)
PRODUCT/RESULT
Case studies1
Valid LibQUAL+™ protocol
LibQUAL+™
QUAN Test
instrument
Web-delivered survey
Reliability/validity
analyses: Cronbachs
Alpha, factor analysis,
SEM, descriptive statistics
Scalable process
Enhanced understanding of
user-centered views of service
quality in the library
environment2
QUAL
Refine theory
of service quality
Unstructured interviews at
Health Sciences and the
Content analysis
Smithsonian libraries
Cultural perspective3
QUAL
Refine LibQUAL+™
instrument
E-mail to survey
administrators
Content analysis
Refined survey delivery
process and theory of service
quality4
QUAN
Test LibQUAL+™
instrument
Web-delivered survey
Reliability/validity analyses
including Cronbachs Alpha,
factor analysis, SEM,
descriptive statistics
Refined LibQUAL+™
instrument5
Focus groups
Content analysis
QUAL Refine theory
Iterative
Vignette
Re-tooling
2004
315 Libraries English, Dutch, Swedish,
German LibQUAL+™ Versions
160,000 anticipated respondents
old.libqual.org
Local contextual
understanding of
LibQUAL+™ survey
responses6
Dimensions
2000
2001
2002
2003-2006
41 items
56 items
25 items
22 items
Affect of Service
Affect of Service
Service Affect
Service Affect
Library as Place
Library as Place
Library as Place
Library as Place
Reliability
Reliability
Personal Control
Information
Control
Provision of
Physical
Collections
Self-Reliance
Information
Access
Access to
Information
Access to
Information
old.libqual.org
Survey Structure – Page 2
(Detail View)
old.libqual.org
Rapid Growth in Other Areas
• Languages
–
–
–
–
–
–
–
–
• Types of Institutions
–
–
–
–
–
–
–
–
–
American English
British English
French
Dutch
Swedish
Norwegian
Finnish
Danish
• Consortia
– Each may create 5 local
questions to add to their
survey
Academic Health Sciences
Academic Law
Academic Military
College or University
Community College
European Business
Hospital
Public
State
• Countries
– Australia, Canada, Denmark,
Egypt, Finland, France, New
Zealand, the Netherlands,
Norway, South Africa,
Sweden, Switzerland, UAE,
U.K., U.S.
old.libqual.org
®
LibQUAL+ Participants
350
316
300
250
200
206
150
210
164
100
50
43
0
12
2000
2001
2002
2003
old.libqual.org
2004
2005
LibQUAL+® Surveys by Type
Years
2000
2003
2004
Academic law
1
25
Academic military
6
College or university
13
2001
41
Community college
2002
111
244
150
191
16
29
3
14
5
Family history library
1
1
35
23
Hospital
New York Public library
10
1
European business
Health sciences library
2005
13
12
10
1
1
1
1
Public
4
Smithsonian
1
State
1
1
1
University/TAFE
2
Electronic
1
Natural Resources
2
FFRDC
5
old.libqual.org
LibQUAL+® Languages
American English
French Canadian
Swedish
British English
Afrikaans
Dutch English
Dutch
Continental French
Swedish
(British English)
German
Norwegian
Finnish
Danish
old.libqual.org
LibQUAL+® Surveys by Language
Years
American English
2000
2001
2002
2003
2004
2005
13
42
164
285
176
207
20
22
31
British English
Continental French
1
Dutch
1
Dutch English
1
1
French Canadian
2
1
4
Swedish
5
2
Swedish English (A.E.)
2
1
Afrikaans
4
old.libqual.org
LibQUAL+® Surveys by Consortia
2000
2001
2002
2003
2004
2005
1
35
21
14
12
AAHSL
AJCU
20
Alabama Academic (NAAL)
10
CES
1
8
6
City University of New York
19
CUC
8
EBSLG
6
FFRDC
5
Hospital/MLA
7
MERLN
6
NELLCO
8
NY3Rs
76
Oberlin
12
9
13
45
1
14
20
17
16
State Universities of Florida
6
2
University of Wisconsin System
14
OhioLINK
57
SCONUL
VALE
2
12
old.libqual.org
Participating Libraries by Country
Country
2000
2001
2002
2003
Australia
Canada
1
3
4
8
2004
2005
1
6
10
15
Denmark
1
Egypt
1
France
1
Ireland
1
Netherlands
1
1
Sweden
3
Switzerland
2
U.A.E.
1
UK
20
South Africa
1
17
4
16
12
old.libqual.org
Surveys by Session 2004-2005
Year
Session 1
Session 2
2004
202
2
2005
199
56
old.libqual.org
World LibQUAL+® Survey
old.libqual.org
Preparing for the Survey Implementation
old.libqual.org
Why is Your Library Participating
in LibQUAL+® ?
• Institutional goals:
– what do you want to get out of the survey?
• Survey requirements:
– people, supplies, technology
old.libqual.org
Get Permission from your
Institutional Review Board
• Local group that approves human subject
research
• May also be called
– Committee for Human Subject Research
– Committee for the Protection of Human
Subjects
• Not all institutions need to obtain permission
old.libqual.org
Get Permission from your
Institutional Review Board
• If permission is required:
– Seek well in advance of survey
– Supply a copy of the survey, if requested
– Inform them that results will be shared
among participants
• No need to inform LibQUAL+® of decision
old.libqual.org
Determine Whom to Survey
• Random sample
or
• Entire population
old.libqual.org
If You Sample…
• Recommendations:
– At least 1,200 random email addresses for
each user group
– Separate sample groups for
undergraduates, graduates, faculty, and
staff
– Think about survey fatigue if you want to
repeat the survey regularly
• Keep notes on your methodology as you will
be asked to describe it in the Post Hoc
Questionnaire
old.libqual.org
If You Survey the
Entire Population….
• Recommendations:
– Think about survey fatigue if you want to
repeat the survey regularly
• Keep notes on your methodology as you will
be asked to describe it in the Post Hoc
Questionnaire
old.libqual.org
Obtain Email Addresses
• Typical sources include:
– Campus computing office
– Campus administrative records office
– Library patron database
old.libqual.org
Promoting your Survey
– Place ads in campus newspaper
– Write article for library newsletter
– Post flyers around campus
– Present at faculty meetings & student
orientations
– Create a survey Web site and feature on
library’s home page
– Take the survey to where users are: dining
halls, study rooms, dorms
old.libqual.org
Become Familiar with
LibQUAL+® Resources
• Management Center
http://www.libqual.org/Manage/index.cfm
• LibQUAL+® Procedures Manual
http://www.libqual.org/Manual/index.cfm
– Updated Version – September 2006
– Much more detail
• Discussion list
[email protected]
old.libqual.org
The LibQUAL+®
Management Center
old.libqual.org
The Survey Process: Initial Steps
old.libqual.org
Mapping Disciplines
old.libqual.org
Mapping Disciplines
• Results notebooks summarize findings by
user group and provide a chart for, both,
standard and custom disciplines
• Standard disciplines
• Customized disciplines
– Recommend no more than 10 disciplines, if
possible
old.libqual.org
The Standard Disciplines
• Agriculture/Environmental
Studies
• Architecture
• Business
• Communications/
Journalism
• Education
• Engineering/Computer
Science
• General Studies
• Health Sciences
•
•
•
•
•
•
•
Humanities
Law
Military/Naval Science
Other
Performing & Fine Arts
Science/Math
Social Sciences/
Psychology
• Undecided
old.libqual.org
Customized Disciplines
• Use your local terminology to map to the
standard disciplines
• Cautions:
– Need to provide representativeness data
for each discipline
– Too many choices present challenges to
users
old.libqual.org
Standard Disciplines:
A PA Academic Library
old.libqual.org
Customized Disciplines:
A PA Academic Library
old.libqual.org
Getting Practice with Mapping
Let’s practice mapping your disciplines
old.libqual.org
Approving and Running Your Survey
old.libqual.org
Understanding Your Results
old.libqual.org
Results Notebooks
• Sections for Overall, Undergraduates, Graduates,
Faculty, Staff, Library Staff include:
– Demographic Summary
– Core Questions Summary
– Dimensions Summary
– Local Questions
– General Satisfaction Questions
– Information Literacy Outcomes Questions
– Library Use Summary
• Appendix describing changes in the dimensions and
the questions included in each dimension.
old.libqual.org
Results
• Results Notebooks (PDF)
http://old.libqual.org/Manage/Results/index.cfm
– Group Notebook (PDF)
– Unified Notebook (PDF)
– User Comments
– Excel Data File
SPSS data files (if requested at registration)
will be e-mailed to the primary contact two to
three months after the session has closed.
old.libqual.org
Core Items and Dimensions
22 core items (i.e., questions)
Three dimensions:
• Affect of Service – 9 questions
• Information Control – 8 questions
• Library as Place – 5 questions
old.libqual.org
Understanding a Radar Chart
old.libqual.org
Key to Radar Charts
old.libqual.org
What Do the Colors Mean?
Green
And
Blue
old.libqual.org
What Do the Colors Mean?
Red
old.libqual.org
What Do the Colors Mean?
Little Yellow
old.libqual.org
What Do the Colors Mean?
A lot of Yellow
old.libqual.org
Dimension Summary
old.libqual.org
Library Use Question (Unified)
old.libqual.org
2006 LibQUAL+® Highlights:
Overall
old.libqual.org
Qualitative Analysis:
User Comments
• About one-half of users include comments on their
surveys
• User Comments available on the LibQUAL+® Web
site
– Download comments in Excel or text file
• Skim the comments
• Conduct ATLAS.ti analysis
old.libqual.org
General Discussion and Q&A
old.libqual.org
Summary and Closure
old.libqual.org
LibQUAL+® Resources
• LibQUAL+® Web site:
http://old.libqual.org
• Publications:
http://old.libqual.org/publications
• Events and Training:
http://old.libqual.org/events
• Gap Theory/Radargraph Introduction:
http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:
http://old.libqual.org/Publications/index.cfm
old.libqual.org
The LibQUAL+® Team
• MaShana Davis
Technical Communications Liaison
[email protected]
• Richard Groves
Customer Relations Coordinator
[email protected]
• Martha Kyrillidou
Director, Statistics and Service Quality Programs
[email protected]
old.libqual.org