Transcript An Introduction to LibQUAL+

Using LibQUAL+®
Selena Killick
Association of Research Libraries
LibQUAL+® Workshop
l’Université de Paris V
28th January 2008
old.libqual.org
Introduction
• Background to LibQUAL+®
– The LibQUAL+® Questionnaire
– Results from SCONUL
– Feedback from UK Participants
• LibQUAL+® Process Overview
• The votes are in! What next?
– Reading the Charts
– Making the most out of your data
old.libqual.org
Introductions
• Institutional Contact for Cranfield
University since 2003
• Administrated the SCONUL Consortium
since 2003
– Consortium co-ordinated by Stephen Town
• European LibQUAL+® Contact since 2006
– On behalf of ARL
old.libqual.org
The need for LibQUAL+®
• Underlying need to demonstrate our worth
• The reallocation of resources from
traditional services and functions
• Rapid shifts in information-seeking
behavior
• Increasing user demands
old.libqual.org
The LibQUAL+® Premise
PERCEPTIONS
SERVICE
“….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999).
Delivering quality service. NY: The Free Press.
old.libqual.org
LibQUAL+® Development
• An ARL/Texas A&M University joint
developmental effort based on SERVQUAL.
• LibQUAL+® initially supported by a 3-year grant
from the U.S. Department of Education’s Fund
for the Improvement of Post-Secondary
Education (FIPSE)
• Initial project established an expert team, regrounded SERVQUAL concepts, and designed
survey methodology
• Survey conducted at over 700 libraries resulting
in a data base of over half a million user
responses
old.libqual.org
LibQUAL+® Participants
350
250,000
307
308
300
205,639
250
286
200,000
250
176,360
204 151,460
150,000
200
128,958
Number of
Institutions
Number of
Responses
164
150
113,480
100,000
78,863
100
Number of Institutions
50
43
13
0
50,000
Number of Responses
20,416
4,407
2000
0
2001
2002
2003
2004
2005
Survey Year
old.libqual.org
2006
2007
Growth in other areas
• Languages
–
–
–
–
–
–
–
–
–
–
–
–
–
–
• Types of Institutions
Afrikaans
American English
British English
Chinese
Danish
Dutch
Finnish
French
German
Japanese
Norwegian
Spanish
Swedish
Welsh
• Consortia
*Each may create 5 local questions to add
to their survey
• Countries
–
Australia, Canada, Denmark, Egypt,
Finland, France, Hong Kong, Ireland,
New Zealand, the Netherlands,
Norway, South Africa, Sweden,
Switzerland, UAE, U.K., U.S.
old.libqual.org
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Academic Health Sciences
Academic Law
Academic Military
College or University
Community College
Electronic
European Business
European Parliament
Family History
Research Centers (FFRDC) Libraries
High School
Hospital
National Health Service England
Natural Resources
New York Public
Public
Smithsonian
State
University/TAFE
LibQUAL+® Languages
American English
French
Canadian
Continental
French
Swedish
Japanese
Swedish
(British
English)
British
English
German
Spanish
Welsh
old.libqual.org
Afrikaans
Norwegian
Dutch
English
Dutch
Finnish
Danish
LibQUAL+® Surveys by Language
Year
Language
2000
2001
2002
2003
2004
Afrikaans
American English
13
42
164
British English
2005
2006
4
1
2007
285
176
207
236
217
20
22
31
50
38
Chinese
4
Danish
1
Dutch
Dutch English
1
1
2
1
2
Finnish
1
French Canadian
2
French Continental
1
4
1
26
1
German
1
Norwegian
1
Swedish
5
2
Swedish British English
1
1
Swedish English (A.E.)
2
old.libqual.org
1
1
Participating Libraries by Country
Year
Country
2000
2001
2002
2003
Australia
Canada
1
3
4
8
2004
2005
2006
2007
1
6
2
3
10
15
11
75
Denmark
1
2
Egypt
1
2
Finland
1
2
France
1
2
Hong Kong
1
11
Ireland
1
1
2
Mexico
1
1
Netherlands
1
1
5
New Zealand
1
Norway
2
old.libqual.org
3
Participating Libraries by Country
(cont’d)
Year
Country
2000
2001
2002
2003
2004
South Africa
Sweden
3
Switzerland
2
U.A.E.
1
U.K.
20
old.libqual.org
17
2005
2006
2007
12
8
5
4
4
2
2
16
33
21
LibQUAL+® and SCONUL
• Coordinated on behalf of the Society of College,
National & University Libraries (SCONUL)
Working Group on Performance Improvement
(WGPI)
• 2003 20 UK Higher Education (HE) institutions
agree to pilot the survey in a consortium of
SCONUL Members
• Pilot seen as a success
• Consortium of SCONUL Libraries has
participated in LibQUAL+® annually since 2003
• 67 Different institutions in 6 years
old.libqual.org
The LibQUAL+® Questionnaire
old.libqual.org
Process Overview
• Register with ARL (2008 cost $3,000)
• Institutional contact sets survey to local needs
• Local Questions
• Disciplines
• Send out a URL to the survey via email
• Mounted on ARL servers
• Watch the surveys come in
• Close the survey when ready, institutional results
available after a couple of weeks
• PDF
• SPSS
• Excel
old.libqual.org
Time frame
•
•
•
•
December – Registration
February to May – Session I
July to December – Session II
January 2009 – Consortium results
available
old.libqual.org
Survey Composition
• 22 Core Questions
– Affect of Service
– Information Control
– Library as Place
•
•
•
•
•
•
5 Local Questions (optional)
5 Information Literacy Questions
3 General Satisfaction Questions
Library Usage Patterns
Demographics
Free Text Comments Box
old.libqual.org
Five Local Questions
• Participants can choose 5 questions to
add to their survey from a range of over
100
• Helping participants focus on local issues
• Maintaining standardisation for
benchmarking purposes
old.libqual.org
Free-Text Comments Box
• About 40% of participants provide open-ended
comments, and these are linked to
demographics and quantitative data
• Users elaborate the details of their concerns
• Users feel the need to be constructive in their
criticisms, and offer specific suggestions for
action
• Available in real-time enabling prompt responses
to concerns
old.libqual.org
Usage & Demographics
•
•
•
•
•
•
Library Usage
User group
Discipline
Age
Sex
Gender
• Attached to SPSS and Excel results
• Enabling detailed further analysis by type
old.libqual.org
Survey Instrument
old.libqual.org
Gap Theory
• For the 22 items LibQUAL+® asks users’ to rate
their:
• Minimum service level
• Desired service level
• Perceived service performance
• This gives us a ‘Zone of Tolerance’ for each
question; the distance between minimally
acceptable and desired service ratings
• Perception ratings ideally fall within the Zone of
Tolerance
old.libqual.org
Gap Theory
Desired
Minimum
Perceived is greater
than desired
Perceived
Perceived is greater
than minimum, less
than desired
Perceived is less
than minimum
Perceived
Perceived
Minimum
old.libqual.org
Desired
Results from SCONUL
old.libqual.org
Core Questions
old.libqual.org
SCONUL Core Question Summary 2007
old.libqual.org
ARL College & University Core Question
Summary 2007
old.libqual.org
SCONUL Results by Dimension
Information Control
Affect of Service
8.50
8.50
8.00
8.00
8.00
7.50
7.50
7.50
7.00
7.00
6.50
Minimum Mean
Desired Mean
6.50
7.00
Mean
Mean
Mean
8.50
Library as Place
Perceived Mean
Minimum Mean
Desired Mean
6.50
Perceived Mean
Affect of Service
6.00
6.00
6.00
5.50
5.50
5.50
8.50
8.00
5.00
2003 2004
5.00
2005 2006 2007
2003 2004
2005
5.00
2006 2007 2003
7.50
Minimum Mean
7.00
Desired Mean
old.libqual.org
Perceived Mean
2004
2005
2006
2007
SCONUL Results by User Group
Undergraduates Overall
Postgraduates Overall
8.50
8.50
8.50
8.50
8.00
8.00
8.00
8.00
7.50
7.50
7.50
7.50
Desired Mean
7.00
Perceived Mean
Desired Mean
7.00
Perceived Mean
Desired Mean
Perceived Mean
6.50
6.50
6.50
6.50
6.00
6.00
6.00
6.00
5.50
Library Staff Overall
Minimum Mean
Mean
7.00
Minimum Mean
Mean
7.00
Mean
Mean
Minimum Mean
Academic Staff Overall
5.50
5.50
5.50
old.libqual.org
2003 2004 2005 2006 2007 2003 2004 2005 2006 2007 2003 2004 2005 2006 2007 2003 2004 2005 2006 2007
General findings
• Highly desired
• Making electronic resources accessible from my
home or office
• Print and/or electronic journals I require for my work
• A haven for study, learning or research
• Lowest
• Library staff who instil confidence in users
• Giving users individual attention
• Space for group learning and group study
old.libqual.org
Feedback from UK Participants
old.libqual.org
Why use LibQUAL+®?
Feedback from LibQUAL+® Users
“Why did you choose to use LibQUAL+®?”
• LibQUAL+® was recommended to us as offering
a well designed, thoroughly Library-focused set
of survey tools
• Cost-effectiveness
• Automated processing & fast delivery of results
• Opportunity to benchmark
• Respectability and comparability (with others
and historically)
old.libqual.org
The benefits of LibQUAL+®
LibQUAL+® has enabled us to find out what
a broad range of our users thought of the
services we offer; what level of servicedelivery quality we had achieved in their
eyes, and to get a clear picture of what
they actually wanted the Library to deliver
(as opposed to what we thought they
wanted).
UK HE Institution, 2006
old.libqual.org
SCONUL Frequency
How often do SCONUL members run the
LibQUAL+® survey?
• Every year = 8 Institutions
• Every 2 years = 11 Institutions
• Every 3 years = 5 Institutions
old.libqual.org
LibQUAL+® vs. Local
• Advantages
• Advantages
– Administered for you
– Ability to focus on local
issues
– Customisable
• Limited local effort
–
–
–
–
Analysis conducted for you
Cost effective
Benchmarking capabilities
Respectability
• Disadvantages
• Disadvantages
– Limited ability to focus on
local issues
– User Group Demographics
not customisable
old.libqual.org
– Question writing
– Requires a lot of staff time
•
•
•
•
Design
Administering
Marketing
Collating and Analysing
In Summary, LibQUAL+®…
• Focuses on success from the users’ point of view
(outcomes)
• Demonstrates that a web-based survey can handle large
numbers; users are willing to fill it out; and survey can be
executed quickly with minimal expense
• Requires limited local survey expertise and resources
• Analysis available at local, national and inter-institutional
levels
• Offers opportunities for highlighting and improving your
status within the institution
old.libqual.org
LibQUAL+® Process Overview
old.libqual.org
Preparing for the Survey Implementation
• Why is Your Library Participating
in LibQUAL+® ?
– what do you want to get out of the survey?
• Get permission from your Institutional
Review Board (if necessary)
• Decide who to sample
– Random Sample
– Entire Population
old.libqual.org
Random Sample or Entire Population
• If you sample
– Recommend at least 1,200 random email addresses
for each user group
– Separate sample groups for undergraduates,
graduates, faculty, and staff
• If you survey the entire population
– Think about survey fatigue if you want to repeat the
survey regularly
• Keep notes on your methodology as you will be
asked to describe it in the Post Hoc
Questionnaire
old.libqual.org
Obtain Email Addresses
• Typical sources include:
– Campus computing office
– Campus administrative records office
– Library patron database
old.libqual.org
Marketing Your Survey
•
•
•
•
Place ads in campus newspaper
Write article for library newsletter
Post flyers around campus
Present at faculty meetings & student
orientations
• Create a survey Web site and feature on
library’s home page
• Take the survey to where users are: dining halls,
study rooms, dorms
• More ideas on
http://www.libqual.org/Publications/index.cfm
(Publication Type = Special)
old.libqual.org
Become Familiar with
LibQUAL+® Resources
• Management Center
– http://old.libqual.org/Manage/index.cfm
• LibQUAL+® Procedures Manual
– http://old.libqual.org/Manual/index.cfm
• Discussion list
– [email protected]
• SCONUL Performance Portal
– http://vamp.diglib.shrivenham.cranfield.ac.uk/quality/li
bqual
old.libqual.org
The Survey Process: Initial Steps
old.libqual.org
LibQUAL+® Management Center:
Getting Started
• Login (Required to use the Management Center)
• Center Sections:
–
–
–
–
–
–
–
Personal Profile
Institution Profile
Manage Users
Manage Your Survey
Survey Results
Resources
Directory
old.libqual.org
Survey Process: Manage Your Survey
Eight Sections:
• Preferences
• Customization
• Preview
• Representativeness
• Monitor Survey Progress
• Incentive Winners
• Post Hoc
• Evaluation
*Sections must be completed
in order
old.libqual.org
Preferences
Fields marked
with * are
required
old.libqual.org
Customization: Optional/Local Questions
old.libqual.org
Customization: Disciplines
• Results notebooks summarise findings by
user group and provide a chart for both
standard and custom disciplines
• Standard disciplines (based on your
institution type, i.e., College/University)
• Customised disciplines
– Recommend no more than 16 disciplines, if
possible
old.libqual.org
Customised Disciplines
• Tailor your disciplines to suit your
institution
• Map them back to the standard disciplines
to aid benchmarking
• Cautions:
– Need to provide representativeness data for
your customised disciplines
– Too many choices present challenges to
users, difficult to navigate on screen
old.libqual.org
Manage Your Survey:
Customization – Customized Disciplines
old.libqual.org
Standard Disciplines:
An Academic Library
old.libqual.org
Customized Disciplines:
An Academic Library
old.libqual.org
Customization: Branch Libraries
The library you
use most
often.
old.libqual.org
Approving and Running
Your Survey
old.libqual.org
Previewing and Approving
Your Survey
Preview
• Complete at least one
full run of your
preview survey
• Test in different
settings, using
different platforms
and Web browsers
• Get library staff
involved in testing
Approved
• Can no longer make
changes
• Live survey URL will
appear at the top of
the “Manage Your
Survey” page marked
as Survey URL
old.libqual.org
Distributing
• Once you have your URL you can send
out invitations via email to complete the
survey
• Can be done in own time
• Recommend you run Library staff training
before your invitations go out if possible
– Ideal way to capture Library Staff views
old.libqual.org
Sending Reminders
• 3-5 reminders to sample populations
• Thank respondents who have completed the
survey
• Boost up marketing efforts around campus
– More fliers, table tents, ads in campus newspaper
– Get professors and other staff involved
– Increase number or types of incentives
• Examples of marketing and downloadable logos
are available on the LibQUAL+® Web site.
old.libqual.org
Reviewing Comments
• About one-half of users include comments
on their survey
• Download via the Monitor Survey Progress
page (Excel file or text file)
• http://old.libqual.org/Manage/Results/Com
ments/index.cfm
old.libqual.org
Completing Your
Representativeness Questionnaire
• Determines how your institutional profile
compares to your survey data
• Requires the following information:
–
–
–
–
Number of individuals per user group
Number of individuals within each discipline
Number of males and females
Library Statistics
• Volumes added during the year – Gross/Total
• Total number of current serials received (including electronic
serials)
• Total library expenditures (in U.S. dollars)
• Personnel – professional staff, full-time equivalent (FTE)
• Personnel – support staff, full-time equivalent (FTE)
• Must be complete BEFORE closing survey
old.libqual.org
Completing Your
Representativeness Questionnaire
Representativeness
Completed
Representativeness
NOT Completed
old.libqual.org
Closing Your Survey
• We recommend a survey run of a
minimum of three weeks
• Once you close your survey:
– Retrieve list of incentive winners
– Fill out Post Hoc and Evaluation
Questionnaires
old.libqual.org
Completing Post Hoc and
Evaluation Questionnaires
• Post Hoc
Questionnaire
• Information about
your survey
– Sample size
– Number of e-mails
sent
– Number of invalid email addresses
– Incentives offered
– Marketing techniques
– Etc.
• Evaluation
Questionnaire
• Feedback about your
LibQUAL+®
experience
• All survey liaisons
and assistants are
encouraged to
complete this
questionnaire
old.libqual.org
LibQUAL+® Management Center:
Retrieving Survey Results
• Survey Results Notebook (PDF)
– Individual & Group Analyses
• Comments
– About half of users provide comments
– Download to a text or Excel file under Monitor Survey
Progress (see Manage Your Survey)
• Excel data files
• SPSS (delivered 2-3 months after survey)
• Additional Services:
–
–
–
–
Customized Discipline Analysis
Library Branch Analysis
Other Customized Analyses (upon request)
Print Copies
old.libqual.org
Summary
• Tasks for Institutional Contact:
– Survey dates
– Optional questions
– Discipline mapping
– Representativeness data
– Marketing & distribution
old.libqual.org
The votes are in! What next?
old.libqual.org
Results Notebooks
• Sections for Overall, Undergraduates, Graduates,
Faculty, Staff, Library Staff include:
– Demographic Summary
– Core Questions Summary
– Dimensions Summary
– Local Questions
– General Satisfaction Questions
– Information Literacy Outcomes Questions
– Library Use Summary
• Appendix describing changes in the dimensions and
the questions included in each dimension.
old.libqual.org
Results
• Results Notebooks (PDF)
http://old.libqual.org/Manage/Results/index.cfm
– Group Notebook (PDF)
– Unified Notebook (PDF)
– User Comments
– Excel Data File
SPSS data files (if requested at registration)
will be e-mailed to the primary contact two to
three months after the session has closed.
old.libqual.org
Reading the Charts
old.libqual.org
Key to Radar Charts
Key Term:
Zone of Tolerance
old.libqual.org
Radar Chart Basics
Desired
Minimum
Perceived is greater
than desired
Perceived
Perceived is greater
than minimum, less
than desired
Perceived is less
than minimum
Perceived
Perceived
Minimum
old.libqual.org
Desired
What Do the Colors Mean?
Key Term:
Zone of Tolerance
Green
And
Blue
old.libqual.org
What Do the Colors Mean?
Red
Key Term:
Zone of Tolerance
old.libqual.org
What Do the Colors Mean?
Little Yellow
Key Term:
Zone of Tolerance
old.libqual.org
What Do the Colors Mean?
A lot of Yellow
Key Term:
Zone of Tolerance
old.libqual.org
Understanding Your Individual
Results: Thermometer Charts
Key Term:
Zone of Tolerance
old.libqual.org
Dimension Summary
Key Term:
Zone of Tolerance
old.libqual.org
Making the most out of your data
old.libqual.org
Making the Most of the Data
• Further data analysis is necessary to
maximise the potential of your results
• Initial results will provide an overview of
your service
• Detailed analysis allows further issues to
be highlighted
old.libqual.org
Tools to Help You
old.libqual.org
Quantitative Analysis: Excel Data
• Excel data files available on the LibQUAL+® Web site
http://old.libqual.org/Manage/Results/index.cfm
• Use customized radar chart template to create custom
analyses
http://old.libqual.org/Manage/Resources/SampleRadarC
hart/index.cfm
old.libqual.org
Analytics Tool
old.libqual.org
LibQUAL+® Resources
• LibQUAL+® Website:
http://old.libqual.org
• Publications:
http://old.libqual.org/publications
• Events and Training:
http://old.libqual.org/events
• Gap Theory/Radar Graph Introduction:
http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Bibliography:
http://www.coe.tamu.edu/~bthompson/servqbib
• LibQUAL+® Procedures Manual:
http://old.libqual.org/Information/Manual/index.cfm
old.libqual.org
Acknowledgements
• Fred Heath, Bruce Thompson, Colleen Cook,
Martha Kyrillidou and the rest of the LibQUAL+®
team
• Stephen Town and rest of the SCONUL Working
Group on Performance Improvement
• All SCONUL LibQUAL+® Participants
old.libqual.org
Selena Killick
Research & Development Officer
Cranfield University
[email protected]
Tel: +44 (0)1793 785561
old.libqual.org