Transcript Slide 1

LibQUAL+™
How to Listen to your Customers
Using LibQUAL+™
GUGM Reference Section
May 19, 2005
Presentation by
Brian Mathews, Georgia Tech
Caroline Killens, UGA
Definition of Assessment
 In an academic environment assessment can be defined as “an ongoing
process aimed at understanding and improving student learning. It
involves:
 making our expectations explicit and public;
 setting appropriate criteria and high standards for learning quality;
 systematically gathering, analyzing, and interpreting evidence to determine how
well performance matches those expectations and standards; and
 using the resulting information to document, explain, and improve performance.
When it is embedded effectively within larger institutional systems, assessment can help
us focus our collective attention, examine our assumptions, and create a shared
academic culture dedicated to assuring and improving the quality of higher education
(Thomas A. Angelo, AAHE Bulletin, November 1995, p.7).
The Importance of Assessment
 To evaluate your service.
 To understand what patrons “really” want.
 To guide the decision-making process.
 To support administrative initiatives.
What is LibQUAL?
 A survey sponsored by ARL, as part of the New Measures initiative, to
augment the collection count and fiscal input measures of ARL
that comprise the ARL Index and ARL Statistics. The survey is comprised of “22
questions + a box”

Is an
assessment tool that lets customers judge quality
 Produces outcome measures on service quality and satisfaction of users.
Is a way of “listening” to users
 Provides institutional data and reports that enables you to assess
whether your library services are meeting user expectations
 Provides libraries with comparable assessment information from
peer institutions
Why Use LibQUAL+™?
 Great value with minimal effort.
 To gain an objective understanding of your patron’s
unique needs and perceptions.
 Provides comparative data with other libraries.
 Can be utilized with campus library reviews, SACS
reviews, accreditation reports, etc.
How To Prepare for LibQUAL+™
 Complete LibQUAL+™ questionnaire on-line to activate your account.
Determine a good source for your campus statistics!
 Determine the best dates to conduct your survey.
 Determine if you need permission from your campus IR Board.
 Decide who to include in the survey and how it will be conducted.
Everyone on campus, random selection, etc.;
If using a random sample, how will you generate your sample? Circulation files, Human
Resources files, Registrar’ files, etc.
E-mail, website, mail paper copy
 Determine how much advertisement you want to conduct prior to the survey
Campus newspapers, website, posters, alert departments, etc.
 Decide if you will use incentives.
iPods, no fines, money, sponsored gifts, etc.
 Prepare letter to be sent to participants. (sample provided by LibQUAL+™)
What Happens While the Survey
Takes Place
 You can track the number of surveys completed for your institution
online in real time.
 You can begin reviewing comments immediately.
 Send out reminders if it looks like things are slowing down.
 Delete undeliverable addresses from your files so your reminders
will go only to active addresses.
 Have someone dedicated to answer questions by phone or e-mail –
 RELAX! and let the survey work for you.
What Do You Get From
LibQUAL+™?
 LibQUAL+™ report within a month of your survey’s
closing
 SPSS and Excel files with all of the raw data
 Interactive files at the LibQUAL website
 List of the comments made on your survey
 Access to reports from all other participating libraries
How The Survey Works
Participants are asked to score each question in 3 sub-categories,
based on a scale of 1-9:
The sub-categories for each question:
 Minimum = “what they need” minimum acceptable level of service
required by
users
 Perceived = “what they have” - the level of service users
think/believe the library is providing (perceived)
 Desired = “what they want” - the level of service users want to have
(desired)
When it comes to…..
My minimum service level My perceived service level My desired service
low
1)Employees who instill confidence users
high
1 2 3 4 5 6 7 8 9
low
high
1 2 3 4 5 6 7 8 9
low
high
1 2 3 4 5 6 7 8 9
Zone of Tolerance
GAPS
The scores from the 3 sub-categories can be utilized to show the
following gaps:
1. ADEQUACY GAP : how well the library is meeting the users’
“minimum” requirements/needs.
Formula: Perceived score minus the minimum score
2. SUPERIORITY GAP : how far the library is from meeting or
exceeding the users’ desired level of service
Formula: Perceived score minus the desired score
One way of viewing GAPS
(shows minimum perceived & desired scores on 1 graph)
Another view of Gaps
UGA Adequacy Gaps
by Question
Overall Adequacy Gap in Ascending Order
1.2
1.07
0.99
1
0.95
Mean Score
0.8
0.66
0.6
0.4
0.38
0.45
0.45
IC-2
IC-6
0.48
0.5
0.53
0.55
0.68
0.69
0.71
0.73
0.75
0.78
0.79
0.81
0.58
0.58
IC-7
LP-1 LP-3 LP-2 LP-4 AS-5 AS-7 AS-3 AS-4 AS-8 AS-6 AS-2 AS-1 LP-5
0.41
0.2
0
IC-8
IC-3
IC-5
IC-1 AS-9 IC-4
Question IDs
Comparing gaps over time
GT Graduates Adequacy 2003 & 2004
RADAR GRAPHS
Radar graphs are a good way to show results of all
questions in one chart. This is a good visual
way to compare yourself overall with other
libraries.
RED – Scores are below your users minimum expectations
BLUE – shows the gap between your minimum and perceived
Yellow – shows the gap between your perceived and desired
GREEN – shows scores above the desired expectations
LibQUAL+™ Comparisons:
ARL Colleges & Universities and UGA
Core Questions Summary
ARL
UGA
Benchmarking
 An external focus on internal activities, functions,
or operations in order to achieve continuous
improvement.
 LibQUAL+ enables you to compare your library
with peer institutions.
 LibQUAL+ enables you to identify areas of
excellence and areas for potential improvement.
GT Benchmarking Examples
Top Engineering Schools
Undergraduates, Perceptions
Printed Materials
Top Engineering Schools
Undergraduates, Adequacy
Knowledge to Answer User Questions
COMMENTS
The survey offers a “comments box” soliciting openended user views that provide a more detailed picture
of how users view our resources and services
466 UGA participants utilized this box to give 799
specific comments.
231 GT participants utilized this box to give 400
specific comments.
Comments on Online Tools :
Summary
 Many comments expressed confusion or dissatisfaction with doing
research online, but were not specific enough to apply to one tool such
as GIL, GALILEO, EJL, etc.
 Areas of difficulty/frustration expressed most frequently by survey
participants:
-the difficulty of finding journals online
-too many disparate search engines available
-online tools not user-friendly
-searches that result in few or no hits.
 Dissatisfaction with online tools seems to decrease with experience in
searching and/or doing research, though all user groups had a high
proportion of constructive comments.
Comments on Online Tools:

“Most
frustrating problem for GALILEO, etc., is the amount of drilling down
to reach needed databases.” – Graduate Student, Education
 “…sometimes GALILEO can be very confusing, particularly with all of the
different search engines … is there any way to put it all into ONE search?” –
Undergraduate Student, Business
 “The only major complaint I have is that searches can be very in depth and
complicated without assistance from a library employee at least until you
have a lot of practice.” – Undergraduate Student, Journalism
 “GIL is a huge problem for me. Half the time books don’t show up on
general or advanced searches even though there are probably over 40
books available.” – Undergraduate Student, Social Science
 “The online catalog search systems seem as though they were written by
undergraduate computer science classes. You need a professionally written
search system that is designed the way scientists and engineers use
information. What good is a library if one can't find the information?” –
Faculty, Science
Memorable Comments: 
 “The Student Learning Center is fantabulous. It’s better than a cell phone –
how did I live without it?!?!?! – Undergraduate, Physical Sciences
 “[Suggestion:] A good cafeteria – empty stomach is also a devil’s workshop.”
– Graduate Student, Ag. & Env. Sciences
 “Please have a drive-up renewal/fine window. That would be fabulous!” Graduate Student, Humanities
 “Allow us to check out laptops like books.” – Undergraduate, Engineering
 “I think the employees of the library are doing a great job. They have always
been very courteous and always able to answer my questions.”
Undergraduate, Engineering
 “I feel that the employees could be more courteous and willing to help.”
Undergraduate, Engineering
What did GT and UGA hear from their
users?
 We are not even meeting the minimum expectations for
faculty and/or graduate students in many of the information
control questions which includes electronic access, electronic
information & print materials. (Is this because the needs are
very specialized and/or diverse? our collections are inadequate? or
because users are unable to identify and locate what we do have?)
 All users groups, especially undergraduates, have difficulties using
our online tools. There is particular confusion for undergraduates
over the terminology of GIL, GALILEO & EJL
 Users find it difficult to locate materials on the shelves, especially
those that GIL indicates are “not checked out.” There is also some
confusion on the arrangement of materials in the Library.
DO IT AGAIN?
Yes or No?
Why?
LibQUAL+ Assessment Websites
 http://old.libqual.org
Here you will find information about all aspects of the survey
 http://www.libs.uga.edu/assessment/index.html
This is the assessment site at UGA
 http://www.library.gatech.edu/about_us/libqual/
This is the LibQUAL+ results and analysis site at GT
Contacts:
Brian Mathews : [email protected]
Caroline Killens : [email protected]