Transcript Drilling Down: LibQUAL Quantiative and Qualitative Data

Drilling Down: LibQUAL
Quantitative
and Qualitative Data
Allison Sivak
Assessment Librarian
University of Alberta Libraries
October 25, 2007
Assessment Challenges
•
•
•
•
•
Gathering meaningful data
Acquiring methodological skills
Managing assessment data
Organizing assessment as a core activity
Interpreting data within the context of user
behaviors and constraints
Troll Covey, 2002
Major Trends
• Appetite for electronic access constantly increasing
• Access and content are the same thing
• Expectation that electronic access & searching is easy
and reliable
• I need it, now is when I want it, as easy and as quickly as
possible, by myself, from wherever I am
• Library space solution a challenge of conflicting
expectations – not yet solved
• ILL service a problem with GSs, declining issue with UGs
(specificity vs. satisficing?)
• Inconsistencies in customer service noticed and noted
How We’ve Responded
• Strong focus on improving electronic access
– Proxy server replaced with seamless remote access
– Link resolver
– Aggressive e-collection development
• Hours
• Study space zoning
– Space planning and renovation
• Customer Service Standards
• Notifications / request services
– User-initiated online holds, recalls, renewal
– Retrieve and send material to library of choice
• Follow-up studies: web experience testing, collections data,
e-books survey, faculty /partnership investigation
Ryan, 2006
Overall Quality of Service Satisfaction,
2002 - 2007
9.00
UG
GS
FA
8.00
7.00
6.00
5.00
4.00
2002
2003
2004
2005
2006
2007
Overall Satisfaction with Library Support for Learning,
Research, and/or Teaching Needs, 2002 - 2007
9.00
UG
GS
FA
8.00
7.00
6.00
5.00
4.00
2002
2003
2004
2005
2006
2007
Overall Satisfaction with Treatment at Library
2002 - 2007
9.00
UG
GS
FA
8.00
7.00
6.00
5.00
4.00
2002
2003
2004
2005
2006
2007
Year
Cases
Desired Mean
2004
2007
615
452
527
7 .9 4 7
7 .8 9 6
7 .9 8 4
7 .9 0 6
5
Desired Maximum
9
A dequacy Gap Mean
2006
589
Desired Minimum
Perceived Mean
2005
4 .7 2 7
9
5
9
1 .5 9 1
9
7 .0 1
7 .0 1 8
7 .1 2 4
6 .9 3 5
0 .4 8 3
0 .5 4 5
0 .5 2 9
0 .3 7 2
Most Important Services: Information Control
Undergraduate Students
Making electronic resources available from my home or
office
Print and/or electronic journal collections I require for my
work
Modern equipment that lets me easily access needed
information
The electronic information resources I need
Easy-to-use access tools that allow me to find things on my
own
Making information easily accessible for independent use
A library Web site allowing me to locate information on
my own
Graduate Students
Print and/or electronic journal collections I require for my
work
A library Web site allowing me to locate information on
my own
Making electronic resources accessible from my home or
office
The electronic information resources I need
Making information easily accessible for independent use
Easy-to-use access tools that allow me to find things on my
own
Modern equipment that lets me easily access needed
information
The printed library materials I need for my work
Desired
Score
8.30
8.20
8.12
8.11
8.10
8.06
8.01
Desired
Score
8.58
8.52
8.50
8.51
8.38
8.36
8.15
8.06
Most Important Services: Information Control
Faculty
Making electronic resources accessible from my home or
office
Print and/or electronic journal collections I require for my
work
The electronic information resources I need
A library Web site allowing me to locate information on
my own
Making information easily accessible for independent use
Easy-to-use access tools that allow me to find things on my
own
Modern equipment that lets me easily access needed
information
Desired
Score
8.66
8.64
8.54
8.40
8.37
8.36
8.15
Most Important Services: Affect of Service
Undergraduate Students
Employees who have the knowledge to answer user
questions
Employees who are consistently courteous
Graduate Students
Adequate hours of service
Readiness to respond to users’ questions
Employees who are consistently courteous
Employees have the knowledge to respond to user
questions
Dependability in handling users’ service problems
Willingness to help users
Faculty
Dependability in handling users’ service problems
Employees have the knowledge to respond to user
questions
Employees who are consistently courteous
Desired
Score
8.08
8.08
Desired
Score
8.23
8.20
8.18
8.17
8.16
8.09
Desired
Score
8.11
8.04
8.01
Most Important Services: Library as Place
Undergraduate Students
A getaway for study, learning, or research
Desired
Score
8.03
Graduate Students - No ‘Desired level’ scores over 8.0 in this dimension.
Faculty - No ‘Desired level’ scores over 8.0 in this dimension.
Undergraduate Students
Library space that inspires study and learning
Making electronic resources accessible from my home or
office
Making me aware of library resources and services
The electronic information resources I need
Quiet space for individual activities
Print and / or electronic journal collections I require for my
work
A getaway for study, learning, or research
Easy-to-use access tools that allow me to find things on my
own
Teaching me how to access, evaluate, and use information
A library website enabling me to locate information on my
own
A comfortable and inviting location
Modern equipment that lets me easily access the information I
need
Dependability in handling users' service problems
Making information easily accessible for independent use
Employees who are consistently courteous
The printed library materials I need for my work
Employees who deal with users in a caring fashion
Community space for group learning and group study
Timely ILL / document delivery
Convenient business hours
Informing me of useful library services
2002
n/a
-1.68
2003
-1.30
-1.43
2004
-1.30
-1.23
2005
-1.25
-1.28
2006
-1.45
-1.38
2007
-1.86
-1.33
n/a
n/a
n/a
n/a
n/a
-1.14
-1.28
-1.32
n/a
-1.20
-1.06
-1.29
n/a
-1.17
-0.87
-1.31
n/a
-1.20
-1.20
-1.16
-1.23
-1.20
-1.18
-1.18
n/a
-1.18
-0.92
-1.51
-0.94
-1.11
-0.88
-1.14
-1.10
-1.00
-1.17
-1.06
n/a
-1.08
n/a
-1.17
n/a
-1.04
n/a
-1.10
-0.73
-1.08
-1.02
-1.02
-0.65
-1.19
-0.95
-1.02
-1.16
-0.85
-0.74
-0.97
-1.02
-0.98
-1.02
-0.99
-1.11
-1.08
-0.64
n/a
-1.19
n/a
-1.03
-1.49
n/a
-0.97
-1.10
-1.13
-1.10
-0.97
-0.97
-1.09
-1.20
n/a
-0.87
-0.88
-0.86
-0.98
-0.81
-0.67
-0.88
-1.14
-0.82
-0.97
-0.93
-0.91
-1.02
-0.74
-0.52
-0.87
-1.03
-1.05
-1.03
-0.83
-1.08
-0.97
-0.78
-0.50
-0.96
-0.92
-1.02
-0.95
-0.95
-0.84
-0.82
-0.74
-0.68
n/a
n/a
n/a
Graduate Students
2002 2003 2004
2005
2006
2007
The electronic information resources I need
n/a -1.40 -1.43
Print and / or electronic journal collections I require for my
n/a -1.72 -1.62
work
Library space that inspires study and learning
n/a -1.05 -1.13
A library website enabling me to locate information on my
-0.91 -1.20 -1.36
own
Making electronic resources accessible from my home or office -1.45 -1.17 -1.25
Easy-to-use access tools that allow me to find things on my
-1.01 -1.47 -1.22
own
Quiet space for individual activities
n/a -0.90 -0.72
-1.39
-1.47
-1.06
-1.14
-1.38
-1.27
-0.77
-1.08
-0.87
-0.94
-1.22
-1.20
-1.29
-1.10
-0.95
-0.89
-1.15
-1.14
-0.73
-0.66
-1.10
Modern equipment that lets me easily access needed
information
A getaway for study, learning, or research
Making me aware of library resources and services
The printed library materials I need for my work
Making information easily accessible for independent use
Adequate hours of service
-0.96 -1.06 -0.92
-0.96
-0.76
-0.84
n/a -0.85 -0.88
n/a n/a n/a
n/a -1.32 -1.15
-1.04 -1.15 -1.04
n/a n/a n/a
-0.83
n/a
-1.17
-0.91
n/a
-0.65
n/a
-0.72
-0.71
n/a
-1.07
-1.05
-1.05
-1.01
-0.95
Ease and timeliness in getting materials from other libraries
Employees who are consistently courteous
Informing me of useful library services
n/a n/a n/a
-0.64 -0.92 -0.79
n/a n/a -1.03
n/a
-0.81
-1.03
n/a
-0.73
-0.74
-0.93
-0.93
n/a
Faculty
2003
2007
Ability to navigate library web pages easily
Print and / or electronic journal collections I require for my
work
A library website enabling me to locate information on my
own
Easy-to-use access tools that allow me to find things on my
own
The printed library materials I need for my work
n/a
-1.78
-1.75
-1.71
-1.30
-1.59
-1.40
-1.55
-1.41
-1.43
Making electronic resources accessible from my home or office
The electronic information resources I need
Ease and timeliness in getting materials from other libraries
Dependability in handling users’ service problems
Modern equipment that lets me easily access needed
information
Adequate hours of service
Library space that inspires study and learning
Making information easily accessible for independent use
Timely document delivery / interlibrary loan
Convenient service hours
Convenient access to library collections
-1.52
-1.21
n/a
-0.72
-1.07
-1.40
-1.43
-1.28
-1.11
-1.15
n/a
-0.38
-1.13
-1.85
-1.25
-1.20
-1.10
-1.11
n/a
n/a
n/a
n/a
Number of Comments by User Group and Year
300
250
200
GS
UG
Faculty
150
100
50
0
2003
2004
2005
2006
2007
Number of Comments by Broad Category and Year
400
350
300
250
Affect of Service
200
Information Control
Library as Place
150
100
50
0
2003
2004
2005
2006
2007
Total Comments by User Group and Broad Category
600
500
400
Affect of Service
300
Information Control
Library as Place
200
100
0
Faculty
GS
UG
Library as Place: Subissues
c onvenient
s ervic es
c onnec tivity
100%
90%
80%
lac k of room
70%
group s pac e
60%
c omfort
50%
lighting
40%
maintenanc e
30%
wayfinding
20%
aes thetic s
10%
nois e
0%
2003
2004
2005
2006
2007
Library as Place: Proportions of Criticism / Praise
100%
80%
60%
general praise
general criticism
40%
20%
0%
2003
2004
2005
2006
2007
Affect of Service: Subissues
100%
90%
80%
70%
Affect of Service staff
60%
Affect of Service
reference & instruction
50%
Affect of Service general
praise
40%
Affect of Service general
criticism
30%
20%
10%
0%
2003
2004
2005
2006
2007
Affect of Service: Staff Courtesy
100%
90%
80%
70%
60%
positive
50%
negative
40%
30%
20%
10%
0%
2003
2004
2005
2006
2007
Size Matters
• Affect of Service wasn’t as far from
“desired service level” for UAL users in
quantitative ratings
• Library as Place not ranked above 8.0 in
importance (except for undergraduates)
– Length of comments suggest impact when
unsatisfied!
• Unevenness of service: “Good but….”
• Lengthy descriptions of outstandingly positive or
negative experiences
•
Although I get good service most of the time, I have occasionally encountered
a library staff member who seemed reluctant to expend the effort in fully
assisting me. The worst example of such happened this 2006 Winter term
when I was trying to access my [] reserve material at [] Library. This was my
first time doing so and I was unaware that there were call numbers for reserve
material. I'd misplaced my syllabus (which probably had that info) but I
thought I could just obtain some staff assistance in finding the reserve
material instead of making another special trip to []. The staff member I first
approached at the reserve desk told me that I needed to find the call numbers.
When I stood there looking confused, she showed me how to get to the
webpage on her computer. Then she indicated that I should learn to find the
call numbers on my own at a different computer (which were all full by the
way). I would have appreciated it if she had walked me through ALL of the
steps...especially considering that this was my first time, she was already at
the right webpage to start searching, and there were NO OTHER STUDENTS IN
LINE. In fact, I was the ONLY student being helped at that moment for the
entire reserve desk and if the search had become longer than expected or if
there were other people waiting for assistance, then I would have gladly taken
myself elsewhere to continue it. I left the reserve desk feeling angry since I
did not feel that circumstances warranted that type of treatment. Fortunately,
when I finally returned to the reserve desk, I received better help from a
different person the second time around. Not only did I obtain the reserve
material, but she also helped me renew a couple of items on my account. I
appreciated the fact that she didn't tell me to go somewhere else to do it on
my own!
• This is a slightly difficult survey to respond to, because my overall
concern is with the variability of service at [] Library. There is a core
of highly professional and knowledgeable librarians and technicians
who staff Rutherford; these are highly capable professionals who have
greatly enhanced my own research, the quality of research at the
University of Alberta generally, and who clearly provide highly
conscientious stewardship over library collections and services,
respectively. I pray that my students get them when they go to the
library. But I know very well that this is purely the luck of the draw,
and that it is quite possible that they will be helped by someone
whose qualifications and abilities are significantly lower than what I
am evoking here. Thus I would suggest that the library make it a top
priority to make the *point of contact* librarians more universally
professional. This really will have a major impact on whether
students seek out the help of a librarian in doing research, and will, I
believe, have a major impact on the quality of work they are able to
do in our libraries. I talk up our libraries whenever I can, as I really
believe that they are a big part of what makes the U of Alberta a
great place to work or to be student. But seriously. I feel undercut in
this advocacy when I hear tales of library staff with very little
academic or professional formation helping students or researchers. I
have been helped by a fair number of such folks myself, so I know
very well that this is more than just student bellyaching.
• I have primarily used the [] library and feel I have to comment on
the staff in this library. Any time I have required information or
assistance they have been more than willing to help and this is
greatly appreciated by myself and others. I know of students, in at
least two other faculties, that specifically come to the [] library
because the staff is more helpful and they prefer this library on
campus. I believe this says much about the staff and we
appreciated the friendly, courteous manner in which we are treated
by these individuals.
• My biggest concern, however, is the out-of-control situation
regarding students. The library's decision to allow food and drink
has been disastrous, not only for the long-term safety of the
collections, but for those of us who use the library for working
purposes. Even the allegedly "silent" areas echo to the sound of
munching and crunching, fiddling with food wrapping, and the
pervasive smell of chinese food/burgers/fries etc. It is unbelievably
distracting at best, and for those of us brought up to worry about
insect damage and grease on pages, it's a situation which is
genuinely distressing. Add to this the endlessly ringing cellphones,
computer games, high-volume iPods, etc. and you have something
which resembles a high school cafeteria more than a library. Can
we stop the political nonsense about "accessibility" please? There
is nothing "inaccessible" about a library with sensible rules. As it
stands, the library is become "inaccessible" for those who need it
most: students genuinely trying to study and faculty members
desperate to make the most of that morning or afternoon of
research time. We want our libraries back!
What Else?
SLIS Studies
• Given, Lisa M. 2007. Faculty and librariansﾕ perspectives on
academic space: Setting the stage for undergraduatesﾕ
information behaviors. In The Library as Place: History, Community and
Culture, edited by John Buschman and Gloria J. Leckie, 177-189.
Westport, CT: Libraries Unlimited.
• Given, Lisa M. [In press]. Emotional entanglements on the
university campus: The role of affect in undergraduatesﾕ
information behaviors. In The Emotional Information Environment: The
Emergent Affective Paradigm in Information Behavior Research and Theory
[working title], edited by Diane Nahl and Dania Bilal. Medford, NJ:
American Society for Information Science & Technology
(Information Today, Inc.).
• Sadler, Elizabeth (Bess), and Lisa M. Given. 2007. Affordance
theory: A framework for graduate studentsﾕ information
behaviors. Journal of Documentation 63, no.1:115-141.
What Else?
Internal Assessment Projects
– new service model
•
•
•
•
•
Single help / access services desk
Librarian and associate staff work side by side
Research referral
Self-sufficiency in circulation
Self-serve reserve and e-reserve
– Delving deeper into website
•
•
•
•
Assessment of user guides
Assessment of digitized collections
Creating web usability testing plan
Usability testing of link resolver
– Talking to institutions that score higher on Information Control
Questions?
[email protected]
492.7324