Transcript Developing Tools to Assess Digital Libraries

TM
:
2002 Results
Colleen Cook
Fred Heath
April 2, 2003
Texas Library Association
Houston
Project web site www.arl.org/libqual/
Library Remains a
Credible Resource
 98% agree with statement, “My … library
contains information from credible and
known sources”
Note. Digital Library Federation and Council on Library and Information
Resources. (2002). Dimensions and Use of the Scholarly Information Environment.
Information Seeking
Behaviors are Changing
 15.7% agreed with the statement “The
Internet has not changed the way I use
the library”
Note. Digital Library Federation and Council on Library and Information
Resources. (2002). Dimensions and Use of the Scholarly Information Environment.
Finding Electronic Journals
for Research




92.7% find out about e-journals on-line
21.7% report using print resources to find
16.5% would ask a person for assistance
Only 2.5% would prefer to ask a
librarian
Note. Digital Library Federation and Council on Library and Information
Resources. (2002). Dimensions and Use of the Scholarly Information Environment.
Finding Print Journals
for Research
 Only 13.9% ask a librarian
 Only 3.2% consider consulting a librarian a
preferred way of identifying information
Note. Digital Library Federation and Council on Library and Information
Resources. (2002). Dimensions and Use of the Scholarly Information Environment.
TotalTotal
Circulation
Circulation
600,000
550,000
500,000
450,000
400,000
91
19
92 993
19
1
94
19
95
19
96 997
19
1
98
19
99 000
19
2
01
20
Note. M. Kyrillidou and M. Young. (2002).
ARL Statistics 2000-01. Washington, D.C.: ARL, p.8.
In House
Use
ofMaterials
Materials
Use of
In House
950,000
900,000
850,000
800,000
750,000
700,000
650,000
600,000
550,000
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
500,000
Note. M. Kyrillidou and M. Young. (2002).
ARL Statistics 2000-01. Washington, D.C.: ARL, p.8.
Reference
Transactions
Reference
Transactions
170,000
160,000
150,000
140,000
130,000
120,000
110,000
19
91
19
92
19
93
19
94
19
95
19
96
19
97
19
98
19
99
20
00
20
01
100,000
Note. M. Kyrillidou and M. Young. (2002).
ARL Statistics 2000-01. Washington, D.C.: ARL, p.8.
The Imperative for our Research
In an age of accountability, there is a pressing need
for an effective and practical process to evaluate
and compare research libraries. In the aggregate,
among the 124 Association of Research Libraries
(ARL) alone, over $2.8 billion dollars were
expended in 1999/2000 to satisfy the library and
information needs of the research constituencies in
North America (Kyrillidou & Young, 2001, p. 5).
LibQUAL+ Description
LibQUAL+ is a research and
development project undertaken to define
and measure library service quality across
institutions and to create useful qualityassessment tools for local planning.
LibQUAL+ Process

SERVQUAL dimensions served as a
priori theoretical starting point
Relationships: Perceptions,
Service Quality and Satisfaction
….only customers judge quality;
all other judgments are essentially
irrelevant”
Zeithaml, Parasuraman, Berry. (1999).
Delivering quality service. NY: The Free Press.
LibQUAL+ Resources
 An ARL/Texas A&M University joint
developmental effort based on SERVQUAL
 LibQUAL+ initially supported by a 3-year
grant from the U.S. Department of Education’s
Fund for the Improvement of Post-Secondary
Education (FIPSE)
 Initial project established an expert team,
regrounded concepts, and designed survey
methodology
 Survey conducted at over 200 libraries resulting
in a data base of over 78,000 user responses
LibQUAL+ Project Goals
 Establishment of a library service quality
assessment program at ARL
 Development of web-based tools for
assessing library service quality
 Development of mechanisms and
protocols for evaluating libraries
 Identification of best practices in
providing library service
LibQUAL+ Participants
Year 3
164
Participants
Year 1
12
Participants
Spring
2000
Year 4
316
Participants
Year 2
43
Participants
Spring
2001
For More Information about Participants:
Visit the LibQUAL+ web site.
Spring
2002
Spring
2003
LibQUAL+ Fundamental Contributions
to the Measurement of Effective Delivery
of Library Services
 Shift the focus of assessment
from mechanical
expenditure-driven metrics to
user-centered measures of
quality
 Determine the degree to which
information derived from local
data can be generalized,
providing much needed “best
practices” information
 Re-ground gap theory for the
library sector, especially
academic libraries
 Demonstrate the efficacy of
large-scale administration of
user-centered assessment
transparently across the web
 Grounded questions yield
data of sufficient granularity
to be of value at the local
level
 Makes little demand of local
resources and expertise
Multiple Methods
of Listening to Customers











Transactional surveys*
Mystery shopping
New, declining, and lost-customer surveys
Focus group interviews
Customer advisory panels
Service reviews
Customer complaint, comment, and inquiry capture
Total market surveys*
Employee field reporting
Employee surveys
Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
76 Interviews Conducted






York University

University of Arizona

Arizona State

University of Connecticut 
University of Houston

University of Kansas
University of Minnesota
University of Pennsylvania
University of Washington
Smithsonian
Northwestern Medical
LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred
Dimensions of Library
Service Quality
Affect of Service
Library
Service
Quality
Personal Control
Empathy
Ease of Navigation
Responsiveness
Convenience
Assurance
Modern Equipment
Reliability
Information Access
Library as Place
Scope
Utilitarian space
Timeliness
Symbol
Convenience
Refuge
Affect of Service
 Absorbed several of the original
SERVQUAL questions measuring
Responsiveness, Assurance and Empathy
 In the current analysis also includes
Reliability
 All in all: the Human Dimension of
Service Quality
Affect of Service
“I want to be treated with respect. I want you
to be courteous, to look like you know what
you are doing and enjoy what you are
doing. … Don’t get into personal
conversations when I am at the desk.”
Faculty member
Reliability
“You put a search on a book and it’s just gone;
it’s not reacquired. … There’s more of a
problem of lost books, of books that are
gone and nobody knows why and nobody’s
doing anything about it.”
Faculty member
Service as Performance
“…as users have metamorphosed from
penitents to self-reliant information surfers,
the rules of engagement have changed.
Service is not something dispensed; rather, it
is enacted as an elaborate cultural ritual, the
texture and fabric of which is changing in
front of us. Service may now embody
multiple overlays of meaning, many too
dense for anything but an anthropological
fieldwork study to uncover” (Lincoln, p. 15).
Access to Information
 Covers scope, timeliness, and convenience
of access
 Adequacy of collections
 Comprehensiveness, quality, and depth of
information resources
 All in all: required information delivered
in the format, location, and time of choice
Comprehensive Collections
“I think one of the things I love about
academic life in the United States is that as
a culture…, we tend to appreciate the
extraordinary importance of libraries in the
life of the mind.”
Faculty member
Ubiquity of Access
“Over time my own library use has become
increasingly electronic. So that the amount
of time I actually spend in the library is
getting smaller and the amount of time I
spend at my desk on the web … is
increasing.”
Faculty member
Cultural Perspective Collections
“In the physical [vs. virtual] reality, ‘texture’
has become important. Density of
collections becomes important, and, if
collections are not complete, users want to
know where they can find missing volumes,
journal articles, and/or how swiftly
interlibrary loan will work for them”
(Lincoln, p. 11).
Personal Control
 Covers ease of navigation, convenience,
and support services
 Personal control of the information
universe in general and web navigation in
particular
 All in all: How users want to interact with
the modern library
Personal Control
“By habit, I usually try to be self-sufficient.
And I’ve found that I am actually fairly
proficient. I usually find what I’m looking
for eventually. So I personally tend to ask a
librarian only as a last resort.”
Graduate student
Personal Control
“…first of all, I would turn to the best search
engines that are out there. That’s not a
person so much as an entity. In this sense,
librarians are search engines [ just ] with a
different interface.”
Faculty member
Cultural Perspective –
Self-reliance
“If Foucault is correct that we in the West live in
surveilled societies, then what function does selfreliance serve? …the library user who wishes to
navigate resources with as little help as possible –
seeks a kind of privacy from the surveillance of
librarian help …Having found the relative
anonymity of cyberspace and a virtual world, this
self-reliant user now seeks the same independence
and lack of surveillance in the text-based and
digitized universe of information resources known
as the library” (Lincoln, p. 12).
Library as Place
 Covers usefulness of space, symbolic
value, and refuge for work and study
 Transcends the SERVQUAL dimension of
Tangibles to include the idea of the library
as the campus center of intellectual activity
 All in all: As long as physical facilities are
adequate, library as place may not be an
issue
Library as Place
“I guess you’d call them satisfiers. As long as
they are not negatives, they won’t be much
of a factor. If they are negatives, they are a
big factor.”
Faculty member
Library as Place
“One of the cherished rituals is going up the
steps and through the gorgeous doors of the
library and heading up to the fifth floor to
my study. … I have my books and I have
six million volumes downstairs that are
readily available to me in an open stack
library.”
Faculty member
Library as Place
“The poorer your situation, the more you need
the public spaces to work in. When I was
an undergraduate, I spent most of my time
in the library, just using it as a study space.”
Faculty member
Cultural Perspective –
Library as Place
 “…It’s beyond the ease [with] which you can find
information, just because the library experience is
something like Greece or Athens…” (Undergraduate)
 “…the library needs to welcome them in. It needs to
make them feel like this is a place where they can be in
almost a haven, a refuge” (Business professor)
 “writing an undergraduate thesis with this big dome
over his head…he felt really like a scholar” (Linguistics
professor)
 Writing a dissertation in a particular library for another
scholar “was an emotional experience”
Survey Design Considerations
 Three scales exploring optimal, minimal,
and actual service levels
 Twenty-five questions clustered around
four dimensions
 Survey covers a sample of targeted user
population
 Data illuminates gap between desired level
of service and perception of experience
LibQUAL+ Core Questions Y1
_____________________________________________________________________________
Factor_
______ _
No.
I
II
III
IV
Item Core
_____________________________________________________________________________
32
.84947
.12848
.24465 .13335
1 Willingness to help users
33
.80847
.13662
.25348 .14147
1 Giving users individual attention
7
.80757
.17881
.12781 .21125
1 Employees deal with users caring fashion
50
.79273
.19288
.18847 .12497
1 Employees who are consistently courteous
31
.77262
.16358
.26461 .20061
1 Employees have knowledge answer questions
5
.74072
.14754
.18453 .29624
1 Employees understand needs of users
3
.74052
.15102
.17296 .20793
1 Readiness to respond to users' questions
18
.71718
.19757
.18289 .26766
1 Employees who instill confidence in users
43
.62487
.22402
.29970 .28256
0 Dependability handling service problems
20
2
19
25
41
.16556
.17739
.22362
.16013
.20398
.87679
.83172
.83147
.80492
.80204
.11430
.08498
.14705
.18894
.17599
.16236
.13901
.22566
.16628
.20255
2 A haven for quiet and solitude
2 A meditative place
2 A contemplative environment
2 Space that facilitates quiet study
2 A place for reflection and creativity
37
28
14
45
17
29
.22528
.19602
.33339
.30467
.35390
.30136
.12353
.09611
.16156
.23784
.18467
.21018
.78405
.75780
.60389
.59090
.55690
.55341
.15466
.13173
.31109
.28919
.41864
.38474
* website enabling me locate info on my own
* elec resources accessible home or office
* access tools allow me find on my own
3 Modern equip me easily access info I need
* info easily accessible for independent use
4 Convenient access to library collections
11
39
16
9
8
.13494
.14894
.29445
.27782
.22850
.23183
.23743
.19831
.05333
.18484
.18868
.29367
.22384
.16331
.13137
.73636
.60350
.60107
.57866
.56343
3 Comprehensive print collections
3 Complete runs of journal titles
3 Interdisciplinary library needs addressed
4 Timely document delivery/interlibrary loan
0 Convenient business hours
________________________________________________________________
Sample Survey
Spring 2002
Sample Survey…continued
Sample Survey…continued
Sample Survey…continued
Sample Survey…continued
LibQUAL+ 2002 Iteration
 42 — ARL Libraries
 35 — Health Sciences Libraries
 36 — State Colleges & Universities
(excluding ARL)
 34 — Private Colleges & Universities
(excluding ARL)
 15 — Community Colleges
 2 — Special & Public Libraries
(Smithsonian & NYPL)
Respondents by Age
(Excludes NYPL)
Note: LibQUAL+ Spring 2002 Aggregate Survey Results. (2002). vol. 1, p. 19
Respondents by Sex
Note: LibQUAL+ Spring 2002 Aggregate Survey Results. (2002). vol. 1, p. 20
4-Year Institution Respondent by Discipline
(n=54,073)
Note: LibQUAL+ Spring 2002 Aggregate Survey Results. (2002). Vol. 1, p. 38
Aggregate Dimension Means
(n=70,445)
Dimension
Minimum
Desired
Perceived
SA Gap
Access to Information
6.57
7.93
6.82
0.25
Affect of Service
6.51
7.90
7.11
0.60
Library as Place
5.98
7.41
6.62
0.64
Personal Control
6.74
8.15
7.07
0.33
Note: LibQUAL+ Spring 2002 Aggregate Survey Results. (2002). vol. 1, p. 24
Mean Perceived Scores
2001/2002 Trend (n=34)
7.2000
7.0000
2002 Data
6.8000
6.6000
6.4000
6.2000
6.0000
6.0000
6.2000
6.4000
6.6000
2001 Data
6.8000
7.0000
7.2000
TAMU Faculty Item Summary
Note: LibQUAL+ Spring 2002 Survey Results. (2002). vol. 2, p. 40
ARL Faculty Item Summary
Note: LibQUAL+ Spring 2002 Survey Results - ARL. (2002). Vol. 5, p. 46
Institutional Norms for Perceived
Means on 25 Core Questions
Note: Thompson, B. LibQUAL+ Spring 2002 Selected Norms, (2002).
Overall Mean Scores and Service Adequacy Gap Scores
By Cohort Group
2002 LibQUAL+ Iteration
(n=162)
Community
Colleges
7.26
(.55)
Private
Colleges
6.90
(.49)
AAHSL
7.07
(.56)
State Colleges
& Universities
6.38
(.30)
ARL
Top 40
6.84
(.46)
ARL
Other
6.74
(.27)
LibQUAL+TM Related
Documents
• LibQUAL+TM Web Site
http://www.arl.org/libqual/
• LibQUAL+TM Bibliography
http://www.coe.tamu.edu/~bthompson/servqbib
woof