Transcript Sam Kalb
Service Quality Bench-marking on a National Scale
LIBQUAL CANADA 2007
2008 IFLA Satellite Conference on Global Statistics
Sam Kalb, Library Assessment & IT Projects Coordinator
Queen’s University Library, Kingston, Ontario, Canada
Email: [email protected]
WHY NOT JUST DEVELOP A CANADIAN SURVEY?
LibQUAL+™ established survey instrument for
academic libraries
Challenges & costs to build a better Canadian survey
instrument & national support infrastructure
LIBQUAL+™ AND THE CANADIAN CONTEXT
20 Canadian LibQUAL+™ participants to 2006 but
never more than 10 in any given year
Need to develop relevant comparators reflecting the
realities of Canadian education
All
Canadian universities publicly funded
Education
a provincial (state) jurisdiction
By 2006, LibQUAL+™ was the primary instrument
used by Canadian academic libraries to assess library
service quality
ORIGIN OF THE LIBQUAL+ CANADA CONSORTIUM
Est. & funded by Canadian Association of Research
Libraries (CARL) in Jan. 2006
Goal: create a larger database of Canadian content
that would offer more meaningful benchmarking of
services for Canadian academic research libraries
Unique Opportunity: to engage the broader Canadian
academic and research library community in
developing a national service quality assessment
survey
THE 2007 LIBQUAL+ CANADA CONSORTIUM
Largest LibQUAL+™ consortium: 46 universities, 7
community colleges and 3 federal government
libraries from across Canada
66% of the libraries had never done the survey
including some smaller institutions who would not
have considered participating on their own
Bilingual Environment: English language, Frenchlanguage, and bilingual institutions
BUILDING THE LIBQUAL CANADA CONSORTIUM
What factors went into establishing and conducting this
large and successful consortial project?
Governance and Support
Project Organization & Management
Communication & Engagement
Active recruitment of participants
GOVERNANCE & SUPPORT
Governing body: CARL Committee on Effectiveness
Measures and Statistics
Funding: annual budgets for 2006 &2007
Admin. Support: CARL staff
PROJECT ORGANIZATION & MANAGEMENT
Coordination: Dedicated Project Leader working in
consultation with participants (official contacts)
Underlying assumption: most members did not have
dedicated assessment staff to manage the process
successfully on their own
Project management objective: guide consortium
members through the planning process, via discrete,
manageable sets of actions; each stage with its own
timelines and deliverables.
COMMUNICATION & ENGAGEMENT
Moderated discussion/announcement list
Members encouraged to contribute in shaping each
phase of the project
Timelines and action items were revised at each stage
based on member input.
Highest priority: Every query answered in a timely
fashion &, in most cases, exchange shared with the
membership
ACTIVE RECRUITMENT OF PARTICIPANTS
Building critical mass (invitations to join via national
& regional library councils)
Individual invitations to encourage maximum
participation by leading Canadian institutions
Rapid response to queries from potential participants,
incl. support documentation to help persuade
reluctant or wary administrators
WEB PRESENCE
Major recruitment & project management tool
Goal: to provide an easy to use, one-stop resource for
member libraries – with material, relevant to
Canadian libraries, that could be readily adapted by
individual libraries for their use.
Updated “look” throughout the project (from early
focus on attracting participants to final focus on the
survey results & their analysis
LIBQUAL & BEYOND: A WORKSHOP
OTTAWA, ONT. CANADA, OCT. 2007
Helped consortium participants to analyze their
LibQUAL+™ results effectively
1st Canadian library assessment conference
Provided 1st forum for Canadian librarians
engaged/interested in assessment to meet &
network
Attempted to encourage libraries to start building a
“culture of assessment”
CONSORTIAL DELIVERABLES
Standard LibQUAL™ consortial notebook, aggregated
by user category, library type, and survey language
The Consortial, on behalf of CARL and regional
councils in Ontario (OCUL) and Quebec (CREPUQ),
contracted with ARL for custom consortial notebooks
representing their member libraries
The councils all approved the posting of the aggregate
notebooks on the consortial web site.
DATA SETS
The Consortium received the complete data set
representing the results for all 48,000 consortium
respondents
Data set and subsets made available to all
consortium participants in spreadsheet or SPSS
format (with individual identifiable data, such as the
institution name, names of campus libraries, local
discipline groups, etc. replaced with masking codes)
SURVEY OF CONSORTIUM PARTICIPANTS
93.6% of wanted to take the LibQUAL+™ survey again
as members of the consortium
80% preferred LibQUAL+™ over developing a homegrown alternative; slight preference among
respondents for a more abbreviated LibQUAL+ ™ Lite
over the full survey
Members split evenly between 2 & 3 year options for
preferred frequency of future consortial surveys
Ratings for consortial support and responsiveness
were very high
CHALLENGES FOR CONSORTIUM & ITS MEMBERS
Demands on staff time to plan the survey & to review,
analyze & act on the results – greatest for libraries
with fewer staff. Limited data analysis expertise.
Few community college participants in the 2007
survey & widely differing mandates among the
Canadian provinces as to clientele served and types
of academic and non-academic programs. Need for
more web resources aimed at community colleges
Limited benchmarking value for federal government
libraries who each have such widely different clientele
and mandates
HOW TO IMPROVE THE SURVEY FOR OUR MEMBERS
Alternative, briefer LibQUAL+™ surveys
Alternative delivery mechanisms
Customizable set of user types linkable to a set of
standard user categories (similar to discipline group
mapping)
Customizable labels mapped to the same survey
concepts for different cultures (e.g. “gender” instead
of “sex”)
More effective mapping & management of survey
questions in different languages
CONCLUSION
Despite the challenges, the 48,000 consortial responses to the 2007
survey have provided a rich, unique resource of assessment data
for Canadian academic and research libraries that can only grow
more valuable each time the consortium runs the survey.
QUESTIONS?
Thank You!