Transcript 1. Sam Kalb (Queen’s University). Using the LibQUAL +® notebooks and other LibQUAL+ services to analyze and present your results (50 minutes) 2. Eun-ha Hong (Wilfrid.

1.
Sam Kalb (Queen’s University). Using the
LibQUAL +® notebooks and other LibQUAL+
services to analyze and present your results (50
minutes)
2.
Eun-ha Hong (Wilfrid Laurier University). Using
statistical tools to further analyze your
LibQUAL+® data (15 minutes)
3.
Questions/discussion (25 minutes)
• Any LibQUAL participant can access the
results of all other libraries and consortia
who participated in the same year
• “In an example of collaboration,
LibQUAL+® participants are sharing their
results within the LibQUAL+® community
with an openness that nevertheless
respects the confidentiality of each
institution and its users.” Martha Kyrillidou, 2010
LibQUAL +® survey, introduction.
• Use the LibQUAL +® Notebooks & analytic
utilities to perform some simple analyses of
your LibQUAL+® survey results
• Present the results to your stakeholders
• Use the data to target areas for
improvement
From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa,
Ontario, Canada, October 24-25, 2007
¾ of members chose the Lite format
Total LibQUAL+® Canada Members 2010
Universities
Community Colleges
Full
Lite
Total
11
33
43*
% of Protocol
84.6
91.7
% of Total Cases
23.4
70.2
91.5*
2
3
4*
15.4
8.3
% of Total Cases
4.3
6.4
8.5*
Count
13
36
47*
% of Protocol
100
100
% of Total Cases
27.7
76.6
Count
Count
% of Protocol
Total
(by Survey Protocol)
100*
• 2010 LibQUAL+® Lite: 61.7%
• 2010 LibQUAL+® full: 54.3%
• 2007 LibQUAL+® full: 48.8%
Completed Survey. The user has supplied a rating for all items on
the survey.
There are no absolute high or low scores.
2. Scores are relative indicators
3. Scores are only meaningful in comparison with
other scores in the same survey, your survey
from another year, other individual libraries
and the consortial totals.
1.
1.
Zone of tolerance
• Perceptions vs. expectations
• meeting users minimum expectations (Adequacy
Gap)
• Approaching users’ desired expectations (Superiority
Gap)
2.
My scores over time (longitudinal)
• Am I doing better or worse compared to last time I
measured my performance
3.
Peer comparisons
From presentation by M. Kyrillidou, ALA, June 2007
User Assigned:
• Desired. How highly do I value it?
• Minimum. What is my least expectation?
• Perceived. My actual rating
Calculated Scores:
• Adequacy Gap. Perceived – Minimum
• Superiority Gap. Desired – Perceived
1.
2.
Desired. How important is a service
relative to other services; relatively to
same services among other libraries
Adequacy Gap. How far above or
below Minimum expectation is the
service delivered.
Focus mainly on your results by User Group
The major academic user groups (faculty, grads and undergrads)
have quite different perspectives and expectations of library
services.
Overall totals from different libraries or different years could be
skewed by variations in user groups ratios.
•
•
•
•
Look at the 5 most and the 5 least desired questions by each
user group
Look at the services with 5 highest and the 5 lowest gap scores
(Bear in mind that Superiority Gap scores are usually negatives.
Look for any correlation between the high/low Desired and the
high/low Gap scores
Look at the average (mean) scores for each service dimension
by user group
Queen's 2007
Dim.
Question
Canadian Consortium (Universities) 2010
Queen’s 2010
Score
Dim.
Question
Score
Dim.
Question
Score
Overall
Overall
Overall
IC-1
Making electronic resources
accessible from my home or
office
8.51
IC-1
Making electronic resources
accessible from my home or
office
8.51 IC-1
Making electronic resources accessible
from my home or office
8.22
IC-8
Print and/or electronic journal
collections I require for my
work
8.48
IC-2
A library Web site enabling me to
locate information on my own
8.38 IC02
A library Web site enabling me to locate
information on my own
8.17
IC-4
The electronic information
resources I need
8.39
IC-8
Print and/or electronic journal
collections I require for my work
8.32 IC-8
Print and/or electronic journal collections
I require for my work
8.16
IC-2
A library Web site enabling
me to locate information on
my own
8.38
IC-4
The electronic information
resources I need
8.16 IC-6
Easy-to-use access tools that allow me to
find things on my own
8.06
IC-6
Easy-to-use access tools that
allow me to find things on my
own
8.30
IC-6
Easy-to-use access tools that
allow me to find things on my own
8.15 AS-3 Employees who are consistently
courteous
8.06
• In addition to analyzing your results by User
Group, you can analyze your results by Discipline
using ARL’s Analytics utility (to be covered later in the
presentation).
• Focus on LibQUAL +®’s Standard Disciplines
› Facilitates comparison with other libraries
› Most useful for Information Control dimension
•
Report - results notebook (PDF)
•
Comments - respondents’ free-text comments (CSV or TXT)
•
Consortial Notebooks – Consortium, CARL, CREPUQ, OCUL
(PDF)
•
Raw Data – individual records of every respondent’s survey in
CSV format (spreadsheet)
•
Data Keys & SPSS Syntax – description of the raw data
elements or fields (MS Word)
http://www.libqual.org/SurveyInstruments/LibQual/DataRepository.aspx
http://library.queensu.ca/webir/canlibqual/results-e.htm
• Sections for Overall, Undergraduates, Graduates, Faculty,
Staff, Library Staff include:
› Demographic Summary
› Core Questions Summary
› Dimensions Summary
› Local Questions
› General Satisfaction Questions
› Information Literacy Outcomes Questions
› Library Use Summary
• Appendix describing changes in the dimensions and the
questions included in each dimension.
* Queen’s University 2010 LibQUAL +® Survey
Population
Respondents
Response Rate
Undergrads
473,406
30,173
6.7%
Grads
104,867
11,201
10.7%
All Faculty
47,867
51,45
10.7%
Full-time Faculty
29,039
4,606
15.9%
• Tables & Charts
• Individual questions – average scores & standard deviations
• Dimensions summary
How closely does a mean score in a notebook represent all
the individual respondents scores for the particular item. If all
respondents rated AS-1 Desired as 7.64, the SD would be 0
Desired
Perceived
Target
Minimum
Zone of
tolerance
• Affect of Service – Customer/client service
• Information Control – Collections & access to
collections
• Library as Place – Physical facilities
Range of
Mean scores
is relatively
narrow
(6.27 – 8.03)
• Potential areas for improvement or further
investigation
Survey Year
Minimum
Mean
Perceived
Mean
Desired
Mean
Adequacy
Gap
current
6.72
7.45
7.85
0.73
previous
6.33
7.29
7.80
0.96
Always check the Minimum mean score when
evaluating the rise or fall in your Adequacy Gap
score. A lower Gap score just may result from
rising expectations
From presentation by M. Kyrillidou, ALA, June 2007
From presentation by M. Kyrillidou, ALA, June 2007
• Local Questions
• General Satisfaction Questions
• Information Literacy Outcomes
Questions
• Library Use Summary
• You can compare your individual local question
results with:
• Your past surveys
• Consortial or peer library results (assuming
sufficient number of respondents, e.g. 100+)
LibQUAL Canada 2010. Item 3.4. Location Questions Summary
• These are simple indicators without the context
of minimum and desired ratings
LibQUAL+ 2010 – General Satisfaction
LibQUAL+
Canada
Queen's
In general, I am satisfied with the way in which I
am treated at the library.
7.54
7.59
In general, I am satisfied with library support for
my learning, research, and/or teaching needs.
7.11
7.14
How would you rate the overall quality of the
service provided by the library?
7.19
7.32
From presentation by M. Kyrillidou, ALA, June 2007
• Almost ½ of respondents fill in the Comments
box
• Provide valuable insights and suggestions for
improvements
• Essential component in understanding the
reasons behind the survey scores.
• Available on LibQUAL+® Web site
Repository
www.libqual.org
Data
Download file in csv(Excel) or txt (text)
 Record Content:

› ID:
› UserGroup:
› Discipline:
› Branch:
› Age:
› Sex:
› Comment:
•
•
Skim the comments
Load csv file into text analysis program, e.g.
ATLAS.ti, InMagic, etc.
• http://db.library.queensu.ca/libqual/index2010.htm
•
•
Conduct analysis
Identify major themes, e.g. study space,
library catalogue, noise, etc.
› http://library.queensu.ca/webir/libqual-2007/issues&actions.html
•
Analytics
•
Norms Tables
www.libqual.org
• Institutional Explorer (peer comparison)
• Representativeness graphs
• Radar chart
• Library Use chart
• Thermometer chart
• Cumulative percentile distribution
• Longitudinal Analysis (by survey year)
• Cumulative percentile distribution
http://www.libqual.org/SurveyInstruments/LibQual/Analytics.as
px
2010 Queen’s LibQUAL +® results
Undergraduate Business students
Heath Sciences Graduate Students
• ARL provides templates to assist you in preparing
customized radar and thermometer charts to create
custom analyses
http://www.libqual.org/SurveyInstruments/LibQual/Resources.aspx
www.libqual.org
•
Identify your score
•
Compare it to a relevant norms table
• Year
• Subgroup
• Dimension
•
Norms are stable and are not calculated on an annual
basis anymore (last calculated 2005)
•
Norm Conversion Tables facilitate the interpretation of
observed scores using norms created for a large and
representative sample.
http://people.cehd.tamu.edu/~bthompson/libq2005.htm
From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada,
October 24-25, 2007
• Customized Discipline Analysis
• Library Branch Analysis
• Other Customized Analyses (upon
request)
• Print Copies
http://www.libqual.org/about/about_lq/fee_schedule
www.libqual.org
• Identify all of the stakeholders or constituents
who want and need to know about the survey
results
• Consider the “stake” of each of the above;
what specific aspect of LibQUAL+® will be of
most interest / concern
• Determine how to communicate with each
identified stakeholder
From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October
24-25, 2007
1st Priority. Particularly those whom you asked to
participate in the survey. As soon as you can:
1. Announce incentive award winners
2. Inform users of highlights of survey results
3. Present weak areas as challenges and
opportunities not as negatives
4. Most importantly, what the library intends to do.
Describe action items begun and planned
Example: http://library.queensu.ca/libqual%202010
From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October
24-25, 2007
•
Compare your results with the corresponding consortial
results and those of peer Canadian libraries
•
Compare your results over multiple LibQUAL +® surveys
(longitudinal analysis)
• Look at results to determine if users are not aware of
what the library already does
• Explore one question by discipline and user group
• Probe the questions that had meaningful gaps between
perceived results and minimum expectations (Adequacy
Gap)
From presentation by M. Kyrillidou, ALA, June 2007
From all of the data, determine what can and should
be addressed
• Prioritize some action items
•
• Align with mission, vision and goals of parent
•
•
•
•
organization
Address users’ top priorities, by user group
Improve areas of strong user dissatisfaction
Build on strengths, if they are truly user needs and
priorities
Identify work that can be de-emphasized and
resources that can be reallocated
From: presentation by R. Bowlby and M. Kyrillidou,, LibQUAL+® Canada Workshop, Ottawa, Ontario, Canada, October
24-25, 2007
• Carefully crafted Executive Summary
• Library performance (strengths &
opportunities for improvement/
enhancements in services and priorities
• Present results via explanatory text with
supporting tables and charts in an
appendix. Don’t rely on tables and chart to
convey your point.
2007 Findings
Actions
Example from Library Web Site: http://library.queensu.ca/webir/libqual2007/issues&actions.html
Queen’s University LibQUAL+ 2010 - Core Questions
Summary by User Group
User Group
Minimum
(Mean/Avg)
Desired
(Mean/Avg)
Perceived
(Mean/Avg)
Adequacy
Gap
Queen's
CanCon
Queen's
CanCon
Queen's
CanCon
Queen's
CanCon
Faculty
6.55
6.75
7.72
7.83
7.18
7.03
0.63
0.28
Graduate
6.62
6.67
7.93
7.97
7.11
7.04
0.49
0.37
Undergraduate
6.27
6.47
7.83
7.87
6.89
6.96
0.62
0.49
Example: Queen’s University 2010 user group scores compared
with the LibQUAL Canada Consortial results
Queen’s University Library – 2007 Survey
Results Presentation Plan
•
•
•
•
•
Completed Report discussed at Management Team;
consultation plan developed
Report, comments database, and plan distributed to all staff
All-Staff information session
Units and functional teams
• Meetings of individual units and functional teams identify
the issues in their areas of responsibilities and recommend
appropriate actions.
Management Team
• Reviewed the compilation of issues and objectives in
developing the 2008/09 Budget Report. Compiled and
approved action items prepared by the functional teams
and units.
• Engage the staff in reviewing the
results and developing solutions
• Focus on assessment of services
not people; don’t make the survey
a staff assessment tool
• Keep the experience a positive
one for the staff
old.libqual.org
• The academy is an intensive five-day
program that focuses on both qualitative
and quantitative methods for collecting
and analyzing library service quality data.
• Next scheduled for March 14-18, 2011,
Toronto, Ont.
• Deadline for application: December 15,
2010
http://www.arl.org/stats/statsevents/sqacademy/index.shtml
•
LibQUAL Canada:
http://library.queensu.ca/webir/canlibqual/c
arl-libqual.htm
•
LibQUAL +®: www.libqual.org
•
Other library web sites
•
http://library.queensu.ca/webir/canlibqual/canlibs.htm
•
http://www.libqual.org/about/about_survey/related_sit
es