Qualitative Assessment in Libraries: Adding Meaning to

download report

Transcript Qualitative Assessment in Libraries: Adding Meaning to

Qualitative Methods in
Assessing Libraries, Users
and ICT Applications
Dr. R. Sevukan
Associate Professor & Head
Dept. of Library & Information Science
Pondicherry University
Puducherry, India
[email protected]
NACLIN, 9-11 December, 2014
What is an Assessment?
It is the process of gathering information
from the stakeholders using various
methods to systematically gauge the
effectiveness of an institution.
Why is assessment important?
Are we delivering what we think we are
Are customers getting what they are
supposed to be getting?
Are customers satisfied with what they are
Is there a way to deliver the information
better, thereby promoting satisfaction?
Answers to these questions
Information sources needed for research,
teaching, and learning;
Reasons and frequency of library use;
Campus computer network connectivity;
Use of electronic resources;
Instructional needs and effectiveness;
Satisfaction on the services available; and
Enhance the quality of services rendered.
Methods of Assessment
(Source: Niglas, 1999)
Qualitative Vs. Quantitative
Qualitative Research is primarily exploratory research
used to gain an understanding of underlying reasons,
opinions, and motivations. It provides insights into the
problem or helps to develop ideas or hypotheses for
potential quantitative research.
Quantitative Research is used to quantify the problem by
way of generating numerical data or data that can be
transformed into useable statistics. It is used to quantify
attitudes, opinions, behaviors, and other defined
variables – and generalize results from a larger sample
Qualitative Methods:
Naturalistic –
Emergent –
Purposeful –
Investigate social processes as they unfold
naturally rather than reasoning from either the conditions
under which they occur or the outcomes that correlate with
Willingness to adapt methods of analysis as
knowledge and understanding deepens
Cases are selected because they are
“information rich”
Source: Patton, Michael Q. Qualitative Research and Evaluation Methods. 3rd ed. (Thousand Oaks, CA: Sage, 2002), pp. 4041.
Data Collection Characteristics of
Qualitative Method
Thick description –
Personal experience and engagement –
Empathic stance –
Attention to process –
The goal is deep understanding
about phenomena and experiences.
researcher is the primary instrument of data collection and
Sensitivity, respect, awareness,
responsiveness, and openness.
Situational and contextual
changes occur throughout the life of the study.
Analytical Characteristics of
Qualitative Method
Unique case orientation – Every case is special and
Inductive and creative – Research process guided by
analytical principles rather than rules
Holistic perspective – Phenomena under study are
complex and cannot be reduced to a few variables
Context sensitivity – Avoid broad generalizations;
investigation does not rest in isolation from the larger social/
cultural/ economic/political environment
Reflexivity – Critical self-awareness of one’s own
presence in the research process
Selective Applications in
Library Settings
To assess individualized outcomes
 To explore internal dynamics of programs and services
 To evaluate critical cases of program or service utilization
 To acquire details about program implementation
 To obtain information about the nuances of program or service
 To personalize the evaluation process and/or outcomes
 To discover effects of specific programs or services
 To add depth, detail, and meaning to statistical data
 To provide privileged access to processes, causes, and effects.
Source: Patton, Michael Q. How to Use Qualitative Methods in Evaluation. 1st ed. (Thousand Oaks, CA: Sage, 1987), pp. 41-42.
Some Techniques for Collecting
Assessment Data
In-Depth Interviews – speaking with and listening to
our users
Observation – witness what is going on in a particular
setting and explore the experiences of others
Qualitative Case Studies – holistic understanding of
social phenomena in library settings
In-Depth Interviews
Outcomes for Assessment
What is meaningful or important in the respondent’s
own voice?
Researcher’s knowledge, expertise, and interpersonal
skills can be used to explore interesting or unexpected
concepts or issues raised by respondents
Opportunity for deep probing of issues
Source: Patton, Michael Q. Qualitative Research and Evaluation Methods. 3rd ed. (Thousand Oaks, CA:
Sage, 2002)
Observation: Outcomes for
Can facilitate multidimensional and long-term
imbeddedness in the “natural setting”
Melds together aspects of looking and listening
and watching and asking (connect what people are
saying to what they are actually doing)
Unmasks everyday interaction between people
and organizations; reveals the mundane
Cases can be revisited repeatedly
Case Studies Design Considerations
Individuals – comparative sample of individual experiences
Groups – sample of individuals who share a common identity
Communities – sample of individuals with ethnic, cultural,
spiritual or other types of characteristics in common
Programs – comparative sample of projects or services
intended to meet a public need
Policies – comparative sample of plans of action
Organizations – comparative sample of a formal group of
people with one or more shared goals
Qualitative Case Studies
Applications for Assessment
Illustrative: Descriptive and intended to add “realism” and in-depth
examples to other information about a program or policy.
Exploratory: Also descriptive, but aimed at generating hypotheses for later
investigation rather than for purposes of illustrating.
Deviant/Critical Instance: Examines a single instance of unique interest
or serves as a critical test of an assertion about a program, problem, or
Program Implementation: Investigates operations, often at several sites,
and with a subjective, value laden, and/or affective emphasis.
Program Effects: This application uses the case study to examine causality
and usually involves multi-site, multi-method assessments.
Cumulative: This brings together findings from many case studies to
answer an evaluation question, whether descriptive, normative, or causeand-effect.
Assessment Tools: A Glance
The Challenge of
Assessment in Libraries
Traditional statistics emphasize inputs, expenditures,
acquisitions, holdings, etc.
Helping funding
No demonstrable relationship between expenditures and
service quality
Lack of metrics describing outcomes: success from the
user’s point of view
Need to redesign library services to better meet
changing patterns of use
Building the climate, tools, and skill set for library
No Single Method!!!
“The difficulty lies in trying to find a single model or set
of simple indicators that can be used by different
institutions, and that will compare something across
large groups that is by definition only locally applicable –
i.e., how well a library meets the needs of its institution.
Librarians have either made do with oversimplified
national data or have undertaken customized local
evaluations of effectiveness, but there has not been
devised an effective way to link the two.”
Sarah Pritchard, Library Trends, 1996
Successful Model
“Different is not
always better but
better is always
“Let us replicate but not duplicate research”
--- H.K.Kaul