Transcript LIS403 Evaluation of Information Services

LIS403 Evaluation of
Information Services
Feb 11, 2009 - Session #2
For today…
•
•
•
•
4 articles to read and consider;
One to critique
Re-read Weiss, ch 6-8
Begin to consider other forms of
evidence and how the pieces (writing,
argument, concept of warrant, literature
review) collective answer the research
question(s)
Consumer … towards
producer
• What to ask if you’re an informed
consumer?
– Did you create a criteria? Consider a
published one?
When reading an article, see if you can fill in the blanks:
The author is studying __________
because s/he wants to find out who/how/why ______,
in order to understand how/why/what _______.
From Booth et al.
The bigger picture
• What rights do you to expect?
– In forms of structure of the argument
– Textual, numeric, graphic
• Reflective Inquiry
– Is the author clear about the topic?
– Where do you think the authors got the ideas?
• Sources: journals, lit, calls for proposals, brainstorming,
recognizing a pattern
– Is the topic manageable?
• [Money, time, ethics, capabilities of researchers, interest,
data sources]
• Does this contribute to knowledge?
Reflective inquiry
• Procedures: research design &
collection methods
• Data analysis: gathering, processing,
and analyzing data
• Reliability, validity, credibility,
confirmability
• Presentation [how, where, when]
In the articles …
• Problem statement and hints why valuable?
– “The goal of the study is …”
– “There is little research considering …”
• Can you see what is to be covered and what
is not covered?
• Are there associated problems?
• Are the title and research question
“qualified”?
Problem statement +
literature
• Is the literature appropriate?
• BTW,why do we cite others?
– Homage
– Seminal works
– To correct or to criticize
– To dispute
– To verify
What frames the question
and research?
• Any theory you can identify?
– Any specific theory cited?
• What’s the purpose of theory?
– Any thoughts from Pettigrew?
– Your own?
– Helps to see patterns and principles to base
productive action
– Explains “why” - suggest possible actions and
facilitates broader interpretation
Objectives of the evaluation
• Are the objectives clear to you?
• Are the values appropriate?
– Example: employee review; LISSA “evals”
• Writing: active verbs instead of phrases
• Objects clear? (e.g., “satisfaction”,
“students”)
Quan vs Qual
• Can you identify whether the work is
qualitative [observing events from
individual perspective] or
• Quantitative? [quantification of study
parameters]
– E.g., if quantitative, can you ID the
hypothesis? (e.g., “It is expected that …”
– Your own examples?
Examples of research
approaches
•
•
•
•
•
•
•
Field studies/Ethnolography
Surveys
Experiments (control of variables)
Historical
Case Studies
Delphi
Content Analysis
Examples, con’t
•
•
•
•
•
•
Bibliometrics
Transaction analysis
Longitudinal studies
Meta analyses
Usability studies
User satisfaction
Questions & Hypotheses
• Hypotheses: based on comparing or
contrasting objectives
– May be positive or negative
– Must define concepts and variables!
– Brief statement
– Testable: hypothesis must be T or F
– Sample - not population
• generalization
Hypothesis example
• “As a result of the program, recidivism
among Hispanic males, 18-25, will
decrease by a small, but statistically
significant, rate.”
• Your examples?
Most LIS work is
descriptive, not hypothesis
driven
• Many library and archive situations are
descriptive - hence qualitative
• Why do you think that is?
• Here’s an example …
“The information needs of various groups, such as
faculty and students in academic institutions,
member of Congress, engineers, and the general
public have been investigated. No study, however,
has examined the information-seeking behavior of
female farmers and, more specifically, their use and
non-use of public libraries. Under what
circumstances do they turn to public libraries? What
types of information do they seek and for what
purposes? If they do not use any public libraries,
why not?”
Who or what is studied?
• Unit of analysis:
– Individual, group, organizations, social
artifacts
• What kind of data can be extracted from
these groups?
Methods
• Research Design: answers the
questions of the study
– How data will be collected
– What approach or approaches?
• How does the researcher decide?
Experimental approach
• Want to take some before-and-after
action (predictive or inferential)
– Take action (treatment or independent
variables)
– Observe the consequences (outcomes or
dependent variables)
• Even obvious terms must be
operationally defined (e.g., “service”,
“satisfaction”)
What kinds of problems are
associated with these
• Content analysis
techniques?
•
•
•
•
•
•
Observation
Interviews
Focus groups
Surveys
Historical Studies
Case Studies