Transcript Evaluation Research

Evaluation Research
COMT 502
topics
• Evaluation research
– Process of determining whether intervention
has its intended result
Formulating the problem
• Specifying outcomes
– Response variable
• Measuring relevant contexts
– Aspects of context which might affect outcome
• Specifying intervention
– May need to measure extent or quality of participation
or exposure
• If differences in outcome among treatment group, need to
figure out why program worked better for some than others
Measurement
• Specifying population
– Monitor relative severity of subjects’ problems,
risk behavior, etc.
• New vs. existing measures
–
–
–
–
Pros & cons
Body of possible comparisons
Evidence
Reliability & validity
Operationalizing
• How much of an effect is considered to be a
success when measuring a program?
Types of designs
• Experimental
• Quasi-experimental
– Lack random assignment
• Time-series design
– Measurements taken over time
– Only one group
• Non-equivalent control groups
– Existing, similar “control” group used…not exactly equal
– Better than no comparison
• Multiple time-series design
– Non-equivalent control w/ time-series data
– Can look at how control groups changed
Designs
• Qualitative evaluations
– In-depth interviews
– Content-analysis
– Focus groups
• In-depth understanding
• Combo is ideal
Social context
• Logistical problems
• Ethical issues
• Use of results
– People don’t understand
– Contradicts deeply held beliefs
– People have vested interests
Social indicators
• Aggregated statistics that reflect social
condition of society or subgroup
– Used to monitor aspects of social life (e.g.,
GNP)
– Used for comparison across groups, over time
– www.socio.com