Transcript Causal-Comparative Studies

Educational Research:
Causal-Comparative Studies
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.
Research...

The systematic application of a
family of methods employed to
provide trustworthy information
about problems
…an ongoing process based on many
accumulated understandings and
explanations that, when taken together
lead to generalizations about problems
and the development of theories
The basic steps of research...
Scientific and disciplined inquiry is an
orderly process, involving:
 recognition and identification of a
topic to be studied (“problem”)
 description and execution of
procedures to collection information
(“method”)
 objective data analysis
 statement of findings (“results”)
Research methods...
Quantitative…
…collects and analyzes numerical data
obtained from formal instruments
Quantitative methods...
descriptive research (“survey research”)
 correlational research
 causal-comparative research
(“ex post facto research”)
 experimental research


causal-comparative research (“ex post
facto research”)
…at least two different groups are
compared on a dependent variable
or measure of performance (called
the “effect”) because the
independent variable (called the
“cause”) has already occurred or
cannot be manipulated
Research variables...
Independent…
…an activity of characteristic
believed to make a difference
with respect to some behavior
…(syn.) experimental variable,
cause, treatment
dependent variable…
…the change or difference occurring
as a result of the independent
variable
…(syn.) criterion variable, effect,
outcome, posttest
A causal-comparative study…
…a study in which the researcher
attempts to determine the cause, or
reason, for pre-existing differences in
groups of individuals
…called an “ex post facto” study
because both the effect and the
alleged cause have already occurred
and must be studied in retrospect
Differences in causal-comparison
and correlational studies…

causal-comparative studies…
…attempt to identify cause-effect
relationships

correlational studies…
…attempt to identify relationships

causal-comparative studies…
…involve two (or more) groups and one
independent variable

correlational studies…
…typically involve two (or more)
variables and one group

causal-comparative studies…
…involve making comparisons

correlational studies…
…involve establishing relationships
Differences in causal-comparison
and experimental studies…

causal-comparative studies…
…individuals are not randomly selected but
selected because they belong to groups

experimental studies…
…individuals are randomly selected and
assigned to two (or more) groups

causal-comparative studies…
…the researcher cannot manipulate the
independent variable

experimental studies…
…the researcher manipulates the
independent variable

causal-comparative studies…
…the independent variable has already
occurred and cannot be manipulated

experimental studies…
…the researcher manipulates the
independent variable to determine its
effects

causal-comparative studies…
…the random sample is selected from two
already-existing populations

experimental studies…
…the random sample is selected from a
single population
Conducting a causal-comparative
study…
1. select the problem
2. select participants and instrument
3. design and procedure
4. data analysis and interpretation
1. select the problem…
…the researcher starts with an effect
and seeks its causes
…the independent variable cannot or
should not be manipulated
2. select the participants and instrument…
…select samples representative of
their respective populations and
similar with respect to critical
variables other than the independent
variable
…called “comparison groups”
3. design and procedure…
…the performance of the groups is
compared using some valid
dependent variable measure
(“instrument”)
…lack of randomization, manipulation,
and control are sources of weakness

control
…the process by which the researcher
attempts to ensure that the findings
are as free of researcher bias and
error as possible

types of control
…random assignment of participants to
groups
…pair-wise matching
…comparing homogeneous groups
…comparing homogeneous subgroups
…factorial analysis of variance
…analysis of covariance

random assignment of participants to
groups
…not possible in causal-comparative
studies because the groups already
exist and have already received the
treatment

pair-wise matching
…first: find a participant in the second (third,
fourth, etc.) group with the same or similar
score on the control (nonmanipulated)
variable as the participant in the first group
…second: if a participant in either group
does not have a suitable match, the
participant is eliminated from the study

comparing homogeneous groups
…control for extraneous variables that
are homogeneous with respect to the
extraneous variables
…limitation: lowers the number of
participants in the study and, of
course, limits the generalizability of
the findings

comparing homogeneous subgroups
…form subgroups within each group that
represent all levels of the control
(nonmanipulated) variable
…controls for the variable and also permits
the researcher to determine whether the
independent variable affects the
dependent variable differently at different
levels of the control (nonmanipulated)
variable

factorial analysis of variance (“FANOVA”)
…building the control (nonmanipulated)
variable into the research design
…then use FANOVA to analyze the results
to determine the effect of the independent
and control (nonmanipulated) variable on
the dependent variable, both separately
and in combination
…FANOVA allows the researcher to
determine if there is an interaction
between the independent variable
and the dependent variable such that
the independent variable operates
differently at different levels of the
independent variable building it into
the research design

analysis of covariance (“ANCOVA”)
…statistically adjusts initial group
differences on a dependent variable for
initial differences on some other variable
related to performance on the dependent
variable
…removes initial differences so that the
results can be fairly compared as if the two
groups started equally

symbolic representation of the basic
causal-comparative design
Group
(E)
(C)
Independent
Variable
(X)
Dependent
Variable
O
O
Where: E (experimental group); C (control group);
X (independent variable); O (dependent variable)
Group
(E)
(C)
Independent
Variable
(X1)
(X2)
Dependent
Variable
O
O
Where: E (experimental group); C (control group);
X (independent variable); O (dependent variable)
4. Data analysis and interpretation…
…researcher uses a variety of
descriptive and inferential statistics:
t-test
mean
analysis of
standard
variance
deviation
chi squared

mean
…the descriptive statistic indicating the
average performance of an individual
or group on a measure of some
variable

standard deviation
…the descriptive statistic indicating the
spread of a set of scores around the
mean

t-test
…the inferential statistic indicating
whether the means of two groups are
significantly different from one
another

analysis of variance (“ANOVA”)
…the inferential statistic indicating the
presence of a significant difference
among the means of three or more
groups

chi squared (Χ2)
…the inferential statistic indicating that
there is a greater than expected
difference among group frequencies
Mini-Quiz…

True and false…
…causal-comparative studies
attempt to identify the causeeffect relationships; correlational
studies do not
True
…causal-comparative studies typically
involve two (or more) groups and one
independent variable, whereas
correlational studies typically involve
two (or more) variables and one group
True
…causal-comparative studies involve
relation, whereas correlational
studies involve cause
False
…oftentimes, causal-comparative
research is undertaken because the
independent variable could be
manipulated but should not
True
…one of the most important reasons
for conducting causal-comparative
research is to identify variables
worthy of experimental investigation
True
…“lack of control” means that the
researcher can and should
manipulate the independent variable
False
…each group in a causal-comparative
study represents a different
population
True
…the more similar two groups are on
all relevant variables except the
independent variable, the stronger
the study is
True
…there is random assignment to
treatment groups from a single
population in causal-comparative
studies
False
…lack of randomization, manipulation
of the independent variable, and
control are all sources of weakness
in a causal-comparative design
True
…matching, comparing homogenous
groups or subgroups, and covariate
analysis are strategies that enable
researchers to overcome problems
of initial group differences on an
extraneous variable
True
…interpretation of the findings in a
causal-comparative study requires
considerable caution because the
cause may be the effect and the
effect may be the cause
True
…extraneous variables or confounding
factors may be the real “cause” of
both the independent and dependent
variables
True

Fill in the blank…
…groups selected for a causalcomparative study which differ on
some independent variable and
comparing them on some dependent
variable
comparison groups

Fill in the blank…
…unexplained variables that influence
a dependent variable
confounding factors
extraneous variables

Fill in the blank…
…a method for controlling extraneous
variables by comparing groups that
are homogeneous with respect to
the extraneous variable
comparing homogeneous groups

Fill in the blank…
…a method for controlling extraneous
variables by forming subgroups
within each group that represent all
levels of the control variable
comparing homogeneous subgroups

Fill in the blank…
…a statistical tool to determine the
effects of the independent variable
and the control variable on the
dependent variable, both separately
and in combination
factorial analysis of variance

Fill in the blank…
…a statistical tool to adjust initial
group differences on variables
analysis of covariance

Fill in the blank…
…the descriptive statistic indicating
the average performance of a group
on a measure of some variable
mean

Fill in the blank…
…the descriptive statistic indicating
how clustered or spread out around
the mean a set of scores is
standard deviation

Fill in the blank…
…the inferential statistic determining
whether there is a significant
difference between the means of two
groups
t-test

Fill in the blank…
…the inferential statistic determining
whether there is a significant
difference between the means of
three or more groups
analysis of variance

Fill in the blank…
…the inferential statistic determining
whether there is a greater than
expected difference among group
frequencies
chi squared

Fill in the blank…
…activities by which a researcher
endeavors to ensure that the results
of a causal-comparative study are
not tainted by extraneous variables
control
This module has focused on...
causal-comparative studies
…which identify the cause, or
reason, for existing differences in
the behavior or status of groups
The next module will focus on...
experimental studies
...which test hypotheses to establish
cause-and-effect relationships