Transcript Experimental Research - University of Puget Sound

Experimental Research
Validity and Confounds
What is it?

Systematic inquiry that is
characterized by:

An investigator’s direct manipulation
(variation) of some factor or factors (IV or
IV’s) thought to be causally related to some
outcome or outcomes (DV or DV’s) which are
observed as data while holding all other
factors constant.
Independent Variable

The independent variable: conditions in the
experiment






Different situations (rooms, experiences, etc)
Different instructions
Different tasks
Different treatments
Experimental vs. control groups
Bystander study: Manipulate the number of
bystanders
Essential Features of Experiments

Independent Variable



Types?
Control group
Dependent variable
Manipulated vs. Subject Variable
Subject variables are characteristics that you cannot
manipulate: gender, choosing subjects with high or low anxiety.

Subject variable studies also called “natural groups studies”

Manipulated variables true experimental designs

Create groups where you place/force subjects to be in one group or
another


Create anxiety for ½ of the group.
Limitation with a natural groups study can’t draw causal
conclusions because you can’t rule out other possible factors that
might be the “true” causal factor.


Example: gender
Extraneous variables vs. Confounds
Extraneous Variable: uncontrolled,
unsystematic factors or variance. Can be ANY
variable that is related to the IV


Creates “noise” and may affect the outcome
variable, just not systematically.
Confounding variable: uncontrolled,
systematic variance


Affects the different levels of the IV differently.
2 things to control for confounds:



Random assignment
Within Subjects designs
Internal vs External Validity
External validity: the extent to which the
results of ones study can be generalized to
subjects and/or conditions outside of the
study. Often done by replication studies

Internal validity: found when results are
characterized by the ABSENCE of competing
explanations. You are confident that changes
in the IV did indeed lead to changes in the DV.




Experimental control is the key to internal validity
An absence of confounds and extraneous variables
Control groups help establish internal validity
Threats to Internal Validity
Post test study



Treatmentobservation
Ritalin and ADHD
Pre test-post test study


Observation treatmentobservationOffers a baseline measurement from which
we can compare
Threats to Internal Validity
In pre test-post test studies we find..
History and maturation

History: Something external happens and affects changes in your
independent variable
maturation: external to your study, but internal to your subjects.
Development or experience


Regression to the mean

When people are chosen because they are ‘extreme’ in some way and you
measure their change over time in that extreme dimension
you’re likely to see a change in the score closer to the mean.


Instrumentation and testing

Change in the researchers (or subjects) instrumentation or measuring over
time.



Testing: when actually taking the test affects the behavior or helps “practice”.
IDU study
Instrumentation: changes in the instrument used to measure from pre to post.
Controlling Against TtIV

Use a Control Group
Observation Treatment Observation
AND
 Observation---------------- Observation
OR
 Observation Placebo--- Observation


But, what if these threats don’t affect both
groups equally?
Example

Want to know the sexual assault opinions of
UWT students
Group 1: Wild life video
2 weeks later…
 Group 2: Assault video
But during those 2 weeks there was a string of sexual
assaults on campus
 What kind of threat?
 Solution?

Other Threats
Other threats come from HOW a subject is chosen for
the study and how comparison groups are formed

Subject selection- if subjects chosen for one group
are different than those in another group before the
study takes place.


Solution: Random assignment of subjects into groups
Attrition/subject mortality- When subjects drop
out or refuse to take part in your experiment.


Possible solution: try to interview the people who dropped out
to find out why and see if that is related to the IV