Fall, 2008

Psych 5500/6500

Confounding Variables 1

Confounding Variables

We are running a t test in an attempt to establish a cause and effect relationship between the independent and dependent variables. If we reject H0, we are saying that the difference in the group means is statistically significant, i.e. that the difference is larger than would be expected if only random error were involved. We would then like to conclude that it was the independent variable that made the groups means different. We can only conclude this, however, if we have no

confounding variables

in our experiment.

2

Confounding (cont.)

A

confounding variable

is a variable other than our independent variable (and other than random chance) that could have caused the group means to be different.

For example: I.V.=gender, D.V.=respiration rate. We run the experiment and find that the difference between the two groups is statistically significant. We would like to conclude that gender affects respiration rate. But what if we measured the men in the morning and the women in the evening, then ‘time of day’ could account for the significant difference between the groups, ‘time of day’ would be a ‘confounding variable’.

3

Confounding (cont.)

If we have a confounding variable, and we reject H0, then we do not know whether it was our independent variable or the confounding variable that caused the group means to differ. If we do not have a serious confounding variable then when we reject H0 we can conclude that it was our independent variable that caused the group means to differ 4

Handling Confounding Variables

Handling confounding variables is an important topic in experimental design, the important point in a statistics class is to realize that rejecting H0 only indicates that the difference between the means was unlikely if only chance were involved. To then conclude that the independent variable had an effect is only reasonable if there are no serious confounding variables. Even though this is more a topic for an experimental design class, I think it would behoove us to take a quick look at how to handle confounding variables. We’ll look at four possibilities: 5

Handling Confounding Variables

1) Pretest to see if there is a problem: for example, before you run your experiment you could run a quick pretest to see if time of day influences respiration, if you find it does not then don’t worry about time of day being a confounding variable in your experiment.

6

Handling Confounding Variables

2) Control the variable: if time of day is a confounding variable then design the study

so that it effects both groups equally,

in this way you are

controlling

the confounding variable. For example, run both groups at the same time, or test one male and one female at 8:00 in the morning, one male and one female at 8:30 and so on.

7

Control and Treatment Groups

Often the experimenter wants to examine the effect of some manipulation (treatment). To do so two groups are created, the ‘control’ group that does not get the treatment and the ‘treatment’ group that does. The goal is to control all of the possible serious confounding variables, so that the only difference between the control group and the treatment group is the treatment itself.

8

Handling Confounding Variables

3) Randomize the variable: sometimes you may not be able to control the confounding variable. If this is the case, then arrange for its effect to be randomly divided among the participants. For example, randomly determine what time of day each participant is measured. The confounding variable might still effect the data (e.g. by luck more males may be measured in the morning than females) but at least its effect will be random, and will have an equal chance of contradicting your theory’s prediction as supporting it. Randomly assigning subjects to groups randomizes the effect of individual differences.

9

Handling Confounding Variables

4) Include the confounding variable in your model: there are a lot of advantages to looking at how both time of day

and

gender affect respiration (rather than trying to get rid of time of day so you can just look at gender). We will be covering this next semester when we get to the ‘Model Comparison Approach’.

10

Experimental Designs

The t test for independent groups can be used in three different types of experimental designs. Some of these designs introduce additional confounding variables.

1. True Experimental Design 2. Quasi-Experimental Design 3. Static Group Design 11

True Experimental Design

Attributes: 1. Subjects are randomly divided into groups.

2. The independent variable is manipulated by the experimenter If performed correctly, then the only reason the groups should differ before the IV is applied is because of random chance. This design may still have confounding variables (introduced by the how you manipulate the independent variable), but it tends to have fewer confounding variables than the other two designs.

12

Quasi-Experimental Design

Attributes: 1. Subjects are not randomly divided into groups, they are in pre-existing groups that you hope are essentially the same (i.e. as if you had randomly divided into groups).

2. Independent variable manipulated by the experimenter.

This design is used when you would like to do a true experimental design but practical reasons make that impossible. Additional confounding variables are introduced by possible pre-existing, non-random, differences between the groups. Possible solutions: search for groups that are equal, test for possible confounding variables.

13

Quasi-Experimental Design (cont.) For more information on quasi-experimental designs you may want to read the influential work of Campbell. You could start with: Campbell, Donald T. (1969). Reforms as experiments.

American Psychologist, 24

, 409-429. In looking for what he has done recently I ran across the following (which I haven’t read) but looks good for light summer reading (seriously, looks good if you really want to get into the topic of establishing cause and effect with experiments).

Shadish, W. R., Cook, T. D., Campbell, D. T. (2002)

Experimental and quasi-experimental designs for generalized causal inference.

Boston: Houghton Mifflin and Company. 623 pp.

14

Static Group Design

Attributes: 1. Subjects are not randomly divided into groups, the groups are pre-existing.

2. The independent variable is not manipulated by the experimenter, instead it is the variable you use to define the two groups.

In this design you are explicitly looking for pre-existing differences between the groups (which is your I.V.). In the gender example the I.V. (gender) was what you used to divide participants into the two groups. Additional confounding variables are pre-existing differences other than your I.V. (i.e. other variables that systematically differ along with the I.V., such as height differences that occur when you divide participants based upon gender).

15

Static Group Design (cont.)

The static group design is considered to be a correlational design as the independent variable is not manipulated. It shows up here, however, rather than in the latter lecture on correlation as the focus here is on comparing the two group means, rather than upon a correlation between two variables.

16