S519: Evaluation of Information Systems

Social Statistics Inferential Statistics Chapter 11: ANOVA

This week

 When to use F statstic  How to compute and interpret  Using FTEST and FDIST functions  How to use the ANOVA

The problem with t tests…

 We could compare three groups with multiple ttests: M1 vs. M2, M1 vs. M3, M2 vs. M3

What is ANOVA?

 “

An

alysis

o

f

Va

riance”  A hypothesis-testing procedure used to evaluate mean differences between two or more treatments (or populations).

 Related to: t-tests using independent-measures or repeated- measures design.

 Advantages:   1) Can work with more than two samples.

2) Can work with more than one independent variable

What is ANOVA?

 In ANOVA an independent or quasi independent variable is called a factor.

 Factor = independent (or quasi-independent) variable.

   Levels = number of values used for the independent variable.

One factor → “single-factor design” More than one factor → “factorial design”

What is ANOVA?

 An example of a single-factor design  A example of a two-factor design

F value

 Variance between treatments can have two interpretations:  Variance is due to differences between treatments.

 Variance is due to chance alone. This may be due to individual differences or experimental error.

Three Types of ANOVA

 Independent measures design: Groups are samples of independent measurements (different people)  Dependent measures design: Groups are samples of dependent measurements (usually same people at different times; also matched samples) “Repeated measures”  Factorial ANOVA (more than one factor)

Excel: ANOVA

 Three different ANOVA:    Anova: single factor - independent Anova: two factors with replication - factorial Anova: two factors without replication - dependent

Example (independent)

 Three groups of preschoolers and their language scores, whether they are overall different?

Group 1 Scores 87 Group 2 Scores 86 76 56 78 98 77 66 75 67 87 Group 3 Scores 89 85 99 91 96 85 79 81 82 87 89 90 89 78 85 91 96 96 93

F test steps

 Step1: a statement of the null and research hypothesis  One-tailed or two-tailed (there is no such thing in ANOVA)

H

0 :  1   2   3

H

1

: at least one



is different

F test steps

 Step2: Setting the level of risk (or the level of significance or Type I error) associated with the null hypothesis  0.05

F test steps

 Step3: Selection of the appropriate test statistics   See Figure 11.1 (S-p227) Simple ANOVA (independent)

F test steps

 Between-group degree of freedom=k-1  k: number of groups  Within-group degree of freedom=N-k  N: total sample size

F test steps

 Step4: determination of the value needed for rejection of the null hypothesis using the appropriate table of critical values for the particular statistic    Table B3 (S-p363) df for the denominator = n-k=30-3=27 df for the numerator = k-1=3-1=2

F test steps

 Step5: comparison of the obtained value and the critical value    If obtained value > the critical value, reject the null hypothesis If obtained value < the critical value, accept the null hypothesis 8.80 and 3.36

F test steps

 Step6 and 7: decision time  What is your conclusion? Why?

 How do you interpret F (2, 27) =8.80, p<0.05

Example (dependent)

Five participants took a series of test on a new drug

P1 P2 P3 P4 P5 2 0 0 T1 3 0 1 1 1 T2 4 3 4 3 4 T3 6 3 5 4 3 T4 7 6

F test steps

 Step1: a statement of the null and research hypothesis  One-tailed or two-tailed (there is no such thing in ANOVA)

H

0 :  1   2   3   4

H

1

: at least one



is different

F test steps

 Step2: Setting the level of risk (or the level of significance or Type I error) associated with the null hypothesis  0.05

F test steps

 Step3: Selection of the appropriate test statistics   See Figure 11.1 (S-p227) Simple ANOVA (independent)

F test steps

 Between-group degree of freedom=k-1  k: number of groups  Within-group degree of freedom=N-k  N: total sample size  Between-subject degree of freedom=n-1  n: number of subjects  Error degree of freedom=(N-k)-(n-1)

F test steps

 Step4: determination of the value needed for rejection of the null hypothesis using the appropriate table of critical values for the particular statistic    Table B3 (S-p363) df for the denominator = (N-k)-(n-1)=16-4=12 df for the numerator = k-1=4-1=3

F test steps

 Step5: comparison of the obtained value and the critical value    If obtained value > the critical value, reject the null hypothesis If obtained value < the critical value, accept the null hypothesis 24.88 and 3.49

F test steps

 Step6 and 7: decision time  What is your conclusion? Why?

 How do you interpret F (3, 12) =24.88, p<0.05

Factorial ANOAVA

 Next week