Transcript Hypothesis Testing – The 7

STAT 3130
Statistical Methods I
Session 3
ANOVA POST HOC Testing
ANOVA – Post Hoc Testing
From the previous notes, we found an F-statistic of 24.33. This
equated to a very low p-value (<.001), allowing us to reject the Null
Hypothesis and conclude that a difference amongst the means
was present…
This does not really guide our decision making – although we
concluded that at least one of the plans is different, at this point
we have no information regarding WHICH plan(s) is (are) different.
To determine this, we need to execute a Post Hoc test.
ANOVA – Post Hoc Testing
Post Hoc tests are a critical step in the ANOVA process because
they allow us to determine WHICH plans are different from the
others.
There are several possible Post Hoc tests to choose from based
upon the specifics of your analysis. We will look at the two most
common: Fishers LSD and Tukey.
<Other post hoc tests which you may hear about include Duncan,
Scheffe’, Bonferroni, Dunnett, Sidak, Waller, Welch>
ANOVA – Post Hoc Testing
Both Fisher’s LSD test and Tukey’s test are commonly accepted
tests amongst statisticians. You really cannot go too far wrong with
either. Here are a few general points of comparison:
1. Fisher’s LSD (least significant difference) is less conservative than
Tukey’s HSD (honestly significant difference).
2. Fisher’s LSD executes a series of ttests amongst the pairs,
resulting in a higher Type 1 error rate, while Tukey’s HSD specifies
a “family error rate” across all comparisons
3. Tukey’s HSD calculates “Tukey’s Yardstick” – any mean
differences which are at least as large as the Yardstick are
considered to be different.
ANOVA – Post Hoc Testing
Additional sources for detailed information
regarding Post Hoc tests:
http://support.sas.com/documentation/cdl/en/statug/63347/HTML/default/viewer.htm#statug_glm_a0000000867.htm
http://www.uwsp.edu/psych/cw/statmanual/posthocs.html
http://pages.uoregon.edu/stevensj/posthoc.pdf
http://practicalstats.labanca.net/index.php?title=Selecting_a_Post_Hoc_test