Transcript Test on Pairs of Means – Case I

Bootstrap Test for Pairs of Means of a Non-Normal
Population – small samples
• Suppose X1, …, Xn are iid from some distribution independent of
Y1, …, Ym are iid from another distribution. Further suppose that both n
and m are small and we are interested in testing whether the two
populations have the same means.
• Can use the t-test (pooled or unpooled) since it is robust as long as
there are no extreme 1outliers and skewness.
• Alternatively, we can use bootstrap hypothesis testing.
STA248 week 12
1
Bootstrap Hypothesis Testing - Introduction
• Suppose X1, …, Xn is a random sample of size n, independent
from another random sample Y1, …, Ym of size m. and we wish
to test H 0 :  x   y vs H a :  x   y.
• As a test statistics we will use V  X  Y .
• The P-values of this test is PV  x  y | H 0 is true .
• We want the bootstrap estimate of this P-value.
STA248 week 12
2
Bootstrap Test Procedure
• To obtain the bootstrap estimate of the P-value we need to generate
samples with H0 true.
• One way of doing this (assuming X and Y have same distribution) is
to combine 2 samples into 1 of size n+m.
• Then re-sample with replacement from this combined sample such
that each re-sampling has two groups …
• For each bootstrap sample calculate the bootstrap estimate of the test
statistics v j  x *j  y *j , j = 1, …, B.
• The bootstrap estimate of the P-value is ….
STA248 week 12
3
Example
STA248 week 12
4
Data Collection
• There are three main methods for collecting data.
 Observational studies
 Sample survey
 Planned / designed experiments
• These methods differ in the strength of conclusion that can be
drawn.
STA248 week 12
5
Observational Studies
• In some cases, a study may be undertaken retrospectively.
• In observational studies we simply collect information about
variables of interest without applying any intervention or controlling
for any factors.
• When factors are not controlled we are not able to infer a causeeffect relationship.
• Other problems with observation studies are:
 Confounding – can’t separate effect of one variable from another.
 Lack of generalization.
STA248 week 12
6
Sample Surveys
•
•
•
•
•
•
•
•
Sample surveys are observational in nature.
Surveys require existence of physically real population.
Data is collected on a random sample from the target population.
Survey design includes selection of sample so it is representative of
the population as a whole.
Use statistics to make inference about entire population.
Confounding is still a problem. However, the results can be
generalized to the population.
Cause of any observed differences cannot be determined.
To allow generalization and to avoid bias – sample must be chosen
randomly e.g., SRS.
STA248 week 12
7
Planned / Designed Experiments
• There are few key features of designed experiments that distinguish
it from any other type of study.
• Independent variables of interest are carefully controlled by the
experimenter in order to determine their effect on a response
(dependent) variable.
• Researcher randomly assign a treatment to the subjects or
experimental units.
• Control of independent variables and randomization make it possible
to infer cause and effect relationship.
• Use of replication – multiple observation per treatment.
• Replication allows measurement of variability.
STA248 week 12
8
• Treatments are sometimes called predictor variables and sometimes
called “factors”.
• The values of a factor are its “levels”.
• A design is balanced if each treatment has the same number of
experimental units.
• Problem: can’t always carry out an experiment.
STA248 week 10
9
Randomization
• The use of randomization to allocate treatments to experimental
units (or vice versa) is the key element of well-designed experiment.
• Random allocation tends to produce subgroups which are
comparable with respect to the variables known to influence the
response.
• Randomization ensures that no bias is introduced in allocation of
treatments to experimental units.
• Randomization reduces the possibility that factors not included in
the design will be confounded with treatment.
STA248 week 12
10
Cautions Regarding Experiments
• “Effective sample size” – all statistical techniques we have learned
assume observations are independent. If they are not but treated as if
they were, get more power and smaller CI than you should.
• “Fishing expedition” – if doing 100 tests at α = 0.05 significant
level, expect 5 of 100 tests to show significant differences from H0
even when H0 is always true (type I errors).
STA248 week 10
11
Controlling for Type I error
• One widely use method for controlling for type I error uses
Bonferoni Inequality….
• If Ai is the event that the ith test has a type I error, and typically
P(Ai ) = α, then by Bonferoni Inequality we that: ..
That is the probability of committing at least one type I error in k
tests is at most kα.
• Therefore, if use significant level of α/k for each individual test, then
the “overall significant level” (P(at least 1 type I error)) is at most α.
• The Bonferoni method is very conserevative.
STA248 week 10
12
Analysis of Variance – Introduction
• Generalization of the two sample t-procedures (with equal
variances).
• The objective in analysis of variance is to determine whether there
are differences in means of more than 2 groups.
• The statistical methodology for comparing several means is called
analysis of variance, or simply ANOVA.
• When studying the effect of one factor only on the response we use
one-way ANOVA to analyze the data.
• When studying the effect of two factors on the response we use twoway ANOVA.
STA248 week 10
13
One-Way ANOVA model
• The response variable Y is measured on each experimental unit in
each treatment group. Measure Yij for the jth subject in the ith group.
• The one-way ANOVA model is:
Yij = μi + εij for i = 1, 2,…, k and j = 1, 2, …, ni.
• μi is the unknown mean response for the ith group.
• The εij are called “random errors” and are assumed to be i.i.d
N(0, σ2).
• The parameters of the model are the population means μ1, μ2,…, μk
and the common standard deviation σ.
• The objective of one-way ANOVA is to test whether the mean
response in each treatment group is the same.
• The null and alternative hypotheses are….
STA248 week 10
14
Derivation of Test Statistics
STA248 week 10
15