Testing of hypothesis

Dr.L.Jeyaseelan

Dept. of Biostatistics Christian Medical College Vellore, India

Statistics

Inferential Statistics

Hypothesis testing

Comparison of means Comparison of proportions ( incidences / prevalences) Descriptive Statistics

Summarize mean / proportion

(incidence / prevalence)

Hypotheses Research Question

Is there a (statistically) significant difference between two groups with respect to the outcome?

Null Hypothesis

There is no (statistically) significant difference between two groups with respect to the outcome.

Alternative Hypothesis

There is a (statistically) significant difference between two groups with respect to the outcome.

Two groups – two independent populations Outcome – scores obtained Intervention – Educational training

P - Value

Probability of getting a result as extreme as or more extreme than the one observed when the null hypothesis is true.

When our study results in a probability of 0.01, we say that the likelihood of getting the difference we found by chance would be 1 in a 100 times.

It is unlikely that our results occurred by chance and the difference we found in the sample probably due to the teaching programme.

(i) Chance Variation (ii) Effect Variation Variation

The difference that we might find between the two groups’ exam achievement in our sample might have occurred by chance, or it might have occurred due to the teaching programme

‘P’ as a significance level

P < 0.05 result is statistically significant P > 0.05 result is not statistically significant.

These cutoffs are arbitrary & have no specific importance.

COMPARISON OF MEANS

t - tests

A bit of history...

W.A. Gassit (1905) first published a t-test. He worked at the Guiness Brewery in Dublin and published under the name Student. The test was called Studen

t Test

(later shortened to

t

test).

Types of t-tests

 One sample t-test  t-test for two independent (uncorrelated) samples (i) Equal variance (ii) Unequal variance  t-test for two paired (correlated) samples

Comparison of two independent Means

(Student’s t-test / unpaired t-test)

A t-test is used when we wish to compare two means Type of data required Independent Variable One nominal variable with two levels E.g., (i) boy/girl students; (ii) non-smoking/heavy smoking mothers Dependent Variable Continuous variable E.g., (i) marks obtained by the students in the annual exam; (ii) Birth weight of children

Assumptions

 The samples are random & independent of each other  The independent variable is categorical & contains only two levels  The distribution of dependent variable is normal. If the distribution is seriously skewed, the t-test may be invalid.

 The variances are equal in both the groups

Example data

A study was conducted to compare the birth weights of children born to 15 non-smoking with those of children born to 14 heavy smoking mothers.

Non-smoking Mothers (n = 15)

3.99

3.79

3.60

3.73

3.21

3.60

4.08

Heavy smoking Mothers (n = 14)

3.18

2.84

2.90

3.27

3.85

3.52

3.23

t  | x 1  x 2 | S 1 n 1  1 n 2 Where, 3.61

3.83

3.31

4.13

3.26

3.54

3.51

2.71

2.76

3.60

3.75

3.59

3.63

2.38

2.34

S 2  (n 1 )s 1 2  n 1  n (n  2 2 )s 2 2 2

Checking the Normality

Unequal Variances

Sometimes we wish to compare two groups of observations where the assumption of normality is reasonable, but the variability in the two groups are markedly different Two questions arise: (1)How different do the variances have to be before we should not use the two sample t-test?

(2)What can we do if this happens?

Unequal Variances – Contd..

(1)

Levene’s test for equality of variances

Null Hypothesis : The variances are equal Alternative Hypothesis : The variances are not equal If Levene’s test is not significant …. P>0.05

Report “equal variances assumed” If Levene’s test is significant ……... P<0.05

Report “equal variances not assumed” (2)

Use Modified t-test in the presence unequal variances

How to report the results?

Heavy smoking mothers (n=14) Mean SD Non-smoking mothers (n=15) Mean SD

Diff in means (95% CI) P-Value

Birth weight of children

3.20

0.49

3.60

0.37

0.4 (0.06 – 0.72) 0.022

The difference between birth weight of children born to non-smoking and heavy smoking mothers found by chance is only 2 in a 100 times.

The distribution of data

Normal data: SD < ½ mean Skewed / Non-normal data: SD > ½ mean use t-test use Non parametric Mann - Whitney test / log – transformed t-test Note: Applicable only for variables where negative values are impossible (e.g., Rate of GFR change) Ref: Altman DG, 1991

Clinical Significance Vs Statistical Significance

A possible antipyretic is tested in patients with the common cold.

500 receive the candidate drug 500 receive a placebo control Temperatures measured 4 hours after dosing

Drug Control N

500 500

Mean

39.950 40.058

StDev

0.653 0.699

SE Mean

0.029 0.031

p value = 0.011

Because the sample size is so large we are able to detect a very small change in temperature

Misuses of t-test

• t-test for non-normal data.

Hospital 1 Mean (SD) n Hospital 2 Mean (SD) n Length of Stay (in days)

26 (17) 11 Heterogeneous data – SD > ½ (mean) 79 (57) 13

Correct Method:

Non-parametric Mann-Whitney test with Median and Range values • t-test for paired observations

BP Levels Before intervention After intervention (n = 12) Mean SD Mean SD

142.0 30.5

120.5

31.5

Correct method:

Paired t-test

Misuses of t-test (Contd. ..)

• Multiple t-test Comparison of length of stays between three hospitals

Length of Stay (in days) Hospital 1 Mean (SD) n 25 (5) 12 Hospital 2 Mean (SD) n 75 (20) 13 Hospital 3 Mean (SD) n 30 (10) 14

Hospital 1 vs Hospital 2 Hospital 1 vs Hospital 3 P- value = ?

P- value = ?

Hospital 2 vs Hospital 3 P- value = ?

The effective p-value for 3 comparison is 3 x 0.05 = 0.15

Correct method:

ANOVA with Bonferroni correction.

Two groups of paired Observations

• • •

Paired t-test

Same individuals are studied more than once in different circumstances eg. Measurements made on the same people before and after intervention The outcome variable should be continuous The difference between pre - post measurements should be normally distributed

A study was carried to evaluate the effect of the new diet on weight loss. The study population consist of 12 people have used the diet for 2 months; their weights before and after the diet are given below.

Patient No.

8 9 10 11 12 1 2 3 4 5 6 7

Weight (Kgs) Before Diet After Diet

75 60 68 70 54 58 98 83 89 65 93 78 84 60 78 95 80 100 108 77 90 76 94 100 The research question asks whether the diet makes a difference?

Paired t test output

t- test

 To examine the difference between two independent groups

paired t-test

 To examine the difference between pre & post measures of the same group

How do we compare more than two groups means??

Example: Treatments: A, B, C & D Response : BP level How does t-test concept work here?

A versus B A versus C A versus D B versus C B versus D C versus D The rate of error increases exponentially by the number of tests conducted… 1-(1-0.05) 6 = 0.27

Instead of using a series of individual comparisons we examine the differences among the groups through an analysis that considers the variation across all groups at once.

Analysis of Variance (ANOVA)

WHY ANOVA not ANOME?

Although means are compared, the comparisons are made using estimate of variance. The ANOVA test statistic or F statistics are actually ratios of estimate of variance.

Hypotheses

The main analysis is to determine whether the population means are all equal. If there are K means then the null hypothesis is

H o

  1   2  ...

 

k

Alternative hypothesis is given by

H A

  1   2  ...

 

k

Type of data required

Independent Variable One nominal variable (>2 levels) E.g., Socio economic status (low / medium / high) Dependent Variable Continuous variable (normally distributed) E.g., hb level

Assumptions

 The samples are random & independent of each other  The independent variable is categorical & contains more than two levels  The distribution of dependent variable is normal. If the distribution is seriously skewed, the ANOVA may be invalid.

 The groups should have equal variances

Example data

A study was conducted to assess the hb levels of women in low, medium and high socio economic status SL No 1 2 3 4 5 6 7 8 9 10 Low (n = 20) 8.10

8.00

6.90

11.40

10.70

10.20

8.90

9.90

6.80

9.10

Medium (n = 18) 8.40

11.10

10.80

11.00

12.20

8.70

12.30

11.50

11.60

12.90

High (n = 17) 12.70

11.80

13.10

12.30

10.90

12.60

13.20

14.20

11.80

12.40

SL No 11 12 13 14 15 16 17 18 19 20 Low (n = 20) 9.20

7.40

10.70

11.40

7.70

6.10

11.00

11.10

7.90

10.60

Medium (n = 18) 12.00

10.90

11.70

11.00

12.20

11.20

10.70

9.90

High (n = 17) 12.70

13.40

14.30

13.80

15.00

14.20

9.20

Source of Variation

ANOVA separates the variation in all the data into two parts: The variation between the each group mean and the overall mean for all the groups (the between group variability) and the variation between each study participant and the participants group mean (the within-group variability).

If the between-group variability is much greater than the within-group variability, there are likely to be difference between the group means.

ANOVA data Group 1 Group 2 Group 3

ANOVA output

Multiple Comparisons procedure

ANOVA is a " group comparison " that determines whether a statistically significant difference exists somewhere among the groups studied. If a significant difference is indicated, ANOVA is usually followed by a " multiple comparison combinations of procedure groups to " that examine compares further any differences among them. The most common multiple comparison procedure is the " pairwise comparison ", in which each group mean is compared (two at a time) to all other group means to determine which groups differ significantly.

Bonferroni Test

Uses t tests to perform pairwise comparisons between group means, but controls overall error rate by setting the error rate for each test to the experiment wise error rate divided by the total number of tests.

Disadvantage with this procedure is that true overall level may be so much less than the maximum value ‘  ’ that none of individual tests are more likely to be rejected.

Tukey’s Method

Uses the studentized range statistic to make all of the pairwise comparisons between groups.Sets the experiment wise error rate at the error rate for the collection for all pairwise comparisons This method is applicable when 1.

2.

Size of the sample from each group are equal.

Pairwise comparisons of means are of primary interest that is Null hypothesis of the form.

to be considered.

Scheffé test

Performs simultaneous joint pairwise comparisons for all possible pairwise combinations of means. Uses the F sampling distribution.

This method is recommended when 1.

2.

The size of the samples selected from the different populations are unequal.

Comparisons other than simple pairwise comparison between two means are of interest.

Analysis of Covariance (ANCOVA)

Analysis of covariance

•     ANCOVA is an another ANOVA technique which combines the ANOVA with regression to measure the differences among group means The advantages that ANCOVA has over other techniques are: The ability to reduce the error variance in the outcome measure.

The ability to measure group differences after allowing for other differences between subjects. In ANOVA two sets of variables are involved in the analysis the independent and the dependent variable. With ANCOVA a third type of variable is included: the covariate which is continuous

Assumptions

1.

2.

3.

4.

5.

6.

The groups should be mutually exclusive.

The variance of the groups should be equivalent.

The dependent variable should be normally distributed.

The covariate should be a continuous variable.

The covariate and the dependent variable must show a linear relationship.

The direction and strength of relationship between the covariate and dependent variable must be similar in each group (homogeneity of regression across groups).

Steps for the analysis

   Check whether the dependent variable is normally distributed.

(Use rule of thump)

Sum chol

•  Test whether the variance of the dependent variable is similar across groups (Bartlett’s test for equal variances)

Oneway chol group, tabulate

•   Measure the correlation between cholesterol and age.

Corr chol age Twoway (scatter chol age)

Cont..

  Homogeneity of regression across groups is equivalent to testing interaction between the covariate and the independent variable.

Anova chol group age age*group, contin(age)

   If interaction is significant one could study the effect of age on cholesterol in each of the two groups separately.

If the interaction is not significant then the assumptions are met and it is appropriate to do ANCOVA.

anova chol group age age*group, contin(age)

Summary

 ANCOVA is an extension of ANOVA that allows us to remove additional sources of variation from the error term, thus enhancing the power of our analysis.

 ANCOVA Should be used only after careful consideration has been given to meeting the underlying assumptions.

 It is especially important to check for homogeneity of regression, because if that assumption is violated, ANCOVA can lead to improper interpretations of results.

Example

 In a survey to examine relationships between the nutrition and the health of women in middle west, the concentration of cholesterol in the blood serum was determined on 56 randomly selected subjects of Iowa and 130 in Nebraska  After controlling for age, do the two groups (Iowa, Nebraska) differ significantly on the cholesterol levels?

Dataset

ANOVA without adjusting for age

Testing Homogeneity of Variances across groups

Measuring the correlation between cholesterol and age

Correlations between the dependent variable and the covariate

Testing Homogeneity of regression across groups

Testing the homogeneity of regression across groups

Model shows that the interaction term is not significant (Assumption is met)

The Interaction term is eliminated from the model (Full Factorial model)

The ANCOVA results

Interpretation of the findings

 After controlling for the covariate age the two groups, (IOWA and Nebraska) do not differ significantly in their cholesterol levels.

 Note that the error variance was very high when age is not adjusted in the model