Research supported by TLRI

Randomisation: part 2

Randomisation S1

Study design and inference Experiments & the Randomisation Test: •

The difference between two means

Watch out for: • The

‘chance is acting alone’

explanation •

How

we

assess the plausibility

of the ‘chance alone’ explanation – (test for ‘chance alone’) Randomisation S2

What does ‘chance alone’ look like?

iNZightVIT

Randomisation

Chapter 1

The Walking Babies Experiment

Does a special exercise programme lower walking age?

Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315 10 male infants (& parents) were randomly assigned to one of two treatment groups.

First walked without support: Treatment Exercise Control 9 13.25

9.5

Age (months) 9.75

10 11.5

12 13.5

11 11.5

Randomisation S4

The Walking Babies Experiment

9 10 11 12 Age (months) 13 14 Randomisation S5

Is chance alone likely to generate differences as big as our difference?

Re-randomisation distribution of differences (under chance alone) Tail proportion: roughly ____%

• • •

The Walking Babies Experiment

Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the exercise has no effect) We can

rule out

‘chance is acting alone’ as a

plausible

explanation for the difference between the two groups.

We have evidence

against

‘chance is acting alone’ We have evidence that

chance is not acting alone

Randomisation S7

The Walking Babies Experiment

Possible explanation:

If chance is not acting alone, then what else

•

difference?

(the exercise has no effect) We can

rule out

‘chance is acting alone’ as a

plausible

• •

We have evidence

against

‘chance is acting alone’ We have evidence that

chance is not acting alone

Randomisation S8

The Walking Babies Experiment Conclusion:

Because the male infants (& parents) were

randomly assigned

to the groups, we may

claim

that the

exercise was effective

in lowering the walking age.

Because these subjects in this experiment were volunteers (

not randomly selected

), then we would need to consider carefully as to which wider group(s) this conclusion may apply.

Randomisation S9

Two types of Inference

There are two types of inference 1. Sample-to-population eg x = 172cm so the population mean is about 172cm.

2. Experiment-to-causation eg The treatment was effective Randomisation S10

The Walking Babies Experiment

Does a special exercise programme lower walking age?

Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn” Science, Vol. 176 (1972), pp314-315 10 male infants (& parents) randomly assigned to either the Exercise group or the Control exercise group.

First walked without support: Treatment Exercise Control exercise 9 11 9.5

Age (months) 9.75

10 10 15 11.75

13 10.5

Randomisation S11

The Walking Babies Experiment

9 10 11 12 Age (months) 13 14 15 Randomisation S12

The Walking Babies Experiment

Possible explanation: One possible explanation for the observed difference between these two groups:

Chance is acting alone

(the exercise has no effect) Randomisation S13

The Walking Babies Experiment

Possible explanation: One possible explanation for the observed difference between these two groups:

Chance is acting alone

(the exercise has no effect) • Is the ‘Chance alone’ explanation simply not plausible?

Would our observed difference be unlikely when chance is acting alone? • How do we determine whether a difference is unlikely when chance is acting alone?

• See what’s

likely

and what’s

unlikely

acting alone when chance is

Randomisation S14

Our observed difference = 1.4 months Is chance alone likely to generate differences as big as our difference?

Re-randomisation distribution of differences (under chance alone) Tail proportion: roughly ___/1000 (____%)

• • •

Do the actual exercises lower the walking age?

Our tail proportion of ____

/1000 = ___%

means:

___ times

out-of-a-1000 times we get a difference of 1.4 months or more, when chance is acting alone. Under chance alone, it’s not unusual to get a difference bigger than or equal to our observed difference of 1.4 months .

A difference of 1.4 months or greater is not unusual when chance is acting alone, . . . therefore

chance could be acting alone.

Randomisation S16

The Walking Babies Experiment

Possible explanation: One possible explanation for the observed difference between these two groups: Chance is acting alone (the treatment has no effect) • • • We can

NOT rule out

‘chance is acting alone’ as a

plausible

explanation for the observed difference between the two groups.

We have

no

evidence

against

‘chance-is-acting-alone’ BUT something else, as well as chance,

COULD

ALSO be acting.

Randomisation S17

The Walking Babies Experiment

Possible explanation: • Chance is acting alone (the treatment has no effect)

observed difference?

• • We can

NOT rule out

‘chance is acting alone’ as a

Remember:

explanation for the observed difference between the two groups.

different treatment.

We have

no

evidence

against

‘chance-is-acting-alone’ Chance could be acting alone . . . BUT something else, as well as chance,

COULD

ALSO be acting.

Randomisation S18

The Walking Babies Experiment

• • Conclusion: Because the male infants (& parents) were

randomly assigned

to the groups, we conclude that the observed difference is the result of EITHER

chance acting alone

OR

an exercise effect together with chance acting

– we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one. Randomisation S19

Did she brush her teeth (2)?

1. Formulate statement to test.

2. Data (information at hand).

1.

She has brushed her teeth.

2. The toothbrush is wet.

3. Consider 1. and the data:

If 1. is true, then what are the chances of getting data like that in 2.?

3. The-toothbrush-is-wet would be likely if she had brushed her teeth. 4. Review the statement in 1. in light of 3. together with the data in 2.

4. Therefore, she could have brushed her teeth.

We have no evidence that she has not brushed her teeth.

I do not know Randomisation S20

Is the actual exercise effective?

1. Formulate statement to test.

2. Data (information at hand).

1. Chance is acting alone.

2. Observed diff = 1.4 months 3. Consider 1. and the data:

If 1. is true, then what are the chances of getting data like that in 2.?

3. A difference of 1.4 months or greater is 4. Review the statement in 1. in light of 3. together with the data in 2.

Randomisation S21

Is the actual exercise effective?

1. Formulate statement to test.

1.

Chance is acting alone.

2. Data (information at hand).

2. Observed diff = 1.4 months 3. Consider 1. and the data:

If 1. is true, then what are the chances of getting data like that in 2.?

4. Review the statement in 1. in light of 3. together with the data in 2.

3. A difference of 1.4 months or greater is

not unusual when chance is acting alone.

4. Therefore, chance could be acting alone OR

something else could be acting along with chance.

we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one.

Randomisation S22

Guidelines for assessing ‘Chance alone

’ • • • When the tail proportion is

small

(less than 10%):

the observed difference would be unlikely when chance is acting alone . . . therefore, it’s a fairly safe bet chance is not acting alone.

we have evidence

against

‘chance-is-acting-alone’ we have evidence that chance is

not

acting alone

Chapter 1

Guidelines for assessing ‘Chance alone’

• • • When the tail proportion is

large

(10% or more ) then:

the observed difference is not unusual when chance is acting alone, therefore chance could be acting alone we have NO evidence against ‘chance is acting alone’ EITHER chance could be acting alone OR something else as well as ‘chance’ COULD also be acting.

- we do NOT have ENOUGH INFORMATION to MAKE A CALL as to which one.

Randomisation S24

Questions… Chapter 1