Transcript Consulting a biostatistician

Getting the most out of a
biostatistics consultation
John Pearson
University of Otago, Christchurch
2/9/8
Themes
• Who we are
• What we do
• How we work
2
Full time consultants
Assoc Prof Elisabeth Wells
Department of Public Health and General Practice
Email [email protected]
Research Interests
Psychiatric epidemiology Survey methods
Dr John Pearson
Department of Pathology
Email [email protected]
Research Interests
Microarrays
Genetics Bioinformatics
Survey Methods Statistical Computing
3
How to find us
• http://www.chmeds.ac.nz/departments/pub
health/biostat.htm
• Stephen Sharp (admin, ground floor) can
make bookings
– [email protected]
– Tel: 378 6026
• Supervisors should attend the first meeting
4
Two-way communication issues in
consulting
Client:
Be patient with a biostatistician who doesn’t know
your research area and is struggling to understand
enough to advise well
Biostatistician:
Be respectful and try to communicate with nonstatisticians, taking account of the amount of
statistics they know and trying to translate, if
required
5
Biostatistical consulting
UOC biostatistics webpage
http://www.uoc.otago.ac.nz/departments/pubhealth/biostat.htm
says:
“Biostatistical consultancy involves advice on
study design, methodology, computer software,
data analysis and preparation, and revision of
articles and reports. The biostatisticians can also
act as collaborators during research analysis or
on long term studies, and provide training in
statistical methods.”
6
Biostatistics webpage (ctd)
“There is no charge for consultation services
provided by biostatistical consultancy staff for
students and members of staff of the School and
the Canterbury District Health Board. However
biostatistical work should be costed as part of
research grant applications.”
WE ARE FREE BUT IN SHORT SUPPLY
7
Process
• Think first (preferably think lots)
• Read the literature
• Consult a biostatistician, well in advance
of an ethics or grant application
In this sequence each step may result in
your going back to an earlier step and
revising
8
Reasons for revising
Examples:
• Literature may show your project has been
done or that you need to measure other
variables
• The biostatistician’s questions may require
you to re-read articles or search again
• There may be ways of analysing the data
which you had not thought of so you may
revise your aims and objectives
9
Looping back and revising
• Looping back and revising is part of the
usual process of setting up a research
project
• The more you can think through things in
advance the better
• Nonetheless revision is often required as
you think further, and in more detail, and is
nothing to be ashamed of
10
What to bring to a biostatistician
• Aims/objectives/hypotheses
• Suggested study design
• Expected size of effect based on the
literature/clinical experience
or
the desired precision of some estimate
• Any key papers
• Draft data collection instrument (if
available)
11
Study design
• Observational studies:
- cross-sectional studies
- cohort studies
- case-control studies
• Intervention/experimental studies:
- for human trials usually only one or two
interventions are used
- for laboratory studies there will usually be
a dose variable with several levels and
often other experimental conditions as well12
Study design
Within the main study designs there will be
options:
• Is a case-control study unmatched or
matched (individually or frequency
matched)
• In a treatment trial are there parallel
groups or a cross-over design
• In follow-up of patients is it better to study
more patients or fewer more often?
13
What sample size do I need
• This is sometimes asked with no context
• That is like asking a travel agent ‘What
does it cost to travel?’
• The answer depends on what you want to
do.
14
Effect size or single estimate
When planning a study you need to
decide if you are:
• Estimating the size of an effect (eg the
difference between Tx A and Tx B) and
testing the hypothesis that there is an
effect
• Estimating a single quantity (eg the
proportion of the population living below
the poverty line) – no hypothesis here
15
Effect sizes expected
• Use the literature as a guide to the size of
effect you might find (similar studies,
similar treatment for different problem, size
of effect that would change clinical
practice or policy)
• Consider differences in proportions, odds
ratios, or differences in means (try to find
out the standard deviations if possible)
16
Effect size and sample size
• For comparisons, expected effect sizes
have to be thought through before a
statistician can come up with a sample
size
• Various scenarios can be looked at before
deciding one is best
• If sample size is fixed, then it is possible to
see what size of effect could be detected
17
Power and sample size
• Ethics applications mention the power of
the study. This is the probability of
obtaining a statistically significant result in
your study if the world is as you suppose it
(ie there is an effect of the size you
propose).
• Power is often set at 80% (odds of 4:1) but
sometimes at 90% or 95% for definitive
studies.
18
Power (ctd)
• If the power is less than 80% then there is
not much point in doing the study
• Biostatisticians don’t like multiple small
studies which are too small to detect the
likely effects of different treatments
19
Hypothesis testing vs confidence
intervals
• In the mid 1980s biostatisticians tried to
move health research away from
significance testing (results reported as
significant or not) to confidence intervals
around an estimate
• The estimate shows the most likely effect
and the confidence interval shows the
interval within which the true value is likely
to fall
20
Precision of single estimates
• Sometimes there is no ‘size of effect’ but just the
precision of a single estimate.
• This is often how sample sizes for national
surveys are set but it also can apply to small
surveys
• You may want to know if only a small, moderate
or high proportion of patients had a 12 month
follow-up outpatient visit
• However if the survey result is 40% you want to
know the precision (“margin of error”); 10%-70%
is a different estimate from 35%-45%
21
Planning for precision
Planning for precision is based on the
size of the confidence interval expected:
• If 4 out of 20 children have asthma then the
prevalence (P) is 20%, with a 95% confidence
interval (CI) of (8%, 42%)
• For 20/100, P=20%, 95%CI=13%, 29%
• For 40/200, P=20%, 95%CI=15%, 26%
Agresti, A., and Coull, B. Approximate is better than 'exact' for interval
estimation of binomial proportions. The American Statistician 52: 119-126, 1998
22
Planning for precision
Planning for precision is based on the
size of the confidence interval expected:
Proportion
CI for P=0.2
0.6
0.4
0.2
0.0
0
50
100
150
200
Sample size
23
Planning for precision
Precision depends on P so it is necessary
to look at plausible values for P
Precision vs proportion
0.2
50
0.15
CI width
• N=100 P=50%
95%CI=40%, 60%
• N=100 P=5%
95%CI=2%, 11%
0.1
100
0.05
200
0
0
0.2
0.4
0.6
Proportion
What precision do you want?
What precision can you afford?
24
Diminishing returns
• Precision and power depend on the
square root of N, not N
• To 1/2 a confidence interval you need to
4x the sample size, (not 2x it)
• Costs increase linearly with sample size
• There are diminishing returns from larger
and larger samples
25
Data collection
Biostatisticians can advise on data
collection:
• This should be planned well in advance of
starting collection
• A database or EXCEL spreadsheet should
be set up
• Procedures for checking data are required
26
Questionnaires
• Most biostatisticians have experience in
questionnaire design
• Biostatisticians will think about data entry
issues, coding and analysis of the data,
not just the questions themselves
• Questionnaires may look easy but there
are many traps
27
Computing
• Almost no research is done without
computers now
• Biostatisticians can advise on what
software packages to use
(but we use only some of them so if you
choose another then the support will be
limited.)
28
Analysis
Whether you carry out your own analyses or
whether a biostatistician does them
depends on:
• How much you know
• Whether or not you are a student
(students have to learn to do their own
analyses)
• How complicated the analyses are
29
Writing up the results
Biostatisticians can help with this:
• For papers or reports biostatisticians often
write up the statistical methods and may
write some or much of the results
• Students have to write their own results
but will receive guidance from their
supervisor(s) and may check with a
biostatistician
30
Responding to referee criticism
The point of referees is to find weaknesses
in studies (as well as to praise good
points). Depending on the issues raised
biostatisticians may be involved in:
• Rebutting criticism or making design
changes suggested by grant reviewers
• Rebutting criticism or carrying out
additional analyses from journal or report
reviewers
31
Summary
• Biostatisticians can be involved with all
stages of a project from design to
publication
• Sometimes they are required only for
technical advice at some part (but fixing
problems which occurred because advice
was not sought earlier is not liked)
• Sometimes they are part of the team
32
Take home messages
• Check statistics early
• Statisticians are mere mortals
33