Transcript Tips for a Sensible Animal Justification Section

Tips for a Sensible Animal
Justification
Reid D. Landes
Disclaimer: the opinions and thoughts contained herein are
not necessarily those of UAMS or any of its affiliates
(including the IACUC).
Outline
• Reviewers’ questions
• Answers for
– Non-statistical justifications
– Statistical justifications
• Elements of a (normal) power analysis
• Pilot studies
Reviewers ask
•
•
•
•
What are you measuring?
What are the groups?
What do you want to learn?
Do the animal numbers match your need?
Non-statistical Justifications
• Reviewer: Measuring what? Learning what?
– PI: “From the animals? Nothing.”
• Typical protocols with these answers
– Breeding protocols
– Materials protocols
Comment #1
• Breeding protocols: How many animals…
– are needed for future research objectives and
maintenance of colony?
– will not be used for any purpose?
• Materials protocols: How much material…
– is needed & why that amount?
– does one animal provide?
Tips for Statistically Justifying
Your Animal Use
Answers to Reviewers’ Questions
for Statistical Justifications
Comment #2
• Many protocols have more than one
experiment
• Some differ in their design or size
• Some differ in their outcomes
• Usually power analyses are needed for
each unique design-size-outcome
combination
What are you measuring?
• Define your outcome measures
• Give their units
• Several outcomes in one experiment?
– Which is most important?
– Which is most variable?
– On what outcome is the power analysis
based?
Outcome Measures Examples
• Cancer study: “Tumor burden”
– Number of cancer cells per ml. Or tumors.
– Volume of largest tumor. Or all tumors.
• ALS Study: Several measures
– Time on a rotating rod (seconds)
– Survival time (days; power based on this)
– Oxidative damage markers (more definition
needed here)
What are the groups?
• Describe the “experiment design”
• List the groups (and how they are formed)
– The Methods section contains what happens
to animals in the various groups
Experiment Design Example
Strain
WT
Drug
--
Dose
0
A
KO
B
--
A
B
Lo Hi Lo Hi 0 Lo Hi Lo Hi
• “Mice within a strain equally randomized among the 5
drug-dose combinations”
• 6 mice / strain-drug-dose combo X
10 strain-drug-dose combos = 60 mice
Experiment Design Example (cont)PO
Strain
WT
Drug
--
Dose
0
A
KO
B
--
A
B
Lo Hi Lo Hi 0 Lo Hi Lo Hi
• Define strains, drugs, and drugs’ low & high
doses outside of the table
• If sample size differs among Strain-Dose-Drug
combos, then add a new row
What do you want to learn?
• “What are your research hypotheses?”
• Identify comparisons are of interest
• Sometimes, comparisons are ordered
– Primary, secondary, tertiary
– Base power analyses on primary ones
• Often possible comparisons exceed
comparisons of interest
Comparisons of Interest Example
Strain
WT
Drug
--
Dose
0
A
KO
B
--
A
B
Lo Hi Lo Hi 0 Lo Hi Lo Hi
• Possible comparisons: 10C2 = 45 pairs
• Primary: Within Drug, compare strains (2)
• Secondary: Within Strain and Drug, compare
each dose to control (2x2x2=8)
Do the animal numbers match your need?PO
•
•
Provide a well-described power
analysis
Attributes of “well-described”
1. Describes statistical analyses that are
consistent with experiment design
2. Describes a power analysis that
•
•
is consistent with the statistical analysis
can be reproduced
3. Presents and justifies the parameters and
assumptions required in the power analysis
Comment on Attribute #1
• Must you describe the statistical analysis?
– For me? Not required, but definitely appreciated
• Often, a well-described power analysis
sufficiently describes the statistical analysis
– For simpler experiments
• If a statistical / power analysis is beyond
investigators’ capabilities, consult / collaborate
with a statistician
Comment on Attribute #2
• Must have reproducible power analysis?
– I’m looking for this
• Sample sizes based on heuristics (trial &
error, “standard” for a particular field, etc.)
are generally turned back
Comment on Attribute #3
• Parameters of a power analysis
– Significance level, Power, Sample size, Effect size,
…and others depending on statistical analysis
– All are required to reproduce the power analysis
Particular parameter = F(other parameters)
• Often have to assume / estimate one or more of
these
– Explanation or references for estimates should be
given
Elements of a Power Analysis
Normal-distribution based
• Need to identify (or calculate) the previous
4 parameters, plus
– parameters specific to the experiment (e.g.,
number of groups, factors and levels)
– statistical analysis (e.g., 1-way, 2-way
ANOVA, two-sample t-test, paired t-test, etc.)
– Possibly comparisons of interest
Effect Size Comment
• What about a SD? Or difference to detect?
– Encompassed in an effect size
• When comparing two groups’ means
an effect size = difference in means
within-group SD
• difference = 5, SD = 10  effect size = ½
• difference = ¾, SD = ¼  effect size = 3
Example of a Power Analysis
Strains-Drugs-Doses
• Assuming
– Y, outcome of interest, is normal
– All Ys are mutually independent
– k =10 groups are mutually independent
– “one-way” ANOVA
• Significance level=.05, power=.80, and…
• and n=6 … forces effect size=
– 1.17 (not controlling for multiple tests)
– 1.30 (controlling for 2 tests with Bonferroni’s
method)
• and effect size=1.0 … forces n=
– 8 (not controlling for multiple tests)
– 10 (controlling for 2 tests with Bonferroni’s
method)
Example Write Up
“With 6 mice per strain-drug-dose combination, we
can detect a difference of 1.3 SDs for the
primary comparisons of interest with at least .80
power using a .05 level two-sided t-test, adjusted
for 2 tests with Bonferroni’s method and
conducted within an one-way ANOVA context.
SDs of survival times have been estimated to be
as high as 10 days [refs]. A difference of 2
weeks was found between Drug A and a control
in [ref]; hence, providing evidence that a
difference of 13 days is not unreasonable to
expect.”
Pilot StudiesPO
• Reviewers still ask the same questions
• Animal measurements informing scientific
objectives call for justification
• Often, statistical justifications are possible
and sensible for pilot studies
• When not possible, decision rules should
be in place
Objectives of Pilot studies
• Estimating a statistical parameter; e.g.,
SD, mean, proportion, etc.
– Report widths of 95% Confidence Intervals for
requested n
• Determining feasibility
– Feasibility defined in terms of measurement
– i.e., define a rule by which feasibility is
determined
Pilot Study Example #1
Estimating a statistical parameter
• Estimate SD of Y with n=10 animals
“Assuming Y is normal in distribution, we will
have 95% confidence that the true SD is no
more (less) than 1.83 (0.68) times the
estimated SD.”
Pilot Study Example #2
Determining Feasibility
• Want to know if a new procedure is
reliable (i.e., feasible) in producing a
desired effect.
• Requesting 6 animals
• Investigator defined rule: “The procedure
will be deemed ‘feasible’ if the effect is
produced in all 6 animals. We will examine
each animal in sequence stopping if the
effect is not produced.”
Contact Information
•
•
•
•
•
Reid Landes
[email protected]
501-526-6714
COPH Room 3224
www.uams.edu/biostat/landes