Transcript Multiple Comparison Correction in SPMs Will Penny SPM short course, Zurich, Feb 2008

Multiple Comparison
Correction in SPMs
Will Penny
SPM short course, Zurich, Feb 2008
image data
parameter
estimates
design
matrix
kernel
realignment &
motion
correction
General Linear Model
smoothing
model fitting
statistic image
Random Field
Theory
normalisation
anatomical
reference
Statistical
Parametric Map
corrected p-values
Inference at a single voxel
NULL hypothesis, H: activation is zero
a = p(t>u|H)
We can choose u to ensure
a voxel-wise significance level of a.
u=2
t-distribution
This is called an ‘uncorrected’ p-value, for
reasons we’ll see later.
We can then plot a map of above threshold
voxels.
Inference for Images
Noise
Signal
Signal+Noise
Use of ‘uncorrected’ p-value, a=0.1
11.3%
11.3%
12.5%
10.8%
11.5%
10.0%
10.7%
11.2%
Percentage of Null Pixels that are False Positives
10.2%
9.5%
Using an ‘uncorrected’ p-value of 0.1 will lead us to conclude on average that 10% of
voxels are active when they are not.
This is clearly undesirable. To correct for this we can define a null hypothesis for
images of statistics.
Family-wise Null Hypothesis
FAMILY-WISE NULL HYPOTHESIS:
Activation is zero everywhere
If we reject a voxel null hypothesis
at any voxel, we reject the family-wise
Null hypothesis
A FP anywhere in the image
gives a Family Wise Error (FWE)
Family-Wise Error (FWE) rate = ‘corrected’ p-value
Use of ‘uncorrected’ p-value, a=0.1
Use of ‘corrected’ p-value, a=0.1
FWE
The Bonferroni correction
The Family-Wise Error rate (FWE), a, for a family of N independent
voxels is
α = Nv
where v is the voxel-wise error rate. Therefore, to ensure a particular
FWE set
v=α/N
BUT ...
The Bonferroni correction
Independent Voxels
Spatially Correlated Voxels
Bonferroni is too conservative for brain images
Random Field Theory
• Consider a statistic image as a discretisation of a
continuous underlying random field
• Use results from continuous random field theory
Discretisation
Euler Characteristic (EC)
Topological measure
– threshold an image at u
- EC = # blobs
- at high u:
Prob blob = avg (EC)
So
FWE, a = avg (EC)
Example – 2D Gaussian images
α = R (4 ln 2) (2π) -3/2 u exp (-u2/2)
Voxel-wise threshold, u
Number of Resolution
Elements (RESELS), R
N=100x100 voxels,
Smoothness FWHM=10,
gives R=10x10=100
Example – 2D Gaussian images
α = R (4 ln 2) (2π) -3/2 u exp (-u2/2)
For R=100 and α=0.05
RFT gives u=3.8
voxels
data matrix
scans
=
design matrix
Estimated component fields

?
parameters
+
errors
?
^

 estimate

parameter
estimates


=
Each row is
an estimated
component field
residuals
estimated variance
estimated
component
fields
Applied Smoothing
Smoothness
smoothness » voxel size
practically
FWHM  3  VoxDim
Typical applied smoothing:
Single Subj fMRI: 6mm
PET: 12mm
Multi Subj fMRI: 8-12mm
PET: 16mm
SPM results I
Activations
Significant at
Cluster level
But not at
Voxel Level
SPM results II
Activations
Significant at
Voxel and
Cluster level
SPM results...
False Discovery Rate
ACTION
Don’t
Reject
TRUTH
At u1
Reject
H True (o)
TN=7
FP=3
H False (x)
FN=0
TP=10
Eg. t-scores
from regions
that truly do and
do not activate
FDR = FP/(# Reject)
a = FP/(# H True)
FDR=3/13=23%
a=3/10=30%
oooooooxxxooxxxoxxxx
u1
False Discovery Rate
ACTION
Don’t
Reject
TRUTH
At u2
Reject
FDR=1/8=13%
a=1/10=10%
H True (o)
TN=9
FP=1
H False (x)
FN=3
TP=7
Eg. t-scores
from regions
that truly do and
do not activate
FDR = FP/(# Reject)
a = FP/(# H True)
oooooooxxxooxxxoxxxx
u2
False Discovery Rate
Noise
Signal
Signal+Noise
Control of Familywise Error Rate at 10%
Occurrence of Familywise Error
FWE
Control of False Discovery Rate at 10%
6.7%
10.4%
14.9%
9.3% 16.2% 13.8% 14.0% 10.5% 12.2%
Percentage of Activated Pixels that are False Positives
8.7%
Summary
• We should not use uncorrected p-values
• We can use Random Field Theory (RFT) to ‘correct’ p-values
• RFT requires FWHM > 3 voxels
• We only need to correct for the volume of interest
• Cluster-level inference
• False Discovery Rate is a viable alternative