Estimating Power for fMRI & Classification Directions in fMRI

Thomas Nichols Clinical Imaging Centre GlaxoSmithKline

Overview

• Power Exploration – ROIs (small/big, lots/few) ?

GD Mitsis, GD Iannetti, TS Smart, I Tracey & R Wise Regions of interest analysis in pharmacological fMRI: How do the definition criteria influence the inferred result?

Epub NeuroImage • Power Prediction • Classification

Power Review: 1 Test

• Power: The probability of rejecting H 0 when H A is true • Specify your null distribution – Mean=0, variance=σ 2 • Specify the effect size ( Δ), which leads to alternative distribution • Specify the false positive rate, α Null Distribution 0.4

0.35

0.3

0.25

0.2

0.15

0.1

0.05

0 -4 -2 0 Alternative Distribution α Power 2 4 Δ/σ 6 8

Power: 100,000 Tests?

• Avoid Multiple Testing Problem if possible – Typically study will use well-characterized paradigm – Expected region of response should be known • But… – Variation in functional and structural anatomy – “Perfect” region never known • Should we use focal ROI?

• Voxel-wise search in neighborhood?

• Over whole brain anyway?

Qualitative Power Exploration

• Simplified power setting – Not voxel-wise; instead largish (>1000 voxel) VOIs – Large VOIs: Assuming σ within << σ between • Hence different sized VOI’s will have similar variance – Large VOIs: Assuming independence between VOIs • Consider impact of many vs. fewer VOI’s – Many VOIs • Better follows anatomy, possible shape of signal • Worse multiple testing correction – Fewer VOIs • Will dilute localized signal • Fewer tests to correct for

AAL & Derived ROI Atlases

Atlas 0 (AAL) k = 116 regions α FWE = 0.00043

Atlas 3 k = 17 regions α FWE = 0.00294

(surrogate for correlated voxel-wise search) Atlas 1 (AAL symmetric) k = 58 regions α FWE = 0.00086

Atlas 4 (Lobar AAL) k = 6 regions α FWE = 0.00833

Atlas 2 k = 28 regions α FWE = 0.00179

Atlas 5 (whole GM) k = 1 region α FWE =0.05000

L Amygdala

Atlas 0 (AAL) k = 116 regions α FWE = 0.00043

Signal # VOIs = 1 Strength = 100% Atlas 1 (AAL symmetric) k = 58 regions α FWE = 0.00086

Signal # VOIs = 1 Strength = 47% Atlas 2 k = 28 regions α FWE = 0.00179

Signal # VOIs = 1 Strength = 47% Atlas 3 k = 17 regions α FWE = 0.00294

Signal # VOIs = 1 Strength = 4.9% Atlas 4 (Lobar AAL) k = 6 regions α FWE = 0.00833

Signal # VOIs = 1 Strength = 0.6% Atlas 5 (whole GM) k = 1 region α FWE =0.05000

Signal # VOIs = 1 Strength 0.1%

Power: L Amygdala, True ROI

• True ROI best (of course) • Rich ROI atlas (k=116) beats coarser atlases – Dilution more punishing than greater multiple testing

Power: L Amygdala, Shifted ROI

• True ROI best • Wrong (unshifted) ROI next • Rich ROI atlas still beats coarser atlases

Power: ½ of Mid-Cingulate

• Whole Mid Cing ROI best • Again, huge (k=116) atlas next best • But we’ve assumed RFX – No precision gain for large ROI’s, as shrinking σ WiN is no help

Power: ½ of Mid-Cingulate: FFX

• Whole Mid Cing ROI best • Now Symmetric AAL atlas (k=58) best!

– If σ BTW small, precision increase with large ROIs has impact

Power Exploration Conclusions

• Compared Range of Scales – Whole Brain, Lobar (k=6),…, AAL (k=116) • Focal structures – Focal ROI’s best • More extended signals, with heterogeneity – Rich atlas best • Dilution of signal worse than Bonferroni • But whole-brain always less powerful than reduced volume – Suggests voxel-wise / “Multiple Endpoint” result preferred, constrained coarsely

Why Doesn’t Bonf. Hurt More?

• Example – 1100 total voxels – 100 voxels have β=Δ • A test with 50% power on average will detect 50 of these voxels with true activation – 1000 voxels have β=0 • α=5% implies on average 50 null voxels will have false positives • 1 Signal ROI – 1 opportunity for a positive • 100 Signal Voxels – 100 opportunities for a positive Reject H 0

Truth (unobserved)

H 0 True H 0 False Type I Error α

50

Power

50

Accept H 0 Correct

950 1000

Type II Error

50 100

Formal Power analysis

• N: Number of Subjects – Adjusted to achieve sufficient power • α: The size of the test you’d like to use – Commonly set to 0.05 (5% false positive rate) • Δ: The size of the effect you’re interested in detecting – Based on intuition or similar studies • σ 2 : The variance of Δ – Has a complicated structure with very little intuition – Depends on many things …

Temporal autocorr.

Cov(Y)= σ 2 w V

Power for Group fMRI

Subject 1 Subject 2

. . .

Between subject variability, σ 2 B

J. Mumford & TE. Nichols. NeuroImage 39:261 –268, 2008 Subject N http://www.fmripower.org

Level 1

Y k = X k β K + ε k =

β k0 β k1 β k2 β k3

+ • Y k • X k • β k • ε k : T : T k k -vector timeseries for subject k  p design matrix : p-vector of parameters : T k -vector error term, Cov( ε k )= σ 2 k V k

β cont

=

Level 2

X g β g β g1 β g2

+ +

ε g

= ^ k • X g : N  p g design matrix • β g : p g -vector of parameters • ε g : N-vector error term – Cov(ε g ) = V g = diag { c(X T k V k -1 X k ) -1 σ k 2 c T } + σ B 2 I N Within subject variability Between subject variability

Alternative distribution

• For a specific H A :c g β g = Δ • t is distributed T n-pg, ncp – ncp= Δ/c g (X g T V g -1 X g )c g T N # subj α FPR Δ Effect Mag.

σ 2 Effect SD c g 2 nd X g Level Model σ 2 W V W/in Subj SD σ 2 B Btw Subj SD c 1 st X k Level Model σ 2 k V k (σ WN ,σ AR ,ρ) Noise Mag. Noise Autocorrelation

known guessed

Model

• Block design 15s on 15s off • TR=3s • Hrf: Gamma, sd=3 • Parameters estimated from Block study – FIAC single subject data – Read 3 little pigs • Same/different speaker, same/different sentence • Looked at blocks with same sentence same speaker

Power as a function of run length and sample size • Assumes fixed maximal scanner time • 21 Ss optimal • Btw 23 and 18 subjects sufficient – 17 subjects cannot obtain sufficient power

More importantly….cost!

•Cost to achieve 80% power •Cost=$300 per subject+$10 per each extra minute

Power, Accounting for searching over space?

S Hayasaka, AM Peiffer, CE Hugenschmidt, PJ Laurienti.Power and sample size calculation for neuroimaging studies by non-central random field theory. NeuroImage 37 (2007) 721 –730

Univariate vs. Multivariate

• Mass Univariate Modelling – Model each voxel independently (account for dependence at inference stage) – Great for localization – Doesn’t acknowledge spatial structure • Multivariate Modelling – Model entire volume simultaneously – Explicitly uses spatial structure – Not as good for localization

Multivariate Classification: Classification of Subjects

• ICA Components appear to distinguish NC vs. SZ vs. BP – fMRI Experiment: Auditory oddball task • But no one voxel responsible VD Calhoun, PK Maciejewski, GD Pearlson, KA Kiehl. Temporal Lobe and ‘‘Default’’ Hemodynamic Brain Modes Discriminate Between Schizophrenia and Bipolar Disorder. Human Brain Mapping, Epub 2007 Sep 25

Multivariate Classification: Classification of Subjects

• ICA Components appear to distinguish NC vs. SZ vs. BP – fMRI Experiment: Auditory oddball task • But no one voxel responsible VD Calhoun, PK Maciejewski, GD Pearlson, KA Kiehl. Temporal Lobe and ‘‘Default’’ Hemodynamic Brain Modes Discriminate Between Schizophrenia and Bipolar Disorder. Human Brain Mapping, Epub 2007 Sep 25

Multivariate Classification

• Even very simple method can give very good performance – Define average IC grp for each group – Label subj k with group that has minimum Euclidian distance (btw IC k & IC grp )

Multivariate Classification: Prediction Time Series

Pittsburgh Brain Activity Interpretation Competition Inferring Experience Based Cognition from Virtual Reality fMRI

Greg Siegle, Walter Schneider, Maureen McHugo, Melissa Thomas, Lori Koerbel, Lena Gemmer, Kate Fissell, Sudhir Pathak, Dan Jones, Kevin Jarbo

University of Pittsburgh

Inferring Cognition

• Virtual Reality fMRI Paradigm – Subjects explore neighborhood, looking for fruit, guns, dogs – 11 features rated continuously • e.g. arousal, valance, movement, dog, cell phone, etc – 3 Sessions of fMRI data • Features only given for 1 st 2 sessions R 2 =.79

17 minutes

Surprisingly accurate results

Arousal Valence Hits SearchPeople SearchWeapons SearchFruit Instructions Dog Faces FruitsVegetables WeaponsTools InteriorExterior Velocity 1 st place -.2

0 .2

.4

.6

Correlation .8

1 • Very different methods gave similar scores (based on pre and post processing) • Similar methods (e.g., support vector machines) gave very different results.

Lessons from Contest

• Pre-processing mattered – Detrending details had big impact • Multivariate, but not un informed – Winners used masks • Weighting salient voxels, ignoring uninformative ones – Post-processing clean up • In general, extensive tuning per feature to be predicted Subject14 visual cortex Use for “Interior Exterior” Subject13 auditory cortex Use for “Dog”

Conclusions

• Power for fMRI – Focused ROI’s, but not too focused – Exact power predictions possible • As always, based on guesses • Classification – Uses entire brain to predict subject identity or cognitive state – New direction, methods still evolving • e.g. Support Vector Machines work well, but never with out appreciable feature selection/tuning