Transcript Data Analysis for fMRI Computational Analyses of Brain Imaging CALD 10-731 and Psychology 85-735 Tom M.

Data Analysis for fMRI
Computational Analyses of Brain Imaging
CALD 10-731 and Psychology 85-735
Tom M. Mitchell and Marcel Just
January 15, 2003
1
Ten Minutes of Activity for One Voxel
Indicates
experimental
condition
2
3
…
4
fMRI Data Visualization
[from W. Schneider]
Slice View
3 D view
Time Series
Rendered View
Inflated View
5
Many Types of Analysis
• Transformation from fourier space into spatial images,
adjusting for head motion, noise, drift,... (FIASCO, SVM)
• Warping individual brains to canonical structure (Talairach,
AIR, SPM)
• Identifying voxels activated during task (t-test, F-test,…)
• Finding temporally correlated voxels (clustering)
• Factoring signal into few components (PCA, ICA)
• Modeling temporal evolution of activity (diffeqs, HMMs)
• Learning classifiers to detect cognitive states (Bayes, SVM)
• Modeling higher cognitive processes (4CAPS, ACT-R)
• Combining fMRI with ERP, behavioral data, …
6
Identifying Voxels Activated
During Task
For each voxel, vi, calculate t statistic
comparing activity of vi during task
versus rest condition.
Retain voxels with t-statistic above
some threshold
7
Mental Rotation of Imagined Objects
Clock rotation
Shephard-Metz rotation
both
[Just, et al., 2001]
8
Study of Men and Women Listening
“Men listen with only one side of their brains,
while women use both”
 Men listening
Women listening
(IU School of Medicine Department of Radiology)
9
Identifying Voxels with Similar
Time Courses
(functional connectivity)
10
Increase in functional connectivity between parietal and
inferior temporal areas with workload
(from Diwadkar, Carpenter, & Just, 2001)
Easier
Harder
The activation in two cortical areas (parietal/dorsal and inferior
temporal/ventral) becomes more synchronized as the object
recognition task becomes more difficult.
11
Factoring fMRI Signals into
Fewer Components
PCA, ICA, SVD, Hidden Units
12
Independent Component Analysis
of fMRI time-series
• ICA discovers statistically independent
components that combine to form the observed
fMRI signal
• ICA is a data-driven approach, complementary to
hypothesis-driven methods (e.g. GLM) for
analyzing fMRI data
• Finds reduced dimensionality descriptions of
poorly understood, high dimensional spaces
• Requires no a-priori knowledge about
hemodynamics, noise models, time-courses of
subject stimuli,…
13
Independent Component Analysis
of fMRI time-series: data-model
(McKeown et al., 1998)
14
Independent Component Analysis
of fMRI Time-series
[from W. Schneider]
ICA
algorithm
.
.
.
IC #1
IC #2
.
.
.
.
.
.
.
fMRI time-series
IC #T
15
Independent Component Analysis
of fMRI time-series
ICA
Solution
GLM
Solution
IC1
IC2
Images Elia Formisano & Rainer Goebel 2001
16
Advantages of ICA
• Interpretation of non-explicit condition
manipulation
– Not just AB type designs
– Applications driving, reading, problem solving
• Identify dimensions of poorly understood spaces
– Reduce high dimension data to few components
– Applications: structure of semantic memory, processes
underlying visual scene analysis in visual cortex
17
Learning Classifiers to Decode
Cognitive States from fMRI
Bayes classifiers, SVM’s, kNN, …
18
Study 1: Word Categories
[Francisco Pereira et al.]
•
•
•
•
•
•
Family members
Occupations
Tools
Kitchen items
Dwellings
Building parts
•
•
•
•
•
•
4 legged animals
Fish
Trees
Flowers
Fruits
Vegetables
19
Training Classifier for Word Categories
Learn fMRI(t)  word-category(t)
– fMRI(t) = 8470 to 11,136 voxels, depending on subject
Feature selection: Select n voxels
–
–
–
–
Best single-voxel classifiers
Strongest contrast between fixation and some word category
Strongest contrast, spread equally over ROI’s
Randomly
Training method:
– train ten single-subect classifiers
– Gaussian Naïve Bayes  P(fMRI(t) | word-category)
20
Results
Classifier outputs ranked list of classes
Evaluate by the fraction of classes ranked ahead of true class
0=perfect, 0.5=random, 1.0 unbelievably poor
21
Impact of Feature Selection
22
23
Summary
• Able to classify instantaneous cognitive state
– in contrast to describing average activity over time
• Significance
– Virtual sensors for mental states
– Step toward modeling sequential cognitive processes?
– Potential clinical applications: diagnosis = classification
24
Modeling temporal evolution of
activity
HMMs, Diffeqs, …
25
Challenge: learn process model -- HMM’s?
a=6,… 3x+a=2
recall correct
start
transform correct
read
problem
answer
recall error
transform error
…
time 
26
DCM [Friston 2002]
Aim: Functional integration and the modulation of specific pathways
Contextual inputs
Stimulus-free u2(t)
BA39
{e.g. cognitive set/time}
Perturbing inputs
Stimuli-bound u1(t)
{e.g. visual words}
y
STG
V4
y
BA37
y
V1
y
y
27
The DCM and its bilinear
approximation [Friston 02]
neuronal
changes
Input
u(t)
induced
connectivity
induced
response
 x1   a11  a n1  b11  bn1   x1   c1 
          u          u   
  
 
    
 x n  a1n  a nn  b1n  bnn   x n  cn 
b23
c1
intrinsic
connectivity
The bilinear model
a12
x  ( A   u j B j ) x  Cu
activity
x2(t)
activity
x1(t)
j
activity
x3(t)
  {A, B, C}
y
y
y
y(t )  h( x(t ))
Hemodynamic model
28
Overview
[Friston, 2002]
Models of
Constraints on
•Hemodynamics in a single region
•Neuronal interactions
•Connections
•Hemodynamic parameters
p( y |  )
p ( )
p( | y)  p( y |  ) p( )
Bayesian estimation
Applications
•Simulations
•Plasticity in single word processing
•Attentional modulation of coupling
29
Cognitive Models Grounded in
fMRI Data
30
4CAPS Model of Language Processing
[Just, et al., 2002]
31
The player was followed by the parent.
[Just, et al., 2002]
32
4CAPS Prediction of fMRI Activity
CU in 4CAPS
comprehension
model components
Model
prediction
Model CU
transform
fMRI
data
33
[Anderson, Qin,
& Sohn, 2002]
34
What We’d Like
Cognitive
model:
Understand
question
Recognize
word
See word
Understand
statement
Answer
question
Hypothesized
intermediate
states,
representations,
processes:
Observed
image
sequence:
time 
35
Machine Learning
Problems
What We’d Like
Cognitive
model:
Understand
question
Recognize
word
See word
Understand
statement
Answer
question
Hypothesized
intermediate
states,
representations,
processes:
Observed
image
sequence:
time 
6
• Learn f: image(t)  cognitiveState(t)
• Discover useful intermediate abstractions
• Learn process models
36