Transcript Temporal Aspects of Visual Extinction

Spatial Processing
 Chris Rorden
– Spatial Registration
 Motion correction
 Coregistration
 Normalization
– Interpolation
– Spatial Smoothing
– Advanced notes:
 Spatial distortions of EPI scans
 Image intensity distortions
 Matrix mathematics
1
Why spatially register data?
Statistics computed individually for voxels.
Only meaningful if voxel examines same
region across images.
Therefore, images must be in spatially
registered with each other.
2
Spatial Registration
We use spatial registration to align images
– Motion correction (realignment) adjusts for an
individuals head movements.
– Coregistration aligns two images of different
modalities from the same individual.
– Normalization aligns images from different people.
3
Within-subject registration
 With-in subject registrations
– Assumption: same individual, so there should be a good linear solution.
Motion correction
Registration of
the fMRI scans
(across time)
Coregistration
Registration of fMRI
scans with high
resolution image.
4
Rigid Body Transforms
Translation
Rotation
 By measuring and correcting
for translations and rotations,
we can adjust for an object’s
movement in an image.
5
How many parameters?
 Each transform can be applied in 3 dimensions.
 Therefore, if we correct for both rotation and
translation, we will compute 6 parameters.
X
Y
Translation
Z
Pitch
Rotation
Yaw
Roll
6
Motion Correction
 Motion correction
aligns all in time
series.
 Translations and
rotations only
– ‘rigid body registration’
– Assumes brain size
and shape identical
across images.
7
Motion Correction
Motion Correction (‘Realignment’) is crucial:
– We want to compare same part of the brain across
time.
– If we do not MC, there will be a lot of variability in
our data.
Mathematically, MC is easy
– We assume that all images show the same brain,
so rigid body transform is sufficient.
– All images have the same contrast.
8
Motion Correction Cost Function
2
=
Target
Reslice
2
=
Target
When aligned,
Difference
squared = 0
When
unaligned,
Difference
squared > 0
Reslice
cfmi.georgetown.edu/classes/BootCamp/
9
Local Minima
 Search algorithm is iterative:
2. Test cost function
3. Repeat until cost function does
not get better.
 Search algorithm can get stuck at
local minima: cost function
suggests that no matter how the
transformation parameters are
changed a minimum has been
reached
Value of Cost Function
1. move the image a little bit.
Local
Minimum
Global
Minimum
Translation in X
cfmi.georgetown.edu/classes/BootCamp/
10
Motion correction cost function
 Motion correction uses variance to check if images are a
good match.
 Smaller variance = better match (‘least squares’)
 Iterative: moves image a bit at a time until match is worse.
Image 1
Image 2
Difference
Variance (Diff²)
11
Coregistration
Coregistration is more complicated than
motion correction
– Rigid body not enough:
Size differs between images (must add zooms).
fMRI scans often have spatial distortion not seen in
other scans (must add shears).
– Variance cost function will fail: relative contrast of
gray matter, white matter, CSF and air differences
between images.
12
Coregistration
Coregistration is used to align images of
different modalities from the same individual
 Uses ‘mutual information
cost function’: Note
aligned images have
neater histograms.
 Uses entropy reduction
instead of variance
reduction as cost
function.
13
Coregistration
 Used within individual, so linear transforms should be sufficient
 Typically 12 parameters (translation, rotation, zooms, shear each in 3 dimensions)
 Though note that different MRI sequences create different non-linear distortions
T1 image
Coregistered FLAIR
14
Between-subject: Normalization
 Allows inference about general population
Subject 1
Template
Subject 2
Average activation
Normalization
15
Why normalize?
 Stereotaxic coordinates analogous to longitude
– Universal description for anatomical location
– Allows other to replicate findings
– Allows between-subject analysis: crucial for inference that
effects generalize across humanity.
16
Normalization
 Normalization attempts to register scans from different people.
 We align each persons brain to a template.
– Template often created from multiple people (so it is fairly average in shape,
size, etc).
– We typically use template that is in the same modality as the image we want
to normalize
 Therefore, variance cost function.
 If different groups use similar templates, they can talk in common
coordinates.
Popular MNI Template
based on T1-weighted scans
from 152 individuals.
17
Coordinates - normalization


Different people’s brains look different
‘Normalizing’ adjusts overall size and orientation
Raw Images
Normalized Images
18
Coordinates - Earth
For earth (2D surface) we use latitude and
longitude
– Origin for latitude is equator
Explicit: defined by axis of rotation
– Origin for longitude is Greenwich.
Arbitrary: could be Paris
What is crucial is that we
we agree on the same origin.
19
Coordinates - stereotaxic
For the brain, left-right side is
obvious.
– Interhemispheric Fissure
analogous to equator
How about Anterior-Posterior and
Superior-Inferior?
We need an origin for these
coordinates.
20
Coordinates - Talairach
Anterior Commissure (AC) is the origin for
neuroscience.
– We measure distance from AC
57x-67x0 means ‘right posterior middle’.
Three values: left-right, posterior-anterior, ventral-dorsal
21
Coordinates - Talairach
 The AC is not enough
 We need second origin to define horizontal plane.
?
?
?
22
Coordinates - Talairach
 Axis for axial plane is defined by anterior commissure
(AC) and posterior commissure (PC).
 Both are small regions that are clear to see on most
scans.
Y+
Y PC
Z+
Z AC
23
Templates
 Original Talairach-Tournoux atlas based on histological slices
from one 69-year old woman.
– Single brain may not be representative
– No MRI scans from this woman
 Modern templates were at some stage aligned to images from
the Montreal Neurological Institute.
– MNI space slightly different from T&T atlas (larger in every dimension).
24
SPM uses modality specific template
– MNI T1 template, plus custom templates
FSL uses MNI T1 template for all modalities
– Requires intra-modal cost functions
T1
T2*
PET
25
Affine Transforms (aka linear, geometric)
Translation
Rotation
Zoom
Shear
26
Affine Transforms
Co-linear points
remain co-linear after
any affine transform.
Transform influences
entire image.
27
Spatial Processing
Non-linear transforms can match features that
could not aligned with affine transforms.
SPM uses basis functions.
28
Nonlinear functions and normalization
Scans from 6 people
Linear Only
Linear + Nonlinear
http://imaging.mrc-cbu.cam.ac.uk/imaging/SpmMiniCourse
29
Nonlinear basis functions
 Here are the functions SPM uses.
 They can be combined to create subtle deformations
30
A simple problem?
31
Regularization
Regularization penalizes bending energy
What is the best way to show graph points with
a smooth line? Heavy regularization is a poor
fit, heavy regularization causes local distortion
Heavy Regularization
Medium Regularization
Little Regularization
32
Regularization
Regularization is a parameter that you can
adjust that influences non-linear normalization
Medium Regularization
http://www.fmri.ox.ac.uk/fsl/fnirt/
Little Regularization
33
Spatial Processing
Affine Transforms are robust – they influence
the entire brain
Note that non-linear functions can have local
effects.
– This can improve normalization
– This can also lead to image distortion.
E.G. In stroke patients, the injured region may not match
the intensity of the template…
34
Non-linear transforms
Stroke image
Template image
Stroke variance image
35
Non-linear transforms
If you work with pathological brains, only use
non-linear transforms appropriately…
Linear
only
Linear
+Nonlinear
Mask+Linear
+Nonlinear
Template
36
Sulcal matching
Normalization conducted on smoothed images.
– We are not trying to precisely match sulci (would
cause local distortion).
Sulcal matching only approximate
– www.loni.ucla.edu/~thompson/
Post-normalization
alignment of calcarine
sulcus, precentral
gyrus, superior
temporal gyrus.
37
Alternatives
 SPM/FSL normalization will roughly match orientation and shape of
head.
– Good if function is localized to proportional part of brain
– Poor if function is localized to specific sulci (e.g. early visual area V1 tied to
calcarine fissure).
 Alternatively, use sulci as cost function (Goebel et al., 2006).
– Image below: mean sulcal position for 12 people after standard normalization (left)
followed by sucal registration (middle).
– Note: This technique improves sulcal alignment, but distorts cortical size.
38
Alternatives
SPM and FSL normalize overall brain
shape.
Individual sulci largely ignored.
What are different normalization
strategies?
– Sulci are crucial for some tasks (Herschl’s
gyrus and hearing)
– Perhaps less so for others (e.g. Amunts
et. al 2004 with Broca’s variability)
39
Interpolation
Top and bottom
images each
rotated 12º.
Top image looks
jagged, bottom
looks smooth.
Difference is in the
interpolation used
in reslicing.
40
cfmi.georgetown.edu/classes/BootCamp/
Interpolation
Reslicing data after
spatial registration will
require interpolation.
Rotations, zooms, etc
mean that there is not a
perfect source voxel for
each output voxel.
41
Interpolation
 How do we estimate
values that occur
between discrete
samples?
 Four popular methods:
1.
2.
3.
4.
Nearest neighbor
Linear
Spline
Sinc



42
Linear Interpolation
 For neuroimaging we usually use linear interpolation.
– Much more accurate than nearest neighbor.
– There is some loss of high frequencies – spline or sinc
interpolation are better but much slower to compute.
– Since we spatially smooth data after spatial registration, we
will lose high frequencies eventually.
1D Linear Interpolation
Weighted mean of 2 samples
2D Bilinear Interpolation
3D Trilinear Interpolation
Weighted mean of 4 samples Weighted mean of 8 samples
43
Linear Interpolation – High Frequency Loss
180o
20o
Original
10o
90o
360o
•Linear interpolation loses high frequencies
•Multiple successive resampling will lead to blurry image
•Solution: Minimize number of times the data is resliced.
cfmi.georgetown.edu/classes/BootCamp/
44
Advanced Interpolation
 Spline and Sinc interpolation can retain high frequency
information.
 Especially useful if multiple transformations will be applied.
 Computationally much slower to apply.
 Not necessary if you will heavily blur your data with a broad
smoothing kernel.
Sinc Function
cfmi.georgetown.edu/classes/BootCamp/
45
Smoothing
The need for spatial registration (motion
correction, registration, normalization) is
obvious.
However, intentionally blurring images seems
unintuitive: we are throwing away information.
46
Smoothing
Smoothing has several benefits:
– Each voxel is a noisy measure. A blurred image
minimizes noise and amplifies coherent signal.
Voxels are arbitrary: neighbors should show similar
signal.
– The statistics lectures will describe additional
benefits:
Reduces the number of independent statistics
Makes our data more ‘normal’ – fits the assumptions of
our statistics.
47
A smoothing kernel
 A ‘kernel’ describes how neighboring samples
influence a sample.
 We will start describing a one-dimensional filter.
[0.25,0.5,0.25] =
 Output values are equal to 50% of the
source value plus 25% of each neighbor.
 This acts to smooth the data.
 Kernels tend to sum to 1
 Values >1 will amplify the signal
 Values < 1 will attenuate the signal
48
A smoothing kernel
A wider kernel is influenced by more neighbors
– more blur, less noise.
[0.1,0.2,0.4,0.2,0.1]=
 Output values are equal to 40% of the
source value plus 20% of immediate
neighbors and 10% their neighbors.
 This acts to heavily smooth the data.
49
An edge-detection kernel
We can do interesting image processing with
kernels.
| [-1,0,1] |
=
 Output values are equal the difference
between the two neighbors.
 The || symbols mean the output is the
absolute value (always + sign)
 This detect edges.
50
2D Kernels
We can apply kernels to 2D and 3D images.
.1
.1
.1
.1
.2
.1
.1
.1
.1
=
 Output values are equal to 20% of the
source value plus 10% of each neighbor.
 This acts to smooth the data.
51
2D Edge Detection Kernels
We can also do edge
detection in 2D or 3D using
kernels.
52
Spatial Smoothing
 Each voxel is noisy. However, neighbors tend to show
similar effect. Smoothing results in a more stable signal.
 Smooth also helps statistics: smoothed data tends to be
more ‘normal’ – fits our assumptions. Also, allows RFT
thresholding (see Statistics lecture).
Gaussian Smoothing
=
53
FWHM
2 = 1
2 = 2
f(x)
2 = 3
2 = 4
Dispersion Differs
 Smoothing referred to a convolution: the output intensity
based on neighbors.
 The relative weighting of the neighbors is referred to as
the ‘kernel’.
 The most popular kernel is the gaussian function (a
normal distribution).
 The ‘full width half maximum’ adjusts the amount of
gaussian smoothing.
 FWHM is a measure of dispersion (like standard deviation
or variance)
 Large FWHMs lead to more blurry images.
 For fMRI, we typically use a FWHM that is ~x2..x3 our
original resolution (e.g. 8mm for 3x3x3mm data).
 However, the FWHM tunes the size of region we will be
best able to detect.
– E.G. If you want to look for a brain region that is around
10mm diameter, use a 10mm FWHM.
54
Smoothing
Spatial smoothing useful for between-subject
analyses.
– Spatial normalization is only approximate: smoothing
minimizes individual sulcal variability.
– Smoothing controls for variation in functional
localization between people.
None
4mm
8mm
12mm
55
Smoothing Limits Inference
 Extra activation observed
comparing strong with light taps.
 After smoothing we can not
distinguish between:
– Increased activation of the same
population of neurons
– Recruitment of more neighboring
neurons.
 Example: note that after
smoothing broad low contrast
looks line looks like focused high
contrast line.
56
Smoothing Alternatives
 Gaussian smoothing is great for
‘normal’ data – assumes few
outliers.
– Outliers will contaminate
neighbors.
 If your data has dropouts or
high-frequency artifacts,
consider alternative filters.
Gaussian
Smoothing
Median Filter
– Median filters
– FSL’s SUSAN
57
Intensity unwarping
 Motion correction creates a spatially
stabilized image.
 However, head motion also changes
image intensity – some regions of the
brain will appear brighter/darker.
– SPM: EPI unwarping corrects for brightness
changes (right)
– FSL: You can add motion parameters to
statistical model (FEAT stats page).
 Problem: We will lose statistical power if
head motion is task related, e.g. pitch
head every time we press a button
 Above: motion
related image
intensity
changes.
58
 The head distorts the magentic
field.
 Shimming attempts to make field
level homogeneous.
 Even after shimming, there will
be varying field strengths.
 Specifically, regions with large
density changes (sinus/bone of
frontal lobe).
 This inhomogeneity leads to
intensity and spatial distortion.
www.bruker-biospin.de/MRI/applications/medspec_hcalc.html
Spatial unwarping
59
Spatial unwarping
 We can measure field in
homogeneity.
 This can be used to unwarp
images (FSL’s B0 unwarping,
SPM’s FieldMap).
Raw EPI
Structural
Unwarped
EPI
60
Bias correction
Inhomogeneity also leads to variability
in image intensity.
– Bias correct anatomical scans (e.g.
SPM’s segmentation, N3).
Field homogeneity issues more severe
with higher field strength.
Parallel Imaging (collecting MRI with
multiple coils) can dramatically reduce
effects.
61
Euler angles
Most people like to describe spatial transforms
as Euler angles (‘yaw’, ‘pitch’, ‘roll’).
In neuroimaging, we use mathematical
matrices to describe linear spatial transforms.
– Very compact: just 12 numbers can describe the
result of an infinite number of linear transforms
without losing precision.
– Avoids ‘gimbal lock’ problem of Euler angles.
62
Euler transforms
 We like to talk about ‘yaw’, ‘pitch’ and ‘roll’.
 However, order of these transforms is crucial.
 Consider a plane that pitches 90º and then rolls 90º
 Different direction to a 90º roll followed by 90º pitch
63
Euler angles
Euler angles: order is important
– Mathematically, we can not explain all rotations with only
three numbers for yaw, pitch and roll (we also need to
specify the order).
– While directions make sense from pilots point of view, they
can be confusing from the ground (e.g. the first rotation of
90º shifts the frame of reference for following rotations).
E.G. Computer warnings of Apollo 11 lunar lander: used moon’s
surface as frame of reference. Only 3 gimbals were used
threatening ‘gimbal lock’ problems.
64
Vectors
Vectors
– Vectors have a direction and a length
The 2D vector [1,0] points East and has a length of 1
The 2D vector [1,2] points North-East and has a length of 2.23
– 2D vectors have two values [x,y], 3D vectors have three
values [x,y,z]
1, 0
0, 1
1, 2
65
2D Rotation matrix
2D transformation matrix as two vectors (horizontal
and vertical)
– Can be described by four numbers
Original
Zoom
Rotation
Shear
1, 0
1, 0
.7,-.7
1, 0
0, 1
0, 2
.7, .7
1, 1
Flip
-1, 0
0, 1
66
2D Transformation matrix
Transformation matrix is a rotation matrix plus
translation values
– 9 values, final row is always 0 0 1 (must have as many
rows as columns).
Original
Translated
Translated
1, 0, 0
1, 0, 1
1, 0, .2
0, 1, 0
0, 1, 0
0, 1, .2
0, 0, 1
0, 0, 1
0, 0, 1
67
Mathematical Matrices
SPM, FSL, etc use matrix mathematics to
compute linear spatial transforms.
We will start with a matrix for 2D space. This has
six numbers that concern us (3 columns, 2 rows).
To transformed horizontal position for x
fx = (x*i)+(y*j)+k
To transformed vertical position y
i
j
k
l
m
n
fy = (x*l)+(y*m)+n
68
The identity matrix
The identity matrix has 1 on the diagonal, and
zeros in all other positions.
With this matrix, the input and output are
identical:
1
0
0
=
0
1
0
69
Translations
Translations are performed by adding a value
to the last column.
– Top-left value moves left-right
– Bottom-left value moves up-down
Horizontal
Position
1
0
0
=
0
1
20
Vertical Position
70
Shear
You can skew an image by adding/subtracting a
value at the position multiplied by the orthogonal
direction.
Horizontal
Shear
1
0
0
=
-0.2 1
Vertical Shear
0
71
Scaling
Zooms are performed by multiplying all the
values in a row by your scale factor.
– Top row shrinks/stretches horizontally
– Bottom row shrinks/stretches vertically
– Use a negative value to mirror-flip in the axis.
1.3 0
0
=
0
1
0
72
Rotations
To rotate a 2D matrix by an angle :
cos() sin()
-sin() cos()
Note: translation values (right column) set the
center for pivoting.
.93 -.37 0
=
.37 .93
0
73
Combining Matrix Tranforms
 Our six numbers can store all of the possible rotations,
shears, translations and scaling.
– Simply multiply previous matrix with our transform (order crucial).
– E.G. Zoom an image we have rotated and translated…
.93 -.37 5
.37 .93 -5
1.3 0
0
1
1.2 -.48 7
0
0
=
.37 .93
-5
74
2D Matrices are 3x3
 In actual fact, our 2D matrices have 9 values
– To multiply matrices together, the first matrix must have
exactly as many rows as the second matrix has columns.
– So it is useful to use matrices with equal numbers of rows
and columns.
 However, last row is always ‘0 0 1’
 Therefore, the identity matrix really looks like this:
1
0
0
0
1
0
0
0
1
75
3D Matrices are 4x4
We can generate 4x4 matrices that will allow
us to work with 3D images.
A 4x4 matrix, but the last row always ‘0 0 0 1’
fx = (x*i)+(y*j)+(z*k)+l
fy = (x*m)+(y*n)+(z*o)+p
fz = (x*q)+(y*r)+(z*s)+t
i
j
m n
q r
0 0
k l
o p
s t
0 1
76
Matrices and 3D space
 3D matrices work just like 2D matrices
–
–
–
–
–
–
The identity matrix still has 1’s along the diagonal
Translations are the values in the final column (constants)
Zooms are done by scaling all values of a row.
Shears are values added to the relevant orthogonal value
Rotations use sine/cosine in dimensions of plane.
Our twelve numbers can store all of the possible rotations,
shears, translations and scaling.
 Simply multiply previous matrix with our transform (order crucial).
77