Jan Kamenický Mariánská 2008

2

 ◦ ◦ ◦ We deal with medical images Different viewpoints - multiview Different times - multitemporal Different sensors – multimodal  Area-based methods (no features)   Transformation model Cost function minimization 3

 ◦ Transformation model Displacement field

u

(

x

)

I R T

(

x

)  

x I S

 

u

(

x

)  4

 ◦ Transformation model Displacement field

u

(

x

)

I R T

(

x

)  

x I S

 

u

(

x

)   ◦ ◦ ◦ Cost function Similarity measure (external forces) Smoothing (penalization) term (internal forces) Additional constraints (landmarks, volume preservation)

R

,

I S

)  

R

,

I S

)    

C soft

5

 ◦ Transformation model Displacement field

u

(

x

)

I R T

(

x

)  

x I S

 

u

(

x

)    ◦ ◦ ◦ Cost function Similarity measure (external forces) Smoothing (penalization) term (internal forces) Additional constraints (landmarks, volume preservation)

R

,

I S

)  

R

,

I S

)   Minimization

T

ˆ 

T C T I R

,

I S

)   

C soft

 

I R

,

I S

) 6

     Translation Rigid (Euler) ◦ Translation, rotation Similarity ◦ Translation, rotation, scaling Affine ◦ B-splines Control points - regular grid on reference image 

x k

 

N x p k

 3 (

x



x k

) 8

9

    Sum of Squared Differences Normalized Correlation Coefficients Mutual Information Normalized Gradient Field 10

 ◦ Sum of Squared Differences (SSD) Equal intensity distribution (same modality)

SSD



I R

,

I S

)  1 

R x i

 

R



I R x i



I S

 ( )

i

 2     Normalized Correlation Coefficients Mutual Information Normalized Gradient Field 11

    ◦ Sum of Squared Differences Normalized Correlation Coefficients (NCC) Linear relation between intensity values (but still same modality)

NCC I R

,

I S

) 

x i

 

R x i

 

R



I



R I R x i x i



I



R



I

2

R

 

x i

 

R I S

 

I S

 ( )

i

  ( )

i I



S

 

I S

2  Mutual Information Normalized Gradient Field 12

   Sum of Squared Differences Normalized Correlation Coefficients ◦ Mutual Information Any statistical dependence

MI I R

,

I S

)   

R

 2  

R S

  Normalized Gradient Field   13

 ◦ Mutual Information (MI) From entropy    2          | )  ) 2 

p X Y

 14

 ◦ Mutual Information (MI) From Kullback-Leibler distance   

i

 2 

p X Y

 15

 ◦ ◦ Mutual Information (MI) For images 

p

(

x

)

MI

… normalized image histogram

I R

,

I S

)     2 

p R R

Normalized Mutual Information (NMI)

S

   

NMI I R

,

I S

)   

R R

2  

R R

  

S



S

2   2

S

 16

 ◦ Mutual Information (MI) Joint probability estimation  Using B-spline Parzen windows   1 

R x i

 

R w R

 

r



I



R R x i

  

w S

 

s



I S

 

S

( )

i

    

R



S

17

    ◦ Sum of Squared Differences Normalized Correlation Coefficients Mutual Information Normalized Gradient Field (NGF) Based on edges

e

 

I

 2 2 

e

2

NGF I R

,

I S

)  

x i

 

R



T e R

, ) ( (

i e S



x i

 2 18

 ◦ Elastic Elastic potential (motivated by material properties)

P elas

    4   

x j u k

 

x k u



j

2   2  div

u

 2

dx

 ◦ Fluid Viscous fluid model (based on Navier-Stokes)  ◦ Diffusion Much faster

P diff

 1 2 

l

 

u l

2

dx

19

 ◦ Curvature Doesn’t penalize affine transformation

P curv

 1 2

l d

  1   

l

2

dx

 Bending energy (Thin plate splines)       2

u p r

 2

dx

20

curvature elastic diffusion fluid 21

  ◦ Landmarks (fiducial markers) “Hard” constraint

C j r j u r j



t j

,

j

 1, 2,...,

m

◦ “Soft” constraint

C soft

 

m j

 1

C j

2 Volume preservation

C soft

  

R



u x



dx

22

   ◦ ◦ Full Grid ◦ Used with multi-resolution Random Random subset of voxels is selected Improved speed 23



k

 1

k

 

k



a d k k

,  0,1, 2,...

25

 ◦ Gradient Descent (GD) Linear rate of convergence 

k

 1  

k



a g k

( 

k

)     Quasi-Newton Nonlinear Conjugate Gradient Stochastic Gradient Descent Evolution Strategy 26

  ◦ Gradient Descent Quasi-Newton (QN) Can be superlinearly convergent 

k

 1  

k



H

 1 ( 

k



k

)    Nonlinear Conjugate Gradient Stochastic Gradient Descent Evolution Strategy 27

   Gradient Descent Quasi-Newton ◦ Nonlinear Conjugate Gradient (NCG) Superlinear rate of convergence can be achieved

d k

 

g

( 

k

)  

k d k

 1   Stochastic Gradient Descent Evolution Strategy 28

     Gradient Descent Quasi-Newton Nonlinear Conjugate Gradient Stochastic Gradient Descent (SGD) ◦ Similar to GD, but uses approximation of the gradient (Kiefer-Wolfowitz, Simultaneous Perturbation, Robbins-Monro) Evolution Strategy 29

     Gradient Descent Quasi-Newton Nonlinear Conjugate Gradient Stochastic Gradient Descent ◦ ◦ Evolution Strategy (ES) Covariance matrix adaptation Tries several possible directions (randomly according to the covariance matrix of the cost function), the best are chosen and their weighted average is used 30

 ◦ ◦ ◦ Data complexity Gaussian pyramid Laplacian pyramid Wavelet pyramid  ◦ ◦ Transformation complexity Transformation superposition Different B-spline grid density 31

  ◦ ◦ ◦ ◦ Registration toolkit based on ITK Handles many methods Similarity measures (SSD, NCC, MI, NMI) Transformations (rigid, affine, B-splines) Optimizers (GD, SGD-RM) Samplers, Interpolators, Multi-resolution, … http://elastix.isi.uu.nl

32