Content-based Retrieval of 3D Medical Images

Y. Qian, X. Gao , M. Loomes, R. Comley, B. Barn School of Engineering and Information Sciences Middlesex University, UK

R. Hui, Z.Tian

Department of Neurosurgery, General Navy Hospital, P.R.China

Contents

1. Background 2. Methodology 3. Experiment Results 4. Conclusion and Future Work

1. Background

MIRAGE ( M

iddlesex medical

I

mage

R

epository with a CBIR

A

rchivin

G E

nvironment

)



Aim: To develop a repository of medical images benefiting MSc and research students in the immediate term and serve a wider community in the long term in providing a rich supply of medical images for data mining, to complement MU current online e-learning system.



So far 100,000 2D images and 100 images in 3D form.

 http://image.mdx.ac.uk/

JSIC

 

Innovation in the use of ICT for education and research.

http://www.jisc.ac.uk/

Content-Based Image Retrieval (CBIR)

CBIR can index an image using visual contents that an image is carrying, such as colour, texture, shape and location.

e.g. Query by Example Image(QBE)

Proposed Framework for MRIAGE

GIFT Framework

GIFT(GNU Image Finding Tool) is open framework for content-based image retrieval and is developed by University of Geneva.

  

Query by example and multiple query Relevance Feedback Distributed architecture (Client - Server)

Demo :

Content-Based 3D Brain Image Retrieval 2D brain images ----- 3D Brain



Shape-based

Surface of a 3D object(e.g. tumor) 

Texture-based

Inside of a 3D object( e.g.textures representing tissue structure properties

Aim: To develop a fast texture-based 3D brain retrieval method

2. Methodology

Proposed framework for 3D image retrieval

Pre-processing 1) Spatial Normalization---Statistical Parametric Mapping (SPM5 )

Transform each individual brain into a standard brain template

2) Divide 3D brain into 64 non-overlapping equally sized blocks

Extraction of Volumetric Textures

1) 3D Grey Level Co-occurrence Matrices (3D GLCM) 2) 3D Wavelet Transform (3D WT) 3) 3D Gabor Transform (3D GT) 4) 3D Local Binary Pattern (3D LBP)

1) 3D Grey Level Co-occurrence Matrices (3D GLCM)

3D GLCM is two dimensional matrices of the joint probability of occurrence of a pair of gray values separated by a displacement

d = (dx,dy,dz)

.

  

52 Displacement vectors: 4 distance * 13 direction = 52 4 Haralick texture features: energy, entropy, contrast and homogeneity Feature vector: 208 components (=4 (features) * 52 (matrices)).

2) 3D Wavelet Transform (3D WT)

3D WT provides a spatial and frequency representation of a volumetric image.   

2 scales of 3D WT Mean and Standard deviation Feature vector: 30 components (2 (features) +15 (sub-bands))

3) 3D Gabor Transform (3D GT)

3D GT generates a set of 3D Gabor filters

Gabor filters g



x

,

y

,

z

,

F

,  ,    ^

g



x

,

y

,

z

 exp 

j

2  

F

sin  cos 

x



F

sin  sin 

y



F

cos 

z

 

Gabor Transform: GT i



f



x

,

y

,

z



144 Gabor filters x

,

y

,

z

,

F i

, 

i

, 

i



i

 1 , 2 , 3 ...

144

4 (

F) *6( θ)*6(Φ) =144  

Mean and Standard deviation Feature vector: 288 components (2 (features) +144(filters))

4) 3D Local Binary Pattern (3D LBP)

Local binary pattern(LBP) is a set of binary code to define texture in a local neighbourhood. A histogram is then generated to calculate the occurrences of different binary patterns.

 

59 binary patterns Feature vector: 177 components (=59(patterns)*3(planes)

Similarity Measurement

 Histogram Intersection(3D LBP)

D



Q

,

I

  

i

min 

Q i

,

I i

  Normalized Euclidean distance (3D GLCM,3D WT,3D GT)

D



Q

,

I

  

i Q i

 

i I i

2

Lesion Detection Assume bilateral symmetry of a normal brain along its mid-plane

3. Experimental Results

Test Dataset



100 MR brain images



Size: 256



256



44



DICOM (Digital Imaging and Communications in Medicine) format



Collected from Neuro-imaging Centre at Beijing General Navy Hospital, China

Experimental Results ------ Lesion Detection

Experimental Results -------Retrieval

Experimental Results -------Query time

4. Conclusion and Future work

1) Conclusion:



Comparative results demonstrate that LBP outperforms four 3D texture methods in terms of retrieval precision and processing speed.



The query time with VOI selection offers 4 times faster operation than that without.

2) Future work:



Test on the larger dataset



Plug 3D image retrieval into GIFT framework (MIRAGE 2011)

Thank You.