Transcript Cellular Pattern Quantification and Automatic Bench

Cellular Pattern Quantification
and Automatic Bench-marking
Data-set Generation
on confocal microscopy images
Chi(Chrissie) Cui
Advisor: Joseph JaJa
Master of Engineering Thesis Defense
1
Outline
•
•
•
•
•
•
•
Problem Definition
Background
Feature Extraction
Feature Quantification
Acceleration using CUDA
Synthetic dataset generation
Conclusion and Future work
Master of Engineering Thesis Defense
2
Problem Definition
• Thesis goal:
– Extract patterns from confocal microscopy images of
giloma cells and build pattern descriptors
– Compute quantification measurements to profile the
responses of cells to different treatments.
– Focus on one cellular protein, the “F-actin”
– Particular interested: Schweinfurthin A(SA) , a
powerful inhibitor for brain tumors
Master of Engineering Thesis Defense
3
Problem Definition
• Major challenges
– What “features” to extract and how?
– Definition: General
– Extraction: Robust + Repeatable
– How to quantify the features and how to validate the
results
– Descriptive
– Intuitive
– Effective for profiling
Master of Engineering Thesis Defense
4
Background
• Actin
– A globular, roughly 42-kDa protein.
– Participate in many important cellular
processes , E.g. muscle contraction, cell
motility, the establishment of cell shape
and etc.
– G-Actin: Individual subunits of actin
– F-Actin: long filamentous polymers
assembled by G-Actin.
• Glioma
– a type of tumor that starts in the brain or
spine
Master of Engineering Thesis Defense
Fig 1. G-Actin
Fig 2. F-Actin; surface
representation of 13
subunit repeat based
on Ken Holmes' actin
filament model
5
Background
• Laser Scanning Confocal Microscopy
– Key Feature: Optical Sectioning (Fig. 5).
– Image Formation: The out-of-focus light is
suppressed by the pinhole. (Fig. 4)
Fig 3. Confocal Microscopy
in operation
Fig 5. An
example for
Optical
Sectioning
Fig 4. Imaging Principle of Confocal Microscopy
Master of Engineering Thesis Defense
6
Background
• DAPI(Nuclei), GFP(Green Fluorescent Protein), DMSO
• Schweinfurthin A(SA): (Fig. 6)
– Isolated and identified at the National Cancer Institute
– Cause dramatic changes in the organization of the actin
cytoskeleton
– 1000-fold selective for central nervous system tumor
cells.
Fig 6. Schweinfurthin A
Master of Engineering Thesis Defense
7
Background
• Data Preparation
– Mouse Glioma Cell Line
– 6 Drugs: cytochalasin B (cytoB), Y-27632 (Y),
OSW1, Schweinfurthin A (SA), cephalostatin 1
derivative (ceph), control group (DMSO)
– Focus: F-actin, stained by Green Fluorescent
Protein (GFP)
– 2 Separate Channels: cell nuclei + F-actin
– 2D Image analysis
Master of Engineering Thesis Defense
8
Previous Work
• “Interpretation of Protein Location Patterns
from Fluorescence Microscopy” - Prof. Robert
Murphy’s Group from CMU
• “Cell Profiler” – BROAD Institute
http://www.cellprofiler.org/
Fig 6. Robert Murphy’s Group
Fig 7. Cell Profiler
Master of Engineering Thesis Defense
9
Previous Work
• Comparison
cell profiler
Protein location
Pattern
Our work
Feature Extraction cell based
cell based
image based
Result Format
Discrete Class
labels
Continuous Numerical
Results
Classification Goal Tell positive from
negative
Different types of
proteins
Different organizational
and morphological pattern
of a single protein (Factin)
Motivation
Highlight the
difference
Show the conditional
variation trend
Binary Indicator
Highlight the
difference
Table 1. A comparison between our method and two previous work
Master of Engineering Thesis Defense
10
Analysis Framework
• Ratio Scalar Quantification
Fig 8. Flow Chart for obtaining the features regions
For Ratio Scalar Quantification
Master of Engineering Thesis Defense
11
Analysis Framework
• Cytoplasm Texture Analysis Quantification
Fig 9. The flow chart for obtaining cytoplasm Texture Analysis Quantification
• Vector Quantification [Nuclei Cytoplasm Relative Distribution]
Fig 10. The flow chart for obtaining nuclei cytoplasm relative distribution
Master of Engineering Thesis Defense
12
Features Extraction (Top Down)
• Tier 1: Tri-band
– Intensity based definition
– WBA (Brightest Part), Cytoplasm (Medium Brightness),
Background (Darkest Part)
Fig 11. Red :White Bright Actin (WBA); Green: Cytoplasm; Blue:
Background
Master of Engineering Thesis Defense
13
Feature Extraction (Top Down)
• Tier 2-a: Sub-Cellular Structure
– WBA Component Image
• Location Based
• Cortical Actin (Cell Boundary); Inner
Punctuate Dots (Inside Cells)
– Cytoplasm Component Image
• Image Enhancement + Thresholding
• Stress Fibers
Fig 12. Examples for the
images of cortical actin,
inner punctuate dots and
stress fibers inside the
cell
Master of Engineering Thesis Defense
14
Feature Extraction
• Multiscale Texture Analysis for Cytoplasm
– Quad-tree decomposition
– Multiscale Weighted Gray Scale Co-occurrence Matrix
Fig 13. MWGLCM Formulation: k denotes
the type of the measurement; S denotes
the maximum decomposition depth, Ns
denotes the number of patches associated
with scale s, Ws denotes the patch size at
decomposition level s and GLCMi,sk
denotes the type k GLCM measurement of
block i at decomposition level s
Master of Engineering Thesis Defense
15
Tri-band Segmentation
• Two Steps
• Pixel-wise Feature Vector Generation
• Fuzzy C Means
• Multiscale Point Wise Feature Extraction
– 5x5 Gaussian kernel
– Filtering three times
– Append the original Intensity
– Length 4 feature vector per pixel
Master of Engineering Thesis Defense
16
Tri-band Segmentation
• Fuzzy C means Segmentation
Master of Engineering Thesis Defense
17
Feature Quantification
• Goal
– Find the closest treatment
– Show perceived changes and trend between images
– Help make new discoveries
Master of Engineering Thesis Defense
18
Feature Quantification
• Scalar Quantification
– Single Measurement per image
– 10 Numbers
– Ratios, resolution oblivious
Fig 14. Scalar Quantification Measurement
Master of Engineering Thesis Defense
19
Feature Quantification
• Nuclei Cytoplasm relative distribution (Vector)
– Geodesic distance transformation based
Fig 16. Left: the cells treated under
cytoB in 10um/ml ; right: the cells
treated under DMSO.
Fig 15: the visualization of cytoplasm
nuclei relative distribution
quantification for cytoB
Master of Engineering Thesis Defense
20
Feature Extraction Validation
• By manual labeled image
• 19 images and selectively labeled
• 2 performance metrics
– Detected Rate
– Mis-detected Rate
• Better results than general recognition testing
C.A.
I.P.D.
S.F.
Detected Rate
100%
95.3%
98.7%
Mis-detected Rate
1.2%
6.5%
8.6%
Table 2. The computed detected rate and mis-detected rate for all the images
in the manual validation dataset. Here C.A. is short for cortical actin; I.P.D. is
short for inner punctuate dots and S.F. is short stress fibers.
Master of Engineering Thesis Defense
21
Feature Extraction Validation
• By Computer Simulated Images
• 5 difference noise conditions
• 100 image for each case
• Compute the ratio between the positively detected
region and the ground truth as the benchmark metric
Gaussian std.
0.002
0.004
0.008
0.01
0.02
C.A.
96.4%
94.5%
91.2%
89.7%
86.9%
I.P.D
88.4%
86.8%
86.1%
83.8%
78.5%
S.F.
89.5%
87.5%
86.4%
85.0%
82.2%
Table 3. the detected rate for the computer generated images
for the three sub-cellular structures.
Master of Engineering Thesis Defense
22
Feature Quantification Validation
• Find closest treatment to single treatment
– E.g. SA (10, 100, 1000 nm/ml)
– Two Facts:
• 23'deoxy-ceph 1 has similar effects on F-actin as SA
• OSW-1 has similar effects on F-actin as SA
– Visualization: Bar Graph (Next 3 Pages)
– Conclusions:
• SA(10nm/ml) is the most similar to OSW1(1nm/ml)
• SA(100nm/ml) is the most similar to ceph(1nm/ml)
• SA(1000nm/ml) is the most similar to SA(100nm/ml)
Next
Master of Engineering Thesis Defense
23
Feature Quantification Validation
Fig 17. SA(10nm/ml) is the most similar to OSW1(1nm/ml)
Back
Master of Engineering Thesis Defense
24
Feature Quantification Validation
Fig 18. SA(100nm/ml) is the most similar to ceph(1nm/ml)
Back
Master of Engineering Thesis Defense
25
Feature Validation Quantification
Fig 19. SA(1000nm/ml) is the most similar to SA(100nm/ml)
Back
Master of Engineering Thesis Defense
26
Feature Quantification Validation
• “Closeness” Extension: Pair-wise visualization
Fig 20. The pair-wise
distance map of all the
treatment conditions in
our experiment. Darker
color means smaller
distance, in other word,
greater similarity while
brighter color means
less in common. The
closest treatment for
each row is highlighted
by blue dot
Master of Engineering Thesis Defense
27
Feature Quantification Validation
• Show trend in series
Fig 21. The quantification result of
the ratio of the stress fiber area to
the cell area of SA
Fig 22. The images corresponding to the
quantification result Fig.6.4.
(UL) DMSO
(UR) SA(10nm/ml)
(LL) SA(100nm/ml) (LR) SA(1000nm/ml)
Master of Engineering Thesis Defense
28
Feature Quantification Validation
• Make “new discoveries” beyond observation
– E.g. Weighted multi-scale Cytoplasm Texture Energy
– Proved by non-optical methods
Fig 23. The weighted multi-scale cytoplasm texture
energy for ceph
Master of Engineering Thesis Defense
29
CUDA Introduction
Fig 24. CUDA Programming Model
(Courtesy Image from NVIDIA)
Fig 25. CUDA Memory Model
Master of Engineering Thesis Defense
30
Accelerating using CUDA
Fig 26. computation step and memory layout
of the point wise feature extraction step.
Table 4. the pseudo code of the
Point Wise Feature Extraction Step.
Master of Engineering Thesis Defense
31
Accelerating using CUDA
Fig 27. The memory layout for
the membership matrices
Table 5. The pseudo code of the Fuzzy C
Means Segmentation Step.
Master of Engineering Thesis Defense
32
Accelerating using CUDA
• Experimental Hardware Environment
– CPU: Intel(R) Core(TM)2 Duo T5800
• 2.00GHz each core + 2.00GB memory
• OS: 32 bit Windows Vista
– GPU: NV9600 MGT
• 512MB device memory
• 1.25GHz clock rate
• 4 multi-core processor per die.
• Experimental Software Enviroment
– Matlab V7.1.4
– CUDA 2.1
Master of Engineering Thesis Defense
33
Accelerating using CUDA
Table 6. the execution configurations of all the kernels.
"Width" and "Height“ denotes those of the input image.
Table 7. The comparison of the runtime results of the CUDA version
and Matlab implementation of our tri-band Segmentation method.
Master of Engineering Thesis Defense
34
Synthetic Dataset Generation
• Bottom up Approach
• Component Images:
– Cell Bodies
– Cortical Actin (No punctuate dots)
– Stress Fibers
• Post Processing
–
–
–
–
Diffusion
Add Poisson Noise
Add Gaussian Noise
Convolute with PSF
Fig 28. The computational framework of
our F-actin highlighted confocal
microscopy image simulation
Master of Engineering Thesis Defense
35
Synthetic Dataset Generation
Fig 29. Left to Right:
1) distance map generated from the cell nuclei mask
2)Watershed segmented color labeled image
3) image after selecting the visible part with probability p and
erode the image with disk shape structuring element.
Fig 30. Left to Right: 1) Generated cortical actin mask.
2) Generated stress fiber mask. 3) Final Result
Master of Engineering Thesis Defense
36
Synthetic Image Generation Validation
• Synthetic Benchmark Dataset
– Apply feature extraction methods to the synthetic images
– Use the preciseness as a metric
WBA
CYTO
BK
C.A.
S.F.
Average 1.005
0.998
1.00
1.023
1.106
0.03
0.005
0.03
0.08
s. t. d.
0.02
Table 8. the preciseness of the tri-band segmentation and subcellular structure extraction algorithm when applied to the
artificial images. C.A. denotes cortical actin and S.F. denotes
stress fibers.
Master of Engineering Thesis Defense
37
Synthetic Image Generation Validation
• Synthetic Benchmark dataset
Fig 31. A: Image of Factin labeled cells. B:
tri-band segmentation.
C: Simulated image of
F-actin in cells. D: Triband segmentation of
the simulated image.
Master of Engineering Thesis Defense
38
Conclusion and Future Work
• Introduce an image analysis framework for quantifying
the F-actin organization patterns in confocal microscopy
images.
• Decide what measures to be taken for quantification so
as to make the final results more reliable and descriptive
• Map Tri-band Segmentation to CUDA and obtain 15 ~ 20
speed up on different sized images
• Propose a framework for simulating 2-D F-actin confocal
microscopy images.
• Future work: running more validations to test the
effectiveness of the quantitative measurements
Master of Engineering Thesis Defense
39
Thank you!
Questions?
Master of Engineering Thesis Defense
40