Transcript Automated detection of cell connections in digital images

Automated detection of TNT in cell
images.
Erlend Hodneland, University of Bergen
Automated detection of TNT in cell
images.
Erlend Hodneland, University of Bergen
Automated detection of
TNTs(Tunnelling NanoTubes) in cell
images
Erlend Hodneland, University of Bergen
Automated detection of
TNTs(Tunnelling NanoTubes) in cell
images
Erlend Hodneland, Arvid Lundervold, Xue-Cheng Tai,
Steffen Gurke, Amin Rustom, Hans-Hermann Gerdes.
Erlend Hodneland, University of Bergen
3D session at fluorescence
microscope


Dimension : 520x688x40
Better resolution in xy
plane than in z direction.
Erlend Hodneland, University of Bergen
Two image channels

50
100
150
200
250
300
350

400
450
500
100
200
300
400
500
600
100
200
300
400
500
600
50
100
150
200
250
300
350
400
The channels appear
from biological stainings
of sample.
The stainings are photo
sensible to specific
wavelengths and
accumulate in certain
compartmens of the
cells.
450
500
Erlend Hodneland, University of Bergen
First channel displaying cell borders
and TNTs
Erlend Hodneland, University of Bergen
Gaussian noise and undesired
structures
Erlend Hodneland, University of Bergen
60
50
40
30
20
10
5 10 15 20 25 30 35 40
Video of image stack
Erlend Hodneland, University of Bergen
Second channel displaying cell
cytoplasma
Erlend Hodneland, University of Bergen
Biological relevance of TNTs




TNTs are until recently
unknown cell structures.
Play a role in cell to cell
communication.
Transport of virus?
Spread of cancer?
Cell 1
Cell 2
Virus moving?
Erlend Hodneland, University of Bergen
Automated detection of TNTs


A very challenging problem due to large variability
between images.
The basis methods are built up around
 Zerocross and Canny edgedetectors.
 Morphology incl. Watershed segmentation,
binary filling, dilation, erosion, closing and
opening.
Erlend Hodneland, University of Bergen
Morhpological operators*
T ranslateA by elementx,
defined as
(A)x  {c c a  x, for a  A}.
Reflectionof B is defined as
Br  {x x  b, for all b  B}.
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
Erlend Hodneland, University of Bergen
Morhpological operators*
T ranslateA by elementx,
defined as
(A)x  {c c a  x, for a  A}.
20
40
60
80
100
120
140
Reflectionof B is defined as
160
Br  {x x  b, for all b  B}.
200
180
20
40
60
80
100
120
140
160
180
200
20
40
60
80
100
120
140
160
180
200
20
40
Dilation:D( A, B) {x ((Br ) x  A)  Ø}
60
80
100
120
140
160
180
200
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
Erlend Hodneland, University of Bergen
Morhpological operators*
T ranslateA by elementx,
defined as
(A)x  {c c a  x, for a  A}.
20
40
60
80
100
120
140
Reflectionof B is defined as
160
Br  {x x  b, for all b  B}.
200
180
20
40
60
80
100
120
140
160
180
200
20
40
60
80
100
120
140
160
180
200
20
40
Erosion:E( A, B) {x (B) x  A}
60
80
100
120
140
160
180
200
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
Erlend Hodneland, University of Bergen
Morhpological operators*
T ranslateA by elementx,
defined as
(A)x  {c c a  x, for a  A}.
20
40
60
80
100
120
140
Reflectionof B is defined as
160
Br  {x x  b, for all b  B}.
200
180
20
40
60
80
100
120
140
160
180
200
20
40
60
80
100
120
140
160
180
200
20
40
Opening: O( A, B)  D( E( A, B), B)
60
80
100
120
140
160
180
200
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
Erlend Hodneland, University of Bergen
Morhpological operators*
T ranslateA by elementx,
defined as
(A)x  {c c a  x, for a  A}.
20
40
60
80
100
120
140
Reflectionof B is defined as
160
Br  {x x  b, for all b  B}.
200
180
20
40
60
80
100
120
140
160
180
200
20
40
60
80
100
120
140
160
180
200
20
40
Closing:C ( A, B) E( D( A, B), B)
60
80
100
120
140
160
180
200
*Serra, J 1982, Image analysis and mathematical morphology., Academic Press.
Erlend Hodneland, University of Bergen
Step #1 : Find cellular regions


Using canny edge
detector to find borders
of cells.
Edge detectors create
lots of broken parts, we
need to combine these
parts.
50
100
150
200
250
50
Erlend Hodneland, University of Bergen
100
150
200
250
300
Step #1 : Find cellular regions

Use morphological
closing and dilation to
combine edges into
closed regions.
Dilation
and closing
50
100
50
100
150
150
200
200
250
250
50
100
150
200
250
300
Erlend Hodneland, University of Bergen
50
100
150
200
250
300
Step #1 : Find cellular regions


Use morphological filling
to fill closed regions.
Cells shown as white,
filled regions.
50
50
50
Filling
100
100
100
150
150
150
200
200
200
250
250
250
50
100
150
200
250
300
Erlend Hodneland, University of Bergen
50
50
100
100
150
150
200
200
250
250
300
300
Step #1 : Find cellular regions

3-D representation of binary
cell image.
Combine
All planes.
50
100
150
200
250
50
100
150
200
250
300
Erlend Hodneland, University of Bergen
Step #2 Find important edges in cell
border channel


The first channel displays
TNTs and cell borders.
TNTs have low
intensities compared to
cell borders but they
have a large gradient.
50
100
150
200
250
50
100
150
200
250
300
50
100
150
200
250
300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #2 Find important edges in cell
border channel

Remove edges inside
cells.
50
100
150
200
250
50
50
100
100
150
150
200
200
250
250
50
100
150
200
250
300
Erlend Hodneland, University of Bergen
50
100
150
200
250
300
50
100
150
200
250
300
Watershed segmentation


A segmentation
procedure specially
designed for images with
natural minima.
A reliable segmentation
method, but it needs
suitable minima regions
as input for the region
growing.
50
100
150
200
250
300
350
400
450
500
100
200
300
400
500
600
Cell Cell
1100
1000
900
800
Background
700
600
500
400
300
200
0
100
Erlend Hodneland, University of Bergen
200
300
400
500
600
700
Watershed segmentation

Pathwise criterion of
Watershed lines W:
Region 1
min(W(a,b))
a
b
min(Ai(a,b))
For all Ai(a,b),
min(W(a,b)) ≥ min(Ai(a,b))
”Moving on the top of the hill”
Region 2
Region 3
Erlend Hodneland, University of Bergen
Watershed segmentation

The minima seeding
regions are extremely
important and decide
where the watershed
lines will appear.
Minima regions
50
100
150
200
250
50
100
150
200
250
300
Minima imposed on image
50
50
100
100
150
150
200
200
250
250
50
100
150
200
250
300
Erlend Hodneland, University of Bergen
50
100
150
200
250
300
Watershed segmentation

Results improve when
the minima seeding
regions are close to the
crest of the desired
structures.
50
100
150
200
250
50
Watershed image, {1,2 … 7}
1
6
50
100
150
200
250
300
The boundaries of cells
50
100
2
4
150
200
100
150
5
3
200
7
250
50
100
150
200
250
250
300
Erlend Hodneland, University of Bergen
50
100
150
200
250
300
Watershed segmentation

Results improve when
the minima seeding
regions are close to the
crest of the desired
structures.
50
100
150
200
250
50
Watershed image, {1,2 … 7}
1
6
50
100
150
200
250
300
The boundaries of cells
50
100
2
4
150
200
100
150
5
3
200
7
250
50
100
150
200
250
250
300
Erlend Hodneland, University of Bergen
50
100
150
200
250
300
Step #3 Watershed segmentation to
find crest of structures from edges

TNTs are thin and
narrow, approximately 34 pixles wide (50200nm).
50
100
1
150
200
250
50
10
100
150
200
250
300
1
20
TNT
30
40
50
60
70
80
10
Erlend Hodneland, University of Bergen
20
30
40
50
60
70
80
Step #3 Watershed segmentation to
find crest of structures from edges


Problem : The structures
from the edge image are
not always continuous
and they are not marking
the crest of the structure.
Solution : Use watershed
segmentation to create
connected lines on the
crest of the high
intensity structures.
50
100
150
200
250
50
100
150
200
250
300
50
100
150
200
250
300
50
100
150
200
250
Erlend Hodneland, University of Bergen
Step #3 Watershed segmentation to
find crest of structures from edges

Problem : The structures
from the edge image are
not always continuous
and they are not marking
the crest of the structure.
10
20
TNT
30
40
50
60
70
80
10
10
20
30
40
50
60
70
80
10
20
30
40
50
60
70
80
1
50
20
30
100
40
150
1
50
60
200
70
250
80
50
100
150
200
250
300
Erlend Hodneland, University of Bergen
Step #3 Watershed segmentation to
find crest of structures from edges



Important: TNTs can
cross several planes.
Therefore we use a
projection in 3-D  2-D
to include the whole
TNT.
All projections are
ranging over the same
planes as the structure
we investigate.
z
20
TNT
Cell 1
Cell 2
10
Erlend Hodneland, University of Bergen
y
x
Step #3 Watershed segmentation to
find crest of structures from edges
Plane 10
20
20
40
40
60
60
Plane 11
80
100
100
120
120
140
140
20
Plane 12
80
40
60
80
100
120
140
20
20
40
40
60
60
Plane 13
80
100
120
120
140
140
40
60
80
100
120
140
40
60
80
100
120
140
20
40
60
80
100
120
140
80
100
20
20
Erlend Hodneland, University of Bergen
Step #3 Watershed segmentation to
find crest of structures from edges

Using the maximum
projection of the
structure from the edge
image to take advantage
of 3-D information.
Maximum
projection
and closing
10
20
30
10
20
TNT
30
40
50
60
70
80
10
10
20
30
40
50
60
70
80
10
20
30
40
50
60
70
80
1
20
30
40
40
50
50
60
60
70
70
80
10
20
30
40
50
60
70
80
80
Erlend Hodneland, University of Bergen
Step #3 Watershed segmentation to
find crest of structures from edges

We use information from
the segmentation of cells
to construct minima
regions to seed the
Watershed segmentation.
10
20
TNT
30
40
50
60
70
80
10
20
30
40
50
60
70
80
70
80
10
50
20
100
Background
30
40
150
50
200
60
Cells
70
250
50
100
150
200
250
300
80
10
Erlend Hodneland, University of Bergen
20
30
40
50
60
Step #3 Watershed segmentation to
find crest of structures from edges
We use information from
the segmentation of cells
to construct minima
regions to seed the
Watershed segmentation.

Impose (1) on (2)
1
Morphological
opening
2
10
20
TNT
30
40
50
60
70
80
10
10
20
30
40
50
60
70
80
60
70
80
Minima regions
20
30
10
10
20
20
40
30
30
50
40
40
50
50
60
60
60
70
70
70
80
10
80
10
20
30
40
50
60
70
20
30
40
50
60
70
80
80
80
10
Erlend Hodneland, University of Bergen
20
30
40
50
Step #3 Watershed segmentation to
find crest of structures from edges

For Watershed we use
the sum projection of
the image to take
advantage of 3-D
information and for
Gaussian noise
supression.
Sum
projection
10
20
TNT
30
40
50
60
70
80
10
20
30
40
50
60
70
80
10
20
30
40
50
60
70
Image stack
80
10
Erlend Hodneland, University of Bergen
20
30
40
50
60
70
80
Step #3 Watershed segmentation to
find crest of structures from edges

10
20
TNT
30
40
50
60
70
80
10
10
20
20
30
40
50
60
70
80
Minima
regions
Watershed
segmentation
30
40
Using Watershed
segmentation to achieve
a connected line on the
crest of the structure
from the edge image.
10
20
TNT
30
40
50
50
60
60
70
70
80
10
20
30
40
50
60
70
80
80
10
Erlend Hodneland, University of Bergen
20
30
40
50
60
70
80
Step #4 Removal of false TNT
candidates

We end up with
numerous TNT
candidates, some false
and some true.
50
100
150
200
250
50
Watershed
segmentation
50
100
150
150
200
200
250
250
100
150
200
250
150
200
250
300
250
300
50
100
50
100
300
Erlend Hodneland, University of Bergen
TNT candidates
50
100
150
200
Step #4 Removal of false TNT
candidates


Each TNT candidate must undergo an evaluation of
correctedness. Remove candidates
 having low intensities compared to their
surroundings.
 not crossing between two cells.
 not beeing straigth lines using hough transformation.
 crossing at the nearest distance of the cells.
We are left with ”true” TNT structures after the
exclusion evaluation.
Erlend Hodneland, University of Bergen
Results


We have employed our algorithm to 51 3-D image
stacks:
 Success rate 67%
 False positive 50%
 False negative 33%
compared to manual counting.
The high number of false positive TNTs is mostly due
to large image variations and irregularities of the cells.
Erlend Hodneland, University of Bergen
Results


Large irregularites.
Main reason for false
positive or false negative
TNTs.
50
100
150
200
250
300
350
50
Erlend Hodneland, University of Bergen
100
150
200
250
300
Results


Large irregularites.
Main reason for false
positive or false negative
TNTs.
20
40
60
80
100
120
140
160
180
200
20
40
Erlend Hodneland, University of Bergen
60
80
100
120
140
160
180
200
Results


Nano experiments to
grow the cells on predefined matrices.
This will improve the
automated detection.
Erlend Hodneland, University of Bergen
Conclusion



We have developed an automated method for counting
TNTs in cell images.
The method is essentially based on existing image
processing techniques like edge-detectors, watershed
segmentation and morphological operators.
We report a success rate of 67% compared to manual
counting.
Erlend Hodneland, University of Bergen