Transcript Document

Digital Image Analysis in Flow
Think outside the dot.
Visual Analysis: Nuclear Translocation
• Does treatment cause nuclear
translocation of NFkB?
Untreated
Treated
• Decision is immediate and based on
interpretation of image collected by eye
Digital Imaging in Flow
7/16/2015
Visual Analysis: Nuclear Translocation
• Does treatment cause nuclear
translocation of NFkB?
• Decision complicated by variation
• Despite the high information content of the
images, the interpretation is qualitative
Digital Imaging in Flow
7/16/2015
Digital Image Analysis
• Some applications require visual
observation of changes
• Images contain lots of information per
object
• Desire to convert image into numeric
score that reflects the perceived change
(Quantitation)
• Desire to measure large numbers of cells
per sample
(Statistical power)
Digital Imaging in Flow
7/16/2015
Pathway to quantitative image analysis
• Numerical measurement of features
associated with an image; performed on
many images per sample
• To achieve this:
Convert cells
into photons
Convert photons
into digits
2
Digital Imaging in Flow
3
3
Convert digits
into features
Plot features of
cell populations
2
2
37
121
2
2
43
18
1
1
2
2
2
Area = 753
Aspect Ratio = 0.84
Centroid X = 43
Intensity = 153,119
7/16/2015
Traditional Flow cytometry:
Architecture
Cells maintained in
fluidic suspension
Pressure differential
Hydrodynamic focus
Focused laser
irradiation of cells
Photomultiplier
signal capture
Digital Imaging in Flow
7/16/2015
Traditional microscopy and cytometry
• Traditional Flow cytometry
• Quantitation of probes on many cells
but:
• zero spatial resolution: no location information
• Fluorescence microscopy
• Good spatial resolution
but:
• Measurement of only a small numbers of cells
Digital Imaging in Flow
7/16/2015
ImageStream Flow cytometry:
Architecture
detector
Spectral
decomposition
element
cells in flow
autofocus
brightfield
illuminator
Digital Imaging in Flow
velocity
detector
laser
7/16/2015
Converting cell into photons
Although
some of
ultra-structural
details
Stain components
cell with fluorescent
dye can
be observed
lookingsource
at brightfield
Expose
the cell toby
excitation
images of cells, precise molecular
Fluorescent dye releases photons
identification is achieved by labeling
Resulting
photons
by the
specimen
molecules
withemitted
specific
probes
thatare
can
projected onto the detector
generate a detectable signal.
Probes used in imaging and flow cytometry
genererate light (photons) when excited
by a source.
Light source
Digital Imaging in Flow
Specimen
Emission photons
7/16/2015
Magnification
Because detector units are larger than the
structures to be analyzed, the photons
emmanating from the sample must be
spread from one another (magnified).
Add microscope
Photon
projection,
objective
no mag to magnify
Object
Objective
Photons
Photons
= spatial
no spatial
resolution
resolution
Digital Imaging in Flow
Detector Array
7/16/2015
Magnification
Light gathering ability of objective = numeric
aperture (NA)
Increased NA allows for increased optical
resolution (magnification). This also
decreases depth of focus.
Ideally, the optical resolution of the
microscope should match the spatial
resolution of the detector
Digital Imaging in Flow
7/16/2015
Converting photons into digits
• Advantages
Detector used
of for
digital
capturing
imaging:
images =
camera.
• Data generated is quantitative
• Traditional
Easy to communicate
silver halide-based cameras convert
incoming
photons
to image
on film
• incorporate
into docs
(papers/presentations)
• E-mail/website
• Digital
cameras convert incoming photons into
• No scanner required (as for film-based images)
digits.
• Easy to copy/store/archive
• Feedback is nearly instantaneous (no film
development time)
• More sensitive
• Comparable spatial resolution
Digital Imaging in Flow
7/16/2015
Digital camera detector: the CCD
• Consists of a rectangular array of detectors (pixels)
in a silicone wafer (semiconductor)
• Each pixel acts as a:
• Photodiode (converts incident photons into electrons)
and
• A charge storing device
• After time, pixels transfer the stored electrons to an
amplifier which converts the current into analog
voltage
• Analog to digital converter translates the voltage to
digital number
• The entire process maintains spatial registry,
meaning each digit represents the intensity of light
coming from a spatially distinct region of the cell
Digital Imaging in Flow
7/16/2015
Digital camera detector: the CCD
1. Photons strike polysilicon gate on the face of the pixel; electrons
liberated into the potential well
2. After defined time, electrons conducted to on-chip amplifier which
boosts the electron signal and converts the current into analog voltage
3. Analog to Digital converter translates the voltage into a digital
number for each pixel and digital value is stored in computer memory
2
3
3
2
2
37 121
2
2
43
18
1
1
2
2
2
Incident photons
Digital Imaging in Flow
7/16/2015
Digital camera detector: the CCD
1. Integrate light to build charge in the pixel wells
2. Shift charge one row
down, dumping charge from
bottom of imaging area to
the serial array
3. Charge moved horizontally:
left-most pixel delivered to Amp
4. Amp converts to signal, ADC
to digital data
Parallel array
5. Once serial array is cleared,
parallel array shifts one row
down
Serial array
Amp, ADC
Digital Imaging in Flow
7/16/2015
Digital camera detector: the CCD in TDI
mode
1. Integrate light to build charge in the pixel wells
2. Shift charge one row down in synch with the
object passing in front of the detector, while
continuing to integrate light.
3. Amp and ADC convert charge
dumped to the serial array to build
digital image.
Parallel array
Amp,
ADC
Serial array
Digital Imaging in Flow
7/16/2015
Photonic Sensitivity
• Ability of a system to convert photons
arising from specimen into signal
• Sensitivity = Signal:Noise
S:N =
Quantum Efficiency (QE) x time x photon flux
Dark current + Read noise + Photon noise
• Increase sensitivity by increasing signal or
decreasing noise
Digital Imaging in Flow
7/16/2015
Photonic Sensitivity
Photon dependent components
time x photon flux
S:N =
Dark current + Read noise + Photon noise
QE x
• Photon flux = photons per second
• Photon noise
• Statistical variation of photon arrival rate
• Equals the square root of the photon signal
• As signal increases, S:N derived from signal increases
Digital Imaging in Flow
7/16/2015
Photonic Sensitivity
Detector dependent components
S:N =
QE x time x photon flux
Dark current + Read noise + Photon noise
• QE = photons-to-electron conversion efficiency
• Property of atomic structure of detector
• Dark Current = electrons liberated in absence of light
• Thermal
• Stray light
• Proportional to surface area of detector
• Also creates Dark noise (analogous to photon noise)
• Correctable in software and/or cooling the detector
• Read noise = current to digits
• Amplifier that converts current to analog voltage
• A/D converter
• Proportional to the square root of number of reads (Poisson)
Digital Imaging in Flow
7/16/2015
Photonic Sensitivity: CCD vs PMT
QE x time x photon flux
S:N =
Dark current + Read noise + Photon noise
• CCD
• High QE across visible light spectrum
• Low dark current per pixel due to size, correctable in software
• Read noise proportional to number of pixels per image
• Excellent at storing and transferring charge
• Spatial information intact
• Traditionally used to look at changes over time
• PMT
• Massive electron amplification in tube prior to readout
produces large signal from low light input
• Higher dark current due to size
• Used for detection of weak signals
Digital Imaging in Flow
7/16/2015
ImageStream Sensitivity
QE x time x photon flux
S:N =
Dark current + Read noise + Photon noise
• ImageStream uses CCD but exposes light for short
times to get high throughput, which is traditionally a
PMT specialty. Here’s how it does it with comparable
sensitivity:
• Increase photon flux with powerful laser
• Decrease fluidic rate (100-200 nL/sec)
• TDI enables extended light integration time of moving object
without streaking (analogous to dynode system = readnoiseless amplification)
• Multimode imagery allows concentration of measurement on
pixels only within the object, reducing impact of noise on
intensity calculation (more later…)
Digital Imaging in Flow
7/16/2015
Pixel dynamic range
Maximum
Full well signal
capacity
capacity
(FWC) of pixel
Dynamic Range =
Dark
Detector-associated
current + Read noise
pixel noise
For example,
Max
capacity if
= FWC
Full Well
= 120,000
Capacity
electrons
= Number
withof100
electrons
electrons
it takes
of
to fill athe
noise,
pixel
analog
well dynamic range is 1200 and a 10 bit A/D
conversion would quantize that signal into a digital dynamic
Pixel noise = Dark Current + Read noise
range of 1024 counts
A/D converter quantizes the signal into bits, and the digital data
is matched to the dynamic range of the sensor output.
Saturation level
Pixel
Amplifier
Digital Imaging in Flow
AD convertor
7/16/2015
Image Signal Properties are SizeDependent
In PMT-based flow cytometers, sensitivity and dynamic range
are independent of the size of the object. This is not true with
a pixilated imaging detector.
• Each pixel is an independent noise source
• Spreading a signal over multiple pixels increases noise
• Each pixel has an independent saturation limit
• Spreading a signal increases the saturation limit
Digital Imaging in Flow
7/16/2015
Dynamic Range vs Object Size
Dynamic range is proportional to cell size
How?
bits perxpixel
N x 2# Mpixels
2 bits per pixel
Maximum
2signal
capacity
of pixel
Dynamic Range per pixel
=
object =
Dark
+ Read
noise
total
Dark
Detector-associated
noise
pixel
noise
√Ncurrent
xcurrent
Sn + Read
• Let M = number of A/D bits
• Let N = number of pixels
• Dark Current = correctable with software
• Read noise = governed by Poisson statistics
• Read noise per pixel (Sn) = 1
• Read noise total = square root of number of reads x Sn
Digital Imaging in Flow
7/16/2015
Dynamic Range vs Object Size
• M = A/D bits
3 µm
6 µm
• Sn = noise per pixel = 1
9 µm
cell diameter (0.5 µm pixel)
12 µm
18 µm
32,768
Signal max
N * 2M

Noise
N * Sn
14
16,384
13
8,192
12
4,096
11
2,048


1N * 2 M
N *2 2N
Eq.Bits  log
M 22  log
2 N *1 
M
DN
15
equivalent bits
• N = number of pixels
Conclusion:
Dynamic range is 10
directly proportional 0
to cell diameter
Digital Imaging in Flow
1,024
100
200
300
400
500
600
700
800
900
1000
number of pixels
7/16/2015
Sensitivity vs Object Size
Sensitivity is inversely proportional to cell size
Math:
For example, 200 counts concentrated in one pixel
has a signal of 200 with only one read.
• Set minimum signal at detection limit = 3 x Read noise total
If, on the other hand, the same 200 counts is spread
• Let N = number of pixels
four pixels,
each
has
a signal of only 50
• over
Read noise
= governed
by pixel
Poisson
statistics
counts,
and four
reads
• Read noise
per pixel
(Sn)are
= 1 required.
• Read noise total = sq root of number of reads x Sn = √N
Minimum signal at detection limit = 3 x √N
Digital Imaging in Flow
7/16/2015
Sensitivity vs. object size
3 µm
6 µm
cell diameter (0.5 µm pixel)
9 µm
12 µm
18 µm
100.00
10.00
Read noise total = 1 X √N
Detection limit = 3 x total noise
Detection limit = 3 x √N
Min. signal/pixel: 3 / √N
digital numbers
Read noise/pixel (Sn) = 1
1.00
0
100
200
300
400
500
600
700
800
900
1000
0.10
Conclusion:
Detection limit is inversely
proportional to cell diameter
Digital Imaging in Flow
0.01
number of pixels
7/16/2015
Sensitivity Comparison to
Conventional Flow
6 µm diameter rainbow beads: FITC Channel
18 bit cytometer
Digital Imaging in Flow
ImageStream
7/16/2015
Sensitivity Comparison to
Conventional Flow
3 µm diameter rainbow beads: FITC Channel
Rd = 31.2
Rd = 10.4
18 bit cytometer
Digital Imaging in Flow
ImageStream
7/16/2015
How many bytes per object?
AD converter writes voltage as binary digits (bits) to
memory
1 unit of memory (byte) holds 8 bits of data, so a pixel
with 8 bit dynamic range will be occupy 1 byte of
memory.
ImageStream example: How many bytes per
object?
Object = 6 images x 882 pixels/image = 45,000 pixels
AD dynamic range is 1024 (10 bit) = 1.25 bytes (stored
in 2 bytes) = 90 kilobytes per object
5:1 loss-less compression = 18 KB per object
10,000 event file ~ 180 MB
Digital Imaging in Flow
7/16/2015
Optical and digital resolution and
sensitivity: Binning
Binning =
Pooling charge collection from more than one
pixel on the chip
Advantage – increase signal per pixel &
decrease read noise per object
Disadvantage – lower digital resolution
Reason to do it – decrease signal integration
time (speed up object collection rate) while
preserving sensitivity
Digital Imaging in Flow
7/16/2015
Optical and digital resolution and
sensitivity: Larger pixel array
Larger pixel array =
More pixels per image
Advantage – increase digital resolution
Disadvantage – decreased signal per pixel
and increased read noise per object
For higher digital resolution, increase the
signal integration time (slow down object
collection rate) while preserving sensitivity
If digital resolution is greater than optical
resolution, there is only the downside of
lower signal to noise
Digital Imaging in Flow
7/16/2015
Converting digits into features
• The AD Converter records the value
representing the amplitude of the analog signal
(intensity) and the x and y coordinates of the
signal to the acquisition file
• Therefore the collected images have been
converted into a spatially registered array of
digital values
• What, then, is a feature?
• Specific characteristic that can be used to classify
an object
• With digital image data, algorithms can be applied to
the pixel values to quantify object features
Digital Imaging in Flow
7/16/2015
Converting digits into features
Cell 1 appears larger and more elongated than cell 2
Features = Size and Circularity
(aspect ratio)
Computer first scans the digital array
of the image box looking for localized
increases in pixel value standard
deviation
Then ‘segments’ the so indicated
region of interest (ROI), capturing the
set of pixels that define the cell within
a ‘segmentation mask’
Digital Imaging in Flow
2
3
1
3
2
2
3
3
2
3
23
19
5
2
2
1
11
64
75
9
1
1
2
23
543 102 4
2
2
2
29
35
3
1
1
3
621 219 26
23
1
1
3
5
6
4
3
2
2
3
1
3
2
2
3
3
2
3
1
2
2
2
2
1
8
64
75
3
1
1
2
30
179 145 4
2
2
2
13
365 200 3
1
1
3
4
5
2
1
1
1
3
2
2
1
3
2
31
7/16/2015
Converting digits into features
Cell 1 appears larger and more elongated than cell 2
Area = sum of pixels in the mask
Aspect Ratio = minor axis:major axis
ratio of best-fit ellipse to the mask
The data can be plotted for all collected objects
for robust population-based statistics
Digital Imaging in Flow
7/16/2015
Increasing sensitivity using multimode
imagery
Segment
Detect
and
image
capture
in
multimode to
brightfield
imagery
determine
including
area
and location
brightfield,
for
darkfield
signal
detection
and
in
fluorescence channels.
Apply
brightfield
Load Channel
3 mask to
fluorescence
fluorescencechannel.
image
Sum
signal
pixelall
array
for under
mask.
Determine mean
processing.
background signal.
• Cross channel
information is used to
enhance sensitivity
beyond quantization
level where signals
cannot be segmented.
• Sub-bit signals can be
Feature
Value
3_Backgnd Mean Intensity 30.03
3_Backgnd StdDev Intensity 0.85
3_Intensity
47.12
6_Area
174
Digital Imaging in Flow
Subtract product of
mean background and
mask area from Sum to
determine fluorescence
intensity.
quantified over multiple
pixels
• Example: Average
intensity is 0.3 counts
7/16/2015
Digital Image Analysis
75
Percent Translocated
Percent Translocated
Nuclear Translocation dose response and time course
50
25
100
75
50
EC50 = 6.31 ng/ml
25
0
0
0
15
30
45
60
Time (min)
75
90
-8
-7
-6
-5
-4
-3
-2
Log [LPS] (mg/ml)
High speed digital imaging enables per cell feature
quantitation and population statistics
Digital Imaging in Flow
7/16/2015
Quantitative Image Analysis
• Numerical measurement of features
associated with an image; performed on
many images per sample
• Components
• Excitation light source
• Light (photons) derived from the sample is
magnified and projected on the detector array
• CCD Detector (converts photons into digits)
• Features related to digital data are calculated and
plotted
Digital Imaging in Flow
7/16/2015
The ImageStream System
• ImageStream Imaging Flow Cytometer
Brightfield, scatter, and 4 fluorescent images at >15,000 cells/minute
• IDEAS® Statistical Image Analysis Software
Quantitative cellular image analysis and population statistics
• Novel Applications
Translocation, co-localization, cell classification, apoptosis, etc.
Digital Imaging in Flow
7/16/2015
A “Universal” Relative Sensitivity
Metric
Rd = 1.0
Rd = 0.5
1.8
1.8
1.6
1.6
1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
-3
-2
-1
0
1
2
3
4
5
6
7
-3
-2
-1
0
1
2
3
4
5
6
3
4
5
6
Rd = 1.5
Rd = 1.2
1.8
1.8
1.6
1.6
1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
Md  Mu
Rd 
d u
0
0
-3
-2
-1
0
1
2
Digital Imaging in Flow
3
4
5
6
-3
-2
-1
0
1
2
7/16/2015
ImageStream CCD array
Custom TDI Camera: 6 Ch Acquisition
Inactive Region
12 pixels wide by
512 tall
X6
Active
Channel
Active
Channel
96 pixels
wide by
96 pixels
512 tall
wide
by
X6
512 pixels
tall
Inactive
area
12 pixels
wide by
10 bit A/D / QE > 90% /  < 100 e- / 150K e- FWC
52 KHz max line rate, 30Mpixels / sec
Stage selectable integration
1X, 2X, 4X, 8X on chip binning
10X high gain mode, 18m x 18m pixels
X6
X6
256 FWD Stage Selection
z
128 FWD Stage Selection
32 FWD Stage Selection
8 Stage Selection
Single or multitap readout register suitable for 50KHz line rate
18 tap register suitable for 400KHz line rate
Digital Imaging in Flow
7/16/2015