Transcript Colour logarithmic CMOS image sensors

Modelling, calibration
and rendition of colour
logarithmic CMOS
image sensors
Dileepan Joseph and Steve Collins
Department of Engineering Science
University of Oxford
Outline



Logarithmic CMOS image sensors
Modelling sensor response
Image sensor calibration
 Fixed pattern noise
 Sensation of colour

Rendition of images
 CIE
 IEC

Lab (perceptual error)
sRGB (standard display)
Summary and future work
May 21-23
IMTC 2002
2
Logarithmic CMOS image sensors




CMOS displacing CCD
because of integration of
signal processing and
economies of scale
Logarithmic sensors offer
high dynamic range and
high frame rate
Linear sensors offer low
fixed pattern noise and
good colour rendition
Example images taken
from IMS Chips website
May 21-23
IMTC 2002
Linear CCD
sensor
Logarithmic
sensor
Logarithmic
sensor
3
Modelling sensor response

Since Ik = ∫ fk(λ) s(λ) dλ


And fk(λ) = gL(λ) gk(λ) gP(λ)



For spectral responses of lens gL(λ),
colour filter gk(λ) and photodiode gP(λ)
Approximating a linear combination of
three CIE XYZ basis functions
Then Ik = dk • x


For photocurrent Ik, spectral response
fk(λ) and light stimulus s(λ) at a pixel,
where k = R, G or B
For mask coefficients dk and tricolour
vector x, i.e. s(λ) in CIE XYZ space
Ideally, y = a + b ln (c + Ik) + ε


May 21-23
For digital response y of pixel with
offset a, gain b, bias c and error ε
Pixel-to-pixel variation of a, b or c
causes fixed pattern noise (FPN)
IMTC 2002
4
FPN calibration

Three types of FPN of interest:







Offset variation
Offset and gain variation
Offset, gain and bias variation
Partition pixels by colour filter to
permit FPN calibration of three
monochromatic sensors
Take images of uniform stimuli
under different illuminances
Calibrate each pixel’s response to
average response of all pixels by
least squares estimation of
varying model parameters
Fuga 15RGB sensor exhibits
offset, gain and bias variation
May 21-23
IMTC 2002
5
Colour calibration




Take and segment images of a
standard chart, having patches of
known CIE XYZ colour, under
different illuminances
Calibrate pixel responses to colour
by estimating non-varying model
parameters (e.g. mask dk), using
estimates of varying parameters
Ideal model fails for Fuga 15RGB
because absolute relationship
between y and Ik invalid (strong
inversion component?)
Empirical model y = a + b ln (c +
(α + dk • x)β) worked well, with no
change to relative responses of
pixels or FPN calibration
May 21-23
IMTC 2002
6
Image rendition (CIE Lab)




Images of a Macbeth Colour
Chart, taken by the Fuga 15RGB,
were rendered into CIE Lab space
with the calibrated empirical model
The perceptual error increases in
dim lighting as the bias term c
dominates the photocurrent Ik
Excluding the dimmest image (i.e.
5 lux), the error equals 12 over a
60 dB dynamic range for offset,
gain and bias variation
Images in Digital Photographer
show that conventional (linear)
digital cameras have an error of
15 over a 30 dB dynamic range
May 21-23
IMTC 2002
7
Image rendition (IEC sRGB)



A Fuga 15RGB image of the
Macbeth Chart, taken in 11 lux of
illuminance, was rendered into
IEC sRGB space with the
calibrated empirical model
Results for offset variation (topleft), offset and gain variation (topright), offset, gain and bias
variation (bottom-left) and true
colours (bottom-right) are shown
Two types of residual deviation for
the rendered patches are visible:


May 21-23
Fixed pattern noise
Colour desaturation
IMTC 2002
8
Summary and future work







Logarithmic image sensors offer high dynamic range and frame rate
Combine theories of colour linear sensors and monochromatic
logarithmic sensors to model colour logarithmic sensors
Calibrate FPN, using images of uniform stimuli, by relative
estimation of model parameters that vary from pixel to pixel
Calibrate colour, using images of a colour chart, by absolute
estimation of model parameters that do not vary
Fuga 15RGB results expose limitations of ideal model in absolute
estimation but reveal empirical model that works well
Macbeth Chart results show colour rendition with calibrated Fuga
15RGB competes with conventional digital cameras
Seek to minimise bias variation, so simple FPN models suffice, and
bias magnitude, to improve colour rendition in dim lighting
May 21-23
IMTC 2002
9
Acknowledgements

The authors gratefully acknowledge the support
of the Natural Sciences and Engineering
Research Council (Canada) and the Engineering
and Physical Sciences Research Council (UK)
May 21-23
IMTC 2002
10