Transcript Context Enhancement of Nighttime Surveillance by Image Fusion

Context Enhancement of Nighttime
Surveillance by Image Fusion
Yinghao Cai
Kaiqi Huang, Tieniu Tan and Yunhong Wang
Center for Biometrics Research and Testing
National Laboratory of Pattern Recognition
Institute of Automation, Chinese Academy of Sciences
2006-8-21
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Outline
• Motivation
• Proposed Method
• Conclusions
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
(a) Nighttime Image
(c) Result of context enhancement
National
of Pattern
Recognition
(b)Laboratory
Daytime
background
Institute of Automation, Chinese Academy of Sciences
模式识别国家重点实验室
中国科学院自动化研究所
Motivation
•
•
Few work has been done on nighttime
surveillance.
Difficulties:
 Low contrast
 Low signal to noise ratio
 Limited environmental information( context
information)
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Solution
• The camera is fixed.
• Capture scenes of day and night at the same
viewpoint.
• Make use of the high quality background of
the day to help enhance the context of
nighttime images.
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Questions
• What information should be preserved in the fused
image?
 Moving objects
 Illumination effects
 Daytime background
Moving objects and illumination
effects preserve the fidelity of
the important information of the
nighttime video
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Framework
Nighttime image
Enhanced image
Illumination segmentation
Nighttime image
Final image
Daytime background
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Enhancement of nighttime video
• A tone mapping function is employed to nighttime
video enhancement [Bennett, 2005].
Where x is the pixel of the original nighttime video,
y is the value of the enhanced video,  is a parameter.
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Comparison with Gamma Correction
(a) Original video
(b) By [Bennett, 2005] ’s method
National Laboratory of Pattern Recognition
Institute of Automation, Chinese Academy of Sciences
(c) By Gamma correction
模式识别国家重点实验室
中国科学院自动化研究所
Motion Detection
• Gaussian mixture models [Stauffer, 99].
 Real time motion detection.
 Robust to variations in lighting, moving scene clutter,
multiple moving objects.
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
(a) Original nighttime video
(b) Enhanced video
(c) Motion detection of original video (d) Motion detection of enhanced video
C. Stauffer and W.E.L.Grimson, adaptive background mixture models for real-time tracking , CVPR 99
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Estimation of illumination characteristics
The image I can be represented by the product of reflectance of
the scene R, and illumination coming from the light source L.
I ( x, y )  R ( x, y ) L ( x, y )
In Retinex theory, the illumination can be considered
as the low frequency of image.
In this paper, we represent the illumination characteristics of the
nighttime image as the smoothed version of the original image.

=
Image
Reflectance
Illumination
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Image fusion
Where F is the final image, N is the nighttime image, D is
the daytime background. W is the weight:
Where M is the result of motion detection, L is illumination
characteristic. They are both in the range [0,1].
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Experimental Results
(a) Original nighttime image
(b) Result of context enhancement
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Experimental Results
(a) Original nighttime video
(b) Result of context enhancement
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Comparison with related work
[Li , 2005]’s method
Our method
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
Conclusions
• Simple but effective.
• Provides a real time and robust solution to frontend image pre-processing in nighttime
surveillance.
• The resultant image contains a more accurate and
comprehensive description of the scene which is
more useful for human visual and machine
perception, especially in surveillance.
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所
[email protected]
National Laboratory of Pattern Recognition
模式识别国家重点实验室
Institute of Automation, Chinese Academy of Sciences
中国科学院自动化研究所