Transcript A Robust Scene-Change Detection Method for Video Segmentation

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
A Robust Scene-Change
Detection Method for Video
Segmentation
Chung-Lin Huang and Bing-Yao Liao
Outline
•
•
•
•
•
Introduction
Abrupt Scene-Change Detection
Gradual Scene-Change Detection
Experimental Results
Conclusion
Introduction
• The main problem of segmenting a video
sequence into shots is the ability to distinguish
between scene breaks and normal changes that
happen in the scene
• This paper combines the intensity and motion
information to detect scene changes such as
abrupt scene changes and gradual scene changes
Previous Problems
• The two main problems in most existing algorithms
– they are threshold-dependent algorithms
– they suffer false detection with scenes involving fast camera
or object motion.
• This paper proposes a scene-change detection
algorithm with three contributions
– Relaxing threshold selection problem
– higher detection rate (scene change should not be missed)
– lower false alarm rate
ABRUPT SCENE-CHANGE
DETECTION
• Method
– Measurement of the Changes Between Frames
• Pixel-Based Difference
• Histogram-Based Difference
– Static Scene Test
– Scene Transition Classification
– Detection Algorithm
Detection Algorithm
• First phase
– locate the highest and the second highest peaks of DCimage
difference in the midst of the sliding window, and then
calculate the ratio n between the first and second peaks
Nhigh
genuine
Ambiguous
Nlow
No Scene Change
• Second phase
– Histogram Measure
– Static Scene Test (Most of the false alarms declared by the histogram
detector are due to sudden light changes, while the edge information is
more or less invariant to these changes)
– Scene Transition
Measurement of the Changes
Between Frames
• Pixel-Based Difference:
–
Where Cx and Cy are the DC image of frames X and Y
• Histogram-Based Difference: (X2 Test)
–
Color Histogram
• Efficient representation
– Easy computation
– Global color distribution
• Insensitive to
–
–
–
–
Rotation
Zooming
Changing resolution
Partial occlusions
• Disadvantage
– Ignore spatial relationship
– Sensitive in illumination changes
• Choose illumination-insensitive color channels
Example
• Color space selection & quantization
– Use RGB channels
• Each channel is divided into 2 intervals
• Total number of bins = 23 = 8
– H(I): Color histogram for Image I
– H1 = (7, 7, 7, 7, 9, 9, 9, 9)
• Image 1 has 7 pixels in each C1 to C4, and 9 pixels in each
C5 to C8
Static Scene Test
• Define
– all objects present in the scene exhibit rather small motion
compared to the frame size, and global movement caused by
the camera is slow and smooth.
• Method
– Edge Detection
– Edge Dilation
• Result
– The transition of two consecutive frames with covering ratio
larger than a predefined threshold is considered as a nonstatic or dynamic scene.
Example of edge detection
Edge
Detection
Edge Dilation ( r=3 )
Scene Transition Classification
• Transition Type
– 1) static scene to static scene
– 2) dynamic scene to static scene or vice versa
– 3) dynamic scene to dynamic scene
• Dynamic-to-dynamic transition usually indicates
continuous object or camera motion, rather than a real
scene change
Gradual Scene-Change Detection
• Gradual Scene-Change
– Dissolve
– Fade-In
– Fade-Out
(X=0)
(Y=0)
Scene X to Y
In Duration T
• Why not easy to detect
– Camera and object motions always introduce a larger
variation than a gradual transition.
Intensity Statistics Model
• Normal Case
– For any frames near the reference frame, their
dissimilarity measure almost increases exponentially
with their distance
• Gradual Transition
– The dissimilarity increases linearly with their distance
during the transition
– After the transition is over, the difference measures
are randomly distributed
Normal Sequence
Gradual Transition
Seed : the beginning frame of a
gradual transition
• N-distance measure
– the difference measure generated by comparing a frame with
itself and its successive ( N – 1 ) frames
–
– Ideal model of the N-distance measure
Gradual Scene-Change-Detection
Algorithm
• 1) N-distance measure
• 2) Difference operation
• 3) Low-pass filtering: Implement a simple low-pass filter
to keep the low frequency segments and to remove the
high-frequency segments
Gradual Scene-Change-Detection
Algorithm
• If the number of zero crossings between frame k and frame l (k
and l is zero crossing frame)
– Zero-crossing rate calculation:
–
–
larger than a threshold : high frequency fragment
else : low frequency fragment (gradual scene-change segment)
• a local “score” record mechanism
– Scorei(q) q=0, 1, 2 …, N-1
• High frequency fragment : Scorei(q) = 0
• Low frequency fragment : Scorei(q) = 1
150
100
N-Distance Measure
50
0
1 3 5 7 9 11 13 15 17 19 21 23 25
40
30
Difference operation
20
10
0
-10 1 3 5 7 9 11 13 15 17 19 21 23 25 27
1.5
Local Score
1
0.5
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27
Gradual Scene-Change-Detection
Algorithm
• a global “track” record mechanism
– Track(p)
p = 1, 2, 3, …,L
the total number of
frames in the video
sequence
• After every N-distance Measure of framei , we can
get the local Scorei(q)
– Accumulate the Score record to the Track record
Improve Gradual Scene-ChangeDetection Algorithm
• To develop a fast seed-searching process
– we select one from every S consecutive frames for
N-distance measure.
• Since gradual scene change does occur in
segment 3 only, we need to ignore the scores in
segment 1 due to correlation behavior of the
reference frame and its neighboring frames.
• The correlated distance in segment 1 is C
EXPERIMENTAL RESULTS (1)
EXPERIMENTAL RESULTS (2)
Performance Measure
Performance Result
EXPERIMENTAL RESULTS (3)
Conclusion
• This method avoided the false alarms by using
the validation mechanism.
• It also proves that the statistical model-based
approach is reliable for gradual scene-change
detection.
• Experimental results show that a very high
detection rate is achieved while the false alarm
rate is comparatively low.