Transcript 下載/瀏覽

Motion Adaptive Search for Fast
Motion Estimation
授課老師：王立洋
老師
製作學生：M9535204
蔡鐘葳
1/39
Outline
▓ Introduction
▓ Motion Activity
▓ Motion Adaptive Search Algorithm
▓ Experimental Results
▓ Reference
2/39
Introduction
This paper presents the motion adaptive search (MAS)
that effectively utilizes the motion activity
information to improve
The speed
The accuracy of motion search
3/39
Motion Activity
Different sets of frames in a video have different
motion activities corresponding to various scenes
We use the term
Global motion activity (GMA) to characterize motion
activity over an entire video frame
Local motion activity (LMA) to characterize motion
activity within a small region
4/39
LMA
The motion vectors of spatially neighboring
macroblocks are highly correlated
Therefore, We compute the LMA at a macroblock
position based on
The knowledge of the motion vectors of its neighboring
macroblocks
5/39
Region of Support (ROS)
Let MBi, j be the (i, j)th macroblock in a frame
i and j indicate the horizontal and vertical positions
We define the region of support (ROS) of MBi, j ,
denoted by Ri, j
As the set of macroblocks to the left, above, and
above right of MBi, j , respectively. That is
6/39
The City-block Lengths
We compute the LMA at MBi, j based on the
information of city-block lengths of motion vectors of
the macroblocks in Ri, j
Let Vi. j≡ (xi, j ,y i, j ) be the motion vector of MBi, j
The city city-block length of Vi, j, denoted by LVi j, is
given by
7/39
The Grade of LMA
The grade of LMA at MBi, j , denoted by GLMA (i, j) , is
defined as the maximum city-block length of the
motion vectors of all macroblocks in Ri, j .
Mathematically,
8/39
Example for LMA
Let LMA(i, j) denote the LMA at MBi, j . Based on
GLMA (i, j) , we categorize LMA(i, j) into two classes
as follows
Fig. 1 depicts an example of distribution of GLMA
The LMA=Low at most of the macroblocks belonging to
the background object
The LMA=High at most of the macroblocks belonging to
9/39
the foreground object
Fig. 1
10/39
LMA and Motion Vector Bias
When LMA(i, j) is Low, all the neighborhood motion
vectors of MBi, j are close to (0,0)
LMA(i, j)=High the motion vector of MBi, j is away
from (0,0)
The motion vector histogram commonly observed in
most video sequences reveals the link between local
motion activity and motion vector bias
11/39
Fig. 2
12/39
CMV and SAD
Consider the candidate motion vector set, denoted by
CMV given as follows
We define the predicted motion vector, denoted Vi, j
As the vector in CMV that yields minimum sum of
absolute difference (SAD). Mathematically,
13/39
The Differential Motion Vector
Consider the differential motion vector
The differential motion vector dv is computed for each
macroblock at which LMA=High
The histogram of dv is shown in Fig. 3
14/39
Fig. 3
15/39
LMA and Camera Motion
The camera motions can be divided into two
categories
Nontranslational (e.g., zoom and rotation)
Translational (e.g., pan and tilt)
Translational camera motion is the most often
encountered camera motion in the real world
16/39
Translational Camera Motion (1/3)
The translational camera motion introduces High
LMA over the entire frame
Thus, the presence of translational camera motion is
detected by first investigating
Whether the LMA is High at a majority of the macroblocks
Whether a high peak occurs at a point away from (0,0)
17/39
Translational Camera Motion (2/3)
Let H be the set of motion vectors of those
macroblocks, at which LMA = High
Consider a subset of H, denoted by H1, whose cityblock lengths are greater than one
Let C be the motion vector at which the histogram of
H1 has a peak
The cardinalities of H and H1 are denoted by |H| and
|H1|
18/39
Translational Camera Motion (3/3)
We declare that translational camera motion is present
in the frame if the following two conditions are
satisfied
|H| is greater than or equal to 25% of the total number of
macroblocks
The peak value of H1 is greater than or equal to 0.1× |H1|
Then C is assigned as the candidate motion vector
representing the camera motion
19/39
Global Motion Activity (GMA)
The GMA is estimated based on the motion vectors of
all the macroblocks in a frame to characterize the
overall motion activity in the entire frame
We define the grade of GMA, denoted by GGMA
20/39
Example for GMA (1/2)
An example of variation of GGMA obtained with FS over the
300 frames of the Foreman video sequence is shown in Fig. 4
21/39
Example for GMA (2/2)
From 0 to 150
An unstable camera is focused (a moderate value)
From 150 to 225
The camera pans to the right (the sudden increase due to
the high motion activity caused by the camera motion)
From 225 to 300
The camera focuses on a building under construction (low
22/39
value)
Motion Adaptive Search Algorithm (1/2)
The current frame are estimated from the motion
vector field of the immediately previous predicted
frame (P-frame)
The LMA and GMA information is used to adjust the
search range for each macroblock
The estimated camera motion vector is used as a point
on one of the search patterns
The search patterns are switched intelligently to suit
23/39
the LMA
Motion Adaptive Search Algorithm (2/2)
Before the motion search begins, the current
macroblock is first tested to determine if it is a
stationary macroblock by comparing its SAD at (0,0)
If yes, the search is terminated immediately and its motion
vector is assigned to be (0,0)
Otherwise, an initial motion vector is predicted based on
the LMA and local search is performed to obtain the final
motion vector
A detailed technical description of MAS is provided
24/39
in the following
Estimation of Camera Motion Vector and
Grade of GMA
Let C and GGMA denote the estimated values and the
grade of GMA in the current frame
C' and G'GMA be the true values and the grade of GMA
If the current frame is the very first inter-frame
We set C equal to (0,0) and GGMA equal to 16 times the
temporal distance between the current frame and its
reference frame
Otherwise, GGMA = G'GMA and C = C'
25/39
Adaptation of Search Range
The maximum search range for each macroblock in
the current frame srmax is given by
We adapt the search range for a as follows
sr i, j is the search range
LMA(i, j) is the LMA at MBi, j
26/39
Early Elimination of Search for Stationary
Macroblocks
We compare the SAD of a macroblock MBi, j at (0,0),
denoted by SADi, j (0,0) , with a threshold value T
If SADi, j (0,0) ≤ T
The motion search is terminated immediately
Assigning its motion vector as (0,0)
We call this process early elimination of search (EES)
27/39
Search Patterns
The two search patterns employed in the MAS are
The diamond search pattern (DSP)
The elastic search pattern (ESP) as shown in Fig. 5
28/39
Prediction of Initial Motion Vector
The initial motion vector at MBi, j , denoted by Ii, j, is
expressed as follows
29/39
Intelligent Local Search (1/5)
We categorize the moves of DSP into three types,
Type-1, Type-2 and Type-3
We make use of the vector cross product between
ck −1ck and ckck +1 to determine the type
30/39
Fig. 6
31/39
Intelligent Local Search (2/5)
From vector algebra, we have
Where ‘⊗’ represents the vector cross product
uz is the unit vector perpendicular to both ck −1ck and ck ck +1
32/39
Intelligent Local Search (3/5)
Another definition of cross product is
The value of sinθ is equal to zero when θ= 0o
The value of sinθ is equal to 1 when θ= 90o
The value of sinθ is equal to -1 when θ= 270o
Therefore, the value of Z distinguishes the types of
moves of the DSP
Z=0 (Type-1), Z>0 (Type-2 ) and Z<0 (Type-3)
33/39
Intelligent Local Search (4/5)
If the move of Dk +1 is of Type-1
The other non-overlapping points are the two points on
either side of dnk+1
If the move of Dk +1 is of Type-2
The point next to dnk+1 is the other non-overlapping point
If the move of Dk +1 is of Type-3
The other non-overlapping points is the point previous to
34/39
dnk+1
Intelligent Local Search (5/5)
Thus, the sets of non-overlapping points of Dk +1 are
for Type-1 move
for Type-2 move
for Type-2 move
35/39
The intelligent local search algorithm (1/2)
36/39
The intelligent local search algorithm (2/2)
37/39
Experimental Results
38/39
Reference
[1] Prabhudev Irappa Hosur, Member, IEEE, “Motion
Adaptive Search for Fast Motion Estimation,” IEEE
Signal Processing Lett. Volume 49, No. 4, Nov. 2003
Page(s):1330 - 1340.
39/39