Transcript True-Motion Estimation with 3-D Recursive Search Block

TRUE-MOTION ESTIMATION WITH
3-D RECURSIVE SEARCH
BLOCK MATCHING
1
Gerard de Haan,
Paul W. A. C. Biezen
Henk Huijgen
Olukayode A. Ojo
(Philips Research Laboratories, 5600 JA Eindhoven, the
Netherlands.)
This paper appears in:
Circuits and Systems for Video Technology, IEEE Transactions
on
Page 368–379.388 ,Oct 1993
OVERVIEW
Introduction
 Recursive Search Method for True ME

1-D Recursive Search
 2-D Recursive Search
 3-D Recursive Search

Updating Strategy
 Further Emphasis on Smoothness
 Block Erosion to Eliminate Blocking Effects
 Evaluation Results & Experiments

Modified Mean Square Prediction Error(M2SE)
 Smoothness


Conclusion
2
INTRODUCTION


What is true motion?
Why do we find the true motion?
 Consumer display scan rate conversion[1]-[8].
 Common drawback is decreased dynamic resolution.
 Motion compensation techniques[9]-[12] are too
expensive for consumer television applications.
3
OVERVIEW
Introduction
 Recursive Search Method for True ME

1-D Recursive Search
 2-D Recursive Search
 3-D Recursive Search

Updating Strategy
 Further Emphasis on Smoothness
 Block Erosion to Eliminate Blocking Effects
 Evaluation Results & Experiments

Modified Mean Square Prediction Error(M2SE)
 Smoothness


Conclusion
4
RECURSIVE SEARCH METHOD FOR TRUE ME(1/5)



1-D Recursive Search: similar to 2-D logarithmic search[22]
The candidate set (CSi) & prediction vector (Di-1):
Indicate with S rather than Di-1
as the spatial prediction vector
(pel-recursive algo. [23][24] ):
5
RECURSIVE SEARCH METHOD FOR TRUE ME(2/5)



2-D Recursive Search: two spatial prediction vectors
A 1-D recursive algorithm cannot cope with discontinuities in the
velocity plane.
Assumption (1):
 The discontinuities in the velocity plane are spaced at a
distance that enables convergence of the recursive block
matcher in between two discontinuities.

Two estimators and the selection criterion:

As described in 1-DRS, updating, respectively, prediction vectors:
6
RECURSIVE SEARCH METHOD FOR TRUE ME(3/5)


2-D Recursive Search solves the run-in problem at the
boundaries of moving objects.
The best implementation of 2-DC results with predictions
from blocks 1 and 3 for estimators a and b, respectively:
7
where (X,Y) is the size of block.
RECURSIVE SEARCH METHOD FOR TRUE ME(4/5)




3-D Recursive Search: temporal prediction vectors
Assumption (2):
 The displacements between two consecutive velocity planes,
due to movements in the picture, are small compared to the
block size.
Rather than choosing the additional estimators c and d, applying
temporal prediction vectors as additional candidates:
These convergence accelerators (CA) are taken from a block
shifted diagonally over “ r ” blocks.
8
RECURSIVE SEARCH METHOD FOR TRUE ME(5/5)



3-D RS candidate set CSa & CSb:
The CA's are particularly advantageous at the top of the
screen, where the spatial process starts converging.
The CA's improve the temporal consistency.
9
OVERVIEW
Introduction
 Recursive Search Method for True ME

1-D Recursive Search
 2-D Recursive Search
 3-D Recursive Search

Updating Strategy
 Further Emphasis on Smoothness
 Block Erosion to Eliminate Blocking Effects
 Evaluation Results & Experiments

Modified Mean Square Prediction Error(M2SE)
 Smoothness


Conclusion
10
UPDATING STRATEGY

0 improves the performance for small stationary
image parts but disturbs the convergence.
The asynchronous cyclic search (ACS):
Nbl is the output of a block counter
 lut is a look-up table function


The pseudorandom look-up table (for p=9):
11
symmetrical distribution around 0 with p updates
OVERVIEW
Introduction
 Recursive Search Method for True ME

1-D Recursive Search
 2-D Recursive Search
 3-D Recursive Search

Updating Strategy
 Further Emphasis on Smoothness
 Block Erosion to Eliminate Blocking Effects
 Evaluation Results & Experiments

Modified Mean Square Prediction Error(M2SE)
 Smoothness


Conclusion
12
FURTHER EMPHASIS ON SMOOTHNESS (1/2)

The risks which jeopardize the smoothness:
1)
2)
3)

An element of the update sets may equal a multiple of the
basic period of the structure.
"The other" estimator may not be converged, or may be
converged to wrong value that does not correspond to the
actual displacement.
Directly after a scene change, the convergence accelerators
(CAs) yield the threatening candidate.
Improve the result for risks 1) & 3):

Add penalties to the error function related to the length of the
difference vector between the candidates to be evaluated:
13
FURTHER EMPHASIS ON SMOOTHNESS (2/2)
Respectively, 0.4%, 0.8%, and 1.6% of the maximum error
value, for the cyclic update(Sn), the convergence accelerator
(CA), and the fixed 0 candidate vector.
 The last candidate(0) especially requires a large penalty.


Improve the result for risk 2):

The situation occurs if a periodic part enters the picture from
the blanking or appears from behind an other object.

Advantage of two independent estimators would be lost.
14
OVERVIEW
Introduction
 Recursive Search Method for True ME

1-D Recursive Search
 2-D Recursive Search
 3-D Recursive Search

Updating Strategy
 Further Emphasis on Smoothness
 Block Erosion to Eliminate Blocking Effects
 Evaluation Results & Experiments

Modified Mean Square Prediction Error(M2SE)
 Smoothness


Conclusion
15
BLOCK EROSION TO ELIMINATE BLOCKING EFFECTS

Improve the result for:
Eliminating the visible block structures in the picture.
 Eliminating fixed block boundaries from the vector field
without blurring contours.

F
E

H-1-1
Finally assigned to the pixels in the quadrant:
16
OVERVIEW
Introduction
 Recursive Search Method for True ME

1-D Recursive Search
 2-D Recursive Search
 3-D Recursive Search

Updating Strategy
 Further Emphasis on Smoothness
 Block Erosion to Eliminate Blocking Effects
 Evaluation Results & Experiments

Modified Mean Square Prediction Error(M2SE)
 Smoothness


Conclusion
17
EVALUATION RESULTS & EXPERIMENTS (1/4)

Modified Mean Square Prediction Error(M2SE):↓, quality↑
s identifies the test sequence 1~5
 P . L is the number of pixels in the image excluding margin.


Smoothness Indicator: S(t)↑, smoothness↑

Nb is the number of
blocks in a field.
18
EVALUATION RESULTS & EXPERIMENTS (2/4)

Experiments:
19
EVALUATION RESULTS & EXPERIMENTS (3/4)
Captured from:
Frame Rate Up-Conversion,陳秉昱,January 8,2006
20
EVALUATION RESULTS & EXPERIMENTS (4/4)
Captured from:
Frame Rate Up-Conversion,陳秉昱,January 8,2006
21
CONCLUSION



The newly designed motion estimation algorithm is
emerging as the most attractive of the tested blockmatching algorithms in the application of consumer field
rate conversion.
The bidirectional convergence principle enabled
combination of the conflicting demands for smoothness
and yet steep edges in the velocity field.
Using new test criteria, the suitability of motion estimators
for television with motion compensated field rate doubling
was tested.
22