Transcript H.264: RDO and HRD

H.264 Rate-Distortion Optimization
and Rate Control
MC Course, 2009
Ref:
Siwei Ma, Wen Gao, and Yan Lu,
“Rate-Distortion Analysis for H.264/AVC Video Coding and its Application to Rate Control,”
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 15,
NO. 12, pp. 1533-1544, DECEMBER 2005.
Rate-distortion Optimization


Minimize distortion D, subject to a constraint Rd, on the
number of bits used Rd.
min {D}, subject to R < Rd.
Rate-distortion optimized selection, given target rate
•
Minimize the distortion, subject to the target rate constraint
min  Dn ( mn ),
n
subject to  Rn ( mk , k )  Rd .
n
min J ( mk , k )   Dn ( mn )    Rn ( mk , k )
n
•

Dn(mn): distortion of the
nth
n
MB using mode mn
The optimal mode is such that each MB working at the same RD slope:

The optimal mode in each MB is such that the resulting RD slope
MBs.
2015/7/17
MC 2009
Dn
Rn
Dn
  .
Rn
is the same among different
2
H.264 Rate Control

H.264/AVC test model: Draft ISO/IEC 14 496-
5:2002/PDAM6, ISO IEC TC JTC 1/SC29 N5821, 2003.



based on the R-D model in VM8
Recommendation H.264 does not (at present) specify a rate
control algorithm
Ma, Gao, and Lu: the quantization scheme in H.264/AVC has been
significantly changed, which leads to the nonlinear relation between the
quantization parameter and the true quantization stepsize. Therefore, it is
highly desirable to develop a more efficient R-D model for the rate control
in H.264/AVC encoder.
2015/7/17
MC 2009
3
Hypothetical Reference Decoder

H.264 hypothetical reference decoder (HRD): guarantee that
the buffers never overflow or underflow
• rate allocation: allocate proper bits to each coding unit
according to the buffer status
• quantization parameter adjustment: how to adjust
the encoder parameters to properly encode each unit with
the allocated bits
• Find the relation between the rate and the
quantization parameter
2015/7/17
MC 2009
4
The Relation between QSTEP and QP

In H.264/AVC, the relation between QSTEP and QP is
QSTEP = 2 (QP-4)/6.
2015/7/17
MC 2009
5
The Relation between PSNR and the
quantization parameter QP

The Relation between PSNR and the quantization parameter QP is
where l and b are the constants.
2015/7/17
MC 2009
6
The Relation between PSNR and the
quantization parameter QP, Ma, Gao, and Lu
2.
Rit is the estimated number of coded bits of a macroblock
SADi is the SAD of a motion-compensated macroblock
3.
t = I, P, B
1.
4.
5.
The first item reflects the bits used to code the transform coefficients.
The second item is the bits used to code the header information of a
macroblock.
2015/7/17
MC 2009
7
Some Rate Control Models



TM5: R(QP) = X/QP, X is a constant
2
2
R

X

MAD
/
QP

X

MAD
/
QP
1
i
i
2
i
i
VM8: i
2
2
R

A
(
K

/
QP
TMN8:
i
i
i  C)




MADi is the mean absolute difference of a residual macroblock
Xi: mode parameters
A, K, and C: constants
i2 is the variance of residual coefficients in a macroblock.
2015/7/17
MC 2009
8
MPEG-4 Scalable Rate Control




MPEG-4 Annex
based on VM8
appropriate for a single video object (a rectangle VO that
covers the entire frame) and a range of bit rates and
spatial/temporal resolutions
target bit rate for a certain number of frames, (long delay
of course)
R


X 1S X 2 S
 2
Q
Q
(*)
Q: quantizer step size
S: is the mean absolute difference of a residual frame
2015/7/17
MC 2009
9
SRC Algorithm

Rate control process
1)
2)
3)
4)

Calculate the target bit rate Ri, based on the number of frames, the
number of available bits, the maximum acceptable buffer contents and
the estimated complexity of frame i.
Compute the quantizer step size Q, (to be applied to the whole frame).
Calculate S for the complete residual frame and solve (*) to find Q.
Encode the frame
Update the model parameters X1 and X2 based on the actual number of
bits generated for frame i.
The MB level version of SRC (low delay)
2015/7/17
MC 2009
10
TMN8 Algorithm
Ri  A( K i2 / QPi 2  C)

Rate control process
1)
2)
3)
4)

Measure i, the variance of residual coefficients in a macroblock.
Calculate QPi
Encode MBi
Update the model parameters K and C based on the actual number of bits
generated for MBi.
TMN8 is effective at maintaining good visual quality with a
small encoder output buffer, keeping coding delay to a
minimum.
2015/7/17
MC 2009
11
H.264 RDO and HRD


H.264: RDO and HRD closely related to rate control
RDO


MB modes: INTRA 44, INTRA 1616, INTER 1616,
INTER 168, INTER 816, INTER 88, INTER 84, INTER
48, INTER 44, SKIP and DIRECT.
Lagrangian method:


Optimal mvs for inter (or intra) and optimal coding modes for
blocks
Optimal bit allocation for mvs and residuals
2015/7/17
MC 2009
12
H.264 R-D Optimization, Ma, Gao, and
Lu
For a block in an inter frame, the rate-constrained motion estimation is first
done to find the optimal motion vector by minimizing
s: the original video signal,
c: the coded video signal
Multiple-reference prediction
2015/7/17
MC 2009
13
H.264 R-D Optimization, Ma, Gao, and
Lu

Afterwards, the rate-constrained mode selection is performed to
choose the optimal coding mode by minimizing

Langrage multipliers s have the following relation with QP, m:
constant

With the different QP, the different motion vectors and modes
might be selected.
2015/7/17
MC 2009
14
H.264 Hypothetical Reference
Decoder

Terminology
 CPB: Coded Picture Buffer
 b(n): the size in bits of picture n
 tai (n): the time when the first bit of picture n enters the CPB, the initial
arrival time of picture n
 taf (n): the time when the last bit of picture n enters the CPB, the final arrival
time of picture n, and

tr(n): is the time when the picture n is removed from the CPB
2015/7/17
MC 2009
R: the bit rate at which the
CPB is filled.
15
H.264 Hypothetical Reference
Decoder
Packetization, and others
One packet per frame this case
tc: time of a clock tick
: the earliest time when picture n can reach the buffer
initial_CPB_removal_delay represents the delay between the time when the first bit of the
coded data arrives in the CPB and the time when the first time removal of coded
data from the CPB.
2015/7/17
MC 2009
16
H.264 Hypothetical Reference
Decoder

access unit: one coded picture or slice data partition
2015/7/17
MC 2009
17
H.264 Hypothetical Reference
Decoder
To ensure that the CPB does not overflow or underflow,
Ln: the lower bound of rate
This means that picture n must be in the buffer when it is removed from the buffer.
2015/7/17
MC 2009
18
H.264 Hypothetical Reference
Decoder
CPB should neither overflow nor underflow if
R: the bit rate at which the CPB is filled.
Where be[t] and te[t] denote the bit equivalent of a time t and
the time equivalent of a number of bits b, respectively, and
Usually the bits allocated to a picture are clipped to [Ln, Un], where
  1.0 and   0.9.
2015/7/17
MC 2009
19
Rate-control Algorithm, Ma, Gao,
and Lu
1.
2.
Bit allocation: In this step, a target bit is allocated to each picture in a group of
pictures (GOP)
Initialization for the current macroblock
TM5: R(QP) = X/QP
QSTEP = 2 (QP-4)/6

where X0t is global complexity measure for a picture as defined in TM5, R0t is the coded bits
of the picture, SAD0t is the average SAD of all macroblocks in the picture, BHead is the
average header bits for a macroblock, including motion and mode information, and BCoeff is
the bits used to code luminance and chrominance coefficients.
2015/7/17
MC 2009
20
Rate-control Algorithm,
Ma, Gao, and Lu
(2)
First RDO-based coding mode selection
3.
•
If the current macroblock is the first MB in the frame, QP is set to be the average quantization
parameter of the previous frame QPprev; otherwise, we use
Mi: the number
of remaining uncoded MBs
Ti: the number of available
bits for encoding the
remaining uncoded MBs
Second RDO-based coding mode selection
4.
•
Calculate SADi’ in terms of the selected coding mode and then compute a new quantization
parameter QP’ using SADi’ in the same way as Step 3.
5.
Counter updating
6.
R-D model parameter updating
2015/7/17
MC 2009
21
Rate-control Algorithm,
Ma, Gao, and Lu
(3)



new R-D model: the relationship between the quantization
parameter and the true quantization stepsize is no longer
linear.
proposed a new rate-distortion (R-D) model by utilizing the
true quantization stepsize and then develop an improved
rate-control scheme for the H.264/AVC encoder based on this
new R-D model.
The algorithm performs one-pass operation at frame level
and a partial two-pass operation at the macroblock level.
Since the second RDO-based coding model selection in Step
4 is a conditional operation, the percentage of such
macroblocks can be controlled by parameter ; consequently,
the computing complexity is controllable as well.
2015/7/17
MC 2009
22
2015/7/17
MC 2009
23