Transcript Document

Supplement,
Chapters 6
MC Course, 2009
Chapter 6
H.264/MPEG4 Part 10
Macroblock syntax elements
Reference picture management
1/8 motion estimation
Motion vector prediction
Intra prediction
Deblocking filter
Transform and quantization
Direct and weighted predictions
Data partition
Raw byte sequence payload

H.264 Encoder/Decoder
Forward path
unfiltered
Reconstruction path
unfiltered
2015/7/17
Chapter 6 2009
3
H.264 Profiles
FRExt
High profiles
2015/7/17
Chapter 6 2009
4
H.264 Slice Types
There is minimal inter-dependency between coded slices
which can help to limit the propagation of errors
2015/7/17
Chapter 6 2009
5
Slice Syntax
Skipped (not coded) MBs
2015/7/17
Chapter 6 2009
6
Macroblock Syntax Elements
2015/7/17
Chapter 6 2009
7
Reference Picture Management

Every reconstructed frame can be marked as





Unused for reference
Short term picture
Long term picture
None (directly output)
Adaptive memory control

Managing the short and long term picture indexes.



(Encoder  Decoder)
Short term picture may be assigned a long term frame index
Short/long term picture may be marked as unused for reference
Instantaneous Decoder Refresh Picture (IDR)

An encoder sends an IDR to clear the reference picture buffer
2015/7/17
Chapter 6 2009
8
Reference Picture Management
(P-slice)
Example: Current frame number=250
Number of reference frames=5
2015/7/17
Chapter 6 2009
9
Macroblock to slice group map
types
2015/7/17
Chapter 6 2009
10
Slice Groups:
dispersed 4 groups
2015/7/17
Chapter 6 2009
11
Slice Groups:
Foreground and Background Map
2015/7/17
Chapter 6 2009
12
Slice Groups
Box-out, Raster and Wipe
2015/7/17
Chapter 6 2009
13
Interpolation of Chroma:
eight-sample
a = round ([(8-dx)(8-dy)A+dx(8-dy)B+(8-dx)dyC+dxdyD]/64)
2015/7/17
Chapter 6 2009
14
Motion Vector Prediction
leftmost
topmost
2015/7/17
Chapter 6 2009
15
Motion Vector Prediction (2)


For transmitted partitions excluding 16  8 and 8  16
partition sizes: MVp= median (MVA , MVB , MVC )
For upper 16  8 partitions: MVp = MVB
For lower 16  8 partitions: MVp = MVA

For left 8  16 partitions: MVp = MVA
For right 8  16 partitions: MVp = MVC

For skipped macroblocks:
MVp= (as if the block were encoded in 16  16 Inter mode)
2015/7/17
Chapter 6 2009
16
Intra Prediction

Signaling intra prediction modes


The choice of intra prediction mode for each 44 block must
be signaled to the decoder and this could potentially require a
large number of bits.
However, intra modes for neighboring 44 blocks are often
correlated.
Mode 1
Mode 1
Mode 1 probably
2015/7/17
Chapter 6 2009
17
1:H
Intra Prediction

3
Signaling intra prediction modes


Calculate the “most probable” prediction mode of E in
both the encoder and the decoder
Most probable mode


Minimum of the prediction modes of A and B
Flag: prev_intra4x4_pred_mode


1: Most probable mode = Prediction mode of E
If most probable mode  prediction mode of E


prev_intra4x4_pred_mode = 0 Most probable mode
rem_intra4x4_pred_mode is used and sent to the decoder

Prediction mode of E: 0, …, i-1, i, i+1, i+2, …, 8
rem_intra4x4_pred_mode: 0, …, i-1, -, i , i+1, …, 7

only 3 bits are needed for 9 modes
2015/7/17
Chapter 6 2009
18
Deblocking Filter

Edge filtering order
5
6
7
8
1 23 4
1616 luma

11
12
9 10
88 chroma
Boundary strength (bS)

2015/7/17
Chapter 6 2009
19
Deblocking Filter

Filter decision

(p2, p1, p0, q0, q1, q2) is filtered only if


bS > 0 and
|p0-q0| < , |p1-p0| <  , and |q1-q0| ≦ .


 and  depend on QPs of block p and q.
Filter implementation

bS  {1, 2, 3}




4-tap filter
(p1, p0, q0, q1) → (p’0, q’0)
If |p2-p0|<, (p2, p1, p0, q0) → (p’1) (luma only)
If |q2-q0|<, (q2, q1, q0, p0) → (q’1) (luma only)
bS = 4

If luma block, |p2-p0|< , and |p0-q0|<round(/4)


(p2,p1,p0,q0,q1)→p’0, (p2,p1,p0,q0) →p’1, (p3,p2,p1,p0,q0) →p’2
otherwise (p1, p0, q1) → p’0
If luma block, |q2-q0|< , and |p0-q0|<round(/4)

(q2,q1,q0,p0,p1)→q’0, (q2,q1,q0,p0) →q’1, (q3,q2,q1,q0,p0) →q’2
otherwise (q1, q0, p1) → q’0
2015/7/17
Chapter 6 2009
20
Digital Cosine Transform
Y = A X AT
X = AT Y A
2015/7/17
Chapter 6 2009
21
Digital Cosine Transform (2)
2015/7/17
Chapter 6 2009
22
Digital Cosine Transform (3)
scaling factor
core
d = c/b  0.414
To simplify the implementation: d  0.5
To ensure orthogonal, b is modified
a = ½, b = 52 , d = ½
(2nd, 4th rows of C, 2nd, 4th columns of CT)2, …
2015/7/17
Chapter 6 2009
23
Transform and Quantization

4x4 residual transform and
quantization
Quantization
Pre-scaling matrix
Transform
[Wij ]
round(
/ Qstep)
Input matrix
•
•
Integer arithmetic using +, - and SHIFT
16-bit arithmetic
MF /
2qbits
(Implemented by
right shift)
qbits=15+floor(QP/6)
2015/7/17
Chapter 6 2009
Table lookup
24
Transform and Quantization
avoiding
PF
MF
Z ij  round(Wij 
) division

 Z ij  round(Wij  qbits )
Qstep
2
b2
ab
PF  a , , or
4
2
2
| Z ij | (| Wij |  MF  f )  qbits
sign(Z ij )  sign(Wij )
2qbits / 3
f   qbits
2 / 6
2015/7/17
Chapter 6 2009
for Intra
for Inter
25
Transform and Quantization

4x4 rescaling (in inverse quantizer)
Table lookup

Z × Qstep ×
× 64 2floor(QP/6)
The output doubles for every (QP/6)
Quantized value
scaling factor to avoid rounding errors
Wij'  Zij  Qstep  PF  64
Wij'  Zij Vij  2 floor(QP / 6) , real implementation
2015/7/17
Vij  Qstep  PF  64 Chapter 6 2009
26
Transform and Quantization

A complete process of Transform,
quantization, rescaling, and inverse transform
2015/7/17
Chapter 6 2009
27
Direct Prediction


Spatial direct mode
Temporal direct mode

2015/7/17
Chapter 6 2009
28
Weighted Prediction

Three types




‘Explicit’ prediction


P slice with ‘explicit’ weighted prediction
B slice with ‘explicit’ weighted prediction
B slice with ‘implicit weighted prediction
The weighted factors are determined by the
encoder.
‘Implicit’ prediction

The weighted factors are calculated based on the
relative temporal positions of the reference
pictures.
2015/7/17
Chapter 6 2009
29
Data Partitioned Slices

Different partitions can be placed in
separate NAL unit.

Partition A


Partition B


Slice header, macroblock header.
Coded residual data for I and SI macroblocks.
Partition C

2015/7/17
Coded residual data for inter coded
macroblocks.
Chapter 6 2009
30
Raw Byte Sequence Payload
(RBSP)
2015/7/17
Chapter 6 2009
31