Transcript Document
Supplement,
Chapters 6
MC Course, 2009
Chapter 6
H.264/MPEG4 Part 10
Macroblock syntax elements
Reference picture management
1/8 motion estimation
Motion vector prediction
Intra prediction
Deblocking filter
Transform and quantization
Direct and weighted predictions
Data partition
Raw byte sequence payload
H.264 Encoder/Decoder
Forward path
unfiltered
Reconstruction path
unfiltered
2015/7/17
Chapter 6 2009
3
H.264 Profiles
FRExt
High profiles
2015/7/17
Chapter 6 2009
4
H.264 Slice Types
There is minimal inter-dependency between coded slices
which can help to limit the propagation of errors
2015/7/17
Chapter 6 2009
5
Slice Syntax
Skipped (not coded) MBs
2015/7/17
Chapter 6 2009
6
Macroblock Syntax Elements
2015/7/17
Chapter 6 2009
7
Reference Picture Management
Every reconstructed frame can be marked as
Unused for reference
Short term picture
Long term picture
None (directly output)
Adaptive memory control
Managing the short and long term picture indexes.
(Encoder Decoder)
Short term picture may be assigned a long term frame index
Short/long term picture may be marked as unused for reference
Instantaneous Decoder Refresh Picture (IDR)
An encoder sends an IDR to clear the reference picture buffer
2015/7/17
Chapter 6 2009
8
Reference Picture Management
(P-slice)
Example: Current frame number=250
Number of reference frames=5
2015/7/17
Chapter 6 2009
9
Macroblock to slice group map
types
2015/7/17
Chapter 6 2009
10
Slice Groups:
dispersed 4 groups
2015/7/17
Chapter 6 2009
11
Slice Groups:
Foreground and Background Map
2015/7/17
Chapter 6 2009
12
Slice Groups
Box-out, Raster and Wipe
2015/7/17
Chapter 6 2009
13
Interpolation of Chroma:
eight-sample
a = round ([(8-dx)(8-dy)A+dx(8-dy)B+(8-dx)dyC+dxdyD]/64)
2015/7/17
Chapter 6 2009
14
Motion Vector Prediction
leftmost
topmost
2015/7/17
Chapter 6 2009
15
Motion Vector Prediction (2)
For transmitted partitions excluding 16 8 and 8 16
partition sizes: MVp= median (MVA , MVB , MVC )
For upper 16 8 partitions: MVp = MVB
For lower 16 8 partitions: MVp = MVA
For left 8 16 partitions: MVp = MVA
For right 8 16 partitions: MVp = MVC
For skipped macroblocks:
MVp= (as if the block were encoded in 16 16 Inter mode)
2015/7/17
Chapter 6 2009
16
Intra Prediction
Signaling intra prediction modes
The choice of intra prediction mode for each 44 block must
be signaled to the decoder and this could potentially require a
large number of bits.
However, intra modes for neighboring 44 blocks are often
correlated.
Mode 1
Mode 1
Mode 1 probably
2015/7/17
Chapter 6 2009
17
1:H
Intra Prediction
3
Signaling intra prediction modes
Calculate the “most probable” prediction mode of E in
both the encoder and the decoder
Most probable mode
Minimum of the prediction modes of A and B
Flag: prev_intra4x4_pred_mode
1: Most probable mode = Prediction mode of E
If most probable mode prediction mode of E
prev_intra4x4_pred_mode = 0 Most probable mode
rem_intra4x4_pred_mode is used and sent to the decoder
Prediction mode of E: 0, …, i-1, i, i+1, i+2, …, 8
rem_intra4x4_pred_mode: 0, …, i-1, -, i , i+1, …, 7
only 3 bits are needed for 9 modes
2015/7/17
Chapter 6 2009
18
Deblocking Filter
Edge filtering order
5
6
7
8
1 23 4
1616 luma
11
12
9 10
88 chroma
Boundary strength (bS)
2015/7/17
Chapter 6 2009
19
Deblocking Filter
Filter decision
(p2, p1, p0, q0, q1, q2) is filtered only if
bS > 0 and
|p0-q0| < , |p1-p0| < , and |q1-q0| ≦ .
and depend on QPs of block p and q.
Filter implementation
bS {1, 2, 3}
4-tap filter
(p1, p0, q0, q1) → (p’0, q’0)
If |p2-p0|<, (p2, p1, p0, q0) → (p’1) (luma only)
If |q2-q0|<, (q2, q1, q0, p0) → (q’1) (luma only)
bS = 4
If luma block, |p2-p0|< , and |p0-q0|<round(/4)
(p2,p1,p0,q0,q1)→p’0, (p2,p1,p0,q0) →p’1, (p3,p2,p1,p0,q0) →p’2
otherwise (p1, p0, q1) → p’0
If luma block, |q2-q0|< , and |p0-q0|<round(/4)
(q2,q1,q0,p0,p1)→q’0, (q2,q1,q0,p0) →q’1, (q3,q2,q1,q0,p0) →q’2
otherwise (q1, q0, p1) → q’0
2015/7/17
Chapter 6 2009
20
Digital Cosine Transform
Y = A X AT
X = AT Y A
2015/7/17
Chapter 6 2009
21
Digital Cosine Transform (2)
2015/7/17
Chapter 6 2009
22
Digital Cosine Transform (3)
scaling factor
core
d = c/b 0.414
To simplify the implementation: d 0.5
To ensure orthogonal, b is modified
a = ½, b = 52 , d = ½
(2nd, 4th rows of C, 2nd, 4th columns of CT)2, …
2015/7/17
Chapter 6 2009
23
Transform and Quantization
4x4 residual transform and
quantization
Quantization
Pre-scaling matrix
Transform
[Wij ]
round(
/ Qstep)
Input matrix
•
•
Integer arithmetic using +, - and SHIFT
16-bit arithmetic
MF /
2qbits
(Implemented by
right shift)
qbits=15+floor(QP/6)
2015/7/17
Chapter 6 2009
Table lookup
24
Transform and Quantization
avoiding
PF
MF
Z ij round(Wij
) division
Z ij round(Wij qbits )
Qstep
2
b2
ab
PF a , , or
4
2
2
| Z ij | (| Wij | MF f ) qbits
sign(Z ij ) sign(Wij )
2qbits / 3
f qbits
2 / 6
2015/7/17
Chapter 6 2009
for Intra
for Inter
25
Transform and Quantization
4x4 rescaling (in inverse quantizer)
Table lookup
Z × Qstep ×
× 64 2floor(QP/6)
The output doubles for every (QP/6)
Quantized value
scaling factor to avoid rounding errors
Wij' Zij Qstep PF 64
Wij' Zij Vij 2 floor(QP / 6) , real implementation
2015/7/17
Vij Qstep PF 64 Chapter 6 2009
26
Transform and Quantization
A complete process of Transform,
quantization, rescaling, and inverse transform
2015/7/17
Chapter 6 2009
27
Direct Prediction
Spatial direct mode
Temporal direct mode
2015/7/17
Chapter 6 2009
28
Weighted Prediction
Three types
‘Explicit’ prediction
P slice with ‘explicit’ weighted prediction
B slice with ‘explicit’ weighted prediction
B slice with ‘implicit weighted prediction
The weighted factors are determined by the
encoder.
‘Implicit’ prediction
The weighted factors are calculated based on the
relative temporal positions of the reference
pictures.
2015/7/17
Chapter 6 2009
29
Data Partitioned Slices
Different partitions can be placed in
separate NAL unit.
Partition A
Partition B
Slice header, macroblock header.
Coded residual data for I and SI macroblocks.
Partition C
2015/7/17
Coded residual data for inter coded
macroblocks.
Chapter 6 2009
30
Raw Byte Sequence Payload
(RBSP)
2015/7/17
Chapter 6 2009
31