Transcript Overview of the H. 264/AVC video coding standard

Overview of the H. 264/AVC
video coding standard
History and Naming
Goal of “The Standard”
• Deploy over existing & future networks in
these areas with substantially low bitrate:
–
–
–
–
–
Broadcast
Interactive or serial storage
Conversational services
Video-on-demand or multimedia streaming services
Multimedia messaging services
• Flexibility and Customizability:
Comparison: PSNR
• H.264 improves the coding efficiency over other
coding methods
Design Feature Highlights
• Enhancements of coding efficiency
– Better prediction
•
•
•
•
Variable block-size motion compensation with small block sizes
Quarter-sample-accurate motion compensation
Multiple reference picture motion compensation
Weighted prediction
– Other factors
• Hierarchical block transform
• Short word-length transform
• Enhancements of robustness and flexibility
for operation over a variety of networks
Structure of H.264 Video Encoder
Network Abstraction Layer
• Facilitates the ability to map H.264 VCL data
to transport layers such as RTP/IP, H.32x,
MPEG-2.
– NAL Units
– VCL and non-VCL units
– Parameter sets
– Access units
– Coded video sequence
NAL: Parameter Sets
VCL: Revolution or Evolution?
• There is no single coding element in the VCL that
provides the majority of the significant improvement
in compression efficiency in relation to prior video
coding standards.
• It is rather a plurality of smaller improvements that
add up to the significant gain.
• The basic source-coding algorithm is a hybrid of
inter-picture prediction to exploit temporal
statistical dependencies and transform coding of
the prediction residual to exploit spatial statistical
dependencies.
VCL: Elements
• Pictures, Frames, Fields
• Macroblocks (MBs)(16x16)
• Slices, Slices Groups
VCL: Coding the Slices
• Each slice can be coded using different coding
types as follows:
– I slice: A slice in which all macroblocks of the slice are coded using
intra prediction.
– P slice: In addition to the coding types of the I slice, some macroblocks
of the P slice can also be coded using inter prediction with at most one
motion-compensated prediction signal per prediction block.
– B slice: In addition to the coding types available in a P slice, some
macroblocks of the B slice can also be coded
– SP slice (switching P slice): coded such that efficient switching between
different pre-coded pictures becomes possible.
– SI slice (switching I slice): allows an exact match of a macroblock in an
SP slice for random access and error recovery purposes.
Peek into Encoder
Adaptive Frame/Field Coding
• 3 choices:
– To combine the two fields together and to code them
as one single coded frame (frame mode).
– To not combine the two fields and to code them as
separate coded fields (field mode).
– To combine the two fields together and compress
them as a single frame, but when coding the frame to
split the pairs of two vertically adjacent macroblocks
into either pairs of two field macroblocks or frame
macroblocks before coding them.
Adaptive Frame/Field Coding
• PAFF (Picture-Adaptive Frame/Field)
– The choice between first 2
– >16% better than field only in general
• MBAFF (MacroBlock-Adaptive Frame/Field )
– Non-moving: frame mode, moving: field mode
– Use a pair of MBs (16x32 luma)
– >14% better than PAFF in general
Intra-Frame Prediction
3 modes: Intra_4x4, Intra_16x16 (with chroma
prediction) and I_PCM
Constrained intra coding mode: prediction only from intra-coded
neighboring MBs
Inter-Frame Prediction
• Full sample, half sample, one-quarter
sample
– Half: apply one-dimensional 6-tap FIR filter
– One-quarter: average full and half
• Results:
– Accurate motion representation
– More flexibility in prediction filtering
• Different degrees of low pass filtering could be
chosen
• better preserving hi-frequency content.
Transform, Scaling and Quantization
• 4x4 Integer transform vs. 8x8 DCT
– No mismatch in inverse transform
– Less computations, short processing wordlength
– Less noise
•52 quantization values; step size: 12%
• one step means 12% bit rate change
• 6 steps: factor of 2
Entropy Coding
• CAVLC (Context-Adaptive Variable Length
Coding)
– VLC tables for various syntax elements are switched depending
on already transmitted syntax elements.
– Since the VLC tables are designed to match the corresponding
conditioned statistics, performance is improved in comparison to
schemes using a single VLC table.
• CABAC (Context-Adaptive Binary Arithmetic
Coding)
– allows the assignment of a noninteger number of bits to each
symbol of an alphabet
– permits adaptation to nonstationary symbol statistics
– uses only shifts and table look-ups
– Compared to CAVLC, typically provides a reduction in bit rate
5%–15%
In-Loop Deblocking Filter
• Goal: reduce blockiness and maintain sharpness
Profiles and Levels
• Profiles: Facilitate interoperability between
various applications.
– Defines a set of coding tools or algorithms that can be used in
generating a conforming bitstream
– Incl. Baseline, Main, Extended, High(4)
• Levels: place constraints on certain key
parameters of the bitstream
– Correspond to processing power and memory capability of a
codec
– Set limts for data bitrate, framesize, pic buffer size. Eg. QCIF,
ITU-R601, HDTV etc.
– Universal (15) used in different profile
Profiles: Coding Features
Computation Consumption
• Use large amount of computation to achieve
high quality, reliability and compression ratio
–
–
–
–
Intra/inter frame prediction: ~60%
Integer transform: ~10%
Error resilience: ~20%
De-blocking filter: ~10%
• Encoding: ~10x of MPEG-2’s CPU usage
• For current home PC users, realtime software
decoding (HDTV quality) could be achieved,
but not encoding
• Recommend software:
– QuickTime Alternative (www.free-codecs.com)
– X264 (www.video-lan.org/x264.html)
Licensing
• Standard: no free lunch
• Lessons from MPEG-4 Visual
– Was not friendly at the beginning…
• H.264
– Codec manufactures pay for the consumer
use by/between end users
– Cheap for free television provider
– Free internet broadcast till 2010.12.31
Applications
• HD DVD Video (DVDForum); Blu-ray Disk
(BDA) (High Profiles: mandatory)
• DVB in Europe, DMB in Korea, ISDB-T in
Japan
• Direct broadcast satellite TV (multi-region)
• 3GPP (optional feature)
• IETF: RFC3984 for RTP
• ITU-T: H.32x (in production)
• VoD