Transcript Source Coding for Video Application

Source Coding for Video
Application
Most Common video codecs
•
•
•
•
•
•
•
•
H.261
MPEG-1 Part 2
H.262/MPEG-2 Part 2
H.263/MPEG-4 Part 2
H.264/MPEG-4 Part 10/AVC
H.265/MPEG-H HEVC
WMV (Windows Media Video)
AVS (Standard in China, similar to H.264)
Most Common video codecs
• H.261
•
•
•
•
•
•
MPEG-1 Part 2
H.262/MPEG-2 Part 2
H.263/MPEG-4 Part 2
H.264/MPEG-4 Part 10/AVC
H.265/MPEG-H HEVC
WMV (Windows Media Video)
• AVS (Standard in China, similar to H.264)
H.261
• First ratified in 1988
• First digital video coding standard that was truly useful in practical
• All subsequent international video coding standards have been based closely on
the H.261 design,
• MPEG-1 Part 2
• H.262/MPEG-2 Part 2
• H.263
• MPEG-4 Part 2
• H.264/MPEG-4 Part 10
H.261 Structure
CIF and QCIF Frame Formats
GOB and Resynchronization
• Purpose of Group of Blocks is resynchronization
• GOB starts with a sync code (binary: 00000000 00000001)
• Within a GOB, encoded MBs don’t even start on byte boundaries
•
If there’s a bit error and you lose sync, or you join in the middle, you can’t decode the next bits (you
don’t know where you are in the bit stream)
•
Scan for the next GOB sync code, and then you can start decoding.
Macroblocks
• Macroblock is basic unit for compression
• Each macroblock is 16x16 pixels
• Represent as YUV 4:2:0 data
• 16x16 Luminance (Y) and subsampled 8x8 Cr, 8x8 Cb
• Represent this as 6 Blocks of 8x8 pixels:
Color Representation
• Requires three colors
• Red (R), Green (G), and Blue (B)
• Nominally requires three times B&W bandwidth
• Can compressed into 1.25 – 2 times B&W bandwidth by matrix:
• Luminance component (Y)
• Two chrominance components (I&Q or U&V)
RGB-YIQ conversion
RGB-YUV conversion
𝑌 = 0.299𝑅 + 0.587𝐺 + 0.114𝐵
𝐵−𝑌
𝑈=
2.03
𝑉=
𝑅−𝑌
1.14
Macroblock Coding
Three Rules:
• Don’t send it if is hasn’t changed since last frame.
• Intra-frame compression
• Do DCT, Quantize, Zig-zag, Run-length encoding, and Huffman coding.
• Just like JPEG
• Inter-frame compression
• Calculate difference from previous version of same block
• Can use motion estimation to indicate block being differenced can from a slightly different place in
previous frame
• Same DCT/quant/Huffman coding as Intra, but data is differences rather than absolute values
Intra-frame compression
Intra-frame coding of Microblock is very similar to JPEG
• 8x8 DCT (Discrete Cosine Transformation)
• Quantization
• Uniform quantizer (∆=8) for intra-mode DC coefficients
• Uniform threshold quantizer (∆=2,4,…,62) for AC coefficients in intra-mode and all
coefficients in inter-mode
• Zigzag Scan
• Order coefficients in zig-zag order
• Run-length encode
• Huffman code
Inter-frame compression
Basic compression process is the same as intra-frame compression, but the data is the differences from the
immediately preceding frame rather that the raw samples themselves.
•
Frame differencing
•
Often the amount of information in the difference between two frames is a lot less than in the second
frame itself.
•
Motion estimation
•
Motion in the scene will increase the differences
•
If you can figure out the motion (where each block came from in the previous frame):
•
Encode the motion as a motion vector (two small integers indicating motion in x and y directions)
•
Encode the differences from the moved block using DCT + quantization + RLE + Huffman encoding
General Description of H.261
H.261 achieves compression by removing redundancy
• Temporal redundancy:
reduced by motion compensation
• Spatial redundancy:
reduced by DCT
• Statistical redundancy:
reduced by entropy coding (e.g Huffman / arithmetic coding)
• Perceptual irrelevancy:
reduced by quantization
General Description of H.261
Two type of coding applied to the frames in H.261
• Intra-frame coding: (I frames)
•
•
Removes spatial redundancy by JPEG-like coding
Predictive coding: (P frames)
Normally, every I frame is followed by many
P frames.
• A new I frame would be used when there
is a scene change
• When there is no scene change, I frames
•
Reduce spatial redundancy by JPEG-like coding
•
Reduce temporal redundancy by motion compensation
are inserted periodically to stop any
•
Achieve higher compression ratio than intra-frame coding
error from propagating
General Description of H.261
Two type of coding applied to the frames in H.261
• Intra-frame coding: (I frames)
•
•
Removes spatial redundancy by JPEG-like coding
Predictive coding: (P frames)
•
Reduce spatial redundancy by JPEG-like coding
•
Reduce temporal redundancy by motion compensation
•
Achieve higher compression ratio than intra-frame coding
Intra-Block Encoding
Inter-Block Encoding
Bitstream Structure
H.263
• Son of H.261
• Standardized in 1996
• Replcaing H.261 in many applications
• Basic design is very similar to H.261 (DCT/Quantization based, using
intra or inter framecoding)
• Numerous optional improvements to improve compression, robustness, and
flexibility of use
H.263 Improvements
•
Half-pixel precision in motion vectors (vs full-pixel precision for H.261)
•
New options:
•
•
Unrestricted Motion Vectos,
•
Syntax-based arithmetic coding (replace RLE/Huffman)
•
Advance preodiction (Uses 4 8x8 blocks instead of 1 16x16 which gives better detail)
•
Forward and Backward frame prediction similar to MPEG
Five resolutions (H.261 only have QCIF and CIF):
SQCIF: 128x96
4CIF: 704x576
QCIF: 176x144
16CIF: 1408x1152
CIF: 352x288
History of video coding standards
H.261/263/264, MPEG-1/2/4
Several MPEG standards
• MPEG-1: video compression at 1.5Mbit/s, CD-ROM
• MPEG-2: video compression at 2-20Mbit/s, CCIR601, HDTV
• MPEG-3: dissolved (combined with MPEG2)
• MPEG-4: video compression at less than 28.8kbit/s, Videophone
Major Applications of Video Compression
Digital television broadcasting 2 – 5 Mbps
(10 – 20 Mpbs for HD)
MPEG-2
H.264/AVC
DVD video
HD-DVD
Blu-ray Disk
4 – 8 Mbps
(10 – 20 Mpbs for HD)
MPEG-2
H.264/AVC, VC-1
Internet video streaming
20 – 600 kbps
Proprietary, similar to H.263,
MPEG-4, or H.264/AVC, VC-1
Videoconferencing,
videotelephony
20 – 320 kbps
H.261, H.263, H.264/AVC
Video over 3G wireless
20 – 200 kbps
H.263, MPEG-4, H.264/AVC,
VC-1
References
H.261:
•
ITU-T (1988). "H.261 : Video codec for audiovisual services at p x 384 kbit/s - Recommendation
H.261 (11/88)". Retrieved 2010-10-21.
JPEG compression:
•
http://www.cs.cf.ac.uk/Dave/Multimedia/node234.html
H.263:
•
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=556485
MPEG:
•
http://www.google.com.hk/books?hl=zh-TW&lr=&id=GtCCIlpecBkC&oi=fnd&pg=PR5&dq=MPEG2&ots=Lxyhnl7BVG&sig=4G7kQp7Kj4Qmx5OuyhgBL1dAv1k&redir_esc=y#v=onepage&q=MPEG2&f=false
Q&A
Arrangement of Group of Pictures (GOPs) in a Video Sequence
I = Intra-frame coding
P = Predictive coding
B = Bidirectional coding
Coding Order of I,B,P-frame