Transcript Key Codes
Quantitative Comparison of
H.264/AVC and VP8 Video
Codecs for Video Conferencing
Scott Kristjanson
CMPT-820
Multimedia Systems
Instructor: Dr. Mohamed Hefeeda
Dec 2nd 2010
Presentation Overview
2
Project Overview
Methods
Results
Conclusions
Future Work
References
Scott Kristjanson – CMPT 820 - SFU
Slide 2
Project Overview
3
Compare Video Codecs for use with Video Conferencing:
• VP8
– by Google’s WebM project
• H.264/AVC – used x256 Open Source Codec
Scott Kristjanson – CMPT 820 - SFU
Slide 3
VP8 Overview
4
Video Codec owned by Google and made open Source
• WebM Project: Matroska container with VP8 video and Vorbis audio
• VP8 - Earlier versions VP7 (Skype), VP6 (Flash Video), VP3 (Theora)
Key Features:
• Adaptive Loop Filtering – Identifies Fast/Slow/Static regions and
adapts encoding to match for improved quality/performance
• Golden Frames – for storing reference point to background layer. Can
be used for reference without display them.
• Faster encoding time by avoiding use of Arithemetic VLE (CABAC).
• Dynamically scales resolution. No need to involve application,
encoder decides. No I-frame required as a transition point.
Structure:
• Frame same as H.264, Segment like a Slice
• Partitions used to separate critical data from video data
• I-Frame, P-Frames, Golden-Frames, no B-Frames due to Patents
For more information, see Google’s “Intro to VP8” slides at:
http://www.slideshare.net/DSPIP/google-vp8
Scott Kristjanson – CMPT 820 - SFU
Slide 4
Methods
5
Quantitative Comparison:
Invoke encoders/decoders with various presets:
• Manually from DOS, then use YUV_Compare to compute Y-PSNR
• ViCoS Codec Comparison utility from MSU – gives ranking only
• Used sequences foreman and salesman to simulate video conference calls
• Encode with bit rates 100, 800, 1500 kbps to produce bit rate curves
Extract Quantitative Data
•
•
•
•
•
Y-PSNR
– Signal to Noise Ratio for Luminosity
Y-SSIM
– Structural Simularity Data for Luminosity
Encoding Time
Frame Rate
Bit Rate
Qualitative Comparison
• View original YUV file and encoded/decoded files using YUVviewer
• Test VP8 in Video Conferencing System to verify Quantitative Data
Scott Kristjanson – CMPT 820 - SFU
Slide 5
Reference Video Sequences
6
Salesman
• Ideal choice for simulating video conference
• Not used since not available in CIF format
• Tried to convert from .avi to CIF format but ran out of time
Foreman
• Good choice, but with some background movement
• Used in detailed quantitative analysis
• Used with Vicos for codec over-all evaluation
Mobile
• Slow motion with detailed cartoon-like background
• Used with Vicos for codec over-all evaluation
Stefan
• Lots of motion and lots of background detail
• Used with Vicos for codec over-all evaluation
Scott Kristjanson – CMPT 820 - SFU
Slide 6
DOS Command Line Testing
7
x264 encode/decode:
vp8 encode/decode:
Visually compare
original video to
encoded video
with YUV_Viewer
Scott Kristjanson – CMPT 820 - SFU
Slide 7
Computing Y-PSNR with AviSynth
8
AVS script to read and compare two YUV files:
• Encoded/Decoded YUV video from DOS command line testing
• Original YUV video
yuv_compare_foreman.avs
loadplugin("C:\megui\tools\avisynth_plugin\rawsource.dll")
decodedClip = RawSource("C:\foreman_352x288_decoded_vp8.yuv",352, 288, "I420")
originalClip = RawSource("C:\foreman_352x288.yuv",
352, 288, "I420")
Compare(decodedClip, originalClip, "YUV", "foreman_yuv_compare.log")
foreman_yuv_compare.log
Mean
Max
Max
Absolute
Mean
Pos.
Neg.
Frame
Dev.
Dev.
Dev.
Dev. PSNR (dB)
----------------------------------------------------0
0.7591
+0.0252
5
-3
47.6277
1
0.7566
-0.0050
5
-4
47.6655
…
…
…
…
…
…
299
0.8597
+0.0070
4
-4
46.8949
Total frames processed: 300
Minimum
Average
Maximum
Mean Absolute Deviation:
0.6941
1.2438
3.7930
Mean Deviation:
-0.2703
-0.0194
+0.0527
PSNR:
33.2940
44.9776
48.2338
Overall PSNR:
41.3285
Scott Kristjanson – CMPT 820 - SFU
PSNR
Slide 8
MSU’s Video Codecs Scoring System (ViCoS)
9
Allows custom Codecs to be bound in and tested
•
•
•
•
bind in encoders and decoders (x264 and vp8)
define encoding and decoding parameters and presets to be tested
encodes/decodes test sequences automatically and computes distortion
generates relative ranking and relative scores for PSNR and SSIM only
Versions:
• Demo version used – provides ranking scores only
• Professional version – creates PNSR/SSIM charts automatically
Scott Kristjanson – CMPT 820 - SFU
Slide 9
Video Quality – Peak Signal to Noise Ratio (Y-PSNR)
10
Bit-Rate/Quality Trade-Off:
•
100kbps – Poor Quality
•
800kbps – Acceptible to good
• 1500kbps – Very Good to Excellent
43
41
39
VP8 Best
x264 Placebo
Y-PNSR (db)
37
VP8 Good1
VP8 Good2
VP8 Good
35
x264 Fast
VP8 Good4
x264 Default
33
vp8 1Pass Fast VBR
VP8 RT CBR w DL
PSNR Ranges:
38+
Excellent
36-37
Very good
34-35
Good
26-33
Poor
25 or less Garbage
31
29
27
VP8 RealTime CBR
x264 fast zero latency
x264 Baseline
x264 ultraFast
25
100
800
Bit Rate (kbps)
Scott Kristjanson – CMPT 820 - SFU
1500
Slide 10
Video Quality – Structural Simularity (SSIM)
11
SSIM score under 0.95 is quite noticible:
•
100kbps – Poor Quality for all codecs (especially x264 Fast Zero-Latency & Ultrafast)
•
800kbps – most codecs Acceptible to good (except VP8 RT CBR and x264 Ultrafast)
•
1500kbps – all Very Good
(except x264 Ultrafast)
1.00
0.90
VP8 Best
x264 Placebo
VP8 Good1
VP8 Good2
VP8 Good
Y-SSIM
x264 Fast
VP8 Good4
0.80
x264 Default
VP8 Fast VBR 1Pass
VP8 RT CBR w DL
VP8 RealTime CBR
x264 fast zero latency
x264 Baseline
x264 ultraFast
0.70
0.60
100
800
Bit Rate kbps
Scott Kristjanson – CMPT 820 - SFU
1500
Slide 11
Encoder Frame Rates
12
x264 Encoder significantly faster than VP8 for similar Quality
255
240
225
210
195
VP8 Best
VP8 Good
180
VP8 Good1
Frames per Second
165
VP8 Good2
VP8 Good4
150
VP8 Fast VBR 1Pass
VP8 RT CBR w DL
135
VP8 RealTime CBR
120
--------------------x264 Placebo
105
x264 Fast
x264 Default
90
x264 fast zero latency
75
x264 Baseline
x264 ultraFast
60
45
30
15
0
VP8
100
x264
VP8
800
x264
Bit Rate (kbps)
Scott Kristjanson – CMPT 820 - SFU
VP8
1500
x264
Slide 12
Good Codec Presets for Video Conferencing
13
Need min of 15 fps for Video Conferencing, prefer 30 fps
Want decent quality – Y-PSNR of 34 or better
Low Latency: under 150ms required
4 Presets met these requirements:
Codec Preset
Scott Kristjanson – CMPT 820 - SFU
Slide 13
Conclusion
14
VP8 and x264 provide similar video quality
• For higher bit rates, x264 PSNR better by 2%
• For lower rates, VP8 wins by almost 10%
x264 provides better performance in some cases
• x264 is between 36% and up to 2x faster than VP8
• A big improvement! VP8 used to be 5-20 times slower!
Scott Kristjanson – CMPT 820 - SFU
Slide 14
References:
15
[1] Image Quality Assessment: From Error Visibility to Structural Similarity
Zhou Wang, Alan Conrad Bovik, Hamid, Rahim Sheikh, and Eero P. Simoncelli
IEEE Transactions on Image Processing, Vol 13, No. 4, April 2004
http://ieeexplore.ieee.org.proxy.lib.sfu.ca/stamp/stamp.jsp?tp=&arnumber=1284395
[2] 6th Annual MPEG-4 AVC H.264 video codecs comparison
D. Vatolin, D. Kulikov, and A. Parshin, April 2010
http://compression.ru/video/codec_comparison/h264_2010/
Scott Kristjanson – CMPT 820 - SFU
Slide 15
Backup Slides
16
Scott Kristjanson – CMPT 820 - SFU
Slide 16
H.264/AVC Presets Used
17
Codecs tested with various presets and tunings
• Looking for good balance of quality and speed for video conferencing
• Each tested with 3 target Encoding bit rates: 100, 800, 1500 kbps
x264 Presets:
• Placebo
• 2-Pass Slower Encoding Speed, higher quality
• Fast (tested with 2 tunings)
• 2-Pass Faster Encoding Speed, good quality
• 2-Pass tuned with zero latency specified
• Default
• 1-pass with default settings
• Baseline
• 1-pass fast encoding but poor quality at lower bit rates
• Ultrafast
• When speed is all that matters!
Scott Kristjanson – CMPT 820 - SFU
Slide 17
VP8 Presets Used
18
Best
• 2-Pass encoding
• Maximimizes Quality but slower encode times
Good 2-Pass
• 2-Pass encoding, less quality but faster encode times
• measured with 1, 2, and 4 cpu cores specified
Good 1-Pass
• 1-Pass encoding, less quality but faster encode times
Fast VBR
• 1-Pass variable bit-rate fast encoding
RealTime CBR
• Encoder Frame buffer size 6
• with and without deadline specified
Scott Kristjanson – CMPT 820 - SFU
Slide 18
VP8 Support for MultiCore Processors
19
VP8 Encoding Rate vs # of CPUs
Preset Used: "Good"
90
80
70
Frames Per Second
60
50
1 CPU
2 CPUs
4 CPUs
40
30
20
10
0
100
800
1500
Bit Rate (kbps)
Scott Kristjanson – CMPT 820 - SFU
Slide 19
Encoding Bit Rate vs Target Bit Rate
20
Both Codecs appear to over-run target slightly
VP8 tracks to target bit-rate slightly better
Actual versus Target Bit Rate
1900
1800
1700
1600
1500
1400
1300
Target Bit Rate
1200
VP8 1500 kbps
1100
VP8 800 kbps
1000
VP8 100 kbps
x264 1500 kbps
900
x264 800 kpbs
800
x264 100 kbps
700
600
500
400
300
200
100
0
1
2
3
4
5
6
7
8
Code c
Scott Kristjanson – CMPT 820 - SFU
Slide 20
Video Quality – Structural Simularity (SSIM)
21
SSIM score under 0.95 is quite noticible:
•
100kbps – Poor Quality for all codecs (especially x264 Fast 0-Latency and Ultrafast)
•
800kbps – most codecs Acceptible to good (except VP8 RT CBR and x264 Ultrafast)
•
1500kbps – all Very Good
1.0
100 kbps
0.8
800 kbps
1500 kbps
0.7
64
Fa
s
ul
tr a
x2
64
B
4
x2
6
t
li n
e
as
e
en
cy
la
t
C
B
e
fa
st
ze
ro
lT
im
x2
8
Scott Kristjanson – CMPT 820 - SFU
VP
R
D
L
w
R
ea
VP
8
VP
8
R
T
V
st
Fa
C
BR
1P
as
s
D
ef
4
x2
6
BR
au
lt
d4
oo
G
8
VP
64
x2
G
8
Fa
oo
st
d
d2
VP
VP
8
G
oo
d1
oo
G
8
VP
x2
64
P
8
la
c
B
eb
o
es
t
0.6
VP
Y-SSIM
0.9
Slide 21