Transcript Focus on Production: Hugo Gaggioni's presentation

Compression Strategies for
SD/HD Production Applications
Hugo Gaggioni
Chief Technology Officer
Sony Broadcast and Professional Solutions Company
Agenda
 DTV Production Issues
 Bit Rates: Current and Future Positioning
 Types of Compression
 Bit Rate Reduction Strategies:
practical considerations for broadcast operations
 Applications
 Conclusion
DTV
Production
Issues
Foundations for the Simulcast 16:9
DTV/4:3 NTSC Era
 16:9 Digital Component HDTV
 16:9 Digital Component 525
 4:3 Digital Component 525
Broadcast Industry:
Near Term Imperatives for
DTV Program Origination
525-Line SDTV
-- Component
Digital Video Format
-- Widescreen
Switchable (16:9/4:3)
-- 4:2:2
Sampling Structure
Movement of Television Program Material
HDTV
16:9 DTV System
Up Conversion
NTSC
DTV
Channel
Down Conversion
4:3 525 System
NTSC
Channel
“Aspect Ratio”
The Biggest Program Origination
Issue in DTV
There is
NO Satisfactory Reconciliation
Between
16:9 and 4:3 Picture Framing
Aspect Ratio is:
a Picture Content Issue
The Larger DTV
Picture Content Issue
16:9 HDTV
Picture Content
is Fully
Protected
Picture Content
Curtailed
Format
Convert
16:9 SDTV
4:3 SDTV
Bit Rate Reduction Strategies
HDV, XDCAM-HD, HDCAM / SR
Bitrate positioning
100 - 142 Mbps
Broadcast
90 - 210 Mbps
50 - 90 Mbps
18 - 30 Mbps
??
NAB’05
XDCAM-HD
Technology Demontration
25 Mbps (DV,DVCAM,DVCPro)
Creator
B&I
Production
440 - 1.5 Gbps
(25, 19 Mbps)
25 Mbps DV
SD
HD
Types of Video Compression Algorithms

Intra-Picture Compression:
-- JPEG, DV, HDCAM, MPEG-4 SP

Inter-Picture Compression:
-- MPEG-1, MPEG-2, MPEG-4, AVC, WM9
Intra-Frame Compression system:
Application to 60i signals
…...
…...
Intra-Frame
Compression
Algorithm
…...
…...
1/30th sec
e.g.,
1080 / 60i
Pictures
…...
…...
Frame-based
Video Compressed
Data Files
Inter-Picture Compression system
11
22
I
33
44
55
66
77
88
99
10
10 11
11 12
12 13
13
B B P B B P B B P B B I
I = Intra-Coded Frame
P = Predictive-Coded Frame
B = Bidirectional-Coded Frame
Picture Quality vs. Compression Ratio
Total Picture
Quality
IntraFrame
Inter4-Field
0 dB
Reference
(“4:2:2”)
(MPEG-2)
Inter12-15 Frame
Intra
Field
2.7:1
4.4:1
8.8:1
22:1 Compression
Ratio
HDCAM and HDCAM-SR
Bit Rates and Performance Characteristics
HD Digital Video Sampling Formats
1920
“4 : 2 : 2”
1080
(Studio Quality)
“4 : 2 : 0”
1080
(Transmission,
field acquisition,
Prosumer Quality)
Y
960
1080
1920 / 1440
Y
CR
960 / 720
540
CR
960
1080
CB
960 / 720
540
CB
Uncompressed Bitrates for 1080 Video
1080 System:
1920 pixels x 1080 lines (active area),
59.94 fields/sec (I/P) , 23.98 frames/sec (P), 29.97 frames/sec (P)
 1080P 4:2:2, 10 bit 60 Hz
------
 1080I, 4:2:2, 10 bit 60Hz
------
 1080I, 4:2:2,
8 bit 60Hz
------
 1080I, 4:2:0,
8 bit 60Hz
------
 1080P, 4:2:2,
8 bit, 30 Hz
------
 1080P, 4:2:2,
8 bit, 24 Hz
------
2.48 Gbps
1.24 Gbps
994.33 Mbps
745.75 Mbps
994.33 Mbps
795.59 Mbps
Uncompressed Bitrates for 720 Video
720P System:
1280 pixels x 720 lines (active area),
59.94 fields/sec (P) , 23.98 frames/sec (P), 29.97 frames/sec (P)
 720P 4:2:2, 10 bit 60Hz
------
 720P, 4:2:2,
8 bit 60Hz
------
1.10 Gbps
883.85 Mbps
 720P, 4:2:2,
8 bit, 30 Hz - - - - - -
441.93 Mbps
 720P, 4:2:0,
8 bit, 30 Hz - - - - - -
331.55 Mbps
 720P, 4:2:2,
8 bit, 24 Hz - - - - - -
353.59 Mbps
HDCAM-SR Recording Process
Recording
&
Playback
SMPTE 292M
SMPTE 292M
1920 x 1080
(10-bits)
1920 x 1080
1242 Mbps
1104 Mbps
440
Mbps
1242 Mbps
(10-bits)
1104 Mbps
SMPTE 296M
SMPTE 296M
1280 x 720
1280 x 720
(10-bits)
(10-bits)
Prefilters in HDTV Acquisition
HDTV Camera
The
Scene
Lens
Lens
CCD
Film
Film Camera
Lens
CCD
HDTV Telecine
Digital
VTR /
Server
HDCAM Recording Process
Input
Processing
SMPTE
292M
994 Mbps
1920
x
1080
Horizontal
Digital
Filter
622
Mbps
&
Down
Sampling
(60Hz)
Recording
&
Playback
142
Mbps
1440
x
1080
Output
Processing
SMPTE
292M
622 Mbps
Up
Sampling
&
Horizontal 994 Mbps
Digital
Filter
1920
x
1080
HDCAM/SR Recording Strategies
2/3 inch
CCDs
(1920x1080)
HDCAM
994
Mbps
Digital Pre-filter
HDTV
(1440x1080)
994
Mbps
HDTV
4.4 : 1
Compression
2.7 : 1
Compression
HDCAM-SR
142 Mbps
Video
Recording
440 Mbps
Video
Recording
HDCAM, HDCAM-SR,
Compression Ratios (60 Hz material)
HDCAM-SR (10-bit):
(approx.)
1.24Gbps
440
= 2.78
622Mbps
142
= 4.4
HDCAM (8-bit, 1080i):
(approx.)
Bandwidth Comparison between
HDCAM, HDCAM-SR
24
MHz
30
MHz
HDCAM-SR
HDCAM
10
MHz
20
MHz
30
MHz
CCD Imager HORIZ MTF
Carrier
872 TVL/ph
100 %
CCD Aperture
Horiz
MTF
Optical
Low Pass
50%
With
Lens
Effective
Aperture
30
24
37 MHz
Nyquist Limit
Sin x
x
Horiz. freq
TVL/ph
74.25 MHz Fs
Spectral Response of Video
created in an HDTV Camera
HDCAM
24 MHz
Narrow Angle Shot
MTF
(Horiz)
Medium Angle
HDCAM-SR
30 MHz
SMPTE 274M
Wide Angle Shot
360 5.75 MHz
(SDTV 16:9)
700
872 TVL/Ph
(30 MHz)
XDCAM-HD and HDV
Bit Rates and Performance Characteristics
@NAB2005
Acquisition Field Operation
Prototype/Technology & Concept Demos
Proxy recording on Memory
Proxy Wireless Transmission
Camcorders
Field Recorder
Networking & Edit
Viewer
Technology Demonstration
External Drive
5 inch bay Drive
Deck
Cart
Transfer &
Archives
Juke Server Concept
HD
Sony AV/IT Solutions
Field operation
toward HD
PC Centric
Proxy
Viewer/Edit
Wireless
LAN
High Rez. Edit
PDW-510/530
Memory
Card
5 inch bay
drive
MXF
HD
i-Link
PDW-D1
MediAvenue
High Rez. Edit
On-air
PDW-1500
i-Link
LAN/WAN
HD
Edit room
i-Link
Microwave
Satellite
HDV
HD
Transmission room
Juke Server
Field Recorder
XDCAM Cart
HD
Master
Distribution
Back to Station
Archive
Pool feed
HD Disc
HD
HD
HDHDHDHDHDHD
Ingest room
Archive room
Network Centric
Consideration of HD Infrastructure
Field operation
PC Centric
Proxy
Viewer/Edit
Wireless
LAN
High Rez. Edit
5 inch bay
drive
PDW-510/530
Longer Recording
Memory Time
Card
HD handling on PC
HD
i-Link
PDW-D1
High Rez. Edit
HDV
i-Link
On-air
PDW-1500
i-Link
HD
MediAvenue
Faster Transfer
LAN/WAN
Network Bandwidth
Microwave
Satellite
Field Recorder
HD
Transmission room
Edit room
Juke Server
Contribution Rate
Transcoding
HD
XDCAM Cart
Realtime Back to Station
Pool feed
HD
HD Quality
HD
Disc
Ingest room
Archive Archiving
Rate
Master
Distribution
HDHDHDHDHDHD
Archive room
Network Centric
Considerations for Codec Selection
For HD ENG & News Magazine production applications
Non-proprietary Codec
Longer Recording Time
Editing Capability: Jog & Shuttle
High speed Intra Transcoding for NLE
Network and Server friendly
Bandwidth and transfer speed (DV 25Mbps – IMX 50Mbps)
High speed Long-Intra Transcoding
Contribution/Archive format and bitrate
Microwave from ENG
Strong support for Long GOP operation
 There are 3 important changes occurring in the TV industry:
-- Tape to Disc with MXF file format
-- Intra codec to Long GOP codec
-- Standard definition to High definition
 Long GOP saves operational costs to TV facilities
 Contents creation editing has 2 types of environments:
High
Speed
GOP
Transformation
“Long GOP”
and
“Intra
GOP”
 Key technologies for handling
these
streams
are:
Long
Intra
Intra
Long
Intra
Long
Long
Intra
Intra
Post Production between
Archive Long
Distribution
-- High speed
GOP transformation
and Intra.
Storage
Acquisition
Live Production
-- Direct Stream editing
Long
Intra
-- Artifact-free bitstream variable playback for search
and decision making. Stream Direct Edit
MPEG Line-up
MPEG-1
MPEG-2
MP@ML
AVC
4:2:2P@ML
up to 50Mb L-GOP
MPEG-4
4:2:0 -15Mb
MP@HL 1440
Up to 60 Mbps L-GOP
MPEG-7
MPEG-2 Architecture
PROFILE
FRAME TYPES
CHROMA
SAMPLING
SIMPLE
I&P
4:2:0
HIGH
LEVEL
Samples/Line
Lines/Frames
Frames/Sec
Max Bit-Rate (Mbps)
HIGH 1440
Samples/Line
Lines/Frames
Frames/Sec
Max Bit-Rate (Mbps)
MAIN
Samples/Line
Lines/Frames
Frames/Sec
Max Bit-Rate (Mbps)
LOW
Samples/Line
Lines/Frames
Frames/Sec
Max Bit-Rate (Mbps)
SDTV
720
576
30
15
MAIN
I, P & B
4:2:0
SNR
I, P & B
4:2:0
1920
1152
60
80
1440
1152
60
60
HDTV
720
576
30
15
720
576
30
15
352
288
30
4
SPATIAL
HIGH
I, P & B
4:2:0
I, P & B
4:2:2 & 4:2:0
1920
1152
60
100
1440
1152
60
60
1440
1152
60
80
720
576
30
15
720
576
30
20
4:2:2
I, P & B
4:2:2 & 4:2:0
1920
1080
60
300
720
608
30
50
MPEG GoP : Group of Pictures
11
22
I
33
44
55
66
77
88
99
10
10 11
11 12
12 13
13
B B P B B P B B P B B I
I = Intra-Coded Frame
P = Predictive-Coded Frame
B = Bidirectional-Coded Frame
MPEG Hierarchy
Max. Bit rate
HDCAM-SR
Downward
Compatibility
MPEG-4 Studio Profile
MPEG-2 422P@HL
HDV
MPEG-2 MP@HL-1440
MPEG-2 422P@ML
XDCAM-HD
MPEG IMX,
XDCAM
at 50Mbps
1.2 Gbps
600 Mbps
200
~300 Mbps
80 Mbps
50 Mbps
15 Mbps
MPEG-2 MP@ML
MPEG-1
Betacam SX
at 18 Mbps
1.5
~ 3.0 Mbps
HDV
HDV format features
1. Use of DV tape as recording media
2. MPEG-2 Compression
3. 1080i / 720p Specification
4. Advanced Error Correction
5. Good Quality Compressed Audio
HD/ SD ¼ inch tape formats
HDV format
DV/DVCAM format
720p
specification
720/25p, 720/30p
720/50p, 720/60p
480/60i (NTSC)
1080i
specification
1080/50i, 1080/60i
576/50i (PAL)
MPEG-2 encoding
DV format SD specifications
(Inter-frame compression)
(Intra-frame compression)
DV Cassette Tape
HDV format Specifications
1080i specification
720p specification
Video Signal
1080/60i, 1080/50i
720/60p, 720/30p
720/50p, 720/25p
Number of Pixels
1440(H) X 1080(V)
1280(H) X 720(V)
Approx. 25Mbps
Approx. 19Mbps
Video
Bit rate after Compression
Audio
Compression
MPEG-1 Audio Layer II
Sampling frequency
48 kHz
Quantization
16 bits
Bit rate after compression
384 kbps (2ch)
DIGITAL HD VIDEO CAMERA RECORDER
HVR-Z1
World’s first HDV 1080i Camcorder!
HDV Recording Process
Input
Processing
746 Mbps
1440
x
1080
Chroma
Down
560
Mbps
Sampling
(4:2:0)
Record
&
Playback
25
Mbps
1440
x
1080
Output
Processing
560 Mbps
Analog
Output
Or
25 Mbps
streams
Analog
or
25 Mbps
HD-TS
XDCAM HD and HDV formats
XDCAM HD
Media
HDV
Professional Disc
1080 / 59.94i, 50i,
23.98P, 25P, 29.97P
1080 / 59.94i, 50i
MPEG2 MP@HL(1440)
MPEG2 MP@HL-(1440)
Various Bit-rates to
accommodate from 60 minutes
to more than 120 minutes of HD
storage
25Mbps CBR @ 1080i
19Mbps CBR @ 720P
Audio CODEC
No compression
MPEG1 Audio Layer 2
Audio format
4ch
48kHz sampling
16 (or 24bit TBD)
2ch / 48kHz /16bit sampling
Compressed bit rate : 384kbps
Video format
Video CODEC
Video Bit rate
(Recording time)
DV tape
(720 / 25P, 50P, 29.97P, 59.94P)
276min(full size) / 60min(mini-DV)
MXF
for
MPEG L-GOP
MXF Structure Comparison
MPEG HD
File Body
File Header
H
P
P
Header
Metadata
B
P
P
Edit
Unit
Edit
Unit
Edit
Unit
…
Edit
Unit
B
Index
P
P Table
Edit
Unit
File Footer
B
Index
P
P Table
Edit
Edit
… Unit
Unit
Edit
Unit
…
Edit
Unit
F
B
Index
P
P
Table
P
P
Random
Index
Pack
DV-DIF
File Header
H
P
P
Header
Metadata
Index
Table
File Body
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
File Footer
…
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
F
P
P
D-10
File Header
H
P
P
Header
Metadata
Index
Table
File Body
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
File Footer
…
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
Edit
Unit
MXF
MPEG HD
DV-DIF
D-10
Edit Unit
Variable Length
Constant Length
Constant Length
Body Partition
Partition size: 1GOP to 10sec
No Body Partitions
No Body Partitions
Index Table
In the next Body Partition
In File Header
In File Header
Random Index Pack
attached
not attached
not attached
F
P
P
Related SMPTE Standards
Item
> File
format
> Operational pattern
> Container
> Video Mapping
> Audio mapping
> System Item
> KLV
> MXF Guideline
SMPTE standard
377M
378M
379M
381M
382M
385M
336M
EG41
MXF
Op1a
MXF GC
MPEG into MXF GC
BWF/AES3
(CP compatible)
XDCAM-HD Recording Process
Record
&
Playback
Input
Processing
746 Mbps
1440
x
1080
Chroma
Down
Sampling
(4:2:0)
(60Hz)
Various
560 Mbps
560
Mbps
Bit-Rates
1440
x
1080
Output
Processing
SMPTE
292M
Up
Sampling
&
Horizontal 994 Mbps
Digital
Filter
1920
x
1080
XDCAM-HD, HDV
Compression Ratios
HDV, (25 Mbps) (8-bit, 1080i):
(approx.)
560Mbps
25
=
22.4
=
16
=
22.4
Examples of possible bit-rates
and compression ratios
XDCAM-HD (35 Mbps) (8-bit, 1080i):
(approx.)
560Mbps
35
XDCAM-HD (25 Mbps) (8-bit, 1080i):
560Mbps
(approx.)
25
Frequency Characteristics
for 25 Mbps L-GOP Compression
24 MHz Band
Picture Quality
Headroom
MPEG Quality: MP@HL-1440 -- 35 Mbps
Excellent
5
Good
4
CCIR
500
Units
35 Mbps
Fair
3
Poor
2
Bad
1
0
Time
MPEG Quality:
MP@ML-1440 -- 35 Mbps & 25 Mbps
Excellent
5
Good
4
CCIR
500
Units
35 Mbps
25 Mbps
Fair 3
Poor 2
Bad 1
0
Time
MPEG Quality:
MP@MHL-1440 -- 35, 25, & 18 Mbps
35 Mbps
Excellent 5
Good 4
CCIR
500
Units
25 Mbps
Fair 3
18 Mbps
Poor 2
Bad 1
0
Time
Frequency Characteristics
for 18 Mbps L-GOP Compression
Preliminary Bandwidth Comparison between
HDCAM/SR, XDCAM-HD and HDV
24
MHz
30
MHz
HDCAM-SR
HDCAM
10
MHz
20
MHz
30
MHz
Other HDV Products
Compression Ratios
Recall: 720p bit-rate
8-bit: “4:2:0”
(1280 x 720) x 12 x 59.94
(HDV, 59.94 Hz):
(approx.)
=
662.89 Mbps
662.89 Mbps
19
=
34.9
Too High !!!
(HDV, 29.97 Hz):
(approx.)
331.45 Mbps
19
=
17.45
XDCAM-HD, HDV: Encoded Streams
VBR, CBR, Constant Bytes per GOP
Input
Video
Black
Plain
Complex
Plain
VBR
Stuffing
Averaged Byte Size
CBR
Target Byte Size
CBG
Networking
Streaming
vs. File transfer
Many standards
SDI / SDTI (AV and data streams)
 Ethernet/Gigabit Ethernet (files)

Fibre Channel (files and streams)
 i.Link (Firewire)

(FAM:
MXF files), (AV/C: DV streams),
(HD-TS: MPEG packets)
MPEG Streams: TS, PES, ES
ES: Elementary Stream
PES: Packetized ES
Stream structure
PES Packet
PES
header
ES
Access Units
PES Packet
PES
header
ES
Access Units
Transport Header
TS: Transport Stream
TS Packet (188 Byte)
HD-TS: High Definition TS
MPEG ES and TS
MPEG Elementary Stream
> Less
process, Less time delay
> Variable Speed Operation
> Suitable for Operations in Station
MPEG Transport Stream
> Suitable
for Transmission, Tape-based
streaming (synchronous operations)
HDV, XDCAM-HD, HDCAM/SR
Applications
HDCAM-SR:
High-End Production/Post-Production,
Electronic Cinematogarphy, Compositing, Digital
Intermediates, Telecine Masters, Computer
Graphics/Animation, RGB/60P/3-D applications
HDCAM:
Broadcast Production, Legacy Material Up-Conversion,
Limited Post-production, 24P Acquisition, HD Commercials,
24P Made-for-TV “Films”, HD-ENG, Sports, Real-time NLE
operations
HDV, XDCAM-HD, HDCAM/SR (cont’d)
Applications
XDCAM-HD
Electronic Cinematography, High-end videographer,
Broadcast Production/ENG, Variable-Frame
Acquisition, Extended duration Recordings, HD/DV
switchable recording,
HDV
Entry level HD solution. Products positioned at the
top end of the current DVCAM line-up.
High-end Corporate/Industrial/Professional
videography. Possible Broadcast usage.
Easy migration from SD to HD: HDV / DVCAM / DV
switchable recording
Conclusion
 Marketplace availability of multi-format HD products
 Manufacturers have HD products at all price / performance
levels to complete any production need
 Applications require a “diverse” set of “tools”
-- Use the product that meets or exceeds the
application’s requirements
 For a technology decision
-- Evaluate the production’s expected finished results
-- The allocated budget
-- The end-user’s experience with HD
Conclusion
Future HD Technologies
 A new range of HD products, at various price and
performance levels for production applications, has been
introduced by the use of MPEG L-GoP compression
 Sony is developing technology to complete full
manipulation of these MPEG compressed bit-streams:
-- Variable speed playback
-- Cut-only and effects editing
-- Multi-stream effects