Transcript Image Storage and Compression
Slide 1
این دو عالم علم دارد در نهاد منتخب
وان جهانی رمز دارد در حروف مختصر
سنایی غزنوی
Slide 2
Image/Video Compression & VOD
برنا فیروزي
نوید زرین درفش
محمود قدیمي
مهندسي فناوري اطالعات
بهار 88
Slide 3
Image Compression
3
Slide 4
Compression
فشرده سازی ( ،)Compressionپردازشی است که با حذف
اطالعات اضافی ،داده ها را به عالیم دیجیتالی کاهش می دهد.
این پردازش بسته به پهنای باند مورد نیاز برای انتقال داده ها و
میزان فضای ذخیره سازی ،داده ها را کاهش می دهد.
کاهش پهنای باند مورد نیاز امکان انتقال داده های بیشتری را در
یک زمان واحد می دهد.
كاهش افزونگی ()redundancy
4
Slide 5
ديجيتال ،آنالوگ
پردازش تصاویر در فضاي دیجیتال انجام مي شود
اگر ما یك منبع تصویري آنالوگ داشته باشیم باید قبل از انجام هر
عمل فشرده سازي ابتدا به آن را به دیجیتال تبدیل كنیم
5
Slide 6
Purpose of Image Compression
6
Saving storage space
Saving transfer time
Easy processing
Easy to transmitted over network
reduce cost
Slide 7
هدف
در حالت ایده آل ما خواستار
حداكثر كیفیت تصویر
حداقل فضاي ذخیره سازي و پردازش منابع
ما نمي توانیم هر دو هدف را در بهترین شرایط داشته باشیم
بهترین فشرده سازي چگونه است؟
7
Slide 8
Why we want to compress?
To transmit an RGB 512X512, 24 bit image
via modem 28.2 kbaud(kilobits/second)
( 512 512 pixels )( 24 bits / pixel )
( 28 . 8 1024 bits / second )
8
213 second
( 4 min)
Slide 9
Image Compression Coding
lossless
compression
Image
Compression
lossy
compression
9
Huffman
Coding
Run-length
encoding
Predictive
Coding
Transform
Coding
Slide 10
دو كالس اصلي فشرده سازي تصاوير
)Lossyپراتالف(
از روي دیتاهاي تصویر ذخیره
شده مي توان به تصویري نزديك
به تصویر اصلي رسید
تصویر ذخیره شده خود تصویر
اصلی نیست ،بلکه شبیه آن است و
اطالعاتی را از دست داده است
Compression rate:
high compression
( Losslessبدون اتالف(
از روي دیتاهاي تصویر ذخیره
شده مي توان دقیقا به تصوير
اصلي رسید
تصویر ذخیره شده بدون از دست
دادن كمترین داده ای ،خود تصویر
است.
Compression rate:
)2:1 (at most 3:1
10
Slide 11
General compression system model
O utput ima ge
Input im a ge
Source decoder
Source encoder
D ecoder
E ncoder
C hannel decoder
C hannel encoder
C hannel
11
Slide 12
Compression System Model
Compression
Input
Preprocessing
Encoding
Compressed
File
• Decompression
Compressed
File
12
Decoding
Postprocessing
Output
Slide 13
Compression Ratio
Compressio n Ratio
Uncompress ed File Size
Compressed
File Size
CR
Ex Image 256X256 pixels, 256 level grayscale
can be compressed file size 6554 byte.
Original Image Size = 256X256(pixels) X
1(byte/pixel)
= 65536 bytes
compressio n Ratio
13
65536
6554
10
Slide 14
Bits per Pixel
(2) Bits per Pixel Number of Bits
Number of Pixels
Ex Image 256X256 pixels, 256 level grayscale
can be compressed file size 6554 byte.
Original Image Size = 256X256(pixels) X
1(byte/pixel)
= 65536 bytes
Compressed file
= 6554(bytes)X8(bits/pixel)
= 52432 bits
14
Bits per Pixel
52432
65536
0 .8
Slide 15
Key of compression
Reducing Data but Retaining Information
Various amounts of data can be used to represent the same
amount of information. It’s “Data redundancy”
Relative data redundancy
RD 1
15
1
CR
Slide 16
Entropy
Average information in an image.
L 1
Entropy p k log 2 ( p k )
k 0
• Average number of bits per pixel
L 1
La
l
k
pk
k 0
pk
16
nk
n
, where k 0 ,1, , L 1
Slide 17
Compression Standard
Standard:
17
Image
Video
ISO
JPEG
JPEG2000
MPEG1,MPEG2,
MPEG4, MPEG7
ITU
N/A
H.261, H.263, H.263+,
H.26L
Slide 18
Image Compression Coding
lossless
compression
Image
Compression
lossy
compression
18
Huffman
Coding
Run-length
encoding
Predictive
Coding
Transform
Coding
Slide 19
Loseless Compression
19
No data are lost
Can recreated exactly original image
Often the achievable compression is mush less
Slide 20
Huffman Coding ( VLC,Entropy)
Using Histogram probability
5 Steps
1.
2.
3.
4.
5.
20
Find the histogram probabilities
Order the input probabilities(smalllarge)
Addition the 2 smallest
Repeat step 2&3, until 2 probability are left
Backward along the tree assign 0 and 1
Slide 21
Huffman Coding(cont)
40
30
20
10
Step 1 Histogram Probability
p0 = 20/100 = 0.2
p1 = 30/100 = 0.3
p2 = 10/100 = 0.1
p3 = 40/100 = 0.4
0
1
2
3
Step 2 Order
p3 0.4
p1 0.3
p0 0.2
p2 0.1
Slide 22
Huffman Coding(cont)
Step 3,4 Add 2 smallest
p 3 0 .4 0 .4 0 .4
p1 0 . 3 0 . 3
0 .6
p 0 0 .2
0 .3
0 .6
0 .4
p 2 0 .1
Step 5 assign 0 and 1
22
Natural Code
00
Probability
0.2
Huffman Code
010
01
0.3
00
10
0.1
011
11
0.4
1
Slide 23
Huffman Coding(cont)
The original Image :average 2 bits/pixel
The Huffman Code:average
3
La
l
i
p i 3 ( 0 . 2 ) 2 ( 0 . 3 ) 3 ( 0 . 1) 1( 0 . 4 ) 1 . 9
i0
3
Entropy p i log 2 ( p i )
i0
( 0 . 2 ) log 2 ( 0 . 2 ) ( 0 . 3 ) log 2 ( 0 . 3 )
(
0
.
1
)
log
(
0
.
1
)
(
0
.
4
)
log
(
0
.
4
)
2
2
23
1 . 846 bits / pixel
Slide 24
Run Length Encoding
24
Run-length encoding (RLE) is a very simple form
of data compression in which runs of data (that is,
sequences in which the same data value occurs in
many consecutive data elements) are stored as a
single data value and count, rather than as the
original run. This is most useful on data that
contains many such runs: for example, relatively
simple graphic images such as
icons, line drawings, and animations.
Slide 25
Run-Length Coding
25
Counting the number of adjacent pixels with
the same gray-level value
Used primarily for binary image
Mostly use horizontal RLC
Slide 26
Run-Length Coding(cont)
Binary Image 8X8
26
horizontal
0
0
0
0
0
0
0
0
1st Row
8
1
1
1
1
0
0
0
0
2nd Row
4,4
0
1
1
0
0
0
0
0
3rd Row
1,2,5
0
1
1
1
1
1
0
0
4th Row
1,5,2
0
1
1
1
0
0
1
0
5th Row
1,3,2,1,1
0
0
1
0
0
1
1
0
6th Row
2,1,2,2,1
1
1
1
1
0
1
0
0
7th Row
4,1,1,2
0
0
0
0
0
0
0
0
8th Row
8
Slide 27
Example of RLE
Let us take a hypothetical single scan line, with B
representing a black pixel and W representing white:
WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWW
WWWWWWWWWWWWWWWWWWWWBWWWWWWWWWW
WWWW
If we apply the run-length encoding (RLE) data compression
algorithm to the above hypothetical scan line, we get the
following:12W1B12W3B24W1B14W
27
Slide 28
Lossy Compression
28
Allow a loss in the actual image data
Can not recreated exactly original image
Commonly the achievable compression is
mush more
Such as JPEG
Slide 29
Predictive Coding
Common Predictive
Coding
29
DM (Delta Modulation)
DPCM (Differential Pulse
Code Modulation)
Slide 30
The principle of Predictive Coding
30
The system consists of an encoder and a decoder,
each containing an identical predictor. As each
successive pixel of the input image, is introduced to
the encoder, the predictor generates the anticipated
value of that pixel based on some number of past
inputs. The output of the predictor is then rounded
to the nearest integer.
Slide 31
Predictive coding model I
Input image
n
fn
Symbol
encoder
fˆn
Predictor
Compressed image
Output image
fn'
Predictor
31
‘
fˆ n '
Symbol
decoder
Slide 32
Predictive coding model II
The predictor :
fˆ n F ( f n 1 , f n 2 , , f n k )
n
f n fˆ n
The symbol encoder : generate the next element of the
compressed data stream
Decoder : perform the inverse of encoding
The linearity predictor :
32
fˆ n F ( f n 1 , f n 2 , , f n k )
n 1
a
kl
k
fk ,
a
k
1
Slide 33
Delta Modulation I
Delta modulation (DM or Δ-modulation) is an analog-todigital and digital-to-analog signal conversion technique
used for transmission of voice information where quality
is not of primary importance.
DM is the simplest form of differential pulse-code
modulation (DPCM) where the difference between
successive samples is encoded into n-bit data streams. In
delta modulation, the transmitted data is reduced to a 1bit data stream.
33
Slide 34
Differential Pulse Code Modulation
34
Differential Pulse Code Modulation (DPCM) compares
two successive analog amplitude values, quantizes and
encodes the difference, and transmits the differential
value.
Slide 35
Transform Coding I (DCT)
35
Transform coding is a type of data compression for
"natural" data like audio signals or photographic
images.
The transformation is typically lossy, resulting in a
lower quality copy of the original input.
Slide 36
Transform Coding II
36
Slide 37
Transform Coding III
A transform coding system
Input image
Construct
subimages
Compressed
image
37
Quantizer
Forward
transform
Symbol
encoder
Compressed
image
Symbol
decoder
Inverse
transform
Merge
subimages
Slide 38
BMP (Bitmap) - lossless
38
Use 3 bytes per pixel, one
each for R, G, and B
Can represent up to 224 =
16.7 million colors
No entropy coding
File size in bytes =
3*length*height, which can be
very large
Can use fewer than 8 bits per
color, but you need to store
the color palette
Performs well with ZIP, RAR,
etc.
Slide 39
GIF (Graphics Interchange Format)
Can use up to 256 colors from
24-bit RGB color space
–
39
If source image contains
more than 256 colors, need
to reprocess image to fewer
colors
Suitable for simpler images
such as logos and textual
graphics, not so much for
photographs
Slide 40
JPEG (Joint Photographic Experts
Group) - lossly
40
Most dominant image
format today
Typical file size is about
10% of that of BMP (can
vary depending on
quality settings)
Unlike GIF, JPEG is
suitable for photographs,
not so much for logos
and textual graphics
Slide 41
Joint Picture Expert Group
41
The name "JPEG" stands for Joint Photographic Experts Group,
the name of the committee that created the standard. The group
was organized in 1986, issuing a standard in 1992, which was
approved in 1994 as ISO 10918-1. JPEG is a commonly used
method of compression for photographic images. The degree of
compression can be adjusted, allowing a selectable tradeoff
between storage size and image quality.
Slide 42
JPEG2000
42
JPEG 2000 is a wavelet-based image compression standard. It
was created by the Joint Photographic Experts Group committee
in the year 2000 with the intention of superseding their original
discrete cosine transform-based JPEG standard (created about
1991).
Slide 43
JPEG Encoding Steps
43
Preprocess image
Apply 2D forward DCT
Quantize DCT coefficients
Apply RLE, then entropy encoding
Slide 44
JPEG Block Diagram
8x8 blocks
Source
Image
B
G
R
DCT-based encoding
FDCT
44
Quantizer
Entropy
Encoder
Table
Table
Compressed
image data
Slide 45
Image Compression- JPEG
Using the DCT, the entries in Y will be organized
based on the human visual system.
The most important values to
our eyes will be placed in the
upper left corner of the matrix.
The least important values
will be mostly in the lower
right corner of the matrix.
Most
Important
SemiImportant
Least
Important
45
Slide 46
JPEG
8 x 8 Pixels
46
Image
Slide 47
Image Compression
Gray-Scale Example:
Value Range 0 (black) --- 255 (white)
63 33 36 28 63 81
27 18 17 11 22 48
72 52 28 15 17 16
132 100 56 19 10 9
187 186 166 88 13 34
184 203 199 177 82 44
211 214 208 198 134 52
211 210 203 191 133 79
47
86 98
104 108
47 77
21 55
43 51
97 73
78 83
74 86
Slide 48
Image Compression
2D-DCT of matrix
-304 210
-327 -260
93 -84
89 33
-9 42
-5 15
10
3
12 30
48
104
67
-66
-19
18
-10
-12
0
-69
70
16
-20
27
17
-1
-3
10
-10
24
-26
-7
32
2
-3
20
-15
-2
21
-17
-15
3
-6
-12 7
21 8
-5 9
-3 0
29 -7
-4 7
-2 -3
12 -1
Slide 49
Image Compression
-304 210
-327 -260
93 -84
89 33
-9 42
-5 15
10
0
0
0
104
67
-66
-19
18
0
0
0
-69
70
16
-20
0
0
0
0
10 20 -12 0
-10 -15 0 0
24 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
As you can see, we save a little over half the original
memory.
49
Slide 50
Reconstructing the Image
New Matrix and Compressed Image
55 41 27 39 56 69 92 106
35 22 7 16 35 59 88 101
65 49 21 5 6 28 62 73
130 114 75 28 -7 -1 33 46
180 175 148 95 33 16 45 59
200 206 203 165 92 55 71 82
205 207 214 193 121 70 75 83
214 205 209 196 129 75 78 85
50
Slide 51
Can You Tell the Difference?
Original
51
Compressed
Slide 52
Image Compression
Original
52
Compressed
Slide 53
Example - One everyday photo with
file size of 2.76 MB
53
Slide 54
Example - One everyday photo with
file size of 600 KB
54
Slide 55
Example - One everyday photo with
file size of 350 KB
55
Slide 56
Example - One everyday photo with
file size of 240 KB
56
Slide 57
Example - One everyday photo with
file size of 144 KB
57
Slide 58
Example - One everyday photo with
file size of 88 KB
58
Slide 59
Analysis
Near perfect image at 2.76M, so-so image at 88K
Sharpness decreases as file size decreases
Which file size is the best?
–
–
59
No correct answer to this question
Answer depends upon how strict we are about image quality, what
purpose image is to be used for, and the resources available
Slide 60
Conclusion
60
Image compression is important
Image compression has come a long way
Image compression is nearly mature, but there
is always room for improvement
Slide 61
Video Compression
Slide 62
Video Data Size
size of uncompressed video in gigabytes
1 sec
1 min
1 hour
1000 hours
1920x1080
0.19
11.20
671.85
671,846.40
1280x720
0.08
4.98
298.60
298,598.40
640x480
0.03
1.66
99.53
99,532.80
320x240
0.01
0.41
24.88
24,883.20
160x120
0.00
0.10
6.22
6,220.80
image size of video
1280x720 (1.77)
62
640x480 (1.33)
320x240
160x120
Slide 63
Video Bit Rate Calculation
width * height * depth * fps
= bits/sec
compression factor
width ~ pixels(160, 320, 640, 720, 1280, 1920, …(
height ~ pixels
(120, 240, 480, 485, 720, 1080, …(
depth ~ bits (1, 4, 8, 15, 16, 24, …(
fps ~ frames per second (5, 15, 20, 24, 30, …(
compression factor (1, 6, 24, …(
63
Slide 64
Effects of Compression
storage for 1 hour of compressed video in megabytes
1:1
3:1
6:1
25:1
100:1
1920x1080
671,846
223,949
111,974
26,874
6,718
1280x720
298,598
99,533
49,766
11,944
2,986
640x480
99,533
33,178
16,589
3,981
995
3 bytes/pixel, 30 frames/sec
64
320x240
24,883
8,294
4,147
995
249
160x120
6,221
2,074
1,037
249
62
Slide 65
Channel Bandwidths
65
Slide 66
Channel Bandwidth
66
Slide 67
Application Requirements
67
Slide 68
Source Video Formats
68
Slide 69
The Need for Video Compression
High-Definition Television (HDTV)
1920x1080
– 30 frames per second (full motion)
– 8 bits for each three primary colors (RGB)
Total 1.5 Gb/sec!
–
Cable TV: each cable channel is 6 MHz
Max data rate of 19.2 Mb/sec
– Reduced to 18 Mb/sec w/audio + control …
Compression rate must be ~ 80:1!
–
69
Slide 70
The Need for Video Compression
Some figures
–
Uncompressed video -> big amount of data
–
Communication and storage capacities limits
70
Color picture 800x320 pix, 24 bits/pix -> 6.3 Mbit/s
SDTV 720x480, 30Hz, 16 bits/pix
-> 166 Mbit/s
HDTV 1920x1080, 30Hz, 16 bits/pix -> 1Gbit/s
Cable or satellite bandwidth : 38 Mbit/s
ADSL : 1 to 8 Mbit/s
DVD capacity : 5 to 8 GB
Slide 71
كاربرد
Video compression is now everywhere :
–
–
–
–
–
71
TV broadcasting over cable, satellite or terrestrial
networks,
CD-ROM, DVD, PC video storage,
Videophone and teleconferencing,
(VoD, IPTV),
Video over moInternet streaming biles.
Slide 72
Standardization (1/2)
Video codecs
Technologies
1950’s
1960’s
1970’s
1980’s
DPCM 52/80
MC Prediction 72/89
Transform Coding 65/80
Wavelet 85/-H.261
CD-ROM 1-1.5Mb/s
Digital TV, DVD
25 to 50 Mb/s
Videophone
Video streaming & post-prod
MPEG-1
4 to 80 Mb/s
Camcorder, VTR
Standards
2000’s
Entropy Coding 49/76
Videophone 56Kb/s – 2Mb/s
72
1990’s
30 Kb/s
30 Kb/s to 600Mb/s
H.262/MPEG62
DVCPRO
H.263
MPEG4 ASP
H.264/AVC
Convergence of all video applications, digital cinema
30 Kb/s to 600Mb/s
SMPTE/VC1
H.264/SVC
Slide 73
Bit rate evolution for SDTV Broadcast
Bit rate evolution
C. Ratio from 4:2:2
6
28
1st generation encoders
2nd generation encoders
(Stat-Mux + Rate control improvements)
5
3rd generation encoders
(advanced Pre-processing)
Mbit/s
4
3
1st generation
encoders
2
2nd generation
encoders
1
166
0
73
1995
1997
1999
2001
2003
2005
2007
2009
Slide 74
MPEG Compression
Compression through
–
–
74
Spatial
Temporal
Slide 75
Video Redundancies
75
Spatial
Neighboring pixels in a frame are statistically
related.
Temporal
Pixels in consecutive frames are statistically
related.
One can achieve higher compression ratios
by exploiting both spatial and temporal
redundancies
Slide 76
Spatial Redundancy
76
Take advantage of similarity among most
neighboring pixels
Slide 77
Spatial Redundancy Reduction
77
RGB to YUV
– less information required for YUV (humans less
sensitive to chrominance)
Macro Blocks
– Take groups of pixels (16x16)
Discrete Cosine Transformation (DCT)
– Based on Fourier analysis where represent signal as
sum of sine's and cosine’s
– Concentrates on higher-frequency values
– Represent pixels in blocks with fewer numbers
Quantization
– Reduce data required for co-efficients
Entropy coding
– Compress
Slide 78
Motion Compensation
Macro Block
Motion Vector
1 2 3
4 5 6
7 8 9
78
16 x 16
1 2 3
4 5 6
7 8 9
Slide 79
Video compression in MPEG-1&2
Spatial redundancy reduction (DCT example)
139
144
150
159
159
161
162
162
144
151
155
161
160
161
162
162
149
153
160
162
161
161
161
161
153
156
163
160
162
161
163
161
155
159
158
160
162
160
162
163
155
156
156
159
155
157
157
158
155
156
156
159
155
157
157
158
155
156
156
159
155
157
157
158
DC T
1260 -1 -12
-23 -17 -6
-11 -9 -2
-7 -2
0
-1 -1
1
2
0
2
-1
0
0
-3
2 -4
-5
-3
2
1
2
0
-1
-2
2
-3
0
1
0
-1
0
2
-2
0
-1
0
-1
1
2
1
-3
0
-1
0
1
1
1
-1
Q uan tsi a toi n
158
-1
-1
0
0
0
0
0
79
0
-1
0
0
0
0
0
0
-1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
z gi -zag scan
158 0 -1 -1 -1 -1 EO B
1
-1
0
0
1
-1
-1
0
Slide 80
Spatial Redundancy Reduction
“Intra-Frame
Encoded”
Quantization
• major reduction
• controls ‘quality’
80
Zig-Zag Scan,
Run-length
coding
Slide 81
Loss of Resolution
Original (63 kb)
Low (7kb)
Very Low (4 kb)
81
Slide 82
Temporal Redundancy
82
Take advantage of similarity between
successive frames
950
951
952
Slide 83
Temporal Redundancy Reduction
83
Slide 84
Temporal Redundancy Reduction
84
Slide 85
Temporal Redundancy Reduction
frames are independently encoded
P frames are based on previous I, P frames
– Can send motion vector plus changes
B frames are based on previous and following I and P
frames
– In case something is uncovered
Slide 86
Group of Pictures (GOP)
Starts with an I-frame
Ends with frame right before next I-frame
“Open” ends in B-frame, “Closed” in P-frame
MPEG Encoding a parameter, but ‘typical’:
–
–
86
IBBPBBPBBI
IBBPBBPBBPBBI
Slide 87
Question
87
When may temporal redundancy reduction be
ineffective?
Slide 88
Answer
When may temporal redundancy reduction be
ineffective?
–
–
88
Many scene changes
High motion
Slide 89
Non-Temporal Redundancy
89
Sometimes high motion
Slide 90
Typical Compress. Performance
90
Type Size Compression
--------------------I
18 KB
7:1
P
6 KB
20:1
B
2.5 KB
50:1
Avg 4.8 KB
27:1
--------------------Note, results are Variable Bit Rate, even if
frame rate is constant
Slide 91
Inter and Intra coding
To exploit spatial redundancies within a
frame (Intra coding):
8x8 DCT, similar to JPEG
To exploit temporal redundancies between
frame (Inter coding):
Motion Estimation
91
Slide 92
Frame Types
Two frame types:
92
Intra-frames (I-frames): I-frame provides an
accessing point, it uses basically JPEG.
Inter-frames (P-frames): P-frames use from previous
frame ("predicted"), so frames depend on eachother.
Slide 93
Some video formats
The 4:2:2 format
–
–
–
–
–
–
–
The 4:2:0 format
–
93
Y sampling @ 13.5 MHz
C sampling @ 6.75 MHz
8 bits per pixel
720 active points per line
576 lines active lines per image (2 fields) (625 lines)
and 480 active lines (525 lines)
Pixels are not square (e.g. for 480 lines, only 640 active points
are needed - VGA format)
Image size 720*576 or 720*480
Vertical chrominance resolution reduced by a factor 2
(average on two successive lines)
Slide 94
Some video formats
SIF format (Source Intermediate Format)
Half the vertical & horizontal resolution of 4:2:0
For 50Hz countries:
–
–
CIF format (Common Intermediate Format)
–
–
–
Intermediate format used in videoconferencing
(communication between US & Europe)
resolution: 360*288
Sampling frequency: 30 Hz
QCIF (Quarter CIF)
–
94
Luminance: 360*288
Chrominance: 180*120
Half the vertical & horizontal resolution of CIF.
AV Compression / Alain Bouffioux
December, 20, 2006
Slide 95
MPEG
95
MPEG was an early standard for lossy compression
of video and audio.
Development of the MPEG standard began in May
1988.
Slide 96
MPEG Coding Performance
Decoding is easy
–
–
–
Encoding is expensive
–
–
–
–
96
MPEG1 decoding in software on most platforms
Hardware decoders are widely available ($150/board)
Windows graphics accelerators with MPEG decoding
now entering market (e.g., Matrox, Diamond, …(
Sequential software encoders are 20:1 real-time
Real-time encoders use parallel processing
Real-time hardware encoders are expensive (e.g.,
$12K-$50K for MPEG1 and $100K-$500K for MPEG2)
Hardware-assisted off-line MPEG1 encoders (3:1) used
for multimedia authoring at reasonable cost ($5k)
Slide 97
MPEG
MPEG1: low bitrate
MPEG2: VCD, DVD
MP3: MPEG2 profile 3, for music
MPEG4: Network streaming
MPEG7: Searching and indexing
Slide 98
MPEG Standards
MPEG-1 ~ 1-1.5Mbps (early 90s) - vhs quality
(1992)
–
–
–
–
98
Frame encoding
For compression of 320x240 full-motion video at rates
around 1.15Mb/s
Applications: video storage (VCD)
CIF images, 4:2:0 sampling, 1.5 Mbs
Slide 99
MPEG Standards
MPEG-2 ~ 2-80Mbps (mid 90s) - broadcast quality
(1994)
–
–
–
–
–
–
–
99
Frame and field encoding
For higher resolutions
Support interlaced video formats and a number of features
for HDTV
Address scalable video coding
Also used in DVD
CCIR 601 images, 4:2:2 sampling, 15 Mbs
Interlaced and progressive scanning
Slide 100
MPEG Today
100
MPEG4 ~ 9-40kbps (later 90s)
–
Around Objects, not Frames
–
For very low bit rate video and audio coding
–
Applications: interactive multimedia and video telephony
–
Lower bandwidth
MPEG-7
–
Provide a fast and efficient searching, filtering
–
New standard
–
Internet orientated
–
VOP (Video Object Plane)
–
Profiles and levels
Slide 101
H.26x
H.261: first generation
H.262 (MPEG2)
H.263: video conference
H.263++: enhancement
H.26L: latest
Slide 102
VOD
Watch
what you want
when you want
Slide 103
Standardization (2/2)
DVB Transport
DVB-S
DVB-S2
DVB-C
Satellite TV
Cable
DVB-T
Terrestrial TV
DVB-H
Mobile TV
DVB-IPI in progress
103
1995
1997
1999
2001
2003
2005
2007
IPTV
2009
Slide 104
Video on Demand
One video server
Many video data
Many clients
Client want to watch at any time
Clients can send request to the server and request to
watch a particular video. The server have to
response by streaming the requested video to the
client. We want to be able to support large
number of clients, and clients should be able to
watch at anytime.
104
Slide 105
Streaming
–
Streaming media:
the ‘real-time’ playing of a video-, audioand/or datastream on a machine from the
moment the first bytes come in.
–
VoD such as YouTube, MSN Video, Google
Video, Yahoo Video, CNN…
2006 April to December: MSN Video service
Client-server mode
Covering over 520 million streaming requests for
more than 59,000 videos.
–
105
Slide 106
Streaming: live vs on-demand
unicast
multicast
archive
realtime
VoD
Eventdriven
scheduled
Eventdriven
Slide 107
Streaming: live vs on-demand
107
Live audio and/or video streaming is a completely
different sport from on-demand streaming.
VoD (Video on demand) is unicast streaming from
an archive.
Scheduled (a tv like netcast) is usually multicast
streaming from an archive
Live (realtime) streaming can be done both in
unicast and multicast.
Slide 108
Motivation
108
VoD such as YouTube, MSN Video, Google
Video, Yahoo Video, CNN…
As the trend of increasing demands on such
services and higher-quality videos, it becomes
a costly service to provide.
Slide 109
Video-on-Demand Distribution
Model
M u lt ic a s t / B r o a d c a s t C a p a b le
N etw ork
STB
STB
STB
VID EO
SERVER
109
A client can tune in to receive any ongoing
media delivery using its Set Top Box
True broadcast: Satellite and cable TV
networks
109
Slide 110
Rewind TV
VoD servers support rewind function
VoD servers
customer premise
TV
IP backbone
Unicast
Set-top box
110
Slide 111
IPTV/VOD : Full Operator model
Content Provider
Service Provider
CPE
Network Provider
DSL (Copper)
IP Streamer
Broadcast
feed
DSLAM
RT Encoder
Encryption
Server
Optical Fiber
VOD Server
Encoder
Media
database
ADSL
Modem/Router
Content
Management
Server
Core &
Aggregation
Networks
STB
Optical
Line
Terminatio
n
Optical Network
Termination
HFC (Coax)
Licence
Server
EPG
metadata
111
Application
Servers
Remote
Management
Server
111
Subscriber
Management
and Billing
Server
Cable
Modem
Termination
System
Cable
Modem/Router
Slide 112
What is IPTV?
The fundamentals
–
IPTV = Internet Protocol Television
112
Digital TV service delivered over a broadband network
using the Internet Protoc0l
IPTV usually refers to TV services over a
Network Operator’s quality controlled network.
Internet TV = IPTV over the public Internet
Slide 113
IPTV Network
One of the first
One of the largest
150 TV channels
250,000 subscribers
113
Slide 114
Basic IPTV Structure
114
Slide 115
Set Top Box – Home gateway (BST)
115
Slide 116
IPTV Set-Top Box
Broadcast TV
116
DVR
Movies On Demand
On Demand
116
Slide 117
Other Services
VOD – Video on Demand
–
DVR
–
117
Record Live TV from your set top box
HD
–
Watch what you want when you want
High Definition TV. The evolution of Television
On line Gaming
Slide 118
Logout | Help
Search
My Content
Annotate
Go
Title: The Hidden Child
Description: Of the 1,600,000 Jewish
Selected Video
children who lived in Europe before
World War II, only 100,000 survived
the Holocaust.
Title: Ralph Golzio interview on
Paterson Silk Strike of 1913
Description: Ralph Golzio was a
teenager at the time of the Paterson
Silk Strike of 1913.
>>
<<
Title: Sondra Gash interview
Description: Evelyn Hershey of the
American Labor Museum interviews
Search For and Select a Video for
Annotation
118
--
Slide 119
Logout | Help
Selection of
Search
My Content
video found
through search
Annotate
Go
Title: The Hidden Child
Description: Of the 1,600,000 Jewish
children who lived in Europe before
World War II, only 100,000 survived
the Holocaust.
Title: Ralph Golzio interview on
Paterson Silk Strike of 1913
Description: Ralph Golzio was a
-
teenager at the time of the Paterson
Silk Strike of 1913.
>>
<<
Title: Sondra Gash interview
Description: Evelyn Hershey of the
American Labor Museum interviews
Drag and Drop Thumbnails Here To Create
an Annotation
119
--
Slide 120
Logout | Help
Once an annotation is
saved it will appear in the
videos timeline.
Amsterdam, The
Netherlands
Search
Start Time:
My Content
5:01
End Time:
8:34
Title:
Amsterdam, The Netherlands
Note:
City street in Amsterdam.
-
Private
My Institution Only
Indexed/Searchable:
>>
Annotate
NJVid Wide
Save
<<
Start and stop
markers can be
placed.
Metadata
placed here.
Annotate and Capture Start/End Times
120
Slide 121
این دو عالم علم دارد در نهاد منتخب
وان جهانی رمز دارد در حروف مختصر
سنایی غزنوی
Slide 2
Image/Video Compression & VOD
برنا فیروزي
نوید زرین درفش
محمود قدیمي
مهندسي فناوري اطالعات
بهار 88
Slide 3
Image Compression
3
Slide 4
Compression
فشرده سازی ( ،)Compressionپردازشی است که با حذف
اطالعات اضافی ،داده ها را به عالیم دیجیتالی کاهش می دهد.
این پردازش بسته به پهنای باند مورد نیاز برای انتقال داده ها و
میزان فضای ذخیره سازی ،داده ها را کاهش می دهد.
کاهش پهنای باند مورد نیاز امکان انتقال داده های بیشتری را در
یک زمان واحد می دهد.
كاهش افزونگی ()redundancy
4
Slide 5
ديجيتال ،آنالوگ
پردازش تصاویر در فضاي دیجیتال انجام مي شود
اگر ما یك منبع تصویري آنالوگ داشته باشیم باید قبل از انجام هر
عمل فشرده سازي ابتدا به آن را به دیجیتال تبدیل كنیم
5
Slide 6
Purpose of Image Compression
6
Saving storage space
Saving transfer time
Easy processing
Easy to transmitted over network
reduce cost
Slide 7
هدف
در حالت ایده آل ما خواستار
حداكثر كیفیت تصویر
حداقل فضاي ذخیره سازي و پردازش منابع
ما نمي توانیم هر دو هدف را در بهترین شرایط داشته باشیم
بهترین فشرده سازي چگونه است؟
7
Slide 8
Why we want to compress?
To transmit an RGB 512X512, 24 bit image
via modem 28.2 kbaud(kilobits/second)
( 512 512 pixels )( 24 bits / pixel )
( 28 . 8 1024 bits / second )
8
213 second
( 4 min)
Slide 9
Image Compression Coding
lossless
compression
Image
Compression
lossy
compression
9
Huffman
Coding
Run-length
encoding
Predictive
Coding
Transform
Coding
Slide 10
دو كالس اصلي فشرده سازي تصاوير
)Lossyپراتالف(
از روي دیتاهاي تصویر ذخیره
شده مي توان به تصویري نزديك
به تصویر اصلي رسید
تصویر ذخیره شده خود تصویر
اصلی نیست ،بلکه شبیه آن است و
اطالعاتی را از دست داده است
Compression rate:
high compression
( Losslessبدون اتالف(
از روي دیتاهاي تصویر ذخیره
شده مي توان دقیقا به تصوير
اصلي رسید
تصویر ذخیره شده بدون از دست
دادن كمترین داده ای ،خود تصویر
است.
Compression rate:
)2:1 (at most 3:1
10
Slide 11
General compression system model
O utput ima ge
Input im a ge
Source decoder
Source encoder
D ecoder
E ncoder
C hannel decoder
C hannel encoder
C hannel
11
Slide 12
Compression System Model
Compression
Input
Preprocessing
Encoding
Compressed
File
• Decompression
Compressed
File
12
Decoding
Postprocessing
Output
Slide 13
Compression Ratio
Compressio n Ratio
Uncompress ed File Size
Compressed
File Size
CR
Ex Image 256X256 pixels, 256 level grayscale
can be compressed file size 6554 byte.
Original Image Size = 256X256(pixels) X
1(byte/pixel)
= 65536 bytes
compressio n Ratio
13
65536
6554
10
Slide 14
Bits per Pixel
(2) Bits per Pixel Number of Bits
Number of Pixels
Ex Image 256X256 pixels, 256 level grayscale
can be compressed file size 6554 byte.
Original Image Size = 256X256(pixels) X
1(byte/pixel)
= 65536 bytes
Compressed file
= 6554(bytes)X8(bits/pixel)
= 52432 bits
14
Bits per Pixel
52432
65536
0 .8
Slide 15
Key of compression
Reducing Data but Retaining Information
Various amounts of data can be used to represent the same
amount of information. It’s “Data redundancy”
Relative data redundancy
RD 1
15
1
CR
Slide 16
Entropy
Average information in an image.
L 1
Entropy p k log 2 ( p k )
k 0
• Average number of bits per pixel
L 1
La
l
k
pk
k 0
pk
16
nk
n
, where k 0 ,1, , L 1
Slide 17
Compression Standard
Standard:
17
Image
Video
ISO
JPEG
JPEG2000
MPEG1,MPEG2,
MPEG4, MPEG7
ITU
N/A
H.261, H.263, H.263+,
H.26L
Slide 18
Image Compression Coding
lossless
compression
Image
Compression
lossy
compression
18
Huffman
Coding
Run-length
encoding
Predictive
Coding
Transform
Coding
Slide 19
Loseless Compression
19
No data are lost
Can recreated exactly original image
Often the achievable compression is mush less
Slide 20
Huffman Coding ( VLC,Entropy)
Using Histogram probability
5 Steps
1.
2.
3.
4.
5.
20
Find the histogram probabilities
Order the input probabilities(smalllarge)
Addition the 2 smallest
Repeat step 2&3, until 2 probability are left
Backward along the tree assign 0 and 1
Slide 21
Huffman Coding(cont)
40
30
20
10
Step 1 Histogram Probability
p0 = 20/100 = 0.2
p1 = 30/100 = 0.3
p2 = 10/100 = 0.1
p3 = 40/100 = 0.4
0
1
2
3
Step 2 Order
p3 0.4
p1 0.3
p0 0.2
p2 0.1
Slide 22
Huffman Coding(cont)
Step 3,4 Add 2 smallest
p 3 0 .4 0 .4 0 .4
p1 0 . 3 0 . 3
0 .6
p 0 0 .2
0 .3
0 .6
0 .4
p 2 0 .1
Step 5 assign 0 and 1
22
Natural Code
00
Probability
0.2
Huffman Code
010
01
0.3
00
10
0.1
011
11
0.4
1
Slide 23
Huffman Coding(cont)
The original Image :average 2 bits/pixel
The Huffman Code:average
3
La
l
i
p i 3 ( 0 . 2 ) 2 ( 0 . 3 ) 3 ( 0 . 1) 1( 0 . 4 ) 1 . 9
i0
3
Entropy p i log 2 ( p i )
i0
( 0 . 2 ) log 2 ( 0 . 2 ) ( 0 . 3 ) log 2 ( 0 . 3 )
(
0
.
1
)
log
(
0
.
1
)
(
0
.
4
)
log
(
0
.
4
)
2
2
23
1 . 846 bits / pixel
Slide 24
Run Length Encoding
24
Run-length encoding (RLE) is a very simple form
of data compression in which runs of data (that is,
sequences in which the same data value occurs in
many consecutive data elements) are stored as a
single data value and count, rather than as the
original run. This is most useful on data that
contains many such runs: for example, relatively
simple graphic images such as
icons, line drawings, and animations.
Slide 25
Run-Length Coding
25
Counting the number of adjacent pixels with
the same gray-level value
Used primarily for binary image
Mostly use horizontal RLC
Slide 26
Run-Length Coding(cont)
Binary Image 8X8
26
horizontal
0
0
0
0
0
0
0
0
1st Row
8
1
1
1
1
0
0
0
0
2nd Row
4,4
0
1
1
0
0
0
0
0
3rd Row
1,2,5
0
1
1
1
1
1
0
0
4th Row
1,5,2
0
1
1
1
0
0
1
0
5th Row
1,3,2,1,1
0
0
1
0
0
1
1
0
6th Row
2,1,2,2,1
1
1
1
1
0
1
0
0
7th Row
4,1,1,2
0
0
0
0
0
0
0
0
8th Row
8
Slide 27
Example of RLE
Let us take a hypothetical single scan line, with B
representing a black pixel and W representing white:
WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWW
WWWWWWWWWWWWWWWWWWWWBWWWWWWWWWW
WWWW
If we apply the run-length encoding (RLE) data compression
algorithm to the above hypothetical scan line, we get the
following:12W1B12W3B24W1B14W
27
Slide 28
Lossy Compression
28
Allow a loss in the actual image data
Can not recreated exactly original image
Commonly the achievable compression is
mush more
Such as JPEG
Slide 29
Predictive Coding
Common Predictive
Coding
29
DM (Delta Modulation)
DPCM (Differential Pulse
Code Modulation)
Slide 30
The principle of Predictive Coding
30
The system consists of an encoder and a decoder,
each containing an identical predictor. As each
successive pixel of the input image, is introduced to
the encoder, the predictor generates the anticipated
value of that pixel based on some number of past
inputs. The output of the predictor is then rounded
to the nearest integer.
Slide 31
Predictive coding model I
Input image
n
fn
Symbol
encoder
fˆn
Predictor
Compressed image
Output image
fn'
Predictor
31
‘
fˆ n '
Symbol
decoder
Slide 32
Predictive coding model II
The predictor :
fˆ n F ( f n 1 , f n 2 , , f n k )
n
f n fˆ n
The symbol encoder : generate the next element of the
compressed data stream
Decoder : perform the inverse of encoding
The linearity predictor :
32
fˆ n F ( f n 1 , f n 2 , , f n k )
n 1
a
kl
k
fk ,
a
k
1
Slide 33
Delta Modulation I
Delta modulation (DM or Δ-modulation) is an analog-todigital and digital-to-analog signal conversion technique
used for transmission of voice information where quality
is not of primary importance.
DM is the simplest form of differential pulse-code
modulation (DPCM) where the difference between
successive samples is encoded into n-bit data streams. In
delta modulation, the transmitted data is reduced to a 1bit data stream.
33
Slide 34
Differential Pulse Code Modulation
34
Differential Pulse Code Modulation (DPCM) compares
two successive analog amplitude values, quantizes and
encodes the difference, and transmits the differential
value.
Slide 35
Transform Coding I (DCT)
35
Transform coding is a type of data compression for
"natural" data like audio signals or photographic
images.
The transformation is typically lossy, resulting in a
lower quality copy of the original input.
Slide 36
Transform Coding II
36
Slide 37
Transform Coding III
A transform coding system
Input image
Construct
subimages
Compressed
image
37
Quantizer
Forward
transform
Symbol
encoder
Compressed
image
Symbol
decoder
Inverse
transform
Merge
subimages
Slide 38
BMP (Bitmap) - lossless
38
Use 3 bytes per pixel, one
each for R, G, and B
Can represent up to 224 =
16.7 million colors
No entropy coding
File size in bytes =
3*length*height, which can be
very large
Can use fewer than 8 bits per
color, but you need to store
the color palette
Performs well with ZIP, RAR,
etc.
Slide 39
GIF (Graphics Interchange Format)
Can use up to 256 colors from
24-bit RGB color space
–
39
If source image contains
more than 256 colors, need
to reprocess image to fewer
colors
Suitable for simpler images
such as logos and textual
graphics, not so much for
photographs
Slide 40
JPEG (Joint Photographic Experts
Group) - lossly
40
Most dominant image
format today
Typical file size is about
10% of that of BMP (can
vary depending on
quality settings)
Unlike GIF, JPEG is
suitable for photographs,
not so much for logos
and textual graphics
Slide 41
Joint Picture Expert Group
41
The name "JPEG" stands for Joint Photographic Experts Group,
the name of the committee that created the standard. The group
was organized in 1986, issuing a standard in 1992, which was
approved in 1994 as ISO 10918-1. JPEG is a commonly used
method of compression for photographic images. The degree of
compression can be adjusted, allowing a selectable tradeoff
between storage size and image quality.
Slide 42
JPEG2000
42
JPEG 2000 is a wavelet-based image compression standard. It
was created by the Joint Photographic Experts Group committee
in the year 2000 with the intention of superseding their original
discrete cosine transform-based JPEG standard (created about
1991).
Slide 43
JPEG Encoding Steps
43
Preprocess image
Apply 2D forward DCT
Quantize DCT coefficients
Apply RLE, then entropy encoding
Slide 44
JPEG Block Diagram
8x8 blocks
Source
Image
B
G
R
DCT-based encoding
FDCT
44
Quantizer
Entropy
Encoder
Table
Table
Compressed
image data
Slide 45
Image Compression- JPEG
Using the DCT, the entries in Y will be organized
based on the human visual system.
The most important values to
our eyes will be placed in the
upper left corner of the matrix.
The least important values
will be mostly in the lower
right corner of the matrix.
Most
Important
SemiImportant
Least
Important
45
Slide 46
JPEG
8 x 8 Pixels
46
Image
Slide 47
Image Compression
Gray-Scale Example:
Value Range 0 (black) --- 255 (white)
63 33 36 28 63 81
27 18 17 11 22 48
72 52 28 15 17 16
132 100 56 19 10 9
187 186 166 88 13 34
184 203 199 177 82 44
211 214 208 198 134 52
211 210 203 191 133 79
47
86 98
104 108
47 77
21 55
43 51
97 73
78 83
74 86
Slide 48
Image Compression
2D-DCT of matrix
-304 210
-327 -260
93 -84
89 33
-9 42
-5 15
10
3
12 30
48
104
67
-66
-19
18
-10
-12
0
-69
70
16
-20
27
17
-1
-3
10
-10
24
-26
-7
32
2
-3
20
-15
-2
21
-17
-15
3
-6
-12 7
21 8
-5 9
-3 0
29 -7
-4 7
-2 -3
12 -1
Slide 49
Image Compression
-304 210
-327 -260
93 -84
89 33
-9 42
-5 15
10
0
0
0
104
67
-66
-19
18
0
0
0
-69
70
16
-20
0
0
0
0
10 20 -12 0
-10 -15 0 0
24 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
As you can see, we save a little over half the original
memory.
49
Slide 50
Reconstructing the Image
New Matrix and Compressed Image
55 41 27 39 56 69 92 106
35 22 7 16 35 59 88 101
65 49 21 5 6 28 62 73
130 114 75 28 -7 -1 33 46
180 175 148 95 33 16 45 59
200 206 203 165 92 55 71 82
205 207 214 193 121 70 75 83
214 205 209 196 129 75 78 85
50
Slide 51
Can You Tell the Difference?
Original
51
Compressed
Slide 52
Image Compression
Original
52
Compressed
Slide 53
Example - One everyday photo with
file size of 2.76 MB
53
Slide 54
Example - One everyday photo with
file size of 600 KB
54
Slide 55
Example - One everyday photo with
file size of 350 KB
55
Slide 56
Example - One everyday photo with
file size of 240 KB
56
Slide 57
Example - One everyday photo with
file size of 144 KB
57
Slide 58
Example - One everyday photo with
file size of 88 KB
58
Slide 59
Analysis
Near perfect image at 2.76M, so-so image at 88K
Sharpness decreases as file size decreases
Which file size is the best?
–
–
59
No correct answer to this question
Answer depends upon how strict we are about image quality, what
purpose image is to be used for, and the resources available
Slide 60
Conclusion
60
Image compression is important
Image compression has come a long way
Image compression is nearly mature, but there
is always room for improvement
Slide 61
Video Compression
Slide 62
Video Data Size
size of uncompressed video in gigabytes
1 sec
1 min
1 hour
1000 hours
1920x1080
0.19
11.20
671.85
671,846.40
1280x720
0.08
4.98
298.60
298,598.40
640x480
0.03
1.66
99.53
99,532.80
320x240
0.01
0.41
24.88
24,883.20
160x120
0.00
0.10
6.22
6,220.80
image size of video
1280x720 (1.77)
62
640x480 (1.33)
320x240
160x120
Slide 63
Video Bit Rate Calculation
width * height * depth * fps
= bits/sec
compression factor
width ~ pixels(160, 320, 640, 720, 1280, 1920, …(
height ~ pixels
(120, 240, 480, 485, 720, 1080, …(
depth ~ bits (1, 4, 8, 15, 16, 24, …(
fps ~ frames per second (5, 15, 20, 24, 30, …(
compression factor (1, 6, 24, …(
63
Slide 64
Effects of Compression
storage for 1 hour of compressed video in megabytes
1:1
3:1
6:1
25:1
100:1
1920x1080
671,846
223,949
111,974
26,874
6,718
1280x720
298,598
99,533
49,766
11,944
2,986
640x480
99,533
33,178
16,589
3,981
995
3 bytes/pixel, 30 frames/sec
64
320x240
24,883
8,294
4,147
995
249
160x120
6,221
2,074
1,037
249
62
Slide 65
Channel Bandwidths
65
Slide 66
Channel Bandwidth
66
Slide 67
Application Requirements
67
Slide 68
Source Video Formats
68
Slide 69
The Need for Video Compression
High-Definition Television (HDTV)
1920x1080
– 30 frames per second (full motion)
– 8 bits for each three primary colors (RGB)
Total 1.5 Gb/sec!
–
Cable TV: each cable channel is 6 MHz
Max data rate of 19.2 Mb/sec
– Reduced to 18 Mb/sec w/audio + control …
Compression rate must be ~ 80:1!
–
69
Slide 70
The Need for Video Compression
Some figures
–
Uncompressed video -> big amount of data
–
Communication and storage capacities limits
70
Color picture 800x320 pix, 24 bits/pix -> 6.3 Mbit/s
SDTV 720x480, 30Hz, 16 bits/pix
-> 166 Mbit/s
HDTV 1920x1080, 30Hz, 16 bits/pix -> 1Gbit/s
Cable or satellite bandwidth : 38 Mbit/s
ADSL : 1 to 8 Mbit/s
DVD capacity : 5 to 8 GB
Slide 71
كاربرد
Video compression is now everywhere :
–
–
–
–
–
71
TV broadcasting over cable, satellite or terrestrial
networks,
CD-ROM, DVD, PC video storage,
Videophone and teleconferencing,
(VoD, IPTV),
Video over moInternet streaming biles.
Slide 72
Standardization (1/2)
Video codecs
Technologies
1950’s
1960’s
1970’s
1980’s
DPCM 52/80
MC Prediction 72/89
Transform Coding 65/80
Wavelet 85/-H.261
CD-ROM 1-1.5Mb/s
Digital TV, DVD
25 to 50 Mb/s
Videophone
Video streaming & post-prod
MPEG-1
4 to 80 Mb/s
Camcorder, VTR
Standards
2000’s
Entropy Coding 49/76
Videophone 56Kb/s – 2Mb/s
72
1990’s
30 Kb/s
30 Kb/s to 600Mb/s
H.262/MPEG62
DVCPRO
H.263
MPEG4 ASP
H.264/AVC
Convergence of all video applications, digital cinema
30 Kb/s to 600Mb/s
SMPTE/VC1
H.264/SVC
Slide 73
Bit rate evolution for SDTV Broadcast
Bit rate evolution
C. Ratio from 4:2:2
6
28
1st generation encoders
2nd generation encoders
(Stat-Mux + Rate control improvements)
5
3rd generation encoders
(advanced Pre-processing)
Mbit/s
4
3
1st generation
encoders
2
2nd generation
encoders
1
166
0
73
1995
1997
1999
2001
2003
2005
2007
2009
Slide 74
MPEG Compression
Compression through
–
–
74
Spatial
Temporal
Slide 75
Video Redundancies
75
Spatial
Neighboring pixels in a frame are statistically
related.
Temporal
Pixels in consecutive frames are statistically
related.
One can achieve higher compression ratios
by exploiting both spatial and temporal
redundancies
Slide 76
Spatial Redundancy
76
Take advantage of similarity among most
neighboring pixels
Slide 77
Spatial Redundancy Reduction
77
RGB to YUV
– less information required for YUV (humans less
sensitive to chrominance)
Macro Blocks
– Take groups of pixels (16x16)
Discrete Cosine Transformation (DCT)
– Based on Fourier analysis where represent signal as
sum of sine's and cosine’s
– Concentrates on higher-frequency values
– Represent pixels in blocks with fewer numbers
Quantization
– Reduce data required for co-efficients
Entropy coding
– Compress
Slide 78
Motion Compensation
Macro Block
Motion Vector
1 2 3
4 5 6
7 8 9
78
16 x 16
1 2 3
4 5 6
7 8 9
Slide 79
Video compression in MPEG-1&2
Spatial redundancy reduction (DCT example)
139
144
150
159
159
161
162
162
144
151
155
161
160
161
162
162
149
153
160
162
161
161
161
161
153
156
163
160
162
161
163
161
155
159
158
160
162
160
162
163
155
156
156
159
155
157
157
158
155
156
156
159
155
157
157
158
155
156
156
159
155
157
157
158
DC T
1260 -1 -12
-23 -17 -6
-11 -9 -2
-7 -2
0
-1 -1
1
2
0
2
-1
0
0
-3
2 -4
-5
-3
2
1
2
0
-1
-2
2
-3
0
1
0
-1
0
2
-2
0
-1
0
-1
1
2
1
-3
0
-1
0
1
1
1
-1
Q uan tsi a toi n
158
-1
-1
0
0
0
0
0
79
0
-1
0
0
0
0
0
0
-1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
z gi -zag scan
158 0 -1 -1 -1 -1 EO B
1
-1
0
0
1
-1
-1
0
Slide 80
Spatial Redundancy Reduction
“Intra-Frame
Encoded”
Quantization
• major reduction
• controls ‘quality’
80
Zig-Zag Scan,
Run-length
coding
Slide 81
Loss of Resolution
Original (63 kb)
Low (7kb)
Very Low (4 kb)
81
Slide 82
Temporal Redundancy
82
Take advantage of similarity between
successive frames
950
951
952
Slide 83
Temporal Redundancy Reduction
83
Slide 84
Temporal Redundancy Reduction
84
Slide 85
Temporal Redundancy Reduction
frames are independently encoded
P frames are based on previous I, P frames
– Can send motion vector plus changes
B frames are based on previous and following I and P
frames
– In case something is uncovered
Slide 86
Group of Pictures (GOP)
Starts with an I-frame
Ends with frame right before next I-frame
“Open” ends in B-frame, “Closed” in P-frame
MPEG Encoding a parameter, but ‘typical’:
–
–
86
IBBPBBPBBI
IBBPBBPBBPBBI
Slide 87
Question
87
When may temporal redundancy reduction be
ineffective?
Slide 88
Answer
When may temporal redundancy reduction be
ineffective?
–
–
88
Many scene changes
High motion
Slide 89
Non-Temporal Redundancy
89
Sometimes high motion
Slide 90
Typical Compress. Performance
90
Type Size Compression
--------------------I
18 KB
7:1
P
6 KB
20:1
B
2.5 KB
50:1
Avg 4.8 KB
27:1
--------------------Note, results are Variable Bit Rate, even if
frame rate is constant
Slide 91
Inter and Intra coding
To exploit spatial redundancies within a
frame (Intra coding):
8x8 DCT, similar to JPEG
To exploit temporal redundancies between
frame (Inter coding):
Motion Estimation
91
Slide 92
Frame Types
Two frame types:
92
Intra-frames (I-frames): I-frame provides an
accessing point, it uses basically JPEG.
Inter-frames (P-frames): P-frames use from previous
frame ("predicted"), so frames depend on eachother.
Slide 93
Some video formats
The 4:2:2 format
–
–
–
–
–
–
–
The 4:2:0 format
–
93
Y sampling @ 13.5 MHz
C sampling @ 6.75 MHz
8 bits per pixel
720 active points per line
576 lines active lines per image (2 fields) (625 lines)
and 480 active lines (525 lines)
Pixels are not square (e.g. for 480 lines, only 640 active points
are needed - VGA format)
Image size 720*576 or 720*480
Vertical chrominance resolution reduced by a factor 2
(average on two successive lines)
Slide 94
Some video formats
SIF format (Source Intermediate Format)
Half the vertical & horizontal resolution of 4:2:0
For 50Hz countries:
–
–
CIF format (Common Intermediate Format)
–
–
–
Intermediate format used in videoconferencing
(communication between US & Europe)
resolution: 360*288
Sampling frequency: 30 Hz
QCIF (Quarter CIF)
–
94
Luminance: 360*288
Chrominance: 180*120
Half the vertical & horizontal resolution of CIF.
AV Compression / Alain Bouffioux
December, 20, 2006
Slide 95
MPEG
95
MPEG was an early standard for lossy compression
of video and audio.
Development of the MPEG standard began in May
1988.
Slide 96
MPEG Coding Performance
Decoding is easy
–
–
–
Encoding is expensive
–
–
–
–
96
MPEG1 decoding in software on most platforms
Hardware decoders are widely available ($150/board)
Windows graphics accelerators with MPEG decoding
now entering market (e.g., Matrox, Diamond, …(
Sequential software encoders are 20:1 real-time
Real-time encoders use parallel processing
Real-time hardware encoders are expensive (e.g.,
$12K-$50K for MPEG1 and $100K-$500K for MPEG2)
Hardware-assisted off-line MPEG1 encoders (3:1) used
for multimedia authoring at reasonable cost ($5k)
Slide 97
MPEG
MPEG1: low bitrate
MPEG2: VCD, DVD
MP3: MPEG2 profile 3, for music
MPEG4: Network streaming
MPEG7: Searching and indexing
Slide 98
MPEG Standards
MPEG-1 ~ 1-1.5Mbps (early 90s) - vhs quality
(1992)
–
–
–
–
98
Frame encoding
For compression of 320x240 full-motion video at rates
around 1.15Mb/s
Applications: video storage (VCD)
CIF images, 4:2:0 sampling, 1.5 Mbs
Slide 99
MPEG Standards
MPEG-2 ~ 2-80Mbps (mid 90s) - broadcast quality
(1994)
–
–
–
–
–
–
–
99
Frame and field encoding
For higher resolutions
Support interlaced video formats and a number of features
for HDTV
Address scalable video coding
Also used in DVD
CCIR 601 images, 4:2:2 sampling, 15 Mbs
Interlaced and progressive scanning
Slide 100
MPEG Today
100
MPEG4 ~ 9-40kbps (later 90s)
–
Around Objects, not Frames
–
For very low bit rate video and audio coding
–
Applications: interactive multimedia and video telephony
–
Lower bandwidth
MPEG-7
–
Provide a fast and efficient searching, filtering
–
New standard
–
Internet orientated
–
VOP (Video Object Plane)
–
Profiles and levels
Slide 101
H.26x
H.261: first generation
H.262 (MPEG2)
H.263: video conference
H.263++: enhancement
H.26L: latest
Slide 102
VOD
Watch
what you want
when you want
Slide 103
Standardization (2/2)
DVB Transport
DVB-S
DVB-S2
DVB-C
Satellite TV
Cable
DVB-T
Terrestrial TV
DVB-H
Mobile TV
DVB-IPI in progress
103
1995
1997
1999
2001
2003
2005
2007
IPTV
2009
Slide 104
Video on Demand
One video server
Many video data
Many clients
Client want to watch at any time
Clients can send request to the server and request to
watch a particular video. The server have to
response by streaming the requested video to the
client. We want to be able to support large
number of clients, and clients should be able to
watch at anytime.
104
Slide 105
Streaming
–
Streaming media:
the ‘real-time’ playing of a video-, audioand/or datastream on a machine from the
moment the first bytes come in.
–
VoD such as YouTube, MSN Video, Google
Video, Yahoo Video, CNN…
2006 April to December: MSN Video service
Client-server mode
Covering over 520 million streaming requests for
more than 59,000 videos.
–
105
Slide 106
Streaming: live vs on-demand
unicast
multicast
archive
realtime
VoD
Eventdriven
scheduled
Eventdriven
Slide 107
Streaming: live vs on-demand
107
Live audio and/or video streaming is a completely
different sport from on-demand streaming.
VoD (Video on demand) is unicast streaming from
an archive.
Scheduled (a tv like netcast) is usually multicast
streaming from an archive
Live (realtime) streaming can be done both in
unicast and multicast.
Slide 108
Motivation
108
VoD such as YouTube, MSN Video, Google
Video, Yahoo Video, CNN…
As the trend of increasing demands on such
services and higher-quality videos, it becomes
a costly service to provide.
Slide 109
Video-on-Demand Distribution
Model
M u lt ic a s t / B r o a d c a s t C a p a b le
N etw ork
STB
STB
STB
VID EO
SERVER
109
A client can tune in to receive any ongoing
media delivery using its Set Top Box
True broadcast: Satellite and cable TV
networks
109
Slide 110
Rewind TV
VoD servers support rewind function
VoD servers
customer premise
TV
IP backbone
Unicast
Set-top box
110
Slide 111
IPTV/VOD : Full Operator model
Content Provider
Service Provider
CPE
Network Provider
DSL (Copper)
IP Streamer
Broadcast
feed
DSLAM
RT Encoder
Encryption
Server
Optical Fiber
VOD Server
Encoder
Media
database
ADSL
Modem/Router
Content
Management
Server
Core &
Aggregation
Networks
STB
Optical
Line
Terminatio
n
Optical Network
Termination
HFC (Coax)
Licence
Server
EPG
metadata
111
Application
Servers
Remote
Management
Server
111
Subscriber
Management
and Billing
Server
Cable
Modem
Termination
System
Cable
Modem/Router
Slide 112
What is IPTV?
The fundamentals
–
IPTV = Internet Protocol Television
112
Digital TV service delivered over a broadband network
using the Internet Protoc0l
IPTV usually refers to TV services over a
Network Operator’s quality controlled network.
Internet TV = IPTV over the public Internet
Slide 113
IPTV Network
One of the first
One of the largest
150 TV channels
250,000 subscribers
113
Slide 114
Basic IPTV Structure
114
Slide 115
Set Top Box – Home gateway (BST)
115
Slide 116
IPTV Set-Top Box
Broadcast TV
116
DVR
Movies On Demand
On Demand
116
Slide 117
Other Services
VOD – Video on Demand
–
DVR
–
117
Record Live TV from your set top box
HD
–
Watch what you want when you want
High Definition TV. The evolution of Television
On line Gaming
Slide 118
Logout | Help
Search
My Content
Annotate
Go
Title: The Hidden Child
Description: Of the 1,600,000 Jewish
Selected Video
children who lived in Europe before
World War II, only 100,000 survived
the Holocaust.
Title: Ralph Golzio interview on
Paterson Silk Strike of 1913
Description: Ralph Golzio was a
teenager at the time of the Paterson
Silk Strike of 1913.
>>
<<
Title: Sondra Gash interview
Description: Evelyn Hershey of the
American Labor Museum interviews
Search For and Select a Video for
Annotation
118
--
Slide 119
Logout | Help
Selection of
Search
My Content
video found
through search
Annotate
Go
Title: The Hidden Child
Description: Of the 1,600,000 Jewish
children who lived in Europe before
World War II, only 100,000 survived
the Holocaust.
Title: Ralph Golzio interview on
Paterson Silk Strike of 1913
Description: Ralph Golzio was a
-
teenager at the time of the Paterson
Silk Strike of 1913.
>>
<<
Title: Sondra Gash interview
Description: Evelyn Hershey of the
American Labor Museum interviews
Drag and Drop Thumbnails Here To Create
an Annotation
119
--
Slide 120
Logout | Help
Once an annotation is
saved it will appear in the
videos timeline.
Amsterdam, The
Netherlands
Search
Start Time:
My Content
5:01
End Time:
8:34
Title:
Amsterdam, The Netherlands
Note:
City street in Amsterdam.
-
Private
My Institution Only
Indexed/Searchable:
>>
Annotate
NJVid Wide
Save
<<
Start and stop
markers can be
placed.
Metadata
placed here.
Annotate and Capture Start/End Times
120
Slide 121