Transcript ECE-C490 Winter 1999 Image Processing Architecture

ECE-C490 Winter 2004
Image Processing Architecture
Lecture 1, 1/6/2004
Introduction
Huffman Coding
Oleh Tretiak
Drexel University
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 1
Introduction

Instructor: Oleh Tretiak



Lecture 1
Room 7-302, x2214
[email protected]
Office hours: Wednesdays 1-3, and by appointment
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 2
Introduction (continued)

Grading policy



Homework, two exams, final, Relative weight 1:2:2:3
Exams on January 29 and February 26, 2002
Homework due on Tuesdays
 Full credit on due date,
 50% if late

Reference book:

Lecture 1
Vasudev Bhaskaran and Konstantinos Konstantinides, Image and
Video Compression Standards: Algorithms and Architectures
(2nd edition), Kluwer Academic Publishers, 1997.
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 3
Course Overview


Introduction
Methods of Compression









Lossless
Lossy
Video Compression
JPEG Standard
MPEG Standards
Processing Requirements
RISC & DSP cores
Architectures for DCT, Motion Estimation, and Entropy Coding
Implementations: JPEG, Video, Multimedia
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 4
Digital Images
233 233 232 233 233 233 233 232 232 232 231 232 234 232 232 235 234 234 235 233 233 235 236 233 233 235 234 233 233 234 235
233 233 233 234 234 233 232 232 228 232 232 230 232 233 233 233 234 232 232 235 237 233 231 232 233 233 233 234 234 233 233
232 232 232 232 232 232 231 231 234 232 231 232 234 232 231 232 231 234 231 231 233 235 233 234 233 232 232 234 234 232 232
232 232 232 231 231 231 232 233 229 230 233 236 234 230 230 234 232 235 235 235 233 235 232 231 233 232 232 233 233 232 232
232 232 232 233 233 233 233 233 230 229 228 229 232 234 233 231 234 231 230 235 230 232 232 232 232 233 233 232 232 233 233
230 230 231 232 233 232 230 229 233 230 232 235 232 227 228 232 229 234 233 234 229 236 234 230 230 233 234 232 232 234 233
231 230 229 230 230 230 229 228 226 235 235 209 171 156 175 199 209 227 234 240 236 239 234 238 231 232 232 232 232 232 232
237 234 230 229 230 231 231 231 229 239 214 142
86
95 141 173 192 202 206 218 209 193 190 221 234 231 230 230 230 230 231
204 220 233 236 231 227 228 226 236 210 158 134
89
98 150 180 192 191 166 125 114
168 174 182 194 207 222 238 242 232 166 140 158 146 148 167 162 147 141 100
75
90
89 172 240 232 228 235 231 229 232
76 100 179 181 238 234 232 232 231 236 235
178 174 169 166 165 181 211 226 207 168 176 167 163 167 155 147 145 121 120 120 109
96 108 195 241 230 229 231 231 233 227
180 178 179 180 172 172 176 171 172 174 172 167 173 170 170 169 155 134 112 116 174 192 195 233 230 225 228 230 223 224 224
179 175 175 179 180 183 181 166 173 167 156 165 165 163 160 148 139 141 139 166 195 197 217 209 217 225 229 234 231 238 234
180 178 176 176 175 181 186 177 168 165 174 172 162 168 152 148 144 135 158 181 181 177 162 178 173 179 180 193 197 203 199
180 177 172 173 176 180 181 174 183 188 191 190 182 161 154 159 160 160 177 175 181 176 180 178 180 180 177 182 177 175 177
181 178 175 177 180 181 181 176 193 192 201 193 194 173 172 175 177 175 175 178 179 177 177 177 178 179 176 178 180 181 176
180 177 177 177 176 178 176 169 196 200 203 188 194 190 188 189 189 191 174 177 179 181 177 179 179 179 179 178 178 178 178
178 177 177 176 175 177 179 180 196 209 198 194 198 198 192 189 195 198 180 179 179 176 177 179 177 177 177 177 177 177 177
174 175 175 174 173 173 175 179 190 208 195 206 196 197 198 198 203 203 184 176 182 176 179 177 176 176 176 176 177 177 177
173 175 175 175 177 174 172 175 198 203 192 205 188 187 198 207 209 206 192 173 179 174 176 174 175 175 175 176 177 177 176
170 171 172 174 176 174 173 177 196 197 196 202 201 194 194 200 205 206 203 173 174 171 173 177 177 177 177 177 178 178 177
170 171 171 171 172 173 176 180 197 204 201 187 204 199 194 201 206 206 203 175 174 172 169 173 173 173 173 173 173 173 173
Red
Lecture 1
Green
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Blue
Page 5
Historical Perspective



VLSI
Communications
Digital Imaging




Computer Graphics (games)
Multimedia
Digital cameras and camcorders
Video
 Digital Television
 HDTV

Standards


Lecture 1
JPEG, MPEG, Px64
Firewire
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 6
Pictures and Movies

Still pictures





Documents
Monochrome (black and white) photographic prints
Color drawings
Color photographs
Moving pictures





Lecture 1
Gif animations
(Black and white film)
Color film
Television
Camcorder
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 7
Digitized Pictures

8.5”x11” document, scanned at 100 dpi (dots per inch)



2 dimensional array
Each array entry (pixel) is a bit (B/W)
How many bits?
 100x100 = 104 bits per square inch
 8.5x11 = 93.5 square inches
 93.5 x 104 ~ 106 bits

5”x7” color photograph, scanned at 200 dpi



2 dimensional array
Each array entry contains 3 numbers (R, G, B components)
Each component represented by 8 bits
 24 bits per array entry
 5x7x200x200 = 1,400,000
 24x1.4e6 = 3.4e7 bits = 4.2e6 bytes
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 8
Need for Compression

Example: Fax document




8.5x11”, 200 dpi, 8.5x11x(200)2 = 3.74 Mbits
@28.8 kbits/sec, 3740000/28800 = 130 sec
Typical compression = 15, with compression 8.65 sec.
Example: Video




Lecture 1
640x480 pictures, 24 bits/sample, 30 frames/sec
640x480x24x30 = 2.21E+08 bits/sec = 2.76E+07bytes/sec
A CD-ROM stores 650 Mbytes -> playing time = 650/27.6 = 23.5 sec
With compression, 74 minutes of low (VHS) quality video can be stored on a
CD-ROM
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 9
Lossless Compression

Lossless compression


The reconstructed (digital) data must be identical to original
Coding efficiency
 Compression ratio. Usually measured in bits/sample or bits/second.
This is limited by the entropy of the source. A random signal is hard to
compress.
 Coder complexity. How much calculation must the encoder/decoder
perform? Measured in memory requirements, operations/second,
and/or power requirements.
 Coding delay. Typically, more effective compression requires more
computation. This limits the throughput of the system. A more powerful
computing system may decrease this.
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 14
Lossy Compression

Images and audio are analog: not possible to reproduce exactly.
The signal may contain noise, which is not worth reproducing. In
addition to coding efficiency, coder complexity, and delay we
also trade off signal quality: the encoder may distort (add noise)
the signal.

Signal quality is usually measured in terms of SNR (signal-to-noise
ratio) or PSNR (peak signal to noise ratio)
input signal energy
noise energy
peak  to  peak signal
PSNR  20 log 10
rms noise
Warning: noise power may not be equivalent to noise appearance.
SNR  10 log 10

Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 15
Typical Encoder System
Signal
Data
Preprocessor

Entropy Coder
Issues

Constant Bit Rate vs. Variable Bit Rate
 In lossless encoder, bit rate depends on compression efficiency
 Variable bit rate is undesirable in real-time applications
 In lossy encoder, bit rate can be kept constant by varying quality

Single- or multiple-sample encoding
 Multiple sample encoding is usually more efficient but also is more
complex.
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 16
Entropy and Huffman Codes

Theory




Information
Entropy
Shannon Compression Theorem
Practice


Deriving symbols from signals
Huffman encoding
 Coder construction
 Encoders
 Decoders
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 19
Example: Compression



We have 1000 symbols from alphabet {a b c d}
Coding: 2 bits per symbol, total bits = 2x1000 = 2000
Variable length coding, if some symbols are more frequent
Symbol
a
b
c
d
Number Code
900
0
50
10
25
110
25
111
Bits
900
100
75
75
Total bits = 900 + 100 + 75 + 75 = 1150
Average bits/symbol = 1150/1000 = 1.15 < 2
Compression = source bits/code bits = 2/1.15 =1.74
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 20
Example: Compression
Symbol
a
b
c
d
Number Code
900
0
50
10
25
110
25
111
Bits
900
100
75
75
Encoding: Source = abaaaaca Code = 01000001100
Decoding: Code 0011100100 Decode = aadaaba
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 21
Information Theory

A signal consist of a sequence of ‘symbols’ from a finite
‘alphabet’, N is the number of symbols in the alphabet.





A scanned document is a sequence of bits, N =2
A text document is a sequence of letters, N ~ 64
A digital color video image pixel is represented by R, G, B
components, each a byte N =2^24 = 16,777,216
Note: we can represent each symbol by a binary number with
log2 N  bits.
Let the i-th symbol have probability pi. The information of this
symbol is defined as log2(1/ pi) bits. The average information of
this symbol is pi log2(1/ pi) bits. The entropy of this symbol set is
defined as
N
H   pi log2(1/ pi ) bits
i 1
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 22
Entropy Examples
Example 1
N = 2, binary source
H = .469 bits
i
pi
1
2
0.1
0.9
log 2 (1/p i )
3.322
0.152
pi log 2 (1/p i )
0.332
0.137
Example 2: Symbols are tosses of a die, N = 6, p = 1/6 for all symbols
H  6 (1/ 6log2(6))  log2 6 
Lecture 1
log10 6
 2.58 bits
log10 2
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 23
More Entropy Examples

We are given the sequence of numbers 4 5 8 6 4 7 8 9 4 8
We interpret his as a source with symbols 4, 5, 6, 7, 8, 9.
s1 = 4, s2 = 5, etc. Compute the relative frequency, and find that
p1 = p5 = 3/10, p2 = p3 = p4 = p6 = 1/10. Plug and chug, and get
H = 2.371 bits.
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 24
Shannon’s Source Coding Theorems



If we have a long sequence of symbols from a source whose per
symbol entropy is H, there is a way of encoding this sequence
with about H bits per symbol.
If the symbols are statistically independent, it is impossible to
encode these with fewer than H bits per symbol.
Comment: These are statistical results.
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 25
Variable Length Codes





Symbol set: si, i = 1 … N, pi — symbol probability
Code: ci, li, where ci is a sequence of 1’s and 0’s of length li.
The code words must be decodable: the transmitted bit stream
is just a set of 1’s and 0’s, code word boundaries are not
indicated. For a decodable code, the code word boundaries can
be found uniquely from the transmitted sequence.
Sufficient condition for decodability: A code is decodable if no
code word is at the beginning (prefix) of another code word. This
is called the prefix condition
N
Average code word length: Lave   pi l i
i 0
For a decodable code, Lave ≥ H
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 26
Example of a Variable-length Code
Symbol Probability Codeword
a
0.5
0
b
0.25
10
c
0.125
110
d
0.125
111
H = 1.75
Observe: this code satisfies the prefix condition.
Average number of bits per source symbol:
Av. bits. per symbol  1 0.5  2  0.25  3  0.125  3  1.25
 0.5  0.5  0.375  0.375  1.750
The code is perfect!
Compression = code bits/source bits = 2/1.75 =1.143
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 27
Huffman codes

Features






Must know the symbol probabilities
Procedure to design variable length codewords for the symbols
Encoding is performed one input symbol at a time
Code words are decoded uniquely, one symbol at a time (code has
prefix property)
Guaranteed to produce smallest Lave of all decodable codes that
encode one input symbol at a time
Steps



Lecture 1
Code construction
Encoding procedure
Decoding procedure(s)
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 28
Construction of a Huffman Code

Tree construction


Order the symbols according to probabilities
Apply contraction process to the two symbols with lowest
probability
 assign a new (hypothetical) symbol to these two,with probabilities equal
to sum of the code word probabilities


Repeat until one symbol is left
Code construction


For the two initial branches of the tree, attach bit ‘0’ and ‘1’ at end
of code word.
For both branches
 If (branch is a source symbol) done
 else repeat above process
Lecture 1
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 29
Code construction example
Symbol Probability
?
0.10
e
0.30
k
0.05
l
0.20
r
0.20
u
0.10
w
0.05
H = 2.55
ST 1
e 0.30
l 0.20
r 0.20
? 0.10
u 0.10
k 0.05
w 0.05
ST 2
e 0.30
l 0.20
r 0.20
? 0.10
u 0.10
A 0.10
ST 4
e 0.30
C 0.30
l 0.20
r 0.20
ST 6
ST 5
D 0.40 D 0.40
e 0.30 E 0.60
C 0.30
E=e,C
D=l,r
C=B,?
B=u,A
A=k,w
00
01
10
1100
11010
11011
111
Lecture 1
ST 3
e 0.30
l 0.20
r 0.20
B 0.20
? 0.10
r
l
e
u
k
w
?
?
e
k
l
r
u
w
111
10
11010
01
00
1100
11011
3
2 Lave
5
2
2
4
5
Image Processing Architecture, © 2001-2004 Oleh Tretiak

N
 pi l i
i 0
 0.10  3  0.30  2 
0.05  5  ...  0.05  5
 2.6
Page 30
Variable Length Code: Encoding

?
e
k
l
r
u
w
Source sequence: werule?
111
10
11010
01
00
1100
11011
Lecture 1
3
2
5
2
2
4
5
11011100011000110111
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 31
Prefix code: bit-serial decoding
11011100011000110111

Algorithm steps: bold denotes output symbols

Lecture 1
ECBAwEeDrECBuDl...
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 32
Prefix code: table decoding

Let k be the maximum code symbol length. Construct table with
2k entries. Each table location contains input symbol and code
word length. Order code by binary value. A code symbol of
length l will have 2k-l entries.
00
01
10
111
1100
11010
11011
Lecture 1
r
l
e
?
u
k
w
Since k = 5, we use 32 table entries. Code symbol ‘00’ will
use 25-2=8 entries. Each entry will have output symbol ‘r’ and
length 2. The next 8 entries will be for ‘l’. The following 8
entries will be for ‘e’. The following 4 entries will be for ‘?’, etc.
Decoding:Take k = 5 bits from encoded sequence. Decode it
by table lookup. From table, find the symbol length, discard
these many bits from the code word used for lookup, take
additional bits from encoded sequence.
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 33
Lookup table decoding example
11011100011000110111
00
01
10
111
1100
11010
11011
r
l
e
?
u
k
w





Lecture 1
First lookup code: 11011. Output = ‘w’, l = 5, read 5
bits from input stream.
Lookup code: 10001. Output = ‘e’, l = 2, discard the
initial ‘10’ from lookup code, read two more bits
Lookup code 00110. Output = ‘r’, l = 2, discard the
initial ‘00’ from lookup code, read two more bits.
Lookup code 11000. Output = ‘u’, l = 4, discard initial
‘1100’, read 4 more bits
Lookup code 01101. Output = ‘l’, ...
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 34
Summary


How to count bits per picture, bits per second on video
Principle of lossless compression:




Shannon’s entropy




Fixed length to variable length
Frequent symbols - few bits (short codes)
Infrequent symbols - many bits (long codes)
Formula H = –∑pi log2 pi
Can encode with about H bits per symbol
Can’t do better!
VLC



Lecture 1
Prefix condition
Huffman code construction
Huffman code decoding
Image Processing Architecture, © 2001-2004 Oleh Tretiak
Page 35