Transcript Coding is more than error

Inspired by Claude Shannon
Coding is more than error correction
A bit of magic
A.J. Han Vinck
A.J. Han Vinck, Joburg September 2012
A.J. Han Vinck, Joburg September 2012
Coding is needed in high tech devices
A.J. Han Vinck, Joburg September 2012
fundamental limits set by Shannon (1948)
The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, October, 1948.
Bell Labs, 1955.
Fundamental problem: reproducing at the receiver a message selected at the transmitter
Shannon‘s contribution:
- bounds on the efficiency (capacity)
- how to achieve the bounds
A.J. Han Vinck, Joburg September 2012
Introduction to the basic concepts
message:
Alpha
codeword:
(0 1 0 1 0 1 0 1)
transmission error
(0 0 0 1 0 1 0 1)
decode
(0 1 0 1 0 1 0 1)
translate:
Alpha
How can we do that?
A.J. Han Vinck, Joburg September 2012
It works as follows
•
Use the difference in transmitted vectors to recover the transmitted original
Example:
A
B
C
D
4 messages encoded as
00 000
01 011
10 101
11 110
the words differ in at least 3 positions
Hence, one difference (error) can be detected and corrected:
receive:
10110
the transmitted message is? 11110
A.J. Han Vinck, Joburg September 2012
1-Error correction started by Hamming (1950)
2 1 3
(3,1)
123
(7,4)
000
111
3567 124
Every circle has even parity
(31,26)
0000 000
1000 110
0100 101
•••
1111 111
info parities
minimum distance dmin = 3
A.J. Han Vinck, Joburg September 2012
What are the classical research problems
Codebook: maximize
- # of codewords of length n
- minimum distance dmin
Channel characterization:
- Types of errors; memory in the noise behavior ?
- Channel Capacity?
Decoder:
- Design of a decoding/decision algorithm
- Minimize complexity
A.J. Han Vinck, Joburg September 2012
Lecture content
Goal: give examples, different from just error correction
- improve storage efficiency in flash memory
- give biometrics privacy
- code design for variable efficiency in networking
- permutation codes for non-traditional error models
A.J. Han Vinck, Joburg September 2012
Ibm punch card „a flash memory“
Write, read
and refresh
Write, read and throw away, but •••
Write, read
and erase, but •••
On the Capacity of Write-Unidirectional Memories with Nonperiodic Codes”, IEEE Transactions on Information theory, April 2004, pp.
A.J. HanFu,
Vinck,
Joburg
2012 Yeung)
649-656, (Fang-Wei
A.J. Han
Vinck,September
Victor Wei, Raymond
A modern ROM: the Flash Memory
Cell array in a flash memory
A flash memory cell
(Floating gate)
-
-
A flash memory cell stores charge / data
We can only increase the cell charge
If charge top level is reached, erase memory block
A.J. Han Vinck, Joburg September 2012
Writing in a q-ary flash memory cell
Strategy for writing: increase charge content or stay at present content
Start at 0
Input
1, 0, 1, 0, 0, 0, 1, 1, 0, 1
Cell Level 0 1, 2, 3, 4, 4, 4, 5, 5, 6, 7
A.J. Han Vinck, Joburg September 2012
bit
cell charge level
value
7
6
5
4
3
2
1
0
1
0
1
0
1
0
1
0
Use the memory T=4 times:
how many sequences we can write into the memory?
q = 2:
T+1 = 5 words
write
0000
0001
0011
0111
1111
Note: writer and reader must know the content!
q = 3:
5 +T(T-1)/2 = 11 words
write
Capacity:
0000
0001
0011
0012
0111
0112
0122
1111
1112
1122
1222
log2|#sequences | ≈ (q-1) log2 (T+1)
A.J. Han Vinck, Joburg September 2012
Performance of the 1 step up strategy
probability
P(1) = 1 – P(0) = p
cell level
p
1-p
Probability (level change) = 2p(1-p)
p
p
1-p
1
0
1
0
1
0
1
0
1
0
1-p
average number of
writes to reach top level
T
 T  q 1
the # of bits stored is : log 2   
log 2 (T  1)
2
 pT 
which is 50% of the capacity !
A.J. Han Vinck, Joburg September 2012
q 1
2p(1  p)
Example of Codes: 2 bits in 3 cells
before erasing
00
01
10
11
2 bits
00
01
2 bits
10
11
A.J. Han Vinck, Joburg September 2012
An artist‘s view of fingerprints
A.J. Han Vinck, Joburg September 2012
Coding for Noisy Biometrics used as a key/password
I am Han,
can I come in?
- verify claimed identity with stored information
problems:
• Errors at authentication:
False Rejection (FRR)/ False Acceptance (FAR)
• Leakage: attacker finds information about bio from Data Base (DB)
A.J. Han Vinck, Joburg September 2012
the Juels-Wattenberg scheme (binary)
b‘(io)
b(io)
Random
Codeword c
cb
c  b  b‘ = c  e
cb
cb
Enrollment
Hash(c)
Authentication
Data
Base
Hash(c)
Hash(c)
=?
Y/N
Our contributions: use knowledge of bio at enrollment to encode
use c  b from database to improve decision on c
Yanling Chen and Han Vinck, A fresh look into the biometric authentication: Perspective from Shannon’s secrecy system and a special
wiretap channel, Secrypt, 2011
V. B. Balakirsky, A. R. G. and A. J. Han Vinck, Permutation
biometrical
A.J. Han block
Vinck,coding
Joburgfrom
September
2012authentication, Proc. 7th Int. Conf. on
Computer Science and Info. Technologies, 2009
Juels-Wattenberg for linear codes
Secret s { 1,2,•••,2k } → RS codeword
c
b(io) changes t positions
b
Enrollment:
store
= c1,c2,c3,••• ,cn
=> ---  --- ----c*
SECURITY: 2t +1 > dmin
Observer „cannot decode“
Authentication:
points
b‘
c*
t positions in c*
b‘
=> ---  --- -----
due to „dust“ b‘ points at (t – e) incorrect and e correct positions (e errors in b‘)
DECODABILITY: dmin – t > 2e
WHY?
Intuitive: Increasing dmin improves decodability, but decreases security and v.v.
A.J. Han Vinck, Joburg September 2012
Research topics we work on
• Translation from Bio to Template
– Iris, fingerprint, etc
011001…
• Investigation of error behavior
– Burst, impulsive, non-linear
• Performance estimation for real data
Practical performance FAR, FRR, etc.
• Using other techniques, such as data compression
– Fault tolerant hashing
A.J. Han Vinck, Joburg September 2012
000001…
Found this one somewhere
Escher
A.J. Han Vinck, Joburg September 2012
Communication over powerlines
• Disturbances:
– noise, attenuation, EMC, cable properties
• Noise:
– Narrowband noise
– Impulsive noise (broadband)
• Catastrophic effect on OFDM (multi-carrier)
A.J. Han Vinck, Joburg September 2012
A simple solution: Spread-FSK
2-FSK
mode
Send: f0 or f1 :
f0
f1
f
Detector: uses FSK or On-Off keying mode
Advantages: robust against narrowband noise
On-off
mode
alternative
f0
Send: (f0,f1 ) or (f1,f0 ) i.e. R = 1/2
f1
against: narrowband + impulsive noise + random
A.J. Han Vinck, Joburg September 2012
f
t
Detector: select max. # of agreements
Advantages: robust
f1
f0
f1
t
effect of correctable noise (simplified)
here 2 x 2
f0
f0
f0
f1
f1
f1
t
t
f0
f0
f1
f1
t
t
Red always has the majority!
A.J. Han Vinck, Joburg September 2012
t
General idea: represent code words as 2-dimensional
Time and frequency picture
(123)
t
f
(132)
f
t
f
t
(231)
t
(312)
f
t
(213)
t
(321)
f
f
Minimum number of differences is TWO !
A.J. Han Vinck, Joburg September 2012
effect of noise on permutation codes (simplified)
here 3 x 3
t
f
Transmitted
Background-insertion
Background-deletion
narrowband-jammer
frequency selective fading
f
Impulsive-broadband
A.J. Han Vinck, Joburg September 2012
Research questions:
• Design of code books: bounds on cardinality
| C |
# of permutations
M!
(Dp  1)!
• Design of decoding algorithm: low complexity
Hendrik C. Ferreira, A. J. Han Vinck, Theo G. Swart, Ian de Beer. Permutation
IEEE Transactions on Communications, 2005: 1782~1789
price for distance
„performance“
trellis codes.
• Performance analysis: practical channels
Constructing Coset Codes With Optimal Same-Symbol Weight
for Detecting Narrowband Interference in M-FSK Systems
IEEE Transactions on Information theory, December 2010, pp. 6347-6353
Versfeld, D. J. J. Vinck, A. J. H. Ridley, J. N. Ferreira, H. C.
A.J. Han Vinck, Joburg September 2012
Our (Olaf hooijen and me) modem design based on permutation codes,
1997, Cebit Hannover
A.J. Han Vinck, Joburg September 2012
Piet Mondriaan
Simple ideas need creativity
(De Zuid-Afrikaanse gruwelprenten van Piet Mondriaan, wie kent ze nog?)
A.J. Han Vinck, Joburg September 2012
Some lessons I learned
• Coding has a mathematical and a systems aspect
• Applications are widespread: from networking to chip level
• Implementations lead to low complexity algorithms
– Viterbi (Convolutional Codes)
– Berlekamp Massey (RS-codes)
• Background knowledge in Information and Communication
theory and combinatorics necessary
• Cooperation with different disciplines important
A.J. Han Vinck, Joburg September 2012
Journal Papers with UJ-Wits-PCS scientists
Constructing Coset Codes With Optimal Same-Symbol Weight for Detecting Narrowband Interference in M-FSK Systems
IEEE Transactions on Information theory, December 2010, pp. 6347-6353
Versfeld, D. J. J. Vinck, A. J. H. Ridley, J. N. Ferreira, H. C.
-„Combined Spectral Shaping Codes and OFDM Modulation for Narrowband Interference Channels,”
K. Ouahada, H.C. Ferreira, A.J. Han Vinck, A.J. Snijders and T.G. Swart
SAIEE Africa Research Journal, pp. 11-17, March 2008, ISSN 1991-169
- “Permutation Trellis Codes,”
IEEE Tr. On Communications, pp. 1782-1789, November 2005,
H.C. Ferreira, A.J. Han Vinck, T.G. Swart and I. de Beer.
-“Power Line Communication”, Encyclopedia of Electrical and Electronics Engineering, Wiley, New York, Edited by John
Webster, pp. 706-716, 199
H. Ferreira, H. Grove, O. Hooijen and A.J. Han Vinck,
-“Insertion/Deletion Correction with Spectral Nulls”
IEEE Trans. on Information Theory, March 1997, pp. 722-731.
(H.C. Ferreira, W.A. Clarke, A. Helberg, K.A.S. Abdel-Ghaffarand A.J. Han Vinck)
-"On the Undetected Error Probability of Linear Block Codes on Channels with Memory"
IEEE Trans. on Information theory, Vol. 42, No. 1, January 1996, pp. 303-309.
(A. Kuznetsov, F. Swarts, A.J. Han Vinck and H.C. Ferreira)
-"A Class of DC-Free, Synchronization Error Correcting Codes"
IEEE Trans. on Magnetics, Vol. MAG-29, No. 5, pp. 4048-4049, September 1993.
(A.S.J. Helberg, W.A. Clarke, H.C. Ferreira and A.J. Han Vinck)
A.J. Han Vinck, Joburg September 2012
my scientific Genealogy (coincidence?)
http://genealogy.math.ndsu.nodak.edu/
• Friedrich Leibnitz (1597, Leipzig)
• Carl Friedrich Gauß (Göttingen)
• John M. Wozencraft (MIT) (1925-2009)
• Me (University Duisburg-Essen, Germany)
– Digital Communications and - Coding techniques
A.J. Han Vinck, Joburg September 2012
baie dankie!
ngiyabonga ka khulu
A.J. Han Vinck, Joburg September 2012
EARLIEST DEPICTION OF JUGGLING known shows skillful Egyptian women on the 15th Beni Hassan
tomb of an unknown prince from the Middle Kingdom period of about 1994 to 1781 B.C
Shannon juggling machine and juggling theorem
A.J. Han Vinck, Joburg September
2012