Transcript Error Correcting Code Dr. W. C. Shiu Hong Kong Baptist University Department of Mathematics.

Error Correcting Code
Dr. W. C. Shiu
Hong Kong Baptist University
Department of Mathematics
1
2
Examples:
 Let us vonsider the following example.
 I am a dog, so you have to worship me.
 Meet me at 8:00 c.m.
Parity Check:
ISBN-10 (International Standard Book Number)
3
0-471-61884-5
1 0  2  4  3  7  4  1  5  6  6  1  7  8  8  8  9  4
 225  5 (mod 11)
In general, assume the first 9 digits of an ISBN is
a1a2…a9, where 0  ai  9 for 1  i  9, then the tenth
digit (the check digit) is
9
1a1  2a2  3a3  4a4  5a5  6a6  7a7  8a8  9a9   iai
 a10 (mod11),
i 1
where 0  a10  10. If a10=10, then use X to instead of it.
4
Parity Check: ISBN-13
Jan. 1, 2007,
ISBN-13 will instead of ISBN-10
How do we convert ISBN-10 to ISBN-13?
5
Parity Check: ISBN-13
 Drop the check digit from the existing ISBN10
 Add the prefix 978 or 979 (usually is 978)
 Calculate the check digit using modulo 10:
Suppose the 13 digits is a1a2…a13, where
0  ai  9 for 1  i  13, then it satisfies
a1  a3  a5  a7  a9  a11  a13
 3(a2  a4  a6  a8  a10  a12 )  0 (mod 10)
6
Parity Check: ISBN-13
Original ISBN-10: 0-471-61884-5
New ISBN-13: 978- 0-471-61884-?
(9  8  4  1  1  8  ?)  3(7  0  7  6  8  4)
 30  ? 3(32)  ? 3(2)  ? 6 (mod 10)
?4
The new ISBN-13 is 978- 0-471-61884-4
7
香港身份證號碼
X354670(?)
身份證號碼的「結構」，可以用  abcdef (z)表示。
「」可能是「空格」或是一個英文字母，「  」則
必定是英文字母。「 abcdef 」代表一個六位數字，而
「z」是作為檢碼之用，它的可能選擇是0, 1, 2, ... , 9,
A(代表10)。這些代號的背後，都可配上一個編碼值。
透過編碼值，便可找出
9+8+7a+6b+5c+4d+3e+2f+z
的總和。該總和特別之處，是必須被11整除。利用這
特點，我們便能找出括號內的數字。試試看！
8
 或  的編碼值:
空格
A
B
C
D
E
F
G
H
58
10
11
12
13
14
15
16
17
I
J
K
L
M
N
O
P
Q
18
19
20
21
22
23
24
25
26
R
S
T
U
V
W
X
Y
Z
27
28
29
30
31
32
33
34
35
9
X354670(?)
9(58)+8(33)+7(3)+6(5)+5(4)+4(6)+3(7)+2(0)+z
= 902+z
被11整除，所以z=0。
我們可利用Modular arithmetic來簡化運算。
z = 987a6b5c4d3e2f
 2+3+4a+5b5c4d3e2f (mod 11)
所以
z  2(58)+3(33)+4(3)+5(5)  5(4)  4(6)  3(7)  2(0)
 2(3)+3(0)+12+252024210
 6+0+1+3+22+10=11  0 (mod 11)
即 X354670(0) 是正確的香港身分證號碼。
10
Message
We think of a message as a block of symbols
from a finite alphabet.
A commonly used alphabet is the set of two
symbols 0 and 1. A possible message is 1001.
This message is then transmitted over a
communications channel that is subject to some
amount of noise.
11
No Encoding-Decoding System
Noise e
Message
u=1001
Channel
Received Message
v=0001
12
Encoding-Decoding System
Noise e
Message
Encoder
u=1001
Eu=b=1001100
Message
Decoder
E-1(Dc)=1001
Dc=1001100
Channel
Received Message
c=b+e=0001100
13
Binary Symmetric Channel (BSC)
No memory. Receives and transmits two symbols 0 and 1.
The BSC has the property that with probability q a
transmitted digit will received correctly, and with
probability p=1q it will not be.
Boolean sum and product:
 0 1
0 0 1
1 1 0
 0 1
0 0 0
1 0 1
14
Encoded Message
Digits in the original message :
Information digits
Digits added by the encoder :
Redundancy digits
Length of a code = the number of information
digits + the number of redundancy digits
15
Examples of single-error detecting code:
1. Repetition code: Each message is repeated once.
E
Channel
1001 1001100110001001
2. Even-parity check code: Add an extra digit to
the message. It is a 0 if there are an even
number of 1’s, otherwise a 1.
E
Channel
1001 10010 10000
16
Examples of Single-Error Correcting Code
Repetition code: Each message is repeated twice.
E
Channel
10011001100110
01  
1001100010
01
D
E 1
1001100110
01 
1001
It can also correct some double errors and more
errors.
100110011001100010111001 100110011001 
100110011001100010001001 100010001000 
100110011001100010110101 100110011001 
17
Examples of Single-Error Correcting Code
Hamming (7,4) code: Given a matrix
1 0 0 0 0 1 1


0 1 0 0 1 0 1
G
0 0 1 0 1 1 0


0 0 0 1 1 1 1
If u=(u1,u2,u3,u4), then we decode it as b=uG.
E
Channel
1001 (1001)G  (1001100)  1000 100  c
18
Examples of Single-Error Correcting Code
 0 0 0 1 1 1 1
Let d=DcT, where D   0 1 1 0 0 1 1


 1 0 1 0 1 0 1
For this example d  (1 0 0)
T
The error occurred at the 4th position.
D is called a parity check matrix.
19
Probability for Correct Sending
Suppose we are transmitting a message of 4 digits on
a binary channel and q=0.9. Then the probability of
correctly sent for
No coding: q4=0.6561;
Repetition code:
q12  12 q11 p 
12  9 10 2 12  9  6 9 3 12  9  6  3 8 4
q p 
q p 
q p  0.8926 ;
2!
3!
4!
Hamming code:  q7+7q6p  0.8503.
20
Probability for Correct Sending
Rate of a code is the ratio of the number of
information digits with the length of the code.
Rate of the repetition code is 1/3.
Rate of the Hamming code is 4/7.
21
Linear Codes
An (n, k) linear code C over a finite field F is a kdimensional vector subspace in F n. Elements of C is
called codeword. If F={0, 1}, then C is called a binary
code
Any kn matrix G whose rows are the basis vectors of
C is called a generator matrix. This means that for each
C, there exists b=(b1,…,bk) such that =bG.
There exists (will show a proof later) an (nk)n
matrix H called parity check matrix of C such that

C  {  F | H  0}
n
T
22
Linear Codes
By applying a sequence of elementary row operations
and column operations, we may assume
G  ( I k A)
Theorem : If an (n, k) code C over a field has a
generator matrix G=(Ik A), then H=(-AT In-k) is a parity
check matrix of C.
Proof: H ( xG)T  HGT xT ,
and rank(H)=n-k
HG  O
T
23
Hamming (7,4) code
Generating matrix
1

0
G
0

0
0
1
0
0
0
0
1
0
0
0
0
1
0
1
1
1
Parity check matrix
1
0
1
1
1

1
0

1
0 1 1 1 1 0 0


H  1 0 1 1 0 1 0
1 1 0 1 0 0 1


 0 0 0 1 1 1 1
D   0 1 1 0 0 1 1


 1 0 1 0 1 0 1
24
Decoding Scheme
Let   Fn . The weight of  is defined by wt()
the number of nonzero digits in =(a1,…,an).
The distance of two vectors ,  Fn is defined
by d(,)=wt().
The method of decoding a received vector to the
closest codeword. It is called maximumlikelihood decoding.
25
Decoding Scheme
For binary code, if u is transmitted and v is
received, then the error e is defined to be
v u  v u
Then d(u,v)=wt(e). Thus the decoded codeword
is chosen u  e  v such that
wt (e)  min{wt (e) | e  v  C}.
26
Mini-example
Consider a binary code C with generator matrix
1 0 1 0


0 1 1 1
Therefore, C={0000, 1010, 0111, 1101}. The
number of all binary sequences of length 4
are 16.
27
Mini-example
A decoding scheme is
0000
1000
0100
0001
1010
0010
1110
1011
Another decoding scheme is
0000 1010
0010 1000
0100 1110
0001 1011
0111
1111
0011
0110
1101
0101
1001
1100
0111
0101
0011
0110
1101
1111
1001
1100
28
Detect and Correct Errors
The minimum weight d of a code is defined to
be the minimum value among all nonzero
codewords.
Theorem: C can detect t errors if and only if
t=d-1.
Theorem: C can correct t errors if and only if
t=[(d-1)/2].
29
Decoding Scheme
Suppose u is transmitted, v is received and e is
the error, i.e., u+e=v.
Then HvT=H(u+e)T=HuT+HeT=HeT.
The decoding scheme is to choose a (fixed)
T
T

e

vector
such that Hv  He
, and then to
decode the received vector to
v  e  v  e
30
Hamming Code
Hamming (7,4)-code or Ham(3,2)
 0 0 0 1 1 1 1


H   0 1 1 0 0 1 1
 1 0 1 0 1 0 1


Hamming (15,11)-code or Ham(4,2)
0

0
H 
0

1
0
0
1
0
0
0
1
1
0
1
0
0
0
1
0
1
0
1
1
0
0
1
1
1
1
0
0
0
1
0
0
1
1
0
1
0
1
0
1
1
1
1
0
0
1
1
0
1
1
1
1
0
Any Hamming code can only correct one error.
1

1
1

1
31
Hamming Code
Given 15 coins which should be of equal weight,
but you are told that there may be one that is
different from the others. You have a balance on
which to compare the weights of different coins or
sets of coins. Devise a scheme using four balance
checks to determine which coin, if any, has the
wrong weight.