Transcript Document

Memory, Definiteness, and Information
Losslessness of Finite Automata
1
Zvi Kohavi and Niraj K. Jha
Memory Span
Memory span: the amount of past input and output information needed to
determine the machine’s future behavior
Memory span w.r.t. input-output sequences (finite-memory machines): An
FSM M is defined as a finite-memory machine of order , if  is
the least integer s.t. the present state of M can be determined
uniquely from the knowledge of the last  input symbols and the
corresponding  output symbols
• I.e., every input sequence of length  is a homing sequence
2
Testing Table and Testing Graph
Example: Consider machine M1 and its testing table and testing graph
AB
0/0
BD
0/0
CD
0/0
1/0
0/0
AC
1/1
BC
0/0
AD
Uncertainty
pair
Implied
pair
3
Conditions for Finite Memory
Theorem: A sequential machine M has a finite memory if and only if its
testing graph is loop-free
Example: the testing graph of M1 has two loops – hence it’s not finite
memory
• An arbitrary long string of 0 input symbols: will never resolve uncertainty
(CD)
• Similarly, if initial uncertainty is (AC): input sequence 0101…01 will
transfer the machine to (BD), (AC), (BD), and so on
AB
0/0
BD
0/0
CD
0/0
1/0
0/0
AC
1/1
BC
0/0
AD
4
Corollary
Corollary: Let G be a loop-free testing graph for machine M. If the length of
the longest path in G is l, then  = l + 1
Example: Machine M2 for which  = (n-1)n/2
AB
0/0
1/0
BC
0/0
CD
0/0
1/0
AC
0/0
BD
1/0
0/0
AD
5
Determining Whether a Graph is Loopfree
Connection matrix of directed graph G with p vertices: a pxp matrix, whose
(i,j)th entry is 1 if there is an arc emanating from vertex i and
terminating at vertex j, and is 0 otherwise
•
•
If G is loop-free: then it has one or more terminal vertices
The subgraph resulting from the removal of a terminal vertex and all arcs
leading to it is also loop-free
Example: connection matrix for machine M2 with  = 6
6
Memory Span wrt Input Sequences
(Definite Machines)
A sequential machine M is called a definite machine of order  if  is the
least integer s.t. the present state of M can be determined
uniquely from the knowledge of the last  input symbols to M
•
•
•
•
•
A definite machine is thus said to have finite input memory
Definite machine of order  :  -definite machine
A  -definite machine is also finite memory of order equal to or smaller
than 
The knowledge of any  past input values is always sufficient to
completely specify the present state of a  -definite machine
Canonical realization:
x
D
x1
D
x2
Combinational logic
D
xu
z
7
Properties of Definite Machines
A machine is  -definite if and only if every sequence of length  is a
synchronizing sequence
•
Length of the longest path to a singleton uncertainty in the synchronizing
tree: order of definiteness
Example: Machine M3: definite of order 3
Level
(ABCD)
0
0
1
(AC)
0
(BD)
0
1
(A)
(BD)
0
(C)
(C)
1
1
(B)
2
1
(B)
3
8
Machine Contraction
Example: Machine M4 and its contracted table M4
If M is
 -definite: M is ( -1)-definite
– Conversely, if M is k-definite: then M is (k + 1)-definite
– If M is not definite: neither is M
9
Test for Definiteness
First test for definiteness:
1. Determine the subsets of states whose Ik-successors are identical
2. Select one representative in each subset
3. Obtain the contracted table M by replacing each subset with its
representative, and modifying the table accordingly
4. Regard M as a new table and repeat the previous steps until no new
contractions are possible
• M is definite if and only if the final contracted table consists of just a
single state
Example: Machine M4 and its contractions
10
Theorem
Theorem: If machine M is  -definite, then  <= n-1. Moreover the order of
definiteness is equal to the number of contractions needed to
obtain a one-state machine
• Since for machine M4, four contractions are necessary to obtain a onestate machine: its  = 4
11
Testing Table/Graph for Definiteness
Example: Testing table and testing graph for machine M3
CD
1
AB
0
1
0
1
AC
0
AD
BD
0
BC
Theorem: A machine is  -definite if and only if its corresponding testing
graph G is loop-free. If the length of the longest path in G is l,
then  = l+1
•
Machine M3: definite of order 3
12
Memory Span wrt Output Sequences
An FSM M is said to have an output memory of order  if  is the least
integer s.t. the knowledge of the last  output symbols suffices to
determine the state of M at some time during the last  transitions
Testing table for output memory:
Example: Machine M5
Output-successor
table
13
Example (Contd.)
Example (contd.): Testing graph for M5
1
AB
1
CD
1
AD
BD
0
0
BC
1
0
AC
Theorem: An FSM has a finite output memory if and only if its
corresponding testing graph G is loop-free. Furthermore, if G is
loop-free and the longest path in G is of length l, then M has an
output memory of order  = I + 1
• For M5:  = 4
14
Determining the State of the Machine
Example: For machine M5
• Suppose the output sequence is 1110
• Initially, the machine could have been in: A, B, or D
• Thus, initial uncertainty: (ABD)
15
Information Lossless Machines
An FSM M is information lossless if the knowledge of the initial state,
output sequence, and final state is sufficient to determine
uniquely the input sequence
Conditions for lossiness:
x2/z2
Si
x1/z1
xn/zn
Sc
Sf
x1/z1
xn/zn
x2/z2
Sj
Example: Machine M6 is lossy
A
0/0
1/0
A
B
1/0
0/0
B
16
Information Losslessness of Finite Order
An FSM is said to be information lossless of finite order if the knowledge of
the initial state and the first  output symbols is sufficient to
determine the first input symbol uniquely
Example: Machine M7: lossless machine of first order
17
Test for Information Losslessness
Two states, Si and Sj, are said to be output-compatible if there exists some
state Sp s.t. both Si and Sj are its Ok-successors, or if there
exists a pair of states Sr,St, s.t. Si,Sj are their Ok-successors
• In the latter case: (SiSj) is implied by (SrSt)
Example: Machine M8: testing table for information losslessness
18
Testing Graph
A machine is lossless if and only if its testing table does not contain any
compatible pair consisting of repeated entries
Testing graph for M8: lossless
AE
1
AD
0
1
1
BC
0
AB
1
0
DE
0
AC
19
Theorem
Theorem: A machine M is lossless of order  = l + 2 if and only if its testing
graph is loop-free and the length of the longest path in the graph
is l
•
•
Case of  = 1: detected by the absence of compatible pairs
Case of  = 2: detected by the absence of arcs in the graph
Example: Since the testing graph of M8 is not loop-free: it is not lossless of
finite order
AE
1
AD
0
1
1
BC
0
AB
1
0
DE
0
AC
20
Example
Example: Machine M9: lossless of order 3
AB
0
AC
0
AD
1
0
0
BC
1
1
BD
1
CD
21
Retrieval of the Input Sequence
Example: Consider machine M8
• Assume the machine was
initially in A
• Suppose output sequence:
110001100101
• The machine terminates in B
22
Inverse Machines
An inverse Mi is a machine which, when excited by the output sequence of a
machine M, produces (as its output) the input sequence to M, after
at most a finite delay
•
M must be lossless of finite order
Example: Machine M7 and its inverse M7i
23
A Deciphering System
Schematic diagram:
Inverse machine
(u-1)-delay register
D
Input
x(t)
Logic
Output
z(t)
S(t)
D
D
Combinational logic
z(t-u+1)
Decoded
message
x(t-u+1)
Logic
S(t-u+1)
Delays
Delays
Coding machine M
Copy of original
machine M
24
Minimal Inverse Machine
Example: Machine M10: lossless of third order
• Knowledge of initial state and three successive output symbols: yields the first
input symbol
• Possible triples: (A,0,0), (A,1,1), (B,0,1), (B,1,0), (C,0,0), (C,0,1), (D,1,0), (D,1,1)
• For every state of M10: the next inverse state is a triple whose members are
obtained as follows:
– First member: the state to which M10 goes when it is initially in the state that
is the first member of the present inverse state, and when it is supplied with
the first input symbol
– Second member: third member of the corresponding present inverse state
– Third member: present output of M10
25
Example (Contd.)
Example (contd.): Suppose M10 is initially in state A, and produces either
00 or 11 in response to some input sequence
•
•
Then, two time units later: M10i must be in the state that corresponds to A
and the appropriate output sequence, i.e., (A,0,0) or (A,1,1)
Since S4 = (B,1,0) is the only state from which M10i that can reach (A,0,0)
and (A,1,1), when supplied with input sequences 00 or 11: if the initial
state of M10 is A, the initial state of M10i must be (B,1,0)
State of M10
Input to M10
Output of M10
i
State of M10
i
Output of M10
A C B D C A D D C B
0 1 0 0 0 1 1 0 1
0 0 0 1 0 1 1 1 0
S4 S5 S1 S5 S3 S1 S6 S2 S2 S1
0 1 0 0 0 1 1
26
Synchronizable and Uniquely
Decipherable Codes
Source alphabet: {A,B,C,…}
Code alphabet: L = {0,1,2,…}
Binary code: L = {0,1}
Code word: concatenation of a finite number of code symbols
Code: finite number of distinct code words of finite length, each
representing a source symbol
Coded message: concatenation of code words, without spacing or any
other punctuation
Example: L = {0,1} and set of code words  1= {00,01,11,10}
•
Sequence ABDC: coded as 00011011
27
Uniquely Decipherable Code
Example: Not in every case can we work backwards to find a unique
sequence of source symbols for a given binary sequence
• If  2 = {0,00,01} represents {A,B,C}: then 0001 may be decoded as AAC
or BC
Uniquely decipherable code: if and only if every coded message can be
decomposed into a sequence of code words in only one way
•  1 is uniquely decipherable, 2 is not
• Whenever the number of code symbols is not the same for all code
words: the code is not necessarily uniquely decipherable
• On the other hand: 3 = {1,01,001,0001} is uniquely decipherable since
symbol 1 actually serves as a separator between successive code words
– Such a separator is called a comma
– Such a code is called a comma code
• Block code: in which all code words contain the same number of symbols
• Variable-length code: in which the number of symbols representing code
words is not the same
28
Instantaneous Code
Instantaneous code: whenever each code word can be deciphered without
knowing the succeeding code words
•
 1and 3 are instantaneous codes: while 4 = {1,10,100} is not since
sequence 10 cannot be deciphered until we verify that the next symbol is
a1
Let  =  1 2...n be a code word: then  1 2...m ,m<=n, is called a prefix
of 
•
A necessary and sufficient condition for a code to be instantaneous is
that no code word is a prefix of some other code word
Reason for using variable-length codes: reduction in the average length of
messages
•
•
Use shorter code words for more frequently used symbols
Average length of code:  Pili
– Pi and li: probability of occurrence and length of the code word
representing the ith source symbol
29
Test of Unique Decipherability
A code is uniquely decipherable with a finite delay  if and only if  is the
least integer s.t. the knowledge of the first  symbols of the
coded message suffices to determine its first code word
•
•
Insert a separation symbol S at the beginning and end of each code
word in 
In every code word representing source symbol N: insert symbol Ni
between its ith symbol and its (i+1)st symbol
Example: If source symbols are {A,B,C} and
words are
•
 = {0,01,1010}, then the code
Separation symbol to the right of the code symbol: k -successor,
denoted Ri, of the left separation symbol
– C1: 1-successor of S because S1C1 occurs in the third code word
30
Testing Table for Unique Decipherability
Two successors, Ri and Rj, are compatible if S kRi and S kRj occur, or if
Rpk Ri and Rq kRj occur, and Rp and Rq are compatible
•
(RiRj) is said to be the compatible implied by (RpRq)
Testing table can be constructed in the following manner:
1. The column headings of the table are the symbols of the code alphabet
2. The first row heading is S. The other row headings are the compatible
pairs
3. The entries in row RpRq, columnk, are the compatibles implied by
(RpRq) under k
Example: Testing table for our example
31
Testing Table (Contd.)
If a repeated pair (SS) occurs in the table: the code is not uniquely
decipherable, else it is
•
•
•
Implies that there exists some compatible pair (RiRj) s.t. S is the
-successor of both Ri and Rj
However, since both Ri and Rj are reachable from S by a binary
sequence that corresponds to two or more different sequences of source
symbols: the code is not uniquely decipherable
Tracing back the compatibles which implied pair (SS): we can find
shortest ambiguous messages
– Ambiguous message 01010 may be interpreted as AC or BBA
S
0
(SB1)
1
(SC1)
0
(B1C2)
1
(SC3)
0
(SS)
32
Testing Graph
Example: Testing table and graph for code = {1,10,001}
•
Uniquely decipherable since (SS) is not generated
S
1
SB1
0
SC1
0
C1C2
A code is uniquely decipherable with a finite delay  if and only if its testing
graph is loop-free
•
Delay  = I + 1: l is the length of the longest path in the testing graph
33
Deciphering a Coded Message
Example: Consider code = {11,011,001,01,00}, which is known to be
uniquely decipherable
•
•
We want to decode 0011101100011010011
Scanning from left: insert a lower comma whenever a sequence, which
corresponds to a legitimate code word, is detected
, , , ,
,
, , ,
,
,
, ,
0 0,1,1,1,0 1,1,0 0,0 1,1,0 1,0 0,1,1
•
•
Next, scan the coded message from the right and insert an upper
comma: whenever a sequence which corresponds to the inverse of a
legitimate code word is scanned
If the code is uniquely decipherable: the message can be decoded by
retaining only those commas that occur in the upper and lower spaces
simultaneously
34
Test for Synchronizability of Codes
A code is said to be synchronizable of order  if  is the least integer s.t.
the knowledge of any  consecutive code symbols is sufficient to
determine a separation of code words within these symbols
Testing a code for synchronizability: analogous to testing an FSM for finite
output memory
•
A code is synchronizable if and only if it is uniquely decipherable and its
testing graph is loop-free. It is synchronizable of order  if and only if the
longest path in the graph is of length  - 1
Example: Testing table and testing graph for
 = {1,10,001}
B1C1
0
B1C2
SC2
1
C 1C 2
SB1
0
SC1
0
35