Theory of Computing Lecture 24 MAS 714 Hartmut Klauck

Size of Automata?

• • • We know L is regular, if there is a DFA with a finite number of states for L Minimum number of states is the Myhill Nerode Index Questions: – How can we find a minimal DFA for L?

– Is it unique (up to renaming states)?

– Are NFA smaller than DFA?

– Are two-way DFA smaller than DFA?

Question 4

• • • • Consider the language L={xi : x 2 {0,1} n and i 2 {0,1} log n – n is fix, this is a finite language and x i =1} It is easy to see that there are 2 n rows in the communication matrix – Rows labeled x contain the string on columns i=1…n Hence any DFA for L has 2 n states.

Can give a 2-way DFA with O(n 2 ) states: – go to the right end of the input, read i into the state, move n-i steps left, accept if there is a 1

Question 3

• • • • • Consider L={xy: x,y – 2 {0,1} n , x  This is a FINITE language, n is fixed y} Exercise: there is an NFA with O(n 2 ) states for L DFA-size: the matrix has more than 2 n rows – DFA size is exponential distinct Hence NFA can be exponentially smaller than DFA for some languages Example where they are not: complement of L

2) Uniqueness

• • • Uniqueness of the minimal DFA (up to vertex names) follows from the Myhill-Nerode characterization The optimal DFA has exactly one state for each set of equal rows of the comm. matrix Edges are also determined uniquely

1) Optimal DFA

• Theorem: For every regular L there is a unique DFA A that has minimum size. Given a DFA M for L we can find A in polynomial time.

• • Corollary: Given DFA A,B we can decide in polynomial time whether they compute the same language – Minimize DFA, compare graphs (note that this is not an instance of the (hard) graph isomorphism problem, since we know the starting state and edge labels) Corollary: Given DFA A, we can decide in polynomial time whether L A is empty

Note

• • The corresponding questions for Turing machines are undecidable These properties (easy to find unique representation, easy to find small representation, easy to check for emptiness and easy to compare) make DFA a good data structure for languages/Boolean functions

DFA Minimization

• • • • • • All algorithms try to recover equivalence classes in the Myhill-Nerode characterization I.e., try to find the smallest partition of the rows of the communication matrix into equal rows Input: DFA with n states and alphabet size k Hopcroft: O(nk log n) Moore: O(n 2 k) Idea in both: – Start from partition of states into accepting/rejecting – Refine partition until no longer possible

Proof

• • • • We are given a DFA M with n states and alphabet size k Want to find an optimal DFA A First: remove unreachable – – states unreachable means in the graph, cannot be reached from q 0 on any input Identify them with DFS from the starting state Equivalent states: q,q’ are equivalent, if all strings y, starting from q or q’ lead to the same result (acc/rej) – can assume |y|< n, otherwise y contains a loop in M – equivalent states have the same row in the communication matrix

A simple algorithm

• Algorithm: – Remove unreachable states – Build a n £ n matrix D that containing blank entries – For all rows q, and columns q’, where q 2 is not, set D[q,q’]=D[q’,q]= ² F and q’ – Iterate [until nothing to do, at most n times] • Loop over all letters a and all pairs q,q’ with D[q,q’] blank • If D[ ± (q,a), ± (q’,a)] is not blank, set D[q,q’]=a – Join states q,q’ that have D[q,q’] still blank

A simple algorithm

• • • D allows us to find a equivalent: D[q,q’]=a, then find recursively the witness y for ± witness (q,a) and ± that q,q’ are not (q’,a), witness for q,q’ is ay – Follow ay from q and from q’, one leads to accept, the other to reject Witnesses need not be longer than n-1 If no witness exists, then q,q’ are equivalent – by definition of equivalence

Time

• • Each iteration takes time O(n 2 k) If q and q’ are not equivalent, a witness of length no more than n-1 can be found, i.e., number of iterations is at most n-1 – typically much less: longest simple path in DFA

Correctness

• • • All q,q’ with D[q,q’] not blank are not equivalent Claim: algo will find a witness for non-equivalent q,q’ Induction over length of witness – length 0: in first iteration – Assume length k witnesses are found in k iterations – If q,q’ are not equivalent, and there is a string ay of length k such that ± (q,ay) accepts, ± (q’,ay) rejects, then some witness y’ for ± (q,a) and ± (q’,a) of length k-1 is found earlier – ay’ is a witness and found in iteration k Claim: all equivalent states will never be declared not equivalent – Clear from the witness computed

Minimizing NFA?

• • • NFA are frequently smaller than DFA Can we minimize their size?

Theorem:

– NFA-minimization is PSPACE-hard – This remains true if the input is a DFA but we search the smallest NFA – Even approximation is hard

Conclusion

• • • DFA can be transformed into unique minimal DFA in polynomial time – Can then compare, decide emptiness etc.

NFA and 2-way DFA/NFA can be exponentially smaller, but cannot be minimized efficiently Limits of DFA: – Communication method

Regular Languages (Definition 2)

• • Simple recursive description of languages Definition: Regular expression over alphabet § – – – – – – a 2 § is a r.e.

The empty word ² is a r.e.

; is a r.e. [empty set] R 1 [ R 1 R 2 R 2 is regular [union] Notation: r is regular [concatenation] 1 R 1 * is regular [Kleene] | r 2

Examples

• • • • • • 0*10*: strings with one 1 § * 1 § *: strings with at least one 1 ( §§ )*: even length 1*(011*)*: every 0 followed by a 1 (0*10*10*)*: even number of 1’s 0*|1*: contains only 0’s or only 1’s

Regular vs. DFA

• • • • Definition: a language describable by a regular expression is called regular Theorem: The set of regular languages is the same set as the set of languages decidable by DFA Proof: We show – NFA can simulate regular expression – Regular expression can be constructed from DFA

Regular to NFA

• • • • • Consider an NFA with ² – transitions Edges can be labeled with ² • (empty word) Exercise: NFA with ² transitions can be transformed into NFA, same number of states R a regular expression Inductively find an NFA for R Basis cases: R= ² , R=a 2 § , R= ; – NFA is trivial (at most 3 states) Induction: Find NFA for Union, Concatenation, Kleene Hull

Regular to NFA

• • Union R 1 [ – R 2 : NFA M 1 M 1 and M 2 , starting state and to start in M 2 ² -move to start in Concatenation: • Kleene:

DFA to regular

• • • • Generalized DFA: edges can carry regular expressions instead of letters Idea 1: start with a normal DFA, shrink to a generalized DFA with 1 edge Idea 2: dynamic programming type argument R i,j,k regular expression for the set of strings that lead from q i to q j using q 0 ,…, q k only

DFA to regular

• • • • • • • R 0,0,0 =(a|b|c)* if there are edges labeled a,b,c from q 0 to q 0 R i,i,0 = ² otherwise R i,j,0 = a|b etc., if there are edges labeled a and b from q i to q j R i,j,k = R i,j,k-1 | R i,k,k-1 (R k,k,k-1 )* R k,j,k-1 Finally take union over all R 0,i,n , where q i accepting is Result: a reg. expr. for the DFA Works even for NFA

Conclusion

• Theoretical Computer Science – Algorithms – Computability – Complexity – Machine Models – Formal Languages – More: Cryptography, Information/Communication Theory …

Topics:

• • Algorithms: – Basic Algorithms [Sorting/Searching] – Graph Algorithms – Paradigms [Greedy, Dynamic Programming] – Linear Programming – Algorithms for hard problems [Approximation] Other theory: – Hardness and reductions [eg. NP-completeness] – Computability [eg. Halting problem, Universality] – Time-Hierarchy [etc.] – Finite Automata