Transcript Lec-01(Automata) - Computer Science Unplugged

CSC312
Automata Theory
Lecture # 1
Introduction
Administrative Stuff
• Instructor: Dr. Mudasser Naseer
[email protected]
Cabin # 1, Faculty Room C7
• Lectures:
Sec-B: Wed 1630, Thu 1800 hrs.
Sec-C: Wed 1800, Thu 1630 hrs.
•Office Hrs: Tue & Thu 1400 – 1600 hrs
(or by appointment)
•Prerequisite: CSC102 - Discrete Structures
Course Organization
Text Book:
i) J. E. Hopcroft, R. Motwani, & J. D. Ullman
Introduction to Automata Theory,
Languages, and Computation, Third Edition,
Pearson, 2008.
ii) Denial I. A. Cohen Introduction to Computer
Theory, First Edition, John Wiley & Sons, 1986.
Instruments:
There will be 2~3 assignments, 4~5 quizzes,
Weights: Assignments
10%
Quizzes
15%
S-I
10%
S-II
Final Exam
15%
50%
Course Outline
Study and mathematically model various abstract
computing machines that serve as models for
computations and examine the relationship between
these automata and formal languages. Regular
expressions, NFAs. Core concepts of Regular
Languages and Finite Automata; Decidability for
Regular Languages; Non-regular Languages; Contextfree Languages and Pushdown Automata; Decidability
for Context-free Languages; Non-context-free
Languages; Turing Machines and Their Languages are
important part of the course. Transducers (automata
with output).
Schedule of Lectures
Lect.#
Topics/Contents
1
Introduction to Automata theory, Its background, Mathematical Preliminaries,
Sets, Functions, Relations, Graphs, Proof Techniques
2
Formal Languages, Introduction to defining languages, alphabet, language,
word, null string, length of a string, reverse of a string, Palindrome, Kleene
closure.
Defining languages recursively, Recursive definitions of languages like EVEN,
INTEGERS, POSITIVE, POLYNOMIALS, ARITHMETIC EXPRESSIONS
3
4
5
6
7
8
Theorems relating to recursive definitions and their proofs.
Formal definition of Regular Expressions, Defining languages with regular
expressions, Languages associated with regular expressions.
Equality of Regular Expressions, Introducing the language EVEN-EVEN.
9
More examples related to regular expressions.
Introducing Finite Automata., Defining languages using Finite Automata.
Constructing Finite Automata for different languages.
Recognizing the language defined by the given Finite Automata.
10
11
More examples related to Finite Automata.
Sessional I
Schedule of Lectures (Cont…)
Lect.#
Topics/Contents
12
Transition Graphs with examples, Generalized Transition Graphs, Nondeterminism in case of Transition Graphs.
13
Non-deterministic FA’s. Differences between FA, TG and NFA.
14
Finite Automata with output, machines and Mealy machines with
examples. 1’s Complement machine, Increment machine.
15
Transducers as models of sequential circuits, Theorems for Converting
Moore machines into Mealy machines and vice versa.
Regular Languages, Closure properties (i.e. , Concatenation and Kleene
closure) of Regular Languages with examples. Complements and
Intersections of Regular Languages, Theorems relating to regular
languages and the related examples.
Decidability, decision procedure, Blue-paint method, Effective decision
procedure to prove whether two given RE’s or FA’s are equivalent.
Non-Regular Languages, The pumping Lemma, Examples relating to
Pumping Lemma.
Myhill-Nerode theorem, Related Examples, Quotient Languages.
16
17
18
19
20
21
22
23
Context-Free Grammars, CFG’s for Regular Languages with examples.
CFG’s for non-regular languages, CFG’s of PALINDROME, EQUAL and
EVEN-EVEN languages, Backus-Naur Form.
Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix
and Postfix notations and their evaluation.
Sessional II
Schedule of Lectures (Cont…)
Lect.#
Topics/Contents
24
Ambiguous and Unambiguous CFG’s, Total language tree. Regular
Grammars, Semi-word, Word, Working String, Converting FA’s into
CFG’s.
25
Regular Grammars, Constructing Transition Graphs from Regular
Grammars. Killing null productions.
26
Killing unit productions, Chomsky Normal form with examples, Left most
derivations.
27
28
29
30
31
32
Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack.
Nondeterministic PDA.
Examples related with PDA, PDA for Odd Palindrome, Even Palindrome,
PalindromeX.
Proving CFG = PDA with examples. Constructing PDA from CFG in CNF
and making trace table for any word.
Context Free Languages, their closure properties, , Concatenation and
Kleene Closures using CFG’s and PDA’s.
Top down parsing, Bottom up parsing with examples, Turing machines,
Examples of Turing Machines with trace tables, Converting FA’s into
Turing machines.
The subprogram Insert, The subprogram Delete, Constructing Turing
machine for the language EQUAL using Insert subprogram.
Some basics
• Automaton = A self-operating machine or
mechanism (Dictionary definition), plural is
Automata.
• Automata = abstract computing devices
• Automata theory = the study of abstract
machines (or more appropriately, abstract
'mathematical' machines or systems, and the
computational problems that can be solved
using these machines.
•
•
•
•
Mathematical models of computation
Finite automata
Push-down automata
Turing machines
History
• 1930s : Alan Turing defined machines more powerful
than any in existence, or even any that we could
imagine – Goal was to establish the boundary
between what was and was not computable.
• 1940s/150s : In an attempt to model “Brain
function” researchers defined finite state
machines.
• Late 1950s : Linguist Noam Chomsky began the
study of Formal Grammars.
• 1960s : A convergence of all this into a formal
theory of computer science, with very deep
philosophical implications as well as practical
applications (compilers, web searching, hardware,
A.I., algorithm design, software engineering,…)
Computation
CPU
Courtesy Costas Busch - RPI
memory
10
temporary memory
input memory
CPU
output memory
Program memory
Courtesy Costas Busch - RPI
11
Example:
f ( x)  x
3
temporary memory
input memory
CPU
output memory
Program memory
compute
xx
compute
2
x x
Courtesy Costas Busch - RPI
12
f ( x)  x
temporary memory
3
input memory
x2
CPU
output memory
Program memory
compute
xx
compute
2
x x
Courtesy Costas Busch - RPI
13
temporary memory
f ( x)  x
3
z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
output memory
Program memory
compute
xx
compute
2
x x
Courtesy Costas Busch - RPI
14
temporary memory
f ( x)  x
3
z  2*2  4
f ( x)  z * 2  8
input memory
x2
CPU
f ( x)  8
Program memory
output memory
compute
xx
compute
2
x x
Courtesy Costas Busch - RPI
15
Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory
Courtesy Costas Busch - RPI
16
Different Kinds of Automata
Automata are distinguished by the temporary memory
• Finite Automata:
no temporary memory
• Pushdown Automata:
stack
• Turing Machines:
random access memory
Courtesy Costas Busch - RPI
17
Finite Automaton
temporary memory
Finite
Automaton
input memory
output memory
Example: Vending Machines
(small computing power)
Courtesy Costas Busch - RPI
18
Pushdown Automaton
Stack
Push, Pop
Pushdown
Automaton
input memory
output memory
Example: Compilers for Programming Languages
(medium computing power)
Courtesy Costas Busch - RPI
19
Turing Machine
Random Access Memory
Turing
Machine
input memory
output memory
Examples: Any Algorithm
(highest computing power)
Courtesy Costas Busch - RPI
20
Power of Automata
Finite
Pushdown
Automata
Turing
Automata
Machine
Less power
More power
Solve more
computational problems
Courtesy Costas Busch - RPI
21
Mathematical Preliminaries
• Sets
• Functions
• Relations
• Graphs
• Proof Techniques
Courtesy Costas Busch - RPI
22
SETS
A set is a collection of elements
A  {1, 2 , 3}
B  {train , bus , bicycle , airplane }
We write
1 A
ship  B
Courtesy Costas Busch - RPI
23
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
finite set
S = { 2, 4, 6, … }
infinite set
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
Courtesy Costas Busch - RPI
24
A = { 1, 2, 3, 4, 5 }
U
A
6
1
7
2
4
5
10
Universal Set:
8
3
9
all possible elements
U = { 1 , … , 10 }
Courtesy Costas Busch - RPI
25
Set Operations
A = { 1, 2, 3 }
B = { 2, 3, 4, 5}
B
A
• Union
A U B = { 1, 2, 3, 4, 5 }
2
1
3
4
5
• Intersection
U
A
B = { 2, 3 }
2
3
• Difference
A-B={1}
1
B - A = { 4, 5 }
Venn diagrams
Courtesy Costas Busch - RPI
26
• Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 }
4
A = { 4, 5, 6, 7}
A
1
5
A
2
6
3
7
A=A
Courtesy Costas Busch - RPI
27
{ even integers } = { odd integers }
Integers
1
odd
2
3
even
0
5
6
4
Courtesy Costas Busch - RPI
7
28
DeMorgan’s Laws
U
A
U
AUB=A
B
B=AUB
Courtesy Costas Busch - RPI
29
Empty, Null Set:
={}
SU
=S
S
=
U
S-
= Universal Set
=S
-S=
Courtesy Costas Busch - RPI
30
Subset
A = { 1, 2, 3}
B = { 1, 2, 3, 4, 5 }
B
U
Proper Subset: A
U
A
B
B
A
Courtesy Costas Busch - RPI
31
Disjoint Sets
A = { 1, 2, 3 }
A
U
A
B = { 5, 6}
B=
B
Courtesy Costas Busch - RPI
32
Set Cardinality
• For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
Courtesy Costas Busch - RPI
33
Powersets
A powerset is a set of sets
S = { a, b, c }
Powerset of S = the set of all the subsets of S
2S = {
, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S|
Courtesy Costas Busch - RPI
( 8 = 23 )
34
Cartesian Product
A = { 2, 4 }
B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
AXBX…XZ
Courtesy Costas Busch - RPI
35
FUNCTIONS
domain
4
A
1
2
5
If A = domain
range
B
f(1) = a
a
b
3
c
f : A -> B
then f is a total function
otherwise f is a partial function
Courtesy Costas Busch - RPI
36
RELATIONS
Let A & B be sets. A binary relation “R” from A to B
R = {(x1, y1), (x2, y2), (x3, y3), …}
Where x i  A and y i  B
R⊆AxB
xi R yi to denote ( a , b )  R
e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
Courtesy Costas Busch - RPI
37
Equivalence Relations
• Reflexive:
xRx
• Symmetric:
xRy
• Transitive:
x R y and y R z
yRx
xRz
Example: R = ‘=‘
•x=x
•x=y
• x = y and y = z
y=x
x=z
Courtesy Costas Busch - RPI
38
Equivalence Classes
Given an equivalence relation R on a set A and an
element x  A , the equivalence class of x is the
set
 x    y  A | ( x, y )  R
Example:
R = { (1, 1), (2, 2), (1, 2), (2, 1),
(3, 3), (4, 4), (3, 4), (4, 3) }
Equivalence class of 1 = {1, 2}
Equivalence class of 3 = {3, 4}
Courtesy Costas Busch - RPI
39
GRAPHS
A directed graph
e
node
b
d
a
c
• Nodes (Vertices)
V = { a, b, c, d, e }
• Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
Courtesy Costas Busch - RPI
40
Labeled Graph
2
6
a
b
1
5
3
e
6
2
d
c
Courtesy Costas Busch - RPI
41
Walk
e
b
d
a
c
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
Courtesy Costas Busch - RPI
42
Path
e
b
d
a
c
Path is a walk where no edge is repeated
Simple path: no node is repeated
Courtesy Costas Busch - RPI
43
Cycle
base
a
3
2
e
b
1
d
c
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
Courtesy Costas Busch - RPI
44
Euler Tour
8
b
4
a
7
3
6
5
base
e
1
2
d
c
A cycle that contains each edge once
Courtesy Costas Busch - RPI
45
Hamiltonian Cycle
5
b
4
a
3
base
e
1
2
d
c
A simple cycle that contains all nodes
Courtesy Costas Busch - RPI
46
Finding All Simple Paths
e
b
d
a
c
origin
Courtesy Costas Busch - RPI
47
Step 1
e
b
d
a
c
(c, a)
origin
(c, e)
Courtesy Costas Busch - RPI
48
Step 2
e
b
d
a
(c, a)
(c, a), (a, b)
c
origin
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Courtesy Costas Busch - RPI
49
Step 3
e
b
d
a
(c, a)
(c, a), (a, b)
c
origin
(c, a), (a, b), (b, e)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Courtesy Costas Busch - RPI
50
Step 4
e
b
(c, a)
d
a
c
(c, a), (a, b)
(c, a), (a, b), (b, e)
origin
(c, a), (a, b), (b, e), (e,d)
(c, e)
(c, e), (e, b)
(c, e), (e, d)
Courtesy Costas Busch - RPI
51
root
Trees
parent
leaf
child
Trees have no cycles
Courtesy Costas Busch - RPI
52
root
Level 0
Level 1
Height 3
leaf
Level 2
Level 3
Courtesy Costas Busch - RPI
53
Binary Trees
Courtesy Costas Busch - RPI
54
PROOF TECHNIQUES
• Proof by induction
• Proof by contradiction
Courtesy Costas Busch - RPI
55
Induction
We have statements P1, P2, P3, …
If we know
• for some b that P1, P2, …, Pb are true
• for any k >= b that
P1, P2, …, Pk imply Pk+1
Then
Every Pi is true
Courtesy Costas Busch - RPI
56
Proof by Induction
• Inductive basis
Find P1, P2, …, Pb which are true
• Inductive hypothesis
Let’s assume P1, P2, …, Pk are true,
for any k >= b
• Inductive step
Show that Pk+1 is true
Courtesy Costas Busch - RPI
57
Example
Theorem: A binary tree of height n
has at most 2n leaves.
Proof by induction:
let L(i) be the maximum number of
leaves of any subtree at height i
Courtesy Costas Busch - RPI
58
We want to show: L(i) <= 2i
• Inductive basis
L(0) = 1
(the root node)
• Inductive hypothesis
Let’s assume L(i) <= 2i for all i = 0, 1, …, k
• Induction step
we need to show that L(k + 1) <= 2k+1
Courtesy Costas Busch - RPI
59
Induction Step
height
k
k+1
From Inductive hypothesis: L(k) <= 2k
Courtesy Costas Busch - RPI
60
Induction Step
height
k
L(k) <= 2k
k+1
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
(we add at most two nodes for every leaf of level k)
Courtesy Costas Busch - RPI
61
Remark
Recursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1
Courtesy Costas Busch - RPI
62
Proof by Contradiction
We want to prove that a statement P is true
• we assume that P is false
• then we arrive at an incorrect conclusion
• therefore, statement P must be true
Courtesy Costas Busch - RPI
63
Example
Theorem:
2 is not rational
Proof:
Assume by contradiction that it is rational
2 = n/m
n and m have no common factors
We will show that this is impossible
Courtesy Costas Busch - RPI
64
2
= n/m
Therefore,
2 m2 = 4k2
n2
2 m 2 = n2
is even
m2 = 2k2
n is even
n=2k
m is even
m=2p
Thus, m and n have common factor 2
Contradiction!
Courtesy Costas Busch - RPI
65