Transcript No Slide Title

Theory of Computation
計算理論
1
Instructor: 顏嗣鈞
E-mail: [email protected]
Web: http://www.ee.ntu.edu.tw/~yen
Time: 9:10-12:10 PM, Monday
Place: BL 103
Office hours: by appointment
Class web page:
http://www.ee.ntu.edu.tw/~yen/courses/TOC-2009.html
2
TEXTBOOK
Introduction to
Automata Theory,
Languages, and
Computation 3rd Edition
John E. Hopcroft,
Rajeev Motwani,
Jeffrey D. Ullman,
(Addison-Wesley, 2006)
3
2nd Edition
Introduction to Automata
Theory, Languages, and
Computation
John E. Hopcroft,
Rajeev Motwani,
Jeffrey D. Ullman,
(2nd Ed. Addison-Wesley, 2001)
4
1st Edition
Introduction to
Automata Theory,
Languages, and
Computation
John E. Hopcroft,
Jeffrey D. Ullman,
(Addison-Wesley, 1979)
5
Grading
HW/Project : 20%
Midterm exam.: 40%
Final exam.: 40%
6
A simple computer
BATTERY
input: switch
output: light bulb
actions: flip switch
states: on, off
7
A simple “computer”
f
BATTERY
start
on
off
f
input: switch
output: light bulb
actions: f for “flip switch”
bulb is on if and only if
there was an odd number
of flips
states: on, off
8
Another “computer” 1
1
start
off
2
BATTERY
off
1
2
2
2
1
2
off
1
on
inputs: switches 1 and 2
actions: 1 for “flip switch 1”
actions: 2 for “flip switch 2”
states: on, off
bulb is on if and only if
both switches were flipped
an odd number of times
9
A design
problem
4
1
BATTERY
2
3
?
5
Can you design a circuit where the light is on if and only
if all the switches were flipped exactly the same number
of times?
10
A design problem
• Such devices are difficult to reason about,
because they can be designed in an infinite
number of ways
• By representing them as abstract
computational devices, or automata, we will
learn how to answer such questions
11
These devices can model many
things
• They can describe the operation of any
“small computer”, like the control
component of an alarm clock or a
microwave
• They are also used in lexical analyzers to
recognize well formed expressions in
programming languages:
ab1 is a legal name of a variable in C
5u= is not
12
Different kinds of automata
• This was only one example of a
computational device, and there are others
• We will look at different devices, and look
at the following questions:
– What can a given type of device compute, and
what are its limitations?
– Is one type of device more powerful than
another?
13
Some devices we will see
finite automata
Devices with a finite amount of memory.
Used to model “small” computers.
push-down
automata
Devices with infinite memory that can be
accessed in a restricted way.
Used to model parsers, etc.
Turing Machines Devices with infinite memory.
Used to model any computer.
time-bounded
Infinite memory, but bounded running
Turing Machines time.
Used to model any computer program
that runs in a “reasonable” amount of
14
time.
Why Study Automata Theory?
Finite automata are a useful model for
important kinds of hardware and software:
• Software for designing and checking digital
circuits.
• Lexical analyzer of compilers.
• Finding words and patterns in large bodies of
text, e.g. in web pages.
• Verification of systems with finite number of
states, e.g. communication protocols.
15
Why Study Automata Theory? (2)
The study of Finite Automata and Formal
Languages are intimately connected.
Methods for specifying formal languages
are very important in many areas of CS,
e.g.:
• Context Free Grammars are very useful when
designing software that processes data with
recursive structure, like the parser in a compiler.
• Regular Expressions are very useful for
specifying lexical aspects of programming
languages and search patterns.
16
Why Study Automata Theory? (3)
Automata are essential for the study of
the limits of computation. Two issues:
• What can a computer do at all?
(Decidability)
• What can a computer do efficiently?
(Intractability)
17
Applications
Compiler
Prog. languages
Comm. protocols
circuits
Pattern
recognition
Supervisory
control
Quantum
computing
Computer-Aided
Verification
...
Theoretical Computer Science
Automata Theory, Formal Languages,
Computability, Complexity …
18
Aims of the Course
•
To familiarize you with key Computer Science
concepts in central areas like
- Automata Theory
- Formal Languages
- Models of Computation
- Complexity Theory
•
To equip you with tools with wide applicability
in the fields of CS and EE, e.g. for
- Complier Construction
- Text Processing
- XML
19
Fundamental Theme
• What are the capabilities and
limitations of computers and computer
programs?
– What can we do with
computers/programs?
– Are there things we cannot do with
computers/programs?
20
Studying the Theme
• How do we prove something CAN be done
by SOME program?
• How do we prove something CANNOT be
done by ANY program?
21
Example: The Halting Problem (1)
Consider the following program. Does it
terminate for all values of n  1?
while (n > 1) {
if even(n) {
n = n / 2;
} else {
n = n * 3 + 1;
}
}
22
Example: The Halting Problem (2)
Not as easy to answer as it might first seem.
Say we start with n = 7, for example:
7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5,
16, 8, 4, 2, 1
In fact, for all numbers that have been tried
(a lot!), it does terminate . . .
. . . but in general?
23
Example: The Halting Problem (3)
Then the following important undecidability
result should perhaps not come as a total
surprise:
It is impossible to write a program that
decides if another, arbitrary, program
terminates (halts) or not.
What might be surprising is that it is possible
to prove such a result. This was first done
by the British mathematician Alan Turing.
24
Our focus
Automata
Computability
Complexity
25
Topics
1.
Finite automata, Regular languages, Regular
grammars: deterministic vs. nondeterministic,
one-way vs. two-way finite automata,
minimization, pumping lemma for regular
sets, closure properties.
2. Pushdown automata, Context-free languages,
Context-free grammars: deterministic vs.
nondeterministic, one-way vs. two-way PDAs,
reversal bounded PDAs, linear grammars, counter
machines, pumping lemma for CFLs, Chomsky
normal form, Greibach normal form, closure
26
properties.
Topics (cont’d)
3. Linear
bounded automata, Contextsensitive languages, Context-sensitive
grammars.
4. Turing machines, Recursively
enumerable sets, Type 0 grammars:
variants of Turing machines, halting
problem, undecidability, Post
correspondence problem, valid and
invalid computations of TMs.
27
Topics (cont’d)
5. Basic recursive function theory
6. Basic complexity theory: Various
resource bounded complexity classes,
including NLOGSPACE, P, NP,
PSPACE, EXPTIME, and many more.
reducibility, completeness.
7. Advanced topics: Tree Automata,
quantum automata, probabilistic
automata, interactive proof systems,
oracle computations, cryptography.
28
Who should take this course?
YOU
29
Languages
The terms language and word are used in
a strict technical sense in this course:
A language is a set of words.
A word is a sequence (or string) of
symbols.
 (or ) denotes the empty word, the
sequence of zero symbols.
30
Symbols and Alphabets
• What is a symbol, then?
• Anything, but it has to come from an
alphabet which is a finite set.
• A common (and important) instance is
= {0, 1}.
•  , the empty word, is never an symbol
of an alphabet.
31
Computation
CPU
memory
32
temporary memory
input memory
CPU
output memory
Program memory
33
Example:
f ( x)  x
3
temporary memory
input memory
CPU
output memory
Program memory
compute
xx
compute
x x
2
34
f ( x)  x
temporary memory
3
input memory
x2
CPU
output memory
Program memory
compute
xx
compute
x x
2
35
temporary memory
z  2*2  4
f ( x)  z * 2  8
f ( x)  x
3
input memory
x2
CPU
output memory
Program memory
compute
xx
compute
x x
2
36
temporary memory
z  2*2  4
f ( x)  z * 2  8
f ( x)  x
3
input memory
x2
CPU
Program memory
compute
xx
compute
x x
f ( x)  8
output memory
2
37
Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory
38
Different Kinds of Automata
Automata are distinguished by the temporary memory
• Finite Automata:
no temporary memory
• Pushdown Automata:
stack
• Turing Machines:
random access memory
39
Finite Automaton
temporary memory
Finite
Automaton
input memory
output memory
Example: Vending Machines
(small computing power)
40
Pushdown Automaton
Stack
Pushdown
Automaton
Push, Pop
input memory
output memory
Example: Compilers for Programming Languages
(medium computing power)
41
Turing Machine
Random Access Memory
Turing
Machine
input memory
output memory
Examples: Any Algorithm
(highest computing power)
42
Power of Automata
Finite
Pushdown
Automata
Turing
Automata
Machine
Less power
More power
Solve more
computational problems
43