Transcript Other models of computation

Other Models of
Computation
Prof. Busch - LSU
1
Models of computation:
•Turing Machines
•Recursive Functions
•Post Systems
•Rewriting Systems
Prof. Busch - LSU
2
Church’s Thesis:
All models of computation are equivalent
Turing’s Thesis:
A computation is mechanical if and only if
it can be performed by a Turing Machine
Prof. Busch - LSU
3
Church’s and Turing’s Thesis are similar:
Church-Turing Thesis
Prof. Busch - LSU
4
Recursive Functions
An example function:
Domain
3
2
f ( n)  n  1
f (3)  10
Prof. Busch - LSU
Range
10
5
We need a way to define functions
We need a set of basic functions
Prof. Busch - LSU
6
Basic Primitive Recursive Functions
Zero function:
z ( x)  0
Successor function: s( x)  x  1
Projection functions: p1( x1, x2 )  x1
p2 ( x1, x2 )  x2
Prof. Busch - LSU
7
Building complicated functions:
Composition:
f ( x, y )  h( g1( x, y ), g 2 ( x, y ))
Primitive Recursion:
f ( x,0)  g1( x)
f ( x, y  1)  h( g 2 ( x, y ), f ( x, y ))
Prof. Busch - LSU
8
Any function built from
the basic primitive recursive functions
is called:
Primitive Recursive Function
Prof. Busch - LSU
9
A Primitive Recursive Function:
add ( x,0)  x
add ( x, y )
(projection)
add ( x, y  1)  s(add ( x, y ))
(successor function)
Prof. Busch - LSU
10
add (3,2)  s (add (3,1))
 s ( s (add (3,0)))
 s ( s (3))
 s ( 4)
5
Prof. Busch - LSU
11
Another Primitive Recursive Function:
mult ( x, y )
mult ( x,0)  0
mult ( x, y  1)  add ( x, mult ( x, y))
Prof. Busch - LSU
12
Theorem:
The set of primitive recursive functions
is countable
Proof:
Each primitive recursive function
can be encoded as a string
Enumerate all strings in proper order
Check if a string is a function
Prof. Busch - LSU
13
Theorem
there is a function that
is not primitive recursive
Proof:
Enumerate the primitive recursive functions
f1, f 2 , f3 , 
Prof. Busch - LSU
14
Define function
g (i )  fi (i )  1
g differs from every fi
g
is not primitive recursive
END OFProf.
PROOF
Busch - LSU
15
A specific function that is not
Primitive Recursive:
Ackermann’s function:
A(0, y )  y  1
A( x,0)  A( x  1,1)
A( x, y  1)  A( x  1, A( x, y ))
Grows very fast,
faster than any primitive recursive function
Prof. Busch - LSU
16
  Recursive Functions
y( g ( x, y))  smallest y such that g ( x, y)  0
Accerman’s function is a
  Recursive Function
Prof. Busch - LSU
17
  Recursive Functions
Primitive recursive functions
Prof. Busch - LSU
18
Post Systems
• Have Axioms
• Have Productions
Very similar with unrestricted grammars
Prof. Busch - LSU
19
Example: Unary Addition
Axiom:
1  1  11
Productions:
V1  V2  V3  V11  V2  V31
V1  V2  V3  V1  V21  V31
Prof. Busch - LSU
20
A production:
V1  V2  V3  V11  V2  V31
1  1  11  11  1  111  11  11  1111
V1  V2  V3  V1  V21  V31
Prof. Busch - LSU
21
Post systems are good for
proving mathematical statements
from a set of Axioms
Prof. Busch - LSU
22
Theorem:
A language is recursively enumerable
if and only if
a Post system generates it
Prof. Busch - LSU
23
Rewriting Systems
They convert one string to another
• Matrix Grammars
• Markov Algorithms
• Lindenmayer-Systems
Very similar to unrestricted grammars
Prof. Busch - LSU
24
Matrix Grammars
Example:
P1 : S  S1S2
P2 : S1  aS1, S2  bS2c
P3 : S1   , S2  
Derivation:
S  S1S2  aS1bS2c  aaS1bbS2cc  aabbcc
A set of productions is applied simultaneously
Prof. Busch - LSU
25
P1 : S  S1S2
P2 : S1  aS1, S2  bS2c
P3 : S1   , S2  
n n n
L  {a b c : n  0}
Theorem:
A language is recursively enumerable
if and only if
a Matrix grammar
generates
it
Prof. Busch
- LSU
26
Markov Algorithms
Grammars that produce 
Example:
ab  S
aSb  S
S  .
Derivation:
aaabbb  aaSbb  aSb  S  
Prof. Busch - LSU
27
ab  S
aSb  S
S  .
n n
L  {a b : n  0}
Prof. Busch - LSU
28
In general:
*
L  {w : w  }
Theorem:
A language is recursively enumerable
if and only if
a Markov algorithm generates it
Prof. Busch - LSU
29
Lindenmayer-Systems
They are parallel rewriting systems
Example:
a  aa
Derivation:
a  aa  aaaa  aaaaaaaa
L  {a
2
n
: n  0}
Prof. Busch - LSU
30
Lindenmayer-Systems are not general
As recursively enumerable languages
Extended Lindenmayer-Systems: ( x, a, y )  u
context
Theorem:
A language is recursively enumerable
if and only if an
Extended Lindenmayer-System generates it
Prof. Busch - LSU
31
Computational Complexity
Prof. Busch - LSU
32
Time Complexity:
The number of steps
during a computation
Space Complexity:
Space used
during a computation
Prof. Busch - LSU
33
Time Complexity
•We use a multitape Turing machine
•We count the number of steps until
a string is accepted
•We use the O(k) notation
Prof. Busch - LSU
34
Example:
n n
L  {a b : n  0}
Algorithm to accept a string w :
•Use a two-tape Turing machine
•Copy the a on the second tape
•Compare the a and b
Prof. Busch - LSU
35
n n
L  {a b : n  0}
Time needed:
•Copy the a on the second tape
O (| w |)
•Compare the a and b
O (| w |)
Total time:
Prof. Busch - LSU
O (| w |)
36
n n
L  {a b : n  0}
For string of length
n
time needed for acceptance: O(n)
Prof. Busch - LSU
37
Language class:
DTIME(n)
DTIME(n)
L1
L2
L3
A Deterministic Turing Machine
accepts each string of length n
in time O(n)
Prof. Busch - LSU
38
DTIME(n)
n n
{a b : n  0}
{ww}
Prof. Busch - LSU
39
In a similar way we define the class
DTIME(T (n))
for any time function:
T (n)
Examples:
2
3
DTIME(n ), DTIME(n ),...
Prof. Busch - LSU
40
Example: The membership problem
for context free languages
L  {w : w is generated by grammar G}
3
L  DTIME(n )
(CYK - algorithm)
Polynomial time
Prof. Busch - LSU
41
Theorem:
DTIME(n
k 1
k
)  DTIME(n )
DTIME(n
k 1
)
k
DTIME(n )
Prof. Busch - LSU
42
Polynomial time algorithms:
k
DTIME(n )
Represent tractable algorithms:
For small k we can compute the
result fast
Prof. Busch - LSU
43
The class
P
k
P   DTIME(n )
for all k
•Polynomial time
•All tractable problems
Prof. Busch - LSU
44
P
CYK-algorithm

n n
{a b }
{ww}
Prof. Busch - LSU
45
Exponential time algorithms:
n
DTIME(2 )
Represent intractable algorithms:
Some problem instances
may take centuries to solve
Prof. Busch - LSU
46
Example: the Traveling Salesperson Problem
5
4
3
1
2
2
6
3
8
10
Question: what is the shortest route that
connects
Prof. all
Busch -cities?
LSU
47
5
4
3
1
2
2
6
3
8
10
Question: what is the shortest route that
connects
Prof. all
Busch -cities?
LSU
48
A solution: search exhuastively all
hamiltonian paths
L = {shortest hamiltonian paths}
n
L  DTIME(n!)  DTIME(2 )
Exponential time
Intractable problem
Prof. Busch - LSU
49
Example: The Satisfiability Problem
Boolean expressions in
Conjunctive Normal Form:
t1  t2  t3    tk
ti  x1  x2  x3    x p
Variables
Question: is expression satisfiable?
Prof. Busch - LSU
50
Example:
Satisfiable:
( x1  x2 )  ( x1  x3 )
x1  0, x2  1, x3  1
( x1  x2 )  ( x1  x3 )  1
Prof. Busch - LSU
51
Example:
( x1  x2 )  x1  x2
Not satisfiable
Prof. Busch - LSU
52
L  {w : expression w is satisfiabl e}
For n variables:
n
L  DTIME(2 )
exponential
Algorithm:
search exhaustively all the possible
binary values of the variables
Prof. Busch - LSU
53
Non-Determinism
Language class:
NTIME(n)
NTIME(n)
L1
L2
L3
A Non-Deterministic Turing Machine
accepts each string of length n
in time O(n)
Prof. Busch - LSU
54
Example: L  {ww}
Non-Deterministic Algorithm
to accept a string ww :
•Use a two-tape Turing machine
•Guess the middle of the string
and copy w on the second tape
•Compare the two tapes
Prof. Busch - LSU
55
L  {ww}
Time needed:
•Use a two-tape Turing machine
•Guess the middle of the string
and copy w on the second tape
•Compare the two tapes
Total time:
Prof. Busch - LSU
O (| w |)
O (| w |)
O (| w |)
56
NTIME(n)
L  {ww}
Prof. Busch - LSU
57
In a similar way we define the class
NTIME(T (n))
for any time function:
Examples:
T (n)
2
3
NTIME(n ), NTIME(n ),...
Prof. Busch - LSU
58
Non-Deterministic Polynomial time algorithms:
k
L  NTIME(n )
Prof. Busch - LSU
59
The class
NP
k
P   NTIME(n )
for all k
Non-Deterministic Polynomial time
Prof. Busch - LSU
60
Example:
The satisfiability problem
L  {w : expression w is satisfiabl e}
Non-Deterministic algorithm:
•Guess an assignment of the variables
•Check if this is a satisfying assignment
Prof. Busch - LSU
61
L  {w : expression w is satisfiabl e}
Time for n variables:
•Guess an assignment of the variables O(n)
•Check if this is a satisfying assignment O(n)
Total time:
O(n)
Prof. Busch - LSU
62
L  {w : expression w is satisfiabl e}
L  NP
The satisfiability problem is an NP - Problem
Prof. Busch - LSU
63
Observation:
P  NP
Deterministic
Polynomial
Non-Deterministic
Polynomial
Prof. Busch - LSU
64
Open Problem:
P  NP ?
WE DO NOT KNOW THE ANSWER
Prof. Busch - LSU
65
Open Problem:
P  NP ?
Example: Does the Satisfiability problem
have a polynomial time
deterministic algorithm?
WE DO NOT KNOW THE ANSWER
Prof. Busch - LSU
66
NP-Completeness
A problem is NP-complete if:
•It is in NP
•Every NP problem is reduced to it
(in polynomial time)
Prof. Busch - LSU
67
Observation:
If we can solve any NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
P  NP
Prof. Busch - LSU
68
Observation:
If we prove that
we cannot solve an NP-complete problem
in Deterministic Polynomial Time (P time)
then we know:
P  NP
Prof. Busch - LSU
69
Cook’s Theorem:
The satisfiability problem is NP-complete
Proof:
Convert a Non-Deterministic Turing Machine
to a Boolean expression
in conjunctive normal form
Prof. Busch - LSU
70
Other NP-Complete Problems:
•The Traveling Salesperson Problem
•Vertex cover
•Hamiltonian Path
All the above are reduced
to the satisfiability problem
Prof. Busch - LSU
71
Observations:
It is unlikely that NP-complete
problems are in P
The NP-complete problems have
exponential time algorithms
Approximations of these problems
are in P
Prof. Busch - LSU
72