Transcript Chapter 11
CS 3240 – Chapter 11
They may not halt on every possible input!
And not just because the creator of a specific TM was a
doofus
This is related to the major
mathematical/computational discovery of the 20th
century!
There are propositions that cannot be decided (“proven”)
A question is decidable if there is a TM that always
halts and answers “yes” or “no” for each possible
input
The TM therefore constitutes an algorithm
A language is decidable if there is a TM that always
halts and answers “accept” or “reject” whenever an
input string is in the language or not
aka “Turing acceptable”
A function is computable if there is a TM that
always halts with the appropriate output for
each possible input in the function domain
aka “Total function”
Let g(x,y) be some computable function
Let f(x) =
the smallest p where g(x, p) = 1, or
0, if such a p does not exist
“Pseudo-algorithm” for f(x):
m = 0;
while (g(x, m) != 1)
++m;
cout << m;
If there is no m for a given g and x, then we hang!
3 possibilities in general when a TM processes
an input string:
Accepts (goes to an accepting halt state)
Rejects (e.g., crashes, or gives a “no” answer)
Hangs (infinite loop)
A language for which there is a TM that
always halts for and accepts strings in the
language is recognizable
“It knows one when it sees one” :-)
But it may hang on strings not in the language
A function that is defined for only some of its
domain elements is a partial function
It may hang (or “blow up”) on some inputs
e.g., divide by zero
This is the computational analogue to a
recognizable language
you’ll always get an answer with a valid input
it may hang on invalid input
Language
Machine
Grammar
Regular
Finite Automaton
Regular Expression,
Regular Grammar
Context-Free
Pushdown Automaton
Context-Free
Grammar
Recursively
Enumerable
Turing Machine
Unrestricted PhraseStructure Grammar
CS 3240 - Introduction
9
A language L over the alphabet is called recursively
enumerable (aka “recognizable”) if there is a TM T that accepts
every word in L and either rejects (crashes) or loops forever for
every word in the language L', the complement of L:
accept(T) = L
reject(T) + loop(T) = L'
recursively enumerable (r.e.) = recognizable
We just saw one (slide 5)
Let g(x,y) be some computable function
Let f(x) =
the smallest p where g(x, p) = 1, or
0, if such a p does not exist
Let Lg be the set of functions, f, corresponding to
all computable functions g, as explained above
Such functions can be encoded as strings, and are
therefore countable
Lg can then be seen as a language (a set of strings)
A language L over the alphabet is called recursive (aka
“decidable”) if there is a TM T that accepts every word in L and
rejects every word in L'; that is,
accept(T) = L
reject(T) = L'
A TM that accepts a recursive set is a
model of an algorithm. It always halts.
recursive = decidable
There exist languages that are r.e. but not recursive
We just saw one
There exist languages that aren't even r.e.!
(You'll see one soon)
All are “contrived”
Languages generated by grammars are r.e. or
“better”
All r.e. languages are closed under union,
intersection, concatenation, and Kleene*
Everything but complement!
Recursive languages are also closed under
complement
Also: If L and L' are r.e., then L is recursive
Here come the Proofs…
Let M be a machine that decides a recursive
language, L
Form the machine M' by inverting the
acceptability output of M
Goes to a reject state instead
Then M' decides L'
So L' is recursive
Suppose L and L' are both r.e.
Let M recognize L, and M' recognize L'
M may hang on elements of L', but M' doesn't
Form a new machine, M* that calls M and M'
in parallel (non-deterministically)
If M accepts w, so does M*
If M' accepts w, reject w
There are no other possibilities! (No hanging)
Therefore, L is decidable/recursive, by definition
TMs can recognize/accept strings from certain
languages and/or compute functions
If there is a TM, M, that accepts a language, L, and M
always halts, then L is recursive
If there is no such M for L, but there is instead a
machine M that accepts every string in L, but M may
hang on strings not in L, then L is recursively
enumerable
The complement of a recursive language is
recursive
In fact, recursive languages are closed under all
operations, like regular languages are
r.e. languages are closed under intersection
The complement of a r.e. language may not
be r.e.
But if it is, then both languages are actually
recursive!
A Timely Interlude
The real numbers in (0,1) are uncountable
They cannot be mapped in a 1-to-1 fashion to the
counting numbers
Proof:
Assume they can be: r0, r1, r2, …
Arrange their digits in a table a[ ][ ]
▪ each row, a[i], contains the digits or ri
The diagonal sequence (a[n][n]+1) mod 10,
representing a valid real number, is not in the
table! Contradiction!
The power set of the natural numbers, N
the set of all subsets (2N)
Suppose it is countable
The we can enumerate the sets: p0, p1, p2, …
Now consider the set T = {i | i ∉ f(i) = pi }
Certainly T is a set of integers, so T ∈ P(N)
▪ Call it pk
Question: Is k in T = pk?
The power set of any countably infinite set is
uncountable
Much bigger than a countable set!
The number of countable sets is negligible
compared to the number of uncountable sets
Well?
A language over an alphabet Σ is a subset of
Σ*
the latter being an infinite set
The set of all languages over Σ is therefore the
power set of Σ* (2∑*)
which we just showed is uncountable
So… the number of languages over any finite
alphabet is uncountable
The # of TMs are countable, but the # of
languages is not
Therefore, some languages cannot be
recognized by a TM
There aren't enough TMs to go around!
Just like there are more reals than integers
So, non-r.e. languages must exist!
Just take the complement of any r.e. language
that is not recursive
Example (page 279):
Consider all TMs, Mi with alphabet Σ={a}.
Let X = {ai: ai ∈ L(Mi)}
▪ This is r.e., but not recursive
▪ Because we can construct a TM that carries out the
computation, but it may not halt when ai ∉ L(Mi)
Then X’ must be non-r.e.!
Suppose X’ is r.e.
Then there is a TM, Mk, that recognizes it
Now ask the question, is ak ∈ X’ ?
If it is, this means ak ∉ L(Mk) = X’, by definition
Contradiction!
If it isn’t, then this means that ak ∈ L(Mk) = X’
Again, by definition
Again, a contradiction!
⇒ X’ is not r.e.
What we have just described is the
membership problem:
“Given a r.e. language, L, and a string, w, is w in
L?”
We have just shown that the membership
problem is, in general, undecidable
Language
Machine
Grammar
Regular
Finite Automaton
Regular Expression,
Regular Grammar
Context-Free
Pushdown Automaton
Context-Free
Grammar
Recursively Enumerable Turing Machine
CS 3240 - Introduction
Unrestricted
Grammar
29
Left-hand side of the rule is a concatenation
of one or more symbols
There must be at least one variable on the left
▪ λ is not allowed on the left
Any string is allowed on the right, including λ
aka “Type 0” Grammar
S → aAbc | abc | λ
Ab → bA
Ac → Bbcc
bB → Bb
aB → aa | aaA
S ⇒aAbc
⇒abAc
⇒abBbcc
⇒aBbbcc
⇒aabbcc
An A is created as the left-most variable. It travels to the right until it finds a
'c', then creates a new b and c, and becomes a B. The B moves back to create
the extra needed a. The last rule allows the option to do it all over again (by
introducing another A). This is similar to what the Turing machine for the
same language does, except the TM marks instead of generates the letters.
We have already seen
CFGs for this
This unrestricted
grammar:
Introduces X’s and Y’s
together
Allows moving X’s and
Y’s arbitrarily by
swapping them
S → XYS | λ
XY →YX
YX → XY
X→a
Y→b
Cannot use the decidability algorithms for
CFGs
e.g., CYK algorithm does not apply
No “Normal Form”
Non-null productions may create shorter
strings
Terminals can disappear!
For every r.e. language, there is an
unrestricted grammar that generates it
For every unrestricted grammar, there is a TM
that recognizes its language
It may or may not decide it
We will not prove this
but the book does
Like Unrestricted Grammars except:
Right-hand side must be no shorter than left-hand side
▪ So strings never shrink
Hence the name “non-contracting” or “monotonic”
Cannot contain λ anywhere
Context-sensitive languages don't need unlimited
memory
Since intermediate sentential forms never shrink, the
largest memory requirement is proportional to |w|
Accepting machine: Linear Bounded Automaton
S → aAbc | abc
Ab → bA
Ac → Bbcc
bB → Bb
aB → aa | aaA
Every context-sensitive language is recursive
there is a TM that accepts it (i.e., always halts)
But not all recursive languages are contextsensitive
So Context Sensitive languages form a proper
subset of Recursive Languages
Just like Recursive languages are a proper subset of
Recursively enumerable languages
Type 0 grammar
Type 1 grammar
Type 2 grammar
Type 3 grammar
Language
Grammar
Machine
Recursively Enumerable
(“Turing Recognizable”)
Unrestricted Phrase
Structure (Type 0)
TM (that may not halt)
Recursive
(“Turing Decidable”)
"
TM (that always halts)
Context-Sensitive
Context Sensitive (Type 1)
(monotonic/noncontracting)
Linear-bounded automata
(bounded TM)
Context Free
Context Free (Type 2)
PDA
Deterministic Context Free
Regular
"
Regular (Type 3)
(Right-linear or Left-linear)
DPDA
FA