Transcript Document

snick

snack
CPSC 121: Models of Computation
2013W2
Introduction to Induction
Steve Wolfman
1
Outline
•
•
•
•
Prereqs, Learning Goals, and Quiz Notes
Bonus Prelude
A Pattern for Induction
Problems and Discussion
– Single-Elimination Tournaments
– Induction on Numbers
• Next Lecture Notes
2
Learning Goals: Pre-Class
By the start of class, you should be able to:
– Convert sequences to and from explicit formulas
that describe the sequence.
– Convert sums to and from summation/“sigma”
notation.
– Convert products to and from product/“pi”
notation.
– Manipulate formulas in summation/product
notation by adjusting their bounds, merging or
splitting summations/products, and factoring out
values.
5
Learning Goals: In-Class
By the end of this unit, you should be able
to:
– Given a theorem to prove via induction and
the insight into how the problem breaks down
into one or more smaller problem(s), write out
a complete proof strategy.
– Prove naturally self-referential properties by
induction. (That is, identify the “insight” and
flesh out the strategy into a complete proof.)
6
Where We Are in
The Big Stories
Theory
How do we model
computational systems?
Now: Developing a new
proof technique that’s
perfect for algorithms
involving recursion (or
iteration)... which is
almost all interesting
algorithms!
Hardware
How do we build devices to
compute?
Now: Taking a break in
lecture. In lab, nearly at
complete, working
computer! (Although
lab’s taking a break
too, for regular
expressions.)
7
Outline
•
•
•
•
Prereqs, Learning Goals, and Quiz Notes
Bonus Prelude
A Pattern for Induction
Problems and Discussion
– Single-Elimination Tournaments
– Induction on Numbers
• Next Lecture Notes
8
Prelude: A Binary Tree with n
Nodes contains n+1 Empty Trees
Note: We’ll define a binary tree as one of an
empty tree or a “node” with two subtrees
(each itself a binary tree).
(What a “node” is depends on what we want to do with the
tree but doesn’t concern us here!)
Translate the definition and the theorem to
predicate logic; use the proofs sheet to think
about how to prove it.
9
Induction: Regular Old Proof
Plus One Powerful Step
A binary tree is: an empty tree or a node
with two subtrees (each itself a binary tree).
Theorem: A binary tree with n nodes
contains n+1 empty trees.
Induction gives us one extra 10
tool:
assuming our theorem holds for recursive sub-cases.
“Binary Trees”
A “binary tree” is either
an empty tree or a
node with two
children, each of
which is a binary tree.
10
5
2
20
9
7
15
17
11
(They let us do some cool CS things... see CS110, 210, 221.)
New Proof Technique:
(Structural) Induction
1. Identify the key recursive structure in the
problem.
For example, binary trees.
A binary tree is one of:
• An empty tree
• A node with two subtrees, each of which is
a binary tree
The book discusses “weak” and “strong” induction.
12
As far as we are concerned, there can be only one induction: structural.
New Proof Technique:
(Structural) Induction
2. Circle each recursive appearance of the
structure inside its definition.
A binary tree is one of:
• An empty tree
• A node with two subtrees, each of which
is a binary tree
13
So, the two subtrees are recursive appearances of binary trees.
New Proof Technique:
(Structural) Induction
3. Divide the cases into: those without
recursive appearances (“base cases”)
and those with (“recursive” or “inductive”
cases)
A binary tree is one of:
• An empty tree Base case
• A node with two subtrees, each of which
Inductive case
is a binary tree
14
Reuse all of these steps whenever you reuse the recursive structure.
New Proof Technique:
(Structural) Induction
4. Write a predicate describing what you
want to say about the structure.
For example:
P(t) ≡ the number of empty trees in the
binary tree t is one more than the number
of nodes in t
15
Always in terms of one of the recursive structure.
New Proof Technique:
(Structural) Induction
16
This step is easy! 
New Proof Technique:
(Structural) Induction
6. Write out your proof template:
P(a) ≡ [fill in predicate definition]
Theorem: For all [recursive structure] a, P(a) holds.
Proof by structural induction:
Base case: [For each base case you ID’d, write the
structure of that case and prove the theorem
holds for that case. If you don’t END by showing
your theorem true for the base case structure,
17
you probably messed up!]
Example
P(t) ≡ the number of empty trees in the binary tree
t is one more than the number of nodes in t
Theorem: For all binary trees t, P(t) holds.
Proof by structural induction:
Base case: t is an empty tree.
In that case, t has no nodes and one empty tree.
1 = 0 + 1 as expected. 
18
New Proof Technique:
(Structural) Induction
6. Proof template continued:
Inductive Step: [For each recursive case write out
an inductive step…]
Consider an arbitrary [recursive case structure] a.
Induction Hypothesis: Assume the theorem holds
for [each recursive appearance of the structure in
this case].
We now show that the theorem holds for a itself.
19
Example
Inductive Step:
Consider an arbitrary non-empty binary tree t (i.e., a
node with two subtrees, each of which is a binary
tree).
Induction Hypothesis: Assume the theorem holds
for the two subtrees of t.
We now show that the theorem holds for t itself.
20
New Proof Technique:
(Structural) Induction
6. Proof template completed:
[Show that theorem holds! Three signs that something has probably
gone wrong:
1) You don’t END by showing that your theorem holds for a.
2) You don’t USE the “Induction Hypothesis” assumption(s) you
made.
3) You EVER take a recursive appearance and use the recursive
definition to break it down further (Example BAD thing to say:
“Each of the subtrees is also a binary tree; so, it can have subtrees
and so on until we reach an empty tree.)]
21
Example
Consider an arbitrary binary tree t.
Induction Hypothesis: Assume the theorem holds for the
two subtrees of t.
t’s left subtree tl has some number of nodes nl and some
number of empty trees el. By assumption, el = nl + 1.
Similarly for t’s right subtree tr, er = nr + 1.
t’s number of nodes n includes all of tl’s and tr’s nodes plus
its own node. t’s number of empty trees e includes all and
only the empty trees in tl and tr.
Thus n = nl + nr + 1 and e = el + er. Substituting from above:
e = el + er = n/ + 1 + nr + 1= nl + nr + 1 + 1 = n + 1.
Thus, e = n + 1, as we wanted to prove.
22
QED
Outline
•
•
•
•
Prereqs, Learning Goals, and Quiz Notes
Bonus Prelude
A Pattern for Induction
Problems and Discussion
– Single-Elimination Tournaments
– Induction on Numbers
• Next Lecture Notes
23
A Pattern for Induction
1. Identify the recursive structure in the problem.
2. Circle each recursive appearance of the
structure inside its definition.
3. Divide the cases into: those without recursive
appearances (“base cases”) and those with
(“recursive” or “inductive” cases)
4. Write a predicate P(a) describing what you
want to say about the structure.
5. Write your theorem a?,P(a)
24
6. Complete the proof template.
P(a) ≡ [fill in predicate definition]
Theorem: For all [recursive structure] a, P(a) holds.
Proof by structural induction:
Base case: [For each non-recursive case, prove the theorem holds for that
case. End by showing your theorem true for the base case structure!]
Inductive Step: [For each recursive case…]
Consider an arbitrary [recursive case structure] a.
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for a. [And then do so! You should:
1) END by showing that your theorem holds for a.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
It helps to write out what you want to prove rather than just “show the theorem holds for a”. 25
(Even though neither one is strictly necessary.)
Outline
•
•
•
•
Prereqs, Learning Goals, and Quiz Notes
Bonus Prelude
A Pattern for Induction
Problems and Discussion
– Single-Elimination Tournaments
– Induction on Numbers
• Next Lecture Notes
26
Single-Elimination Tournaments
In each round teams play in pairs. Losing
teams are eliminated. The tournament
ends when only one team remains.
27
What is a
Tournament?
Let’s think like CPSC 110ers…
A tournament is one of:
28
The “insight into how the
problem breaks down”
A tournament of n rounds is made up of two
tournaments of (n-1) rounds, like how the
Stanley Cup Finals is set “on top of” the
“east half” and “west half” tournaments.
One extra round:
east winner vs. west winner
29
Single-Elimination
Tournament Problem
What’s the maximum number of teams in a
0-round single-elimination tournament?
a. 0 teams
b. 1 team
c. 2 teams
d. 3 teams
e. None of these
30
Think: what does this correspond to in the data definition?
Single-Elimination
Tournament Problem
What’s the maximum number of teams in a
1-round single-elimination tournament?
a. 0 teams
b. 1 team
c. 2 teams
d. 3 teams
e. None of these
A “one round tournament” has only one set of games.
31
A “two round tournament” has two sets of games (finals and semi-finals).
Single-Elimination
Tournament Problem
What’s the maximum number of teams in a
2-round single-elimination tournament?
a. 0 teams
b. 1 team
c. 2 teams
d. 3 teams
e. None of these
Would there be any “byes” in the tournament?
32
A team with a “bye” gets to advance to the next round “for free”.
Single-Elimination
Tournament Problem
What’s the maximum number of teams in an
n-round single-elimination tournament?
a. n teams
b. 2n teams
c. n2 teams
d. 2n teams
e. None of these
(Induction is for proving our theorem;
33
it’s not for figuring out what we want to prove!)
A Pattern for Induction
Coincidentally, t’s a good
variable name again! 
34
Side Note: How Many Rounds
Does a Tournament Have?
How many rounds does a tournament that’s just a
single winner have?
How many rounds does a tournament that’s a final
game between the winners of two smaller
tournaments have?
35
Side Note: How Many Rounds
Does a Tournament Have?
A tournament has:
• 0 rounds if it’s just a single “winner”
• 1 more round than the largest number of
rounds of either of its subtournaments,
otherwise
36
P(a) ≡ [fill in predicate definition]
Theorem: For all [recursive structure] a, P(a) holds.
Proof by structural induction:
Base case: [For each non-recursive case, prove the theorem holds for that
case. End by showing your theorem true for the base case structure!]
Inductive Step: [For each recursive case…]
Consider an arbitrary [recursive case structure] a.
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for a. [And then do so! You should:
1) END by showing that your theorem holds for a.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
37
P(a) ≡ there are at most 2n teams in tournament t with n rounds
Theorem: For all [recursive structure] a, P(a) holds.
Proof by structural induction:
Base case: [For each non-recursive case, prove the theorem holds for that
case. End by showing your theorem true for the base case structure!]
Inductive Step: [For each recursive case…]
Consider an arbitrary [recursive case structure] a.
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for a. [And then do so! You should:
1) END by showing that your theorem holds for a.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
38
P(a) ≡ there are at most 2n teams in tournament t with n rounds
Theorem: For all tournaments t, P(t) holds.
Proof by structural induction:
Base case: [For each non-recursive case, prove the theorem holds for
that case. End by showing your theorem true for the base case
structure!]
Inductive Step: [For each recursive case…]
Consider an arbitrary [recursive case structure] t.
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for t. [And then do so! You should:
1) END by showing that your theorem holds for t.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]39
I changed all the variables to t just because t’s a good name for a tournament.
P(a) ≡ there are at most 2n teams in tournament t with n rounds
Theorem: For all tournaments t, P(t) holds.
Proof by structural induction:
Base case: A tournament that is just a single “winner”… [TODO: finish!]
Inductive Step: [For each recursive case…]
Consider an arbitrary [recursive case structure] t.
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for t. [And then do so! You should:
1) END by showing that your theorem holds for t.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
40
We’ll leave the proof of the base case until we’ve finished the structural pieces.
P(a) ≡ there are at most 2n teams in tournament t with n rounds
Theorem: For all tournaments t, P(t) holds.
Proof by structural induction:
Base case: A tournament that is just a single “winner”… [TODO: finish!]
Inductive Step:
Consider an arbitrary [recursive case structure] t.
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for t. [And then do so! You should:
1) END by showing that your theorem holds for t.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
41
There’s just the one recursive case.
P(a) ≡ there are at most 2n teams in tournament t with n rounds
Theorem: For all tournaments t, P(t) holds.
Proof by structural induction:
Base case: A tournament that is just a single “winner”… [TODO: finish!]
Inductive Step:
Consider an arbitrary tournament t that is a final game between the winners
of two subtournaments (which we’ll call s1 and s2).
Induction Hypothesis: Assume the theorem holds for [each recursive
appearance of the structure in this case].
We now show the theorem holds for t. [And then do so! You should:
1) END by showing that your theorem holds for t.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
It’s often handy to name those recursive appearances!
42
E.g., subtournaments s1 and s2.
P(a) ≡ there are at most 2n teams in tournament t with n rounds
Theorem: For all tournaments t, P(t) holds.
Proof by structural induction:
Base case: A tournament that is just a single “winner”… [TODO: finish!]
Inductive Step:
Consider an arbitrary tournament t that is a final game between the winners
of two subtournaments s1 and s2.
Induction Hypothesis: Assume the theorem holds for s1 and s2.
We now show the theorem holds for t. [And then do so! You should:
1) END by showing that your theorem holds for t.
2) USE the “Induction Hypothesis” assumption(s) you made.
3) NEVER take a recursive appearance and use the recursive definition to
break it down further, like this BAD example: “each subtree is a binary
tree that can have subtrees and so on until we reach an empty tree.”]
43
44
It helps to write out what you want to prove rather than just “show the theorem holds for a”.
45
46
47
48
The “insight into how the
problem breaks down”
The n-round tournament with the maximum
number of teams is made from two
(n-1)-round tournaments with the
maximum number of teams.
49
Step-by-Step?
Here’s the logical structure of our theorems:
MT(0,20).
nZ0, (n > 0)  MT(n-1,2n-1)  MT(n,2n).
Do these prove MT(1,21)?
a. Yes.
b. No.
c. I don’t know.
50
Step-by-Step?
Here’s the logical structure of our theorems:
MT(0,20).
nZ0, (n > 0)  MT(n-1,2n-1)  MT(n,2n).
Plus, we know MT(1,21).
Do all of these prove MT(2,22)?
a. Yes.
b. No.
c. I don’t know.
51
Step-by-Step?
Here’s the logical structure of our theorems:
MT(0,20).
nZ0, (n > 0)  MT(n-1,2n-1)  MT(n,2n).
Plus, we know MT(1,21) and MT(2,22).
Do all of these prove MT(3,23)?
a. Yes.
b. No.
c. I don’t know.
52
Step-by-Step?
Here’s the logical structure of our theorems:
MT(0,20).
nZ0, (n > 0)  MT(n-1,2n-1)  MT(n,2n).
From this, can we prove MT(n,2n)
for any particular integer n?
a. Yes.
b. No.
c. I don’t know.
53
The Magic of Induction!
Our “proof” is really a recursive program.
Plug in a tournament of a particular number
of rounds n, and it produces a proof that
that tournament cannot have more than 2n
teams in it.
The magic of induction is that we agree that
writing this program is the same as writing
all the proofs the program can create.
54
Outline
•
•
•
•
Prereqs, Learning Goals, and Quiz Notes
Bonus Prelude
A Pattern for Induction
Problems and Discussion
– Single-Elimination Tournaments
– Induction on Numbers
• Next Lecture Notes
55
Natural Numbers and Induction
Can we prove things inductively about the
natural numbers without a recursive data
definition? Are they “self-referential”?
Here’s one answer:
A natural number is:
- 0
- The next number after (successor of) a
natural number
56
Natural Numbers and Induction
Can we prove things inductively about the
natural numbers without a recursive data
definition? Are they “self-referential”?
But, let’s just try one!
Problem: What is the sum of the natural
numbers 0..n?
57
Partly Worked Problem:
Sum of 0..n
Partly Worked Problem: What is the sum
of the natural numbers 0..n?
Induction doesn’t help with insight...
But spoilers do: n(n+1)/2.
Now, how do we prove it?
58
Partly Worked Problem:
Sum of 0..n
Partly Worked Problem: Prove that the
sum of the natural numbers 0..n is
n(n+1)/2.
Is this self-referential? If we knew the sum
of the numbers 0..(n-1), would it tell us
anything about the sum of the numbers
0..n?
59
Sigma, Pi, Factorial, Powers:
All Self-Referential
60
So? So, you can do inductive proofs on them!
Outline
•
•
•
•
Prereqs, Learning Goals, and Quiz Notes
Bonus Prelude
A Pattern for Induction
Problems and Discussion
– Single-Elimination Tournaments
– Induction on Numbers
• Next Lecture Notes
61
Learning Goals: In-Class
By the end of this unit, you should be able
to:
– Establish properties of self-referential
structures using inductive proofs that naturally
build on those self-references.
Note: this learning goal is not yet looking for formal
inductive proofs. Instead, we’re just exploring how we
work with things that are defined in terms of themselves.
62
Next Lecture Learning Goals:
Pre-Class
By the start of class, you should be able to:
– Given a theorem to prove and the insight into
how to break the problem down in terms of
smaller problems, write out the skeleton of an
inductive proof including: the base case(s)
that need to be proven, the induction
hypothesis, and the inductive step that needs
to be proven.
So, take what we did in this lecture and use the
textbook readings to formalize your understanding.
63
We will have much more practice on inductive proofs.
Next Lecture Prerequisites
See the Mathematical Induction Textbook
Sections at the bottom of the “Textbook
and References” page.
64
snick

snack
Extra Slides
65
One Extra Step We
Sometimes Do
Really, we are done. But just to be
thorough, we’ll add:
Termination: the number of rounds n is a
non-negative integer, and each application
of the inductive step reduces it by at least 1.
Therefore, it must reach our base case (0) in
a finite number of steps.
66
Binary Tree Data Definition
A binary-tree is one of:
- An empty tree
- (make-node binary-tree
binary-tree number)
10
5
2
20
9
7
15
17
67
10
Binary Trees’ Height
5
2
20
9
15
7
17
A binary tree’s “height” is the
maximum number of steps it takes
to get from the root down to a node
with only empty trees as children.
This one’s height is:
a. 0
b. 1
c. 2
d. 3
e. 4
68
10
Binary Trees’ Height
5
2
20
15
9
7
17
Problem 1: What’s the maximum number of
nodes in a binary tree of height n?
69
10
Binary Trees’ Height
5
2
20
9
15
7
17
Problem 2: Prove your result.
70
10
Binary Trees,
Take Two
5
2
20
9
15
7
17
Problem 1: Give an algorithm to determine
the height of a binary tree.
Problem 2: Prove that your algorithm is
correct.
71
10
5
Worked: Algorithm for Height
2
9
7
20
15
17
Height(t):
Is t an empty tree?
Yes: evaluate to -1
No: recursively find the height of
each subtree hl and hr and
evaluate to max(hl, hr) + 1.
How big are the subtrees?
72
We need to assume the algorithm works on trees of “those sizes”.
10
Worked: Proof of
Algorithm’s Correctness
5
2
20
9
7
15
17
Proof: We proceed by induction.
Base case: On an empty tree, our algorithm yields -1. We simply make
this correct by definition. (Or, you can imagine we go “up one level” to get
to a node.)
Induction Hypothesis: For an arbitrary (finite-sized) non-empty tree t,
assume our algorithm works correctly on all trees smaller than t.
Inductive step: t is not an empty tree; so, our algorithm finds the height of
each subtree hl and hr. These heights are correct by assumption, since the
subtrees must be smaller than t (lacking at least t itself!).
To reach a node with only leaves as children, we go into either the left or
right subtree. The length of the path through the left subtree is therefore
hl+1, with the +1 for the step down into that subtree. Similarly, the right
path is hr +1. The height is the larger of these possible paths; so, it’s
max(hl+1, hr+1) = max(hl, hr) + 1, which is what our algorithm evaluates to.
Termination: We call this only on finite trees, and we reduce the size of the
tree by at least one node at each inductive step. Thus, we eventually reach
an empty tree.
73
QED