Transcript Informed Search R & N Chapter 4, except 4.4 Heuristics The word heuristic comes from the Greek word  (heuriskein), meaning “to discover”,

Informed Search
R & N Chapter 4, except 4.4
Heuristics
The word heuristic comes from the Greek word 
(heuriskein), meaning “to discover”, which is also the origin of
eureka, derived from Archimedes’ reputed exclamation, heurika
(“I have found”), uttered when he had discovered a method for
determining the purity of gold.
We typically use the term heuristic to refer to a technique that is
likely, but not guaranteed, to produce a good (but not
necessarily best) answer or the best answer with a good but not
necessarily best expenditure of effort.
Heuristics and Human Behavior
The availability heuristic is an oversimplified rule of thumb, or heuristic, which occurs when
people estimate the probability of an outcome based on how easy that outcome is to imagine. As
such, vividly described, emotionally-charged possibilities will be perceived as being more likely
than those that are harder to picture or are difficult to understand, resulting in a corresponding
cognitive bias.
For example, most people think that dying from a shark attack is more likely than dying from
injuries sustained from falling airplane parts, yet the opposite is true by a factor of 30. Perhaps this
is because sharks are inherently terrifying or because shark attacks receive more media coverage.
Many people seem to fear plane crashes, yet a dark fate is much more likely to befall you in an
automobile accident on the way to the airport.
One important corollary finding to this heuristic is that people asked to imagine an outcome
immediately view it as more likely than those that were not. And, that which was vividly described
is viewed as more likely than that which was wrought a pallid description. This tendency is often
seen in politics. For example, when justifying an invasion of Iraq in the 2003 State of the Union
Address, President George W. Bush said: "Imagine those 19 hijackers with other weapons and other
plans, this time armed by Saddam Hussein. It would take one vial, one canister, one crate slipped
into this country to bring a day of horror like none we have ever known."
An opposite effect of this bias, called denial, occurs when an outcome is so upsetting that the very
act of thinking about it leads to an increased refusal to believe it might occur. In this case, being
asked to imagine the outcome actually made subjects view it as less likely.
This phenomenon was first reported by psychologists Amos Tversky and Daniel Kahneman.
http://en.wikipedia.org/wiki/Availability_heuristic
Heuristic Functions
The job of a heuristic function f(n) is to evaluate a node n in a
search tree so that the “best” node can be selected for
expansion.
Two approaches to defining f:
•f measures the value of the state contained in the node.
•f measures the cost of the state (and possibly the path to it).
Often, in this case, we decompose f:
f(n) = g(n) + h(n), where
g(n) = the cost to get to n, and
h(n) = an estimate of the cost to get from n to a goal
Examples - f Measures the Value
Often, in these cases, f is the weighted sum of a set of
component values:
•Chess
•School lunch planning
•Autonomous Mars rover
•Perry Mason defense lawyer
Examples – f Measures the Cost to a Goal
In many problems, all goal states are equally good. Then all we
care about is the cost of getting to one. So a state is good if the
estimated cost of getting from it to a goal is low. Or we may
also want to take into account the cost, so far, of getting to it.
• 8–Puzzle
• Water jug
• Route planning
Now We Must Evaluate on Generation
But We Still Have to Check for Alternatives
Search Algorithms – Straight Line or Branching
Branching Algorithms
•A* - See handout
(http://www.cs.utexas.edu/users/ear/cs343/A_Star_Algorithm.htm)
•RBFS
•SMA* - A* until it runs out of memory. Then drop the worst
node and back up the value.
•AO* - See handout
(http://www.cs.utexas.edu/users/ear/cs343/AOstarAlgorithm.htm)
A*
A
1
B
1+2
0+2
5
C
5+2
A*
A
1
B
1
D 2+2
1+2
0+2
5
C
2
E 3+5
5+2
A*
A
1
B
1
D 2+2
2
F 4+2
1+2
0+2
5
C
2
E 3+5
5+2
A*
A
1
B
1
2
4+2
2
G
6+2
5
C
2
D 2+2
F
1+2
2+0
E 3+5
5+2
A*
A
1
B
2+0
1
1+2
1
5
C
2
E 3+5
D 2+2
2
1
F
4+2
G
7+2
3
6+2
We’ve found a cheaper path to G. We need to record it.
5+2
The Optimality of A*
A
1
B
1
1+2
0+2
5
2
D 2+2
5+2
C
E 3+5
1
H
2
F
4+2
2
G
6+2
h has underestimated the true cost. But A* will find H.
6+0
The Optimality of A*
A
1
B
1
1+2
0+2
5
2
D 2+2
5+5
C
E 3+5
1
H
6+0
2
F
4+2
4
G
8+0
h has overestimated the true cost and missed the optimal path.
The Optimality of A*
A* is admissible (i.e., it is guaranteed to find an optimal path
to a goal, if one exists) iff h never overestimates the true cost to
a goal.
If h rarely overestimates the true cost by more than , then A*
will rarely find a solution whose cost is more than  greater
than the cost of the optimal solution.
Recursive Best First Search
SMA*
Like A* until it runs out of memory. Then drop the worst
leaf but backs up the value to the parent. This part of the tree
will only be regenerated if all alternatives become worse.
A Version of Depth-First Search - Branch and
Bound
Consider the problem of planning a ski vacation.
Fly to A $600
Fly to B $800
Stay D $200
Stay E $250
(800)
(850)
Total cost
(1200)
Fly to C $2000
Problem Reduction
Goal: Acquire TV
Steal TV
Earn Money
Buy TV
Or another one: Theorem proving in which we reason backwards
from the theorem we’re trying to prove.
We Need a New Algorithm - AO*
AO* - Computing Scores
A
3
(1) B
(2)
1
C (1)
In A*, we assigned to each node a cost f = g + h. We combined
the cost of getting to the node with the expected cost of getting
from it to a solution. Here we assign to each node just the cost
of getting from it to a solution. We assign to each path the
expected cost of getting to a goal by following it. We must
update the path costs at each step.
AO* - Computing Scores
A
3
(1) B
(2)
1
C (1)
1
X (8)
AO* - Computing Scores
A
3
(1) B
(4)
1
C (9)
1
X (8)
AO* - Computing Scores
A
3
(3) B
2
(1) D
1
C (9)
1
X (8)
AO* - Computing Scores
A
3
1
(1) B
C (9)
2
1
(1) D
1
(1) E
X (8)
1
F
(3)
AO* - Computing Scores
A
3
1
(8) B
C (9)
2
1
(6) D
1
(1) E
X (8)
1
F
(3)
AO* - Computing Scores
A
3
1
(8) B
C (9)
2
1
(6) D
1
X (8)
1
1
(1) E
F
(3)
AO* - Computing Scores
A
3
1
(8) B
C (5)
2
1
(6) D
1
X (4)
1
1
(1) E
F
(3)
A Detail – Should We Store Ancestor Links?
A
3
1
(1) B
C (7)
2
(1) D
1
(3) E
1
F
(1)
A Detail – Should We Store Ancestor Links?
A
3
1
(1) B
C (7)
2
(1) D
1
(3) E
1
F
(1)
1
A Detail – Should We Store Ancestor Links?
A
3
1
(1) B
C (7)
2
1
(1) D
1
(3) E
1
F
(1)
1
Good Heuristic Functions
In any searching problem where there are b choices at each
node and a depth of d at the goal node, a naive searching
algorithm would have to potentially search around bd nodes
before finding a solution. Heuristics improve the efficiency of
search algorithms by reducing the branching factor from b to
(ideally) a low constant b*.
Although any admissible heuristic will give an optimal answer,
a heuristic that gives a lower branching factor is more
computationally efficient for a particular problem. It can be
shown that a heuristic h2(n) is better than another heuristic
h1(n) , if h2(n) dominates h1(n), ie. h1(n) < h2(n) for all n.
From http://en.wikipedia.org/wiki/Heuristic
Finding Good Heuristic Functions
•Solution costs of sub-problems often serve as useful estimates
of the overall solution cost. These are always admissible if
subproblems don’t overlap. For example, a heuristic for a 10puzzle might be the cost of moving tiles 1-5 into their correct
places. A common idea is to use a pattern database that stores the
exact solution cost of every subproblem instance.
•The solution of a relaxed problem often serves as a useful
admissible estimate of the original. For example Manhattan
distance is a relaxed version of the n-puzzle problem, because
we assume we can move each tile to its position in a single step.
•Given a set of admissible heuristic functions, the function h(n) =
max{h1(n),h2(n)...hi(n)} is an admissible heuristic that dominates
all of them.
(From: http://en.wikipedia.org/wiki/Heuristic)
Straight Line (Local) Search
•Generate and test – the uninformed version. Start over each
time.
•Smarter strategies, in which we make an initial attempt, then
attempt to improve on it. We’ll use complete state
formulations for these algorithms.
•Hill climbing
•Simulated annealing
•Genetic algorithms
Hill Climbing – The Algorithm
function Hill-Climbing(problem) returns a state that is a local maximum
inputs: problem
local variables: current, a node; neighbor a node
current  Make-Node(initial-state[problem])
loop do
neighbor  a highest-valued successor of current
if value[neighbor]  value[current] then return state[current]
current  neighbor
N-Queens
We could solve the problem with depth-first search:
http://www.math.utah.edu/~alfeld/queens/queens.html
Hill Climbing - N-Queens
x
x
(6)
x
x
x
(4)
x
x
x
x
(2)
x
x
x
f = # of attacking pairs
N-Queens – An Alternative
It turns out that the N-Queens problem can be solved almost all
the time without search at all:
http://www.apl.jhu.edu/~hall/NQueens.html
Hill Climbing – Some Problems
Sometimes the Problem is f
From the initial state, move A to the table. Three choices for what
to do next.
A local heuristic function: Add one point for every block that is
resting on the thing it is supposed to be resting on. Subtract one
point for every block that is sitting on the wrong thing.
Sometimes the Problem is f
From the initial state, move A to the table. Three choices for what
to do next.
A global heuristic function: For each block that has the correct
support structure (i. e., the complete structure underneath it is
exactly as it should be), add one point for every block in the
support structure. For each block that has an incorrect support
structure, subtract one point for every block in the existing support
structure.
Hill Climbing – Is Close Good Enough?
B
A
Is A good enough?
•8 Queens
•Choose winning lottery numbers
Hill Climbing – Is Close Good Enough?
B
A
Is A good enough?
•8 Queens
•Choose winning lottery numbers
•Get the cheapest travel itinerary
Simulated Annealing
function Simulated-Annealing(problem,schedule) returns a solution state
inputs: problem, a problem
schedule, a mapping from time to “temperature”
local variables: current, a node
next, a node
best, a node
T, a “temperature”
current  Make-Node(initial-state[problem])
best  current
for t  1 to  to
T  schedule[t]
if T = 0 then return best
next  a randomly selected successor of current
if GoalTest(next) then return next
if value[next] > value[best] then best  next
E  value[next] - value[current]
if E > 0 then current  next
else current  next with p = eE/T
Following Slightly More than One Path
What if we follow some small number of paths?
•
Local beam search:
1. Start with k randomly generated nodes.
2. Generate k successors of each.
3. Keep the best k out of the k2.
•
Genetic algorithms:
1. Start with k randomly generated nodes called the
population.
2. Generate successors by combining pairs of nodes.
3. Allow mutations.
Minimax Search for Games
A
B
(8)
C
(3)
D
(-2)
Minimax Search for Games
A
B
E
(9)
F
(-6)
C
G
(0)
H
(0)
D
I
(-2)
J
(-4)
K
(10)
Minimax Search for Games
A (-2)
B (-6)
E
(9)
F
(-6)
G
(0)
C (-2)
H
(0)
I
(-2)
D (-4)
J
(-4)
K
(10)
Minimax with Alpha Beta Cutoffs
(a form of Branch and Bound)
A
B (8)
C (-2)
D (-4)
Minimax with Alpha Beta Cutoffs
(a form of Branch and Bound)
A ( 5)
B (5)
E
(9)
F
(6)
C (-2)
G
(5)
D (-4)
Minimax with Alpha Beta Cutoffs
(a form of Branch and Bound)
A ( 5)
B (5)
E
(9)
F
(6)
C ( -3)
G
(5)
H
(-3)
I
J
D (-4)
Other Ideas in Game Tree Search
•Iterative deepening allows a program to use exactly as much
time as it has.
•Waiting for quiescence helps to avoid the horizon effect.
•Secondary search double checks a chosen move before
committing.
•Saving backed up values trades space for time and effectively
extends the depth of the search in common configurations.
•Book moves help at the beginning and the end.