Transcript Heuristic Search cs475 lecture note by Jin Hyung Kim

Problem Solving by Search
by Jin Hyung Kim
Computer Science Department
KAIST
Example of Representation
 Euler
Path
Graph Theory
 Graph
consists of
 A set of nodes : may be infinite
 A set of arcs(links)
 Directed

graph, underlying graph, tree
Notations
 node,
start node(root), leaf (tip node), root,
path, ancestor, descendant, child(children,
son), parent(father), cycle, DAG, connected,
locally finite graph, node expansion
State Space Representation
 Basic
Components
 set of states {s}
 set of operators { o : s -> s }
 control strategy { c: sn -> o }
 State space graph
 State -> node
 operator -> arc
 Four tuple representation
 [N, A, S, GD], solution path
Examples of SSR
 TIC_TAC_TOE
 n2-1
Puzzle
 Traveling salesperson problem (TSP)
Search Strategies



A strategy is defined by picking the order of node expansion
Search Direction s
 Forward searching (from start to goal)
 Backward searching (from goal to start)
 Bidirectional
Irrevocable vs. revocable
 Irrevocable strategy : Hill-Climbing





Most popular in Human problem solving
No shift of attention to suspended alternatives
End up with local-maxima
Commutative assumption
Applying an inappropriate operators may delay, but never prevent
the eventual discovery of solutions.
Revocable strategy : Tentative control

An alternative chosen, others reserve
Evaluation of Strategies

Completeness


Time Complexity


Maximum number of nodes in memory
Optimality


Number of nodes generated/expanded
Space complexity


Does it always find a solution if one exists ?
Does it always find a least-cost solution ?
Time and Space complexity measured by



b – maximum branching factors of the search tree
d – depth of least-cost solution
m - maximum depth of the state space (may be
Implementing Search Strategies

Uninformed search


Search does not depend on the nature of solution
Systematic Search Method


Breadth-First Search
Depth-First Search (backtracking)




Depth-limited Search
Uniform Cost Search
Iterative deepening Search
Informed or Heuristic Search

Best-first Search



Greedy search (h only)
A* search (g + h)
Iterative A* search
X-First Search Algorithm
start
put s in OPEN
yes
Fail
OPEN empty ?
Select & Remove the a node of OPEN
and put it in CLOSE (call it n)
Expand n.
Put successors at the end of OPEN
pointers back to n
any succesor = goal
?
yes
Success
Comparison of BFS and DFS
 BFS
always terminate if goal exist
cf. DFS on locally finite infinite tree
 Gurantee
shortest path to goal - BFS
 Space requirement
 BFS
- Exponential
 DFS - Linear,
keep children of a single node
 Which is better ? BFS or DFS ?
Uniform Cost Search
A
Genaralized version of Breadth-First
Search
 C(ni,
nj) = cost of going from ni to nj
 g(n) =(tentative minimal) cost of a path from s
to n.
 Guarantee
to find the minimum cost path
 Dijkstra Algorithm
Uniform Cost Search Algorithm
start
put s in OPEN, set g(s) = 0
yes
Fail
OPEN empty ?
Remove the node of OPEN whose g(.) value is smallest
and put it in CLOSE (call it n)
n = goal
?
yes
Success
Expand n. calculate g(.) of successor
Put successors to OPEN
pointers back to n
Iterative Deepening Search
 Compromise
of BFS and DFS
proc Iterative_Deeping_Search(Root)
begin
Success := 0;
for (depth_bound := 1; depth_bound++; Success == 1)
{
depth_first_search(Root, depth_bound);
if goal found, Success := 1;
}
end
 Save
on Storage, guarantee shortest path
 Additional node expansion is negligible
Iterative Deeping (l=0)
Iterative Deeping (l=1)
Iterative Deeping (l=2)
Iterative Deeping (l=3)
Properties of IDS


Complete ??
Time ??
(d+1)b0 + db1 + (d-1)b2 + … + bd = O(bd)


Space ?? : O(bd)
Optimal ?? Yes, if step cost = 1


Can be modified to explore uniform cost tree ?
Numerical comparison b=10 and d=5, solution at far
right
N(IDS) = 50 + 400 + 3,000 + 20,000 + 100,000 = 123,450
N(BFS) = 10 + 100 + 1000 + 10000 + 100000 + 999,990 = 1,111,100
Informed Search
Use of Heuristics to select the Best
 Tic-tac-toe
x
x
x
x o
x
o x
o
x
x
o
o
o
x
o
x
o x
Tic-tac-toe
 Most-Win
Heuristics
x
x
x
3 win
4 win
2 win
8-Puzzel Heuristics
3
2
1 8 4
1 2 3
8
4
7 6 5
7 6 5
2 3
1 8 4
7 6 5
2 8 3
1
4
7 6 5
2 3
1 8 4
7 6 5
a
b
c
• # of Misplaced tiles
• Sum of Manhattan distance
Heuristics for Road Map Problem
Best First Search Algorithm( for tree search)
start
put s in OPEN, compute f(s)
yes
Fail
OPEN empty ?
Remove the node of OPEN whose f(.) value is smallest
and put it in CLOSE (call it n)
n = goal
?
yes
Success
Expand n. calculate f(.) of successor
Put successors to OPEN
pointers back to n
Algorithm A
 Best-First
Algorithm with f(n) = g(n) + h(n)
where g(n) : cost of n from start to node n
h(n) : heuristic estimate of the cost
from n to a goal
 Algorithm
is admissible if it terminate with optimal
solution
 What if f(n) = f*(n) where f*(n) = g*(n) + h*(n)
where g*(n) = shortest cost to n
h*(n) = shortest actual cost from n to goal
Algorithm A*
(Branch and Bound method)
 Algorithm
A becomes A* if h(n)
 Algorithm A* is admissible
 can
h*(n)
you prove it ?
h(n)  0, A* algorithm becomes uniform cost
algorithm
 If
 Uniform
 If
cost algorithm is admissible
n* is on optimal path, f*(n*) = C*
 f*(n) > C* implies that n is not on optimal path
 A* terminate in finite graph
Examples of
Admissible Heuristics




8 puzzle heuristic
N queen problem, Tic-tac-toe
Air distance heuristic
Traveling Salesperson Problem
 Minimum
 ….
spanning tree heuristics
Iterative Deeping A*

Modification of A*
 use

threshold as depth bound
To find solution under the threshold of f(.)
 increase
threshold as minimum of f(.) of
previous cycle
 Still
admissible
 same order of node expansion
 Storage Efficient – practical
 but
suffers for the real-valued f(.)
 large number of iterations
Iterative Deepening A* Search Algorithm ( for tree search)
start
set threshold as h(s)
put s in OPEN, compute f(s)
yes
OPEN empty ?
threshold =
min( f(.) , threshold )
Remove the node of OPEN whose f(.) value is smallest
and put it in CLOSE (call it n)
n = goal
?
yes
Success
Expand n. calculate f(.) of successor
if f(suc) < threshold then
Put successors to OPEN if
pointers back to n
Memory-bounded heuristic
Search

Recursive best-first search






A variation of Depth-first search
Keep track of f-value of the best alternative path
Unwind if f-value of all children exceed its best alternative
When unwind, store f-value of best child as its f-value
When needed, the parent regenerate its children again.
Memory-bounded A*


When OPEN is full, delete worst node from OPEN storing fvalue to its parent.
The deleted node is regenerated when all other candidates
look worse than the node.
Monotonicity (consistency)
A
heuristic function is monotone if
for all states ni and nj = suc(ni)
h(ni) - h(nj)
cost(ni,nj)
and h(goal) = 0
 Monotone
heuristic is admissible
More Informedness (Dominate)
 For
two admissible heuristic h1 and h2, h2 is
more informed than h1 if
h1(n)
0
 for
h2(n) for all n
h1(n)
8-tile problem
 h1
h2(n)
h*(n)
: # of misplaced tile
 h2 : sum of Manhattan distance
Generation of Heuristics

Relaxed problem solution is an admissible heuristics



Manhattan distance heuristic
Solution of subproblems
combining several admissible heuristics
h(n) = max{ h1(n), …, hn(n)}

Use of Pattern databases

Max of heuristics of sub-problem pattern database


1/ 1000 in 15 puzzle compared with Manhattan
Addition of heuristics of disjoint sub-problem pattern
database


1/ 10,000 in 24 puzzle compared with Manhattan
disjoint subdivision is not possible for Rubic’s cube
Semi-addmissible heuristics
Dynamic Weighting,
Risky heuristics
n{ h(n) h*(n) + e }, then

If

f(n) = g(n) + h(n) + e[1-d(n)/N] h(n)



C(n)
 (1+e) C*(n)
At shallow level : depth first excursion
At deep level : assumes admissibility
Use of non-admissible heuristics with risk


Utilize heuristic functions which are admissible in the most of
cases
Statistically obtained heuristics
PEARL, J., AND KIM, J. H. Studies in semi-admissible heuristics. IEEE Trans. PAMI-4, 4 (1982), 392-399
Performance Measure
 Penetrance
 how
search algorithm focus on goal rather
than wander off in irrelevant directions
 P=L/T
 Effective
B
Branching Factor (B)
+ B2 + B3 + ..... + BL = T
 less dependent on L
Planning : Monkey and Banana
 Monkey
is on floor at (x1, y1), banana is
hanging at (x2, y2), and box is at (x3, y3).
Monkey can grab banana if he push box
under banana and climb on it. Develop
a state-space search representation for
this situation and show how monkey
can grab banana.
Local Search and Optimization
Local
search
less
memory required
Reasonable solutions in large (continuous)
space problems
Can
be formulated as Searching for
extreme value of Objective function
find i = ARGMAX { Obj(pi) }
where pi is parameter
Search for Optimal Parameter
 Deterministic
Methods
 Step-by-step
procedure
 Hill-Climbing search, gradient search
 ex: error back propagation algorithm

Finding Optimal Weight matrix in Neural Network training
 Stochastic
Methods
 Iteratively

Improve parameters
Pseudo-random change and retain it if it improves
 Metroplis
algorithm
 Simulated Annealing algorithm
 Genetic Algorithm
Hill Climbing Search
1.
2.
3.
4.
Set n to be the initial node
If obj(n) > max { obj(childi(n)) } then exit
Set n to be the highest-value child of n
Return to step 2
 No
previous state information
 No backtracking
 No jumping
 Gradient
 Hill
Search
climbing with continuous, differentiable functions
 step width ?
 Slow in near optimal
State space landscape
Real World
Hill-climbing :Drawbacks

Local maxima





At Ridge
Stray in Plateau
Slow in Plateau
Determination of
proper Step size
Cure

Random restart

Good for Only few
local maxima
Global Maximum
Local Beam Search
 Keep
track of best k states instead of 1
in hill-climbing
 Full utilization of given memory
 Variation:
 Select
Stochastic beam search
k successors randomly
Iterative Improvement
Algorithm

Basic Idea

Start with initial setting



Iteratively improve the quality
Good For hard, practical problems


Generate a random solution
Because it Keeps current state only and No look-ahead
beyond neighbors
Implementation



Metropolis algorithm
Simulated Annealing algorithm
Genetic algorithm
Metropolis algorithm
Modified
Monte Carlo method
Suppose our objective is to reach the
state minimizing energy function
1. Randomly generate a new state, Y, from state X
2. If E(energy difference between Y and X) < 0
then move to Y (set Y to X) and goto 1
3. Else
3.1 select a random number, 
3.2 if  < exp(- E / T)
then move to Y (set Y to X) and goto 1
3.3 else goto 1
From Statistical Mechanics

In thermal equilibrium,
probability of state i
energy of state i
 absolute temperature
 Boltzman constant


In NN

define E  E j  Ei
πj
 E 
 exp  

πi
 T 
 Ei 
1

 i  exp  
Z
 k BT 
1
 Ei 
 i  exp   
Z
 T 
Simulated Annealing
Probability distribution
p
1
eE/T
0
E
Simulated Annealing algorithm
 What
is annealing?
 Process
contraction : cause stress
of slowly cooling
down a compound or a
substance
 Slow cooling let the substance
flow around 
thermodynamic equilibrium
 Molecules get optimum
conformation
Simulated Annealing
 Simulates
slow cooling of annealing process
 Solves combinatorial optimization
 variant of Metropolis algorithm
 by S. Kirkpatric (83)
 finding minimum-energy solution of a neural
network = finding low temperature state of physical
system
 To overcome local minimum problem
 Instead always going downhill, try to go downhill
‘most of the time’
Iterative algorithm comparison

Simple Iterative Algorithm
1. find a solution s
2. make s’, a variation of s
3. if s’ is better than s, keep s’ as s
4. goto 2

Metropolis Algorithm



3’ : if (s’ is better than s) or ( within Prob), then keep s’ as s
With fixed T
Simulated Annealing

T is reduced to 0 by schedule as time passes
Simulated Annealing algorithm
function Simulated-Annealing(problem, schedule) returns a solution state
inputs: problem, a problem
local variables: current, a node
next, a node
T, a “temperature” controlling the probability of downward steps
current  Make-Node(Initial-State[problem])
for t1 to infinity do
T  schedule[t]
if T=0 then return current
next  a randomly selected successor of current
DE  Value[next] – Value[current]
if DE>0 then currentnext
else currentnext only with probability eDE/T
Simulated Annealing
Schedule example
T0
T(n)
Tf


if Ti is reduced too fast, poor quality
if Tt >= T(0) / log(1+t)
- Geman



move
s
System will converge to minimun configuration
Tt = k/1+t
- Szu
Tt = a T(t-1) where a is in between 0.8 and 0.99
n
Simulated Annealing parameters

Temperature T




Schedule



Used to determine the probability
High T : large changes
Low T : small changes
Determines rate at which the temperature T is lowered
Lowers T slowly enough, the algorithm will find a global
optimum
In the beginning, aggressive for searching alternatives,
become conservative when time goes by
Simulated Annealing(10)

To avoid of entrainment in local minima
 Annealing schedule : by trial and error




Choice of initial temperature
How many iterations are performed at each temperature
How much the temperature is decremented at each step as cooling
proceeds
Difficulties
 Determination of parameters
 If cooling is too slow Too much time to get solution
 If cooling is too rapid  Solution may not be the global optimum
Simulated Annealing
higher probability of
escaping local maxima
Local Maxima
Little chance of escaping local
maxima, but local maxima may
be good enough in practical
problmes.