Transcript Heuristic Search

Heuristic Search
Russell and Norvig: Chapter 4
Current Bag of Tricks
Search algorithm
Heuristic Search
2
Current Bag of Tricks
Search algorithm
Modified search algorithm
Heuristic Search
3
Current Bag of Tricks
Search algorithm
Modified search algorithm
Avoiding repeated states

어떤 때에는 검사를 위한 시간이 더 걸릴
수도 있음 – Search Space가 매우 클 때
Heuristic Search
4
Best-First Search
Define a function:
f : node N  real number f(N)
called the evaluation function, whose
value depends on the contents of the
state associated with N
Order the nodes in the fringe in increasing
values of f(N)

순서화 하기 위한 방법이 중요, heap의 사용도 한
방법 --- 이유는?
f(N) can be any function you want, but will it
work?
Heuristic Search
5
Example of Evaluation Function
f(N) = (sum of distances of each tile to its goal)
+ 3 x (sum of score functions for each tile)
where score function for a non-central tile is 2 if it is
not followed by the correct tile in clockwise order
and 0 otherwise
5
8
4 2 1
7 3 6
N
1 2 3 f(N) = 2 + 3 + 0 + 1 + 3 + 0 + 3 + 1
3x(2 + 2 + 2 + 2 + 2 + 0 + 2)
4 5 6
= 49
7 8
goal
Heuristic Search
6
Heuristic Function
Function h(N) that estimate the cost of
the cheapest path from node N to goal
node.
Example: 8-puzzle
5
8
4 2 1
7 3 6
N
1 2 3 h(N) = number of misplaced tiles
=6
4 5 6
7 8
goal
Heuristic Search
7
Heuristic Function
Function h(N) that estimate the cost of
the cheapest path from node N to goal
node.
Example: 8-puzzle
5
8
4 2 1
7 3 6
N
1 2 3 h(N) = sum of the distances of
every tile to its goal position
4 5 6
=2+3+0+1+3+0+3+1
7 8
= 13
goal
Heuristic Search
8
Robot Navigation
N
YN
XN
h(N) = Straight-line distance to the goal
= [(Xg – XN)2 + (Yg – YN)2]1/2
Heuristic Search
9
Examples of Evaluation function
Let g(N) be the cost of the best
path found so far between the initial
node and N
f(N) = h(N)  greedy best-first
f(N) = g(N) + h(N)
Heuristic Search
10
8-Puzzle
f(N) = h(N) = number of misplaced tiles
3
5
3
4
3
4
2
4
2
3
3
1
0
4
5
4
Heuristic Search
11
8-Puzzle
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
3+3
1+5
2+3
3+4
5+2
0+4
3+2
1+3
2+3
4+1
5+0
3+4
1+5
2+4
Heuristic Search
12
8-Puzzle
f(N) = h(N) =  distances of tiles to goal
6
5
2
5
2
4
3
1
0
4
6
5
Heuristic Search
13
Can we Prove Anything?
If the state space is finite and we avoid
repeated states, the search is complete,
but in general is not optimal
If the state space is finite and we do
not avoid repeated states, the search is
in general not complete
If the state space is infinite, the search
is in general not complete
Heuristic Search
14
Admissible heuristic
Let h*(N) be the cost of the optimal
path from N to a goal node
Heuristic h(N) is admissible if:
0  h(N)  h*(N)
An admissible heuristic is always
optimistic
Heuristic Search
15
8-Puzzle
5
8
1
2
3
6
4
2
1
4
5
7
3
N
6
7
8
goal
• h1(N) = number of misplaced tiles = 6 is admissible
• h2(N) = sum of distances of each tile to goal = 13
is admissible
• h3(N) = (sum of distances of each tile to goal)
+ 3 x (sum of score functions for each tile) = 49
is not admissible
Heuristic Search
16
Robot navigation
Cost of one horizontal/vertical step = 1
Cost of one diagonal step = 2
h(N) = straight-line distance to the goal is admissible
Heuristic Search
17
A* Search
Evaluation function:
f(N) = g(N) + h(N)
where:


g(N) is the cost of the best path found so far to N
h(N) is an admissible heuristic
0 <   c(N,N’)
Then, best-first search with “modified search
algorithm” is called A* search
Heuristic Search
18
Completeness & Optimality of A*
Claim 1: If there is a path from the
initial to a goal node, A* (with no
removal of repeated states) terminates
by finding the best path, hence is:
 complete
 optimal
Heuristic Search
19
8-Puzzle
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
3+3
1+5
2+3
3+4
5+2
0+4
3+2
1+3
2+3
4+1
5+0
3+4
1+5
2+4
Heuristic Search
20
Robot Navigation
Heuristic Search
21
Robot Navigation
f(N) = h(N), with h(N) = Manhattan distance to the goal
8
7
7
6
5
4
5
4
3
3
2
6
7
6
8
7
3
2
3
4
5
6
5
1
0
1
2
4
5
6
5
4
3
2
Heuristic Search
3
4
5
6
22
Robot Navigation
f(N) = h(N), with h(N) = Manhattan distance to the goal
8
7
7
6
5
4
5
4
3
3
2
6
77
6
8
7
3
2
3
4
5
6
5
1
00
1
2
4
5
6
5
4
3
2
Heuristic Search
3
4
5
6
23
Robot Navigation
f(N) = g(N)+h(N), with h(N) = Manhattan distance to goal
8+3
8 7+4
7 6+3
6+5
6 5+6
5 4+7
4 3+8
3 2+9
2 3+10
3 4
7+2
7
6+1
6
5
5+6
5 4+7
4 3+8
3
3 2+9
2 1+10
1 0+11
0 1
6
5
2
4
7+0
7 6+1
6
5
8+1
8 7+2
7 6+3
6 5+4
5 4+5
4 3+6
3 2+7
2 3+8
3 4
Heuristic Search
5
6
24
Robot navigation
f(N) = g(N) + h(N), with h(N) = straight-line distance from N to goal
Cost of one horizontal/vertical step = 1
Cost of one diagonal step = 2
Heuristic Search
25
About Repeated States
f(N1)=g(N1)+h(N1)
N
N1
S
S1
f(N)=g(N)+h(N)
• g(N1) < g(N)
• h(N) < h(N1)
• f(N) < f(N1)
• g(N2) < g(N)
N2
f(N2)=g(N2)+h(N)
Heuristic Search
26
Consistent Heuristic
The admissible heuristic h is consistent
(or satisfies the monotone restriction) if
for every node N and every successor N’
of N:
N
c(N,N’)
h(N)  c(N,N’) + h(N’)
N’
h(N)
h(N’)
(triangular inequality)
Heuristic Search
27
8-Puzzle
5
8
1
2
3
6
4
2
1
4
5
7
3
N
6
7
8
goal
• h1(N) = number of misplaced tiles
• h2(N) = sum of distances of each tile to goal
are both consistent
Heuristic Search
28
Robot navigation
Cost of one horizontal/vertical step = 1
Cost of one diagonal step = 2
h(N) = straight-line distance to the goal is consistent
Heuristic Search
29
Claims
If h is consistent, then the function f along
any path is non-decreasing:
N
f(N) = g(N) + h(N)
f(N’) = g(N) +c(N,N’) + h(N’)
c(N,N’)
N’
h(N)
h(N’)
Heuristic Search
30
Claims
If h is consistent, then the function f along
any path is non-decreasing:
N
f(N) = g(N) + h(N)
f(N’) = g(N) +c(N,N’) + h(N’)
h(N)  c(N,N’) + h(N’)
f(N)  f(N’)
Heuristic Search
c(N,N’)
N’
h(N)
h(N’)
31
Claims
If h is consistent, then the function f along
any path is non-decreasing:
N
f(N) = g(N) + h(N)
f(N’) = g(N) +c(N,N’) + h(N’)
h(N)  c(N,N’) + h(N’)
f(N)  f(N’)
c(N,N’)
N’
h(N)
h(N’)
If h is consistent, then whenever A* expands
a node it has already found an optimal path
to the state associated with this node
Heuristic Search
32
Avoiding Repeated States in A*
If the heuristic h is consistent, then:
Let CLOSED be the list of states
associated with expanded nodes
When a new node N is generated:


If its state is in CLOSED, then discard N
If it has the same state as another node in
the fringe, then discard the node with the
largest f
Heuristic Search
33
Complexity of Consistent A*
Let s be the size of the state space
Let r be the maximal number of states
that can be attained in one step from
any state
Assume that the time needed to test if
a state is in CLOSED is O(1)
The time complexity of A* is O(s r log s)
Heuristic Search
34
Heuristic Accuracy
h(N) = 0 for all nodes is admissible and
consistent. Hence, breadth-first and uniformcost are particular A* !!!
Let h1 and h2 be two admissible and consistent
heuristics such that for all nodes N:
h1(N)  h2(N).
Then, every node expanded by A* using h2 is
also expanded by A* using h1.
h2 is more informed than h1
Heuristic Search
35
Iterative Deepening A* (IDA*)
Use f(N) = g(N) + h(N) with admissible
and consistent h
Each iteration is depth-first with cutoff
on the value of f of expanded nodes
Heuristic Search
36
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=4
6
Heuristic Search
37
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=4
4
6
6
Heuristic Search
38
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=4
4
5
6
6
Heuristic Search
39
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
5
4
Cutoff=4
4
5
6
6
Heuristic Search
40
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
6
5
4
5
6
6
4
Cutoff=4
Heuristic Search
41
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
6
Heuristic Search
42
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
4
6
6
Heuristic Search
43
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
4
5
6
6
Heuristic Search
44
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
4
5
7
6
6
Heuristic Search
45
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
5
4
5
7
6
6
Heuristic Search
46
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
5
4
5
5
7
6
6
Heuristic Search
47
f(N) = g(N) + h(N)
with h(N) = number of misplaced tiles
8-Puzzle
4
Cutoff=5
5
4
5
5
7
6
6
Heuristic Search
48
About Heuristics
Heuristics are intended to orient the search along
promising paths
The time spent computing heuristics must be
recovered by a better search
After all, a heuristic function could consist of solving
the problem; then it would perfectly guide the
search
Deciding which node to expand is sometimes called
meta-reasoning
Heuristics may not always look like numbers and
may involve large amount of knowledge
Heuristic Search
49
Robot Navigation
Local-minimum problem
f(N) = h(N) = straight distance to the goal
Heuristic Search
50
What’s the Issue?
Search is an iterative local procedure
Good heuristics should provide some
global look-ahead (at low computational
cost)
Heuristic Search
51
Other Search Techniques
Steepest descent (~ greedy best-first
with no search)  may get stuck into
local minimum
Heuristic Search
52
Other Search Techniques
Steepest descent (~ greedy best-first
with no search)  may get stuck into
local minimum
Simulated annealing
Heuristic Search
53
Other Search Techniques
Steepest descent (~ greedy best-first
with no search)  may get stuck into
local minimum
Simulated annealing
Genetic algorithms
Heuristic Search
54
Informed Search methods
Best-first Search
Evaluation function :::
a number purporting
to described the desirability of expanding the node
Best-first search ::: a node with the
best evaluation is expanded first
자료구조가 중요!!! Queue를 사용하되
linked list로 구현???
Heuristic Search
56
Greedy search
heuristic function h(n)
h(n) = estimated cost of the cheapest path from the state
at node n to a goal state
현재부터 가장 가까운 거리에 있는 것
Hill climbing method
Heuristic Search
57
Iterative improvement
algorithm
Hill climbing method
Simulated annealing
Neural networks
Genetic algorithm
Heuristic Search
58
Hill climbing Method
Simple hill climbing method :::
연산을
적용한 결과가 goal인지 검사 후, 아니면 기존보다
좋으면 무조건 선택, 아니면 다른 연산 적용, 더
이상 적용할 연산이 없으면 실패  선택한
노드에서 다시 탐색
Steepest-ascent hill climbing method
::: 현재 노드(상태)에 모든 가능한 연산을
적용한 결과에 goal이 있으면 성공, 아니면 제일
좋은 것을 선택하여, 기존보다 좋으면 그 노드에서
다시 탐색, 아니면 실패
59
Complete하지 못함Heuristic Search
예제
A
H
G
F
E
D
C
B

Heuristic Search
H
G
F
E
D
C
B
A
60
First move
A
H
G
F
E
D
C
B
Heuristic Search
61
Second move(Three possible
moves)
A
H
G
F
E
D
C
B
(a)
G
F
E
D
C
B
H
A
(b)
Heuristic Search
A
H
(c)
G
F
E
D
C
B
62
Two heuristics
Local :
Add one point for
every block that is resting on
the thing it is supposed to be
resting on. Subtract one point
for every block that is sitting on
the wrong thing
goal : 8
initial state : 4
first move : 6
second move:
(a) 4, (b) 4, (c) 4
Global :
For each block that
has the correct support
structure, add one point for
every block in the support
structure. For each block that
has an incorrect support
structure, subtract one point for
every block in the existing
support structure
goal : 28,
initial : -28;
first move : -21;
second move: (a) –28,
(b) –16, (c)-15
Heuristic Search
63
Hill climbing method의 약점
Local maxima
Plateau
Ridges
Random-restart hill climbing
Simulated annealing
Heuristic Search
64
Simulated annealing
알고리즘
for t  1 to ∞ do
Tschedule[t]
if T=0 then return current
next  a randomly selected successor of current
∆E  Value[next]-Value[current]
if ∆E>0 then current next
else current next only with probability e∆E /T
이동성 P=e∆E /T (P=e-∆E /kT )
일반적으로 에너지가 낮은 방향으로 물리현상은 일어나지만
높은 에너지 상황으로 변하는 확률이 존재한다.
∆E : positive change in energy level
T : Temperature
k : Boltzmann’s constant
Heuristic Search
65
Heuristic Search
f(n)=g(n)+h(n)
f(n) = estimated cost of the cheapest
solution through n
Best-first search
Heuristic Search
66
Minimizing the total path
cost (A* algorithm)
Admissible heuristics ::: an h function
that never overestimates the cost to
reach the goal
If h is a admissible, f(n) never
overestimates the actual cost of the
best solution through n
Heuristic Search
67
The behavior of A* search
Monontonicity ::: along any path from
the root, the f-cost never decreases
-- underestimate하면 non-monotonic할 수 있음
Complete and optimal
f(n’) = max(f(n),g(n’)+h(n’))
(n이 n’의 parent node)
optimality와 completeness를 증명한다
Heuristic Search
68
A* Algorithm의 예
8(16)-puzzle에서 City block distance (Manhattan
distance)를 사용
: 절대 overestimate하지 않는다…
 h1 ::: the number of titles in wrong position
 h2 ::: Manhattan distance
결과비교
d
14
24
IDS
A*(h1)
A*(h2)
3473941(2.83)
539(1.42)
73(1.24)
불가능
39135(1.48)
1641(1.26)
Heuristic Search
69
Inventing heuristic functions
Relaxed problem
(a) 타일은 인접한 장소로 움직일 수
있다
(b) 타일은 빈자리로 이동할 수 있다
(c) 타일은 다른 위치로 이동할 수 있다
h(n)=max(h1(n), …,hn(n))  실제 값과
가장 가까이 예측하는 함수가 좋다
Heuristic Search
70
Heuristics for CSPs
Most-constrained-variable heuristics
:: forward checking  the variable with
fewest possible values is chosen to
have a value assigned
Most-constraining-variable heuristics
:: assigning a value to the variable that
is involved in the largest number of
constraints on the unassigned variables
(reducing the branching factor)
Heuristic Search
71
Example
B
A
GREEN
RED
C
E
F
D
Heuristic Search
72
Memory Bounded Search
Simplified Memory-Bound A*
A
B
G
10
10+5=15
C
E
10
30+5=35
8+5=13
D
10
20+5=25
8
10
F
10
H
20+0=20
30+0=30
A
J
8
8
24+0=24
I
16
24+0=24
K
24+5=29
A
A
A
12
12
8
16+2=18
13
B
13(15)
15
G
G
B
15
13
H
13
)
8
18(
/
A
A
A
A
15(24)
20(24)
15
15(15)
G
B
B
G
B
15
20(
8
)
15
24
C
25(
/
8
8
24(
I
24
Heuristic Search
)
)
D
20
73
Machine Evolution
Evolutions
Generations of descendants


Production of descendants changed from
their parents
Selective survival
Search processes
Searching for high peaks in the
hyperspace
Heuristic Search
75
Applications
Function optimization

The maximum of a function :::
John Holland
Solving specific problems
•
•
Control reactive agents
Classifier systems
Genetic programming
Heuristic Search
76
A program expressed as a tree
+
3
/
x
5
7
4
Heuristic Search
77
A robot to follow the wall
around forever
Primitive functions : AND, OR, NOT, IF
Boolean functions




AND(x,y) = 0 if x = 0; else y
OR(x,y) = 1 if x = 1; else y
NOT(x) = 0 if x = 1; else 1
IF(x,y,Z) = y if x = 1; else z
Actions

North, east, south, west
Heuristic Search
78
A robot to follow the wall
around forever
All of the action functions have their
indicated effects unless the robot
attempts to move into the wall
Sensory inputs ::: n, ne, e, se, s , sw, w,
nw
만약 함수의 수행결과가 값이 없으면
중지
Heuristic Search
79
A robot in a Grid World
nw
n
ne
w
te
xt
e
sw
s
se
Heuristic Search
80
A wall following program
IF
AND
east
IF
OR
NOT
AND
n
ne
south
e
IF
OR
NOT
AND
e
se
west
s
OR
s
Heuristic Search
NOT
sw
north
w
81
The GP process
Generation 0 (0세대): start with a population
of random programs with functions, constants,
and sensory inputs

5000 random programs
Final : Generation 62  60 steps 동안 벽에
있는 방을 방문한 횟수로 평가  32 cells이면
perfects; 10곳에서 출발하여 fitness 측정
Heuristic Search
82
Generation of populations I
(i+1)th generation


10%는 i-the generation에서 copy  5000
populations에서 무작위로 7개를 선택하여
가장 우수한 것을 선택 (tournament
selection)
90%는 앞의 방법으로 두 프로그램(a
mother, a father)을 선택하여, 무작위로
선정한 father의 subtree를 mother의
subtree에 넣는다 (crossover)
Heuristic Search
83
Crossover
Randomly chosen
crossover points
NOT
IF
NOT
AND
AND
AND
OR
se
se
n
se
NOT
se
IF
s
NOT
e
nw
nouth
OR
Father program
west
w
south
Mother program
IF
NOT
s
west
Child program
Heuristic Search
AND
se
NOT
nouth
84
Generation of populations II

Mutation : 1%를 tournament로 선정 
무작위로 선택한 subtree를 제거하고,
1세대에서 개체를 생성하는 방법으로
만들어서 끼워넣는다
Heuristic Search
85
Evolving a wall-following robot
개별 프로그램의 예



(AND (sw) (ne)) (with fitness 0)
(OR (e) (west) (with fitness 5(?))
the best one ::: fitness = 92 (어떤 때)
Heuristic Search
86
The most fit individual in
generation 0
Heuristic Search
87
The most fit individuals in
generation 2
Heuristic Search
88
The most fit individuals in
generation 6
Heuristic Search
89
The most fit individuals in
generation 10
Heuristic Search
90
Fitness as a function of
generation number
350
300
Fitiness
250
200
150
100
50
0
0
1
2
3
4
5
6
7
8
9
10
Generation number
Heuristic Search
91
숙제
Specify fitness functions for use in evolving
agents that


Control an elevator
Control stop lights on a city main street
Determine what the words genotype and
phenotype in evolutionary theory?
Why do you think mutation might or might no
be helpful in evolutionary processes that use
crossover?
Heuristic Search
92
When to Use Search Techniques?
The search space is small, and


There is no other available techniques, or
It is not worth the effort to develop a more
efficient technique
The search space is large, and


There is no other available techniques, and
There exist “good” heuristics
Heuristic Search
93
Summary
Heuristic function
Best-first search
Admissible heuristic and A*
A* is complete and optimal
Consistent heuristic and repeated states
Heuristic accuracy
IDA*
Heuristic Search
94