Transcript research.cs.queensu.ca

CISC453 Winter 2010
Classical Planning
AIMA3e Ch 10
2
What is Planning?
 generate sequences of actions to perform tasks and
achieve objectives
 involving states, actions & goals
 a search for a solution over an abstract space of plans
 the assumptions for classical planning problems
 fully observable
 we see everything that matters
 deterministic
 the effects of actions are known exactly
 static
 no changes to environment other than those caused by agent
actions
 discrete
 changes in time and space occur in quantum amounts
 single agent
 no competition or cooperation to account for
Classical Planning
3
Background: Planning
 real systems help humans in practical applications
 design and manufacturing environments
 military operations, games, space exploration
 notes:
 the methods of Ch 10 (Classical Planning)
 assume the classical environment assumptions
 in Ch 11 (Planning & Acting in the Real World)
 we'll introduce methods to handle real-world situations where the
classical assumptions may not hold
Classical Planning
4
Planning: Language
 What is a good language?
 expressive enough to describe a wide variety of problems
 restrictive enough for efficient algorithms to operate on it
 planning algorithm should be able to take advantage
 of the logical structure of the problem
 historical AI planning languages
 STRIPS was used in classical planners
 Stanford Research Institute Problem Solver
 ADL addresses expressive limitations of STRIPS
 Action Description Language
 adds features not in STRIPS
 negative literals, quantified variables, conditional effects, equality
 we'll use one version of the now de facto standard PDDL
Classical Planning
5
Planning language
 there's now wide agreement on some planning
language basics
 one key is adopting of a factored representation for
states
 each state is represented as a collection of variables
 this contrasts with
 the atomic representation typical of our previous search
algorithms (recall the Romanian driving problem: state = city)
 the language we'll use is a variant of PDDL
 Planning Domain Definition Language
 to see its expressive power, recall propositional agent in the
Wumpus World, which requires 4Tn2 actions to describe a
movement of 1 square
 PDDL captures this with a single Action Schema
Classical Planning
6
State & Action Representations
 each state is represented as a conjunction of fluents
 these are ground, functionless atoms
 in addition, we'll use Database semantics
 1. the Close World Assumption
 fluents not explicitly mentioned are false
 2. the Unique Names Assumption
 different ground terms are different objects: Plane1, Plane2
 this state representation allows alternative algorithms
 it can be manipulated either by logical inference techniques or by
set operations
 actions are defined by a set of action schemas
 these implicitly define the ACTIONS(s) & RESULT(s, a)
functions required to apply search techniques
Classical Planning
7
Action Schemas
 PDDL & the Frame Problem
 recall the representational issue of capturing what stays the
same given some action
 in PDDL we specify what changes, and if something is not
mentioned, it stays the same
 Action Schemas
are a lifted representation (recall Generalized Modus Ponens)
lifts from propositional logic to a restricted subset of FOL
each one stands for a set of variable-free actions
each includes the schema name, list of variables used,
preconditions & effects
 we consider variables as universally quantified, choose values
to instantiate them
 PRECOND: defines states in which an action can be executed
 EFFECT: defines the result of executing the action




Classical Planning
8
Action Schemas
 each represents a set of variable-free actions
 form: Action Schema = predicate + preconditions + effects
 example:
Action(Fly(p, from, to),
PRECOND: At(p, from)  Plane(p)  Airport(from)  Airport(to)
EFFECT: ¬AT(p, from)  At(p, to))
 an action schema in which (p, from, to) need to be instantiated
 the action name and its parameter list
 its preconditions
 a conjunction of function-free literals
 its effects
 a conjunction of function-free literals
Classical Planning
9
Applying Action Schemas
 action a is applicable in state s
 if its preconditions are satisfied in s
 its preconditions are entailed by s
 given variables, there can be multiple applicable instantiations
 for v variables in a domain with k unique object names, worst
case time to find applicable ground actions is O(vk)
 leads to 1 approach for solving PDDL planning problems
 propositionalize by replacing action schemas with sets of ground
actions & applying a propositional solver like SATPlan
 impractical for large v & k
 result of executing a in s is s'




s' is derived from s & action a's EFFECTs
remove -ve literal fluents (the delete list, DEL(a))
add +ve literals in EFFECTs (the add list, ADD(a))
RESULT(s, a) = (s - DEL(a))  ADD(a)
Classical Planning
10
Action Schemas
 PDDL schemas
 1. variables & ground terms
 variables in effects must also be in precondition
 so matching to state s yields results with all variables bound
 i.e. that contain only ground terms
 2. handling of time
 no explicit time terms, unlike axioms we saw for logical agents
 instead time is implicitly represented in PDDL schemas
 preconditions always refer to time: t
 effects always refer to time: t + 1
 3. a set of schemas defines a planning domain
 a specific problem adds initial & goal states
Classical Planning
11
Initial States, Goals, Solutions
 initial state
 conjunction of ground terms
 goal
 conjunction of +ve & -ve literals
 both ground terms & those containing variables
 variables are treated as existentially quantified
 solution
 a sequence of actions ending in s that entails the goal
 example:
 Plane(P1)  At (P1, SFO) entails At(p, SFO)  Plane (p)
 defines planning as a search problem
Classical Planning
12
Some Language Semantics
 How do actions affect states?
 an action is applicable in any state that satisfies its
precondition(s)
 applicability involves a substitution  for the variables in the
PRECOND
 example
State:
At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK) 
Airport(SFO)
Satisfies PRECOND of Fly action:
At(p, from)  Plane(p)  Airport(from)  Airport(to)
With substitution  ={p/P1, from/JFK, to/SFO}
Thus the action is applicable.
Classical Planning
13
Language Semantics
 result of executing action a in state s is the state s’
 s’ is same as s except
 any positive literal P in the effect of a is added to s’
 any negative literal ¬P is removed from s’
 example
Fly(p, from, to)
EFFECT: ¬AT(p, from)  At(p, to),  ={p/P1, from/JFK, to/SFO}
s: At(P1,JFK)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK) 
Airport(SFO)
s': At(P1,SFO)  At(P2,SFO)  Plane(P1)  Plane(P2)  Airport(JFK) 
Airport(SFO)
Classical Planning
14
Example: air cargo transport
 simple actions to illustrate
 moving cargo between New York(JFK) & San Francisco(SFO)
Init(At(C1, SFO)  At(C2,JFK)  At(P1,SFO)  At(P2,JFK)  Cargo(C1)  Cargo(C2) 
Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO))
Goal(At(C1,JFK)  At(C2,SFO))
Action(Load(c, p, a)
PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)
EFFECT: ¬At(c, a) In(c, p))
Action(Unload(c, p, a)
PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)
EFFECT: At(c, a)  ¬In(c, p))
Action(Fly(p, from, to)
PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)
EFFECT: ¬ At(p, from)  At(p, to))
Classical Planning
15
Air Cargo Example
 this planning domain involves 3 actions
 and 2 affects predicates, In(c, p) & At(x, a)
 properly maintaining At predicates could be an issue
 where is cargo that is in a plane?
 without quantifiers, problem of expressing what happens to cargo
that is in a plane
 solve by treating At predicate as applying only to cargo after it is
unloaded, in effect not At anywhere when it is in a plane
 another issue is that this representation allows "empty"
actions that produce contradictory effects
 Fly(P1, JFK, JFK) yields At(P1, JFK)  ¬At(P1, JFK)
 can ignore issues like this, rarely produce incorrect plans, or
 add inequality precondition: from  to
Classical Planning
16
Air Cargo Example
 the solution is pretty obvious
Init(At(C1, SFO)  At(C2,JFK)  At(P1,SFO)  At(P2,JFK)  Cargo(C1)  Cargo(C2) 
Plane(P1)  Plane(P2)  Airport(JFK)  Airport(SFO))
Goal(At(C1,JFK)  At(C2,SFO))
Action(Load(c, p, a)
PRECOND: At(c, a) At(p, a) Cargo(c) Plane(p) Airport(a)
EFFECT: ¬At(c, a) In(c, p))
Action(Unload(c, p, a)
PRECOND: In(c, p) At(p, a) Cargo(c) Plane(p) Airport(a)
EFFECT: At(c, a)  ¬In(c, p))
Action(Fly(p, from, to)
PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)
EFFECT: ¬ At(p, from)  At(p, to))
[Load(C1, P1, SFO), Fly(P1, SFO, JFK), Unload (C1, P1, JFK),
Load(C2, P2, JFK), Fly(P2, JFK, SFO), Unload(C2, P2, SFO)]
Classical Planning
17
Example: flat tire problem
 simple actions to illustrate
 changing a flat tire (in a bad neighbourhood)
Init(At(Flat, Axle)  At(Spare, Trunk))
Goal(At(Spare, Axle))
Action(Remove(Spare, Trunk)
PRECOND: At(Spare, Trunk)
EFFECT: ¬At(Spare, Trunk)  At(Spare, Ground))
Action(Remove(Flat, Axle)
PRECOND: At(Flat, Axle)
EFFECT: ¬At(Flat, Axle)  At(Flat, Ground))
Action(PutOn(Spare, Axle)
PRECOND: At(Spare, Ground) ¬At(Flat, Axle)
EFFECT: At(Spare, Axle)  ¬At(Spare, Ground))
Action(LeaveOvernight
PRECOND:
EFFECT: ¬ At(Spare, Ground)  ¬ At(Spare, Axle)  ¬ At(Spare, Trunk)  ¬
At(Flat, Ground)  ¬ At(Flat, Axle) )
Classical Planning
18
Flat Tire example
 highly simplified, abstracted version of the problem of
constructing a plan to fix a flat tire
 4 actions
 note capturing of "bad neighbourhood" as tires "disappearing"
if the car is left overnight
 the simple solution
[Remove (Flat, Axle), Remove(Spare, Trunk), PutOn(Spare,
Axle)]
Classical Planning
19
Example: Blocks World
 simple actions to illustrate
 building a tower in Blocks World
Init(On(A, Table)  On(B, Table)  On(C,A)  Block(A)  Block(B)  Block(C) 
Clear(B)  Clear(C))
Goal(On(A,B)  On(B,C))
Action(Move(b, x, y)
PRECOND: On(b, x)  Clear(b)  Clear(y)  Block(b)  (b  x)  (b  y)  (x  y)
EFFECT: On(b, y)  Clear(x)  ¬ On(b, x)  ¬ Clear(y))
Action(MoveToTable(b, x)
PRECOND: On(b, x)  Clear(b)  Block(b)  (b  x)
EFFECT: On(b, Table)  Clear(x)  ¬ On(b, x))
 simple plan given the initial state
 note PRECOND of Move uses inequality to prevent empty moves
 Move(B,C,C)
Classical Planning
Blocks World example
20
 illustration of the problem
Classical Planning
Blocks World example
21
 problem domain:
 stack cube-shaped blocks that are on a table
 Notes:
 only 1 block can fit on top of another
 moves can be onto another block or onto the table
 FOL would use quantifiers to express
 there's no block on top of some other block
 without quantifiers, PDDL requires Clear(x) predicate
 Move action schema must be complemented by MoveToTable
to avoid errors with the Clear predicate
 since Clear(Table) is always true
 residual problem if bind y to Table in Move(b, x, y)
 search space grows, though answers still correct
 fix with a Block(m) predicate & adding Block(b)  Block(y) to the
preconditions for Move
Classical Planning
22
Complexity & Classical Planning
 first, we distinguish 2 versions of the planning
problems
 PlanSAT:
 is there a plan that solves the problem?
 Bounded PlanSAT:
 is there a solution of k or fewer steps?
 this can be used to find optimal plans
 so long as we don't allow functions the number of states is
finite & both categories are decidable
 if we allow functions, PlanSAT becomes semi-decidable (may
not terminate on unsolvable problems) though Bounded
PlanSAT remains decidable even with functions
Classical Planning
Complexity & Classical Planning
23
 sadly, in general, both problems are NP-hard
 putting restrictions on the expressiveness may help
 for example, disallowing -ve preconditions lets PlanSAT reduce
to P
 fortunately for planners, many useful problems are easier than
the worst case
 for blocks world & air cargo domains, Bounded PlanSAT is NPComplete, while PlanSAT is in P!
 so, optimal planning is hard, sometimes sub-optimal planning is
easy
 reminder of complexity category relationships
Planning with state-space search
 given the problem formulation
 we can use search algorithms to generate plans
 systematic heuristic search
 local search (while recording trail of states)
 search can be
 in either the forward or the backward direction
 recall the bi-directional example from search topic
 problems related to non-invertible successor functions
 Progression planners
 do a forward state-space search
 consider the effects of all possible actions in a given state
 Regression planners
 do a backward state-space search
 to achieve a goal
 consider what must have been true in the previous state
 from effects through actions to preconditions
24
Progression & Regression
25
 partial samples: plane moving example
Classical Planning
26
Progression algorithm
 formulation as a state-space search problem
 Initial state = initial state of the planning problem
 literals not appearing are false
 Actions = apply those whose preconditions are satisfied
 add positive effects, delete negative
 Goal test = does the state satisfy the goal
 Step cost = for simplicity, each action costs 1
 recall, functions are not permitted in the representation
 any graph search that is complete is a complete planning
algorithm
 for example, A*
Classical Planning
27
Progression algorithm
 Progression approach
 while complete is inefficient
 any applicable action is a possible next step
 typically its problems are related to branching factor
 (1) irrelevant actions
 just because the preconditions of an action are satisfied
 does not mean that it should be done
 (2) requirement for a good heuristic
 in order for improved efficiency of search
Classical Planning
28
Progression: issues
 illustrate with the air cargo example
 assume 10 airports, 5 planes & 20 pieces of cargo at each
 problem: move all cargo at airport A to airport B
 solution is pretty simple, but uninformed search has serious
complexity difficulties
 just considering planes: 50 x 9 = 450 possible flights
 suppose all planes & packages were at 1 airport: 10450 possible
actions (200 x 50 + 50 x 9)
 if 2000 actions available on average, 200041 nodes in search
graph
 fortunately, good (domain-independent) heuristics are
possible, & can be derived automatically
Classical Planning
29
Regression algorithm
 what about Regression planners
 issues:
 How do we determine predecessors?
 as we saw for bi-directional search, the problem may not allow
predecessor determination (n-Queens)
 fortunately PDDL formulation facilitates it
 terminology: predecessor is g', action a, goal g
 g' = (g - Add(a))  Precond(a), since any effects added by a
might not have been true previously & preconditions must have
been met or could not execute a
 this process requires partially uninstantiated actions & states
 use standardized variable names (avoid clashes, retain generality
of variables)
 implicitly quantify over these so 1 description summarizes
possibility of using any appropriate ground term
Classical Planning
30
Regression: relevant actions
 which action?
 forward uses applicable actions (their preconditions are met)
 backward uses relevant actions (they could be the last step
leading to current state)
 relevant: at least 1 effect (+ve or -ve) must unify with a term
in the (current) goal & there can't be any effect that negates
a term in the goal
 process: use the action formed by substituting the most
general unifier into the standardized action schema
 more formal definition of relevant actions
 if goal g contains literal gi & action schema A (standardized to
produce A') has effect literal ej' where Unify (gi, ej') =  & a' =
SUBST(, A') & there's no effect in a' that is the negation of a
literal in g, then a' is a relevant action toward g
 terminates when a predecessor is satisfied by the initial state
Classical Planning
31
Progression & Regression
 status of the regression approach
 1. there's a lower branching factor in the backward direction,
the result of only considering relevant actions
 2. but each state is really a set of states
 the state defined by true or false for the ground fluents, and the
states not mentioned (see mid p 374)
 this representation of stages in the backward search as sets of
states complicates the process of developing heuristics
 progression versus regression
 the greater availability of accurate heuristics for forward
search has resulted in it being used in many more planning
systems
Classical Planning
32
Heuristics for planning
 the complexity of planning problems
 means that we need good heuristics
 below: 2 general approaches to finding admissible heuristics
 recall that an admissible h(n) (i.e. one that never overestimates
cost) allows A* search for optimal solutions
 recall from our (informed search) discussion of finding
heuristics
 as the optimal solution to a relaxed version of the problem
 from problem decomposition
 if we can make the subgoal independence assumption we can
approximate the cost of solving a conjunction of subgoals as the sum
of the costs of solving the corresponding subproblems
 (1) relaxation: search problem viewed as a graph allows
 (a) adding edges to make a path to a solution easier to find
 (b) merging states in an abstraction of the original problem so
searching is in a space with fewer nodes
Classical Planning
Heuristics by problem relaxation
33
 (a) adding edges to the search space
 (1) ignore all preconditions
 assume all actions are applicable in all states, so any goal fluent
takes just a single step
 apparently #steps ≈ #unsatisfied goals, but
 note: some actions may achieve > 1 goal, & others may undo goals
 ignoring the latter, h(n) = min # actions for which the union of their
effects satisfies the goal
 this is a version of the "set-cover problem", known to be NP-hard!
 a greedy algorithm yields set covering within log n of the true min (n
is # literals in goal), but is not guaranteed admissible
 (2) ignore selected PRECONDs of actions
 as in the N-puzzle heuristics: misplaced tiles, manhattan distance
 the factored representation in action schemata for planning
problems allows automated derivation of such heuristics
 (3) ignore delete lists
 if all goals & preconditions have only +ve literals (true for many
problems anyway & others can be converted to this form)
Classical Planning
Heuristics by problem relaxation
34
 (a) adding edges to search space
 (3) ignore delete lists
 with only +ve literals in all goals & preconditions
 then remove -ve literals from effects
 now steps are monotonic progress towards the goal w/o undoing
 but, it's still NP-hard to find an optimal solution, though it can
be approximated in polynomial time with hill climbing
 summary
 adding edges relaxes a planning problem to yield heuristics but
they're still expensive to calculate
 (b) reduce nodes by abstraction to merge states
 return to the air cargo example
 10 airports, 50 planes, 200 packages means the general case has
5010 x 20050+10 or 10155 states
 a specific problem: say all pkgs are at 5 airports & all pkgs at any
airport have 1 destination
Classical Planning
Heuristics by problem relaxation
35
 (b) reduce nodes by abstraction to merge states
 in the air cargo example, in general, 10155 states
 a specific problem has all pkgs at 5 airports & pkgs at any airport
have 1 destination
 an abstraction of this version omits At fluents except those re: 1
plane & 1 pkg at each of 5 airports, reducing # states to 510 x 55+10
or 1017 states
 so a solution is shorter & an admissible h(n) & is extensible to the
original problem by adding more Load & Unload actions
 problem decomposition approaches for h(n)'s
 the relevant pattern is:
 subdivide problem  solve subproblems  combine solutions
 the subgoal independence assumption
 is useful in developing a heuristic as a sum of subgoal costs
 is too optimistic if there are -ve interactions between subplans
 is too pessimistic (inadmissible) when there are redundant actions
in subplans
Heuristics by decomposition
36
 problem decomposition approaches for h(n)'s
 notation: represent the goal as a set of fluents G, where
disjoint subsets of G are G1, …Gn, for which we find plans P1,
… Pn & then use their costs to estimate cost for G
 Cost(Pi) is an estimate, a possible heuristic
 maxi Cost(Pi) is admissible, overly optimistic
 sumi Cost(Pi) is not generally admissible unless all Gi & Gj are
independent (Pi does not affect preconds or goals of Pj) in
which case the sum is admissible & more accurate
Classical Planning
Heuristics by decomposition
37
 the challenge of abstraction is
 to find one that has a significantly lower total cost (defining the
abstraction + doing the abstracted search + mapping back to
original problem) than the original problem
 the pattern database techniques may be useful
 recall them from informed search discussion of heuristics
 amortize the cost of building the database over multiple problem
solution instances
 the FF (FASTFORWARD) planning system is a hybrid
 it uses heuristics from a planning graph (our next discussion)
 it uses local search (storing plan steps)
 at a plateau or local maximum it does iterative deepening
systematic search for a better state or gives up & restarts
38
Planning Graphs
 Planning Graphs
 are an alternative intermediate representation/data structure
 they are polynomial complexity approximations of full
(exponential) trees of all states & actions
 they can't answer the question: is G reachable?
 though they are correct when they say it's not reachable
 they provide an estimate of the number of steps to G
 any such estimate is optimistic, so is an admissible heuristic
 planning graphs
 provide a possible basis for better search heuristics
 and they can also be used directly, for extracting a solution to a
planning problem, by applying the GRAPHPLAN algorithm
Classical Planning
39
Planning Graphs
 a planning graph
 is a directed graph organized in levels
 the levels of nodes correspond to time steps in a plan
 they consist of alternating S levels (in which the nodes are
fluents that hold) & A levels (in which the nodes are actions that
might be applicable)
 level S0 is the initial state
 each level is a set of literals Si or a set of actions Ai
 Si: all the literals that could be true at that time step
 depending on the actions executed at the previous steps
 Ai: all actions that could have PRECONDs satisfied at that step
 depending on which of the literals actually hold
Classical Planning
40
Planning Graphs
 explanation regarding could on the previous slide
 a planning graph only captures a restricted subset of the
possible -ve interactions
 so a literal might appear at a level earlier than it actually would
(if it would at all), though it never appears too late
 despite this error, the level j of the first appearance is a good
estimate of how difficult it is to achieve from the initial state
 refers to the approximate nature of the state/action lists
 in part, this allows efficient construction by recording a restricted
subset of possible -ve interactions among actions
 note:
 planning graphs apply only for propositional problems
 action schemas for problems with variables can be
propositionalized
 their advantages may make this worthwhile, despite the
increased size of the problem description
Classical Planning
Planning Graphs
 our simple example: "have your cake & eat it too"
 the problem description
Init(Have(Cake))
Goal(Have(Cake)  Eaten(Cake))
Action(Eat(Cake)
PRECOND: Have(Cake)
EFFECT: ¬Have(Cake)  Eaten(Cake))
Action(Bake(Cake)
PRECOND: ¬ Have(Cake)
EFFECT: Have(Cake))
 the corresponding planning graph
41
Planning Graph Cake Example
42
 start at level S0, determine action level A0 & next level S1
 A0: all actions whose preconditions are satisfied in the previous
level (initial state)
 actions are shown in rectangular boxes
 lines connect PRECONDs at S0 to EFFECTs at S1
 also, for each literal in Si, there's a persistence action (square box)
& line to it in the next level Si+1
 level A0 contains the actions that could occur
 conflicts between actions are represented by arcs: mutual
exclusion or mutex links
Planning Graph Cake Example
43
 level S1 contains all the literals that could result
 from picking any subset of actions in A0
 so S1 is a belief state consisting of the set of all possible states
 each is a subset of literals with no mutex links between members
 conflicts between literals that cannot occur together are
represented by the mutex links.
 the level generation process is repeated
 eventually consecutive levels are identical: leveling off
Planning Graph Cake Example
44
 mutex relation holds between 2 actions at a level when
 1. inconsistent effects
 one action negates the effect of another
 2. interference
 an effect of one action negates a precondition of the other
 3. competing needs
 a precondition of one action is mutex with a precondition of the
other
Classical Planning
Planning Graph Cake Example
45
 mutex relation holds between 2 literals at a level when
 1. one is the negation of the other
 2. if each possible action pair that could achieve the literals is
mutex (Have(Cake) & Eaten(Cake) at S1)
 this is termed inconsistent support
Classical Planning
46
Planning Graphs & Heuristics
 Planning Graphs
 construction has complexity polynomial in the size of the
planning problem
 given l literals, a actions, & a PG of n levels: O(n(a + l)2)
 the completed PG
 provides information about the problem & candidate heuristics
 1. a goal literal g that does not appear in the final level cannot be
achieved by any plan
 2. the level cost, the level at which a goal literal first appears, is
useful as a cost estimate of achieving that goal literal
 note that level cost is admissible, though possibly inaccurate
since it counts levels, not actions
 we could find a better alternative level cost by using a serial
planning graph variation, restricted to one action per level
 mutex links between every pair of actions except persistence actions
Classical Planning
47
Planning Graphs & Heuristics
 Planning Graph provides
 possible heuristics for the cost of a conjunction of goals
 1. max-level: highest level of any conjunct in the goal
 admissible, possibly not accurate
 2. level sum: the sum of level costs of conjuncts in the goal
 incorporates the subgoal independence assumption
 so may be inadmissible to degree the assumption does not hold
 3. set-level: level where all goal conjuncts are present without
mutex links
 this one dominates the max level heuristic
 & is good where there are interactions among subplans
Classical Planning
48
Planning Graphs & Heuristics
 a Planning Graph is a relaxed version of the problem
 if a goal literal g does not appear, no plan can achieve it, but
if it does appear, is not guaranteed to be achievable
 why?
 the PG only captures pairwise conflicts & there could be higher
order conflicts
 likely not worth the computational expense of checking for them
 similar to Constraint Satisfaction Problems where arc consistency
was a valuable pruning tool
 3-consistency or even higher order consistency would have made
finding solutions easier but was not worth the additional work
 an example where PG fails to indicate unsolvable problem
 blocks world problem with goal of A on B, B on C, C on A
 any pair of subgoals are achievable, so no mutexes
 problem only fails at stage of searching the PG
Classical Planning
49
The GRAPHPLAN Algorithm
 the GRAPHPLAN algorithm
 generates the Planning Graph & extracts a solution from it
function GRAPHPLAN(problem) return solution or failure
graph  INITIAL-PLANNING-GRAPH(problem)
goals  CONJUNCTS(problem. GOAL)
nogoods  an empty hash table
for tl = 0 to  do
if goals all non-mutex in St of graph then
solution  EXTRACT-SOLUTION(graph, goals,
NUMLEVELS(graph), nogoods)
if solution  failure then return solution
if graph and nogoods have both leveled off then return failure
graph  EXPAND-GRAPH(graph, problem)
Classical Planning
50
Example: Spare Tire Problem
 recall the spare tire problem
Init(At(Flat, Axle)  At(Spare, Trunk))
Goal(At(Spare, Axle))
Action(Remove(Spare, Trunk)
PRECOND: At(Spare, Trunk)
EFFECT: ¬At(Spare, Trunk)  At(Spare, Ground))
Action(Remove(Flat, Axle)
PRECOND: At(Flat, Axle)
EFFECT: ¬At(Flat, Axle)  At(Flat, Ground))
Action(PutOn(Spare, Axle)
PRECOND: At(Spare, Ground) ¬At(Flat, Axle)
EFFECT: At(Spare, Axle)  ¬At(Spare, Ground))
Action(LeaveOvernight
PRECOND:
EFFECT: ¬ At(Spare, Ground)  ¬ At(Spare, Axle)  ¬ At(Spare, Trunk) 
¬ At(Flat, Ground)  ¬ At(Flat, Axle) )
Classical Planning
GRAPHPLAN Spare Tire Example
51
 Notes:
 this figure shows the complete Planning Graph for the problem
 arcs show mutex relations
 but arcs between literals are omitted to avoid clutter
 it also omits unchanging +ve literals (for example, Tire(Spare))
& irrelevant -ve literals
 bold boxes & links indicate the solution plan
Classical Planning
GRAPHPLAN Spare Tire Example
52
 S0 is initialized to 5 literals
 from the problem initial state and the CWA literals
 no goal literal in S0 so EXPAND-GRAPH add actions
 those with preconditions satisfied in S0
 also adds persistence actions for literals in S0
 adds the effects at level S1, analyzes & adds mutex relations
 repeat until the goal is in level Si or failure
Classical Planning
GRAPHPLAN Spare Tire Example
53
 EXPAND-GRAPH adds constraints: mutex relations
 inconsistent effects (action x vs action y)
 Remove(Spare, Trunk) & LeaveOvernight: At(Spare, Ground) & ¬At(Spare,
Ground)
 interference (effect negates a precondition)
 Remove(Flat, Axle) & LeaveOvernight: At(Flat, Axle) as PRECOND &
¬At(Flat, Axle) as EFFECT
 competing needs (mutex preconditions)
 PutOn(Spare, Axle) & Remove(Flat, Axle): At(Flat, Axle) & ¬At(Flat, Axle)
 inconsistent support (actions to produce literals are mutex)
 in S2, At(Spare, Axle) & At(Flat, Axle)
Reminder: GRAPHPLAN Algorithm
54
 the GRAPHPLAN algorithm both generates the
Planning Graph & extracts a solution from it
function GRAPHPLAN(problem) return solution or failure
graph  INITIAL-PLANNING-GRAPH(problem)
goals  CONJUNCTS(problem. GOAL)
nogoods  an empty hash table
for tl = 0 to  do
if goals all non-mutex in St of graph then
solution  EXTRACT-SOLUTION(graph, goals,
NUMLEVELS(graph), nogoods)
if solution  failure then return solution
if graph and nogoods have both leveled off then return failure
graph  EXPAND-GRAPH(graph, problem)
Classical Planning
GRAPHPLAN Spare Tire Example
55
 in S2, the goal literals exist, & are not mutex
 just 1 goal literal so obviously not mutex with any other goal
 since a solution may exist, EXTRACT-SOLUTION tries to find it
 EXTRACT-SOLUTION may search backwards for a solution
 initial state: last level of the PG + goals of the planning problem
 actions from Si
 select a set of conflict-free actions in Ai-1 with effects covering the goals
 conflict free = no 2 actions are mutex & no pair of preconditions are mutex
 goal: reach a state at level S0 such that all goals are satisfied
 cost: 1 for each action
GRAPHPLAN Solutions
56
 if EXTRACT-SOLUTION fails
 at that point it records (level, goals) as a "no-good"
 subsequent calls can fail immediately if they require the same
goals at that level
 we already know planning problems are computationally hard
 require good heuristics
 greedy search with level cost of literals as a heuristic works
well
 1. pick literal with highest level cost
 2. to achieve it, pick actions with easier preconds
 action with smallest sum (or max) of level costs for its preconds
57
GRAPHPLAN Solutions
 alternative to backward search for a solution
 EXTRACT-SOLUTION could formulate a Boolean CSP
 variables are actions at each level
 values are Boolean: an action is either in or out of the plan
 constraints are mutex relations & the need to satisfy each goal &
precondition
Classical Planning
GRAPHPLAN Termination
58
 does GRAPHPLAN terminate?
 does it stop & return failure if there's no solution
 the general rationale is presented here
 for some proof details see the textbook & its Ghallab reference
 background
 recall that levelled off means consecutive PG levels are identical
 now note that a graph may level off before a solution can be
found, on a problem for which there is a solution
 example of 2 airports, 1 plane & n pieces of cargo at one of
them, plan required to move all n to the other airport
 but only 1 piece at a time fits in the plane
 graph levels off at level 4, from which full solution can't be
extracted (that would require 4n - 1 steps)
 we need to take account of the no-goods (goals that were not
achievable) as well
 these may decline even after the graph levels off
GRAPHPLAN Termination
59
 does GRAPHPLAN terminate?
 given that no-goods might continue to decrease after the graph
levels off
 we now require a proof that both the graph & no-goods will
level off
 literals increase monotonically
 once a literal appears, its persistence action causes it to stay
 actions increase monotonically
 once a literal that is a precondition appears, the action stays
 mutexes decrease monotonically
 though the graph simplifying conventions may not show it
 if 2 actions are mutex at Ai, they are also mutex at all previous levels
where they appear
GRAPHPLAN Termination
60
 does GRAPHPLAN terminate?
 no-goods decrease monotonically
 if a set of goals is not achievable at level i, they are not achievable
at any previous level
 so literals & actions increase monotonically
 given a finite number of actions and literals, they will level off
 and mutexes & no-goods decrease monotonically
 since there cannot be fewer than 0, they must level off
 if the graph & no-goods have levelled off & a goal is missing or
is mutex with another goal, the algorithm can stop & return
failure
Alternative Planning Approaches
61
 the following slides outline brief discussions of 4
alternative approaches to solving planning problems
 1. convert the PDDL description into a form solvable by a
propositional logic satisfiability algorithm: SATPlan
 2. convert into a situational calculus representation solvable
by First-Order Logic inference techniques
 3. represent & solve as a Constraint Satisfaction Problem
 4. use a "plan refinement" approach to searching in a space of
partially ordered plans
Classical Planning
Planning with Propositional Logic
62
 1. form a propositional logic satisfiability problem for
submission to SATPlan
 iteratively tries plans of increasing length (increasing number
time steps) up to the Tmax parameter value, so finds the
shortest plan, if one exists
 the SATPlan algorithm
 finds models for a (very long) PL sentence that includes initial
state, goal, successor-state axioms, precondition axioms, &
action-exclusion axioms
 assigns true to the actions that are part of the correct plan &
false to the others
 if the planning is unsolvable the sentence will be unsatisfiable
 any model satisfying the sentence will be a valid plan
 an assignment that corresponds to an incorrect plan will not be a
model because of inconsistency with the assertion that the goal is
true
Classical Planning
Planning with Propositional Logic
63
 translation from PDDL to PL form for SATPlan
 the process involves 6 steps
 step 1: propositionalize actions, replacing each schema with a
set of ground actions (constants substituted for variables)
 step 2: define an initial state asserting F0 for all fluents in the
initial state, ¬F0 for all fluents not in the initial state
 step 3: propositionalize the goal - for each variable replace
literals containing it with a disjunction over constants
 step 4: include successor-state axioms for each fluent
Ft+1  ActionCausesFt  (Ft  ¬ActionCausesNotFt), where
 ActionCausesF is disjunction of all ground actions with F in their
add List
 ActionCausesNotF is disjunction of all ground actions with F in
their delete list
 air cargo example: axioms are required for each plane, airport and
time step!
Classical Planning
Planning with Propositional Logic
64
 the 6 steps from PDDL to SATPlan format
 step 5: add precondition axioms (for every ground action A,
add At  PRE(A)t
 step 6: add action exclusion axioms to say every action is
distinct from every other action
 submit to SATPlan
Classical Planning
65
The SATPLAN Algorithm
 the algorithm
 note that in AIMA 3e, this is first presented in the earlier
section on propositional logic agents
function SATPLAN(init, transition, goal, Tmax) returns solution or failure
inputs: init, transition, goal form a description of the planning problem
Tmax, an upper limit to the plan length
for t = 0 to Tmax do
cnf  TRANSLATE-TO-SAT(init, transition, goal, t )
model  SAT-SOLVER(cnf)
if model is not null then
return EXTRACT-SOLUTION(model)
return failure
Classical Planning
Planning with Propositional Logic
66
 some explanatory notes
 distinct propositions for assertions about each time step
 superscripts denote the time step: At(P1,SFO)0  At(P2,JFK)0
 PDDL includes the Closed World Assumption, so conversion
involves the need to specify which propositions are not true
 ¬At(P1,JFK)0  ¬At(P2,SFO)0
 unknown propositions are left unspecified
 goal is associated with some particular time-step: which one?
Classical Planning
Planning with Propositional Logic
67
 some explanatory notes
 we can't know in advance how many steps are required to
achieve a goal so, to determine the time step T where the
goal will be reached, the algorithm loops until success or
some arbitrary time step limit
 repeat
 start at T = 0
 Assert At(P1,JFK)0  At(P2,SFO)0
 on failure: try T = 1
 Assert At(P1,JFK)1  At(P2,SFO)1
 add terms for next time step
 until a solution is found or some max path length Tmax is
exceeded: setting Tmax assures termination
Classical Planning
Models
68
 the pre-condition axioms
 avoid models with illegal actions
 e.g. for T = 1
 Fly(P1, SFO, JFK)0  Fly(P1, JFK, SFO)0  Fly(P2, JFK, SFO)0
 the second action is infeasible, precluded if we include the
appropriate pre-condition axiom: Fly(P1,JFK,SFO)0  At(P1,JFK)0
 since At(P1, JFK)0 false in initial state, Fly(P1, JFK, SFO)0 false in
any model
Classical Planning
69
Models
 without action-exclusion axioms
 a model might include actions, for example having 1 plane fly
to two destinations in a time step
 Fly(P1, SFO, JFK)0  Fly(P2, JFK, SFO)0  Fly(P2, JFK, LAX)0
 though the second P2 action is infeasible, it requires the added
action-exclusion axioms to avoid such solutions
 ¬(Fly(P2,JFK,SFO)0  Fly(P2,JFK,LAX)0)
Classical Planning
70
Planning in First-Order Logic
 alternative 2: convert to the Situation Calculus format
of FOL & apply FOL inferencing
 recall Situation Calculus as a topic in the 352 Knowledge
Representation discussion of time & action
 PDDL omits quantifiers which limits expressiveness but helps
control complexity of algorithms applied to it
 PL representation also has limitations like the requirement
that a "timestamp" becomes part of each fluent
 Situation Calculus replaces explicit timestamps by using
situations that correspond to a sequence or history of actions
 1. situations
 initial state is a situation
 if s is a situation & a an action, RESULT(s, a) is also a situation
 situations correspond to sequences or history of actions
 2 situations are the same only if their start & actions are the
same
Classical Planning
Planning in First-Order Logic
71
 convert to FOL Situation Calculus
 2. fluents are functions or relations that vary over situations
 syntax: situation is conventionally the last argument
 At(x, l, s) is a relational fluent, true when object x is at location l in
situation s & Location(x, s) is a functional fluent such that
Location(x, s) = l in the same situation
 3. a possibility axiom describes preconditions of each action
 (s)  Poss(a, s) indicates when an action can be taken
  is a formula giving preconditions
 Alive(Agent, s)  Have(Agent, Arrow, s)  Poss(Shoot, s)
 4. successor-state axioms for each fluent indicate what
happens, depending on the action taken
 here are the general form & an example from Wumpus World
Action is possible 
(Fluent is true in result state  Action's effect made it true V
It was true before & the action left it unchanged)
Poss(a, s) 
(At(Agent, y, Result(a, s))  a = Go(x, y) V (At(Agent, y, s) Λ a  Go(y, z)))
Planning in First-Order Logic
72
 express problem using FOL Situation Calculus
 5. unique action axioms required to allow deductions such as
 a  Go(y, z)
 for each pair of action names Ai & Aj there's an axiom to say they
are different: Ai(x, …)  Aj(y, …)
 plus, for each action name Aj an axiom indicates 2 uses are only
equal if all arguments are equal
 Ai (x1, …, xn) = Ai(y1, …, yn)  x1 = y1  …  xn = yn
 6. the solution is a situation (action sequence) that satisfies the
goal
 unfortunately, no major planning systems use situation calculus
 basic efficiency issues of FOL inference
 compounded by lack of good heuristics for planning with SC
Planning: Constraint Satisfaction
73
 we noticed in discussions in 352
 the similarities between Constraint Satisfaction Problems &
boolean satisfiability
 conversion to a CSP is similar to that of conversion to a
propositional logic satisfiability problem
 one difference is that it only requires a single variable Actiont
at each time step: its domain is the set of all possible actions
 thus we also don't need the action exclusion axioms
Classical Planning
74
Partial-order planning
 plans may be developed through refinement of partially
ordered plans
 approaches describe so far, including both Progression and
Regression planning, are totally ordered forms of plan search
 they cannot take advantage of problem decomposition
 decisions must be made on how to sequence the actions for all the
subproblems though some are independent (in the air cargo
problem, loading cargo at 2 different airports)
 a least commitment strategy
 would, where possible, delay choices during search
 partially ordered planning implements this
 adds actions to plans without committing to an absolute time/step,
instead dealing mainly with relative constraints
 action A must precede action B
 partially ordered plans are generated by a search in the space of
plans rather than through the state space
 so the planning operators are different from the real-world actions
Classical Planning
A plan to put your shoes on
75
 an illustration from 2e notes
Goal(RightShoeOn  LeftShoeOn)
Init()
Action(RightShoe,
PRECOND: RightSockOn
EFFECT: RightShoeOn)
Action(RightSock,
PRECOND:
EFFECT: RightSockOn)
Action(LeftShoe,
PRECOND: LeftSockOn
EFFECT: LeftShoeOn)
Action(LeftSock,
PRECOND:
EFFECT: LeftSockOn)
Classical Planning
76
A plan to put your shoes on
 a partially-ordered planner can place two actions into
a plan without specifying which comes first
 sample POP and corresponding totally ordered plans
Classical Planning
POP as a search problem
77
 states are (mostly unfinished) plans
 the empty plan contains only the special start & finish actions
 each state (plan) is a partially ordered structure
 it consists of a set of actions (the steps of the plan) and a set of
ordering constraints of the form Before(ai, aj) meaning that one
action occurs before another
 graphically, actions are shown as boxes, ordering constraints as
arrows
 the starting point is an empty plan consisting of the initial state
& goal, with no actions between them
 the search process adds an action to correct a flaw in the plan or if
it can't, it backtracks
 flaws keep a partial plan from being a solution
 as an example, the next slide shows POPs corresponding to the
"flat tire" planning problem, both the empty plan and a solution
POP for the flat tire problem
78
 at the top the empty plan & below it, the solution plan
 Remove(spare, trunk), Remove(flat, axle) can be done in either
order, so long as both are done before PutOn(spare, axle)
79
POP for the flat tire problem
 search in plan space seeks to add to the plan to
correct a flaw
 flaw: not achieving At(spare, axle) in the flat tire empty plan
 adding PutOn(spare, axle) fixes that, but introduces new
flaws in the form of its (unachieved) preconditions
 process repeats, backtracking as necessary
 each step makes the least commitment (ordering constraints
& variable bindings) in order to fix a flaw
 Remove(spare, trunk) must be before PutOn(spare, axle) but
no other ordering requirements
Classical Planning
80
Partial-Order Planning
 was previously popular because of explicitly
representing independent subproblems
 but it lacks an explicit representation of states in the statetransition model & was superseded by progression planners
with heuristics that allowed discovery of independent
subproblems
 nevertheless it's useful for tasks like operations scheduling,
enhanced with domain specific heuristics
 also appropriate where humans must understand the resulting
plans
 plan refinement approach is easier to understand & verify
 example: spacecraft/rover plans are generated by this approach
Classical Planning
81
Solving POP
 POP in action
 the initial plan contains
 only the Start & Finish steps
 Start step/action has the initial state description as its effect
 Finish step/action has the goal description as its precondition
 the single ordering constraint Before (Start, Finish)
 there's a set of open preconditions: all those in Finish
 the successor function
 picks an open precondition p on an action B and generates a
successor plan using an action A that achieves p
 then does goal test
 any remaining open preconditions?
Classical Planning
82
Process summary
 operators on partial plans may
 add link from an existing action to an open precondition
 add a step (a new action) to fulfill an open condition
 order one step wrt another to remove possible conflicts
 plan generation involves gradually moving
 from incomplete/vague plans to complete/correct plans
 plan construction may require backtracking
 if an open condition is unachievable or conflict is unresolvable
Classical Planning
Classical Planning Summary
83
 approaches rely on techniques of search or logic
 search to assemble a plan or to constructively prove one exists
 handling the inherent complexity of these problems is a
key
 identifying independent subproblems provides potential for
exponential speedup, but full decomposability is inconsistent
with -ve interactions between actions
 heuristically guided search can gain efficiency even if
subproblems are not completely independent
 some problems allow efficient solution by ruling out -ve
interactions
 serializable subgoals = there's a subgoal ordering for a solution
that avoids having to undo any subgoals
 tower problems in Blocks World are serializable if subgoals are
ordered bottom to top
84
Classical Planning Summary
 serializable subgoals
 tower problems in Blocks World are serializable if subgoals are
ordered bottom to top
 NASA spacecraft control commands are serializable
(unsurprising since the control systems are designed for easy
control)
 when problem is serializable, a planner can seriously prune
the planning search
 what's next? where does further progress come from?
 probably not simply factored representations & propositional
approaches
 combine efficient heuristics + first-order, hierarchical
representations
Classical Planning
85
Classical Planning Summary
 PDDL (Planning Domain Definition Language)
 initial & goal states as conjunctions of literals
 actions as lists of preconditions & effects
 search in state space may be forward (Progression) or
backward (Regression)
 derive heuristics from subgoal independence assumption &
relaxed versions of the problems
 Planning Graphs
 build from the initial state to the goal with alternating
incremental sets of literals or actions representing possibilities
for each time step, & encoding mutexes between inconsistent
pairs of literals or actions
 layers are supersets of literals/actions possible at that time step
 the basis of good heuristics for state space searches or can
yield a plan by application of the GRAPHPLAN algorithm
Classical Planning
86
Classical Planning Summary
 possible alternative approaches




FOL inference on a situation calculus knowledge base
conversion to boolean satisfiability problem for SATPlan
encoding as a Constraint Satisfaction Problem
search backwards over a space of Partially Ordered Plans
 what's best?
 there's no agreement on a single best representation or
algorithm
 we saw one hybrid that performs well
 a particular domain may provide a best fit to one particular
technique
Classical Planning