Transcript No Slide Title

Altering the ICARUS Architecture
to Model Social Cognition
Pat Langley
Institute for the Study of
Learning and Expertise
Award Period 2/1/12–1/31/15
ONR Cognitive Science and Human-Robot Interaction
6.1 Program Review
June 25–28, 2013
Critique of the ICARUS Architecture
In previous work (Langley et al., 2009), we have developed
ICARUS, an architecture that, despite its accomplishments:
• Relies on exhaustive, deductive inference
• Emphasizes physical activities over mental ones
• Cannot represent or reason about others’ mental states
• Has inflexible mechanisms for execution / problem solving
This project aims to address these drawbacks by developing
a radically new version of the architecture.
Research Objectives
We aim to develop a unified theory of the human cognitive
architecture that supports:
 Representing and reasoning about others’ mental states
 Flexible inference and problem solving in this context
 Structural learning that supports these processes
The research project’s significance lies in its potential to:
 Improve accounts of human reasoning and learning
 Support agents/robots that interact effectively with humans
This effort addresses aspects of high-level cognition that have
received little attention elsewhere.
3
Recent Accomplishments
During the past year, our team’s accomplishments have included:
• Developing new formalisms for:
• Beliefs and goals that refer to other agents’ mental states
• Concepts and skills that involve relations among mental states
• Designing, implementing, and testing an approach to the incremental
abduction of explanations
• Adapting and applying this mechanism to:
• Understanding domain-level plans
• Understanding stories in which agents reason about others
• Explaining and judging behavior in moral contexts
• Reimplementing / improving a flexible framework for problem solving
that incorporates meta-level control rules
Together, these support our aims to produce a more complete account
of human cognitive abilities.
Challenge: Plan Understanding
A basic task that involves reasoning about others' mental states
is plan understanding, which we can define as:
• Given: A sequence S of actions agent A is observed to carry out;
• Given: Knowledge about concepts and activities, organized
hierarchically, that are available to agent A;
• Infer: An explanation, E, in proof lattice form, that accounts for
S in terms of A's goals, beliefs, and intentions.
This is analogous to language understanding in that analysis
produces a connected account of input.
We distinguish it from plan recognition (Goldman et al., 1999),
which assigns observed behavior to some known category.
An Illustrative Example
Consider an action sequence from the Monroe County corpus
(Blaylock & Allen, 2005):
Truck driver tdriver1 navigates the dump truck dtruck1 to the
location brightondump, where a hazard team ht2 climbs into
the vehicle. Then tdriver1 navigates dtruck1 to the gas station
texaco1, where ht2 loads a generator gen2 into dtruck1…
Given such observations and knowledge about possible goals /
activities, we want to infer the latter to explain events.
In this case, we might conclude the driver is collecting people and
a power source for some mission.
Plan Understanding as Abductive Inference
Our theoretical claims about plan understanding are that it:
• Involves inference about the participating agents’ mental states
(beliefs / goals about activities and environment)
• Involves the abductive generation of explanations through the
introduction of default assumptions
• Operates in an incremental fashion to process observations that
arrive sequentially
• Proceeds in a data-driven manner because understanding arises
from observations about agents’ activities
These four assumptions place constraints on our computational
account of this important process.
A Sample Explanation
get-to(ht2, texaco1)
→ get-to(dtruck1, br-dump)
→ get-to(dtruck1, texaco1)
→ drive-to(tdriver1, dtruck1, br-dump)
→ drive-to(tdriver1, dtruck1, texaco1)
— at-loc(dtruck1, _)
— at-loc(dtruck1, br-dump)
— at-loc(tdriver1, _)
— at-loc(tdriver1, br-dump)
→ navigate-vehicle(tdriver1, dtruck1, br-dump)
→ navigate-vehicle(tdriver1, dtruck1, texaco1)
— person(tdriver1)
— person(tdriver1)
— vehicle(dtruck1)
— vehicle(dtruck1)
— can-drive(tdriver1, dtruck1)
— can-drive(tdriver1, dtruck1)
— at-loc(dtruck1, br-dump)
— at-loc(dtruck1, texaco1)
— at-loc(tdriver1, br-dump)
— at-loc(tdriver1, texaco1)
→ get-in(ht2, dtruck1)
→ get-out(ht2, dtruck1)
— not(non-ambulatory(ht2))
— not(non-ambulatory(ht2))
— person(ht2)
— person(ht2)
→ climb-in(ht2, dtruck1)
→ climb-out(ht2, dtruck1)
— at-loc(ht2, br-dump)
— at-loc(ht2, dtruck1)
— at-loc(dtruck1, br-dump)
— at-loc(dtruck1, texaco1)
— fit-in(ht2, dtruck1)
— at-loc(ht2, texaco1)
— at-loc(ht2, dtruck1)
Representing Plan Knowledge
We represent knowledge about activities in a notation similar to
hierarchical task networks. For example:
navigate_vehicle(Driver, Veh, Loc, T_Start, T_End)
at_location(Veh, VLoc, T_1, T_Start),
at_location(Driver, VLoc, T_3, T_Start),
Driver(Driver), vehicle(Veh),
can_drive(Driver, Veh, T_9, T_10),
at_location(Veh, Loc, T_End, T_13),
at_location(Driver, Loc, T_End, T_15),
constraint(before(T_1, T_Start)), constraint(before(T_2, T_Start)),
constraint(before(T_3, T_Start)), constraint(before(T_4, T_Start)),
constraint(inside(T_Start, T_End, T_5, T_6)), constraint(before(T_End, T_14)),
constraint(inside(T_Start, T_End, T_7, T_8)), constraint(before(T_End, T_13)),
constraint(inside(T_Start, T_End, T_9, T_10)), constraint(before(T_End, T_15)),
constraint(inside(T_Start, T_End, T_11, T_12)), constraint(before(T_End, T_16)).
This formalism separates conditions, effects, and invariants in
terms of temporal constraints on antecedents.
The UMBRA Abduction System
We have developed UMBRA, an abductive inference system that:
• Accepts observations and adds them to working memory
• Incrementally extends an explanation by:
- Finding rules with antecedents that unify with wm elements
- Tentatively completing each rule instance's missing antecedents
- Selecting the rule instance R with best evaluation score
- Adding R’s inferred elements to memory as default assumptions
• Continues until no further observations arrive
This data-driven strategy aims to produce a coherent explanation
in terms of available knowledge.
UMBRA is similar in spirit to AbRA (Bridewell & Langley, 2011).
Experiments on Plan Understanding
Experiments with UMBRA on the Monroe corpus show that:
• The system can reconstruct much higher-level plan structure
• Even when only a fraction of agent actions are observed
• Incremental abduction is nearly as effective as batch processing
Results on Plan Understanding
Precision and recall for
each problem on ten
‘batch’ runs.
The former is very high
on some tasks but not as
good on others.
Differences are due to
features of problems in
the Monroe domain.
Recall is mediocre for
similar reasons.
Challenge: Social Understanding in Fables
A more challenging task involves reasoning about plans that
take others' mental states in account.
This ability is required to understand Aesop-style fables like:
The Snake, the Lion and the Sheep. The lion is too old to chase
down animals. The lion announces he is sick. The sheep, believing
he is harmless, follows social convention and visits the lion's caves
to pay his respects. The lion kills and devours the sheep. A snake
watches these events and understands the deception that occurred.
Explanations of such stories include beliefs and goals about
others’ beliefs and goals.
This requires extensions to representations in both working
memory and long-term knowledge.
Extending Working Memory
UMBRA represents agents’ mental states in terms of embedded
structures like:
• belief(fox, has(crow, grapes, 0930, _), 0931, _)
• goal(crow, acquire_edible_food(crow, _, _))
• belief(snake, belief(lion,
at_location(lion, river, 0900, _), 0902, _), 0902, _)
• belief(snake, goal(fox,
trade_food(crow, grapes, fox, grain, 0940, _), 0930, _), 0933, _)
• goal(lion, belief(sheep, sick(lion, 0900, 2400), 0945, _), 0900, _)
Elements of this sort provide building blocks for explanations
of scenarios that involve agents reasoning about others.
Extending Knowledge about Activities
UMRBA also requires planning operators that influence others'
mental states, such as for communicative actions:
announce_falsehood(Actor, Agent2, Content, START, END)
neg(dead(Actor, T1, T2)),
exists(Actor, T3, T4),
belief(Actor, neg(Content), T5, T6),
agent(Actor),
agent(Agent2),
announce_act(Actor, Agent2, Content, T_S, T_END),
belief(Agent2, Content, T_END, T7),
belief(Actor, belief(Agent2, Content, T_END, T8), T_END, T9),
constraint(inside(T_S, T_END, T1, T2)),
constraint(before(T_END, T8)),
constraint(before(T_S, T_END)).
These structures, combined with domain knowledge, support
abductive construction of complex social explanations.
A Testbed for Social Understanding
We have constructed a domain and test scenarios, based largely
on Aesop's fables, with knowledge that includes:
• About 60 distinct skills / operators
– alternative decompositions
– many with overlapping conditions
– only ten percent used in any 'correct' fable explanation
– about 500 domain-level conditions, excluding constraints
• About 100 distinct domain-level predicates
Most of the six scenarios involve plans that depend on one or
more agents reasoning about the mental states of others.
Results on Social Understanding
We have tested UMBRA on ‘fable’ scenarios that involve different
levels of complexity beyond ‘basic’ plan understanding.
Nested understanding: The primary agent interprets another agent's
mental states and/or plan based on observed behavior.
Feeling hungry, a crow travels to a barn and acquires grain by opening a jar. A
snake watches and understands the crow solving her simple problem.
Deeply nested understanding: The primary agent infers a secondary
agent’s inferences about a third agent's mental states.
A fox, watching the snake watching the crow, imagines what the snake thinks
about the crow's situation.
Inferring mistakes in understanding: The primary agent infers
another agent's mistaken beliefs, why they arise, and the true account.
A lion is proud of his mane. He passes by a river, sees his reflection, and
attacks the ‘other’ lion. An observing snake infers why he takes this action.
Results on Social Understanding
Reasoning about opportunism in understanding: The primary agent
understands how another agent capitalizes upon another's false beliefs.
A hungry crow in possession of some sour grapes trades them to a fox, who
assumes they are sweet, in return for delicious grain. A watching snake
explains the interaction.
Reasoning about deception in understanding: The primary agent
infers than another agent deliberately engenders false beliefs in a third
agent in order to achieve some goal.
A lion is too old to chase down animals. The lion announces he is sick. The
sheep, believing he is harmless, follows social convention and visits the lion's
caves to pay his respects. The lion kills and devours the sheep. A snake who
watches these events and understands the deception that occurred.
UMBRA constructs the desired explanations for each scenario, some
of which involve deeply embedded mental models.
Complete Structure of a Fable Explanation
Green = condition
Yellow = effect
Orange = invariant
Blue = constraint
Diamond = task / skill
Portion of a Fable Explanation
Green = condition
Yellow = effect
Orange = invariant
Blue = constraint
Diamond = task / skill
One Element of a Fable Explanation
Green = condition
Yellow = effect
Orange = invariant
Blue = constraint
Diamond = task / skill
Challenge: Moral Judgement
An even more challenging cognitive task involves complex
moral judgement, which we can specify as:
• Given: A sequence S of observed actions, including the agent(s)
A who performed them;
• Given: Knowledge about these and related events, including their
relation to moral concepts;
• Infer: An explanation E that accounts for S in terms of this
knowledge and A’s beliefs, goals, and intentions; and
• Infer: A moral evaluation of S that takes into account the
explanation E.
This task combines plan understanding with evaluation in terms
of moral concepts.
Claims about Moral Judgement
We maintain that complex moral judgement is a form of social
plan understanding in that it:
• Focuses on the mental states of agents who interact in a
given scenario;
• Depends on rules that abstract away from domain-specific
details and focus on relations among mental states;
• Involves the linking of rule instances into some connected
explanation of observed behavior.
However, the process also relies on calculating numeric values
on elements that reflect evaluations of behavior.
A Sample Moral Explanation
Consider a scenario in which one agent (John) causes another
(Kelly) to feel pain by shoving her.
We might infer that John carried out this action deliberately so
that Kelly would experience distress.
Evaluations of Moral Explanations
We plan to extend UMBRA to support the evaluation of moral
explanations by:
• Adding numeric annotations to long-term knowledge structures:
– A default weight for each conceptual predicate
– An upward factor for each rule's antecedent
– A downward factor for each rule's antecedent
• Calculating an evaluation for each element in an explanation by:
– Multiplying the sum of upward factors by the default value and
propagating the result upward to the root(s)
– Multiplying downward factors by the accrued values at root(s)
We also maintain that top-down influences account for the effect
of mitigating factors on judgement scores.
Problem Solving in ICARUS
The current ICARUS architecture incorporates a distinct module
for problem solving that:
• Utilizes means-ends analysis
• Carries out depth-first search
• Interleaves tightly with skill execution
• Cannot reason about others’ mental states
These features do not reflect the character of human problem
solving, which is far more flexible.
Our new framework aims to support such flexibility by using
meta-level knowledge.
Flexible Problem Solving
We have redsigned and reimplemented our meta-level approach
to problem solving to support different:
• Search strategies (depth first, breadth-first, iterative sampling)
• Intention selection strategies (means ends, forward search)
• Intention application strategies (eager, delayed commitment)
• Failure conditions (depth limited, effort limited, loops)
• Solution conditions (single, multiple, all)
These behaviors are produced by differences among meta-level,
domain-independent control rules associated with five modules.
Soar (Laird, 2012) takes a similar but finer-grained approach; our
framework is closer to that in Prodigy (Minton, 1988).
Organization of Problem Solving
Problem solving
occurs in cycles,
with meta-level
rules determining
behavior at each
successive stage.
Meta-level rules
determine the
system’s behavior
for each stage.
Problem Decompositions
Problems play the
central organizing
structure in our
framework.
Down subproblems
have the same state
as their parents.
Right subproblems
have the same goals
as their parents.
This organization is
the same as that in
means-ends problem
solving, but we use
it to support very
different strategies.
Plans for Future Research
Although we have made substantial progress toward the project
goals, we still need to:
• Extend UMBRA to support belief revision when it decides its
default assumptions are faulty
• Augment the meta-level problem solver to support execution of
plans in the environment
• Integrate UMBRA’s inference mechanism with our approach to
flexible problem solving
• Introduce mechanisms for learning structures from explanations
• Carry our experiments to demonstrate these extensions’ benefits
The resulting architecture should offer a more complete account
of high-level cognition in humans.
Summary Remarks
In this talk, I presented elements of a new cognitive architecture
that addresses limitations of ICARUS by:
• Represents mental states in terms of embedded beliefs / goals
• Incorporates an incremental approach to abductive inference
• Combines these to support plan understanding
• Basic explanations of observed physical activities
• Explanations that involve agents reasoning about other agents
• Moral judgements that include inferences about agent intentions
• Uses meta-level control to support flexible problem solving
When integrated, these should give us a new version of ICARUS
that has substantially greater breadth and flexibility.
Publications and Presentations
Langley, P. (2012). The cognitive systems paradigm. Advances in Cognitive Systems, 1,
13.
3-
Langley, P. (2012). Intelligent behavior in humans and machines. Advances in Cognitive
Systems, 2, 3-12.
MacLellan, C., Langley, P., & Walker, C. (2012). A generative theory of problem solving.
Poster Collection / First Annual Conference on Advances in Cognitive Systems, 1-18.
Meadows, B., Langley, P., & Emery, M. (in press). Seeing beyond shadows: Incremental
abductive explanation for plan understanding. Proceedings of the AAAI-2013 Workshop
on Plan, Activity, and Intent Recognition.
Liu, L., Langley, P., & Meadows, B. (in press). A computational account of complex moral
judgement. Proceedings of the Annual Conference of the International Association for
Computing and Philosophy.
The Cognitive Systems Paradigm. Presented at AAAI Fall Symposium on Advances in
Cognitive Systems, Arlington, VA, November, 2011.
Intelligent Behavior in Humans and Machines. Presented at First Annual Conference on
Advances in Cognitive Systems, Palo Alto, CA, December, 2012.
Cooperative Development
Our research on this project has benefited from results produced
on a number of other efforts:
• Commitments to hierarchical concepts / skills borrowed from
initial ICARUS architecture developed under ONR funding
• Representation of mental states developed jointly with ONR
MURI project at CMU
• Ideas on abductive inference co-developed with W. Bridewell
in ONR MURI work at Stanford
These efforts have let us make more rapid progress than would
have been possible otherwise.
Transition Plan
Our research on computational social cognition has clear uses in
virtual agents and human-robot interaction.
In the longer term, we hope to transition our results to applied
settings like:
• Virtual medical assistants that interact with field medics to help
them provide emergency care
• Cognitive robots that interact with Navy personnel dealing
with shipboard problems (e.g., fighting fires)
We hope to take advantage of existing relationships with NRL
researchers to increase the chances of successful transitions.
Project Budget
The research project’s budget, by federal fiscal year, is:
• FY2012: $118K
• FY2013: $179K
• FY2014: $182K
• FY2014: $ 60K
No DURIP were awarded in relation to this project.
End of Presentation