Transcript Intelligent agents - Carnegie Mellon University

Intelligent Agents
Katia Sycara
[email protected]
The Robotics Institute
Joseph Giampapa
[email protected]
www.cs.cmu.edu/~softagents
What is an agent?
• An agent is an autonomous computational
entity, which:
– is reactive and proactive
– is goal driven
– is intelligent:
• able to reason, plan and sometimes learn
• has domain specific intelligence
– interacts with humans, other agents, and the
environment via sensors and effectors in a high
level language/protocol
– anticipates user needs and reacts based on them
– wish list: friendly, understands natural lang.,etc
Multi-Agent Systems (MAS)
• An agent is more useful in the context of
others:
– can concentrate on tasks of its expertise
– can delegate other tasks to other experts
– can take advantage of its ability to intelligently
communicate, coordinate, negotiate
• But, a MAS is not just a collection of agents
– it needs meaningful ways for agents to interact
– it needs some system design and performance
evaluation
MAS - Two Approaches
1. Centralized design
• Build a system that is comprised of agents should provide good performance
• Advantages may arise from:
– possibility to develop each agent as an expert
– incorporation of non-local expertise
– rather simple to have multiple developers
working concurrently
• Example: a system within an organization
MAS - Two Approaches
2. Open MAS
• Usually, the system has no prior static design, only
single agents within
• Agents seek others to provide services, without
knowing in advance who they are
• There is a need for agent finding mechanism
• Other agent may be non-cooperative or untrusted
or malicious
• Example: markets, Internet
Design and Architecture - Outline
•
•
•
•
•
•
•
Design philosophies
Information processing and needs
Reactive architectures
Deliberative architectures
Layered architectures
Belief, Desire, Intention (BDI)
Concurrent architecture (RETSINA)
Agent Design Philosophies
• Agents reside in the environment: the world and
other agents
• The environment can be characterized by a set of
states S={s1,s2,...}
• Agents execute actions A={a1,a2,…}
• An action is a function action: SS, that affects
the environment via state change
• So, in general, an agent is a set of actions that
receive input from the world and manipulate the
state of the world
Agent - Environment Interaction
Agent
Action
Output
Sensor
Input
Environment
Architecture Design
• A map of the internal structures of the
agent, includes:
– data structures
– operation that can be performed on them
– control and data flow between the structures
• Starts from a high-level definition and
traverses through refinements
• Design decisions result in adding details,
getting closer to code level, reducing
generality
Information and Processing Needs
• System architecture design needs knowledge of:
– what the expected inputs are?
– what the required/expected outputs are?
– what processing can provide this relation between
input and output?
• For agents, in particular:
– what are the possible/expected states of the world?
– how should the world state be perceived?
– how should the agent’s reasoning be affected by the
world state?
– how should agent reasoning result in agent action?
Agent Information Needs
• Is the “state of the world” sufficient?
• Yes. Usually it is too much:
– some or most of it may be inaccessible
– dynamic and possibly non-deterministic
– includes other agents, users, internet, etc
• Perception filters and reduces amount of info.
• Design question: what is the minimal set of
data and how to filter/extract it?
Agent Architectures
Reactive architectures
Deliberative architectures
• Layered architectures
• Belief, Desire, Intention (BDI)
• Concurrent architecture (RETSINA)
Agent Architectures (cont)
• Reactive vs. deliberative
– Reactive agents: upon input from the
environment, they react with action execution
– Deliberative agents: upon input, a reasoning
process is invoked. Action is based on the
results of the reasoning
• Agents may populate the whole spectrum
between purely reactive and maximally
deliberative (rational)
Agent Processing Needs
• Reactive agents only need to map between world
states and actions
• Deliberative agents need to reason for action.
May include:
–
–
–
–
–
taking into account historical states of world
creating and maintaining an internal state
reasoning about world and states
planning, re-planning for current/future action
learning
Agent Processing Needs (continued)
• Collaborative (social) agents need, in
addition:
–
–
–
–
maintain models of other agents and the society
reason about others
plan collaborative activity
reason about interaction: communication,
coordination, collaboration
Required Agent Attributes
• Perception - a function perceive: SP, where
P={p1,p2,…} a set of percepts:
– required for both reactive and deliberative agents
– may be provided via sensor or any other input
• Internal state I (records history)
– not necessary for reactive agents
– deliberative agents need to maintain information regarding
past activity to allow for deliberation
• Reasoning: performed mainly by deliberative agents,
but may be useful for reactive, too
• Learning: only in deliberative agents
Reactive Architecture
Perception
Action
Agent
Environment
Agents without State (Reactive)
•
•
•
•
Perception is a function perceive: SP
Action is a function action: SS,
Action selection is a function as: PA
The world state results in a percept via
perception, the percept results in an action
selection, and the action transforms the state
of the world
Example: Subsumption (Brooks)
• An agent decision making is performed by a set of task
accomplishing behaviors (TAB)
• In Brooks’ implementation, each TAB is a finite state
machine
• In other implementation, TABs are rules of the type
situation  action, which maps percepts to actions
• In the subsumption architecture, multiple behaviors can
be activated simultaneously
• Action selection is based on a subsumption hierarchy,
behaviors arranged in layers, which are at different
layers of abstraction (layered architecture)
Reactive: Pros and Cons
• Pros:
–
–
–
–
simplicity, economy
computational tractability
fault tolerance
overall behavior emerges from component interaction
• Cons:
– without a model of the environment, agents need
sufficient info. to determine action
– agents are “short-sighted”, may limit decision quality
– relationship between components not clear.
Agents with State = Deliberative
Agent
Perception
Action
Reasoning
State
Environment
Agents with State (Deliberative)
• Perception is still a function perceive: S  P
• Action is still action: SS
• But action selection (was as: PA), is now
the function as: IA
• In addition, update: P×II is a function that
update the internal state based on percepts
(may include complex reasoning)
Agents with State: Refinement
Agent
Perception
planning
Action
learning
State
inference
Reasoning
Environment
Layered Agent Architectures
• Usually, but not always, deliberative
architectures
• Decision making is performed via
separation to several software layers
• Each layer reasons at a different level of
abstraction. Layers interact
• Two major types:
– vertical layers: perception input and action
output are dealt with by a single layer each
– horizontal layers: each layer directly connects
to perception input and action output
Layers’ design
• Typically, at least two layers, one for
reactive behavior and one for proactive
• No reason not to have multiple layers
• Typology: information and control flow
between the layers, e.g.:
Agent
Perception
Action
Information and Control Flow
Action
output
Layer n
Layer n
Layer n
Layer 2
Layer 2
Layer 2
Layer 1
Layer 1
Layer 1
Perceptual
input
Perceptual Action
input
output
Perceptual
input
Action
output
Horizontal
Vertical (one pass)
Vertical (two pass)
Layers Pros and Cons
• Horizontal
– each layer acts like an agent - provides
independency, simplicity
– for n different behaviors we implement n layers
– competition between layers can cause incoherence
– need for mediation between layers: exponentially
complex, a control bottleneck
• Vertical
– Low complexity, no control bottleneck
– Less flexible and not fault tolerant: one decision
needs all layers
TOURINGMACHINES
• Three layers produce suggestions for action:
– reactive: implements situation-action rules as in
Brooks’ subsumption architecture
– planning: achieves proactiveness via plans based on
a library of schemas
– modeling: model of world, other agents, self,
predicts conflicts, generates goals to resolve them
• Domain of implementation: multiple vehicles
Example: TOURINGMACHINES
Perception
input
Modeling layer
Perception subsystem
Planning layer
Action subsystem
Reactive layer
Action
output
Control subsystem
INTERRAP
• A vertically layered two pass architecture
• Layers have similar purposes as in
TOURINGMACHINES
• Each layer is associated with a knowledge-base
• Layers interact with each other:
– bottom-up: activation
– top-down: execution
Example: INTERRAP
Cooperation layer
Social knowledge
Plan layer
Planning knowledge
Behavior layer
World model
World interface
Perception input
Action output
Belief-Desire-Intention Architecture
• Based on practical reasoning and decision on actions.
Involves:
– decision on what goals we want to achieve: deliberation
– decision on how to achieve these goals: means-ends reasoning
• Choosing some options creates intentions. These:
– usually lead to action
– should persist:
• once adopted, an agent should persist with the intention, attempt to
achieve it
• the intention should be dropped if
– it is clearly non-achievable
– it was already achieved
– the reason for the intention is not there anymore
– are related to beliefs about the future
BDI Architecture Components
• A set of current beliefs about the environment
• A belief revision function (brf) - updates current beliefs
based on perception
• An option generation function - determines available
options (desires) based on beliefs and intentions
• A set of desires (current options) - possible courses of action
available
• A set of current intentions - the options the agents is
committed to trying to perform
• A filter function (deliberation) - determines new intentions
based on current beliefs, desires, intentions
• An action selection function - selects actions based on
intentions
Schematic BDI Architecture
Perception input
brf
beliefs
Generate options
desires
filter
intentions
action
Action output
BDI Pros and Cons
• Key problem: difficult to balance between mental
activities
– Example: dropping intentions requires
reconsideration, which is costly but needed
– Rate of environment change helps set re-consideration
• Questionable: what advantage do mental states
provide?
• Intuitive, provides functional decomposition
• Easy to define formally, using logic, and convert
to code
Concurrent Architectures
(RETSINA: Sycara & al.)
• Include multiple functional and knowledge
modules that work concurrently
• Coherence between the functional modules
is achieved via shared databases
• Typical functional separation:
–
–
–
–
communication and collaboration
planning and reasoning
action scheduling
execution and monitoring
Example: RETSINA Agent Architecture
Functional Components
• Communicator: handles incoming and outgoing
messages in an ACL. Converts requests into
goals/objectives
• Planner: takes objectives and devises detailed plans to
achieve them. Creates tasks, actions and new objectives.
Uses plan fragments from libraries
• Scheduler: schedules actions for execution
• Execution monitor: executes actions and monitors
• Coordination/collaboration: reasons for such activities,
may be internal to planner or to communicator
• Self-awareness: maintains self model: load, state, etc
Planning
• By incremental instantiation of plan
fragments
• Conditional planning mechanisms
• Interleaving planning, information
gathering, and execution
• Declarative description of information flow
and control flow requirements
11
Task decomposition
Knowledge Components
• Objective DB: holds the agent’s objectives
• Task DB: holds the agent’s tasks and actions,
before they are scheduled for execution
• Schedule: holds scheduled actions
• Task reduction library: includes a set of possible
task decompositions
• Task schema library: includes plan fragments,
each provides details on how to perform a task
• Beliefs DB: holds the beliefs of the agent
regarding information relevant to its activity
Architecture Attributes
• Functional components do not directly
interface or synchronize with each other
• Knowledge components do not directly
interface or synchronize with each other
• Functional components work concurrently
• These provide:
–
–
–
–
reusability and substitutability of components
efficient utilization of computational resources
timely task performance
reduced development effort
RETSINA Agent Functionality
• Interacts with humans and other agents
• Anticipates and satisfy human information
needs
• Provides decision support
• Integrates planning, information gathering
and execution
• Acquires, use and disseminate timely and
relevant information
• Adapts to user, task and situation
10