Transcript MAS-2 - Cursuri Automatica si Calculatoare

Multi-Agent Systems
University “Politehnica” of Bucarest
Spring 2010
Adina Magda Florea
http://turing.cs.pub.ro/mas_10
curs.cs.pub.ro
Lecture 2: Agent architectures




Simple agent models
Cognitive agent architectures
Reactive agent architectures
Layered architectures
1. Simple agent models

An agent perceives its environment through sensors
and acts upon the environment through effectors.

Aim: design an agent program = a function that
implements the agent mapping from percepts to
actions.
Agent = architecture + program
 A rational agent has a performance measure that
defines its degree of success.
 A rational agent has a percept sequence and, for
each possible percept sequence the agent should do
whatever action is expected to maximize its
performance measure.
3
Reactive agent model
Decision
component
action
P
Agent
E = {e1, .., e, ..}
P = {p1, .., p, ..}
A = {a1, .., a, ..}
A
Perception
component
see
Execution
component
action
Environment
Reactive agent
see : E  P
action : P  A
env : E x A  E
(env : E x A  P(E))
env (E)
4
Reactive agent model
Several reactive agents
seei : E  Pi
actioni : Pi  Ai
env : E x A1 x … An  P(E)
A1,…, Ai,..
P1,…, Pi,..
(usually the same)
Decision
component
action
Perception
component
see
Agent (A1)
Agent (A2)
Execution
component
action
Agent (A3)
Environment
env
5
Cognitive agent model
S
Decision
component
action, next
P
Perception
component
see
Agent
E = {e1, .., e, ..}
P = {p1, .., p, ..}
A = {a1, .., a, ..}
S = {s1, .., s, ..}
A
Execution
component
action
Environment
env (E)
State Agent
see : E  P
next : S x P  S
action : S  A
env : E x A  P(E)
6
Cognitive agent model
Several cognitive agents
seei : E  Pi
nexti : Si x P  Si
actioni : Si x I  Ai
interi : Si  I
env : E x A1 x … An  P(E)
S1,…, Si,..
A1,…, Ai,..
P1,…, Pi,..
(not always the same)
I = {i1, .., ik,…}
Decision
component
action, next
Perception
component
see
Agent (A1)
Interaction
component
inter
Execution
component
action
Agent (A2)
Agent (A3)
Environment
env
7
Cognitive agent model
Agent with states and goals
goal : E  {0, 1}
Agents with utility
utility : E  R
Nondeterministic environment
env : E x A  P(E)

Utility theory = every state has a degree of usefulness, to
an agent, and that agent will prefer states with higher utility

Decision theory = an agent is rational if and only if it
chooses the actions that yields the highest expected utility,
averaged over all possible outcomes of actions - Maximum
Expected Utility
8
Cognitive agent model
The expected probability that the result of an action (a)
executed in e to be the new state e’
 p ro b(ex(a,e)
') 
e
1
e en v e a
The expected utility of a action a in a state e, from the
point of view of the agent
U (a, e) 
 prob(ex(a, e)  e' )*utility(e' )
e 'env( e,a )
Maximum Expected Utility (MEU)
9

Example: getting out of a maze
– Reactive agent
– Cognitive agent
– Cognitive agent with utility

3 problems:
– what action to choose if several available
– what to do if the outcomes of an action are not
known
– how to cope with changes in the environment
10
2. Cognitive agent architectures

Rational behaviour: AI and Decision theory

AI = models of searching the space of possible
actions to compute some sequence of actions that
will achieve a particular goal

Decision theory = competing alternatives are taken
as given, and the problem is to weight these
alternatives and decide on one of them (means-end
analysis is implicit in the specification of competing
alternatives)

Problem = deliberation/decision
– The agents are resource bounded
– The environment may change during deliberation
11
Interactions
Information about
itself
Communication
Reasoner
Other
agents
Planner
Control
Output
Scheduler&
Executor
State
- what it knows
- what it believes
- what is able to do
- how it is able to do
- what it wants
environment and
other agents
- knowledge
- beliefs
Input
General cognitive agent architecture
Environment
12
FOPL models of agency

Symbolic representation of knowledge + use inferences in FOPL deduction or theorem proving to determine what actions to execute

Declarative problem solving approach - agent behavior represented as
a theory T which can be viewed as an executable specification
(a) Deduction rules
At(0,0)  Free(0,1)  Exit(east)  Do(move_east)
Facts and rules about the environment
At(0,0)
x y Wall(x,y)  Free(x,y)
Wall(1,1)
Automatically update current state and test for the goal state
At(0,3) or At(3,1)
(b) Use situation calculus =describe change in FOPL
Function Result(Action,State) = NewState
At((0,0), S0)  Free(0,1)  Exit(east)  At((0,1), Result(move_east,S0))
Try to prove the goal At((0,3), _) and determines actions that lead to it
- means-end analysis
13
Advantages of FOPL
- simple, elegant
- executable specifications
Disadvantages
- difficult to represent changes over time
other logics
- decision making is deduction and
selection of a strategy
- intractable
- semi-decidable
14
BDI (Belief-Desire-Intention)
architectures

High-level specifications of an architecture for a
resource-bounded agent.

Beliefs = information the agent has about the
world

Desires = state of affairs that the agent would
wish to bring about

Intentions = desires the agent has committed to
achieve

BDI - a theory of practical reasoning - Bratman,
1988

Intentions play a critical role in practical reasoning limits options, DM simpler
15
BDI particularly compelling because:
 philosophical component - based on a theory of
rational actions in humans
 software architecture - it has been implemented and
successfully used in a number of complex fielded
applications
– IRMA (Intelligent Resource-bounded Machine Architecture)
– PRS - Procedural Reasoning System

logical component - the model has been rigorously
formalized in a family of BDI logics
– Rao & Georgeff, Wooldrige
– (Int i  )   (Bel i )
16
percepts
BDI Architecture
Belief revision
Beliefs
Knowledge
Opportunity
analyzer
B = brf(B, p)
Deliberation process
Desires
I = options(D, I)
Intentions
Filter
Means-end
reasonner
I = filter(B, D, I)
Intentions structured
in partial plans
 = plan(B, I)
Library of plans
Plans
Executor
actions
17
BDI Agent control loop
B = B0
I = I0
D = D0
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
execute()
end while
18
Commitment strategies
The option chosen by the agent as an
intention = the agent is committed to that
intention
 Commitments imply temporal persistence of
intentions
Question: How long is an agent committed to
an intention
 Blind commitment
 Single minded commitment
 Open minded commitment

19
B = B0
BDI Agent control loop
I = I0 D = D0
Blind commitment
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
while not (empty() or succeeded (I, B)) do
 = head()
execute()
 = tail()
get next perceipt p
B = brf(B,p)
if not sound(, I, B) then
 = plan(B, I)
Reactivity, replan
end while
end while
20
BDI Agent control loop
B = B0
I = I0 D = D0
Single minded commitment
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
Dropping intentions that are impossible
I = filter(B, D, I)
or have succeeded
 = plan(B, I)
while not (empty() or succeeded (I, B) or impossible(I, B)) do
 = head()
execute()
 = tail()
get next perceipt p
B = brf(B,p)
if not sound(, I, B) then
 = plan(B, I)
Reactivity, replan
end while
end while
21
BDI Agent control loop
B = B0
I = I0 D = D0
Open minded commitment
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
while not (empty() or succeeded (I, B) or impossible(I, B)) do
 = head()
execute()
 = tail()
get next perceipt p
B = brf(B,p)
if reconsider(I, B) then
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
Replan
end while
end while
22
BDI architecture – most popular
 There is no unique BDI architecture

PRS - Procedural Reasoning System (Georgeff)
 dMARS
 UMPRS si JAM – C++

(http://www.marcush.net/IRS/irs_downloads.html)
JACK – Java (http://en.wikipedia.org/wiki/JACK_Intelligent_Agents)
 JADEX – XML si Java, JADE (http://vsis
www.informatik.uni-hamburg.de/projects/jadex/)

JASON – Java (http://jason.sourceforge.net/)
23
3. Reactive agent architectures
Subsumption architecture - Brooks, 1986







DM = {Task Accomplishing Behaviours}
A TAB is represented by a competence module (c.m.)
Every c.m. is responsible for a clearly defined, but not
particular complex task - concrete behavior
The c.m. are operating in parallel
Lower layers in the hierarchy have higher priority and are
able to inhibit operations of higher layers
The modules located at the lower end of the hierarchy are
responsible for basic, primitive tasks
The higher modules reflect more complex patterns of
behaviour and incorporate a subset of the tasks of the
subordinate modules subsumtion architecture
24
Competence
Module (2)
Investigate env
Input
(percepts)
Sensors
Competence
Module (1)
Move around
Output
(actions)
Effectors
Competence
Module (0)
Avoid obstacles
Module 1 can monitor and influence the inputs and
outputs of Module 2
M1 = wonders about while avoiding obstacles  M0
Supressor node
M2 = explores the environment looking for
distant objects of interests while moving
around  M1
 Incorporating the functionality of a subordinated
c.m. by a higher module is performed using
suppressors (modify input signals) and
inhibitors (inhibit output)
Competence
Module (1)
Move around
Inhibitor node
Competence
Module (0)
Avoid obstacles
25

More modules can be added:
• Replenishing energy
• Optimising paths
• Making a map of territory
• Pick up and put down objects
• Investigate environment
• Move around
• Avoid obstacles
Behavior
(c, a) – pair of condition-action describing behavior
Beh = { (c, a) | c  P, a A}
R = set of behavior rules
  R x R - binary inhibition relation on the set of behaviors, total ordering of R
function action( p: P)
var fired: P(R)
begin
fired = {(c, a) | (c, a)  R and p  c}
for each (c, a)  fired do
if   (c', a')  fired such that (c', a')  (c, a) then return a
return null
end
26

1990 - Brooks extends the architecture to
cope with a large number of c.m. - Behavior
Language

Indirect communication

Advantages of reactive architectures

Disadvantages
27
4. Layered agent architectures

Combine reactive and pro-active behavior
At least two layers, for each type of behavior

Horizontal layering - i/o flows horizontally

Vertical layering - i/o flows vertically

Layered agent architectures
Action
output
Layer n
perceptual
input
Action
output
Layer n
Layer n
…
…
Layer 2
Layer 2
Layer 2
Layer 1
Layer 1
Layer 1
…
Action
output
Vertical
Horizontal
perceptual
input
perceptual
input
29
TouringMachine



Horizontal layering - 3 activity producing layers, each layer
produces suggestions for actions to be performed
reactive layer - set of situation-action rules, react to precepts from the
environment
planning layer
- pro-active behavior
- uses a library of plan skeletons called schemas


- hierarchical structured plans refined in this layer
modeling layer
- represents the world, the agent and other agents
- set up goals, predicts conflicts
- goals are given to the planning layer to be achieved
Control subsystem
- centralized component, contains a set of control rules
- the rules: suppress info from a lower layer to give control to a higher one
- censor actions of layers, so as to control which layer will do the actions
30
InteRRaP
Vertically layered two pass agent architecture
 Based on a BDI concept but concentrates on the dynamic
control process of the agent
Design principles
 the three layered architecture describes the agent using various
degrees of abstraction and complexity
 both the control process and the KBs are multi-layered
 the control process is bottom-up, that is a layer receives control
over a process only when this exceeds the capabilities of the
layer beyond
 every layer uses the operations primitives of the lower layer to
achieve its goals
Every control layer consists of two modules:
- situation recognition / goal activation module (SG)
- planning / scheduling module (PS)
31

Cooperative
planning layer
I
n
t
e
R
R
a
P
Local
planning layer
Behavior
based layer
World interface
actions
SG
SG
SG
Sensors
Social KB
PS
Planning KB
PS
World KB
PS
Effectors
Communication
percepts
32
BDI model in InteRRaP
options
Beliefs
Sensors
Situation
Goals
Social model
Cooperative situation
Cooperative goals
Mental model
Local planning situation
Local goals
World model
Routine/emergency sit.
Reactions
filter
Options
Intentions
Cooperative option
Cooperative intents
Effectors
SG
Local option
Local intentions
Reaction
Response
Operational primitive
Joint plans
PS
Local plans
plan
Behavior patterns
33


Muller tested InteRRaP in a simulated loading area.
A number of agents act as automatic fork-lifts that move in the loading
area, remove and replace stock from various storage bays, and so
compete with other agents for resources
34