Transcript PPT
peer-to-peer and agent-based computing
Basic Theory of Agency (Cont’d)
Indistinguishable environments (1)
• Imagine two environmental states
e
1 ,
e
2 Env, such that –
e
1
e
2 and –
see
(
e
1 ) =
see
(
e
2 ) • Such environments are different, but as far as agents are concerned they are indistinguishable !
– The environments are mapped to the same percept – The agent would receive the same perceptual information from different environment states – Agents cannot tell them apart…
2
Indistinguishable environments (2)
• Environment states consisting of – Boolean variable
temp
the room is above 35 o set to true if the temperature of Celsius (otherwise
temp
is false) – Boolean variable
smoke
set to true if smoke has been detected in the environment (otherwise
smoke
is false) • The full set of environment states is {
temp
,
smoke
,
temp
,
smoke
,
temp
,
smoke
,
temp
,
smoke
}
e
1 • The
see e
2
e
3 function of a thermostat agent is:
e
4
see
(
e
)=
p
1 if
e
=
e
1 or
e
=
e
2
p
2 if
e
=
e
3 or
e
=
e
4
3
Agents with State (1)
• Now, consider agents that record information about environment state and history:
4
Agents with State (2)
• Let I be the set of all internal states.
• The perception function see is unchanged: see : E
Per
• The action selection function is now a mapping action : I
Ac
from internal states to actions. • An additional function next is introduced, which maps an internal state and percept to an internal state: next : I
Per
I
5
Agent Control Loop
1. Agent starts in some initial internal state
i
0 2.
i cur
=
i
0 3. Agent observes the environment state
e see
(
e
) and generates a percept 4. Agent’s internal state is updated via
next i next
=
next
5. Action selected by agent is (
i cur
,
see
(
e
function ))
action
(
next
This action is then performed 6. Go to 3 with
i cur
=
i next
(
i cur
,
see
(
e
))
6
Tasks for Agents
• We build agents to carry out tasks for us.
• The task must be specified … • However, we want to tell agents what without telling them how to do it.
to do • So, the question is: – how do we give an agent information about what states are better than others?
7
Utility Functions over States (1)
• Possibility: associate utilities with individual states – Agents pursue states that maximise utility.
• A task specification is a function u : E which associates a real number with every state.
8
Utility Functions over States (2)
• But what is the value of a run? Is it – The minimum utility of all states in a run?
– The maximum utility of all states in a run?
– The sum of utilities of all states in a run?
– The average of utilities of all states in a run?
• Disadvantage: – It is difficult to specify a long term view when assigning utilities to individual states!
9
Utilities over Runs
• Another possibility – assign utilities to runs rather than individual states: u : R • This takes an inherently long term view.
• Other variations: – Incorporate probabilities of different states emerging • Difficulties with utility-based approaches: – Where do the numbers come from?
– We don’t think in terms of utilities!
– Hard to formulate tasks in these terms…
10
Utility in the TILEWORLD
• Simulated 2D grid environment with – Agents – Tiles – Obstacles and holes • Agents can move up, down, left or right – If agent is next to a tile, agent can push tile.
• Holes have to be filled up with tiles by the agent to score points. • Aim : score lots of points!!
• Holes appear and disappear at random… • Utility function of a run
r
: number of holes that appeared in r
11
Expected Utility & Optimal Agents
• P (r |Ag,Env ) denotes probability that run r occurs when agent Ag is placed in environment Env:
r
R(Ag,Env) P (r |Ag,Env ) = 1 • The optimal agent
Ag opt
in an environment Env is the one that maximises expected utility :
Ag opt
= arg max
Ag
AG
u(r)
r
R(Ag,Env) P(r |Ag,Env) (1)
12
Bounded Optimal Agents
• Some agents cannot be implemented on some computers; – E.g. a function Ag : R
E
Ac may need more memory than what is available to implement.
• AG
m
denotes those agents that can be implemented on machine (computer) m:
AG m
= {Ag | Ag AG and Ag can be implemented on m} • We can replace equation (1) with the following, which defines the bounded optimal agent
Ag opt
:
Ag opt
= arg max
Ag
AG m
r
R(Ag,Env) u(r) P(r |Ag,Env) (2)
13
Predicate Task Specifications
• A special case of assigning utilities to histories is to assign 0 (false) or 1 (true) to a run – If a run is assigned 1, then the agent succeeds on that run – Otherwise it fails.
• We call these predicate task specifications • We denote a predicate task specification as : : R {0,1}
14
Task Environments (1)
• A task environment is a pair
Env,
, where Env is an environment, and : R {0,1} is a predicate over runs.
• Let TE be the set of all task environments.
• A task environment specifies: – The properties of the system the agent will inhabit – The criteria by which an agent will be judged to have either failed or succeeded
15
Task Environments (2)
• R (Ag, Env ) denotes the set of all runs of agent Ag in environment Env that satisfy :
R
(Ag, Env ) = {r | r R (Ag, Env ) (r ) = 1} • We then say that an agent Ag in task environment
Env,
is successful if
R
(Ag, Env ) = R(Ag, Env )
16
The Probability of Success
• Let P (r | Ag, Env ) denote the probability that run r occurs when agent Ag is placed in environment Env.
• The probability P ( Ag in Env would be: |Ag,Env ) that is satisfied by P ( |Ag,Env) =
r
R
(Ag,Env ) P (r |Ag,Env )
17
Achievement & Maintenance Tasks (1)
• Most common tasks in agent-based computing are – Achievement tasks: “achieve state of affairs ” – Maintenance tasks: “maintain state of affairs ”
18
Achievement & Maintenance Tasks (2)
• An achievement task is specified by a set G of “good” or “goal” states G E: – An agent succeeds if it is guaranteed to bring about at least one of these states – We do not care which one – they are all considered equally good • A maintenance goal is specified by a set B of “bad” states B E : – An agent succeeds in a particular environment if it manages to avoid all states in B, i.e., it never acts so that a state in B occurs.
19
Agent Synthesis
• Agent synthesis is automatic programming: – The goal is to have a program that will take a task environment and – Automatically generate an agent that succeeds in this environment: syn : TE (Ag { }) (Think of as being like
null
in Java) • Common agent synthesis mechanism: genetic programming – E.g., in the evolution of simple predator and prey agents.
20
Agent Synthesis (2)
• A synthesis algorithm is sound – If it returns an agent, this agent succeeds in the task environment passed as input – I.e., the algorithm satisfies the condition: [syn(
Env,
)=Ag] [R(Ag,Env)=R (Ag,Env)] • A synthesis algorithm is complete – if it is guaranteed to return an agent whenever there exists an agent that will succeed in the task environment given as input; – I.e., the algorithm satisfies the condition:
Ag
AG
.
[R(Ag,Env)=R (Ag,Env)] [syn(
Env,
) ]
21
•
Suggested Reading
An Introduction to Multi-Agent Systems , M. Wooldridge, John Wiley & Sons, 2002. Chapter 2.
Multi-Agent Systems (CS5562) – Wamberto Vasconcelos 22