Transcript bachrachb08.ppt

Distributed Multiagent Resource
Allocation In Diminishing
Marginal Return Domains
Yoram Bachrach(Hebew University)
Jeffrey S. Rosenschein (Hebrew University)
Outline

Multiagent Resource Allocation (MARA)

General problem
 Applications

Centralized and decentralized mechanisms


Specific restricted domain


VCG solution in restricted domain
Allocation by interaction


Selfish behavior challenge
Market motivation behind method
Allocation protocol and suggested strategies

Convergence to optimal allocation
 Strategic and selfish behaviour
 Expected time to convergence

Conclusions and future research
Multiagent Resource Allocation

Allocating resources to users
 Scarce
resources
 Selfish agents with private information


Both users and resource owners
An allocation maps resources to users
MARA Applications
Industrial procurement
 Satellite resources
 Tasks in manufacturing systems
 Grid computing
 RF spectrum and coverage
…

MARA Domain Properties

Divisible / Indivisible


Sharable / Non-Sharable


Can a resource be allocated to several agents simultaneously?
Single-Unit / Multi-Unit


Can parts of a single resource be allocated to several agents?
Are there bundles of identical resources?
Transferable / Non Transferable Utility

Can agents compensate by transferring utility among them?
MARA Approaches

Attempt to maximize social welfare



Centralized mechanisms


A central mechanism gets the agents’ preferences and chooses
an outcome
Decentralized approaches


Other possible goals – Maximin, fairness, …
There may be more than one optimal allocation
Agents actively participate in choosing the outcome
Problem – agents are selfish and attempt to maximize
their own utility
Centralized Mechanisms

The mechanism must elicit the agents’ private information
about allocations


We are interested in incentive compatible mechanisms



But agents may manipulate to increase their own utility
Agents reply truthfully, under a certain rational behavior
Rational behavior captured in a game theoretic solution concept
Vickery-Clarke-Groves (VCG) approach


Tax agents to make truth telling is a dominant strategy
Strategyproof, allocatively efficient but only weakly budget balanced
Distributed Mechanisms

Central mechanisms may not be
appropriate in distributed environments
 Hard
to establish a trusted central authority
 Scalability concerns – the central mechanism
may be a performance bottleneck

Have agents interact among themselves to
choose the allocation
 Need
to define the protocol for interaction
 Selfish agents may still manipulate
Specific Domain

Set of identical agents


Set of resources



Each agent only requires a single resource, and does not benefit
from being allocated more than one resource
Cannot be divided among agents
Can be shared among agents
Diminishing marginal production

The total utility of the agents who are allocated a certain
resource drops as more agents use that resource
Diminishing Marginal Return
10
14
15
5
10
7
5
7
5
Diminishing Marginal Return
Total production is 10
10
14
15
5
10
7
5
7
5
Diminishing Marginal Return
Total production increases to 14
10
14
15
5
10
7
5
7
5
Diminishing Marginal Return
Total production increased by 4 when adding a single agent
Marginal production of 4
10
14
15
5
10
7
5
7
5
Diminishing Marginal Return
Total production increased by 1 when adding a single agent
Marginal production of 1
10
14
15
5
10
7
5
7
5
What needs to be decided?

A mechanism must decide:
 An

allocation – which agent gets which resource
We want to maximize the social welfare – total production
 Utility

Agents gain utility due to the allocation


Resource owners receive nothing
Resource owners hold the private information



transfers
Eliciting this information requires incentivizing the resource
owners to report their production function
Requires giving resource owners some of the utility
We assume the total production across all the
resources can be redistributed in any way
VCG in Restricted Domain

Easy to compute an optimal allocation
 Resources
report total production functions
 Find maximal social welfare by a greedy algorithm


Assign to the resource with maximal marginal production
Induce truthfullness by VCG tax
 Requires


establishing a trusted central authority
Trust and security issues, central bottleneck, …
Weakly budget balanced – some of the total production is
kept in the mechanism and not distributed
Allocation by Interaction

Define a protocol for interaction between agents and
resource owners


Interaction proceeds in discrete time rounds



Simulate a market for services
Each round determines both an allocation and transfers
Design protocol and suggest interaction strategies
so that the optimal allocation is always reached
Challenges


Achieve the optimal allocation despite selfishness
Make sure the optimal allocation is reached quickly
Interaction Protocol
R1
Currently on R1,
getting utility 5
R2
Round Payment (5)
R3
Interaction Protocol
R1
Currently on R1,
getting utility 5
R2
Resource Request
R3
Interaction Protocol
R1
R2
Payment Bid (10)
R3
Interaction Protocol
Switch to R2 with
utility 10
R1
R2
Accept
R3
Interaction Protocol
Stay on R1, with
utility 5
R1
R2
Decline
R3
Interaction Protocol
R1
Currently on R2
with utility 10
R2
Round Payment 10
R3
Interaction Protocol
R1
Currently on R2
with utility 5
R2
Payment Change (5)
The Resource Owner’s
Perspective
Production – 12
Payments – 10
Utility – 2
12
Production – 13
Payments – 12
Utility – 1
13
4
5
4
5
4
Chosen Allocation

The interaction decides both the allocation and
redistribution of the utility





Agents are allocated the last resource whose bid they accepted
Agents get the utility as in the last payment bid they accepted
Resource owners keep the reminder of the production on the
resource not redistributed to the agents
The allocation may change at the end of every round
An allocation is stable if once reached it never changes

Depends on the strategies of the participants

Agents and resource owners
Suggested Strategy - Agents

Each round, randomly choose a resource
and request using the resource
 If
the bid in that resource is better than the
current bid, switch to that resource (accept)
 If the bid is lower than the current resource
offers, stay with current resource
Suggested Strategy –
Resource Owners



Keep the agents’ share of the utility in the level
of the marginal production on the resource
On round start, offer all the agents allocated to
the resource the current last marginal production
Answer resource requests with bid of the next
marginal production on the resource
 If
accepted, set the bid for all the agents to the new
marginal production by a Payment Change message
 If declined – do nothing
Resource Owners - Example
MP = 4
10
MP = 1
14
15
1
10
4
1
4
1
Protocol Stable Allocation

Given a set of strategies for the agents and resource owners, a
protocol stable allocation is one that, once reached, never changes


Under these strategies, no interaction results in an agent switching to a
different resource
Protocol stable under the suggested strategies

No agent is ever given a bid higher than what he is currently getting on
his current resource
 Resource owners bid the next marginal production
 There is no resource where the next marginal production is greater than
the current marginal production on other resources

Similar to greedily allocating agents to resources according to marginal production
Convergence to Optimum

Under the suggested strategies, the chosen allocation
always converges to the optimal allocation

Monotonic improvement


Stability in optimum


The optimal allocation is protocol stable
No “local” optima – protocol stable is optimal


If an agent switches resources, the social welfare increases
If a non optimal allocation is chosen, there is a possible round
where an agent switches resources
What about strategic behavior?
Strategic Behavior

Agents and resource owners have to follow the
protocol, but not the suggested strategies
 Might
obtain higher utility by choosing a different
strategy


Agents may accept a bid lower than what they currently have
Resource owners may suggest a bid different than the
current marginal production



Higher, to attract more agents
Lower, to give a lower share of utility to the agents
Is such strategic behavior rational for self
interested agents?
Strategic Agents (Our domain)

If an agent gained from strategic behavior,
we still reach an optimal allocation
 If
a single agent has deviated from the
suggested strategy and gained utility

Gained utility: a protocol stable allocation has been
reached, in which the agent gets a higher utility
 Then
the reached protocol stable allocation is
also optimal
Strategic Resource Owners

Resource owners who set too high a bid


Attract more agents but pay more and lose utility
Resource owners who set too low a bid

Pay less, but lose agents to competing resources



who offer higher bids
When the domain is competitive for resource owners, such a
manipulation is irrational
Highly competitive settings

Condition that occurs mostly in environments where there are
many resources with similar marginal production values

Similar resources or slight changes in marginal production
Strategic Resource Owners

In our specific domain
 Diminishing
marginal return
 Highly competitive for resource owners

If a resource owner gained from strategic
behavior, we still reach an optimal allocation
 If
a single resource owner has deviated from the
suggested strategy and gained utility

Gained utility: a protocol stable allocation has been reached,
in which the resource owner gets a higher utility
 Then
the reached protocol stable allocation is optimal
Convergence Time

When agents and resource owners behave
rationally, we converge to an optimal allocation
 But

how quickly is the optimal allocation reached?
Under the suggested strategies
 Expected
time to convergence:
 Bound on convergence time:

Quick polynomial convergence
Related Work

TFG-MARA survey


Distributed mechanism design approaches


J. Feigenbaum and S. Shenker. Distributed algorithmic mechanism design: Recent results and
future directions.
Scheduling domains


Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J. A.
Rodríguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation.
B. Heydenreich, R. Muller, and M. Uetz. Decentralization and mechanism design for online machine
scheduling.
Negotiations over resources


U. Endriss, N. Maudet, F. Sadri, and F. Toni. Negotiating socially optimal allocations of resources.
T. W. Sandholm. Contract types for satisficing task allocation.
Conclusions

A distributed approach to resource allocation in
a specific domain
 Achieves
optimal allocation (maximal social welfare)
 No central authority required
 All utility divided among agents and resource owners

“Strongly budget balanced”
 Quick

convergence
Can a similar approach be applied to other
domains (or more general domains)?