Transcript Slides.

Costly valuation
computation/information acquisition
in auctions:
Strategy, counterspeculation, and deliberation
equilibrium
Tuomas Sandholm
Computer Science Department
Carnegie Mellon University
Mainly based on the following papers:
Larson, K. and Sandholm, T. 2001. Costly Valuation Computation in Auctions. In Proceedings of the Theoretical Aspects of Reasoning about Knowledge (TARK).
Larson, K. and Sandholm, T. 2001. Computationally Limited Agents in Auctions. In Proceedings of the International Conference on Autonomous Agents, Workshop on Agent-based Approaches to B2B.
TRACONET, 1990-91
$ 2,000
Auction
$ 1,700
Contract:
Task transferred
[Sandholm NOAS-91, AAAI-93]
Bidders may need to compute their
valuations for (bundles of) goods
• In many (even private-values quasilinear) applications, e.g.
– Vehicle routing problem in transportation exchanges
– Manufacturing scheduling problem in procurement
• Value of a bundle of items (tasks, resources, etc) =
value of solution with those items - value of solution without them
• Our models apply to information gathering as well
3
Software agents for auctions
• Software agents exist that bid on behalf of user
• We want to enable agents to not only bid in auctions,
but also determine the valuations of the items
• Agents use computational resources to compute
valuations
• Valuation determination can involve computing on NPcomplete problems (scheduling, vehicle routing, etc.)
• Optimal solutions may not be possible to determine due to
limitations in agents’ computational abilities (i.e. agents have
bounded rationality)
4
Bounded rationality
• Work in economics has largely focused on descriptive models
• Some models based on limited memory in repeated games
[Papadimitriou, Rubinstein, …]
• Some AI work has focused on models that prescribe how
computationally limited agents should behave
[Horvitz; Russell & Wefald; Zilberstein & Russell; Sandholm & Lesser;
Hansen & Zilberstein, …]
– Simplifying assumptions
• Myopic deliberation control
• Asymptotic notions of bounded optimality
• Conditioning on performance but not path of an algorithm
• Simplifications can work well in single agent settings, but any
deviation from full normativity can be catastrophic in
multiagent settings
Incorporate deliberation (computing) actions into agents’
5
strategies => deliberation equilibrium
Simple model: can pay c to find one’s own valuation
=> Vickrey auction no longer has a dominant strategy
[Sandholm ICMAS-96, International J. of Electronic Commerce 2000]
Thrm. In a private value Vickrey auction with uncertainty about an agent’s own
valuation, a risk-neutral agent’s best strategy can depend on others.
E.g. two bidders (1 and 2) bid for a good.
v1 uniform between 0 and 1; v2 deterministic, 0 ≤ v2 ≤ 0.5
Agent 1 bids 0.5 and gets item at price v2:
pdf
gain
loss
v2
1
v1
1
]   v1  v2 dv1 
nopay
E[1
0
1
v
2 2
Say agent 1 has the choice of paying c to find out v1. Then
agent 1 will bid v1 and get the item iff v1 ≥ v2 (no loss
possibility, but c invested)
1
pay
E[1
]  c 
v
1
v2
 v 2 dv
E[1pay]  E[ nopay
]  v 2  2c
1
Same model studied more recently in the literature on “information acquisition in auctions”
6
[Compte and Jehiel 01, Rezende 02, Rasmussen 06]
Quest for a general fully normative model
Auctioneer
bid(result)
bid(result)
Agent
Agent
Deliberation controller
Deliberation controller
(uses performance profile)
(uses performance profile)
Compute!
result
Compute!
result
Domain problem solver
Domain problem solver
(anytime algorithm)
(anytime algorithm)
7
Normative control of deliberation
• In our setting agents have
– Limited computing, or
– Costly computing
• Agents must decide how to use their limited
resources in an efficient manner
• Agents have anytime algorithms and use
performance profiles to control their
deliberation
8
Anytime algorithms can be used
to approximate valuations
• Solution improves over time
• Can usually “solve” much larger problem instances than complete
algorithms can
• Allow trading off computing time against quality
– Decision is not just which bundles to evaluate, but how carefully
• Examples
– Iterative refinement algorithms: Local search, simulated annealing
– Search algorithms: Depth first search, branch and bound
9
Performance profiles of
anytime algorithms
• Statistical performance profiles characterize the quality of
an algorithm’s output as a function of computing time
• There are different ways of representing performance
profiles
– Earlier methods were not normative: they do not capture all the
possible ways an agent can control its deliberation
• Can be satisfactory in single agent settings, but catastrophic in multiagent
systems
10
Performance profiles
Deterministic
performance profile
Solution
quality
Variance introduced by
different problem instances
Solution
quality
Optimum
Computing time
Computing time
11
[Horvitz 87, 89, Dean & Boddy 89]
Table-based representation of uncertainty in
performance profiles
[Zilberstein & Russell IJCAI-91, AIJ-96]
.08 .19 .24
.15 .30 .17 .39
.16 .10 .16 .25 .30 .22
.08 .04 .17 .20 .22 .30 .24 .19 .15
Solution
quality .09 .10 .20 .22 .23 .37 .31 .13 .15
.11
.14
.33 .18 .21 .18 .08
.22
.17
.25 .24 .15 .13
.40
.31
.15 .19 .05
.15
.20
.03
Conditioning on
solution quality so far
[Hansen &
Zilberstein AAAI-96]
Ignores conditioning
on the path
.03
12
Computing time
Performance profile tree [Larson & Sandholm AAAI-00, AIJ-01, TARK-01]
P(B|A)
B
4
5
4
A
0
Solution
quality
• Normative
10
3
P(C|A)
C
6
15
5
20
2
– Allows conditioning on path of solution quality
– Allows conditioning on path of other solution features
– Allows conditioning on problem instance features (different
trees to be used for different classes)
• Constructed from statistics on earlier runs
13
Performance profile tree…
• The following can be augmented into the model:
– Randomized algorithms
– Agent not knowing which algorithms others are using
– Agent having uncertainty about others’ problem instances
• Agent can emulate different scenarios of others
p(0)
Random node
Value node
0
5
4
6
p(1)
2
Our results hold in this augmented setting
4
10
3
15
14
20
Roles of computing
• Computing by an agent
– Improves the solution to the agent’s own problem(s)
– Reduces uncertainty as to what future computing steps
will yield
– Improves the agent’s knowledge about others’ valuations
– Improves the agent’s knowledge about what problems
others may have computed on and what solutions others
may have obtained
• Our results apply to different settings
– Computing increases the valuation (reduces cost)
– Computing refines the valuation estimate
15
“Strategic computing”
• Good estimates of the other bidders’ valuations can
allow an agent to tailor its bids to achieve higher utility
• Definition. Strong strategic computing: Agent uses
some of its deliberation resources to compute on others’
problems
• Definition. Weak strategic computing: Agent uses
information from others’ performance profiles
• How an agent should allocate its computation (based
on results it has obtained so far) can depend on how
others allocate their computation
– “Deliberation equilibrium” [AIJ-01]
16
Theorems on strategic computing
Auction
mechanism
Single
item for
sale
Counterspeculation
by rational
agents ?
Limited
computing
Costly
computing
First price sealed-bid
yes
yes
yes
Dutch (1st price descending)
yes
yes
yes
Vickrey (2nd price sealed bid)
no
no
yes
English (1st price ascending)
no
no
yes
no
yes
yes
Multiple Generalized Vickrey
items for On which <bidder, bundle>
sale
pair to allocate next
computation step ?
Strategic computing ?
If performance profiles are deterministic, only weak strategic computing can occur
 New normative deliberation control method uncovered a new phenomenon 17
Costly computing in English auctions
• For rational bidders, straightforward bidding is ex post eq.
• Thrm: If at most one performance profile is stochastic, no
strong strategic computing occurs in equilibrium
• Thrm: If at least two performance profiles are stochastic,
strong strategic computing can occur in equilibrium
– Despite the fact that agents learn about others’ valuations by
waiting and observing others’ bids
– Passing & resuming computation during the auction is allowed
– Proof. Consider an auction with two bidders:
• Agent 1 can compute for free
• Agent 2 incurs cost 1 for each computing step
18
Performance profiles of the proof
Agent 1’s problem
p(high1)
Agent 2’s problem
p(high2)
high1
0
0
1-p(high1)
low1
high2
0
1-p(high2)
low2
low2 < low1 < high2 < high1
Since computing one step on 2’s problem does not yield any information,
we can treat computing for two steps on 2’s problem atomically
19
Proof continued…
• Agent 1 has straightforward (ex post eq.) strategy:
– Compute only on own problem & increment bid whenever
• Agent 1 does not have the highest bid and
• Highest bid is lower than agent 1’s valuation
• Agent 2’s strategy:
– CASE 1: bid1 > low1
• Agent 2 knows that agent 1 has valuation high1
• Agent 2 cannot win, and thus has no incentive to compute or bid
– CASE 2: bid1< low2
• Agent 2 continues to increment its own bid
• No need to compute since it knows that its valuation is at least low2
– CASE 3: low1  bid1  low2
• If Agent 2 bids, he should bid bid1 + ε
• His strategy depends on the performance profiles…
20
Decision problem of agent 2 in CASE 3
Agent 2’s utility
low2 < low1 < high2 < high1
Decision node for agent 2
high2
Chance node for agent 1’s
performance profile
Compute on 2’s
Chance node for agent 2’s
performance profile
high
-3
low2
high2
Bid
1
Withdraw
-1
Compute on 2’s
Compute on
1’s problem
Withdraw
Compute on 1’s
Compute on
2’s problem
high2
-1
Withdraw
-2
low2
Bid
high1
Withdraw
0
high2
high2-low1-3
-3
low2
high1
low1
Compute on 1’s
high1
Bid
low1
low2
high2-low1-1
low2-low1-1
-3
high2-low1-3
-2
high2-low1-2
-3
-3
high1
Withdraw
0
high2
low1
-1
low1
high1
Bid
-1
low2
low2
high2
Bid
low1
-3
low1
-2
high2-low1
low2-low1
-2
-2
21
Under what conditions does strong
strategic computing occur?
low2 =3, low1 =12, high2 =22, high1 =30
1
0.8
Probability that
agent 2 will
have its high
valuation
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
Probability that agent 1 will have its high valuation22
1
Other variants we solved
• Agents cannot pass on computing during the auction & resume
computing later during the auction
– Can make a difference in English auctions with costly computing, but
strong strategic computing is still possible in equilibrium
• Agents can/cannot compute after the auction
• 2-agent bargaining (again with performance profile trees)
– Larson, K. and Sandholm, T. 2001. Bargaining with Limited
Computation: Deliberation Equilibrium. Artificial Intelligence, 132(2),
183-217.
– Larson, K. and Sandholm, T. 2002. An Alternating Offers Bargaining
Model for Computationally Limited Agents. In Proceedings of the First
International Joint Conference on Autonomous Agents and Multiagent
Systems (AAMAS), Bologna, Italy, July.
23
Designing mechanisms
for agents whose valuation deliberation
is limited or costly
[Larson & Sandholm AAMAS-05]
Mechanism desiderata
• Preference formation-independent
– Mechanism should not be involved in agents’ preference formation process
• (otherwise revelation principle applies trivially)
• I.e., agents communicate to auctioneer in terms of valuation (or expected valuations)
• Deliberation-proof
– In equilibrium, no agent should have incentive to strategically deliberate
• Non-misleading
– In equilibrium, no agent should follow a strategy that causes others to believe that
its true preferences are impossible
• E.g. agent should not want to report a valuation and willingness to pay higher than his
true valuation
• <= truthful (equivalence in the case of direct mechanisms)
• Thm. There exists no direct or indirect mechanism (where any agent can
affect the allocation regardless of others’ revelations) that satisfies all these 3
properties
27
Recent work on overcoming the impossibility
• Restricted settings
– Not too much asymmetry – tends to avoid strong strategic computing
• Relaxing properties (but not Non-Misleading)
– Relax Deliberation-Proof: Encourage strategic deliberation
• Incentives for the right (cheap) agents to compute & share right information?
–
Some agents as “experts” [Ito et al. AAMAS-03]
• Cavallo & Parkes [AAAI-08] get efficiency and no deficit in (within-period) ex post
equilibrium. Agents report deliberation states and center says which agent deliberates next
–
–
–
Assumptions
» Only one agent can compute at a time
» Valuations increase with computation
» Time is discounted
Without strategic deliberation possibility, achievable using dynamic VCG [Bergemann&Valimaki 07]
With strategic deliberation, use payments such that equilibrium utilities are exactly as they would be if an agent’s
deliberation processes about other agents’ values were in fact about its own value
– Relax Preference-Formation Independent
•
•
•
•
Mechanism guides deliberation
Revealing only some info about agents’ deliberative capabilities?
Related to “search” & sequential preference elicitation
Generalizing [Cremer et al. 03] to multi-step info gathering & to gathering info about other
agents as well
• [Larson AAMAS-06] studies mechanism design for the case where agents can only
deliberate on their own valuations
32