Transcript The El Farol Bar Problem on Complex Networks

The El Farol Bar Problem
on Complex Networks
Maziar Nekovee
BT Research
Mathematics of Networks, Oxford, 7/4/2006
Content
•
•
•
•
•
Motivation.
The El Farol Bar problem.
Solutions extensions and critique.
El Farol on social networks.
Conclusions.
Motivation
• Many real-life situations involve a set of independent
agents/entities competing for the same resource, in an
uncoordinated fashion.
drivers choosing similar travel routes.
visitors to a popular website.
…………………………..
……………………………
wireless devices (wifi, Bluetooth etc) sharing RF spectrum.
Scientific American, March 2006
a cognitive radio
a network of cognitive radios:
independent learners and
decision makers competing the same
resource (RF Spectrum)
The El Farol Bar Problem

Mathematical formulation
N: total number of customers
L: bar comfort level ( resource capcity); L  60
L
P= ; we are mainly interested in P~1
N
customer i attends El farol in week k
x i (k )  10 ifothewise
N
A(k)=  x i (k ): attendance in week k
i=1
A : <A>: average attendance
 2  ( A  L) 2 : measure of efficiency (  volatility)
I k  { A(k  m),... A(k  1)}: avilable information
u i (k )  x i (k )[2( A( k )  L)  1] : customers' utility at time k
Decision making model
• Each customer has a finite set of predictors which s/he uses to
predictor next week’s attendance, based on past attendance history.
{F1i , F2i , F3i ..Fni }
Asi (k  1)  Fsi [{ A(k  m), A(k  m  1),.. A(k  1)]  Fsi [ I k ]
• Each predictor has a score associated to it, which is updated
according to:
U si (k )  U si (k  1)  {[ Asi (k )  L][ A(k )  L)}
reinforced learning
• Customers use the predictor with the highest score to predict next
week’s attendance. Then:
if Aˆ simax  L s/he goes to El Farol
else stays at home
Predictors
...44 78 56 15 23 67 84 34 45 76 40 56 22 35
• The same as last week
35  go
• A (rounded) average of
the last m attendances.
49  go
• The same as 3 weeks
ago.
56
• The trend in the last 8
weeks (bounded by 0 and
100)
• …
76  stay
 go
Simplified El Farol (Minority Game)
Challet and Zhang,
1997.
N=: total number of agents
{-1,1}: set of actions avilable to agents (e.g. go/stay buy/sell)
a i (k ) : action of agent i at time k
N
A(k)=  number of agents choosing A/B
i=1
A : <A>: average attendance
 2  ( A  L) : Volatility
I k  { A(k  m),... A(k  1)}: information
u i (k )   xi (k ) A(k ) : agents achievd utility at time k
minority group is rewarded with | A |
majority is penalised with  | A |
Key questions
• Would bar attendance settles to some stationary state:
1
lim
M  M
M

A(k )  C ?
k  k0
• Can decentralised decision making result in efficient
utilization of the bar:
1
lim
M  M
M

k  k0
( A(k )  L) 2   ?
Nash Equilibrium
W. B. Arthur, 1984.
Critique of El Farol
• Predictor’s choice.
• Global information available to agents
regardless attendance.
• Other learning mechanisms.
• The impacts of inter-agent communication
(via a social network).
Statistical mechanic’s approach
Marsili, Challet, et al
information: Aˆ (k )   ( A(k )  L) Johnson et al
attendance : Iˆ(k )  { Aˆ (k  m),... Aˆ (k  1)  {0,1,0,.....1,1}
m bits
strategy : maps every possible m-history to a decision {0,1}
2m possible histories
2
2m
strategies
ski  {0,1,1.....0,0}
m
22 bits
A strategy soup
1
0
1
0
0
0
0
1
1
0
0
1
0
1
0
0
1
1
1
0
0
1
1
0
0
0
1
0
0
1
1
1
0
1
1
0
1
0
0
0
0
1
1
0
0
0
1
0
1
Marsili, Challet, Otino, 2003
Stochastic solution with simple
adaptive behaviour
Bell, Sethares, Buklew, 2003
• Agents adapt their attendance probabilitypi ( k )
through a simple process of “habit forming”:
• Full information on attendance:
pi (k  1)  pi (k )   ( A(k )  L)
(bounded by 0 and 1)
• Partial information on attendance:
pi (k  1)  pi (k )   xi (k )( A(k )  L)
(simplified) El Farol on networks
El Farol on social networks
Galstyan, Kolar, Lerman, 2003
• N agents connected via a social network.
• Instead of interacting via a global signal of
attendance history, agents interact with K
other (randomly chosen) agents.
x (k  1)  F
i
i
1
i
i
smax
i
2
i
K
l1i
lKi
i
[ x (k ),...x (k )]
{l , l ,...l }: "neighbours" of agent i
Emergence of scale-free influence networks
Toroczkai, Anghel, Basselr,
Korniss, 2004
• A social network of N agents through which
agents communicate (ER random graph).
• Agents play the minority game on the graph,
using reinforced learning to select a leader
among their nearest neighbours:
s i (k  1)  Max{s
lki1
max
(k ) s
lki 2
max
lki g
i
(k ),..smaxi (k ), smax
(k )}
{l1i , l2i ,...lgi i }: first neighbours of agent i
Emergence of scale-free influence network
Toroczkai, Anghel, Basselr,
Korniss, 2004
Conclusions
• The El Farol bar problem (EFBP) is highly relevant to
understanding distributed resource sharing in interacting multi-agent
systems.
• Many unexplored questions remain.
• Information flow via inter-agent networks can greatly impact the
dynamics of EFP.
work in progress
•
EFP on cognitive radio networks.
Thanks to Matteo Marsili for pointing me to the EFBP