– Last position may require special handling Yahoo! Confidential

Next-price equilibrium

• Next-price auction: Not truthful: no dominant strategy • What are Nash equilibrium strategies? There are many!

• • Which Nash equilibrium seems “focal” ?

Locally envy-free equilibrium

[Edelman, Ostrovsky, Schwarz 2005]

Symmetric equilibrium

[Varian 2006] Fixed point where bidders don’t want to move  or  – Bidders first choose the optimal position for them: position i – Within range of bids that land them in position i, bidder chooses point of indifference between staying in current position and swapping up with bidder in position i-1 • Pure strategy (symmetric) Nash equilibrium • Intuitive: Squeeze bidder above, but not enough to risk “punishment” from bidder above Yahoo! Confidential

Next-price equilibrium

• Recursive solution: pos i-1 *adv i *b i = (pos i-1 -pos i )*adv i *v i +pos i *adv i+1 *b i+1 b i = (pos i-1 -pos i )*adv i *v i +pos i *adv i+1 *b i+1 pos i-1 *adv i • Nomenclature: Next price = “generalized second price” (GSP) Yahoo! Confidential

Ad exchanges

• Right Media • Expressiveness

Research

Online Advertising Evolution

1. Direct: Publishers sell owned & operated (O&O) inventory 2. Ad networks: Big publishers place ads on affiliate sites, share revenue AOL, Google, Yahoo!, Microsoft 3. Ad exchanges: Match buy orders from advertisers with sell orders from publishers and ad networks Key distinction: exchange does not “own” inventory

Exchange Basics

[Source: Ryan Christensen]

Advertisers

Netflix Vonage Auto.com

…

Networks

Ad.com

CPX Tribal …

Demand

Exchange

Inventory

Publishers

MySpace Six Apart Looksmart Monster … Yahoo! Confidential

[Source: Ryan Christensen]

Right Media Publisher Experience The publisher can approve creative from each advertiser

• • • •

Publisher can select / reject specific advertisers Green = linked network Light Blue = direct advertiser Publishers can traffic their own deals by clicking “Add Advertiser”

Yahoo! Confidential

[Source: Ryan Christensen]

Right Media Advertiser Experience

• • •

Advertisers can set targets for CPM, CPC and CPA campaigns Set budgets and frequency caps Locate publishers, upload creative and traffic campaigns

Yahoo! Confidential

Expressiveness

• • • • • • • • “I’ll pay 10% more for Males 18-35” “I’ll pay $0.05 per impression, $0.25 per click, and $5.25 per conversion” “I’ll pay 50% more for exclusive display, or w/o Acme” “My marginal value per click is decreasing/increasing” “Never/Always show me next to Acme” “Never/Always show me on adult sites” “Show me when Amazon.com is 1st algo search result” “I need at least 10K impressions, or none” “Spread out my exposure over the month” “I want three exposures per user, at least one in the evening” Design parameters: Advertiser needs/wants, computational/cognitive complexity, revenue Yahoo! Confidential

Research

Expressiveness Example

•

Competition constraints

b xCTR = RPS 3 x .05 = .15

1 x .05 = .05

Research

Expressiveness Example

•

Competition constraints

monopoly bid b xCTR = RPS 4 x .07 = .28

Research

Expressiveness: Design

•

Multi-attribute bidding Male users (50%) Un differentiated Advertiser 1 $1 Female users (50%) $2 $1.50

Advertiser 2 $2 $1 $1.50

Pre-qualified (50%) Other (50%) Advertiser 1 $2 $1 Advertiser 2 $2 $1 Un differentiated $1.50

$1.50

Expressiveness: Less is More

• Pay per conversion: Advertisers pay for user actions (sales, sign ups, extended browsing, ...) – Network sends traffic – Advertisers rate users/types 0-100 Pay in proportion – Network learns, optimizes traffic, repeat • Fraud: Short-term gain only: If advertisers lie, they stop getting traffic Yahoo! Confidential

Expressiveness: Less is More

• “I’m a dry cleaner in Somerset, New Jersey with $100/month. Advertise for me.” • Can advertisers trust network to optimize?

Yahoo! Confidential

Research

Coming Convergence: ML and Mechanism Design

Stats/ML/Opt Engine Mechanism (Rules) e.g. Auction, Exchange, ...

Stats/ML/Opt Engine Stats/ML/Opt Engine Stats/ML/Opt Engine Stats/ML/Opt Engine

Research

ML Inner Loop

• • • •

Optimal allocation (ad-user match) depends on: bid, E[clicks], E[sales], relevance, ad, advertiser, user, context (page, history), ...

Expectations must be learned Learning in dynamic setting requires exploration/exploitation tradeoff Mechanism design must factor all this in! Nontrivial.

Selected Survey of Internet Advertising Research

Source: S. Lahaie

An Analysis of Alternative Slot Auction Designs for Sponsored Search

Sebastien Lahaie

, Harvard University* *work partially conducted at Yahoo! Research

ACM Conference on Electronic Commerce, 2006

Source: S. Lahaie

Objective

• Initiate a systematic study of Yahoo! and Google slot auctions designs.

• Look at both “short-run” incomplete information case, and “long-run” complete information case.

Source: S. Lahaie

Outline •

• • • • Incomplete information (one shot game) Incentives Efficiency Informational requirements Revenue

•

• • • Complete Information (long-run equilibrium) Existence of equilibria Characterization of equilibria Efficiency of equilibria (“price of anarchy”)

Source: S. Lahaie • • • • •

The Model

slots, bidders • • The type of bidder i consists of a

value

per click of , realization a

relevance

, realization is bidder i’s

revenue,

realization Ad in slot is viewed with probability So CTR i,k = Bidder i’s utility function is quasi-linear:

Source: S. Lahaie

The Model (cont’d) • • • •

is i.i.d on according to is continuous and has full support is common knowledge Probabilities are common knowledge.

• •

Only bidder i knows realization Both seller and bidder i know , but other bidders do not

Source: S. Lahaie

Auction Formats • • • • •

Rank-by-bid (RBB): bidders are ranked according to their declared values ( ) Rank-by-revenue (RBR): bidders are ranked according to their declared revenues ( ) First-price: a bidder pays his declared value Second-price (next-price): For RBB, pays next highest price. For RBR, pays All payments are

per click

Source: S. Lahaie •

Incentives

First-price: neither RBB nor RBR is truthful • Second-price: being truthful is not a dominant strategy, nor is it an

ex post

Nash equilibrium (by example): 1 6 1 4 • Use Holmstrom’s lemma to derive truthful payment rules for RBB and RBR: • RBR with truthful payment rule is VCG

Source: S. Lahaie

Efficiency

• Lemma: In a RBB auction with either a first- or second-price payment rule, the symmetric Bayes-Nash equilibrium bid is strictly increasing with

value

. For RBR it is strictly increasing with

product

.

• RBB is not efficient (by example).

0.5

6 1 4 • Proposition: RBR is efficient (proof).

Source: S. Lahaie • • •

First-Price Bidding Equilibria

is the expected resulting clickthrough rate, in a symmetric equilibrium of the RBB auction, to a bidder with value y and relevance 1.

is defined similarly for bidder with product y and relevance 1.

Proposition: Symmetric Bayes-Nash equilibrium strategies in a first-price RBB and RBR auction are given by, respectively:

Source: S. Lahaie

Informational Requirements

• • RBB: bidder need not know his own relevance, or the distribution over relevance.

RBR: must know own relevance and joint distribution over value and relevance.

Source: S. Lahaie

Revenue Ranking

• • Revenue equivalence principle: auctions that lead to the same allocations in equilibrium have the same expected revenue. Neither RBB nor RBR dominates in terms of revenue, for a fixed number .

of agents, slots, and a fixed

Source: S. Lahaie

Complete Information Nash Equilibria

Argument: a bidder always tries to match the next lowest bid to minimize costs. But it is not an equilibrium for all to bid 0. Argument: corollary of characterization lemma.

Source: S. Lahaie

Characterization of Equilibria

• RBB: same characterization with replacing

Source: S. Lahaie Define:

Price of Anarchy

Source: S. Lahaie

Exponential Decay

• Typical model of decaying clickthrough rate: • [Feng et al. ’05] find that their actual clickthrough data is fit well by such a model with • In this case

Source: S. Lahaie

Conclusion

• • • • • Incomplete information (on-shot game): Neither first- nor second-pricing leads to truthfulness.

RBR is efficient, RBB is not RBB has weaker informational requirements Neither RBB nor RBR is revenue-dominant • • • Complete information (long-run equilibrium): First-price leads to no pure strategy Nash equilibria, but second-price has many.

Value in equilibrium is constant factor away from “standard” value.

Source: S. Lahaie

Future Work

• Better characterization of revenue properties: under what conditions on does either RBB or RBR dominate?

• Revenue results for complete information case (relation to Edelman et al.’s “locally envy-free equilibria”).

Source: S. Lahaie

Research Problem: Online Estimation of Clickrates • •

Make virtually no assumptions on clickrates.

Each different ranking yields (1) information on clickrates and (2) revenue.

•

Tension between optimizing current revenue based on current information, and gaining more info on clickrates to optimize future revenue (multi-armed bandit problem...)

•

Twist: chosen policy determines rankings, which will affect agent’s equilibrium behavior.

Equilibrium revenue simulations of hybrid sponsored search mechanisms

Sebastien Lahaie

, Harvard University* *work conducted at Yahoo! Research

David Pennock

, Yahoo! Research

Revenue effects

Overture

Highest bid wins

Google/Yahoo!

Highest bid*CTR wins

Hybrid

Highest bid*(CTR) s wins s=0 s=1 s=1/2 ?

s=3/4 ?

• What gives most

revenue

?

–

Key

: If rules change, advertiser bids will change – Use Edelman et al.

envy-free equilibrium

solution Yahoo! Confidential

Source: S. Lahaie

Monte-Carlo simulations

• 10 bidders, 10 positions • Value and relevance are i.i.d. and have lognormal marginals with mean and variance (1,0.2) and (1,0.5) resp.

• Spearman correlation between value and relevance is varied between -1 and 1.

• Standard errors are within 2% of plotted estimates.

Yahoo! Confidential

Source: S. Lahaie Yahoo! Confidential

Source: S. Lahaie Yahoo! Confidential

Source: S. Lahaie Yahoo! Confidential

Source: S. Lahaie

Preliminary Conclusions

• With perfectly negative correlation (-1), revenue, efficiency, and relevance exhibits threshold behavior • Squashing up to this threshold can improve revenue without too much sacrifice in efficiency or relevance • Squashing can significantly improve revenue with positive correlation Yahoo! Confidential

Source: M. Schwarz

Pragmatic Robots and Equilibrium Bidding in GSP Auctions

Michael Schwarz

, Yahoo! Research

Ben Edelman

, Harvard University

Testing game theory

Thanks: M. Schwarz • Empirical game theory – Analytic solutions intractable in all but simplest settings – Laboratory experiments cumbersome, costly – Agent-based simulation: easy, cheap, allow massive exploration;

Key:

modeling realistic strategies • Ideal for agent-based simulation: when

real

decisions are already delegated to software economic

“If pay-per-click marketing is so strategic, how can it be automated? That’s why we developed Rules-Based Bidding. Rules-Based Bidding allows you to apply the kind of rules you

would use if you were managing your bids manually.” Atlas http://www.atlasonepoint.com/products/bidmanager/rulesbased Yahoo! Confidential

Yahoo! Confidential Source: M. Schwarz

Bidders’ actual strategies

Source: M. Schwarz

Models of GSP

1.

2.

Static game of complete information Generalized English Auction (simple dynamic model) More realistic model • • Each period one random bidder can change his bid Before the move a bidder observes all standing bids Yahoo! Confidential

Pragmatic Robot (PR)

Source: M. Schwarz • Find current optimal position i Implies range of possible bids: Static best response (BR set) • Choose envy-free point inside BR set: Bid up to point of indifference between position i and position i-1 • If start in equilibrium PRs stay in equilibrium Yahoo! Confidential

0.8

0.6

0.4

0.2

0 1.6

1.4

1.2

1 Yahoo! Confidential 100

Convergence of PR Simulation

Source: M. Schwarz 200 300 400 500 simulation rounds - convergence to 0.000001 after 329 iterations 600 Total Surplus Search Engine Revenue Advertiser Surplus Computed Equilibrium 700 800

Yahoo! Confidential

Convergence of PR

Source: M. Schwarz

Convergence of PR

Source: M. Schwarz • The fact that PR converges supports the assertion that the equilibrium of a simple model informs us about the outcome of intractable dynamic game that inspired it ?

Complex game that we can not solve Simple model inspired by a complex game Yahoo! Confidential

Source: M. Schwarz

Playing with Ideal Subjects

Largest Gap

(commercially available strategy) Moves your keyword listing to the largest bid gap within a specified set of positions Regime One: 15 robots all play Largest Gap Regime Two: one robot becomes pragmatic By becoming Pragmatic pay off is up 16% Other assumptions: values are log normal, mean valuation 1, std dev 0.7 of the underlying normal, bidders move sequentially in random order Yahoo! Confidential

Source: M. Schwarz

ROI

• Setting ROI target is a popular strategy • For any ROI goal the advertiser who switches to pragmatic gets higher payoff Yahoo! Confidential

Source: M. Schwarz

If others play ROI targeter

• Bidders

1,...,K-1

targeting strategy bid according to the ROI • What is

K

’s bidder payoffs if bidder

K

plays best response?

bidder ROI targeting PR

1

…

K-1 K

0.0387

0.0457

Yahoo! Confidential

Reinforcement Learner vs Pragmatic Robot

• Pragmatic learner outperforms reinforcement learner (that we tried) • Remark: reinforcement learning does not converge in a problem with big BR set Source: M. Schwarz Yahoo! Confidential

Thanks: M. Schwarz

Conclusion

• A strategy inspired by theory seems useful in practice: PR beats commercially available strategies and other reasonable baselines • Since PR converges and performs well, the equilibrium concept is sound in spite the fact that some theoretical assumptions are violated and there are plenty of players who are “irrational” • When bidding agents are used for real economic decisions (e.g., search engine optimization), we have an ideal playground for empirical game theory simulations Yahoo! Confidential

First Workshop on Sponsored Search Auctions

at ACM Electronic Commerce, 2005

Organizers:

Kursad Asdemir

, University of Alberta

Hemant Bharghava

, University of California Davis

Jane Feng

, University of Florida

Gary Flake

, Microsoft

David Pennock

, Yahoo! Research

Research

Papers

•

Mechanism Design

•

Pay-Per-Percentage of Impressions: An Advertising Method that is Highly Robust to Fraud,

J.Goodman

• •

Stochastic and Contingent-Payment Auctions,

C.Meek,D.M.Chickering, D.B.Wilson

•

Optimize-and-Dispatch Architecture for Expressive Ad Auctions,

D.Parkes, T.Sandholm

• •

Sponsored Search Auction Design via Machine Learning,

M.-F. Balcan, A.Blum, J.D.Hartline, Y.Mansour

• •

Knapsack Auctions,

G.Aggarwal, J.D. Hartline

Designing Share Structure in Auctions of Divisible Goods,

J.Chen, D.Liu, A.B.Whinston

Research

Papers

• •

Bidding Strategies

•

Strategic Bidder Behavior in Sponsored Search Auctions,

Benjamin Edelman, Michael Ostrovsky •

A Formal Analysis of Search Auctions Including Predictions on Click Fraud and Bidding Tactics,

B.Kitts, P.Laxminarayan, B.LeBlanc, R.Meech

User experience

•

Examining Searcher Perceptions of and Interactions with Sponsored Results,

B.J.Jansen, M. Resnick •

Online Advertisers' Bidding Strategies for Search, Experience, and Credence Goods: An Empirical Investigation,

A.Animesh, V. Ramachandran, • • S.Vaswanathan

Research

Stochastic Auctions

C.Meek,D.M.Chickering, D.B.Wilson

• • • • •

Ad ranking allocation rule is stochastic Why?

• • •

Reduces incentive for “bid jamming” Naturally incorporates explore/exploit mix Incentive for low value bidders to join/stay?

Derive truthful pricing rule Investigate contingent-payment auctions: Pay per click, pay per action, etc.

Investigate bid jamming, exploration strategies

Research

Expressive Ad Auctions

D.Parkes, T.Sandholm

• •

Propose expressive bidding semantics for ad auctions (examples next)

• •

Good: Incr. economic efficiency, incr. revenue Bad: Requires combinatorial optimization; Ads need to be displayed within milliseconds To address computational complexity, propose “optimize and dispatch” architecture: Offline scheduler “tunes” an online (real-time) dispatcher

Research

Expressive bidding I

•

Multi-attribute bidding Male users (50%) Un differentiated Advertiser 1 $1 Female users (50%) $2 $1.50

Advertiser 2 $2 $1 $1.50

Pre-qualified (50%) Other (50%) Advertiser 1 $2 $1 Advertiser 2 $2 $1 Un differentiated $1.50

$1.50

Research

Expressive bidding II

•

Competition constraints

b xCTR = RPS 3 x .05 = .15

1 x .05 = .05

Research

Expressive bidding II

•

Competition constraints

monopoly bid b xCTR = RPS 4 x .07 = .28

Research

Expressive bidding III

• • • • • • • • •

Guaranteed future delivery Decreasing/increasing marginal value All or nothing bids Pay per: impression, click, action, ...

Type/id of distribution site (content match) Complex search query properties Algo results properties (“piggyback bid”) Ad infinitum Keys: What advertisers want; what advertisers value differently; controlling cognitive burden; computational complexity

Source: K. Asdemir

Second Workshop on Sponsored Search Auctions

Organizing Committee Kursad Asdemir, University of Alberta Jason Hartline, Microsoft Research Brendan Kitts, Microsoft Chris Meek, Microsoft Research

Source: K. Asdemir

Objectives

 Diversity  Participants  Industry: Search engines and search engine marketers  Academia: Engineering, business, economics schools  Approaches    Mechanism Design Empirical Data mining / machine learning  New Ideas

History & Overview

 First Workshop on S.S.A.

    Vancouver, BC 2005 ~25 participants 10 papers + Open discussion 4 papers from Microsoft Research  Second Workshop on S.S.A.

   ~40-50 participants 10 papers + Panel 3 papers from Yahoo! Research Source: K. Asdemir

Source: K. Asdemir

Participants

 Industry  Yahoo!, Microsoft, Google  Iprospect (Isobar), Efficient Frontier, HP Labs, Bell Labs, CommerceNet  Academia  Several schools

Papers

  

Mechanism design

   Edelman, Ostrovsky, and Schwarz Iyengar and Kumar Liu, Chen, and Whinston  Borgs et al.

Bidding behavior

    Zhou and Lukose Szymanski and Lee Asdemir Borgs et al.

Data mining

 Regelson and Fain  Sebastian, Bartz, and Murthy Source: K. Asdemir

Source: K. Asdemir

Panel: Models of Sponsored Search: What are the Right Questions?

 Proposed by  Lance Fortnow and Rakesh Vohra  Panel members  Kamal Jain, Microsoft Research  Rakesh Vohra, Northwestern University  Michael Schwarz, Yahoo! Inc  David Pennock, Yahoo! Inc

Source: K. Asdemir

Panel Discussions

    Mechanisms    Competition between mechanisms Ambiguity vs Transparency: “Pricing” versus “auctions” Involving searchers Budget  Hard or a soft constraint  Flighting (How to spend the budget over time?) Pay-per-what? CPM, CPC, CPS   Risk sharing Fraud resistance Transcript available!

Research

Web resources

• • • • • •

1st Workshop website & papers:

http://research.yahoo.com/workshops/ssa2005/

1st Workshop notes (by Rohit Khare):

http://wiki.commerce.net/wiki/RK_SSA_WS_Notes

2nd Workshop website & papers:

http://www.bus.ualberta.ca/kasdemir/ssa2/

2nd Workshop panel transcript: (thanks Hartline & friends!)

http://research.microsoft.com/~hartline/papers/ panel-SSA-06.pdf

3rd Workshop website

http://opim-sun.wharton.upenn.edu/ssa3/index.html

4th Workshop website

http://research.yahoo.com/workshops/adauctions2008/

More Challenges

• Unifying search, display, content, offline • Economics of attention • Directly rewarding users, control, privacy 3-party game theoretic equilibrium • Predicting click through rates • • Detecting spam/fraud • Pay per “action” / conversion • Number/location/size of of ads • Improved targeting / expressiveness

$15B Question

: Monetizing social networks, user generated content

Prediction Markets

David Pennock

, Yahoo! Research

Research

Bet = Credible Opinion

Obama will win the 2008 US Presidential election “I bet $100 Obama will win at 1 to 2 odds” • •

Which is more believable?

More Informative?

Betting intermediaries

• • •

Las Vegas, Wall Street, Betfair, Intrade,...

Prices: stable consensus of a large number of quantitative, credible opinions Excellent empirical track record

Research

A Prediction Market

•

Take a random variable, e.g.

Bird Flu Outbreak US 2008?

(Y/N) •

Turn it into a financial instrument payoff = realized value of variable

I am entitled to: $1 if Bird Flu US ’08 $0 if Bird Flu US ’08

Research

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

http://intrade.com

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Prediction Markets: Examples & Research

Research

•

The Wisdom of Crowds

Backed in dollars

•

Where What you can say/learn % chance that

• • • • • • • •

Obama wins GOP wins Texas YHOO stock > 30 Duke wins tourney Oil prices fall Heat index rises Hurricane hits Florida Rains at place/time

• • • • • • • •

IEM, Intrade.com

Intrade.com

Stock options market Las Vegas, Betfair Futures market Weather derivatives Insurance company Weatherbill.com

Research

Prediction Markets

With Money

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Without

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW ) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Research

• • • • • • • • • • • • •

The Widsom of Crowds

Backed in “Points” HSX.com

Newsfutures.com

InklingMarkets.com

Foresight Exchange CasualObserver.net

FTPredict.com

Yahoo!/O’Reilly Tech Buzz ProTrade.com

StorageMarkets.com

TheSimExchange.com

TheWSX.com

Alexadex, Celebdaq, Cenimar, BetBubble, Betocracy, CrowdIQ, MediaMammon,Owise, PublicGyan, RIMDEX, Smarkets, Trendio, TwoCrowds http://www.chrisfmasse.com/3/3/markets/#Play-Money_Prediction_Markets

QuickTime™ and a TIFF (LZW) dec ompressor are needed to s ee this pic ture.

http://betfair.com

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Screen capture 2008/05/07

http://tradesports.com

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Screen capture 2007/05/18

Example: IEM 1992

Example: IEM

Example: IEM

[Thanks: Yiling Chen]

Does it work?

 Yes, evidence from real markets, laboratory experiments, and theory  Racetrack odds beat track experts [Figlewski 1979]  Orange Juice futures improve weather forecast [Roll 1984]  I.E.M. beat political polls 451/596 [Forsythe 1992, 1999][Oliven 1995][Rietz 1998][Berg 2001][Pennock 2002]  HP market beat sales forecast 6/8 [Plott 2000]  Sports betting markets provide accurate forecasts of game outcomes [Gandar 1998][Thaler 1988][Debnath EC’03][Schmidt 2002]  Laboratory experiments confirm information aggregation [Plott 1982;1988;1997][Forsythe 1990][Chen, EC’01]  Theory: “rational expectations” [Grossman 1981][Lucas 1972]  Market games work [Servan-Schreiber 2004][Pennock 2001]

Prediction Markets: Does Money Matter?

Research

The Wisdom of Crowds

With Money Without IEM: 237 Candidates HSX: 489 Movies

20 10 5 actual 100 50 2 1 1 2 5 10 20 50 100 estimate

Research

The Wisdom of Crowds

With Money Without

Research

Real markets vs. market games

HSX FX, F1P6 probabilistic forecasts forecast source

F1P6 linear scoring F1P6 F1-style scoring

betting odds F1P6 flat scoring F1P6 winner scoring avg log score

-1.84

-1.82

-1.86

-2.03

-2.32

Research

Does money matter? Play vs real, head to head

• •

Experiment 2003 NFL Season ProbabilitySports.com Online football forecasting competition

• • • • •

Contestants assess probabilities for each game Quadratic scoring rule ~2,000 “experts”, plus: NewsFutures (play $) Tradesports (real $)

• Used “last trade” prices • •

Results: Play money and real money performed similarly

•

6 th and 8 th respectively Markets beat most of the ~2,000 contestants

•

Average of experts came 39 th (caveat)

Electronic Markets

, Emile Servan Schreiber, Justin Wolfers, David Pennock and Brian Galebach

100 90 30 20 10 0 80 70 60 50 40 Prediction Accuracy

Research

100 Market Forecast Winning Probability and Actual Winning Probability TradeSports: Correlation=0.96

NewsFutures: Correlation=0.94

0 10 20 30 40 50 60 Trading Price Prior to Game 70 80 90 100 Data are grouped so that prices are rounded to the nearest ten percentage points; n=416 teams in 208 games 75 50 25 Prices: TradeSports and NewsFutures Fitted Value: Linear regression 45 degree line 0 0 20 40 60 NewsFutures Prices n=416 over 208 NFL games.

Correlation between TradeSports and NewsFutures prices = 0.97

80

Prediction Performance of Markets Relative to Individual Experts

0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Week into the NFL season

NewsFutures Tradesports 100

Research

Does money matter? Play vs real, head to head

Probability Football Avg TradeSports (real-money) NewsFutures (play-money) Difference TS - NF Mean Absolute Error

= lose_price

[lower is better]

Root Mean Squared Error

= ?Average( lose_price 2 )

[lower is better] 0.443 (0.012) 0.476 (0.025) 0.439 (0.011) 0.468 (0.023)

0.436

(0.012)

0.467

(0.024) 0.003 (0.016) 0.001 (0.033)

Average Quadratic Score

= 100 - 400*( lose_price 2 )

[higher is better]

Average Logarithmic Score

= Log(win_price)

[higher (less negative) is better] 9.323 (4.75) -0.649 (0.027) 12.410 (4.37)

-0.631

(0.024)

12.427

(4.57)

-0.631

(0.025) -0.017 (6.32) 0.000 (0.035) Statistically: TS ~ NF NF >> Avg TS > Avg

Research

A Problem w/ Virtual Currency Printing Money

Alice 1000 Betty 1000 Carol 1000

Research

A Problem w/ Virtual Currency Printing Money

Alice

5000

Betty 1000 Carol 1000

Research

Yootles A Social Currency

Alice 0 Betty 0 Carol 0

Research

Yootles A Social Currency

I owe you 5 Alice -5 Betty 0 Carol 5

Research

Yootles A Social Currency

I owe you 5 credit: 5 credit: 10 Alice -5 Betty 0 Carol 5

Research

Yootles A Social Currency

I owe you 5 I owe you 5 credit: 5 credit: 10 Alice -5 Betty 0 Carol 5

Research

Yootles A Social Currency

I owe you 5 I owe you 5 credit: 5 credit: 10 Alice 3995 Betty 0 Carol 5

Research

• •

Yootles A Social Currency

For tracking gratitude among friends A yootle says “thanks, I owe you one”

QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture.

Combinatorial Betting

March Madness

Research

•

Combinatorics Example March Madness

•

Typical today Non-combinatorial

• • • •

Team wins Rnd 1 Team wins Tourney A few other “props” Everything explicit (By def, small #)

•

Every bet indep: Ignores logical & probabilistic relationships Combinatorial

• •

Any property Team wins Rnd k Duke > {UNC,NCST} ACC wins 5 games

•

2 264 possible props (implicitly defined)

•

1 Bet effects related bets “correctly”; e.g., to enforce logical constraints

Expressiveness: Getting Information

• Things you can say today: – (43% chance that) Hillary wins – GOP wins Texas – YHOO stock > 30 Dec 2007 – Duke wins NCAA tourney • Things you can’t say (very well) today: – Oil down, DOW up, & Hillary wins – Hillary wins election, given that she wins OH & FL – YHOO btw 25.8 & 32.5 Dec 2007 – #1 seeds in NCAA tourney win more than #2 seeds

Expressiveness: Processing Information

• Independent markets today: – Horse race win, place, & show pools – Stock options at different strike prices – Every game/proposition in NCAA tourney – Almost everything: Stocks, wagers, intrade, ...

• Information flow (inference) left up to traders • Better: Let traders focus on predicting whatever they want, however they want: Mechanism takes care of logical/probabilistic inference • Another advantage: Smarter budgeting

Research

[Thanks: Yiling Chen]

Automated Market Makers

• • •

A market maker (a.k.a. bookmaker) is a firm or person who is almost always willing to accept both buy and sell orders at some prices

• • •

Why an institutional market maker? Liquidity!

Without market makers, the more expressive the betting mechanism is the less liquid the market is (few exact matches) Illiquidity discourages trading: Chicken and egg Subsidizes information gathering and aggregation: Circumvents no-trade theorems

• •

Market makers, unlike auctioneers, bear risk. Thus, we desire mechanisms that can bound the loss of market makers Market scoring rules [Hanson 2002, 2003, 2006] Dynamic pari-mutuel market [Pennock 2004]

Overview: Complexity Results

Permutations Boolean Call Market General NP-hard Pair NP-hard Subset General 2-clause Restrict Tourney Poly co-NP complete ?

?

Market Maker (LMSR) #P-hard #P-hard #P-hard #P-hard #P-hard Poly

Research

New Prediction Game

Research

Mech Design for Prediction

Primary Secondary Financial Markets Social welfare (trade) Hedging risk Information aggregation Prediction Markets Information aggregation Social welfare (trade) Hedging risk

Research

Mech Design for Prediction

• •

Standard Properties

• • • • • •

Efficiency Inidiv. rationality Budget balance Revenue Truthful (IC) Comp. complexity Equilibrium

•

General, Nash, ...

• •

PM Properties

• • • • • • •

#1: Info aggregation Expressiveness Liquidity Bounded budget Truthful (IC) Indiv. rationality Comp. complexity Equilibrium

•

Rational expectations

Competes with: experts, scoring rules, opinion pools, ML/stats, polls, Delphi

Research

Discussion

• •

Are incentives for virtual currency strong enough?

• •

Yes (to a degree) Conjecture: Enough to get what people already know; not enough to motivate independent research

•

Reduced incentive for information discovery possibly balanced by better interpersonal weighting Statistical validations show HSX, FX, NF are reliable sources for forecasts

• •

HSX predictions >= expert predictions Combining sources can help

Research

Catalysts

• • • • •

Markets have long history of predictive accuracy: why catching on now as tool?

No press is bad press: Policy Analysis Market (“terror futures”) Surowiecki's “Wisdom of Crowds” Companies:

•

Google, Microsoft, Yahoo!; CrowdIQ, HSX, InklingMarkets, NewsFutures Press: BusinessWeek, CBS News, Economist, NYTimes, Time, WSJ, ...

http://us.newsfutures.com/home/articles.html

CFTC Role

• MayDay 2008: CFTC asks for help • Q: What to do with prediction markets?

• Right now, the biggest prediction markets are overseas, academic (1), or just for fun • CFTC may clarify, drive innovation • Or not

Research

Conclusion

• •

Prediction Markets: hammer = market, nail = prediction

• •

Great empirical successes Momentum in academia and industry

•

Fascinating (algorithmic) mechanism design questions, including combinatorial betting Points-paid peers produce prettygood predictions