DCPs in Forecasting - Edward L Kambour III
Download
Report
Transcript DCPs in Forecasting - Edward L Kambour III
An Intelligence Approach
to Evaluation of Sports
Teams
by
Edward Kambour, Ph.D.
1
Agenda
I.
II.
III.
IV.
V.
VI.
VII.
College Football
Linear Model
Generalized Linear Model
Intelligence (Bayesian) Approach
Results
Other Sports
Future Work
General Background
Goals
Forecast winners of future games
Beat the Bookie!
Estimate the outcome of unscheduled games
What’s the probability that Iowa would have beaten
Ohio St?
Generate reasonable rankings
Major College Football
No playoff system
“Computer rankings” are an element of the
BCS
114 teams
12 games for each in a season
Linear Model
Rothman (1970’s), Harville (1977), Stefani
(1977), …, Kambour (1991), …, Sagarin???
Response, Y, is the net result (point-spread)
Parameter, , is the vector of ratings
For a game involving teams i and j,
E[Y] = i - j
Linear Model (cont.)
Let X be a row vector with
if k i
1
X k 1 if k j
0 otherwise
E[Y]=X
Regression Model Notes
Least Squares Normality, Homogeneity
College Football
Estimate 100 parameters
Sample size for a full season is about 600
Design Matrix is sparse and not full rank
Home-field Advantage
Generic Advantage (Stefani, 1980)
Force i to be home team and j the visiting team
Add an intercept term to X
Adds one more parameter to estimate
UAB = Alabama
Rice = Texas A&M
Team Specific Advantage
Doubles the number of parameters to estimate
Linear Model Issues
Normality
Homogeneity
Lots of parameters, with relatively small
sample size
Overfitting
The bookie takes you to the cleaners!
Linear Model Issues (cont.)
Should we model point differential
A and B play twice
A by 34 in first, B by 14 in the second
A by 10 each time
Running up the score (or lack thereof)
BCS: Thou shalt not use margin of victory in thy
ratings!
Logistic Regression
Rothman (1970s)
Linear Model
Use binary variable
Winning is all that matters
Avoid margin of victory
Coin Flips
Logistic Regression Issues
Still have sample size issues
Throw away a lot of information
Undefeated teams
Transformations
Transform the differentials to normality
Power transformations
Rothman logistic transform
Transforms points to probabilities for logistic
regression
“Diminishing returns” transforms
Downweights runaway scores
Power Transforms
Transform the point-spread
Y = sign(Z)|Z|a
a = 1 straight margin of victory
a = 0 just win baby
a = 0 Poisson or Gamma “ish”
Maximum Likelihood Transform
1995-2002 seasons
Power
-2ln(likelihood)
0.1
52487
0.3
41213
0.5
35128
0.67
32597
0.8
31418
1
31193
MLE = 0.98
Predicting the Score
Model point differential
Y1 = Si – Sj
Additionally model the sum of the points
scored
Y2 = Si + Sj
Fit a similar linear model (different parameter
estimates)
Forecast home and visitors score
H = (Y1 + Y2 )/2, V = (Y2 - Y1)/2
Another Transformation Idea
Scores (touchdowns or field goals) are
arrivals, maybe Poisson
Final score = 7 times a Poisson + 3 times a
Poisson + …
Transform the scores to homogeneity and
normality first
The differences (and sums) should follow suit
Square Root Transform
Since the score is “similar” to a linear
combination of Poissons, square root should
work
Transformation
T S k
Why k?
For small Poisson arrival rates, get better
performance (Anscombe, 1948)
Likelihood Test
LRT: No transformation vs. square root with
fitted k
Used College Football results from 1995-2002
k = 21
Transformation was significantly better
p-value = 0.0023, chi-square = 9.26
Predicting the Score with
Transform
Model point differential
Y1 Si 21 S j 21
Additionally model the sum of the points
scored
Y2 Si 21 S j 21
Forecast home and visitors score
H = ((Y1 + Y2 )/2)2 , V = ((Y2 - Y1)/2)2
Note the point differential is the product
Unresolved Linear Model Issues
Overfitting
History
Going into the season, we have a good idea as
to how teams will do
The best teams tend to stay the best
The worst teams tend to stay the worst
Changes happen
Kansas State
Intelligence Model
Concept
The ratings and home-ads for year t are similar
to those of year t-1. There is some drift from
one year to the next.
Model
t t 1 t
where
t ~ N(0, 2 )
Intelligence Model (Details)
Notation
L teams
M seasons of data
Ni games in the ith season
Xi : the Ni by 2L “X” matrix for season i
Yi : the Ni vector of results for season i
i : the Ni vector of results for season I
Details (cont.)
Data Distribution:
For all i = 1, 2, …, M
Yi N Xi i , 2
(independent)
Details (cont.)
Prior Distribution
1
2
I
0 2
N 0,
0 0.05I
0 2
0.25I
i
N i 1 ,
for i 2,..., M
0.01I
0
2 2,0.5
2
Details (finally, the end)
The Posterior Distribution of M and -2 is
closed form and can be calculated by an
iterative method
The Predictive Distribution for future results
(transformed sum or difference) is straightforward correlated normal (given the
variance)
Forecasts
For Scores
Simply untransform
E[Z2] = Var[Z] + E[Z]2
For the point-spread
Product of two normals
Simulate 10000 results
Enhanced Model
Fit the prior parameters
Hierarchical models
Drifts and initial variances
No closed form for posterior and predictive
distributions (at least as far as I know)
The complete conditionals are straight-forward, so
Gibbs sampling will work (eventually)
Results
(www.geocities.com/kambour/football.html)
2002 Final Rankings
Team
Rating
Home
Miami
72.23 (1.03)
0.21 (0.04)
Kansas St
72.04 (1.04)
0.44 (0.03)
USC
71.95 (1.03)
0.04 (0.03)
Oklahoma
71.85 (1.02)
0.18 (0.03)
Texas
71.57 (1.03)
0.36 (0.03)
Georgia
71.49 (1.03)
0.02 (0.03)
Alabama
71.45 (1.03)
-0.09 (0.03)
Iowa
71.30 (1.03)
0.21 (0.04)
Florida St
71.29 (1.02)
0.43 (0.03)
Virginia Tech
71.25 (1.03)
0.12 (0.03)
Ohio St
71.18 (1.03)
0.27 (0.03)
Results
2002 Final Rankings
Team
Rating
Home
Miami
72.23
0.21
Kansas St
72.04
0.44
USC
71.95
0.04
Oklahoma
71.85
0.18
Texas
71.57
0.36
Georgia
71.49
0.02
Alabama
71.45
-0.09
Iowa
71.30
0.21
Florida St
71.29
0.43
Virginia Tech
71.25
0.12
Ohio St
71.18
0.27
Results
2002 Final Rankings
Team
Rating
Home
Miami
72.23
0.21
Kansas St
72.04
0.44
USC
71.95
0.04
Oklahoma
71.85
0.18
Texas
71.57
0.36
Georgia
71.49
0.02
Alabama
71.45
-0.09
Iowa
71.30
0.21
Florida St
71.29
0.43
Virginia Tech
71.25
0.12
Ohio St
71.18
0.27
Bowl Predictions
Ohio St
Miami Fl (-13)
17
31
0.8255
0.5228
Washington St
Oklahoma (-6.5)
21
31
0.7347
0.5797
Iowa
USC (-6)
21
30
0.7174
0.5721
NC State (E)
Notre Dame
20
17
0.5639
0.5639
Florida St (+4)
Georgia
24
27
0.5719
0.5320
2002 Final Record
Picking Winners
522 – 157
0.769
Against the Vegas lines
367 – 307 – 5
0.544
Best Bets
9 – 7
In 2001, 11 - 4
0.563
ESPN College Pick’em
(http://games.espn.go.com/cpickem/leader)
1. Barry Schultz
2. Jim Dobbs
3. Michael Reeves
4. Fup Biz
5. Joe *
6. Rising Cream
7. Intelligence Ratings
5830
5687
5651
5594
5587
5562
5559
Ratings System Comparison
(http://tbeck.freeshell.org/fb/awards2002.html)
Todd Beck
Ph.D. Statistician
Rush Institute
Intelligence Ratings – Best Predictors
College Football Conclusions
Can forecast the outcome of games
Capture the random nature
High variability
Sparse design
Scientists should avoid BCS
Statistical significance is impossible
Problem Complexity
Other issues
NFL
Similar to College Football
Square root transform is applicable
Drift is a little higher than College Football
Better design matrix
Small sample size
Playoff
NFL Results
(www.geocities.com/kambour/NFL.html)
2002 Final Rankings (after the Super Bowl)
Team
Rating
Home
Tampa Bay
70.72
0.29
Oakland
70.57
0.28
Philadelphia
70.55
0.10
New England
70.16
0.12
Atlanta
70.13
0.20
NY Jets
70.10
-0.01
Pittsburgh
69.95
0.28
Green Bay
69.92
0.28
Kansas City
69.90
0.51
Denver
69.89
0.50
Miami
69.89
0.49
2002 Final NFL Record
Picking Winners
162 – 104 – 1
0.609
Against the Vegas lines
135 – 128 – 4
0.513
Best Bets
9 – 8
0.529
NFL Europe
Similar to College and NFL
Square root transform
Dramatic drift
Teams change dramatically in mid-season
Few teams
Better design matrix
College Basketball
Transform?
Much more normal (Central Limit Theorem)
A lot more games
Intersectional games
Less emphasis on programs than in College
Football
More drift
NCAA tournament
NCAA Basketball
Pre-tournament Ratings
Team
Rating
Home
Arizona
100.06
3.97
Kentucky
99.33
4.32
Kansas
95.89
3.85
Texas
93.42
4.44
Duke
92.90
4.66
Oklahoma
90.19
4.31
Florida
90.65
3.99
Wake Forest
88.70
3.65
Syracuse
88.50
3.49
Xavier
87.89
3.37
Louisville
87.88
4.16
NBA
Similar to College Basketball
Normal – No transformation
A lot more games – fewer teams
Playoffs are completely different from
regular season
Regular season – very balanced, strong home
court
Post season – less balanced, home court
lessened
Hockey
Transform
Rare events = “Poissonish”
Square root with k around 1
A lot more games
History matters
Playoffs seem similar to regular season
Balance
Soccer
Similar to hockey
Transform
Square root with low k
Not a lot of games
Friendlys versus cup play
Home pitch is pronounced
Varies widely
Soccer Results
Correctly forecasted 2002 World Cup final
Brazil over Germany
Correctly forecasted US run to quarter-finals
Won the PROS World Cup Soccer Pool
Future Enhancements
Hierarchical Approaches
Conferences
More complicated drift models
Correlations
Individual drifts
Drift during the season
Mean correcting drift
More informative priors