Transcript Document

Outline for Today
• Brief Lecture on weather forecasting
• A comment on the upcoming Exam 4 and a bunch of
other stuff
Theoretical Models and Accuracy
• As we discussed previously, one of the most useful
ways of evaluating a theoretical model is to compare
the predictions of the model against empirical data.
• Unfortunately, many industries do not provide you
with information on the accuracy of their predictions.
– how many people improve after taking a drug
– how many people actually lose weight by eating Subway
sandwiches
– how many horoscopes accurately predict the future
• As a consequence, it is almost impossible for you, as
a consumer, to evaluate the quality of the product.
• In general, weather forecasters make their
predictions based on either mathematical
models/simulations of weather systems, past weather
conditions, intuitive judgments based on the
movement of current weather systems, or “all of the
above”
• Unfortunately, weather forecasters do not tell us—the
consumers—how accurate their forecasts are. We
don’t know what the “track record” is for any
weather/news team. (Moreover, they don’t keep
records of past forecasts on their websites, thereby
preventing consumers from investigating the matter
themselves.)
A scientific analysis of the accuracy of weather
forecasting
• Students recorded the 7-day forecasts (on Sunday
nights) for 1 week for a variety of sources
–
–
Weather.com
Fox News
• We compared the actual, end-of-the-day weather
from an unrelated source: Weather Underground.
• This allows us to compare the predictions made by
the different teams with the actual weather.
80
60
20
40
Actual high temperature
60
40
20
Actual low temperature
80
Actual temperatures observed, as recorded by Weather
Underground, Champaign
1
2
3
4
5
6
7
1
Days
2
3
4
Days
Nov 20 – Nov 26
5
6
7
Predictions made by weather.com
Date
Predicted High
Observed High
Discrepancy (P-O)
11/20
55
45
10
11/21
62
63
-1
11/22
68
70
-2
11/23
52
41
11
11/24
52
40
12
11/25
32
41
-9
11/26
33
32
1
Average absolute discrepancy: 6.6 degrees
~ not too bad ~
WGNTV Chicago | Mean
Discrepancy = 4.3
30
40
50
60
Predictied High
70
80
80
60
50
30
20
20
20
40
Actual High
50
30
40
Actual High
60
70
70
80
80
70
60
50
40
30
20
Actual High
ABC 15 | Mean Discrepancy =
1.3
WGN | Mean Discrepancy = 4.8
20
30
40
50
60
Predictied High
70
80
20
30
40
50
60
Predictied High
70
80
As an additional comparison, we
can generate predictions in
another way: based on “day
before” temperature
Date
11/20
Observed
11/21
63
45
11/22
70
63
In this case, we’re assuming
that the best bet for tomorrow’s
temperature is today’s
temperature.
11/23
41
70
11/24
40
41
11/25
41
40
11/26
32
41
Average discrepancy: 10.8
Predicted
45
32
What about broader weather conditions?
• Broader weather conditions
–
–
–
–
–
Sunny
Partly cloudy
Cloudy
Rain
Snow
• Accuracy in broad prediction (vs. day before method)
–
–
–
–
Weather.com
WGN-Chicago
WGN-Chicago
ABC15-Phoenix
42% (16%)
75% (57%)
57% (0%)
57% (17%)
Summary
• It appears that the various weather teams do a pretty
good job in predicting the temperature
– They were only off by approximately 4 – 7 degrees, on average.
– Their ability to make accurate forecasts was lower for extended
forecasts.
• In predicting weather conditions (e.g., rain, shine),
the various stations were right 40% – 80% of the
time. This was better than the accuracy obtained by
predicting today’s weather from yesterday’s.
• Nonetheless, you wouldn’t know how good they are
without doing the research yourself.