Transcript Gridded OCF Probabilistic Forecasting For Australia Shaun Cooper and Timothy Hume Centre for Australian Weather and Climate Research Probabilistic Forecasts: “Vote Counting” Gridded OCF:

Gridded OCF Probabilistic Forecasting For Australia
Shaun Cooper and Timothy Hume
Centre for Australian Weather and Climate Research
Probabilistic Forecasts: “Vote Counting”
Gridded OCF: Introduction
Gridded Operational Consensus Forecasting (Gridded OCF) is an operational forecast guidance system used by
the Australian Bureau of Meteorology. It consists of a Poor Man's Ensemble (PME) of Numerical Weather
Prediction (NWP) output from a number of international centres. The PME members are bias corrected with
respect to a gridded mesoscale analysis (MSAS) which covers the Australian domain (but precipitation forecasts
are not currently bias corrected). A range of products are then generated from the PME members, including:
This method is easily applied to a range of
variables. For example:
 Precipitation exceeding defined thresholds
 Wind speed exceeding critical speeds
 Temperature (cold events, heat waves and so on)
The “vote counting” method counts the number of PME
members which satisfy a certain condition (e.g. rainfall
exceeding a specified threshold), and express the
probability as a proportion of the total number of
members. These probabilities will not generally be
calibrated.
Weighted average consensus forecasts (used as deterministic guidance by weather forecasters)
 Probabilistic forecasts of particular parameters exceeding specified thresholds

Examples
This poster describes the probabilistic products produced by the Gridded OCF system.
Probability of daily precipitation exceeding 1 mm computed using the
“vote counting” method. 24 hour forecast valid at 2011-0829T00:00:00Z. Colours indicate percentages ranging from 0% (dark
purple) to 100% (light pink).
Probability of the wind speed exceeding 34 knots (gale force winds).
24 hour forecast valid at 2011-08-02T00:00:00Z. Colours indicate
percentages ranging from 0% (dark purple) to 100% (light pink)
Diagram illustrating the method used to produce probabilistic forecasts from the Gridded OCF system.
Calibration of Rainfall Forecasts
Probabilistic forecasts generated using the “vote counting” method
(see above) are not usually calibrated. That is, the observed frequency
of an event turns out to be different than the forecast probability of the
event occurring.
Probabilistic Forecasts: QPF Method
This method assigns probabilities of rainfall
exceeding a threshold based on a quantitative
precipitation forecast (QPF). The higher the QPF,
the higher the probability assigned to exceeding a
defined threshold. The method is based on work
by Sloughter et al. (2007) which employs
Bayesian Model Averaging. It can also be applied
to a single deterministic rainfall forecast. In our
work, we apply it to the PME average forecast;
that is, we average the QPFs from each ensemble
member and use the resulting average QPF to
derive the probability that the daily precipitation
will exceed defined thresholds.
To calibrate precipitation forecasts, the preceding year of uncalibrated
forecasts covering the Australian states of Victoria and New South
Wales are compared with daily rainfall analyses valid at the same
time. A look-up table is constructed to convert uncalibrated
probabilities to calibrated probabilities. The calibration tables are
continuously updated using the most recent year of data.
Probability of rainfall exceeding 0.2 mm computed using the QPF
method. 24 hour forecast valid at 2011-08-31T00:00Z. Colours
indicate percentages ranging from 0% (dark purple) to 100% (light
pink).
Probabilistic forecasts generated using the QPF
method are calibrated
 A continuous range of probabilities are generated,
whereas the vote counting method produces a
discrete number of possible probabilities based on
the number of ensemble members

Example of a daily rainfall analysis (constructed from rain gauge observations) for 2011-08-18.
Analyses such as these are used for calibrating probabilities derived using the “vote counting”
method (see panel above), and also in the QPF method (see panel to the right). Colours indicate
the 24 hour accumulation from 0 mm (grey) to 80 mm (pink-white). No data available over the sea.
Comparison of un-calibrated (top panel) with calibrated
(bottom panel) forecasts of the probability of daily
precipitation exceeding 0.2 mm (made using the “vote
counting method”). 24 hour forecast valid at 2011-0831T00:00Z. Colours indicate percentages ranging from
0% (dark purple) to 100% (light pink).
Example: The figure shows the relationship between the PME's
average QPF and the probability of precipitation exceeding 0.2 mm
over the Australian states of Victoria and New South Wales. The curve
was derived by comparing one year of historical QPF forecasts with the
corresponding daily rainfall analyses (taken as the “truth”) for the year.
Verification
Conclusions
Forecasts of the probability of
precipitation exceeding 0.2mm
over the Australian states of
Victoria and New South Wales
were prepared for the period 1
September 2009 through 29 May
2011 inclusive using both the
calibrated “vote counting” and
QPF methods.
Reliability (top) and ROC (bottom) diagrams for
24 hour forecasts of probability of precipitation
exceeding 0.2 mm (vote counting method)
The probabilistic forecasts were
verified against the operational
daily rainfall analyses (see earlier
panel), and reliability and relative
operating characteristics (ROC)
curves created.
It is clear from the diagrams (vote
counting results on the left, QPF
method on the right) that both
methods
produce
reasonably
reliable forecasts. The area
beneath the ROC curve is
approximately the same for both
forecast methods, indicating they
have comparable skill.
• Two methods of generating probabilistic forecasts from a “Poor Man’s Ensemble” were investigated
•
•
•
•
Reliability (top) and ROC (bottom) diagrams for
24 hour forecasts of probability of precipitation
exceeding 0.2 mm (QPF method)
•
•
The “vote counting” method is simple, and can be easily applied to a variety of meteorological
parameters such as rainfall, wind speed and temperature.
A weakness of the “vote counting” method is that the probability forecasts are “quantised”. For
example, if the PME has 4 members, only probabilities of 0, 25, 50, 75 and 100% are possible.
The “vote counting” method does not usually produce calibrated probability forecasts. An extra
calibration phase is required.
The QPF method generates a continuous range of probabilities, even if there is only a single member
in the PME.
Forecasts made using the QPF method are calibrated.
However, there is not much difference in forecast skill (as shown by the ROC and reliability diagrams)
between the QPF and “vote counting” methods.
Future Work
• Apply the probabilistic forecasting methods to other types of weather, such as the probability of extreme
temperature events.
• Improved calibration techniques.
Acknowledgements
Thanks to Philip Riley for advice and ideas for
probabilistic rainfall forecasting. The original PME
rainfall forecasts in the Bureau of Meteorology were
developed by Elizabeth Ebert and Chermelle Engel.
For more information please contact [email protected]
References
Sloughter, J. M., Raftery, A., Gneiting, T., and Fraley, C. (2007).
Probabilitistic quantitative precipitation forecasting using Bayesian model
averaging. Monthly Weather Review, 135:3209–3320.
© Commonwealth of Australia 2011