Transcript 2nd International Conference on Fog and Fog Collection

PROGRESS ON A KNOWLEDGE-BASED INTERNET
WEATHER FORECASTING SYSTEM
100-day evaluation trial from
25 September, 2002 to 8 January, 2003.
Harvey Stern
(Bureau of Meteorology, Australia)
http://www.weather-climate.com/internetforecasts.html
Location Diagram (1)
Location Diagram (2)
Example (NWP MSL Prognosis)
Example (The System’s Output)
Background
An early version of the knowledge-based system was
presented to the 18th IIPS Conference (Stern, 2002).
The system was developed for the small (227,000 sq km)
southeast Australian State of Victoria.
It was described as being capable of generating forecasts for
public, aviation, marine and media interests, in languages
other than English, and for more than 200 localities in
Victoria.
The large volume of output proposed here (forecasts for than
200 localities) would be difficult (if not impossible) to
generate manually
However, it would be straightforward (to generate) utilising
automated systems.
“Bank” of forecaster experience
A major benefit of the knowledge-based system is that it
incorporates an extensive "bank" of forecaster experience.
Ramage (1993) has proposed an "iterative" approach to
"locking in" improvements in forecasting methodology.
The automated nature of the system lends itself to Ramage's
approach.
The system's skill increases as new knowledge is
incorporated into its operation.
Hence, progress is gradually made towards the realisation of
Ramage's dream.
Not “yet another” instrument of forecast guidance
The system is not seen as "yet another" instrument of
forecast guidance.
Rather, its development is seen as a logical step along the
path of having the computer replicate the best aspects of the
manual side of the forecast process, by systematically
"locking in" new knowledge.
As Brooks (1995), wrote: "technology, which initially allowed
humans to make routine weather forecasts, will soon close
that avenue of human endeavour ... (and thereby permit)
concentration on severe events".
The main human-interaction is in utilising forecast
verification data (after the event) to iteratively incorporate
new additional forecaster knowledge into its algorithm.
A Forecasting Research Tool
A simple illustration is now presented of how the system for
generating Internet Forecasts might be employed as a
research tool to enhance understanding.
The first Figure following depicts the MSL pressure (MSLP)
distribution for Synoptic Type 41 (strong cyclonic SSW flow).
A Forecasting Research Tool (cont.)
The Figure plots the annual march of PoP forecasts for
strong cyclonic SSW flow, with 850 hPa temperatures of 0
deg C, & with 700 hPa RHs of 30% & 70%.
One might ask why there is such a marked difference
between winter & summer responses to the same situation.
A Forecasting Research Tool (cont.)
An illustration is now presented of how one may proceed
from the enhanced understanding achieved via the preceding
analysis, to increase the potential accuracy of the forecasts
generated.
A study of Synoptic Type 41 cases suggests that the strength
of the SSW flow between SE Australia and over waters to the
west of that region might be significant.
Regression analysis confirms that adding cyclonicity to the
other predictors, cyclonicity proves to be the most
significant.
A Forecasting Research Tool (cont.)
The Figure plots the annual march of PoP forecasts for
strong cyclonic SSW flow with RH=30% and the new
equation.
It reveals just how important cyclonicity is in assessing the
likelihood of precipitation.
November 2001 trial
The system's forecast performance (during November 2001)
was evaluated for the city of Melbourne.
The evaluation showed that, although superiority over
climatology was achieved, the forecasts (on most measures)
proved to be inferior to the official forecasts.
The results of the November 2001 trial were analysed,
potential improvements in the forecasting process (employed
by the system) were identified, and those improvements were
"locked in".
The system (so modified) was then subjected to another
evaluation for the city of Melbourne, this time over 100 days
(between late September 2002 and early January 2003).
Overall Accuracy (in 2001)
Modifications
The 2001 system operated by producing its predictions from
a restricted number of discrete "forecast sets".
The set that was chosen (by the system) was largely
determined by the particular synoptic pattern suggested by
the selected NWP model.
The 2002/2003 modification utilises regression analysis to
allow predictions to be selected from a continuous array of
possible forecasts.
The particular form of regression analysis employed is
parameter enveloping.
Parameter enveloping allows definition of how the various
predictors impact upon, or envelope, the influence (on a
predictand) of other predictors.
An illustration of parameter enveloping
Overall Accuracy (2002/2003)
RMS Error of Min Temp Forecasts
RMS Error of Max Temp Forecasts
% Correct Rain/No Rain
RMS Error of QPFs
Need for Further Testing
The 100-day verification trial suggests that substantial
progress, albeit uneven, towards achieving computer
replication of the manual forecast process, has been made.
Specifically, the impact of the "locked in" improvements upon
the skill displayed by the system has been considerable,
particularly for Day-1 forecasts.
Nevertheless, the performance of the system over the 100day trial is so impressive, that one must entertain the
possibility that it was simply a "fluke".
For this reason, further testing needs to be undertaken.
Implications
It was considered that, should further testing confirm the 100day trial results, the implications would be profound.
We could see ourselves at the "dawn" of the operational
implementation of Ramage's approach to weather forecasting
which, through computer replication of the manual process,
would allow for:
•Systematic incorporation of new procedures that may lead to
a quantum leap in the accuracy of the forecast products (in a
related piece of work, Ryan et al. (2003) have developed a
system that archives the subjective inputs to the forecast
process and makes them available for statistical analysis);
•The opportunity to greatly increase the number and variety
of such products.
Further Testing
So far, further testing is, indeed, confirming the 100-day trial
results.
A modest post-trial exercise, involving the derivation of
forecasts for Day-1 only for the (now) 130 days ended
February 1, 2003, yields:
•A CSS of 61.4 for the Internet Forecasts; and,
•A CSS of 60.3 for the official forecasts.
http://www.weather-climate.com/internetforecasts.html