Transcript National Weather Service Verification Program Overview

National Weather Service
Verification Program Overview
Brenton MacAloney II
National Weather Service Headquarters
Silver Spring, MD
July 19th, 2006
Summer WAS*IS 2006
Verification Programs
The National Weather Service (NWS) Performance Branch
collects warning and forecast products, as well as
observations to generate verification scores.
These verification
scores are displayed
on the Performance
Management website
accessible only to
NWS employees.
Summer WAS*IS 2006
Verification Programs
Verification data is available via the Stats on Demand program.
Summer WAS*IS 2006
Verification Programs
Verification statistics are calculated on two types of products.
Forecasts
Summer WAS*IS 2006
Warnings
Forecast Verification
Aviation
– Terminal Aerodrome Forecast (TAF)
• Ceiling, Visibility, Significant Weather, Wind Speed,
Direction, and Gust, and Flight Category
Hydrology
– River Stage Forecast
– Quantitative Precipitation Forecasts (QPF)
Summer WAS*IS 2006
Forecast Verification
Marine
– Coastal and Offshore Forecast
• Wave Height, Wind Speed, Wind Direction
Fire Weather
– National Fire Danger Rating System Forecasts (NFDRS)
• State of the Weather, Temperature, Humidity, Wind Speed
Public
– Point Forecast Matrices (PFM) Forecasts
• Max/Min Temperature, Probability of Precipitation (PoP)
Summer WAS*IS 2006
Warning Verification
Short Duration Warnings
•
•
•
•
Tornado (TOR)
Severe Thunderstorm (SVR)
Flash Flood (FFW)
Special Marine (SMW)
Long Duration Warnings*
• Winter Storm (WSW)
• High Wind (NPW)
• Coastal Flood (CFW)
* Verification of long duration warnings is currently done by hand. Starting in
October 2006, the verification of long duration warnings will be automated.
Summer WAS*IS 2006
Warning Verification
As far back as 1986, warnings have been verified using
counties warned in the Universal Geographic Code (UGC)
line of a warning product.
Verifiable events are collected, entered into storm data at the
forecast offices (WFO), and sent to NWS Headquarters monthly.
Summer WAS*IS 2006
Warning Verification
Any event occurring during the valid time of warned county,
verifies the warning. The event is also considered “warned”.
Summer WAS*IS 2006
Warning Verification
The NWS is changing how they issue short duration
warnings. In the next few years, warnings will be issued
where the “polygon” is the warning area, not the county.
This “polygon” information
is currently available on all
SVR, TOR, FFW, and
SMW products.
Summer WAS*IS 2006
Brent’s Dream World
• Due to a limited budget, the majority of the money the
Performance Branch receives is spent on the
development of new verification programs.
• Having verification information is great, but it does not
tell the whole story.
• I would love to see other areas of “Performance
Evaluation” explored.
Summer WAS*IS 2006
Questions and Issues
• Performance evaluation questions I would like to
find answers to:
– What is an acceptable False Alarm Ratio to
the public? to Emergency Managers? to
businesses? to others who depend on our
warnings?
Summer WAS*IS 2006
Questions and Issues
• Other questions:
– Are there any new or existing metrics that we
are not using and might be useful in showing
a better picture of our performance?
Summer WAS*IS 2006
Questions and Issues
• Other questions:
– How can we get the forecasters to better
understand the current metrics so that they
are able to improve their forecasts and
warnings?
Summer WAS*IS 2006
Questions and Issues
• Other questions:
– Are there any ways for the WFOs to easily do
their own service assessments? Does
anyone know of any software that might help?
What types of relationships can be set up
within the local community that can help
channel feedback to the WFO on their
performance?
Summer WAS*IS 2006
Contact Information
Brenton MacAloney II
[email protected]
Phone: 301-713-0090 x135
Summer WAS*IS 2006