Transcript Developing of Evaluation Metrics and Indices for Applications Galia Guentchev and the NCPP Core and Tech team.

Developing of Evaluation Metrics
and Indices for Applications
Galia Guentchev and the NCPP Core and
Tech team
Community and Collaboration
NCPP mission - to accelerate the provision of climate information
on regional and local scale for use in adaptation planning and
decision making through collaborative participation of a
community of scientists and practitioners.
Applications related groups
Applications Related Metrics and Indices
Major focus - the development of a capability for objective and
quantitative evaluation of downscaled climate information in
support of applications.
Example of a Health Impacts Case Study
– Focus: Historical changes in the temperature related
triggers used in Heat Health Warning Systems (HHWS)
– May – September 1971-2000
– Variables – tmax, tmin
• The temperature/mortality relationships, when used as
trigger setting approaches for HHWS, have been found to
most closely identify the most dangerous days in terms of
excess mortality - Hajat et al 2010
– Indices:
• 92.5th, 95th and 97.5 percentile exceedances of Tmax and
Tmin with a minimum duration of 2 days (based on relative
threshold exceedances for each summer day with a 5 day
window);
– Parameters – frequency, timing of occurrence,
duration
– Areas of interest – 3 metropolitan areas – Boston,
Indianapolis, Washington DC; N Carolina counties
Information that can be obtained from
the Advanced Search capability
• Comparisons of downscaled data vs
Maurer02v2
– Average summer maximum and minimum
temperature comparisons
– 95th percentile of maximum or minimum
temperature
– Annual number of tropical nights (tmin>20degC)
• Average, median, maximum, minimum
Average summer minimum temperature
Minimal differences in the order of -0.5 to 0.5 degrees C regardless of
downscaling method or model
(some variation in spatial pattern for the ARRM MIROC MEDRES data).
95th percentile of summer minimum temperature
Average annual number of tropical nights
Large differences between the methods and the
models used.
Maximum annual number of tropical nights
Even larger divergence between the method and
model used.
Information that can be obtained from
the Open Climate GIS capability
• Observed and downscaled minimum and
maximum temperature
– To be used to calculate exceedances above a
threshold using dynamical percentiles or a simple
absolute threshold
– For area of interest – the 3 metropolitan areas and
the North Carolina counties
• Pre-calculated indices results from the NCPP
evaluation
– To be used to extract evaluation results for area of
interest
Metrics calculated on Tmax or Tmin for
summer months:
•
•
•
•
•
•
•
•
Mean
Median
Maximum
Minimum
Standard Deviation
Absolute threshold exceedances
Number of values between thresholds
Consecutive occurrences above/below an absolute
threshold
• Frequency of spells
• Exceedances using dynamical percentiles for each
day – 90, 92.5, 95, 97.5 percentiles
Comparison between observed and downscaled
data for exceedances above the 97.5 dynamical
percentile
Exceedances of minimum daily
temperature above 97.5p for Indianapolis
1971-2000
Exceedances of minimum daily temperature
above 97.5 p for Boston 1971-2000
40
40
35
35
30
30
Maurer
25
BCCA-GFDL
Maurer
25
BCCA-GFDL
20
ARRM-GFDL
15
15
10
10
5
5
0
0
6
7
8
ARRM-GFDL
20
6
7
8
Need for Evaluation of Indices
• General evaluation based only on distributional
characteristics of variables of interest does not
contain sufficient usable information for
applications
• There is a need to evaluate applications related
indices and metrics
• Following good practices to incorporate uncertainty
is important – use of various downscaling methods
and an array of GCMs and emissions scenarios is
recommended (STARDEX, 2005; TGICA 2004)
During our working times with the
applications groups we will be working with
the tools, and will be discussing what is
usable interpretation of these evaluation
results and what guidance is needed on the
use of climate information in their work.
END
Summer Ave Tmax
95th percentile of
summer tmax
Minimum annual number of tropical nights