Transcript RT5: Evaluation Albert Klein Tank & Elisa Manzini ENSEMBLES General Assembly Prague, Czech Republic, 14 November 2007

RT5: Evaluation
Albert Klein Tank & Elisa Manzini
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
RT5: Evaluation
Albert Klein Tank & Elisa Manzini
Objective:
to perform a comprehensive and
independent evaluation of the
performance of the ensemble
simulation/prediction system against
observations/analyses
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
• With some delay, all RT5 deliverables and
milestones are met
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
• With some delay, all RT5 deliverables and
milestones are met
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
global
multidecadal
global
s2d
regional
multidecadal
impact
models
• With some delay, all RT5 deliverables and
milestones are met
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
global
multidecadal
• Evaluation of global models used in
multidecadal simulations (RT2a)
– D5.15: impact of improved land-surface physics on
the simulation of the Boreal summer surface
climate (INGV)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
global
multidecadal
global
s2d
• Evaluation of forecast quality with global
models in s2d simulations (RT2a)
– D5.11: representation of the tropical intra seasonal
variability and its impact on seasonal
predictability (IPSL)
– D5.12: predictability in perfect-model mode over the
North Atlantic/European sector (IfM)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
global
multidecadal
global
s2d
regional
multidecadal
• Evaluation of regional models used in
multidecadal simulations, with focus on
extremes (RT2b and RT3)
– D5.18: ENSEMBLES daily gridded dataset for Europe
(UEA)
– D5.20: observational station data collected for the
ENSEMBLES daily gridded dataset for Europe
(KNMI)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
global
multidecadal
global
s2d
regional
multidecadal
impact
models
• Evaluation of impact models (RT6)
– D5.13: challenges of interpreting and application of an
ensemble of imperfect models on seasonal
time scales (LSE)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Achievements
global
multidecadal
global
s2d
regional
multidecadal
impact
models
• Some examples:
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
GCM representation of vertical circulation regimes
(ω500 dynamical proxy) for tropical eastern Pacific
-> double ITCZ syndrome (INGV)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Evaluation of DEMETER Asian summer monsoon
intraseasonal variability ISV (Xavier, LMD)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
KNMI Climate Explorer  ENSEMBLES data server
(climexp.knmi.nl by Van Oldenborgh, KNMI)
Select data in the Climate Explorer
Data is retrieved from ECMWF
s2d server with OpenDAP
Available for analysis, e.g. skill scores
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
T2m JJA forecast, ini: May, DEMETER, 1960-2001
cross-validation using modified ranked probability skill score
RPSS (Weigel et al., MeteoSwiss)
RPSSD
-1
-0.5
0
0.5
ECMWF
Multi-model (equal weights)
UKMO
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
1
T2m JJA forecast, ini: May, DEMETER, 1960-2001
cross-validation using modified ranked probability skill score
RPSS (Weigel et al., MeteoSwiss)
RPSSD
-1
-0.5
0
0.5
1
ECMWF
Multi-model (weighted + climatology)
(more orange + no blue)
UKMO
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
ECA dataset for gridding (KNMI and MeteoSwiss)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
ECA dataset for gridding (KNMI and MeteoSwiss)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
ECA dataset for gridding (KNMI and MeteoSwiss)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Best gridding method for daily rainfall
(Hofstra et al., Oxford Univ.
and Haylock et al., Univ. of East Anglia)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Comparison with existing daily datasets
(Hofstra et al., Oxford University)
Daily precipitation
Compound relative
error score CRE
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Comparison with ERA40
(Hofstra et al., Oxford University)
Daily Tmax
RMSE
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
ECMWF seasonal forecast storms, %total insured value
SwissRe portfolio (Della-Marta & Liniger, MeteoSwiss)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Verification of wheat yield at the Bologna University
experimental farm forecasted with DEMETER
downscaled input (ARPA)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
Plans
• Another 16 deliverables until the end of the
project, including all-GCM evaluation efforts
• Start of more regional work now that the RT5
observational data set and the RCM data are
available
• Note that good access to all data remains
essential for RT5!
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007
RT5 review
• Taken up the recommendation to further
integrate RT5 work within ENSEMBLES:
– Monday workshop on RCM and SDS evaluation
(together with RT2b and RT3)
– Thursday workshop on model error (with RT4)
– Thursday workshop on weighting, credibility and
reliability (with RT3; Geert Lenderink)
ENSEMBLES General Assembly
Prague, Czech Republic, 14 November 2007