Deterministic vs Probabilistic Forecast J.P. Céron – Météo-France The Predictability « a Thunderstorm will be observed next Sunday over the Toulouse « Météopole »

Download Report

Transcript Deterministic vs Probabilistic Forecast J.P. Céron – Météo-France The Predictability « a Thunderstorm will be observed next Sunday over the Toulouse « Météopole »

Deterministic vs
Probabilistic Forecast
J.P. Céron – Météo-France
The Predictability
« a Thunderstorm will be observed next Sunday
over the Toulouse « Météopole » between
15h and 16h »
 Irrealistic, the confidence that one can
have in this forecast is very low
« a rainy system will cross the Toulouse region
Sunday afternoon »
 realistic, one can be quite confident in
this forecast
The Predictability
The predictability depends on :
 The scale of the forecasted phenomenum
(Thunderstorm, Easterly Wave, Blocking
situation, ENSO, …)
 The Range of the forecast (NowCasting, Short ,
Medium , Seasonnal , Climatic)
Deterministic formulation  error or lost of
informations
Probabilistic formulation  more possibilities in
the forecast but interpretation problems
The different Forecasts


How do you play to Horse races?
 Informations about the form of horses, trainers, jockey,
race conditions (type of soil, weather, …), predictions,
…
 Synthesis then making a bet on horses (in a more or less
subjective choice)
Use of seasonnal forecasts.
 Informations about the states of atmosphere, continental
surfaces, oceanic system and its evolutions, the
different forecasts, …
 Synthesis and decision/action (i.e. make a bet on the
real solution in a more or less objective way)
The different Forecasts


The horse n°5 will win the race.
It will rain 650 mm at Niamey during the next
rainy season.
Consequences : gain or lost depending of the
success or not of the forecast.
The different Forecasts
Deterministic forecast
The
different
Forecasts
Deterministic
forecast
The different Forecasts
Deterministic forecast
Limits of Numerical Forecasting
ANALYSIS
- errors (or uncertainty) from measurements, assimilation.
- lack of observations over some regions
Network density of surface observations
Over the whole globe
The forecast errors can come from different part of
the forexast system :
But also ...
Limits of Numerical Forecasting
ANALYSIS
- errors (or uncertainty) from measurements, assimilation.
- lack of observations over some regions
MODELS LIMITS
- mesh, number of vertical levels (resolution of the model)
- Equations, parameterisations
- small scale phenomena (under the mesh or badly know)
- L'équation vectorielle du mouvement :
Sur l'horizontale :
1 




h p - 2Vh + Fh
V

dVh
=
dt
Sur la verticale : équation de l'hydrostatique d'après les approximations
1 p
g
0=
 Z
Equations are generally
But also ...
- L'équation de la Thermodynamique :
d (cpT)
RT dp
=
+Q
dt
p dt
etc...
“simplifyed”
and one calibrate “parametrisations”
in the model, that is to say that one
use data computed in an approximate
form (even sometime as “constant”).
Limits of Numerical Forecasting
ANALYSIS
- errors (or uncertainty) from measurements, assimilation.
- lack of observations over some regions
MODELS LIMITS
- mesh, number of vertical levels (resolution of the model)
- Equations, parameterisations
- small scale phenomena (under the mesh or badly know)
INTERPRETATION
- Models generally provide “raw data” and
consequently the interpretation is often difficult)
And finally ...
Limits of Numerical Forecasting
ANALYSIS
- errors (or uncertainty) from measurements, assimilation.
- lack of observations over some regions
MODELS LIMITS
- mesh, number of vertical levels (resolution of the model)
- Equations, parameterisations
- small scale phenomena (under the mesh or badly know)
INTERPRETATION
- Models generally provide “raw data” and
consequently the interpretation is often difficult)
COMMUNICATION
- misfitted vocabulary
- misreading of users’ needs
- dissemination without feedbacks from the user
Limits of Numerical Forecasting

Natural variability of the Ocean/Atmosphere system and
the Atmospheric respons to external forcing
Limits of Numerical Forecasting







Uncertainties of the initial state of the Climatic system
Modelisation Error (both Oceanic and Atmosphéric)
Natural varibility of Atmosphere and its respons to
external forcing
Interpretation of the forecast
To sample the initial state uncertainty  disturbances
of the analysis
To sample the model uncertainty  distubances of the
model
To sample in the forecast all the possible solutions of
the Ocean/Atmosphere system
Limits of Numerical Forecasting
The numerical forecast using "ARPEGE« model at short range is a
« déterministic » forecast. It uses the initial state description of the climate
system and, using model’s equation, allow to perform the time evolution of the
atmospheric state.
Major errors of this type of forecasts mainly come from initial state errors
intoduced inside the model. The uncertainty spread increase with the range of
the forecast.
error at time t0 + range
error at time t0
(initial uncertainty)
Limits of Numerical Forecasting
To take into account the probability of the deterministic evolution, one
perform several forecasts starting from the same initial time but using slightly
modified values of the parameters of the simulation
(namely inside the probable range of errors introduced at the initial time).
At this range:
Strong probability
region
value of the parameter
Deterministic
forecast
Forecast range
Initial time
Limits of Numerical Forecasting
Because of computer ressources, one use a larger mesh model (to
limit the computation time) and one perform, typically, around thirty
forecast using different conditions for the model.
1
0 - 6%
25 - 50%
6 - 25%
50 - 75%
75 - 100%
Deterministic model
verification
850 hPa Temperature plumes (Toulouse)
base 21 september 1999 at 12h UTC
Limits of Numerical Forecasting
One can look at the parameter’s dispersion as a function of the time
intergration and describe the encountered value distribution rath
rather than the values themselves. One can also give a confidence
indice with a more or less high value (the larger the dispersion of the
forecasts, the lower the indice that is to say the lower the confidence).
1
0 - 6%
25 - 50%
6 - 25%
50 - 75%
75 - 100%
Deterministic model
verification
850 hPa Temperature plumes (Toulouse)
base 21 september 1999 at 12h UTC
Temperature dispersion plumes as a function of the time integration
This method is named “Ensemble forecast”
Limits of Numerical Forecasting
The ensemble forecast (Resume)





To sample the initial state uncertainty  disturbances
of the analysis
To sample the model uncertainty  distubances of the
model
To sample in the forecast all the possible solutions of
the Ocean/Atmosphere system
Several forecasts – trying to get the distribution of the
possible solutions instead of a single value
Several models – How can we merge the informations
coming from many different models – empirical, AGCM,
COAGCM ? Multimodel approach.
Limites of Statistical Forecasting

Uncertainty already included inside the statistical tools
The different Forecasts

The horse n°5 will win the race.
It will rain 650 mm at Niamey during the next
rainy season.

The horse n°5 has good chances to be in the
firsts in this race.
The situation of the Ocean/Atmosphere
system and its probable evolutions indicate
that the next rainy season in Niger has a good
probability to be « above the Normal ».
Consequences : gain or lost depending of the
success or not of the forecast

The different Forecasts
Probabilistic Forecast
The different Forecasts
Probabilistic Forecast
The different Forecasts
Probabilistic Forecast
The different Forecasts
Probabilistic Forecast
The different Forecasts

Analogue technics
Probabilistic Forecast : formulation





1 model et n members
p models et n members
Gaussian : mean + standard deviation
Analogues
Statistical Methods (Discriminant Analysis,
Multiple Regression, Probabilistic Regression,
…)
Catégories Forecast
Probabilistic Forecast
Catégories Forecast

How to define the categories?




How to evaluate the forecasted probabilities for
each category?




Number
Categories Limits
Needs of user?
Frequency/Probability , Climatological Probabilities,
Conditionnal probabilities, Confidence Indice
Statistical Models
Numerical Models
How to transform the forecast in « readable and
comprehensive » form for the user?
Probabilistic Forecast: verification



Quadratic Scores (Brier, RPS, …)
Relative Operating Characteristic (FA vs ND)
Cost/Lost ratio approach

Deux categories: + dry / + wet et Ratio c/L=0.5
obs
non
obs
obs
prev
c
c
prev
n11 n12
non
prev
L
0
non
prev
n21 n22
C1= averaged cost using a climatological forecast
C2 = averaged cost using a perfect forecast
C3= averaged cost using a the model forecast
non
obs
V100 C1C3
C1C2
Brier Scores :
BrSc = 1/N  (pi – oi)2
pi probabilité prévue pour l’événement
oi variable indicatrice de l’observation de l’événement
1
BrSc=  o(p) 1-p2 + 1-o(p) p2  g(p) dp
0
g(p) Fréquence relative avec laquelle l’événement est prévu avec une probabilité comprise
entre p et p+dp
1
1
1
0
0
0
BrSc= f 1- f  +   p1-o(p) 2-  f-o(p) 2  g(p) dp
Uncertainty
Reliability
Resolution
Pour un système qui aurait toujours prévu la probabilité climatique d’occurrence ( pflt )
LCBrSc= f 1- f  + f 1- flt
[L/S]CBrSkSc = 1 – BrSC / [L/S]CBrSc Brier Skill Score
BSSREL = 1 – BSREL // [L/S]CBrSc Brier Reliability SC
BSSRSL = BSRSL / Uncertainty
Brier Resolution SC
Probabilistic Forecast: verification

Reliability Diagrams
Probabilistic Forecast

Cost/Lost ratio approach
Probabilistic Forecast

Differents users
Probabilistic Forecast

Comparison between Deterministic and
Probabilistic formulation
Probabiliste
Déterministe
Probabilistic Forecast

Resume