Transcript Statistical Downscaling Approach & its Application in EMC Bo Cui , Zoltan Toth

Statistical Downscaling Approach & its
Application in EMC
Bo Cui1, Zoltan Toth2, Yuejian Zhu2
1SAIC
at Environmental Modeling Center, NCEP/NWS
2Environmental Modeling Center, NCEP/NWS
Acknowledgements
Ken Mitchell, EMC/NCEP
Manuel Pondeca, EMC/NCEP
Downscaling Method with Decaying Averaging Algorithm
 True = High Resolution Analysis
• operational North American Real-Time Mesoscale Analysis (RTMA)
• 5x5 km National Digital Forecast Database (NDFD) grid (e.g. G. DiMego et al.)
• 4 variables available: surface pressure, T2m, 10m U and V
• other data can also be used
 Downscaling Method: apply decaying averaging algorithm
Downscaling Vector = (1-w) * prior DV + w * (GDAS – RTMA)
 four cycles, individual grid point, DV = Downscaling Vector
 GDAS analysis interpolated to RTMA grids
 regime (not flow) dependent
 choose different weight: 0.5%, 1%, 2%, 5%, 10%
 Downscaling Process
Downscaled Forecast = Bias-corrected Forecast – Downscaling Vector
 subtract DV from bias-corrected forecast valid at analysis time
 bias-corrected forecast interpolated to RTMA grids
Downscaling Application & Verification
 Experiments
• control 1: operational GEFS ensemble after interpolated to RTMA grids
• control 2 : NAEFS bias corrected ensemble after interpolated to RTMA grids
• 5 downscaled ensembles : 0.5%, 1%, 2%, 5%, 10% weights when calculating DV
 Application
• off-line experiments starting from 08/11/2006, different decaying weights
• baseline for evaluating other sophisticated flow dependent downscaling methods
 Verification
• Domain averaged bias (absolute values) comparison: before & after downscaling
 Accumulated bias are derived from 7 experiments against RTMA with 2% weight
Accumulated Bias = (1-w) * prior accumulated bias + w * ( mean forecast – RTMA)
•
•
•
•
 mean forecast from control 1, 2 and 5 downscaled ensemble mean, respectively
Ensemble mean & bias comparison before & after downscaling: 00 hr, 24 hr
Continues Ranked Probability Scores (CRPS)
Ensemble mean RMSE and ensemble spread
Downscaling Vector comparison
2m Temperature: Accumulated Bias Before/After RTMA Downscaling
1.7
1%
2%
10%
0.2
Black- control 1, operational ensemble mean, bias range 1.1- 1.7
Red - control 2, NAEFS bias corrected ensemble mean, bias range 1-1.6
Blue- downscaled & bias corrected ensemble mean, 1%, bias range 0.5-0.6
Green- downscaled & bias corrected ensemble mean, 2%, bias range 0.3- 0.5
Yellow- downscaled & bias corrected ensemble mean, 10%, bias range 0.2-0.4
10m U Wind: Accumulated Bias Before/After RTMA Downscaling
0.7
1%
2%
10%
0.2
Black- control 1, operational ensemble mean, bias range 0.7-0.85
Red - control 2, NAEFS bias corrected ensemble mean, bias range 0.6-0.8
Blue- downscaled & bias corrected ensemble mean, 1%, bias range 0.4-0.45
Green- downscaled & bias corrected ensemble mean, 2%, bias range 0.3-0.35
Yellow- downscaled & bias corrected ensemble mean, 10%, bias range 0.2-0.3
10m V Wind: Accumulated Bias Before/After RTMA Downscaling
Black- control 1, operational ensemble mean, bias range 0.6-0.8
Red - control 2, NAEFS bias corrected ensemble mean, bias range 0.55-0.65
Blue- downscaled & bias corrected ensemble mean, 1%, bias range 0.4-0.5
Green- downscaled & bias corrected ensemble mean, 2%, bias range 0.3-0.4
Yellow- downscaled & bias corrected ensemble mean, 10%, bias range 0.2-0.25
00hr GEFS Ensemble Mean & Bias Before/After Downscaling 10%
10m U Wind
2m Temperature
Before
After
Before
After
24hr Ensemble Mean & Bias Before/After RTMA Downscaling 10%
Before
Before
After
 Left top: operational ens. mean and its bias vs. RTMA
 Right top: bias corrected ens. mean and its bias
 Left bottom: bias corrected & downscaled ( 10% ) ens.
mean and its bias vs. RTMA
After Downscaling
 More detailed forecast information
 Bias reduced, especially high topography areas
24hr Ensemble Mean & Bias Before/After RTMA Downscaling 10%
Before
Before
After
 Left top: operational ens. mean and its bias vs. RTMA
 Right top: bias corrected ens. mean and its bias
 Left bottom: bias corrected & downscaled ( 10% ) ens.
mean and its bias vs. RTMA
After Downscaling
 More detailed forecast information
 Bias reduced, especially high topography areas
24hr Downscaled Ensemble Bias Comparison : 2%, 5% & 10%
2%
5%
10%
Bias corrected & downscaled ensemble mean
and its bias left vs. RTMA: T2m
 more bias are reduced for 10% test
 lakes have different bias from surrounding areas.
10% can eliminate most of the cold bias.
 Downscaling vectors display the similar bias
over lakes. Which system is doing the poor job of
analysis over lake? GFS analysis or RTMA?
GDAS Analysis & Downscaling Vector ( 10% )
GDAS Analysis & Downscaling Vector ( 5% )
GDAS Analysis & Downscaling Vector ( 2% )
2m Temperature: Continuous Ranked Probability Scores (CRPS)
Average for 20070212 to 20070305
After downscaling
CRPS: 2m Temperature
CRPS: 10m U
CRPS: 10m V
Preliminary results:
 2%, 5% & 10% have significant improvements
compared with raw & calibrated fcst. till day 7
 10% is better than 2% and 5% in short range, 2%, 5%
and 10% are close for long range
Limitation:
 small samples, 22 cases
 more samples in short range than long range
2m Temperature: Ensemble Mean RMSE and Ensemble Spread
Average for 20070212 to 20070305
2m Temperature
10m U
10m V
Preliminary results:
 downscaled forecast have reduced RMSE compared
with raw & bias-corrected forecast
 small ensemble spread changes for different tests
 distance between RMSE and ensemble spread has a
decreasing tendency with forecast lead time. The ens.
mean RMSE and spread curves are becoming close for
long lead time
Summary & Future Plan
 Summary
• systematic (time mean) error: downscaling method with decaying averaging
algorithm can effectively reduce systematic forecast errors. The 10% weight has
the best performance, ~ 70% of T2m, 10m U and V wind systematic errors are
reduced.
• more detailed forecast information available in the downscaled forecast.
• CRPS show that the downscaled & bias-corrected ensemble forecasts have been
improved compared with the raw and bias corrected ensembles.
• RMSE: downscaled forecasts have reduced ensemble mean RMSE compared
with the raw and bias-corrected forecasts, 10% - 20% of RMS errors reduced.
 Future Plan
•
•
•
•
more weight factor tests choosing 20% and do comparison with 10%.
study on systematic and random error components, respectively.
add downscaled 10-50-90 percentile forecast values for selected variables.
Downscaled method scheduled to be implemented later in 2007.
Background !!!!!
Introduction

Definition of downscaling for this discussion (there are other ways to define):
•
Relationship between coarse & fine resolution geo-science information
 How we create fine resolution info based on coarse resolution fields
 In space and/or in time
 Related to finite spatial/temporal resolution
 Not directly related to any numerical model
 Eg, relationship between low and high resolution analyses: How can we predict hires from low-res?

Definition of “forecast bias correction”
•

Relationship between bias correction & downscaling
•

Removal of systematic (time mean) error from forecast
 On original forecast grid
 Related to “drift” of numerical model forecast
They are “orthogonal”
 Different problems, different methods can (or should?) be used
 Benefit from understanding differences between forecast bias correction & downscaling
 Some studies address both forecast bias correction & downscaling with same method
Objectives for downscaling
•
•
Systematic (time mean) errors on fine scale
• Want to eliminate
• From all ensemble members
Variability on fine scales
• Want to insert variability so down-scaled info has realistic variance
• Will hurt individual forecast
• Will help ensemble forecast
• Ensemble mean should be unchanged
Statistical Post-Processing Issues
 Statistical Downscaling Method
• Regression: MOS techniques; Bayesian technique
• “PDF” matching: (aka CDF matching)
• Matching cumulative probability density functions (frequency of occurrence)
• Neural Networks
• Analog Methods: Tom Hamill and Jeff Whitaker; Self Organizing Maps (SOMs)
• Downscaling Vectors (DV) with decay averaging algorithm
• requires an independent hi-res analysis (e.g. RTMA applied to GFS forecast)
• derive static DV for past N times: subtract hi-res anal from low-res fcst valid at anal time
• averaged DV: apply decreasing weights to past N static DVs (above bullet)
•
Canonical Correlation Analysis: CCA
• e.g. widely used in CPC (Huug Van den Dool)
• Redundacy Analysis: a variant of CCA
• Singular Value Decomposition: a variant of CCA
 Bias Correction vs. Downscaling
• Bias Correction : remove lead-time dependent bias on model grid
 Working on coarser model grid allows use of more complex methods
 Feedback on systematic errors to model development
• Downscaling: downscale bias-corrected forecast to finer grid
 Further refinement/complexity added
• No dependence on lead time, No dependence on model
Downscaling: Some basic preliminaries
by Ken Mitchell
 Definition:
• Any method to represent a global or regional model’s atmospheric, oceanic or
land analysis or forecast at substantially finer resolution than the original product
 Common Motivation:
• Represent the influence of finer scale topography
 Main Types of Methods
• Dynamical: predictive models, with temporal evolution
• imbed finer scale predictive model inside parent global or regional model
• often the most expensive approach to develop, execute and maintain
• Physical: no temporal predictive element
• Kinematic: includes adjustment of large scale wind and pressure field to finer scale
topography
• Static: usually applied 1-D in vertical, without spatial adjustment of wind and pressure
field to finer terrain
• Statistical: analog techniques, neural networks, others
• The broadest category, usually requires long archive of parent model prediction
• Combination: combination of two or more of above
 Prediction Range
• short-range (1-3 days), medium-range (1-15 days), sub-seasonal (1-6 weeks) or
seasonal to annual (1-12 months)
Bias Correction Method & Application
 Bias Correction Techniques – array of methods
 Estimate/correct bias moment by moment (e.g., D. Unger et al.).
• Simple approach, implemented partially
• May be less applicable for extreme cases
 Bayesian approach (e.g., Roman Krzysztofovicz)
• Allows simultaneous adjustment of all modes considered, under development
 Moment-based method at NCEP: apply adaptive (Kalman Filter type) algorithm
decaying averaging mean error = (1-w) * prior t.m.e + w * (f – a)
For separated cycles, each lead time and individual grid point, t.m.e = time mean error
6.6%
3.3%
1.6%
Toth, Z., and Y. Zhu, 2001
• Test different decaying weights.
0.25%, 0.5%, 1%, 2%, 5% and
10%, respectively
• Decide to use 2% (~ 50 days)
decaying accumulation bias
estimation
Bias Before/After Bias Correction ( NCEP NH)
500hPa height
850hPa temperature
Before bias correction (1x1)
After bias correction (1x1)
Sea level pressure
2m Temperature
24hr Ensemble Mean Forecast & Bias Left: 2%,5% & 10%
2%
10%
5%
Bias corrected & downscaled ensemble mean
and its bias left vs. RTMA: 10m U:
 more bias are reduced for weight 10% test
24hr Downscaled Ensemble Mean & Bias Comparison : 2%,5% & 10%
2%
5%
10%
Bias corrected & downscaled ensemble mean
and its bias left vs. RTMA: T2m
 more bias are reduced for weight 10% test
 lakes have different bias from surrounding
areas. 10% can eliminate most of the cold
bias. Downscaled vectors also display this
bias. Which system is doing the poor job of
t2m analysis over lake? The GFS analysis or
RTMA?