Transcript Welcome to the Gap Filling Comparison Workshop

Welcome
to the
Quic kTime™ and a
TIFF ( LZW) dec ompres s or
are needed to s ee this pic ture.
Gap Filling Comparison
Workshop
September 18-20, 2006
Antje Moffat
1
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Welcome
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
• 12 of the 14 members from the gap filling
comparison
• Over 30 participants - from all over Europe
(Germany, Italy, Netherlands, Switzerland),
US, Canada, Russia, and Australia
2
Goals of the Workshop
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
• Review of Gap Filling Techniques
• Completion of the Gap Filling Comparison
– Discussion of the Results
– Review of Paper
– Evaluation of the Techniques
• Work Sessions and Plenary Debates to
Exchange our Experiences and Expertise
• Generate Ideas for Further Improvement of
the Gap Filling
• New Insights into the Eddy Flux Data
3
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Outline
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Workshop Program
GFC Analysis
Performance of Techniques
Error on Annual Sum
4
Program: This Morning (Monday)
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
9:00 Registration of the Participants
If you need any kind of help, please contact Ulli or Silvana!
9:30 Setting the Stage of the Workshop
• Antje Moffat: “Welcome”
• Martin Heimann: “Biogeochemistry Research at the MPI in Jena”
• Dario Papale: “The CarboeuropeIP Ecosystem Component Database: data
processing and availability”
• Markus Reichstein: "Gap filling: Why and how?”
11:00 COFFEE BREAK (Foyer)
11:30 Review of Gap Filling Techniques: SPM and ANNs
• Vanessa Stauch: “Semi-parametric models”
• Dario Papale: “Gap filling of eddy fluxes with artificial neural networks”
• Rob Braswell: "Gap filling by iterative regression using a regularized neural
network”
12:30 LUNCH (Campus Cafeteria)
5
This Afternoon
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
14:00 Review of Gap Filling Techniques (cont.): NLRs and UKF
• Ankur Desai: “Towards a robust, generalizable non-linear regression gap filling
algorithm”
• Asko Noormets: “NLR_AM - AQRTa-Model” (10 min recording)
• Andrew Richardson: “Maximum likelihood non-linear regression model”
• David Hollinger: “Data assimilation for eddy flux filling: The unscented Kalman
filter”
15:30 COFFEE BREAK
16:00 Review of Gap Filling Techniques (cont.): Models and comparison
• Zisheng Xing: “A gap-filling model for tower-based NEP measurements”
• Jens Kattge: “Model parameter inversion against Eddy Covariance Data using
a Monte Carlo Technique”
• Bart Kruijt: “Comparing gap filling using neural networks and the CarboEurope
tool, for Fluxes and Meteo data”
19:30 Dinner Suggestion: Restaurant Papiermuehle (Please sign up!)
6
Tuesday Morning
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
9:00 Eddy and Component Flux Measurements
• Corinna Rebmann: “Eddy covariance measurements and their shortcomings for
the determination of NEE”
• David Hollinger: “Uncertainty in eddy flux data and its relevance to gap filling”
• Eva van Gorsel: “Nocturnal Carbon Efflux: Can eddy covariance and chamber
measurements be reconciled?”
• Pasi Kolari: “Gapfilling submodel selection based on measured component
fluxes”
10:30 COFFEE BREAK
11:00 Gap Filling of Grassland and Agricultural Sites
• Christof Ammann: “Gap-Filling of CO2 Fluxes of Frequently Cut Grassland”
• Mauro Colavincenzo: “A gap filling methodology used at a agricultural site in
Southern Italy”
• Irene Lehner: “Carbon balance of a maize canopy: comparison of different gap
filling strategies”
12:00 LUNCH
END of OPEN SESSIONS!
7
Tuesday Afternoon
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
13:00-15:00 Parallel Work Sessions Part 1
• Group 1: “Analysis of the partitioned GPP/ER comparison results” (Ankur
Desai)
• Group 2: “Gap filling of meteorological data and water and energy fluxes”
(Dario Papale)
15:00 COFFEE BREAK
15:30-17:30 Parallel Work Sessions Part 2
• Group 3: "Gap filling of sites with non-steady time series, e.g. cut grassland,
cropland" (Christof Ammann)
• Group 4: "Using gap-filling techniques for estimating random errors in eddy
covariance data" (Andrew Richardson)
Please sign up for the work sessions!
19:30 WORKSHOP DINNER (Restaurant: Weinbauernhaus “Im Sack”)
8
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Wednesday
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Roundtable on the Gap Filling Comparison
8:30 Review of Gap Filling Comparison Paper, Antje Moffat
- Interpretation of the comparison results
- Derivation of key findings
- Evaluation of techniques
12:30 LUNCH
13:30 Minutes from the four Work Sessions
14:30 COFFEE BREAK
15:00 Plenary Debates
- Site dependency of gap filling technique performances
- Filling of long gaps using previous years
- Conception of a public domain code library with filling routines
- Extended gap filling comparison for urban and crop
- Workshop resume and outlook
17:00 End of Workshop
9
Handling of Presentations
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
• 20 min presentations: 15 min talk plus 5 min
discussion
• Please transfer your presentation onto
common laptop during coffee or lunch break
(Ulli or Silvana)
• Publicized on GFC webpage after workshop
Questions?
10
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Gap Filling Comparison
Analysis
11
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Basic Principle
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.

Keyfile: Artificial Gap Flags

Golden File - fragmented:

Superimposition

Comparison of Observed NEP and Predicted NEP:
12
Statistical Metrics
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
• Bias Error
BE 
1
N
 p
i
 oi 
•p - predicted NEP
•o - observed NEP

• Root Mean
Square Error
RMSE
1
N
p
 oi 
2
i
• Correlation
Coefficient

2
( pi  p )(oi  o) 


2
R 
 ( pi  p ) 2 (oi  o) 2
13
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Analysis
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Predicted versus Observed
• Half-hourly basis
• Daily sum basis for full day artificial
gaps
•Daytime/Nighttime data
DSumd  xd
x


i
Nd
•Weighted ALL data

DSumALL 
xd * hhd xn * hhn

48
48
14
Challenge of the Analysis
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
*
*
*
*
*
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
5 artificial gap length scenarios (single hh - 12 days)
10 permutations
3 subsets: day, night, all
12 golden sites
19 submissions
15 statistical metrics: RMS, R2, Bias, Daily Sum,
normalized, benchmarked, …
513,000 comparison results!
(see selection on posters in foyer)
15
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
16
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Some Comparison
Findings
17
Correlation Coefficient R2
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
RMSE and R2:
Half-hourly Basis
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Daytime
Nighttime
Root Mean Square Error (gCm-2)
Performance of gap filling techniques from bottom
• MIM, MDV, UKF_LM, NLR & Others
• 3 ANNs leading
18
RMSE and R2:
Half-hourly & Daily Sums
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Correlation Coefficient R2
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Daytime
Daytime
Nighttime
Nighttime
Root Mean Square Error (gCm-2)
Half-hourly basis
Root Mean Square Error (gCm-2)
Daily sum basis
Daytime
•
R2: 0.6 - 0.8
•
RMSE: 2.5 - 4.0 gCm-2
Nighttime
•
R2: 0.2 - 0.4
•
RMSE: 1.5 - 2.5 gCm-2
Daytime
•
R2: 0.8 - 0.95
•
RMSE: 1.0 - 1.8 gCm-2
Nighttime
•
R2: 0.75 - 0.9
•
RMSE: 0.5 - 1.0 gCm-2
 Good filling performance for daytime
but not for nighttime
 Very good filling performance for
daytime and nighttime data
Techniques:
•
MIM, MDV, UKF_LM, NLR & Others, 3
ANNs leading
Techniques:
•
MIM, Others, ANNs leading
19
DailySum Bias per Site Year:
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Bias
Medium Gaplength, ALL
Medium gap length (1.5 days):
Bias of <0.07 gCm-2 per filled day
20
DailySum Bias per Site Year:
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Bias
Long Gaplength, ALL
Long gap length (12 days):
Bias of <0.2 gCm-2 per filled day
21
Annual Sum Error Estimate
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Assumption:
• representative choice of golden sites
• good technique (red stars)
 Error estimate on the annual sum
Annual Sum Error
• Small to med gaps: <0.07 gCm-2 per filled day equivalent
• Periods of longer gaps: <0.2 gCm-2 per filled day equivalent
 Quality of long gap filling critical
22
Calculation Example
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Example for average file with 35% gaps:
• 18% small to medium gaps
• 18% periods of longer gaps of 5-10 days
18% ^= 66 filled days
Error estimation
• 66 x 0.07 gCm-2: 5 gCm-2
• 66 x 0.2 gCm-2: 13 gCm-2
 Total error induced by filling of the
gaps on the annual sum:
±18 gCm-2
23
Test using Real Gap Filling Results
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Are 18gCm-2 an appropriate estimate of the
error on the annual sum prediction?
S ite year N L R _F _M O DN L R _F M _O L S A N N _B R
A N N _P S
be1 _2 0 0 0
- 3 4 5 .3 3
- 3 3 4 .8 8
- 3 4 5 .1 1
- 3 3 2 .5 9
be1 _2 0 0 1
- 5 1 4 .9 0
- 5 2 1 .1 2
- 5 1 6 .6 0
- 5 1 2 .0 7
de3 _2 0 0 0
- 5 1 6 .8 3
- 5 2 9 .6 1
- 5 3 3 .8 3
- 5 0 6 .7 5
de3 _2 0 0 1
- 5 0 5 .4 4
- 4 7 8 .8 4
- 5 0 7 .3 0
- 5 0 1 .4 3
fi1 _2 0 0 1
- 1 7 5 .2 1
- 1 7 0 .2 3
- 1 7 7 .8 7
- 1 7 3 .0 9
fi1 _2 0 0 2
- 2 1 9 .3 2
- 2 1 5 .2 6
- 2 2 4 .5 9
- 2 1 0 .8 8
fr1 _2 0 0 1
- 5 6 5 .3 8
- 5 5 8 .1 7
- 5 4 8 .9 0
- 5 6 0 .2 0
fr1 _2 0 0 2
- 5 4 7 .2 7
- 5 5 0 .1 8
- 5 4 5 .8 6
- 5 5 3 .6 2
fr4 _2 0 0 2
- 2 9 7 .5 5
- 3 0 4 .5 4
- 2 8 9 .6 7
- 3 2 3 .7 6
il1 _2 0 0 2
- 1 7 0 .6 4
- 1 7 2 .5 6
- 1 1 8 .6 4
- 1 4 7 .3 6
it3 _2 0 0 2
- 2 5 .9 4
- 2 3 .3 7
- 3 5 .3 9
- 2 9 .6 7
M DS
- 3 4 4 .8 5
- 5 1 0 .0 5
- 5 1 8 .7 6
- 4 9 8 .6 2
- 1 6 6 .8 4
- 2 1 9 .4 6
- 5 5 3 .0 4
- 5 4 2 .5 1
- 3 2 2 .5 5
- 1 6 0 .0 2
- 3 5 .5 5
SP M
- 3 3 0 .7 2
- 5 1 4 .6 8
- 5 1 6 .2 5
- 5 0 7 .2 9
nan
- 1 9 3 .8 3
- 5 4 2 .4 0
- 5 4 2 .0 3
- 3 2 8 .8 8
nan
- 2 1 .8 4
M ean S D eviation
- 3 3 8 .9 1
6 .9 0
- 5 1 4 .9 0
3 .8 2
- 5 2 0 .3 4
9 .8 4
- 4 9 9 .8 2
1 0 .8 4
- 1 7 2 .6 5
4 .2 9
- 2 1 3 .8 9
1 0 .8 5
- 5 5 4 .6 8
8 .2 9
- 5 4 6 .9 1
4 .4 8
- 3 1 1 .1 6
1 6 .0 81)
- 1 5 3 .8 4
22.09 1)2)
- 2 8 .6 3
5 .9 3
1) no soil temperature
2) 30 day system failure
•
Standard deviation between techniques of filling the real dataset with 
35% gaps ≤ 16 gCm-2
24
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Questions?
25
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Let’s fill our
“knowledge gaps”
Quic
kTime™
andaa
and
have
TIFF ( LZW) dec ompres s or
are needed to s ee this pic ture.
fun and productive
workshop!
26
Separation of Daytime and
Nighttime Data
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Keyfile: 10% artificial gaps
Fragmented Golden File: 80% daytime NEP data, 35%
nighttime NEP data
• Real gap filling:
20% real day gaps, 65% real night gaps
1:3
• Artificial gap filling:
8% artificial day gaps, 3.5% artificial night gaps
2:1
 Important to consider daytime and nighttime data
separately
27
Bias on daily sums
Daytime data
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
• Distribution of bias error of the individual daily sums
Bias Error (gCm-2)
Daytime data
Daily Bias Error:
- up to 4 gCm-2
ANNs leading
ANN_BR
28
Bias on daily sums:
Nighttime data
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee t his pic ture.
Quic kTime™ and a
TIFF ( LZ W ) dec ompres s or
are needed to s ee this pic ture.
• Distribution of bias error of the individual daily sums
Bias Error (gCm-2)
Daytime data
Daily Bias Error:
- up to 2 gCm-2
ANN_PS leading
29