Transcript Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Operational QC on raw data of the new Swiss ground-based measurement.

Federal Department of Home Affairs FDHA
Federal Office of Meteorology and Climatology MeteoSwiss
Operational QC on raw data
of the new Swiss ground-based
measurement network
Roulet Y.-A., Laubscher S., Rast J., Heimo A.,
Overney P. and Calpini B.
MeteoSwiss
TECO-2006, WMO, Geneva, Switzerland
4-6 December 2006
SwissMetNet (SMN) Project
Renewal of all meteorological networks existing
in Switzerland, operational under the
responsibility of MeteoSwiss
ca. 130 automatic weather stations
connected to a secured network
(see P1(16) from A. Heimo)
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
2
SMN data flow architecture
Sensors
BV-NET
ADAS
CDAS/NIMDAS
Data Warehouse
(DWH)
MCH LAN
DataMart
(DM)
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
3
SMN station in Aigle
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
4
SMN station in Gütsch (alpine)
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
5
Main purposes of a QC
• Avoid gaps in time series (detect instruments’
failures as quickly as possible), and guarantee
high data availability to our customers
• Improve know-how on measurements’ techniques
• Determine MTBF for each sensor and optimize
inventory control for spare parts and on-site
interventions
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
6
1st QC step: real time QC
A product (MetConsole) developed by
ALMOS / TELVENT for MeteoSwiss
Providing min, max, mean, standard deviation and
number of valid samples for each measured parameter.
Integrated on-line QC using e.g. threshold values,
dead band criterias or jumps.
Purpose: to detect instantaneously
instrumental failure  alarm list
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
7
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
8
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
9
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
10
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
11
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
12
2nd QC step: time series (QCT)
Off-line QC of time series raw data
measurements (3 months)
Purpose: to detect as quickly as possible
drifts or trends due to instrumental problems
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
13
2nd QC step: time series (QCT)
1st phase: determine parameters sensitive to
instrumental problems (indicators).
2nd phase: determine criteria to detect instrumental
problems (e.g. by analysing past known cases, testing
instruments behavior in laboratory, etc.).
3rd phase: implement the criteria in an operational
mode, with alarm in case of suspicious parameter
behavior.
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
14
QCT operational
Station
Network
X
Y
Z
dkl010z0
dkl010za
dkl010zc
dkl010zn
dkl010zx
gre000z0
gre000za
gre000zc
gre000zn
gre000zx
hto000s0
lu2000z0
lu2000za
lu2000zc
lu2000zn
lu2000zx
oli000z0
…
Search
procedure
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
Type
ABO
SMN
AIG
SMN
WIND
WIND
HKD
WIND
WIND
RAD
RAD
HKD
RAD
RAD
SNOW
RAD
RAD
HKD
RAD
RAD
RAD
L 14512
L 14512
L 14512
L 14512
L 14512
CM21
CM21
CM21
CM21
CM21
SR50
LICOR
LICOR
LICOR
LICOR
LICOR
L 14512
L 14512
L 14512
L 14512
L 14512
CM21
CM21
CM21
CM21
CM21
SR50
LICOR
LICOR
LICOR
LICOR
LICOR
CG4
…
Data extraction
from the DM
15
QCT operational
Read the rule
Select data from
the DM
DataMart
Upload data
from DM
Loop over
all stations
Execute
QC test
Produce alarms
(email)
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
16
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
17
QCT procedure: Example I
Number of zero wind speed measurements
72
66
Number of measurements [-]
60
5d_AIG
5d_FEY
54
48
Sensor replaced
42
36
30
QCT  action!
24
18
12
6
0
1.01 1.11 1.21 1.31 2.10 2.20 3.01 3.11 3.21 3.31 4.10 4.20 4.30 5.10 5.20 5.30 6.09 6.19 6.29 7.09 7.19
Time
Date
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
18
QCT procedure: Example II
Pressure derivative for PIL and LUZ stations, 05/01/01 to 05/04/20
4
Pressure derivative [hPa/10 min]
3
2
1
0
-1
dP/dt PIL
(dP/dt)+2 LUZ
-2
-3
1
6
11
16
21
26
31
5
10
15
20
25
2
7
12
17
22
27
1
6
11
16
Day
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
19
QCT procedure: Example III
PAY, Soil
WAE,
Soil temperature
temperature difference,
difference, 01.01.04-30.06.04
78
T(-5) - T(-20)
T(-10) - T(-20)
AVG dT(-5,-20)
AVG dT(-10,-20)
6
6
5
Temperature differencs [K]
4
4
2
3
20
1
-2
0
-4
-1
-6
-2
-3
-8
1
11
21
31
10
20
1
11
21
31
10
20
30
10
20
20
30
30
99
19
19
29
29
Day
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
20
Perspectives
 Operational implementation and continuous
development of QCT
 Development of QCS (spatial dimension):
 develop algorithms to interpolate meteorological
data measured by the SMN network
 use GIS to spatially check the quality of raw
measurements using adjacent stations
 such a system will be able to distinguish between
a broken sensor and a long-term drift of errors
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
21
Thank you for your attention!
Operational QC on SMN raw data | TECO-06, WMO, Geneva
Y.-A. Roulet et al., MeteoSwiss
22