Transcript Quality management, calibration, testing and comparison of instruments and observing systems M. Leroy, CIMO ET on SBII&CM.

Quality management, calibration,
testing and comparison of
instruments and observing
systems
M. Leroy,
CIMO ET on SBII&CM
Quality management, some aspects




Explicit identification of uncertainty objectives
Representativeness of observations
Calibration and maintenance
Selecting instruments : knowing their characteristics
– Testing
– Intercomparisons
 Composite observing systems
 Value analysis
 ISO 9001-2000 is an international frame
Quality management process




Setting and documenting objectives
Defining associated indicators
Regular survey of indicators
Regular survey of users’ (customers) needs and
satisfaction.
 Correcting and improving
Identification of uncertainty objectives
 Identification of users of observation data
– Forecasters and climatologists
– Few direct users of observation : special case is aeronautic
users, with needs clearly defined in Annex 3 of ICAO.
 Getting the users’ needs is not always easy
– They are not always aware of the possible uncertainty and
associated cost.
– A value analysis is necessary to get a good compromise.
 The CIMO guide (WMO doc n° 8) is a guide.
– Annex 1B indicates users’ needs and achievable measurements
(best state of the art).
 The measurement uncertainty must be demonstrated.
So realistic choices must be done.
An example
 For his proprietary Radome network, mainly
dedicated for real-time observation, MétéoFrance has set up the following objectives;
some are less ambitious than the achievable
measurement uncertainty stated by the CIMO
guide.
–
–
–
–
–
–
0.5 hPa for pressure
6% for relative humidity
0.5°C for air temperature
10% for wind speed
5-10% for precipitation amount
5% for daily amount of solar radiation.
 For other objectives, additional care can be
necessary
 Example : US Climate Reference Network
–
–
–
–
Redundancy and cross check of measurements.
3 independent air temperature measurements.
Dual fence rain gauge.
Associated measurement of wind speed.
 Such efforts cannot be made in each station of a
climatological cooperative network.
Quality indicators
 Availability of data in the (final) data base
– Within a given limit of time : example 95% of expected surface
observations available within 10 minutes.
– Availability of data in the climatological data base : example 98%
for the Radome network.
 Measurement uncertainty
– % of sensors calibrated in a stated delay : example 12 months
nominal, 15 months maximum, objective is more than 90%.
– % of sensors found outside an Acceptable User Limit : example
hygrometers found with one (or more) control point more than
5% from the reference. These sensors are flagged as having
drifted.
– % of sensors for which the periodicity of preventive maintenance
is respected.
– % of large errors when compared to numerical models
(monitoring)
Some examples
Knowing the characteristics of an instrument
 Technological survey : reading publications,
participating to conference and instrument
exhibitions.
 Testing of instruments : please, publish results
and make them available.
 Intercomparisons, WMO intercomparisons.
 Read the manufacturers’ technical
documentation, having in mind the common
characteristics of instruments, for example, thru
the CIMO guide.
Recent WMO intercomparisons
 Radiosonde intercomparisons
 WMO Laboratory Intercomparison of Rainfall
Intensity gauges
 WMO Field Intercomparison of RI gauges : May
2007-May 2008, Vigna di Valle, Italy
 WMO Intercomparison of Thermometer
Screens/Shields in conjunction with Humidity
Measurements : 2007, Ghardaïa, Algeria
WMO Field Intercomparison of Rainfall
Intensity Instruments
 The laboratory Intercomparison was a first step, easier to handle
than a field test.
 Not all instrument types available on the market were tested in
laboratory.
 Looking for a possible site, Vigna di Valle, Italy (Italian
Meteorological Service, ReSMA) was selected.
August 2007  August 2008
Field Intercomparison
 Call for instruments (March 2006)
 24 instruments of different measuring principles were selected (54
proposed !).
 Reference based on a set of high quality devices, inserted in a pit
gauge.
 Draft Rec. 4.2/2 : Procedure and reference instruments for field
rainfall intensity intercomparisons
WMO Combined Intercomparison of Thermometer Screens/Shields, in
conjunction with Humidity Measuring Instruments
 Looking for a possible site in harsh conditions, Ghardaïa, Algeria
was selected.
 Desert region
 Jan. 2007  Jan. 2008
 Instruments’ calibration before and after
Screens & Hygrometers Intercomparison
 Call for instruments (March 2006)
 16 screens/shields’ types (29 proposed) and 11 hygrometers’
types(17 proposed) were selected.
 Reference : Thygan
 Data analysis :
– International Organizing Committee
– ISO standard 17714 (draft)
Conclusion
 There are many other subjects and actions concerning
quality measurement :
– Quality control directly at the level of the measurement systems.
– Quality control using spatial and temporal cross-check.
– Representativeness of observations, due to the nearby
environment of the measuring site.
– …
 Quality management and ISO 9001-2000 certification
force us to explicit the objectives of observing networks
and the proofs demonstrating their achievement.