Transcript Slide 1

Quality Control Standards for SeaDataNet
• Review status at 1st Annual Meeting
(March 2007)
• Review developments over last year
• Current status
• Future work
Quality Control Standards for SeaDataNet
OUTLINE OF QUALITY CONTROL DOCUMENT
• Introduction
• Why is quality control is needed?
• Information to accompany data
• Automatic checks
• “Scientific” quality control
• CTD (temperature and salinity)
• Current meter data (including ADCP)
• Wave data
• Sea level
• Biological data, etc.,
• Quality flags
• Documentation
Quality Control Standards for SeaDataNet
Data quality control has the following objective:
“To ensure the data consistency within a single
data set and within a collection of data sets and to
ensure that the quality and errors of the data are
apparent to the user who has sufficient
information to assess its suitability for a task.”
(IOC/CEC Manuals and Guides 26, 1993)
Quality control, if done well, brings about a
number of key advantages:
• Maintaining standards
• Consistency
• Reliability
Quality Control Standards for SeaDataNet
For all types of data information is required about:
• Where the data were collected: location (preferably as latitude
and longitude) and depth/height
• When the data were collected (date and time in UTC or clearly
specified local time zone)
• How the data were collected (e.g. sampling methods,
instrument types, analytical techniques)
• How the data are referenced (e.g. station numbers, cast
numbers)
• Who collected the data, including name and institution of the
data originator(s) and the principal investigator
• What has been done to the data (e.g. details of processing
and calibrations applied, algorithms used to compute derived
parameters)
• Comments for other users of the data (e.g. problems
encountered and comments on data quality)
Quality Control Standards for SeaDataNet
IODE/JCOMM Forum on Oceanographic
Data Management and Exchange Standards
• Date and time: recommended to adopt the ISO8601 standard (using extended format) where
appropriate while recognising some limitations
• Latitude, longitude, altitude: recommended to
adopt the ISO-6709 standard
• Countries: recommended to adopt the ISO-3166
(3166-1 and 3166-3) standard
Reference:
Intergovernmental Oceanographic Commission 2008 IODE/JCOMM Forum on
Oceanographic Data Management and Exchange Standards, IOC Project Office for
IODE, Oostende, Blegium 21-25 January 2008. Oostende, Belgium: IOC/IODE Project
Office, 45pp. (IOC Workshop Report No. 206) (English)
Quality Control Standards for SeaDataNet
Basic automatic checks for all data types:
• Date and time
• Latitude and longitude
• Position must not be on land
Other automatic checks:
•
•
•
•
•
•
•
•
•
Impossible speed
Spike
Gradient
Density inversion
Pressure increasing
Global range
Regional range
Deepest pressure
Check for duplicates
}
} ? QC flags ?
}
Quality Control Standards for SeaDataNet
Visual inspection of data
•
•
•
•
•
•
Pressure/depth series (e.g. CTD)
Property-property plot
Time series (e.g. current meter, sea level)
Scatter plot (e.g. current meter)
Map covering the locations of series
Ensure that data are free from instrumentgenerated spikes, gaps, spurious data at the start
and end of the record and other irregularities
• Apply quality flags
• Quality flags do not change the data
• Visual inspection can be subjective, dependent on
experience
Quality Control Standards for SeaDataNet
Outcomes of Standards Forum relevant to QC
No recommended standards identified, but:
• Temperature and salinity profiles: GTSPP to revise IOC
•
•
•
•
Manuals and Guides No. 22
Surface T&S: GOSUD to revise their QC document
Sea level: revise ESEAS QC document in close
collaboration with the GE-GLOSS
Currents: consolidate input from US IOOS with IOC
Manuals and Guides No. 26; add info on HF radar
Surface waves: update IOC Manuals and Guides No. 26;
US IOOS will provide the US national waves plan
• SeaDataNet QC flags are a sensible extension to handle
situations appropriate for a greater range of variables
• Develop “Guidelines for a Manual on Quality Control”
document
Quality Control Standards for SeaDataNet
SeaDataNet quality control flags (L201)
Flag
0
1
2
3
4
5
6
7
8
9
A
Short description
No quality control
The value appears to be correct
The value appears to be probably good
The value appears probably bad
The value appears erroneous
The value has been changed
Below detection limit
In excess of quoted value
Interpolated value
Missing value
Incomplete information
Based on IGOSS/UOT/GTSPP & Argo quality flags
Quality Control Standards for SeaDataNet
Data Documentation
• Comprehensive documentation to accompany the
data
• All data sets need to be fully documented to
ensure they can be used in the future without
ambiguity or uncertainty
• Compiled using:
• information supplied by the data originator (e.g. data
reports, comments on data quality)
• any further information gained during QC
• Includes: instrument details, mooring details, data
quality, calibration and processing carried out by
the data originator and data centre processing and
quality control
Quality Control Standards for SeaDataNet
Other data types: Discrete Water Sample Data
•
•
•
•
•
•
•
•
•
•
•
How trace values (values below the detection limit)
identified
What is the precision of the methods (e.g. number of
significant figures)
What analyses performed (use parameters
descriptions as described in the ICES green book)
What units are used
Duplicate samples taken?
Comments describing each station
Supply calibration document
Station number, site details, sample identifier (or bottle
number), type of station, continuous flow etc.,
Check profiles vs. regional climatology
Check calibration information available
Compare parameters for predictable relationships (e.g.
parameter ratios)
Quality Control Standards for SeaDataNet
Other data types: Example – nutrients (Baltic)
•
•
•
•
•
•
•
•
•
Plot nutrients against salinity
Compare how nutrients behave – look for outliers in
homogeneous water mass
Check sum NO2 + NO3 + NH4 (or phosphate) less than
or equal to total nitrogen (or phosphorus)
Check for stable relationship between phosphorus and
nitrogen
If NO2 > NO3 check carefully to see if OK
During productive season nutrients should decrease
If hydrogen sulphide present then NO2 and NO3 should
go to zero and NH4 increase
Look at (plot) dissolved oxygen and chlorophyll (this
checks for productivity)
Regional and seasonal behaviour important
Quality Control Standards for SeaDataNet
Next steps/future work
•
Version 1 of the QC manual to be completed and
available on the project web site by end April 2008
•
Extend range of parameters to include, for example:
• Surface underway data (e.g. thermosalinograph,
shipboard ADCP)
• Nutrients, dissolved oxygen (and other chemistry)
• Marine geophysics
•
Extend and improve information on biological data
quality control
•
Follow Standards Forum progress and update QC
manual as appropriate
•
Version 2 of the manual to be available April 2009