Transcript No Slide Title

GODAE Quality Control Pilot Project
Status Report August 2007
IGST Meeting
Department Fisheries Ocean
St. John’s, New Foundland, Canada
1.
2.
3.
4.
Outline
New Data Sources
Output File and Reference Data
Plans and Issues
Argo FSI Float QC Outcomes
GODAE Quality Control Pilot Project
New Data Sources
• UKMO
– long list form of UKMO QC output now available in real-time
– UKMO back filled long list data to 2003 – thanks to Matt Martin
– long list provides additional QC outcomes and both in situ and
potential temperatures (short list had only potential temperatures)
– convention is to use in situ temperature in QC databases
• in situ calculated from potential when salinity reported
• not always the case in the UKMO short list files
• BMRC
– pushing QC data from BoM system to GODAE server
– currently have data for 2005 and 2006 (nothing from 2007 yet)
– working on BMRC data decoder
GODAE Quality Control Pilot Project
New Output File Format and Reference Data
• Call sign database conversion to NetCDF format complete
– based on Coriolis/Argo multiple profile NetCDF file structure
– easily expanded with new QC variables or new QC data providers
– sorted by observation time and data center for ease of use
• Navy GDEM climate and climate variability profiles added as
reference profiles in call sign database
–
–
–
–
temperature and salinity climatology
global, monthly, 0.25 degree resolution
78 vertical levels (47 in upper 1000 m of water column)
climate mean and standard deviations interpolated to observed
levels for month in which observation taken
GODAE Quality Control Pilot Project
New MEDS QC Variable in Call Sign Database
• decoded MEDS hex bit codes on QC tests performed
Meds
0 3 6 7 8 11 14 –
17 20 –
23 –
QC Tests (binary string bit position from the RIGHT):
Platform Identification 1 - Impossible Date/Time
2
Position on Land
4 - Impossible Speed
5
Global Impossible Parameter Values
Regional Impossible Parameter Values
Increasing Depth
9 - Profile Envelop
10
Freezing Point
12 – Spike
13
Gradient
15 - Density Inversion
16
Emery and Dewar Clim
18 - Asheville Clim
19
Waterfall
21 - Cruise Track
22
Bottom
24 - Temperature Inversion
Bit # - Test
- Impossible Location
- Impossible Sounding
–
Constant Profile
Top and Bottom Spike
Levitus Seasonal Clim
Levitus Monthly Clim
Profiles
• experience indicates only two tests flagged as typically performed
– Levitus seasonal climate check and temperature inversion
GODAE Quality Control Pilot Project
Plans
• initialize call sign databases with 2003-2006 production run
– NetCDF call sign databases will be hosted on US GODAE server
– target date for completion Oct 2007
Issues
• US GODAE server hardware upgrade has never taken place
– IBM X-346 machines plus 30 TB storage purchased years ago by ONR
– never installed due to security issues – completion date still unknown
• GODAE QC application cannot run on existing server hardware
– cannot do real-time updates of call sign databases as planned
– in process of purchasing additional disk space for personal Linux
workstation to support GODAE QC function
– using my Linux workstation (at best) only monthly updates of NetCDF
call sign databases on GODAE server possible
GODAE Quality Control Pilot Project
FSI Argo Floats Pressure Offset Errors
• discovered in February, 2007 that Argo profiles from SOLO floats
with FSI CTD (Argo Program WHOI) have incorrect pressures
– profile data may be offset upward by one or more pressure levels,
resulting in a significant cold bias for the data
– a data denial list was posted on the Argo web site (287 WMO IDs)
– data no longer distributed on GTS
• Argo science team asked if bad FSI float data had been accepted
into operational assimilation systems
• FSI float QC outcomes from the NRL/FNMOC real-time QC system
were evaluated from 2004 through 2006
– instrumentation error checks
– climate background field check
– cross validation (buddy) check
float track
error probability
of entire profile
(reject if p > 0.95)
GDEM climate
anomaly
cross validation
anomaly
error probability
of profile level
(reject if p > 0.95)
NRL FSI Float QC Outcomes Display
GODAE Quality Control Pilot Project
Summary NRL/FNMOC QC of
FSI Argo Float Pressure Errors
• rejected 11% of the float temperature profiles
• rejected 5% of the float temperature levels
• 185 float profiles rejected due to inadequate vertical sampling as a
result of the vertical level rejections (created gaps in profiles)
Positive feedback from Howard Freeland on call sign databases
•
From: Freeland, Howard [mailto:[email protected]]
Sent: Wednesday, August 01, 2007 2:58 PM
To: Cummings, James [[email protected]]
Subject: Rejected floats
I think the call sign files are exactly what I'd like to look at. I've been doing some simple
investigations about data being rejected by less sophisticated users than GODAE people
and have some interesting results...
END
Call Sign Database Structure
dac
source of data
(FNMOC, NAVO, MEDS,
Coriolis, UKMO, BMRC)
sal_qc_lvl
salinity level quality
sal_flg
salinity QC test failures
dtg
observation time
tmp
observed temperature
dtg_qc
quality observation time
tmp_fcst
temperature forecast
lat, lon
observation position
t_fcst_err
temp forecast error
pos_qc
quality observation position
tmp_clm
temperature climate
n_lvl
number levels
t_clm_err
temp climate variability
lvl
observation depth levels
tmp_qc
temperature profile quality
sal
observed salinity
tmp_qc_lvl
temperature level quality
sal_fcst
salinity forecast
tmp_flg
temperature QC test failures
s_fcst_err
salinity forecast error
sign
observation call sign
sal_clm
salinity climatology
upd_dtg
profile update time at center
s_clm_err
salinity climate variability
qc_conflict
sal_qc
salinity profile quality
flag indicating QC outcomes
differ among data centers
NRL
MEDS
UKMO
Coriolis
BMRC
GODAE QC Application Program Interface (API)
read different QC data formats, check for valid call sign, match profiles (lat, lon, time,
call sign), populate database structure with observation, QC codes, QC flags, and
supplemental information, sort by observation time, update NetCDF call sign database
1900305
A8ES9
52007
Argo Float
Ship XBT
TAO Mooring
Call Sign Databases