Transcript No Slide Title
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
TES GDS Instrument Team Ground Data System Reports Scott Gluck Jet Propulsion Laboratory California Institute of Technology Aura DSWG September 27, 2010
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Overview
• Current State of SIPS Data Production • Current Data Product Version • Current and Upcoming Releases • Issues and Other Items
September 27, 2010 Aura DSWG
2
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Current State of SIPS Data Production
• TES GDS software Release R11.3 (L2 ESDT Version 4) completed reprocessing January ’10 • TES GDS software version: ‘R11.3’, active at SIPS from February ’09 (R11.0 from August ‘08) – L1B ESDTs • Version 003, File version F03_04 – L2 ESDTs • Standard products: Version 004, File Version F05_07 • Summary/Ancillary products: Version 004, File Version F02_07 – L3 ESDTs • L3 products (daily, monthly grid and browse): Version 002, File Version F01_07 • Global Surveys to date: 901, 100% processed with L2 ESDT Version 4 - 6+ Years of data!
• Special Observations to date: 2479, 100% processed with L2 ESDT Version 4
September 27, 2010 Aura DSWG
3
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
SIPS R11 Reprocessing – Completed
September 27, 2010 Aura DSWG
4
• National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Data Processing Configuration
Three processing centers – SIPS: processes nominal Global Survey and Special Observations – SPC: ‘special’ processing: full filter, larger scale science investigations – SCF: supports development and small scale investigations • SIPS – 120 nodes with dual Intel Xeon quad-core CPUs with 16GB RAM and 2TB storage – 190 nodes with dual AMD Opteron single core CPUs with 6GB RAM and 500GB data storage – Total of 310 nodes with 335TB data storage • SPC – 40 nodes with dual AMD Opteron single-core CPUs with 6GB RAM and 500GB storage – Total of 20TB data storage • SCF: 40 nodes with 16 GB memory per compute node, 120 TB data storage
September 27, 2010 Aura DSWG
5
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Hardware/processing upgrades
– Additional cluster is scheduled to arrive on 9/22/10 and to be implemented in SIPS OPS in October 2010 • 60 nodes with dual Intel Nehalem Xeon quad-core CPUs with 24GB RAM and 2TB storage • Total of 120TB data storage – Noted processing timings • The new nodes have ~30% improved processing time – Data throughput bottleneck in Level 2 processing • Implementation of new file server data access methodology is being tested to alleviate PGE data bottleneck. Potential to provide significant improvement over current data staging on individual processing nodes.
September 27, 2010 Aura DSWG
6
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
SIPS Data Throughput
• • TES safe-mode event and calibration scheme modification will reduced data volume by ~40% for 2010. – In the latter half of 2009, the TES pointing control system (PCS) started showing errors and late in 2009 the errors resulted in a degradation in TES data so data collection was halted. The cause was determined to most likely be shorting in one of the telemetry lines in the flex cable that supplies power and control signals to the TES PCS. In order to minimize wear and tear of the cable an alternate calibration methodology was derived and implemented. The TES instrument resumed data collection in April 2009. – TES Level 1B software required calibration and phase alignment updates, delivered July 2010 SIPS has completed the processing of Global Survey and Special Observation data collected while software upgrades to accommodate the calibration schema change were in progress. SIPS was delivering 15-24 Global Surveys/week during this period while keeping up with forwarding processing of TES data.
• Last month SIPS delivered the following volume to ASDC under a rate of 3x; where 1x is the volume for 3.5 Global Surveys: – L1B: 86 GB L2: 26 GB L3: 1 GB
September 27, 2010 Aura DSWG
7
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Calibration changes & Impact on file naming convention
• To account for the TES modification to the on-board data collection techniques and changes to the calibration during telemetry processing there is now two output file naming conventions. The ground data software will calibrate the data with different algorithms but they will be scientifically compatible. To distinguish between the calibration methods a designator has been incorporated into the output product names and information will be written to the file metadata.
– Standard calibration scheme with embedded blackbody and space view measurements • Global surveys with Run IDs <= 11124 => TES-Aura_L2-CO2-Nadir_r
• Special Observations with Run IDs <= 11188 => TES-Aura_L2-CO2-SO-Nadir_r
– New calibration scheme with split-cal blackbody and space view measurements • Global surveys with Run IDs >= 11125 => TES-Aura_L2-CO2-Nadir_r
• Special Observations with Run IDs >= 11189 => TES-Aura_L2-CO2-SO-Nadir_r
September 27, 2010 Aura DSWG
8
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
TES Level 2 Data Version 4
• The Version 4 TES L2 standard product data includes: – Improvements to the temperature retrievals (updated CO 2 – Improvements to the methane retrievals spectroscopy) – Species dependent quality control information • Validation analyses are underway; Validation report will be provided late this year.
• A TES Global Survey quality statement has been released detailing almost no systematic differences between the new versus old calibration strategy. • Data Quality Statement and Data Users Guide provided to ASDC for Version 4 ( http://eosweb.larc.nasa.gov/PRODOCS/tes/table_tes.html
)
September 27, 2010 Aura DSWG
9
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Future State of SIPS Data Production
• TES GDS software Release R12 reprocessing scheduled, October 2010 - January 2012 • ESDT updates for Release 12 – L1B ESDTs • Version 004, File version F04_06 – L2 ESDTs • Standard products: Version 005, File Version F06_08 • Summary/Ancillary products: Version 005, File Version F03_08 – L3 ESDTs • L3 products (daily, monthly grid and browse): Version 003, File Version F01_08
September 27, 2010 Aura DSWG
10
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Release 12 and Upcoming Releases
• Release 12 product enhancements – New standard product files: Ammonia (NH 3 ), Carbon Dioxide (CO 2 ), CO 2 Lite (reduced pressure grid) – Ozone (O 3 ) tropospheric column from ancillary to all standard products – Add Instantaneous Radiative Forcing Kernel (IRFK) of outgoing longwave radiation (OLR) computation of Ozone (O 3 ) L2 standard products – Update Ocean/Ice emissivity algorithm to accurately determine whether water is frozen or not.
– Updates to Surface Temperature algorithm for cases when a scene is a freshwater target surrounded by land. – Updates to daily and monthly Level 3 plots to help differentiation based on pressure levels and color scales.
• Future Product enhancements – NetCDF compatibility – GEOS-5 update incorporation – Joint TES-MLS Carbon Monoxide (CO) and TES-OMI products (delivery dates TBD) – New standard products for Methanol (CH 3 OH) and Formic Acid (HCO 2 H) (delivery dates TBD)
September 27, 2010 Aura DSWG
11
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Accessing TES data
• ASDC Data Pool & WIST Data access; all V3 and V4 data available • Prototype ESDT Version 005 Level 2 standard products have been successfully ingested at ASDC in preparation for Release 12 processing. • Documentation & IDL data readers available from – http://eosweb.larc.nasa.gov/PRODOCS/tes/table_tes.html
• IDL readers provided early in each release, and are backward compatible • Documentation provided via ASDC site – Data Product Specification – Level 2 & Level 3 User’s Guides – Data Quality Statements – Data Versioning – Validation Reports – ATBDs
September 27, 2010 Aura DSWG
12
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
•
New look and feel using “JPL common” attributes
•
The color theme shall be based on the following criteria:
•
Atmosphere: Purple
•
Ocean: Blue
•
Land: Green
•
Other: Amber
•
Updated Science team bios
•
Access to all publications, presentations and posters generated since launch
Tropospheric Emission Spectrometer
TES Web Page Update
September 27, 2010 Aura DSWG
13
National Aeronautics and Space Administration
Jet Propulsion Laboratory California Institute of Technology
Tropospheric Emission Spectrometer
Issues & Other Items
• Naming convention change (due to calibration schema modification) could affect product reader tools • Looking for a firm schedule for GEOS-5 delivery
September 27, 2010 Aura DSWG
14