Transcript Time Series Product Tool
Stennis Space Center
Time Series Product Tool
Don Prados Computer Sciences Corp.
John C. Stennis Space Center USDA Forest Service, Asheville, NC September 6-7, 2007
TSPT (Time Series Product Tool)
Stennis Space Center
• Temporally processes various MODIS (Moderate Resolution Imaging Spectroradiometer) products on a per-pixel basis – MOD02 daily swath radiance and planetary reflectance – MOD09 daily gridded surface reflectance – MOD43 8-day and 16-day gridded BRDF (Bidirectional Reflectance Distribution Function)-adjusted reflectance – MOD13 16-day gridded NDVI (Normalized Difference Vegetation Index) and EVI (Enhanced Vegetation Index) • Calculates NDVI or NDMI (Normalized Difference Moisture Index) from radiance or reflectance • Uses cloud and other metadata to remove bad or low-quality pixels • Fuses images from the Aqua and Terra satellites based on comparative pixel quality National Aeronautics and Space Administration Time Series Product Tool 2
TSPT (continued)
Stennis Space Center
• Uses an outlier detection algorithm to remove suspect pixels • Applies temporal interpolation and smoothing to further reduce noise • Displays NDVI and NDMI time series plots for user-selected pixels at various stages of processing • Recreates images from temporally processed pixels at various stages of processing • Generates an ENVI ® header file for each image containing the map projection and other metadata – Allows images to be displayed in ENVI or ERDAS IMAGINE ® – Tracks processing parameters and settings • Stacks images into 3D ENVI BSQ (Band Sequential) files National Aeronautics and Space Administration Time Series Product Tool 3
TSPT Input
Stennis Space Center
• The TSPT can process output from the MRT (MODIS Reprojection Tool) – Developed software front end for MRT – Reprojects MODIS or simulated VIIRS (Visible/Infrared Imager/Radiometer Suite) HDF (Hierarchical Data Format) files to UTM, LCC, or LAEA • The TSPT also can directly reproject MODIS HDF swath or gridded data into UTM or other map projections and can process the output • The TSPT can convert MODIS HDF sinusoid gridded data to ENVI BSQ sinusoid and can process the sinusoid data • Supports: – Radiance, planetary reflectance, surface reflectance, and BRDF adjusted reflectance – MOD13 NDVI and EVI – MODIS and simulated VIIRS National Aeronautics and Space Administration Time Series Product Tool 4
TSPT State Diagram
Stennis Space Center
Reflectance or Radiance Calc NDVI or NDMI NDVI, EVI, or NDMI Remove Clouds Clear Remove Outliers Filter Ideal Fuse Fuse Filtered Fuse Filtered Fused Fused Remove Outliers Fused Ideal Filter Fused Filtered National Aeronautics and Space Administration Time Series Product Tool 5
TSPT Modules
Stennis Space Center
• TSPT_Reproject: reprojects HDF gridded (sinusoid) or swath (no map projection) data to various map projections • TSPT_GridToBSQ: converts HDF sinusoid to BSQ sinusoid • TSPT_CalcNDVI: calculates NDVI and NDMI • TSPT_Ideal: removes cloudy and bad pixel values • TSPT_Outlier: removes time series outliers • TSPT_Fuse: combines data from the Aqua and Terra satellites • TSPT_Time: interpolates and filters time series • TSPT_Plot: plots time series for selected pixels • TSPT_Stack: stacks images into 3D BSQ files • TSPT_DisplayBSQFiles: displays BSQ images National Aeronautics and Space Administration Time Series Product Tool 6
Stennis Space Center
MOD02 Daily NDVI Cloud Removal
MOD02 Terra NDVI (left) and Aqua NDVI (right) before and after cloud removal. Original NDVI in blue, clear NDVI in black. Clouds typically result in a drop in NDVI.
National Aeronautics and Space Administration Time Series Product Tool 7
Stennis Space Center
MOD02 Fusing of Clear Aqua NDVI and Clear Terra NDVI
Cloudy: 26.2% Mixed: 19.5% Cloudy: 37.2% Mixed: 9.0% Cloudy: 10.0% Mixed: 4.3% MOD02 Terra NDVI, clouds removed MOD02 Aqua NDVI, clouds removed
National Aeronautics and Space Administration
Terra NDVI and Aqua NDVI fused using cloud metadata and sensor zenith angle
Time Series Product Tool 8
Stennis Space Center
MOD02 Outlier Removal
Fused MOD02 Aqua and Terra NDVI before (blue) and after (black) outlier removal
National Aeronautics and Space Administration Time Series Product Tool 9
Stennis Space Center
MOD02 Temporal Interpolation and Filtering
Temporal interpolation applied to replace MOD02 NDVI values eliminated in previous steps, followed by Savitzky-Golay filtering to further reduce noise
National Aeronautics and Space Administration Time Series Product Tool 10
Stennis Space Center
MOD43A Outlier Removal
MOD43A 8-day BRDF-adjusted NDVI before (blue) and after (black) outlier removal
National Aeronautics and Space Administration Time Series Product Tool 11
Stennis Space Center
MOD43A Temporal Interpolation and Filtering
Temporal interpolation applied to replace MOD43A values eliminated in previous steps, followed by Savitzky-Golay filtering to further reduce noise
National Aeronautics and Space Administration Time Series Product Tool 12
Stennis Space Center
MOD13 Cloud Removal
MOD13 16-day Terra NDVI before (blue) and after (black) cloud removal
National Aeronautics and Space Administration Time Series Product Tool 13
Stennis Space Center
MOD13 Outlier Removal
MOD13 16-day NDVI before (blue) and after (black) outlier removal
National Aeronautics and Space Administration Time Series Product Tool 14
Stennis Space Center
MOD13 Temporal Interpolation and Filtering
Temporal interpolation applied to replace MOD13 NDVI values eliminated in previous steps, followed by Savitzky-Golay filtering to further reduce noise
National Aeronautics and Space Administration Time Series Product Tool 15
Next Steps
Stennis Space Center
• Investigate alternative methods for fusing Aqua and Terra data, such as choosing the pixel with the highest value • Investigate alternative outlier removal techniques to reduce chance of removing good pixels • Investigate/Compare temporal filtering methods • Provide guidelines for selecting parameters for outlier removal and filtering techniques • Support GeoTIFF (Geo-referenced Tagged Image File Format) format for TSPT input and output National Aeronautics and Space Administration Time Series Product Tool 16
Participation in this work by Science Systems and Applications, Inc., and by Computer Sciences Corporation was supported by NASA at the John C. Stennis Space Center, Mississippi, under Task Order NNS04AB54T.
Stennis Space Center
Backup Slides
TSPT_Reproject
Stennis Space Center
• Alternative to MRT and MRTSwath • Reprojects MODIS or VIIRS HDF to UTM, LCC, or LAEA BSQ • Creates corresponding ENVI HDR (header) file containing reprojection information • BSQ/HDR files can be read by ENVI and by IMAGINE • Copies metadata from HDF files to header file, modifying metadata where appropriate • Adds new metadata to header file National Aeronautics and Space Administration Time Series Product Tool 19
TSPT_GridToBSQ
Stennis Space Center
• Converts MODIS sinusoid HDF to sinusoid BSQ • Creates corresponding ENVI header file containing sinusoid reprojection information • BSQ/HDR files can be read by ENVI and by IMAGINE • Copies metadata from HDF files to header file, modifying metadata where appropriate • Adds new metadata to header file National Aeronautics and Space Administration Time Series Product Tool 20
TSPT_CalcNDVI
Stennis Space Center
• Calculates NDVI from reflectance or radiance • Optionally adjusts NDVI when reflectances are below a selected threshold: – Reduces sensitivity of NDVI to small changes in reflectance when reflectances are very small – Reduces upward spikes in NDVI due to noise – Prevents NDVI values from reaching or exceeding 1 or -1 • For MOD02 datasets, creates an 8-bit Cloud Mask from the 48-bit MOD35 Cloud Mask • Mosaics NDVI, Cloud Mask, and Sensor Zenith Angle into daily products if day contains more than one dataset National Aeronautics and Space Administration Time Series Product Tool 21
TSPT_Ideal
Stennis Space Center
• Eliminates cloudy pixels using different metadata for different products – MOD35 Cloud Mask for MOD02 planetary reflectances – MOD09 quality metadata for MOD09 surface reflectances – MOD13 quality metadata for MOD13 NDVI and EVI – MOD43B4 quality metadata for MOD43B4 nadir reflectances – MOD43A2 quality metadata for MOD43A4 nadir reflectances • Eliminates pixels flagged as bad and/or set to fill value using different metadata for different products National Aeronautics and Space Administration Time Series Product Tool 22
TSPT_Outlier
Stennis Space Center
• Eliminates outliers – Eliminates downward spikes (large negative slope followed by large positive slope) – Eliminates upward spikes (optional) – User-selectable threshold is based on the angle change calculated from the change in slope – Default of 10 passes, reducing threshold on each pass • Eliminates pixels acquired at high sensor zenith angles using selected maximum sensor zenith angle National Aeronautics and Space Administration Time Series Product Tool 23
TSPT_Fuse
Stennis Space Center
• Fuses data acquired on the same day from the Aqua and Terra satellites – For each pixel, chooses Aqua or Terra NDVI based on cloud information, sensor zenith angle, and other metadata • If Aqua is chosen, uses Aqua metadata and sensor zenith angle for that pixel • If Terra is chosen, uses Terra metadata and sensor zenith angle for that pixel – Can be called after TSPT_Ideal, TSPT_Outlier, or TSPT_Time modules using the current state of TSPT to determine inputs and outputs National Aeronautics and Space Administration Time Series Product Tool 24
TSPT_Time
Stennis Space Center
• Applies temporal interpolation to replace cloudy, bad, or suspect values (optional) • Applies one or more of the following filters to the time series: – Weighted Savitzky-Golay algorithm • weights based on pixel quality and temporal distances • parameters: polynomial order, temporal frame size, full width half maximum for temporal-distance Gaussian function – weighted-average algorithm using the same weights as above – conventional Savitzky-Golay algorithm – median filter • Applies extrapolation to replace cloudy, bad, or suspect pixels at beginning and end of time series (optional) National Aeronautics and Space Administration Time Series Product Tool 25
TSPT_Plot
Stennis Space Center
• Plots time series given pixel locations and a date range • Pixel locations provided as lists of latitudes and longitudes or in a shapefile • Can be called after TSPT_NDVI, TSPT_Ideal, TSPT_Outlier, TSPT_Fuse, or TSPT_Time modules using current state of TSPT to determine what to plot • Optionally, plots pixel quality along with NDVI National Aeronautics and Space Administration Time Series Product Tool 26
TSPT_Stack
Stennis Space Center
• Replaces null values with user-selected value • Writes output to user-selected output directory • Optional: – Creates a binary cloud mask and writes it to the output directory – Stacks daily images into a single multiband BSQ file • one date per band • creates ENVI header for multiband BSQ file – Stacks binary cloud masks into a single multiband BSQ file • allows easy calculation of spatial, temporal, or spatio-temporal cloud statistics National Aeronautics and Space Administration Time Series Product Tool 27
TSPT_DisplayBSQFiles
Stennis Space Center
• Displays BSQ images in MATLAB ® figures using supplied date range • Can be called after TSPT_NDVI, TSPT_Ideal, TSPT_Outlier, TSPT_Fuse, or TSPT_Time modules using current state of TSPT to determine which files to display • User-selectable color map • Optional: – Shows states, roads, other shapefiles – Saves images as JPEG or TIFF with worldfile – Converts null values to user-selected alternative – Converts snow and water values to user-selected alternative National Aeronautics and Space Administration Time Series Product Tool 28
Stennis Space Center
ART Simulation and Reprojection
• Several features from the ART (Application Research Toolbox) are used to prepare data for input into the TSPT • The ART is used to – reproject gridded MODIS HDF (Hierarchical Data Format) datasets to various map projections using the MRT (MODIS Reprojection Tool) – reproject swath MODIS HDF datasets to various map projections using the MRTSwath (MODIS Swath Reprojection Tool) – simulate VIIRS (Visible/Infrared Imager/Radiometer Suite) reflectance products from MODIS reflectance products before or after reprojection National Aeronautics and Space Administration Time Series Product Tool 29
MRT_Command
Stennis Space Center
• Front end to the MRT and to the MRTSwath • Reprojects MODIS or VIIRS HDF to binary UTM, LCC, or LAEA • Creates corresponding ENVI HDR file containing reprojection information • BSQ/HDR files can be read both by ENVI and by IMAGINE • Copies metadata from HDF files to HDR file, modifying metadata where appropriate • Adds new metadata to HDR file • Output ready for input into TSPT National Aeronautics and Space Administration Time Series Product Tool 30
Application Research Toolbox
Stennis Space Center
• The Application Research Toolbox is an integrated set of algorithms and models developed in MATLAB ® that allows users to generate and validate VIIRS and other National Polar-Orbiting Environmental Satellite System Preparatory Project (NPP) systems as well as “Landsat Data Continuity Mission like” systems • The ART provides a simulation solution by using data acquired by existing sensors with higher spatial and spectral resolution to create multispectral images with predefined characteristics • Predefined characteristics include spatial resolution, spectral bands, signal-to-noise ratio, data quantization, and band-to-band registration National Aeronautics and Space Administration Time Series Product Tool 31
ART Simulation/ Reprojection State Diagram
Stennis Space Center
MODIS Swath HDF Sim VIIRS Swath VIIRS Swath HDF Reproject Reproject MODIS Grid HDF Reproject HDF to BSQ MODIS BSQ TSPT Sim VIIRS Grid VIIRS BSQ TSPT Sim VIIRS Grid MODIS BSQ MODIS Sinusoid BSQ TSPT TSPT National Aeronautics and Space Administration Time Series Product Tool 32
Stennis Space Center
ART Modules used in Conjunction with the TSPT
• ART_SimVIIRS_Swath – simulates VIIRS imagery from MOD02 planetary reflectance and from MOD03 sensor zenith angle products, leaving the data in the swath format (no map projection) • ART_SimVIIRS_Grid – simulates VIIRS imagery from MODIS gridded data, such as the output of MRT_Command (below) • MRT_Command – is used as an automated front end to the MRT and to the MRTSwath – supports the following map projections: • UTM (Universal Transverse Mercator) • LCC (Lambert Conformal Conic) • LAEA (Lambert Azimuthal Equal Area) – Input is HDF; output is BSQ format, ready for input into the TSPT or into ART_SimVIIRS_Grid National Aeronautics and Space Administration Time Series Product Tool 33
ART_SimVIIRS_Swath
Stennis Space Center
• MODIS and VIIRS PSFs (Point Spread Functions) are used to simulate VIIRS from MODIS MOD02 swath data • MOD02 reflectance/radiance values in HDF files are replaced by VIIRS values where the VIIRS PSF is larger than the MODIS PSF; otherwise values are not changed • The location of the swath pixels is not changed • The PSFs vary with sensor zenith angle, becoming progressively larger moving away from nadir • MODIS MOD03 sensor zenith angles are replaced by VIIRS sensor zenith angles, accounting for different satellite altitudes • Data remains in HDF files National Aeronautics and Space Administration Time Series Product Tool 34
ART_SimVIIRS_Grid
Stennis Space Center
• Simulates VIIRS UTM BSQ from MODIS UTM BSQ • Updates the ENVI HDR (header) file created by MRT_Command • BSQ/HDR files can be read by both ENVI and IMAGINE • Output is ready for input into the TSPT <