Transcript The Atmospheric Model Evaluation Tool K. Wyat Appel and Robert C.

The Atmospheric Model Evaluation Tool
K. Wyat Appel and Robert C. Gilliam
7th Annual CMAS Conference, Chapel Hill, NC
October 9, 2008
Office of Research and Development
National Exposure Research Laboratory, Atmospheric Modeling Division, Applied Modeling Research Branch
October 8, 2008
Acknowledgements
• EPA
– Steve Howard
– Rob Gilliam
– Jenise Swall
– Alice Gilliland
– Sharon Phillips
• UNC Institute for the Environment
– Alexis Zubrow
– Saravanan Arunachalam
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
What is AMET?
• Atmospheric Model Evaluation Tool
• Two modules
– Meteorology (Rob Gilliam)
– Air Quality (Wyat Appel)
• Combination of several open source software packages
– MYSQL
–R
– Perl
• AMET specifically designed to compare observations against
meteorological (e.g. MM5, WRF) and air quality model (e.g. CMAQ,
CAMx) predictions
– Does not export all gridded data to database
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
AMET-MET Evaluation Flow Chart
Analyses
Observations
•NOAA ESRL
MADIS
Surface-based
Wind Profiler
Precipitation
Upper-air
Observation-Model
Synchronization
Model Output
•MM5 (NetCDF)
•WRF (NetCDF)
Match obs. with
model values in
time and space
Generate
database records
Connect to
database and
insert records
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
Model
Evaluation
Database
MySQL server
database that stores all
model-observation
pairs in tables for
access by analysis
programs.
Model
Performance
Plot (scatter,
range,
histogram and
table)
Diurnal
Statistics
Time series
Spatial Statistics
Time-height
statistics
Text
Upper-air
statistics
Other, User-developed tools
The MySQL database is a standard, widely used and easily
connectable database that allows users to easily connect
and extract data using other software (Excel, Matlab, Perl,
SAS, etc.)
AMET-AQ Evaluation Flow Chart
Observations
STN, IMPROVE,
CASTNet, NADP,
AQS, SEARCH,
MDN
Model Output
•CMAQ (IOAPI)
•MCIP (IOAPI)
Model
Evaluation
Database
Observation-Model
Synchronization
MySQL server database
that stores all modelobservation pairs in
tables for access by
analysis programs.
Match obs. with model values in time
and space using site compare and
compare airs programs
Uses either web
interface or existing
PERL scripts to create
required MySQL tables.
•Uses Combine program
Analyses
Model
Performance
Plots
Diurnal
Statistics
Time series
Spatial
Statistics
Box Plots
Scatter Plots
Generate database records
(FORTRAN)
Populate Database (PERL)
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
Bar Plots
Other, User-developed tools
“Soccer
Goal” Plots
The MySQL database is a standard,
widely used and easily connectable
database that allows users to easily
connect and extract data using
other software (Excel, Matlab, Perl,
SAS, etc.)
Bugle Plots
Advantages of the AMET System
• Somewhat automated/interactive system
• Data stored in relational database
– Data from multiple simulations stored in a single location
– Allows data queries based on many factors
• Time period, geographic location, time of day, etc.
• Pre-generated analysis scripts
– Same analysis for multiple simulations
– Common analyses between different groups
• Open Source
– Easy to create new scripts (if you know R)
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
Available AMET-MET Analyses
• Model Performance Summary
•
•
•
•
•
•
– includes various plots (scatter, box, etc.) along with various
statistics
Timeseries
Spatial Plots
– various statistical values (e.g. NMB, NME, etc.)
Bar Plots
– error, bias, etc.
Rawindsonde
Wind Profiler
Aircraft Profiler
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
Model Performance Summary
Temperature
Wind Direction
Time Series
2-m Mixing Ratio
2-m Temperature
2-m Wind Speed
2-m Wind Dir.
Spatial Statistics
Wind Profiler – Model Comparisons
Aircraft Profile Comparisons (Potential Temp)
Mean Absolute Error
Distribution by level
Aircraft
Mean
WRF
Mean
Available AMET-AQ Analyses
• Scatter Plots
•
•
•
•
•
•
– model to observation
– model to model (at observation points)
Summary Statistics (as csv text file)
Spatial Plots
– various statistical values (e.g. NMB, NME, etc.)
– concentrations (predicted, observed, and difference)
Box Plots
Stacked Bar Plot
Bugle Plot
Soccer Goal Plot
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
Scatter plots
• Model vs. Obs
• Model vs. Model
• Multiple Networks
– select statistics
• Single Network
– additional statistics
• Ozone Specific
• Temporal Averaging
– monthly, seasonal, annual
Spatial Plots
• Statistics
– NMB, NME, Correlation,
etc.
• Concentrations
– model, observation,
difference
• Sub-regions
Time Series Plots
Box Plots
Diurnal Box Plot
Monthly Box Plot
Stacked Bar Plots
Other Plots
“Bugle” Plot
“Soccer Goal” Plot
Public Availability of AMET
• AMET is available for download on the CMAS website
•
•
•
•
– Both Met and AQ versions available
– Script based version
– Extensive users guide included
Contains most of the functionality shown here
Met and AQ versions can be installed together or individually
Includes tutorial data and example output plots
Bugzilla available for AMET
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory
Future Improvements to AMET
• Java interface
– Currently under development
– Runs AMET locally and accesses remote database
– Interactive tabs
– User friendly
– Portable (to a degree)
• Additional analysis scripts
– Developed internally
– Developed externally (user community)
• More query options
Office of Research and Development
Atmospheric Modeling Division, National Exposure Research Laboratory