Transcript Model Evaluation - University of Toledo

Understanding the USEPA’s
AERMOD Modeling System for
Environmental Managers
Model Evaluation
Ashok Kumar
University of Toledo
[email protected]
Air Pollution Research Group
Evaluation Studies on AERMOD



USEPA
Evaluation using field studies
No evaluation using ambient air monitoring network in an urban
area
Air Pollution Research Group
Lucas County: Sources and Monitoring
Stations
NORTHING (UTM) KM
SOURCE AND MONITOR LOCATIONS FOR SO2
4620
4618
4616
4614
4612
4610
4608
286
288
290
292
294
296
298
EASTING (UTM) KM
MAIN SOURCES
SUPERSTACKS
MONITORING STATIONS
Air Pollution Research Group
Input Data Flow in AERMOD
AERM OD STRUCTURE
D E M F ile
S u rfa ce
C lim a te
D a ta
AERM AP
P re p ro c e s so r
S u rfa ce
F ile
U p p e r A ir
D a ta
AERMET
P re p ro ce ss o r
AERMOD
P ro file
F ile
O n -S ite
D a ta
O u tp u t
Air Pollution Research Group
Data Requirements for Model Evaluation
Emission Inventory
 Properties of stacks and super stacks
 Meteorological data
 Receptor data
 Air monitoring data

Air Pollution Research Group
AERMET - Input
1.
Meteorological Input Parameters – Multi-Level WS, WD, and
Temperature, Opaque Cloud Cover, Ceiling Height, RH,
Pressure, Surface Heat Flux, Friction Velocity, and Roughness
Length, Delta-T , Solar Radiation, Upper Air Data
2.
Data Formats -
CD144, SCRAM, SAMPSON (surface data)
1.
TD 3280 (surface data )
2.
TD6201 (upper air data)
3.
On-site (site specific data)
Air Pollution Research Group
AERMET - Output
Boundary Layer File

sensible heat flux

surface friction velocity

convective velocity scale

potential temp. gradient above mixing height

convectively-driven mixing height

mechanically-driven mixing height

Monin-Obukhov length

surface roughness length
 Bowen ratio

albedo
Profile File

Measurement height

WD, WS

Temperature

Standard Dev. of Lateral WD
 Standard Dev. of Vertical WS
Air Pollution Research Group
Atmospheric Stability



AERMOD uses Monin-Obukhov length as the stability parameter
You will need friction velocity u*and the flux of sensible heat H to
compute L
L is defined to be negative in convective conditions and positive
in stable
Air Pollution Research Group
AERMAP

Input data needs for AERMAP:
 DEM formatted terrain data
 User provided receptors and terrain
 Design of receptor grid: AERMAP accepts either polar,
cartesian or discrete receptors
Air Pollution Research Group
Pathways Used in AERMOD Input Runstream





Control
Source
Receptor
Meteorology
Output
Air Pollution Research Group
Statistical Evaluation Methods

Fractional Bias (FB)

Normalized Mean Square Error (NMSE)
Air Pollution Research Group
Statistical Evaluation Methods

Coefficient of Correlation (COR)
r 

( C o  C o )( C o  C

co

p
)
cp
Factor of Two (Fa2)
Fraction of data for which 0.5<Cp/Co<2
Air Pollution Research Group
Statistical Evaluation Methods


Confidence Limits
Confidence limits are the lower and upper boundaries / values of a
confidence interval, that is, the values which define the range of a
confidence interval. The upper and lower bounds of a 95%
confidence interval are the 95% confidence limits.
Q-Q Plots
The quantile-quantile (Q-Q) plot is a graphical technique for
determining if two data sets come from populations with a common
distribution.
Air Pollution Research Group
Results & Discussion
Model is evaluated in the following ways:
 Performance measures and confidence limits for
3-hr average for SO2



Stable Condition
Convective Condition
24-hr average for SO2
 Plots of NMSE vs.FB
 Q-Q plots for 3-hr average for S02
 Q-Q plots for 24-hr average for S02
Air Pollution Research Group
NMSE vs FB plots (3-hr average)
Plot of NMSE VS. FB (Main Street Station - Stable
Case)
2.5
NMSE
2
1.5
1
0.5
0
-1.5
-1
-0.5
0
0.5
1
FB
NMSE CURVE
NMSE VERSUS FB
Air Pollution Research Group
1.5
NMSE vs. FB plots(3-hr average)
Plot of NMSE VS. FB (Main Street Station Convective Case)
4
NMSE
3
2
1
0
-1.5
-1
-0.5
0
0.5
1
FB
NMSE CURVE
NMSE VERSUS FB
Air Pollution Research Group
1.5
NMSE vs. FB plots (3-hr average)
Plot of NMSE VS. FB (Collins Park Station - Stable
Case)
NMSE
6
5
4
3
2
1
0
-2
-1.5
-1
-0.5
0
0.5
1
1.5
FB
NMSE CURVE
NMSE VERSUS FB
Air Pollution Research Group
2
NMSE vs. FB plots (3-hr average)
Plot of NMSE VS. FB (Collins Park Station Convective Case)
4
NMSE
3
2
1
0
-1.5
-1
-0.5
0
0.5
1
FB
NMSE CURVE
NMSE VERSUS FB
Air Pollution Research Group
1.5
NMSE vs. FB plots (24-hr average)
Plot of NMSE VS. FB
2.5
NMSE
2
1.5
1
0.5
0
-1.5
-1
-0.5
0
0.5
1
1.5
FB
NMSE CURVE
NMSE VERSUS FB (Main Street)
NMSE VERSUS FB (Collins Park)
Air Pollution Research Group
Q-Q plots (3-hr average)
70
70
Q -Q p lo t fo r 3 -h r a v e ra g e fo r M a in S tre e t S ta
tio n 80
S ta b le C a s e
90
90
100
150
(μ g /m 3)
C o n cen trat io n s
P red icted
100
80
100
50
0
0.00
20.00
40.00
60.00
80.00
100.00
120.00
O b se rve d C o n ce n tra tio n s (μ g /m 3)
Q -Q plot
R eferenc e line
Air Pollution Research Group
Q-Q plots (3-hr average)
Predicted
Concentrations
(μg/m3)
Q-Q plot for 3-hr average for Main Street Station Convective Case
150
100
50
0
0.00
20.00
40.00
60.00
80.00
100.00
120.00
Observed Concentrations (μg/m3)
Q-Q plot
Reference line
Air Pollution Research Group
140.00
Q-Q plots (3-hr average)
Predicted
Concentrations
(μg/m3)
Q-Q plot for 3-hr average for Collins Park Station Stable Case
80
60
40
20
0
0
10
20
30
40
50
60
Observed Concentrations (μg/m3)
Q-Q plot
Reference line
Air Pollution Research Group
70
Q-Q plots (3-hr average)
Predicted
Concentrations
(μg/m3)
Q-Q plots for 3-hr average for Collins Park Station Convective Case
80
60
40
20
0
0
20
40
60
Observed Concentrations (μg/m3)
Q-Q plot
Reference line
Air Pollution Research Group
80
Q-Q plots (3-hr average)
Observed Concentrations < 20µg/m3
Q-Q plots for 3-hr average for Main Street Station - Stable Case
25
Predicted Concentrations(µg/m3)

20
15
10
5
0
0.00
5.00
10.00
15.00
20.00
25.00
Observed Concentrations (µg/m3)
Air Pollution Research Group
Q-Q plots (3-hr average)
Observed Concentrations < 20µg/m3
Q-Q plots for 3-hr average for Main Street Station - Convective
Case
25
Predicted Concentrations (µg/m3)

20
15
10
5
0
0.00
5.00
10.00
15.00
20.00
25.00
Observed Concentrations (µg/m3)
Air Pollution Research Group
Q-Q plots (3-hr average)
Observed Concentrations < 20µg/m3
Q-Q plots for 3-hr average for Collins Park - Stable Case
25
Predicted Concentrations (µg/m3)

20
15
10
5
0
0
5
10
15
20
25
Observed Concentrations (µg/m3)
Air Pollution Research Group
Q-Q plots (3-hr average)
Observed Concentrations < 20µg/m3
Q-Q plots for 3-hr average for Collins Park - Convective case
25
Predicted concentrations (µg/m3)

20
15
10
5
0
0
5
10
15
20
25
observed concentrations (µg/m3)
Air Pollution Research Group
Q-Q plots (24-hr average)
Predicted Concentration
(µg/m3)
Q-Q plot for 24-hr average for Main Street Station
70
60
50
40
30
20
10
0
0
10
20
30
40
50
60
Observed Concentration (µg/m3)
Q-Q plot
Reference line
Air Pollution Research Group
70
Q-Q plots (24-hr average)
Predicted Concentration
(µg/m3)
Q-Q plot for 24-hr average for Collins Park Station
70
60
50
40
30
20
10
0
0
10
20
30
40
50
60
Observed Concentration (µg/m3)
Q-Q plot
Reference line
Air Pollution Research Group
70
Confidence Limits (3-hr average)


The values of NMSE and FB were significantly
different from zero in the stable case. COR was not
significantly different from zero.
The values of NMSE, FB, and COR were not
significantly different from zero for the convective
case.
Air Pollution Research Group
Confidence Limits (24-hr average)


The values of NMSE and FB were significantly
different from zero. COR, was not significantly
different from zero.
Note: 24-hr data were not divided according to
stability classes.
Air Pollution Research Group