Transcript Document

Accuracy - Relationship to the goal or truth?
High accuracy is represented by small differences between a
outcome or measurement and a know quantity.
Precision - Relationship between individual outcomes or measurements
High precision is represented by small differences between
a set of outcomes or measurements.
Error – measurement based on the known “truth”
Observed – Predicted
Uncertainty – measurement of confidence of the results
Probabilistic approach can be used to quantify and model
uncertainty in spatial data and models.
Potential Sources of Error/Uncertainty in GIS Analysis
1.
Age of Data – Does the factor change rapidly?
NOAA Coastal Services Center
http://www.csc.noaa.gov/
Potential Sources of Error/Uncertainty in GIS Analysis
2. Positional Accuracy
There will always be some error – What can you live with?
Engineering Level Survey - +/- centimeters
GPS without correction - +/- 10 -20 meters (on a good day)
USGS DOQQ’s/7.5 Quad - +/- 30 meters
Data Entry Error/Original Map Error – Hard to determine
PLSS – Area to Point Conversion
3. Resolution – Is the level of detail sufficient for the task?
Original Resolution of Data (i.e. old maps)
Minimum Mapping Resolution
Pixel/Grid Size
“Wrong” Spatial Data Model (Ex. soils – polygon)
Level of Precision of Attributes
Organ Pipe NM
Change in landscape characteristics
Number of types
No. of patches
Mean patch size (ha)
10x10
19
158
106.63
100x100
19
247
68.45
1000
x1000
11
28
625.00
Resolution
Potential Sources of Error in GIS Analysis
4. Misrepresentation generated in the database development process.
Classification
Level of Detail
Number of Classes
Aggregation and Disaggregation Schemes
Generalization
Feature Elimination (Selection)
Simplification (i.e. smoothing)
Change of Dimensionality (point vs. polygon)
Reduction in Level of Precision
Exaggeration or Enlargement (Beware of Old Maps!)
Projections and Coordinate Systems
Generalization: Intentional Error
Generalization is the process that purposely reduces the accuracy of
a map for cartographic or data development reasons. Common in
paper maps where generalization was used to show features in a
small scale map. This error is reproduced if the map is digitized.
Generalization
Simplification
Selection
Classification
Change in Dimension
Potential Sources of Error in GIS Analysis
4. Accuracy of Attributes
Classification/Interpretation (Remote Sensing)
Encoding Error (Typos)
Measurement Errors
Natural Variability (Is it adequately described?)
5. Algorithms
All algorithms make assumptions that can lead to error.
Algorithms can be applied using different parameters.
Example: Different interpolation methods create different
surfaces.
Kernel Density Mapping – Same Data, Different Parameters
Change in S.D. (#/ha)
Standard Deviation
250
200
150
100
S.D.
Patchy
Smooth
50
0
4 6 8 10 12 14 16 18 20 22 24 26 28 30 40 50
10m
15m
Search Distance
20m
40m
Potential Sources of Error in GIS Analysis
6. Models
Different Methods/Models can yield different results.
What is the “best” method may be the function of
the problem scale and the type of input data available.
The process is important! Document What You Do!!!
7. Human Error
Quality control is essential in order to get good data.
Assessing Uncertainty
Where assessing uncertainty is important:
• Resource Allocation – Evaluating “what if” scenarios
• Quantitative Risk Analysis – Probability of an event
occurring and the likely consequences
• Error Propagation – An inherent problem with spatial
data – cascading error
• Decision-making – “Input data and models are always
imperfect, and geoprocessing functions produce
estimates, not truth.” Assessment of sensitivity to
unknown and estimated parameters and the
resulting uncertainty is required to estimate
potential risks
Error Propagation
• All spatial data contains misrepresentations
and inaccuracies.
• When different thematic layers are combined
you would expand your level of uncertainty.
– Cascading error – it can be nonlinear
Error Analysis
• The effects of cascading on error will be complex:
– do errors get worse, i.e. multiply?
– do errors cancel out?
– are errors in each layer independent or are they related?
• Suppose two maps, each with percent correctly
classified of 0.90 are overlaid
– studies have shown that the accuracy of the resulting
map (percent of points having both of the overlaid
classes) is little better than 0.90x0.90=0.81
– when many maps are overlaid the accuracy of the
resulting composite can be very poor
– e.g. 4 maps: 0.90x0.90x0.90x0.90 = 0.66
Error Analysis
• How important is the variable:
– Linear Combination Method
• High weight will amplify the error
– Is the variable used often in an analysis
• DEM can be used to compute:
–
–
–
–
–
Slope
Aspect
Flow direction
Stream channels
Watershed area
Regional Ground Water Vulnerability –
Detail (1500m x 1500m) Scale
Potential Risk for Nitrates – Use of Logistic Regression
11
8
2
15
STATSGO (nationwide)
1:100,000
Landscape Representation
SSURGO (local)
1:24,000
Field/Hillslope
representation
AGWA
• PC-based GIS tool for watershed modeling
– KINEROS & SWAT (modular)
• Investigate the impacts of land cover
change on runoff, erosion, water quality
• Targeted for use by research scientists,
management specialists
– technology transfer
– widely applicable
Conceptual Design of AGWA
PROCESS
PRODUCTS
STATSGO
NALC, MRLC
USGS 7.5' DEM
Build GIS Database
Discretize Watershed
f (topography)
Contributing
Source Area
Characterize Model Elements
f (landcover, topography, soils)
Gravelly loam Soil
 Ks = 9.8 mm/hr
 G = 127 mm
 Por. = 0.453
intensity
Derive Secondary Parameters
look-up tables
View Model Results
link model to GIS
runoff
Build Model Input Files
10-year, 30-minute event
time
time
runoff, sediment hydrograph
Example of watershed discretization for parameterizing KINEROS. Model parameters are
averaged for each overland flow and channel element.
Sim ulated vs. Observed Runoff
Watershed 11 STATSGO soils
14
y = 2.0189x - 0.1804
r2 = 0.7889
Se =1.4907
1:1
10
8
6
4
2
0
0
2
4
6
8
10
12
14
Observed runoff (m m )
Sim ulated vs. Observed Runoff
Watershed 11 SSURGO soils
14
y = 1.4934x - 0.6588
r2 = 0.8249
Se =0.9820
12
Simulated runoff (mm)
Simulated runoff (mm)
12
1:1
10
8
6
4
2
0
0
2
4
6
8
Observed runoff (m m )
10
12
14
Tools for Uncertainty Analysis
• Sensitivity Analysis
– Input data error
– Methods
– Parameters and coefficients
• Monte Carlo Simulation
• Fuzzy Set Theory
• Bayesian Belief Networks
Sensitivity Analysis
• Sensitivity analysis investigates how a model
responds to changes in the information
provided to it as input.
• Assess the potential range of outcomes that
can be expected.
• Assess level of confidence in a result given
the potential error with the data.
• Identify the parts of the model that are critical
and the most important variables
– Identify where you need to improve your data
and those that are not.
http://www.innovativegis.com/basis/MapAnalysis/MA_Intro/MA_Intro.htm
Monte Carlo Simulation
• Execute the model across the range of all
the inputs to obtain a distribution of results.
– Statistical Distributions
– Error Estimates
– Relative Ranges (examine all weight
combinations)
• You can locate central tendencies and the
extremes.
Fuzzy Set Theory
• Fuzzy Inputs
– Polygons vs. Fields
• Continuous range for the outputs
– Few ordinal classes vs. a 0-100 range
• Multiple Fuzzy Logic Models
– Compare ranges of combined scores
Resultant flood risk calculated with three different fuzzy logic
families and defuzzification methods (Jasiewicz 2011)