Transcript Slide 1

Overview of the USGS Plan
for Quality Assurance of
Digital Aerial Imagery
ASPRS 2006 Annual Conference
Reno, Nevada
U.S. Department of the Interior
U.S. Geological Survey
Outline






Background
ASPRS Study recommendations
Who, what, why, how
USGS Plan for Quality Assurance of Digital Aerial
Imagery
When
Summary
2
Background
Why calibration at USGS?






Photogrammetric methods for map generation
Map production primarily done in-house
Quality assurance measure for aerial photography
from aerial contractors
Leadership role in standards development
Unbiased, independent agency with technical
expertise
Quality assurance for The National Map
3
USGS Camera Calibration History

USGS responsible for calibration services for film
camera in United States since 1973

USGS operates Optical Sciences Lab (OSL) in Reston,
VA with a custom-built calibration instrument

Current policy requires current (within 3 years)
camera calibration report on file before award of
contract

The “Catch-22” problem for digital sensors
4
ASPRS Study Recommendations




Continue to provide analog calibration services
Develop and implement digital calibration capabilities
Develop standards for camera and sensor calibrations
Develop and implement in-situ calibration methods
5
Stages of New Technology




Stage 1:
Stage 2:
Stage 3:
Stage 4:
Initiation
Contagion
Control
Integration
6
Example: DOQs




Initiation:
Contagion:
Control:
Integration:
Research and pilot projects
Wild enthusiasm and demand
Define standards and format
DOQs critical to enterprise operation
7
The Barriers to New Technology







The unknown: Does it perform as claimed?
Lack of standards: What are the specifications?
Lack of experience: What I can or can not do?
Business decision: Does it make business sense?
Operational changes: What are the best practices?
Integration: How to integrate into existing processes?
Overcoming inertia: Acceptance by users?
8
The Formula for Quality




An independent validation of manufacturer’s
specifications
Selection of well-qualified Data Providers with
experience and established “best practices”
Standardized procurement specifications and
selection criteria
Quality control measures
9
What is the USGS Strategy for Quality?




Development of a comprehensive quality plan
Acceptance and use by Inter-Agency Digital Image
Working Group composed of 14 Federal agencies
Extension of requirement to other agencies and
contracting offices
Acceptance by general user community
10
What’s Your Perspective?
The major players:
Manufacturers
Data Providers
Procurement officials
End-users
11
The USGS Perspective




Image products and services for other agencies and
cooperators via contracts with Data Providers
Occasional procurement of sensor systems
Quality assurance of end products for The National
Map
Development and establishing standards for
geospatial community
12
USGS Plan for Quality Assurance

Four major parts covering two major processes:

Data Production



Manufacturers Certification
Data Providers Certification
Data Purchasing & Acceptance


Contracting Guidelines
Data Acceptance Standards
13
The USGS Plan
Data Purchasing and Acceptance Domain
User Needs
Data Procurement:
Data Users and Inspectors:
Contracting Guidelines &
Boilerplate Tool
Acceptance Standards
Sensor Manufacturers:
Data Providers:
Manufacturers Certification
Data Providers Certification
Data Production Domain
14
Final Product
Manufacturers Certification

Aerial Digital Imaging is in its “Wild West” phase



Anything & everything being tried
Some metric-quality systems
Many “other” systems

How does the customer know which can produce
mapping-quality data?

USGS to offer “type certification” of mapping-quality
digital aerial sensors


Must be stable, well-quantified, repeatable
Able to routinely generate mapping-quality data

When operated properly!
15
Benefits of Manufacturers Certification






Communicates specifications
Provides evidence of system performance
Independent certification helps to promote sensor
systems
Supports verification of Data Provider’s system
Type certification eliminates burden of calibration for
each sensor sold in the United States (1 time vs. n
times)
Eliminates need for USGS to have custom-built
calibration instrument for calibration purposes
16
USGS Manufacturers Review Team

Review Team Lead - Gregory L. Stensaas
Remote Sensing Technologies Project Manager
USGS Earth Resources Observation and Science Center, Sioux Falls, SD
[email protected]

Systems Engineering Team Member - Jon Christopherson
Science Applications International Corporation (SAIC)
Contractor to the USGS EROS, Sioux Falls, SD
[email protected]

Photogrammetric Engineering Team Member - Dr. George Y. G. Lee
U.S. Geological Survey, Menlo Park, CA
[email protected]

Geo-Spatial and Software Engineering Team Member - Donald Moe
Science Applications International Corporation (SAIC)
Contractor to the USGS EROS, Sioux Falls, SD
[email protected]

Radiometric and Physics Team Member - Dr. Robert Ryan
Science Systems and Applications, Inc.
Contractor to NASA Stennis Space Center, MS
[email protected]
17
Data Providers Certification

Focused on processes and process control


Ensures that Data Providers are operating sensors in
accordance with manufacturer’s instructions and limitations
Ensures that Data Providers follow quality procedures

Focused on documentation and monitoring of
procedures and best practices

ISO-like certification process

Desire to ensure reliability, repeatability, and quality
18
Benefits of Data Providers Certification





Provides evidence of performance of products
Independent certification helps to promote product
specifications and Data Provider’s capabilities
Documents Data Provider’s quality assurance plan
and “best practices”
One certification for Data Provider and not for each
camera
Data Providers no longer have to send cameras to
OSL for calibration, reducing down-time and shipping
expenses
19
Contracting Guidelines

User community is not sure how to contract for digital
imagery






New terms, capabilities, standards, lexicon
Inhibits digital contracting
Addresses boilerplate requiring “USGS Certificate”
Goal is to remove barriers to digital aerial contracts
Encourage digital imaging
Created Federal Digital Imagery General Contract
Guideline
20
Benefits of Contracting Guidelines

Standardized terms and descriptions make the
contracting process easier and more uniform among
agencies

Guidelines help acceptance of digital sensors and
educate end-users on benefits of digital technology

Standardized terms and guidelines help contracting
officers describe their users needs

Standardized performance measures

USGS certifications provide a priori acceptance of
systems and Data Provider’s “best practices”

Manufacturer and Data Providers Certification reduces
necessary documentation in the RFP process
21
Digital Data Acceptance Standards

End-users unsure of how to judge digital aerial data
quality


New terms & capabilities (resolution, spectral, etc.)
Each customer understands things differently

There is a need for common, uniform definitions and
methods for evaluating quality of image data

USGS to work with Inter-Agency Digital Image
Working Group to develop these standards

Goal is a Web-based tool illustrating quality problems,
measurement techniques, and standards
22
Benefits of Acceptance Standards

Data consumers have common standards to evaluate
data products

More consistent acceptance/rejection criteria among
contracting agencies

Clearer standards and guidelines helps to eliminate
false expectations

Ensures high quality products

Increases customer satisfaction
23
Funding Strategy

Manufacturers Certification (Cost-shared by USGS
and manufacturers)

Data Providers Certification (100% by Data Providers)

Contracting Guidelines (100% USGS and IADIWG
funded)

Acceptance standards (100% USGS and IADIWG
funded)
24
Status







Manufacturers Certification Guidelines drafted
Up to 4 manufacturers to be certified this fiscal year
Applanix and Intergraph factory visits completed, reports
pending
A few details but no major issues; certification to be issued soon
Leica and Vexcel-Microsoft visits being discussed now
Four Data Providers have expressed interest working on the
initial round of Data Providers Certification
First draft of Digital Imagery Contracting Guideline completed
and reviewed by limited group
A Web-based tool to help generate contracting language is
being developed
IADIWG to begin drafting guidelines in Fall


25
Next Steps







USGS Plan for Quality Assurance of Digital Aerial
Imagery briefed during ASPRS panel session
Invite comment from broader community
Finalize and obtain ASPRS approval
Complete initial four Manufacturers Certifications
Begin initial Data Providers Certifications
Provide briefings to geospatial community
Participate in international forums to communicate
plans used in the United States
26
Closing Thought
Four Phases of Technology Assimilation:




Phase 1:
Phase 2:
Phase 3:
Phase 4:
Identification and investment
Learning and adaptation
Management control and standards
Maturity and widespread acceptance
27
Summary

To implement a comprehensive, meaningful process
that ensures the quality of data products and services

To cooperatively develop the plan with all elements of
the geospatial community

Good for one is good for all

Education and training for the community

To establish a model to support other new
technologies in the future
28
For more information

IADIWG Web site at: http://calval.cr.usgs.gov/
29
For issues or comments
Contact:

Gregory L. Stensaas
Remote Sensing Systems Characterization Manager
USGS EROS Data Center
47914 252nd Street
Sioux Falls, SD 57198
605-594-2569
[email protected]
30
The USGS Quality Plan
Questions or comments?
31