Transcript Slide 1

Quality Assurance and Data
Quality for Stormwater
Programs
8th Annual EPA Region 6 MS4
Stormwater Conference
Richard Smith
INCOG
June 27, 2006
1
Data Quality and Quality Assurance
• “QA” is an essential component of creating and
maintaining the usefulness of data and information.
• QA addresses:
“QA = CYA”
• “Monitoring” data (e.g. field and lab measurement data)
• Observation data (e.g. visual inspections)
• Records of conversations, meetings, emails, letters, etc.
• Data summarization and analyses (e.g. statistics, estimates)
• Reports and data transfers (e.g. spreadsheets, letters)
• In-house processes for communication and record-keeping
• Storage, calibration and use of equipment and supplies
2
How Will Your Data Be Used ?
• Permitting Authority – (Annual Report / DMRs / Phase II
audits / P.A. pollution & compliance enforcement).
• Other State and Federal Agencies – to fulfill their own
data collection efforts for projects / databases.
• City Councils / County Commissions – program updates,
budgets, justify additional resources or changes to
ordinances, etc.
• Internal Staff Requests – information flow within city.
• Citizens / Local Parties – public education, data requests.
• Local Enforcement Actions – building your case.
• Local Media – radio, newspaper, television.
The Data Needs will dictate how you collect, store and report data.
3
Phase II Data Streaming
DATA IN-FLOWS
DATA OUT-FLOWS
Information to Media
Data for Court
Enforcement Data
Data
Management
System
Reports to Councils
Management Reports
Reports to City Staff
Reports to Citizens
Data to Agencies
ODEQ Annual Report
Program Audits
4
Stormwater Program Manager
Many cities and counties
struggle to define a
management structure.
CITY MANAGER
PUBLIC WORKS DIR.
CITY COUNCIL
LOCAL CITIZENS
& BUSINESSES
CONSULTANTS
STORMWATER
PROGRAM
MANAGER
AGENCIES &
SUBSTATES
VENDORS
MS4 / FACILITY
INSPECTIONS
EDUCATION
& TRAINING
CONSTRUCTION
INSPECTIONS
DATA MANAGER
& REPORTING
5
Most Phase IIs Need Help With...
• Map data. . . . . . . . . . . .
• QA methods . . . . . . .
GIS, GPS, AutoCAD, paper.
What’s required, most efficient.
• Electronic data sets. . . . . . Tier II, RCRA, TRI, etc.
• Field test kits. . . . . . . . . . D.O. pH, Cond., etc.
• Other field gear . . . . . . . . Samplers, bottles, etc.
• Data Management:
• Software for supervision, records, reports
• Forms for inspections, contact records. etc.
• Data and monitoring Quality Assurance (QA)
• Sampling and Lab services
6
Data Management Options
Need for Data
Management
Consider Your Ability to:
• Purchase
• Learn
• Use
Paper Filing
System
Computer
Software:
Spreadsheet or
Database
Commercial
Vendor Software
• Maintain
Customized /
Consultant
Developed
7
Why is QA Important ?
• Many circumstances require verification of
information:
• Measurement data for court cases or enforcement.
• Reports to City Councils, Commissions, citizens,
contractors, consultants, agencies and in-house use.
• Regulatory decision-making (e.g. violations of WQS).
• Political fallout if your data is discredited.
• Formal documentation of QA procedures is wise.
• Don’t expect to have a fully developed QA program
quickly – give yourself time to learn and experiment
with what works best for your local needs.
8
To What Activities Should QA Apply ?
• Technically, all activities pertaining to any type of data
collection, analysis, manipulation and reporting....
• Determine your highest priorities first. For example:
• Visual inspections / lite-monitoring of MS4
• Source tracking inspections
• Construction site inspections
• If private consultants and/or labs are used, be sure that
they employ appropriate and adequate QA to their
activities.
• Make this part of your contract specifications, especially if
their work is needed for enforcement or court evidence.
9
Even Simple Test Kits Need Attention
• Expired chemicals - field test kits have chemicals that expire
– keep records of refresh purchases.
• Some kits require “calibration” – use fresh calibration
standards and keep a log of calibration activities.
• Some kits can measure “check samples” (of known analyte
concentration) – perform checks frequently and keep
records of all tests.
• SOPs help ensure continuous proper use, and aid in training
new employees.
• Maintenance includes verifying equipment function and
replacing defective parts before re-use of the kit. Detailed
maintenance records should be kept.
10
Types of Field Instruments
Comparator
Test Kits
Test Strips
Titrations
Colorimeters
These types of kits rely upon
chemical reactions to indicate
the presence and/or
concentration of one specific
parameter.
An SOP should be written for
each type of kit.
Multi-Test Kits
No endorsement of products is intended.
11
Types of Field Instruments
Test Pens
These types of kits
measure directly the
concentration of one or
more parameters.
Single Parameter
Meters
Multi-Parameter
Meters
Field Fluorometer
An SOP should be
written for each device
and procedure.
Turbidimeter
No endorsement of
products is intended.
Flow Meter
12
QA Concepts
Precision is a measure
of how close multiple
readings are of the same
sample.
Both are important concepts
in assessing quality of data.
Accuracy is a measure
of how close a reading
is to the known value.
Both are normally reported
with laboratory data.
13
More QA Concepts
Representativeness
Does the sample represent what you intend?
Completeness
Do you have enough samples?
Comparability
Are sampling methods useable to others?
Chain of Custody
Sample Labels
Legal documentation of “ownership” of
samples during all phases of transport.
Adequate information on each sample bottle
to show ownership, collection information
and intended analyses.
14
Types of QA Documentation to Consider
• QAPP – QA Project Plan - how you conduct data
gathering, analysis and reporting for your Phase II
program.
• DQOs – Data Quality Objectives – deciding in the
beginning what types of data you will be collecting
and, most important, how it will be used.
• QMP – Quality Management Plan – down the road,
you may consider this – describes your overall data
management and communication system.
• SOPs – Standard Operating Procedures – detailed
instructions for how you perform specific data
collection activities, including use, storage and care
of measurement equipment.
15
What Is Absolutely Required ?
• Each State’s General Permit should provide specific
requirements.
• In Oklahoma, OKR04 Part V lists a number of
requirements for “Monitoring, Record Keeping and
Reporting”
• “Samples and measurements...shall be representative...”
• “If lab analysis is conducted it must [use] 40 CFR Part
136” [methods]
• “Monitoring records must include...[names, dates, times,
place, methods, etc.]”
• “Retain records of all monitoring information” (e.g.
copies of reports, instrument charts, etc.)
16
Are Formal QA Documents Required ?
• Your State’s General Permit will state which, if any, QA
documents will be required (e.g. QAPP, DQOs, QMP
and/or SOPs).
• In Oklahoma, OKR04 does not specifically require any
of these documents.
• However, one must consider the “CYA” aspect of QA as
well as what the minimum permit requirements are.
• Particularly if your data are to be used in court, for
local enforcement or for making regulatory decisions...
• Or for that matter, just giving yourself credibility to
your manager, peers, mayor and citizens.
17
How to Start a Formal QA Program
• Managers, Mayors and Councilmen must learn the
importance of good QA’d data, and how it will help them
defend their programs.
• Don’t try to do it all at once – expect changes as you go.
• Decide what level of QA you need to apply to each type
of data and activity.
• Start with SOPs and a basic QAPP.
• Seek guidance from experienced resources.
• Be comfortable with what you create – stick to basics.
• Amend as your Phase II program grows – QA is dynamic.
18
A Good Beginning QA Program
• Start applying QA to “hard data” first – e.g., chemical data
from field measurements, field test kits and lab data as well
as visual observation data from visual inspections.
• DQOs - Decide upon the level of data quality you want for
the main types of data you will be collecting. Use of data.
• SOPs – identify all routinely repeatable actions and write
detailed step by step instructions and list supplies needed.
• QAPP – prepare an umbrella QA plan for all activities
requiring formal QA attention.
• Lab and Service contracts – include in your contract (and
QAPP) what technical requirements you want for lab
services and from vendors and consultants that will be
performing technical services (e.g., inspections).
19
EPA’s QA Resources
• EPA guidance documents:
• http://www.epa.gov/earth1r6/6wq/ecopro/watershd/monitrng/qappsprt/
• DQO Guidance = QA/G-4
• QAPP Guidance = QA/R-5
• QMP Guidance = QA/R-2
• SOP Guidance = QA/G-6
• These are not approvable by EPA unless they pertain to
projects funded by EPA water quality grants.
• EPA periodically offers QA training courses.
• Contact your Permitting Authority and substates for QA
advice.
20
Thank you for your
infinite patience.
Any Questions ?