Transcript Practical Solutions to Traceability and Uncertainty in

Practical Solutions to
Traceability and Uncertainty in
Accreditation
Presented to CITAC-NCSLI Joint Workshop
“Traceability and Uncertainty:
Key Technical Issues and Laboratory
Accreditation”
PITTCON 2002, New Orleans
Sunday, 17 March 2002
Warren Merkel, A2LA Technical Manager
American Association for Laboratory Accreditation
Presentation Overview
• Relevant ISO/IEC 17025 Requirements
• Accreditation Body Policies
• Practical Approaches for Laboratories
• Future Developments
• Questions
American Association for Laboratory Accreditation
Measurement Traceability 17025 Requirements
– Calibrations & reference materials traceable to
SI where possible
– Where not possible, traceable to certified
reference materials, agreed methods and/or
consensus standards
– …Unless it has been established that the
associated contribution from the calibration
contributes little to the total uncertainty of the
test result
American Association for Laboratory Accreditation
Traceability- Definition
• Relation to stated references through an
unbroken chain of comparisons…
– Traceability to some stated reference available
with most RMs
– Normally not SI unit for amount of substance,
but often other SI units
• All having stated uncertainties
– Little evidence of verification of uncertainty
claims
American Association for Laboratory Accreditation
Traceability in Practice
• In many cases, traceability to the test
method is all that is required by clients
• Uncertainty statements on RMs are
important, but usually not a significant
contributor to uncertainty of test
• Performance in proficiency tests can serve
as an indicator of traceability problems
American Association for Laboratory Accreditation
Measurement Uncertainty 17025 Requirements
• 5.4.6.2 - Testing laboratories shall have and apply
procedures for estimating uncertainty of measurement
• Nature of the test method may preclude rigorous,
metrologically and statistically valid, calculation of
uncertainty of measurement
• Laboratory shall at least attempt to identify all
uncertainty components, make a reasonable
estimation, and ensure that the form of reporting of
result does not give a wrong impression of
uncertainty
American Association for Laboratory Accreditation
Measurement Uncertainty 17025 “Loopholes”
• Note 1 - Degree of rigor depends on:
– Requirements of the method
– Requirements of the client
– Existence of narrow limits on specification conformance
• Note 2 - In cases where a well-recognized test method
specifies limits to the values of the major sources of
uncertainty and specifies the form of presentation of
calculated results, the laboratory is considered to have
satisfied this clause by following the test method and
reporting instructions
American Association for Laboratory Accreditation
Measurement Uncertainty 17025 Reporting Requirements
• 5.10.3.1 c) Information on uncertainty is needed
in test reports when it is relevant to the validity or
application of the test results, when a client’s
instruction so requires, or when the uncertainty
affects compliance to a specification limit
• 5.10.1 - In the case of a written agreement with the
client, results may be reported in a simplified way
American Association for Laboratory Accreditation
A2LA Interim Policy on Measurement
Uncertainty for Testing Laboratories
• Five categories of test methods
• Intended to facilitate transition- pragmatic
approach, not ideal in all cases
• Results from review at annual meeting:
–
–
–
Some modification of language
Publish list of example methods for each category
by field of testing
Publish guidance on determining uncertainty in
testing based on ISO 5725
American Association for Laboratory Accreditation
A2LA Interim Policy on Measurement
Uncertainty for Testing Laboratories
• Five categories of test methods:
I.
Qualitative
No uncertainty calculations required
Examples: Ignitability; Microbiological screening
II.
Well-recognized methods that specify limits to
uncertainty contributions (Note 2)
No further uncertainty calculations required
Examples: Flash point; Hardness
Problems: Modification of method
What if a client wants uncertainty?
American Association for Laboratory Accreditation
A2LA Interim Policy on Measurement
Uncertainty for Testing Laboratories
• Five categories of test methods, continued:
III. Published methods that do not specify limits to
uncertainty sources and/or reporting format
Uncertainty estimated using standard deviation of
laboratory control samples
Examples: Alloy analysis by OES; VOA
Problems: Normal process for analyzing control
samples may lead to an underestimate of
uncertainty
Quality of control sample
American Association for Laboratory Accreditation
A2LA Interim Policy on Measurement
Uncertainty for Testing Laboratories
• Five categories of test methods, continued:
IV.
Methods requiring identification of major
uncertainty components and reasonable estimate
of uncertainty
Examples: PBMS; One-off tests
V.
Methods requiring full uncertainty analysis
consistent with ISO Guide to the Expression of
Uncertainty in Measurement
Example: Reference material value assignment
American Association for Laboratory Accreditation
A2LA Interim Policy Chemical Laboratories
• Majority of methods classified as
Category II, III, IV
• Classification can vary by laboratory and
use
• Use of RMs critical to demonstrating
process control and evaluating bias
• More practical guidance required
American Association for Laboratory Accreditation
17025 Requirements: Quality Control
and Proficiency Testing
• Laboratory shall have procedures for monitoring
validity of tests, including:
– regular use of CRMs or internal QC using secondary
reference materials
– participation in interlaboratory comparison or
proficiency testing programs
• Record data so trends are detectable
• Laboratories in most cases already have sufficient
data for estimating uncertainty
American Association for Laboratory Accreditation
Practical Approach - Category II
• Laboratories performing tests in Category II
can utilize precision data published with
method as uncertainty estimate if:
– Laboratory has data demonstrating that its
repeatability is comparable to the method data
– Material used for precision estimate is similar
to materials tested by lab
– Method not modified
– Basis for estimate clearly stated
American Association for Laboratory Accreditation
Practical Approach - Category III
• Intermediate measures of precision provide
adequate estimate of uncertainty, if:
–
–
–
–
Measurement method standardized
Measurement process is in control
Control sample well characterized (preferably CRM)
Process for collecting data designed to vary all
significant uncertainty components
American Association for Laboratory Accreditation
Steps Laboratories Can Take
• Identify major uncertainty components
(App. D of EURACHEM/CITAC Guide)
• Establish control charts (ISO 8258)
• Attempt to design QC process to ensure
representative variation of inputs
• Fewer points  more emphasis on design
• May need to record additional data related to
precision as objective evidence
American Association for Laboratory Accreditation
Steps Laboratories Can Take
• When required to report uncertainty, clearly
define the basis for the estimate
– Clarify during contract review process
– If possible, determine with client end use of data
• Make use of PT study data
– Compare internal precision data to spread of results
of participants
– If study based on reference value, compare lab
result/uncertainty with reference
American Association for Laboratory Accreditation
Future Developments
• Consensus method development
– More rigor in precision data
– Additional detail regarding uncertainty sources
• Accreditation of RM producers
• Increased focus on PT based on reference
values vs. consensus values
• Increased awareness of traceability
uncertainty issues in user community
American Association for Laboratory Accreditation
Conclusions
• Accreditation bodies and laboratories
reconciling 17025 requirements that are
ahead of the state of development in many
industries
• Pragmatic approach to requirements is
necessary
• Goal: provide data that is fit for purpose
American Association for Laboratory Accreditation
Contact Information
Warren Merkel
A2LA
5301 Buckeystown Pike Suite 350
Frederick, MD, USA 21704
Direct line: 301 644 3204
Main: 301 644 3248
Fax: 301 662 2974
[email protected]
www.a2la.org
American Association for Laboratory Accreditation