(ILS) & Proficiency Testing Programs (PTP)
Download
Report
Transcript (ILS) & Proficiency Testing Programs (PTP)
ASTM International
Proficiency Test Programs and
Interlaboratory Studies Program
Presenters:
Steve Hepburn, PTP Director
Phillip Godorov, ILS Director
1
ASTM Proficiency Testing Programs
What is a Proficiency Testing Program?
Designed as a statistical quality control tool
enabling participating laboratories to assess their
performance in conducting ASTM or other test
methods
Uniform samples are analyzed by
participating labs using specified methods
and results are reported for statistical
treatment
2
September 2012
ASTM Proficiency Testing Programs
ASTM :
Provides management and administrative support to the
program
Program registration, contract negotiations for sample
prep and distribution, data collection and generation of
statistical summary reports
Coordinates the preparation and distribution of test samples
Sample are prepared by outside contractors
Provides reporting instructions, worksheets and electronic
data report forms for submitting data
3
September 2012
ASTM Proficiency Testing Programs
• ASTM statistical summary reports include:
Listing of all lab data
Charts and graphs
• Lab data is coded to maintain lab confidentiality.
• Final summary reports are provided electronically to
participants and appropriate Committee contacts.
• ASTM Custom Company Reports – provided to companies
with multiple lab performance data upon request.
4
September 2012
ASTM #2Diesel Fuel
Sample ID: DF21102
Acid Number – D664
Lab data coded to maintain lab confidentiality
5
September 2012
ASTM Proficiency
Testing Programs
6
September 2012
ASTM Proficiency
Testing Programs
Box and
Whiskers
graph to
compare
methods
7
September 2012
ASTM Proficiency Testing Programs
Globally recognized programs:
Over 4,300 laboratories participating
Over 57% participation outside
continental U. S.
8
September 2012
ASTM Proficiency
Testing Programs
Click here
9
ASTM Proficiency
Testing Programs
10
ASTM Proficiency
Testing Programs
Metals Testing: (E28 and E01)
Mechanical Properties Testing of Metals
Plain Carbon and Low-Alloy Steel (Chemical
Analysis)
Stainless Steel (Chemical Analysis)
Aluminum (Chemical Analysis)
Determination of Gold in Bullion
Plastics Testing: (D20)
Polyethylene Plastics Testing
Multiple Plastics - Mechanical Properties
Testing
Polypropylene - Mechanical Properties Testing
Flammability of Plastics Testing
Thermal Analyses of Plastics
Elemental Analyses of Plastics
11
September 2012
ASTM Proficiency
Testing Programs
Aromatic Hydrocarbons Testing (D16)
Engine Coolants Testing (D15)
Electrical Insulating Liquids: (D27)
Insulating Fluid Quality
Dissolved Gas Analyses-In-Service Insulating
Fluid
Textiles :(D13)
12
Light, Medium, Heavy Fabric Testing
Yarns and Threads
September 2012
ASTM Proficiency
Testing Programs
Petroleum Products and Lubricants (D02)
Turbine Oil
Lubricating Grease
Crude Oil
In-Service Diesel Oil InService Hydraulic Fluids / Oils
Base Oils
Wax
General Gas Oils
Ultra Low Sulfur Diesel
Fuel Ethanol
Biodiesel(B100) Fuel
Petroleum Coke
13
Engine Oil Lubricants
#2 Diesel Fuel
Aviation Turbine (Jet A) Fuel
Motor Gasoline
#6 Fuel Oil
Automatic Transmission Fluid
Gear Oil
Reformulated Gasoline
Hydraulic Fluids & Oils
Lube Additives
Industrial Gear Oil
Naphtha
September 2012
ASTM Proficiency
Testing Programs
ASTM's Cement and Concrete Reference Laboratory (CCRL)
Works with
• ASTM Committee C01 - Cement
• ASTM Committee C09 - Concrete and Aggregates
Located in Frederick, MD
• Operates the following proficiency sample programs:
• Portland cement
• Hydraulic cement concrete
• Masonry mortar
• Blended cement
• Pozzolans
• Masonry cement
• Concrete masonry units
Visit the CCRL web site on ASTM web page for more information.
14
Use of Programs by Participants
Lab QA Tool
Data for monitoring lab strengths and weaknesses
Residual sample material useful as internal QA sample
Tool to evaluate other test methods for lab use
Participation helps satisfy lab accreditation requirements
Can demonstrate testing capability to customers
Results and associated statistics helps ASTM validate test
method performance under real world conditions
15
September 2012
Use of Programs by ASTM
Committees
Obtain valuable feedback on methods
Resource for increased membership
Database of laboratories for test method validation
16
September 2012
Establishing New Programs
Concept approved by Executive Subcommittee
Program pre-proposal
Market research - survey
Final program proposal
Promote & Launch
Bulk sample requirements
Sample preparation and distribution
ASTM staff will provide guidance and assistance.
17
September 2012
ASTM Proficiency Testing Programs
The programs are open to all companies and
industry test professionals worldwide.
18
September 2012
ASTM Proficiency
Testing Programs
For more information please contact:
ASTM - PTP
Steve Hepburn
[email protected]
Phone: (610) 832-9688
“Until a measurement is in a state of
statistical control, it can not be believed in
any logical sense that it's measuring
anything at all." Dr. Churchill Eisenhart
19
September 2012
ASTM International Interlaboratory
Study Program (ILS)
Presenter:
Phillip Godorov, ILS Director
20
September 2012
The Interlaboratory Study (ILS) Program
Who we are and what we do …
We were originated following approval by the Board of Directors
to create a unit that would help to strengthen the perceived
quality of ASTM Test Methods by:
facilitating the production of data to develop Precision & Bias
statements and Research Reports;
providing administrative support to the committees;
offering financial support to the committees;
helping to ensure the confidentiality of participating labs.
21
September 2012
Administrative Support
Review of Experimental Design
Assistance identifying volunteer laboratories
Identification of sample vendors
Coordination of sample distribution
Data collection
Statistical processing
Generation of reports
22
September 2012
Strengths of the
ASTM Interlaboratory
Study Program
Input is encouraged from active Committee
volunteers, as well as non-members, broadening the
range and diversity of the study participants, allowing
the study to most accurately demonstrate expected
“real-world” precision
Scientific neutrality of ASTM in reviewing test data
Providing value-added Quality Assurance Programs
to participating laboratories
23
September 2012
Potential Benefits of Participation
Statistical program to monitor strengths and
weaknesses of lab testing when compared to peers
Assess testing performance and adherence to
written procedures by lab technicians
Recognition in the final Research Report
Upon publication of the results of an ILS, all
participating labs are eligible to receive a free 12 month
listing in ASTM’s Lab Directory
24
September 2012
Benefits to the
Committees
Obtain valuable feedback on
methods, leading to the correction of
errors and omissions, as well as
highlighting the need for technical
updates
Resource for increased
membership
Database of laboratories for test
method validation
Database of suppliers for
common study materials
25
September 2012
Establishing New Programs
Concept registered as a Work Item
Program registered online as an ILS
Initial conference call, with the a technical contact from
the committee, to establish the basic study parameters
Experimental design (with input from the committee’s
statistical support person, if available)
Identification of study materials, suppliers, a distributor,
and volunteer laboratories
26
September 2012
Administering
Programs
Conference Calls or Virtual Meetings with the
participants to discuss specific study instructions
Coordination of acquisition and distribution of study
material
Physical
Electronic
Collection of data
Analysis of data
Compilation of Precision Statement and Draft
Research Report (from registered information)
27
September 2012
Precision
To calculate precision, we need usable data from at
least six laboratories.
The closer that we can get to 30 participants, the better.
The precision statement in an ASTM test method is not
meant to qualify it as good or bad.
The published precision is there to help a user of the
standard understand what can be expected based on the
real world results of others.
28
September 2012
The Statistics (in a nutshell) ~ ASTM E691
All of the test results from one laboratory on one material
constitute a cell.
The cell average is calculated for each laboratory.
The standard deviation of the test results in each cell is
calculated.
Then we calculate the average of all the cell averages for any one
material – this shows up in the P&B as the “average.”
For each laboratory we calculate the cell deviation by subtracting
the cell average from the average of all the cell averages.
Finally, we calculate the Standard Deviation of the Cell Averages.
29
September 2012
Repeatability (r) ranges
With 95% confidence, the same operator, in the same
laboratory, using the same equipment, under the same
conditions, should obtain results when testing the same material
that agree within this range.
Example: repeatability range = 2.4 ppm
Result 1: 79.1 ppm
Result 2: 81.6 ppm
Results differ by 2.5 ppm, therefore: Suspect
Internal laboratory investigation may be advisable
30
September 2012
Reproducibility (R) ranges
With 95% confidence, two operators, in different laboratories,
using different equipment, under conditions meeting those
specified in the standard, should obtain results when testing the
same material that agree within this range.
Example: reproducibility range = 3.2 ppb
Result from lab 1: 50.8 ppb
Result from lab 2: 47.9 ppb
Results differ by 2.9 ppb, therefore: As Expected
31
September 2012
Bias
To calculate bias, we need to include a
reference “standard” among the sample
specimens distributed to the participating
laboratories.
Bias may be determined as the average discrepancy
between the “known” value and the reported values.
32
September 2012
Precision and Bias
33
September 2012
Remember …
You can correct for bias.
You cannot correct for imprecision.
An ILS can be used to demonstrate
improvement as standards are modified.
Compare results from Method 1 with those from
Method 2
The presence of a Precision Statement can
make a standard stronger, but it doesn’t make
it more repeatable.
34
September 2012
Research Reports
The ASTM Form and Style Manual (Section A29.1)
states, "Where numerical data have been generated to
establish the precision and bias of a test method, a
research report is required.”
ASTM Research Report Template
The draft research report should be made available to
committee members while the associated precision
statement is on ballot.
Research report numbers are assigned after the ballot is
approved.
35
September 2011
Parts of a Research Report
List of participating laboratories
Description of samples with their suppliers
A copy of the laboratory instructions
Description of equipment / apparatus used
All of the raw data (lab names hidden)
A statistical summary
A copy of the precision and bias statement
36
September 2012
Research Report Assistance
ILS can assist with the following research
report tasks:
Answer any research report inquiries
Create research reports from ILS studies
Reformat older research reports to fit the
ASTM research report Template
Assign research report numbers
37
September 2012
Research Report Numbers
A research report number will be assigned
by ASTM when all of the following have
been completed:
the research report is submitted to ILS;
it has been reviewed for completeness;
the ballot item to include the
corresponding precision and bias
statement is approved for publication.
38
September 2012
ILS informational Web pages
http://www.astm.org/ILS/
39
September 2012
40
September 2012
41
September 2012
42
September 2012
43
September 2012
If you are
updating
an ILS
that
you’ve
already
started
When you
are
starting a
brand new
ILS
44
September 2012
First Page of the
Registration
45
September 2012
Technical Contact
46
September 2012
Analyses
47
September 2012
Replicates
48
September 2012
Materials
49
September 2012
Suppliers
50
September 2012
Suppliers
Continued…
51
September 2012
Distributor
52
September 2012
Laboratories
53
September 2012
Statistical Support
54
September 2012
ILS Registration
Summary
55
September 2012
ILS Stats
Number of active ILS programs
Number of committees participating
Number of programs registered in 2011
Number of programs registered so far in 2012
Total number of programs completed
Number of programs completed in 2011
Number of programs completed so far in 2012
% of programs for new test methods
Average amount of time taken to complete an ILS (months)
Range of time taken to complete an ILS (months)
56
312
79
104
65
291
67
26
35
18
52
September 2012
Questions
57
ASTM International
Interlaboratory Study Program (ILS)
Phillip Godorov
Director, Interlaboratory Study Program
ASTM International
100 Barr Harbor Drive
West Conshohocken, PA 19428-2959
Phone: 610-832-9715
Fax:
610-834-7011
E-mail: [email protected]
58
September 2012
Thank you for your
participation!
Proficiency / Interlaboratory Crosscheck
Testing Programs
Interlaboratory Study Program (ILS)
59
September 2012