Transcript (ILS) & Proficiency Testing Programs (PTP)

ASTM International
Proficiency Test Programs and
Interlaboratory Studies Program
Presenters:
Steve Hepburn, PTP Director
Phillip Godorov, ILS Director
1
ASTM Proficiency Testing Programs
What is a Proficiency Testing Program?
 Designed as a statistical quality control tool
enabling participating laboratories to assess their
performance in conducting ASTM or other test
methods
 Uniform samples are analyzed by
participating labs using specified methods
and results are reported for statistical
treatment
2
September 2012
ASTM Proficiency Testing Programs
ASTM :



Provides management and administrative support to the
program
 Program registration, contract negotiations for sample
prep and distribution, data collection and generation of
statistical summary reports
Coordinates the preparation and distribution of test samples
 Sample are prepared by outside contractors
Provides reporting instructions, worksheets and electronic
data report forms for submitting data
3
September 2012
ASTM Proficiency Testing Programs
• ASTM statistical summary reports include:
 Listing of all lab data
 Charts and graphs
• Lab data is coded to maintain lab confidentiality.
• Final summary reports are provided electronically to
participants and appropriate Committee contacts.
• ASTM Custom Company Reports – provided to companies
with multiple lab performance data upon request.
4
September 2012
ASTM #2Diesel Fuel
Sample ID: DF21102
Acid Number – D664
Lab data coded to maintain lab confidentiality
5
September 2012
ASTM Proficiency
Testing Programs
6
September 2012
ASTM Proficiency
Testing Programs
Box and
Whiskers
graph to
compare
methods
7
September 2012
ASTM Proficiency Testing Programs
Globally recognized programs:
 Over 4,300 laboratories participating
 Over 57% participation outside
continental U. S.
8
September 2012
ASTM Proficiency
Testing Programs
Click here
9
ASTM Proficiency
Testing Programs
10
ASTM Proficiency
Testing Programs
Metals Testing: (E28 and E01)

Mechanical Properties Testing of Metals
 Plain Carbon and Low-Alloy Steel (Chemical
Analysis)
 Stainless Steel (Chemical Analysis)
 Aluminum (Chemical Analysis)
 Determination of Gold in Bullion
Plastics Testing: (D20)
 Polyethylene Plastics Testing
 Multiple Plastics - Mechanical Properties
Testing
 Polypropylene - Mechanical Properties Testing
 Flammability of Plastics Testing
 Thermal Analyses of Plastics
 Elemental Analyses of Plastics
11
September 2012
ASTM Proficiency
Testing Programs
Aromatic Hydrocarbons Testing (D16)
Engine Coolants Testing (D15)
Electrical Insulating Liquids: (D27)


Insulating Fluid Quality
Dissolved Gas Analyses-In-Service Insulating
Fluid
Textiles :(D13)


12
Light, Medium, Heavy Fabric Testing
Yarns and Threads
September 2012
ASTM Proficiency
Testing Programs
Petroleum Products and Lubricants (D02)
Turbine Oil
Lubricating Grease
Crude Oil
In-Service Diesel Oil InService Hydraulic Fluids / Oils
Base Oils
Wax
General Gas Oils
Ultra Low Sulfur Diesel
Fuel Ethanol
Biodiesel(B100) Fuel
Petroleum Coke
13
Engine Oil Lubricants
#2 Diesel Fuel
Aviation Turbine (Jet A) Fuel
Motor Gasoline
#6 Fuel Oil
Automatic Transmission Fluid
Gear Oil
Reformulated Gasoline
Hydraulic Fluids & Oils
Lube Additives
Industrial Gear Oil
Naphtha
September 2012
ASTM Proficiency
Testing Programs
ASTM's Cement and Concrete Reference Laboratory (CCRL)
 Works with
• ASTM Committee C01 - Cement
• ASTM Committee C09 - Concrete and Aggregates
 Located in Frederick, MD
• Operates the following proficiency sample programs:
• Portland cement
• Hydraulic cement concrete
• Masonry mortar
• Blended cement
• Pozzolans
• Masonry cement
• Concrete masonry units
Visit the CCRL web site on ASTM web page for more information.
14
Use of Programs by Participants

Lab QA Tool






Data for monitoring lab strengths and weaknesses
Residual sample material useful as internal QA sample
Tool to evaluate other test methods for lab use
Participation helps satisfy lab accreditation requirements
Can demonstrate testing capability to customers
Results and associated statistics helps ASTM validate test
method performance under real world conditions
15
September 2012
Use of Programs by ASTM
Committees

Obtain valuable feedback on methods

Resource for increased membership

Database of laboratories for test method validation
16
September 2012
Establishing New Programs
 Concept approved by Executive Subcommittee

Program pre-proposal

Market research - survey

Final program proposal

Promote & Launch
Bulk sample requirements
Sample preparation and distribution
ASTM staff will provide guidance and assistance.
17
September 2012
ASTM Proficiency Testing Programs
The programs are open to all companies and
industry test professionals worldwide.
18
September 2012
ASTM Proficiency
Testing Programs
For more information please contact:
ASTM - PTP
Steve Hepburn
[email protected]
Phone: (610) 832-9688
“Until a measurement is in a state of
statistical control, it can not be believed in
any logical sense that it's measuring
anything at all." Dr. Churchill Eisenhart
19
September 2012
ASTM International Interlaboratory
Study Program (ILS)
Presenter:
Phillip Godorov, ILS Director
20
September 2012
The Interlaboratory Study (ILS) Program
Who we are and what we do …
We were originated following approval by the Board of Directors
to create a unit that would help to strengthen the perceived
quality of ASTM Test Methods by:
facilitating the production of data to develop Precision & Bias
statements and Research Reports;
providing administrative support to the committees;
offering financial support to the committees;
helping to ensure the confidentiality of participating labs.
21
September 2012
Administrative Support

Review of Experimental Design

Assistance identifying volunteer laboratories

Identification of sample vendors

Coordination of sample distribution

Data collection

Statistical processing

Generation of reports
22
September 2012
Strengths of the
ASTM Interlaboratory
Study Program

Input is encouraged from active Committee
volunteers, as well as non-members, broadening the
range and diversity of the study participants, allowing
the study to most accurately demonstrate expected
“real-world” precision
Scientific neutrality of ASTM in reviewing test data

 Providing value-added Quality Assurance Programs
to participating laboratories
23
September 2012
Potential Benefits of Participation
 Statistical program to monitor strengths and
weaknesses of lab testing when compared to peers
 Assess testing performance and adherence to
written procedures by lab technicians
 Recognition in the final Research Report
 Upon publication of the results of an ILS, all
participating labs are eligible to receive a free 12 month
listing in ASTM’s Lab Directory
24
September 2012
Benefits to the
Committees
 Obtain valuable feedback on
methods, leading to the correction of
errors and omissions, as well as
highlighting the need for technical
updates
 Resource for increased
membership
 Database of laboratories for test
method validation
 Database of suppliers for
common study materials
25
September 2012
Establishing New Programs

Concept registered as a Work Item

Program registered online as an ILS
 Initial conference call, with the a technical contact from
the committee, to establish the basic study parameters
 Experimental design (with input from the committee’s
statistical support person, if available)
 Identification of study materials, suppliers, a distributor,
and volunteer laboratories
26
September 2012
Administering
Programs
 Conference Calls or Virtual Meetings with the
participants to discuss specific study instructions
 Coordination of acquisition and distribution of study
material
 Physical
 Electronic

Collection of data

Analysis of data
 Compilation of Precision Statement and Draft
Research Report (from registered information)
27
September 2012
Precision
 To calculate precision, we need usable data from at
least six laboratories.
 The closer that we can get to 30 participants, the better.
 The precision statement in an ASTM test method is not
meant to qualify it as good or bad.
 The published precision is there to help a user of the
standard understand what can be expected based on the
real world results of others.
28
September 2012
The Statistics (in a nutshell) ~ ASTM E691
 All of the test results from one laboratory on one material
constitute a cell.
 The cell average is calculated for each laboratory.
 The standard deviation of the test results in each cell is
calculated.
 Then we calculate the average of all the cell averages for any one
material – this shows up in the P&B as the “average.”
 For each laboratory we calculate the cell deviation by subtracting
the cell average from the average of all the cell averages.
 Finally, we calculate the Standard Deviation of the Cell Averages.
29
September 2012
Repeatability (r) ranges
 With 95% confidence, the same operator, in the same
laboratory, using the same equipment, under the same
conditions, should obtain results when testing the same material
that agree within this range.
 Example: repeatability range = 2.4 ppm
 Result 1: 79.1 ppm
Result 2: 81.6 ppm
 Results differ by 2.5 ppm, therefore: Suspect
 Internal laboratory investigation may be advisable
30
September 2012
Reproducibility (R) ranges
 With 95% confidence, two operators, in different laboratories,
using different equipment, under conditions meeting those
specified in the standard, should obtain results when testing the
same material that agree within this range.
 Example: reproducibility range = 3.2 ppb
 Result from lab 1: 50.8 ppb
 Result from lab 2: 47.9 ppb
 Results differ by 2.9 ppb, therefore: As Expected
31
September 2012
Bias
 To calculate bias, we need to include a
reference “standard” among the sample
specimens distributed to the participating
laboratories.
 Bias may be determined as the average discrepancy
between the “known” value and the reported values.
32
September 2012
Precision and Bias
33
September 2012
Remember …
 You can correct for bias.
 You cannot correct for imprecision.
 An ILS can be used to demonstrate
improvement as standards are modified.
 Compare results from Method 1 with those from
Method 2
 The presence of a Precision Statement can
make a standard stronger, but it doesn’t make
it more repeatable.
34
September 2012
Research Reports
 The ASTM Form and Style Manual (Section A29.1)
states, "Where numerical data have been generated to
establish the precision and bias of a test method, a
research report is required.”
 ASTM Research Report Template
 The draft research report should be made available to
committee members while the associated precision
statement is on ballot.
 Research report numbers are assigned after the ballot is
approved.
35
September 2011
Parts of a Research Report
 List of participating laboratories
 Description of samples with their suppliers
 A copy of the laboratory instructions
 Description of equipment / apparatus used
 All of the raw data (lab names hidden)
 A statistical summary
 A copy of the precision and bias statement
36
September 2012
Research Report Assistance
 ILS can assist with the following research
report tasks:
 Answer any research report inquiries
 Create research reports from ILS studies
 Reformat older research reports to fit the
ASTM research report Template
 Assign research report numbers
37
September 2012
Research Report Numbers
 A research report number will be assigned
by ASTM when all of the following have
been completed:
the research report is submitted to ILS;
it has been reviewed for completeness;
the ballot item to include the
corresponding precision and bias
statement is approved for publication.
38
September 2012
ILS informational Web pages
http://www.astm.org/ILS/
39
September 2012
40
September 2012
41
September 2012
42
September 2012
43
September 2012
If you are
updating
an ILS
that
you’ve
already
started
When you
are
starting a
brand new
ILS
44
September 2012
First Page of the
Registration
45
September 2012
Technical Contact
46
September 2012
Analyses
47
September 2012
Replicates
48
September 2012
Materials
49
September 2012
Suppliers
50
September 2012
Suppliers
Continued…
51
September 2012
Distributor
52
September 2012
Laboratories
53
September 2012
Statistical Support
54
September 2012
ILS Registration
Summary
55
September 2012
ILS Stats
Number of active ILS programs
Number of committees participating
Number of programs registered in 2011
Number of programs registered so far in 2012
Total number of programs completed
Number of programs completed in 2011
Number of programs completed so far in 2012
% of programs for new test methods
Average amount of time taken to complete an ILS (months)
Range of time taken to complete an ILS (months)
56
312
79
104
65
291
67
26
35
18
52
September 2012
Questions
57
ASTM International
Interlaboratory Study Program (ILS)
Phillip Godorov
Director, Interlaboratory Study Program
ASTM International
100 Barr Harbor Drive
West Conshohocken, PA 19428-2959
Phone: 610-832-9715
Fax:
610-834-7011
E-mail: [email protected]
58
September 2012
Thank you for your
participation!

Proficiency / Interlaboratory Crosscheck
Testing Programs

Interlaboratory Study Program (ILS)
59
September 2012