Transcript Document

ITRI
Industrial Technology
Research Institute
Evaluation of Proficiency Testing Results
with a Drifting Artifact –
An Example of Standard Resistor
Chen-Yun Hung
Center for Measurement Standards (CMS),
Industrial Technology Research Institute (ITRI)
Taiwan
July 31, 2014
2014 NCSL International Workshop and Symposium
1
Contents





Introduction
Stability Testing
Performance Evaluation
Confidentiality
Conclusions
2014 NCSL International Workshop and Symposium
2
Introduction




Proficiency testing (PT) is an effective way to verify a
laboratory’s measurement capability
Through PT activities, participants can understand the
differences between their measurement capabilities
and those of other laboratories
PT often leads to improvement in measurement
competence and quality control
In Taiwan, CMS/ITRI organizes the PT which focuses
on calibration laboratories
2014 NCSL International Workshop and Symposium
3
Why CMS/ITRI?

Industrial Technology Research Institute




The biggest non-profit organization in Taiwan
Half budget supported by the government
About 6,000 employees in most industrial fields
Center for Measurement Standards



CMS operates the National Measurement Laboratory
NML is the NMI in Taiwan (like a small NIST)
Maintain 120 national measurement standards

Quality Engineering Department organizes quality
management system in CMS

QED is also a PT Provider operated in accordance with ISO
17043, accredited by TAF (P002)
2014 NCSL International Workshop and Symposium
4
Certificate of accreditation
Accreditation Number
P002
2014 NCSL International Workshop and Symposium
5
Introduction (Cont.)

This paper will present a PT scheme for standard
resistors to demonstrate the evaluation of PT results
with a drifting artifact



A PT of standard resistors numbered “PT2012-KF01” was
performed from October 2012 to February 2013
A total of 16 laboratories participated
Two standard resistors with nominal values of 1 Ω and 10 kΩ
were chosen as PT items
Nominal
Value
1Ω
10 kΩ
Manufacturer
Model
Serial No.
iET
iET
SRL-1
SRL-10k
B2-9425105
B2-9425117
2014 NCSL International Workshop and Symposium
6
The flowchart of the PT scheme
Start
PT Orientation
PT Parameter
Sample Monitoring
PT Item
Data Retrieval
Stability Testing
Data Analysis
PT Plan
Final Report
PT Instruction
End
2014 NCSL International Workshop and Symposium
7
Stability Testing



Ensure the PT items will not undergo any significant
changes during the PT process
If not, the stability should be quantified and regarded
as an additional component of the uncertainty
The criterion for allowing stability in PT2012-KF01
usta  0.3 u
2
lab, min
u
2
ref
Based on the guidelines
for limiting the uncertainty
of the assigned value in
ISO 13528
where
usta is the standard uncertainty of stability testing
ulab, min is the minimum standard uncertainty of the participants’ results
uref is the standard uncertainty of the reference laboratory’s result
2014 NCSL International Workshop and Symposium
8
Stability Testing (Cont.)

The criterion for allowing stability in PT2012-KF01
usta  0.3 u

2
lab, min
u
2
ref
If the criterion is met
The stability uncertainty will not affect the evaluation of the
participants’ performance

If the criterion is not met
The stability should be regarded as an additional component of
the uncertainty associated with the assigned value
More than one assigned values would be provided to limit the
effect of the uncertainty on the performance evaluation
In some cases, PT items would be replaced based on technical
expert judgment
2014 NCSL International Workshop and Symposium
9
Stability Testing for PT2012-KF01

Two months for stability testing before the PT was initiated
Measured Value(Ω)
Measured Value(kΩ)
Nominal Value(1 Ω)
0.9991798
0.9991794
10.000755
Measured Value
Max = 0.9991793 Ω
min = 0.9991784 Ω
1Ω
Nominal Value(10 kΩ)
Measured Value
Max = 10.000753 kΩ
min = 10.000751 kΩ
10 kΩ
10.000754
10.000753
0.9991790
10.000752
0.9991786
Criterion for Allowing Stability
10.000751
PASS
0.9991782
10.000750
Date
Uncertainties
Nominal Value
1Ω
10 kΩ
2012/9/30
2012/9/28
2012/9/26
2012/9/24
2012/9/22
2012/9/20
2012/9/18
2012/9/16
2012/9/14
2012/9/12
2012/9/8
2012/9/10
2012/9/6
2012/9/4
2012/9/2
2012/8/31
2012/8/29
2012/8/27
2012/8/25
2012/8/23
2012/8/21
2012/8/19
2012/8/17
2012/8/15
2012/8/13
2012/8/9
2012/8/11
2012/8/7
2012/8/5
2012/8/3
10.000749
2012/8/1
2012/9/30
2012/9/28
2012/9/26
2012/9/24
2012/9/22
2012/9/20
2012/9/18
2012/9/16
2012/9/14
2012/9/12
2012/9/8
2012/9/10
2012/9/6
2012/9/4
2012/9/2
2012/8/31
2012/8/29
2012/8/27
2012/8/25
2012/8/23
2012/8/21
2012/8/19
2012/8/17
2012/8/15
2012/8/13
2012/8/9
2012/8/11
2012/8/7
2012/8/5
2012/8/3
2012/8/1
0.9991778
Date
usta
0.00000020
0.0000004
ulab, min
Ω
kΩ
0.000008
0.00005
2014 NCSL International Workshop and Symposium
uref
Ω
kΩ
0.00000015
0.0000015
Ω
kΩ
10
Assigned Value

Calibration values reported by NMI were used as
assigned values in this PT scheme


National Measurement Laboratory (NML)
PT items were calibrated for three times by NML to
ensure their stability

If the differences between the three calibration values are all
within the expanded uncertainty
Assigned Value: Median of three calibration values

If not
Assigned Value: Average of the calibration values
2014 NCSL International Workshop and Symposium
11
Assigned Value for PT2012-KF01
Measured
Value(Ω)
Nominal Value(1 Ω)
The differences between the
three calibration values were
larger than the expanded
uncertainty
0.9991812
0.9991808
0.9991804
1Ω
0.9991800
0.9991796
0.9991792
0.9991788
0.9991784
2012/10/01
E120519A
2012/12/07
E120641A
2013/03/08
E130072A
Date/Report No.

Assigned Value (X): Average of the calibration values

Two assigned values were provided
 One was the average of the first and second calibration values
 One was the average of the second and third calibration values

Expanded Uncertainty of Assigned Value (Uref)

Combine the reported uncertainty of the reference laboratory and
the uncertainty of the difference between two calibration values
2014 NCSL International Workshop and Symposium
12
Assigned Value for PT2012-KF01 (Cont.)
Measured
Value(kΩ)
Nominal Value(10 kΩ)
The differences between the
three calibration values were
smaller than the expanded
uncertainty
10.000760
10.000758
10 kΩ
10.000756
10.000754
10.000752
10.000750
10.000748
10.000746
10.000744
2012/10/01
E120520A
2012/12/07
E120642A
2013/03/08
E130073A
Date/Report No.


Assigned Value (X): Median of three calibration values
Expanded Uncertainty of Assigned Value (Uref)

The reported expanded uncertainty of the median
Group
Nominal Value
I
X
II
Uref
X
Uref
1Ω
0.9991795 Ω 0.0000004 Ω 0.9991801 Ω 0.0000005 Ω
10 kΩ
10.000752 kΩ 0.000003 kΩ 10.000752 kΩ 0.000003 kΩ
2014 NCSL International Workshop and Symposium
13
Performance Statistic

En was chosen as the performance statistic
En 
(x  X )
2
2
U lab
 U ref
where
x is the participant’s result
X is the assigned value
Ulab is the expanded uncertainty of the participant’s result
Uref is the expanded uncertainty of the reference laboratory’s assigned value

Criteria for performance evaluation

|En|  1, “satisfactory” performance

|En| > 1, “unsatisfactory” performance
2014 NCSL International Workshop and Symposium
14
PT Results for PT2012-KF01

PT results of the 1 Ω standard resistor in group I
Measured Value(Ω)
1.0030
1.0020
Nominal Value(1 Ω)
Measured Value
Assigned Value = 0.9991795 Ω
|En| > 1
1.0010
1.0000
0.9990
0.9980
(x – X) & Ulabis large
0.9970
0.9960
0.9950
Ref Lab-I
C
D
E
G
K
L
O
Lab Code
2014 NCSL International Workshop and Symposium
15
PT Results for PT2012-KF01 (Cont.)

PT results of the 1 Ω standard resistor in group II
Measured Value(Ω)
1.0000
0.9998
Nominal Value(1 Ω)
Measured Value
Assigned Value = 0.9991801 Ω
0.9996
0.9994
0.9992
0.9990
0.9988
0.9986
0.9984
Ref Lab-
A
B
F
II
H
I
J
M
N
P
Lab Code
Note: The value of Lab J is not available.
2014 NCSL International Workshop and Symposium
16
PT Results for PT2012-KF01 (Cont.)

PT results of the 10 kΩ standard resistor
Nominal Value(10 kΩ)
Measured Value(kΩ)
10.0060
10.0050
Measured Value
Assigned Value = 10.000752 kΩ
10.0040
10.0030
10.0020
10.0010
|En| > 1
10.0000
9.9990
9.9980
9.9970
9.9960
• CMS/ITRI
only
provided
the
PT
results
and
suggestions
Ref
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Lab Code
• Did not judge whether the participants
were qualified laboratories
2014 NCSL International Workshop and Symposium
17
Confidentiality

Each laboratory is given a code that is mailed to the laboratory
in a sealed envelope


ISO/IEC 17043 4.10.1
The identity of participants shall be confidential
and known only to persons involved in the
operation of the PT scheme
Only knows its own laboratory code
The confidentiality should especially be considered when the
assigned value of a drifting artifact is determined

More than one assigned values are provided
The number of assigned values should be controlled to avoid fewer
participants in one group

Using linear regression to predict the assigned value on a given
day
The transfer schedule is not publicly available
PT items should be transferred to participants by PT provider
2014 NCSL International Workshop and Symposium
18
Conclusions

CMS/ITRI has performed PTs in Taiwan for more than 10
years with the focus on calibration laboratories

Ensuring the stability of artifacts always has been given a
high priority

CMS/ITRI shares a PT of standard resistors with quantified
criterion for stability testing using statistical methods

Most of standard resistor calibration laboratories in Taiwan
have good measurement competence

Although the PT items drift over time, it will not affect the
evaluation of the participants’ performance by a welldesigned method
2014 NCSL International Workshop and Symposium
19
ITRI
Industrial Technology
Research Institute
Thanks for your attention
Chen-Yun Hung
Email: [email protected]
2014 NCSL International Workshop and Symposium
20