Transcript Overview – Courses - STT

Statistics & graphics
for the laboratory
Applications
Analytical quality
Dietmar Stöckl
[email protected]
Linda Thienpont
[email protected]
In cooperation with AQML: D Stöckl, L Thienpont &
• Kristian Linnet, MD, PhD
[email protected]
• Per Hyltoft Petersen, MSc
[email protected]
• Sverre Sandberg, MD, PhD
[email protected]
Prof Dr Linda M Thienpont
University of Gent
Institute for Pharmaceutical Sciences
Laboratory for Analytical Chemistry
Harelbekestraat 72, B-9000 Gent, Belgium
e-mail: [email protected]
STT Consulting
Dietmar Stöckl, PhD
Abraham Hansstraat 11
B-9667 Horebeke, Belgium
e-mail: [email protected]
Tel + FAX: +32/5549 8671
Copyright: STT Consulting 2007
Statistics & graphics for the laboratory
2
Content
Content
Metrology
Analytical quality specifications ("Goals")
Method evaluation/method comparison
Statistics & graphics for the laboratory
3
Content
Detailed content
Metrology
•
•
•
•
Terminology & definitions
Metrological concepts (Error, Accuracy, Uncertainty)
Measurement traceability
Conclusion
Annex
•
•
•
•
•
•
•
•
•
•
Glossary
Système International d'Unités: base units
Metrological concepts
Traceability requirements of Directive 98/79/EC
How to meet the traceability requirement?
The elements of a reference measurement system
Validation of metrologically traceable calibration
What if SI-traceability does not apply?
Traceability – to which extent?
Additional references
Analytical quality specifications ("Goals")
• Introduction
• Concepts
– Clinical concepts
– Questionnaires to clinicians
– Goals from biology
– Goals from experts
– State-of-the-art
• Comparison of goals
• Comparison "state-of-the-art" with goals
• Analytical goals – Translation into practice
• Goals – future vision
• Outlook
• References
Statistics & graphics for the laboratory
4
Content
Detailed content
Method evaluation/method comparison
Introduction
• The analytical quality triangle
• Purpose of method evaluation
Performance characteristics of a methodPrecision
• Limit of detection
• Working range …
Method evaluation strategies
Assessment of performance characteristics
• Specific protocols
• Stability/ruggedness
• Multifactor protocols
• Method comparison
• Summary of protocols, statistics & graphics
• Method comparison – The stable basis
Exercises with EXCEL-file
References
Interactive part
• Factors that influence the interpretation of a method comparison: qualitative
use of a difference (Bland & Altman) plot
Final remark
• When to use regression-based interpretation
Exercises
• Case studies 1 - 5
Statistics & graphics for the laboratory
5
Content
Metrology
• Terminology & definitions
• Metrological concepts
– Error
– Accuracy
– Uncertainty
– Nature and measures of error
• Measurement traceability
Statistics & graphics for the laboratory
6
Terminology and definitions
Terminology and definitions
For understanding metrology ("science of measurement"), one needs to know a lot
of definitions and to use the correct metrological terms (quantity; unit; traceability;
accuracy; uncertainty; …). A few of them will be presented in this part, however,
most of them can be found in the Annex.
The measurement
Measurement (in general)
• process of experimentally obtaining one or more quantity values that can
reasonably be attributed to a quantity
Quantity and value
i. Quantities are length, mass, amount-of-substance, time, temperature, etc.
ii. The value of a quantity is expressed by both a number and a unit
The actual measurement
We determine the value of a particular quantity. Example of a particular quantity:
amount-of-substance concentration of glucose in plasma.
The particular quantity is specified by 3 elements:
• System: Plasma
• Component (also called analyte): Glucose
• Kind-of-quantity: Amount-of-substance concentration
NOTE: The particular quantity subject to measurement is called measurand.
Measurand: quantity intended to be measured.
A full measurement report
The full report of a measurement result for glucose would read: “the amount-ofsubstance concentration of glucose in plasma was 5.2 (= number) mmol/L (unit)”
Measurement units
We have to distinguish between SI-units and International units.
Système International d’Unités (SI):
• Only meaningful in connection with components (analytes) whose elementary
entity can be recognized by full physicochemical characterization.
• For components where the elementary entity is not known (identity and/or purity
not known), arbitrary units have to be used to express quantities.
International Units (IUs):
• In case that an internationally accepted calibrator and/or measurement
procedure are available, the quantity of the above components can be expressed
in “IUs”.
• Characteristically, IUs depend on the measurement procedure and the calibrator
used, in contrast to SI units, which are independent thereof.
Units must be "materialized" in standards !
Statistics & graphics for the laboratory
7
Terminology and definitions
What is a standard?
(Measurement) standard: realization of the definition of a given quantity,
with stated quantity value and measurement uncertainty, used as a reference.
Usually, measurements have to be calibrated with standards.
Calibration of measurement
Calibration: operation that, under specified conditions, in a first step establishes
a relation between the quantity values with measurement uncertainties provided
by measurement standards and corresponding indications with associated
measurement uncertainties and, in a second step, uses this information to
establish a relation for obtaining a measurement result from an indication.
>Calibration relates the measured values of particular quantities with the
values
defined by standards.
Example:
Measurement of the amount-of-substance concentration
of sodiumAmount-of-substance
in serum
Example:
concentration of serum sodium.
Defined analyte
SI-Unit:
mol/L
Measured analyte
Correct
value/matrix
Measurement procedure
Signal
Standard
SRM 919a
Calibrator
Measuring function
Accurate
value
Measurement result
135 mmol/L
Sodium
in serum
Proper calibration is the major factor for "overall" trueness. The main input
elements for proper calibration are analyte preparations with certified content and
matrix-free or matrix-corrected calibration solutions.
The actual calibration needs to be controlled by:
• Written calibration protocols
• Sufficient calibration intervals
• Documentation of calibration status
These elements should be covered by the quality system of a laboratory.
Statistics & graphics for the laboratory
8
Metrological concepts
Quality of measurements
The 3 metrological concepts from ISO
Error concept: BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, OIML. Vocabulaire
International des Termes Fondamentaux et Généraux de Métrologie. 3rd ed.
Geneva: ISO, 2007.
Accuracy concept: International Organization for Standardization. Accuracy
(trueness and precision) of measurement methods and results. Part 1: General
principles and definitions, ISO 5725-1. 1st ed. Geneva: ISO, 1994.
Uncertainty concept: International Organization for Standardization. Guide to
the expression of uncertainty in measurement. 1st ed. Geneva: ISO, 1993.
Important terms related to the 3 concepts
Error
concept
Accuracy
concept
Uncertainty
concept
True value
Accepted reference
value
Value
Systematic error (SE)
Trueness
All SE-components
corrected!
Random error (RE)
Precision
Precision (types A & B)
[In]Accuracy (SE & RE)
[Total error]
Accuracy
(Trueness & Precision)
[Combined] Uncertainty
Graphical presentation of measurement quality
Trueness
Good
Bad
Good
Precision
Bad
Good
Precision
Bad
Because the uncertainty concept is relatively new in analytical chemistry, some of
its main features will be explained below.
Statistics & graphics for the laboratory
9
Metrological concepts
Measurement uncertainty (GUM)
Uncertainty
Parameter characterizing the dispersion of the quantity values being attributed to a
measurand, based on the information used.
The GUM philosophy
• Deprecates the terms "error" and "truth"
• Deprecates the distinction between "systematic" and "random"
• Requires the correction of all "biases"
Systematic versus random
Every calibration introduces a small
systematic error. Over time, however,
these errors should be Normal
distributed and can be viewed as
random calibration bias:
>Classification systematic/random
may depend on the observation time!
Propagation of systematic error
GUM
Correct: after correction,
[random] uncertainty remains
Classical
Errors are summed
(sign is respected)
Sum
1
2
3
What is GUM all about?
It is about propagation of random errors!
Calculation may require statistical experience (see below).
For a detailed treatise of the GUM concept, inclusive examples see:
EURACHEM/CITAC Guide: Quantifying uncertainty in analytical measurement
(http://www.measurementuncertainty.org/mu/guide/index.html).
Statistics & graphics for the laboratory
10
Metrological concepts
Rules for propagation of random error
The standard deviation (s) of calculated results (propagation of s)
1. Sums and differences
y = a(±sa) + b(±sb) + c(±sc)  sy = SQRT[sa2 + sb2 + sc2] (SQRT = square root)
Do not propagate CV!
2. Products and quotients
y = a(±sa) • b(±sb) / c(±sc)  sy/y = SQRT[(sa/a)2 + (sb/b)2 + (sc/c)2]
3. Exponents (the x in the exponent is error-free)
y = a(±sa)x  sy/y = x • sa/a
For GUM, see also:
http://physics.nist.gov/cuu/Uncertainty/index.html
http://physics.nist.gov/Pubs/guidelines/
http://www.measurementuncertainty.org
http://www.westgard.com/guest41.htm
Thienpont LM. Calculation of measurement uncertainty–why bias should be
treated separately. Clin Chem 2008;54:1587-8.
Uncertainty of the final result
Input variables
• Calibration traceability & lot variation
• Linearity limits
• Recovery limits
• Reagent lot-/instrument variation
• Drift limits (recommended IQC procedure)
• Specificity (“cross-reactivity”)
• Interference limits (all kinds; continuous update): variation in sample matrix;
common (lipemia, etc.); drug effects; auto-/heterophilic antibodies; genetic variants
Note that, currently, NOT all can be included in a GUM uncertainty statement!
In particular, statistical concepts are missing of how to deal with unspecificity and
interferences.
Statistics & graphics for the laboratory
11
Metrological concepts
Metrological concept used in the course
Here, we apply the error concept, including the statistical uncertainty of the
experimental estimates of error [Stöckl D. Scand J Clin Lab Invest 1996;56:193-7]
Concept
Systematic error (SE)
<
[Total] Error
>
Random error (RE)
Alternative terms
("Un")Trueness#
<
(In)Accuracy
>
(Im)Precision
Experimental
estimates
Bias ±
Confidence limits
Bias & s
± Confidence limits
s
± Confidence limits
#Un-trueness is not defined by VIM
Nature & measures of error
Random error (RE)
• Quantitative expression: SD/CV
Systematic error (SE)
• Quantitative expression: Bias
– Constant
– Proportional
Random & systematic error
• Quantitative expression: Total error (TE)/Uncertainty
Statistics & graphics for the laboratory
12
Metrological concepts
Random error
Nature of imprecision
• Imprecision is an inevitable characteristic of each measurement. It results in a
characteristic dispersion of results (usually Gaussian) of repeated measurements,
even when carried out under apparently identical test conditions.
• The standard deviation (or coefficient of variation) is a direct measure for
imprecision.
Note: Characteristic of imprecision is that results always lie on both sides (±) of a
mean value.
Input factors for good precision are:
• Sufficient pipetting volume
• Positive displacement volumetric equipment
• Gravimetric control of volumes
• Good signal-to-noise ratio
• Adequate scales
• Sufficient reported digits
• Automatisation (and quality thereof)
• Control of environment
Total analytical imprecision (sa,tot) comprises
• Sampling (ssam),
• Sample preparation (sprep), and
• Measurement (smeas)
Total analytical imprecision is calculated as:
• sa,tot = SQRT[(ssam)2 + (sprep)2 + (smeas)2] (SQRT = square root)
The total imprecision is dominated by the step with the highest imprecision
For automatic analyzers, these 3 are not distinguished:  then, imprecision is
understood to consist of all 3 components.
Evaluation of measurement imprecision makes a difference between:
• Total
• Between-day (between-run)
• Within-day (within-run)
Note: To be representative, the within-day imprecision has to be calculated from
cumulated data of several days.
Common situation
CV constant/SD decreasing down
to a certain concentration, then
SD constant and CV increasing
SD/CV
SD and CV in analytical practice
CV
SD
Concentration
Statistics & graphics for the laboratory
13
Metrological concepts
Systematic error
(Un)trueness (systematic error) of analytical methods may have a variety of
origins:
• Calibration errors
• Unspecificity
• Interferences
• System drift/shift
• Carryover
A measure for (un)trueness/ systematic error is the bias, observed as deviation of
the actual measurement result from a target (or expected value, or reference
value).
Control of systematic errors
• Reference methods and materials
• Apply proper calibration (major factor for overall analytical trueness)
• Adequate data processing (e.g., calibration function)
• Select measurement principle that is insensitive to interferences and specific
and/or apply sample purification
• Select stable, high quality instrumentation (low drift/shift/carryover)
• Apply internal quality control
Note: Characteristic of (un)trueness is that results tend to lie on either side of a
target value (either + or -). Opposite to imprecision, bias can be obviated.
Types of systematic error
• Constant &
• Proportional
Method y
30
20
y=x
Constant
error
Proportional
error
10
0
0
10
20
Method x
30
Statistics & graphics for the laboratory
14
Metrological concepts
Total error
Graphical presentation of the total error
From: Westgard JO, Barry
PL. Cost-effective quality
control: managing the quality
and productivity of analytical
processes. 4th printing.
Washington: AACC Press,
1995
Components of total error (TE)
TE comprises
• Imprecision
• Calibration errors (including non-linearity)
• Unspecificity
• Interference (all kinds)
• Carryover
• System instability (drift; shift)
Measures of total error (TE)
We can distinguish 2 situations for the measure of TE
TE for a single measurement:
TE = SE + z • RE, or TE = SE + z • s
z is usually set to 1.96 or 2.58, encompassing 95% or 99% [two-tailed] of a
gaussian distributed population.
Note: s shall be sufficiently reliable (n  30)
TE for multiple measurements:
TE = SE + z • [s/n], or TE = SE + tn-1 • [s/n]
Note that the formula uses the confidence interval for the actual number of
measurements (n) (instead of z • s).
If s is derived from at least 30 replicate measurements, z (at a given confidence
level) is used; if s is derived from the n actual measurements, t n-1 = the student's tvalue for (n-1) degrees of freedom is used.
Statistics & graphics for the laboratory
15
Measurement traceability
Measurement traceability
From the definition of a measurement:
• "Determine the value of a quantity"
• The value of a quantity is expressed by both a number and a unit
• The unit must be meaningful
one can infer that an actual measurement result should be traceable [preferably]
to the SI-unit. Recently, the process of establishment of traceability has been
formalized.
Measurement traceability – The regulations
European Directive 98/79/EC
• Traceability requirement
EN/ISO 17511:2003. In vitro diagnostic medical devices – Measurement of
quantities in biological samples – Metrological traceability of values assigned to
calibrators and control materials
• Traceability establishment
Selected contents of EN/ISO 17511
4 Metrological traceability chain and calibration hierarchy
6 Expression of uncertainty of measurement
5 Calibration transfer protocols
7 Validation of metrologically traceable calibration
Traceability chains
ISO 17511 elaborates 5 different traceability models (see table), depending on the
definition of the analyte and the existence of a reference measurement system.
Case
RMP
Reference
material
Example
1 (SI)
X
(primary)
X (primary)
Cortisol
2 (non-SI)
X
X
3 (non-SI)
X
–
Hemostatic
factors
4 (non-SI)
–
X
hCG
5 (non-SI)
–
–
Tumor
markers
Statistics & graphics for the laboratory
16
Measurement traceability
Metrological traceability chain (SI) (ISO 17511)
Material
Procedure
í Value assignment
î Calibration
Definition of SI-unit
Primary calibrator
Secondary reference
measurement procedure
Manufacturer’s
working calibrator
Manufacturer’s standing
measurement procedure
Manufacturer’s
product calibrator
End user’s routine
measurement procedure
Uncertainty
Traceability
Primary reference
measurement procedure
Routine sample
Result
The metrological traceability chain starts with the definition of the analyte and its
unit. The unit must be materialized in a standard. The unit is traced to the routine
measurement procedure by a cascade of hierarchically different materials and
methods. The methods are used for value assignment and the materials are used
for calibration according to the above scheme.
Example testosterone
ng/ml
[nmol/l preferred!]
Gravimetry
Merck Testosterone
[No primary calibrator!]
ID-GC/MS
Working calibrator
Human sera
Immunoassay
master procedure
Product calibrator
6 calibrators
Immunoassay
end user’s procedure
Routine sample
ng/ml
[nmol/l preferred!]
Statistics & graphics for the laboratory
17
Measurement traceability
Testosterone - Outcome
Calibration traceability
Method comparison with 50
human samples
Assess regression estimates vs
17511 specifications:
• Slope = 1
• Intercept = 0 (“some tolerance”)
Result traceability
Variation around line: “17511 alternative”
 Manufacturer limit: maximum allowable deviation
Analytical specifications from biological variation (www.westgard.com)
• CV
= 4.7%
• Bias
= 6.4%
• Total error = 14% (z = 1.65 for imprecision)
Suppose the manufacturer selects the TE goal for the validation study = 14%
Variation around line, manufacturer limit
• 14% (“biology”) down to 5 nmol/l
• At 5 nmol/l constant absolute deviation, i.e. 14% of 5 nmol = 0.7 nmol
Statistics & graphics for the laboratory
18
Measurement traceability
Traceability – To which extent?
Currently, no generally accepted answer, only models how to generate the
"numbers".
Models, in hierarchical order#
• Clinical concepts
• Concepts based on biological variation
• Expert opinion
• Regulations
• "State-of-the-art"
#Consensus Statement (Stockholm 1999). Scand J Clin Lab Invest 1999;59:585.
Statistics & graphics for the laboratory
19
Conclusion
Which quality?
• Document your own quality
• Compare it with the “state-of-the-art”  Identify therefrom “problem analytes”
• Compare it with international goals  Identify therefrom “problem analytes” or
“problem goals”
• Identify “problem analytes” by communication with the laboratories/clinicians
• Identify “possible” problem analytes by comparison with “biological”
specifications: biological variation, reference intervals
• Verify “possible” problem analytes by comparison with “clinical” specifications
• Check whether you want to improve the quality of some tests, independent of
proposed goals
Conclusion
For valid measurements … adhere to the analytical quality triangle!
Statistics & graphics for the laboratory
20
Notes
Notes
Statistics & graphics for the laboratory
21
Annex - Metrology
Content
Glossary
Système International d'Unités: base units
Metrological concepts
Traceability requirements of Directive 98/79/EC
How to meet the traceability requirement?
The elements of a reference measurement system
Validation of metrologically traceable calibration
What if SI-traceability does not apply?
Traceability – to which extent?
Additional references
Statistics & graphics for the laboratory
22
Annex - Metrology
Glossary
Metrology [1]
field of knowledge concerned with measurement
Measurand [1]
quantity intended to be measured
Quantity [1]
property of a phenomenon, body, or substance, to which a number can be assigned with
respect to a reference
Measurement [1]
process of experimentally obtaining one or more quantity values that can reasonably be
attributed to a quantity
Notes:
• Quantities are length, mass, amount-of-substance, time, temperature, etc.
• The value of a quantity is expressed by both a number and an unit
• The full specification of the quantities measured in the medical laboratory comprises
three elements:
System (e.g., blood plasma)
Component (also called analyte) (e.g., glucose)
Kind-of-quantity (e.g., amount-of-substance concentration)
The full report of a glucose measurement would read: “the amount-of-substance
concentration of glucose in blood plasma was 5.2 mmol/L”
Measurement unit [1]
scalar quantity, defined and adopted by convention, with which any other quantity of the
same kind can be compared to express the ratio of the two quantities as a number
Value of a quantity [1]
number and reference together expressing magnitude of a quantity
EXAMPLE: Length of a given rod: 5.34 m
Measurement standard [1]
realization of the definition of a given quantity, with stated quantity value and
measurement uncertainty, used as a reference
EXAMPLE: 1 kg mass standard.
Statistics & graphics for the laboratory
23
Annex - Metrology
Glossary
Error [1]
difference of measured quantity value and reference quantity value
Systematic error [1]
component of measurement error that in replicate measurements remains constant or
varies in a predictable manner
Bias [1]
systematic measurement error or its estimate, with respect to a reference quantity value
Random error [1]
component of measurement error that in replicate measurements varies in an
unpredictable manner
Trueness [1]
closeness of agreement between the average of an infinite number of replicate measured
quantity values and a reference quantity value
Accuracy [1]
closeness of agreement between a measured quantity value and a true quantity value of
the measurand
Precision [1]
closeness of agreement between indications obtained by replicate measurements on the
same or similar objects under specified conditions
Repeatability condition [1]
condition of measurement in a set of conditions that includes the same measurement
procedure, same operators, same measuring system, same operating conditions and
same location, and replicate measurements on the same or similar objects over a short
period of time
Reproducibility condition [1]
condition of measurement in a set of conditions that includes different locations,
operators, measuring systems, and replicate measurements on the same or similar
objects
Uncertainty [1]
parameter characterizing the dispersion of the quantity values being attributed to a
measurand, based on the information used
[Metrological] Traceability [1]
property of a measurement result whereby the result can be related to a stated reference
through a documented unbroken chain of calibrations, each contributing to the
measurement uncertainty
Statistics & graphics for the laboratory
24
Annex - Metrology
Glossary
Commutability [of a reference material] [1]
property of a reference material, demonstrated by the closeness of agreement between
the relation among the measurement results for a stated quantity in this material, obtained
according to two given measurement procedures, and the relation obtained among the
measurement results for other specified materials
Matrix effect [2]
Influence of a property of the sample, other than the measurand, on the measurement of
the measurand according to a specified measurement procedure and thereby on its
measured value [2]
Influence quantity [1]
quantity that, in a direct measurement, does not affect the quantity that is actually
measured, but affects the relation between the indication and the measurement result
Note: Specificity & Interference are not yet unequivocally defined by ISO.
Selectivity [1]
capability of a measuring system, using a specified measurement procedure, to provide
measurement results, for one or more measurands, that do not depend on each other nor
on any other quantity in the system undergoing measurement (= specificity in chemistry)
Interference [in analysis]
A systematic error in the measure of a signal caused by the presence of concomitants in
a sample (http://goldbook.iupac.org)
specific [in analysis]
A term which expresses qualitatively the extent to which other substances interfere with
the determination of a substance according to a given procedure. Specific is considered
to be the ultimate of selective, meaning that no interferences are supposed to occur
(http://goldbook.iupac.org).
Calibration [1]
operation that, under specified conditions, in a first step establishes a relation between
the quantity values with measurement uncertainties provided by measurement standards
and corresponding indications with associated measurement uncertainties and, in a
second step, uses this information to establish a relation for obtaining a measurement
result from
an indication
Sensitivity [1]
quotient of the change in the indication and the corresponding change in the value of the
quantity being measured
Statistics & graphics for the laboratory
25
Annex - Metrology
Glossary
Linear range
Concentration range over which the intensity of the signal obtained is directly proportional
to the concentration of the species producing the signal (http://goldbook.iupac.org).
Linearity (generic)
Ability of an analytical procedure to produce test results which are proportional to the
concentration (amount) of an analyte, either directly or by means of a well-defined
mathematical transformation.
Working interval [1]
set of values of the quantities of the same kind that can be measured by a given
measuring instrument or measuring system with specified instrumental uncertainty, under
defined conditions
Limit of detection (in analysis)
The limit of detection, expressed as the concentration, cL, or the quantity, qL, is derived
from the smallest measure, xL, that can be detected with reasonable certainty for a given
analytical procedure. The value of xL is given by the equation xL = xbi + k • sbi, where xbi
is the mean of the blank measures, sbi is the standard deviation of the blank measures,
and k is a numerical factor chosen according to the confidence level desired
(http://goldbook.iupac.org).
Limit of detection [1]
measured quantity value, obtained by a given measurement procedure, for which the
probability of falsely claiming the absence of a component in a material is β, given a
probability α of falsely claiming its presence
Ruggedness (generic)
Ability to reproduce the method in different laboratories or in different circumstances.
Ruggedness (USP)
Degree of reproducibility of the results obtained under a variety of conditions, expressed
as %RSD. These conditions include different laboratories, analysts, instruments,
reagents, days, etc.
Robustness (ICH Q2A 1995)
The robustness of an analytical procedure is a measure of its capacity to remain
unaffected by small, but deliberate variations in method parameters and provides an
indication of its reliability during normal usage.
[1] BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, OIML. Vocabulaire International des Termes
Fondamentaux et Généraux de Métrologie. 3rd ed. Geneva: ISO, 2007.
[2] EN/ISO 17511:2003. In vitro diagnostic medical devices – Measurement of quantities
in biological samples – Metrological traceability of values assigned to calibrators and
control materials.
[3] See also: www.clsi.org>Harmonized Terminology Database
Statistics & graphics for the laboratory
26
Annex - Metrology
Système International d'Unités: base units
Quantity
Name
Symbol
Length
meter
m
Mass
kilogram
kg
Time
second
s
Electric current
ampere
A
Thermodynamic
temperature
kelvin
K
Amount of substance
mole
mol
Luminous intensity
candela
cd
Catalytic amount
katal
kat
Metrological concepts
• Error concept [1]
• Trueness concept [2]
• Uncertainty concept [3]
[1] [1] BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, OIML. Vocabulaire International des
Termes Fondamentaux et Généraux de Métrologie. 3rd ed. Geneva: ISO, 2007.
[2] International Organization for Standardization. Accuracy (trueness and
precision) of measurement methods and results. Part 1: General principles and
definitions, ISO 5725-1. 1st ed. Geneva: ISO, 1994.
[3] International Organization for Standardization. Guide to the expression of
uncertainty in measurement. 1st ed. Geneva: ISO, 1993.
Statistics & graphics for the laboratory
27
Annex - Metrology
Traceability requirements of Directive 98/79/EC
Some excerpts
ANNEX I: Essential Requirements
A. General requirements
3. The devices must be designed and manufactured ... taking account of the
generally acknowledged state of the art. They must achieve the performances, in
particular, where appropriate, in terms of analytical sensitivity, diagnostic
sensitivity, analytical specificity, diagnostic specificity, accuracy, repeatability,
reproducibility, including control of known relevant interference, and limits of
detection, stated by the manufacturer.
The traceability of values assigned to calibrators and/or control materials must be
assured through available reference measurement procedures and/or available
reference materials of a higher order.
B. Design and manufacturing requirements
4. Devices which are instruments or apparatus with a measuring function
4.1. Devices which are instruments … must be designed and manufactured in
such a way as to provide … accuracy of measurement within appropriate accuracy
limits, taking into account … available and appropriate reference measurement
procedures and materials.
8. Information supplied by the manufacturer
8.7. Where appropriate, the instructions for use must contain the following
particulars:
(h) the measurement procedure to be followed with the device including as
appropriate:
- … information about the use of available reference measurement procedures
and materials by the user;
(k) information appropriate to users on:
the traceability of the calibration of the device;
ANNEX III: EC Declaration of Conformity
3. The technical documentation ... must include in particular:
- adequate performance evaluation data showing the performances claimed by
the manufacturer and supported by a reference measurement system (when
available), with information on the reference methods, the reference materials, …
Statistics & graphics for the laboratory
28
Annex - Metrology
How to meet the traceability requirement?
EN/ISO 17511:2003
In vitro diagnostic medical devices – Measurement of quantities in biological
samples – Metrological traceability of values assigned to calibrators and control
materials
Some excerpts related to the traceability chain and how it works
Introduction
Objective of applying the traceability chain
… is to transfer the degree of trueness of a reference material, and/or reference
measurement procedure, to a procedure that is of a lower metrological order, e.g.
a routine procedure
Trueness of measurement
… depends on the metrological traceability of the value through an unbroken chain
of alternating measurement procedures and measurement standards (calibrators),
usually having successively decreasing uncertainties of measurement.
Uncertainty of the value
… depends on the stated metrological traceability chain and the combined
uncertainties of its links.
The measurement of quantities in biological samples requires reference
measurement systems including:
• the definition of the analyte in the biological sample with regard to the intended
clinical use of the measurement results;
• a reference measurement procedure for the selected quantity in human samples;
• suitable reference materials for the selected quantity, e.g. primary calibrators and
secondary matrix-based calibrators that are commutable.
• To ensure the validity of a metrological traceability chain, the quantity shall be the
same at all levels.
Statistics & graphics for the laboratory
29
Annex - Metrology
The elements of a reference measurement system
EN/ISO 17511:2003
4.2.2 a) Definition of measurand including SI unit of measurement
4.2.2 b) Primary reference measurement procedure$
shall be based on a principle of measurement proved to be analytically specific,
providing metrological traceability to an SI unit of measurement without reference
to a calibrator for the same quantity, and having a low uncertainty of measurement
(isotope dilution-mass spectrometry, coulometry, gravimetry, titrimetry).
$ Typical examples of measurement principles
Method principle
Analyte
ID-MS
Ca, K
ID-GC/MS
Cholesterol, glucose, creatinine,
… steroid- & thyroid hormones,
drugs
FAES
K, Na
AAS
Ca, Mg, Li, Ni
Ion chromatography
Phosphate
Ion exchange-gravimetry
Na
LC-MS, CZE after HPLC
HbA1c
Potentiometry
pH
Coulometry
Cl
Absorption photometry
Hb, bilirubin, cholesterol, protein
See also: IFCC. www.ifcc.org.
BIPM. Joint Committee on Traceability in Laboratory Medicine (JCTLM)
. www1.bipm.org/en/committees/jc/jctlm.
4.2.2 c) Primary calibrator
that is an embodiment of the unit of measurement with the smallest achievable
uncertainty of measurement. The primary calibrator shall have its value assigned
either directly by a primary reference measurement procedure or indirectly by
determining the impurities of the material by appropriate analytical methods. The
material usually is highly purified containing a physico-chemically well-defined
analyte, examined for stability, compositional integrity, and accompanied by a
certificate (certified reference material, CRM).
Statistics & graphics for the laboratory
30
Annex - Metrology
The elements of a reference measurement system (ctd)
4.2.2 d) Secondary reference measurement procedure
shall describe a measuring system which is calibrated by one or more primary
calibrators…
See, also: EN/ISO 15193:2002
In vitro diagnostic medical devices – Measurement of quantities in biological
samples – Requirements and layout of reference measurement procedures
[Secondary] Reference measurement procedure
Thoroughly investigated measurement procedures shown to have an uncertainty
of measurement commensurate with their intended use, especially in assessing
the trueness of other measurement procedures for the same quantity and in
characterizing reference materials.
Validation of metrologically traceable calibration
EN/ISO 17511:2003
7.1 The following conditions shall apply to the concept of metrologically
traceable calibration:
a) The reference and routine measurement procedures measure the same
quantity.
b) The mathematical relationship between the measurement results generated by
the routine procedure and the measurement results generated by a higher order
measurement procedure, is the same for all relevant human samples.
c) The mathematical relationship between the measurement results generated by
the measurand in a given calibrator using the reference and the routine procedure
is the same as the relationship expected for measurands in routine human
samples. This assumption has been termed commutability of the reference
material (see 3.9).
NOTE 1 The object of the use of metrologically traceable calibrators in routine
measurement procedures, such as those of in vitro diagnostic medical devices, is
to produce a result of measurement of the measurand that is as close as required
to that which would have been obtained if the reference measurement procedure
to which the calibrators are metrologically traceable had been applied to the same
samples. Thus, the trueness of results given by a calibrated routine measurement
procedure derives from that of the reference measurement procedure when such
is available.
NOTE 2 When the conditions a), b), and c) do not apply, the use of a
manufacturer's product calibrator with assigned value cannot guarantee that the
routine results are metrologically traceable to the reference measurement
procedure.
7.2 The commutability of the manufacturer's working calibrator(s) … shall be
assessed by the manufacturer applying both reference measurement procedure
… and the routine measurement procedure …to the manufacturer's working
calibrator and to a set of relevant human (routine) samples.
Statistics & graphics for the laboratory
31
Annex - Metrology
Validation of metrologically traceable calibration (ctd)
7.3 The commutability of the manufacturer's product calibrator, shall be
demonstrated by comparing the results of measurements, made by both the
reference procedure and the calibrated routine procedure on a set of actual
samples of a type to which the routine measurement procedure is intended to be
applied.
The samples shall be authentic, preferably single-donation and unspiked human
samples …
7.4 …For metrological traceability to be achieved, the results by the routine
procedure shall be related to those of the reference procedure by, e.g. a linear
regression of unit slope and zero intercept with a stated probability.
NOTE If linear regression is used,the observed value of the slope should be
stated, including its uncertainty.
A unit slope is expected but a deviation from unit slope within a stated interval of
quantity values may be tolerable. …
The observed value of the intercept should be stated. If a value significantly
different from zero at a given probability is considered tolerable, the reasons for
this shall be stated…An intercept on the axis of the routine measurement
procedure significantly different from zero can indicate a difference of analytical
specificity between the two procedures, which could invalidate the principle of
metrological traceability.
The expected variability of comparison around the regression line (prediction
limits) may be estimated at a given probability on the basis of the number of
samples and the respective uncertainties of the two measurement procedures.
Variations greater than this indicate an aberrant-sample-dependent variability in
the inter-procedure relationship that invalidates metrologically traceable routine
results for certain samples. Alternatively, a limit of maximum allowable relative
variation between results by the reference and calibrated routine procedures may
be specified by the manufacturer….
7.5 If a panel of human samples is used as part of the process of assigning a
value to the manufacturer's product calibrator, the same panel shall not be used
also to validate metrological traceability.
Statistics & graphics for the laboratory
32
Annex - Metrology
What if SI-traceability does not apply?
EN/ISO 17511:2003
Introduction
… but the selection of steps and the level at which metrological traceability for a
given value stops, depend on the availability of higher order measurement
procedures and calibrators. In many cases, at present, there is no metrological
traceability above the manufacturer's selected measurement procedure or the
manufacturer's working calibrator. In such cases, trueness is referred to that level
of the calibration hierarchy until an internationally agreed reference measurement
procedure and/or calibrator becomes available.
Laboratory medicine routinely provides results for 400 to 700 types of quantity…
Depending on the possibility of metrological traceability to SI and on the
availability of various metrological levels of measurement procedures and
calibrators, the following five typical upper ends of the metrological traceability
chain can be identified.
a) Quantities for which results of measurements are metrologically traceable to SI.
b) Quantities for which results of measurements are not metrologically traceable to
SI (4 cases)…
Traceability – to which extent?
No numbers available!
•… generally acknowledged state of the art. They must achieve the performances,
…, stated by the manufacturer (IVD-Directive 98/79/EC)
•… will depend on the state of development of methods of measurement and the
medical uses to which the results are to be applied (EN/ISO 17511:2003)
•… should reflect the medical use (e.g. based on biological variation or other
means) and the "state-of-the-art" of the quality of the IVD MDs (EN/ISO
14136:2004)
Additional references
• Petersen PH, Stöckl D, Westgard JO, Sandberg S, Linnet K, Thienpont L.
Models for combining random and systematic errors. Assumptions and
consequences for different models. Clin Chem Lab Med 2001;39:589-95.
• Thienpont LM, Van Uytfanghe K, De Leenheer AP. Reference measurement
systems in clinical chemistry. Clin Chim Acta 2002;323:73-87.
Statistics & graphics for the laboratory
33
Notes
Notes
Statistics & graphics for the laboratory
34