Transcript Validation - Instituto Nacional de Metrologia

Validation
by
Dr. Michael Scheutwinkel
International Federation for
Consulting GmbH IFC
1
• Governmental
regulations
• Books ….
• Recommendations
of associations
2
References
EURACHEM Guide
The Fitness for Purpose of Analytical Methods A Laboratory Guide to Method Validation
and Related Topics (12/1998)
www.eurachem.com
CITAC / EURACHEM Guide
Guide to Quality in Analytical Chemistry
An Aid to Accreditation (2002)
www.eurachem.com
3
4
Which chapter of ISO/IEC 17025 is dedicated to validation?
5
Do not ask
me, I am
only a
simple guy
from
Alemania!
6
Definition in clause 5.4.5.1 of ISO/IEC 17025
Validation (ISO/IEC 17025)
Validation is the confirmation by examination
and the provision of objective evidence that
the particular requirements for a specific
intended use are fulfilled.
….fit for purpose
7
Validation of an analytical test method
Validation is the process of establishing the
• Performance characteristics
• Limitations of a method
• Identification of the influences which may
change these characteristics and to what extent.
Which analytes can be determined in which matrix
in the presence of which interferences?
Within these conditions what levels of precision
and accuracy can be achieved?
8
Equipment and method valdiation
Equipment validation
DQ
IQ
Soft and hardware
OQ
PQ
Method validation
… different items
9
Validation in order to analytical method
Standardized-methods
Verification
Modified standardized and
in-house methods
Validation
10
Verification of an analytical test method
Verification is the process to demonstrate the
competence in test performance
of an already validated standardized test method.
The tools are the same as for validation,
the laboratory needs only to verify that
the documented performance characteristics
can be met (e.g. accuracy).
11
When should be validated?
• Changes in established methods
• New method developed for a particular problem
• Established method used in different
laboratories, different equipment or different
staff
• Out-of-control situations within internal quality
assurance
• Non-successful participation in PTs
• Demonstration of equivalence between two
methods (e.g. a rapid new test against a standard
method)
12
Validation Strategy
A
Specific test requirements defined by customer or market
B
C
D
Characterization
of the test method
Comparison
of method
charcacteristics
with
requirements
Verification
to give
proof that the
requirements are
fulfilled
13
Description of test method (clause 5.4.4)
Identification / Scope
Parameters in which matrix
Reagents and materials, CRMs
Equipment and environmental
conditions
Calibration
Sampling and
sample preparation
Test performance
including safety instructions
Quality assurance
Documentation , reporting
evaluation criteria
Uncertainty or procedure for
its estimation
14
Please show us the
way…
15
Tools
Determination of method characteristics
Characteristics
of the range
Characteristics
of calibration
Characteristics
of accuracy
Robustness /
Ruggedness
Selectivity or
Specifity
16
Tools
Determination of method characteristics
Perform more
point calibration
Characteristics
of calibration
 Linearity
 Sensitivity
Correlation
coefficient:
r² > 0.99 ??
Linearity is defined by the
correlation coefficient
Sensitivity is defined by the
slope of the calibration graph
17
Commission Decision 2002/657/EC
18
Validation of testing methods
Calibration
• Calibration values should be applied within the working range.
• Lowest point should be the limit of quantification.
• For calibration certified standard solutions shall be used.
• In addition, also standard solutions have to be added to blank
materials, analysed over all process steps.
• For linear calibration functions at least five concentrations are
necessary.
• Test calibration functions for linearity and highlight failing
linearity.
19
Validation of testing methods
Sensitivity (resolution)
What is sensitivity?


It is the difference in an analytical concentration that
corresponds to the smallest difference of a signal in a
method which is still detectable.
Sensitivity can be extracted from the calibration curve
or defined by using samples with different
concentrations.
20
Validation of testing methods
Linearity

Definition based on samples with varying concentrations
and the calculation of the regression of results.
 Signal and concentration do not have to be fully related.
Five standards suffice to produce a calibration curve if
 linearity is o.k.
More standards are necessary once the linearity is
 unsatisfactory.
 Examine repeat samples and standards over a particular
working range to experience whether a reliable line can be
drawn between proof and detection limit.
21
Homogeneity of variances

Characteristic values for the comparison
of standard variations for differing
concentration.
In case of great differences between
standard deviations
(in-homogeneity of variances),
the working range needs to be split.
22
Tools
Determination of method characteristics
Characteristics
of calibration
Characteristics
of the range
Limit of detection
Limit of quantification
Characteristics
of accuracy
Robustness /
Ruggedness
Selectivity or
Specifity
23
Validation of testing methods
Working range
Finding out the appropriate working range using
•
different matrices
•
varying concentrations.
Concentration range within the achievement of
acceptable
•
accuracy and
•
precision is possible.
In general the working range is broader than the linear range.
24
Validation of testing methods
Prerequisites for calibration
 Standards must be faultless.




Precision has to be similar throughout the entire working range.
The model function is applicable: either linear or curved.
Errors only may occur randomly within signals.
Errors have to follow the normal distribution.
25
LOD
Characteristics of the range
Limit of detection
LOD
by
calculation
xLOD = 3 . sL / b (Blank value method)
xLOD = 4 . sxo (Calibration function method)
xLd = Limit of detection
sL = Standard deviation of the blanks
Sxo = Standard deviation of the calibration function
b = Slope of calibration function
Various
conventions
XLOD = Mean of blanks + 3 x SL
26
LOQ
Characteristics of the range
Limit of quantification
LOQ
by
calculation
xLOQ = 9 . sL/ b (Blank value method)
xLOQ = 11 . sxo (Calibration function method)
xLOQ = Limit of quantification
sL = Standard deviation of the blanks
Sxo = Standard deviation of the calibration function
b = Slope of calibration function
Various
conventions
XLOQ = 5 or 6 or 10 x SL
27
Commission Decision 2002/657/EC
28
LOD
Characteristics of the range
Limit of detection
From 2 : 1 to 5 : 1
by
signal to
noise ratio
Limit of quantification
From 5 : 1 to 10 : 1
29
Effect of peak shape on LOD / LOQ
30
Tools
Determination of method characteristics
Characteristics
of the range
Characteristics
of calibration
Characteristics
of accuracy
Robustness or
Ruggedness
Selectivity or
Specifity
31
Selectivity / Specifity
Selectivity
Specifity
Extent to which particular analytes
can be determined in complex mixtures.
A method which is selective for an analyte
is said to be specific.
Selective detectors
in instrumental
analysis
Use of
antibodies
Cross-reactions ?
Selective media in
microbiology
32
The difference between selectivity and specificity
Selectivity
• A selective method gives
correct results for all
interesting analytes whereas a
specific test method gives
correct results for the
interesting analyte whereas
other analytes might interfere
each other.
Specifity
• Outlines the analytical
extent to which an
analytical substance or
substance group can be
determined without
interference from sample
related components.
No problem, usually both words are mixed …
33
Validation of testing methods
Selectivity
• Outlines the extent to which an analytical substance
can be determined without interference from other
components.
• Method selective to one analytical substance
specific.
• Assess selectivity while application to pure solution
up to complex matrices.
• Document disturbances and restrictions of the
method.
34
Tools
Determination of method characteristics
Characteristics
of the range
Characteristics
of calibration
Characteristics
of accuracy
Robustness /
Ruggedness
Selectivity or
Specifity
35
Validation of testing methods
Robustness
 Degree of proneness of a method to
conscious and unconscious alterations
of a working instruction.
• Part of the routine.
• Learn from other analytical institutes, as
the method-establishing laboratory will have proved
robustness before publishing a new method.
36
Robustness
Sensitivity degree of the test method against
small deviations in experimental conditions.
Examples:
• Times within process steps
• Environmental conditions (e.g. temperatures)
• Minor process changes (e.g. pH, flow rates in
HPLC)
37
Ruggedness, according to USP
Ruggedness is the degree of reproducibility obtained
under a variety of conditions, expressed as relative SD,
e.g.:
•
•
•
•
different analysts,
different equipment,
different trade marks of reagents,
etc.
Internal reproducibility
38
Tools
Determination of method characteristics
Characteristics
of the range
Characteristics
of accuracy:
Correctness
Uncertainty
Precision
(Repeatability/
Recovery)
Reproducibility
Characteristics
of calibration
Robustness or
Ruggedness
Selectivity or
Specifity
39
Accuracy = Precision and correctness I
Accuracy
Exactness of an
analytical method
Degree of
repeatability of an
analytical method
Precision
Repeatability
Reproducibility
Correctness /
Trueness
Systematic error /
Correct value
Uncertainty
Recovery /
Bias
40
Random and systematic deviations
precise
and
correct
not precise
but
correct
precise
but
wrong
not precise
and
wrong
41
Accuracy = Precision and correctness II
Characteristics of accuracy
Correctness
Precision
Measure of correctness, covering systematic and
non-systematic mistakes (use of CRM, comparison
to a well characterized method)
Measure of the degree of repeatability,
covering systematic mistakes
Repeatability: one laboratory;
Better: internal reproducibility
Reproducibility: more than one laboratory
42
Validation of testing methods
Repeatability
Use at least one of the following procedures for determination:




Measurement of reference materials.
Fortification of a blank sample with the analytical
substance.
Comparison to a reference procedure.
Participation in a co-operative (interlaboratory) test in
which the “true content” is known.
43
Validation of testing methods
Precision

Information on accuracy of two coinciding and
independent analytical results.
Precision depends on the concentration of the analyte.
Repeatability is part of the precision with respect to
repeated measurements:
– same material,
– same method,
– same analyst,
– same laboratory and
– short time-span in-between the analyses.
44
Validation of testing methods
Correctness / Trueness
Use the one of the following methods to assure correctness:
• Analysis of certified reference material.
• Participation in interlaboratory comparisons.
• Comparison to a known procedure.
If not available:
•
•
•
•
Document any data that prove correctness.
Make an approximation as a first approach to correctness.
Purchase reference material.
Use a similar method in parallel.
45
Validation of testing methods
Recovery
• Performed by addition of the analytical substance to the matrix.
• A matrix shall be free of the analytical substance.
• If not the sample will be enriched with a weak concentration of
the analytical sample.
• Or a simulated matrix will be used.
• Recovery can be determined only when the analytical sample is
available in pure form.
• List the recovery rate (in %) and the standard deviation when
recovery rates are constant.
• Otherwise issue recovery rate as a function.
46
Validation of testing methods
Frame conditions for the
determination of the recovery rate
The analytical substance needs to be added in
that form in which it occurs in nature.
The samples requires good homogenisation.
The native content shall be below the
determination limit.
47
Validation of testing methods
Reproducibility
internally
Prove reproducibility
externally
 Statistically


Comparative standard deviation
Replicate standard deviation
 Through participation in proficiency testing schemes
 or participation in interlaboratory comparisons.
48
Tools
Determination of method characteristics
Characteristics
of calibration
 Linearity
 Sensitivity
Robustness or
Ruggedness
Characteristics
of the range
 Limit of detection
 Limit of quantification
Selectivity or
Specifity
Characteristics
of accuracy
 Correctness
 Uncertainty
 Precision
(Repeatability/
Recovery)
 Reproducibility
49
Validation of testing methods
Classification of the test method
Distinguish between
• Analysis of contaminants (e.g. pesticides, heavy metals,
food additives)
• Analysis of food constituents (salt and minerals)
• Methods which are determining true values
• Test methods which have to be calibrated with instruments
• Physical measurement (pH value, conductivity)
• Qualitative determinations
• Microbiological methods
50
Validation of testing methods
Parameters for validation
Parameter
Analysis of
Analysis of
Conventional
Methods not
Physical
Qualitative
Bacteriological
trace
constituents
methods
necessary to
methods
methods
methods
X
X
X
X
X
contaminants
Description of
calibrate
X
X
X
X
Calibration
X
X
X
Working range
X
X
X
X
Detection limit
X
X
X
Determination
X
X
test method
X
X
X
limit
Recovery
X
X
Repeatability
X
X
Correctness
X
Specificity
Reproducibility
X
X
X
X
X
X
X
X
X
X
if possible
if possible
if possible
if possible
if possible
X
X
X
if possible
if possible
51
Validation of testing methods
Common validation needs of
known methods
Method
Accuracy Precision
Specificity/
Selectivity
Linearity
Detection/ Determination
limits
Robustness
(X)
X
X
X
X
X
AAS, AES, ICP, RF
X
X
Polarography/ Voltametry
X
X
X
X
X
X
X
X
X
X
X
X
X
Spectroscopy
X
X
X
X
X
Release
X
X
X
X
X
Titration
Water assay
X
X
Chromatography/ Capillary
Electrophoresis
TOC
Enzyme Immuno Assays
X
X
X
X
X
X
X
X
X
X
52
Procedures for method validation
The Analysis of Standard Reference Materials
•
Generally accepted method for validation.
•
Such standards are provided with guarantee on
the market.
•
It may be necessary to contract the preparation
of a unique sample in particular matrix in order
to utilise this procedure for method validation.
•
The analyst must demonstrate that the method
provides accuracy and precision.
53
Procedures for method validation
The participation in PT-schemes or in a
laboratory Collaborative Study
The most widely accepted procedure for method
validation.
Serious practical draw backs:
Costly and time consuming
Effort in co-ordination
Shipping of samples and data
Statistical analysis and interpretation of results.
This method is rarely used for the first description
of a method in the literature.




54
Procedures for method validation
Comparison with a currently accepted method
•
Usually done by one analyst
•
or two using a split-sample.
•
using results from the currently accepted method
for verification.
Agreement suggests validation.
Disagreement could also suggest that the
currently accepted method is invalid.
•
In such case, another procedure has to be
employed for the method validation.
•
The more samples are analysed and the
wider the range of concentration the higher the
credibility of the validation method.
55
Procedures for method validation
The Zero-Blind Method
•
One analyst.
•
Using samples at known levels of analyte to demonstrate
recovery, accuracy and precision.
•
Fast, simple and useful but
it leads to subjective results.
•
Suitable for a first approximation requiring minimal time,
manpower, samples and cost.
•
In general, a good start for the overall validation process.
If this methods fails there is no reason to proceed with
further validation of the method.
56
Procedures for method validation
The Single-Blind Method
•
One analyst at the start.
•
Samples are given to a 2nd analyst to whom levels of analyte are
unknown.
•
Results are compiled and compared by the 1st analyst.
•
Comparison by the first analyst makes objectivity questionable.
•
This method still is biased on behalf of the 1st analyst.
•
Suitable after the zero-blind method has been successful and
before additional analysts or the management will be involved.
57
Procedures for method validation
The Double-Blind Method
•
Three analysts.
•
1st analyst prepares samples at known levels.
•
2nd analyst does the actual analysis.
•
3rd analyst (or administrator) compares both
data from the first two analysts.
•
Only the 3rd analyst has access to these data.
Most effective approach to objectiveness
58
Validation of testing methods
Clearance of validated test procedures
Quality Manager

Validation
results
Customer
requirements
59
Validation of testing methods
Decision criteria for the extent of a validation
Technical feasibility and staff disposition?
Analytical viewpoints
Measuring principle, method complexity?
facts
Occurrence of the sample, wide-spread?
Risk potential for client/ company?
Political decisions
Actual aim of the validation?
super-ordinated criteria
Consideration of normative
and other requirements?
60
Validation of testing methods
Extent of validation in analytical sciences
Purpose
• Suitability of the method
• „Marketability“ of the method
Analytical sensibility
Costs
Requirements from the outside
61
ISO/IEC 17025 states:
Validation is always a balance between
costs, risks and technical possibilities.
The validation shall be completed
by a statement by the laboratory that the
method is fit for the intended use.
62
Validation is
always a balance betwee
costs, risks and
technical possibilities.
63
Do not forget:
Self developed software or excel
sheets have to be validated.
Records for that
must be available.
64
Muito
obrigado!
65
66
Bring them
together ….
67
68
69
70
Trabajo en grupos ”Humedada en Alimentos”
Fuente de incertidumbre
Símbolo
Valor
±
[Unidades]
Calibración de la balance
uEst
+ 0.003 g
Calibración de la estufa
uPip
0%
Tiempo
uBurInd
0%
Repetibilidad
uRep
+ 0.1 %
Distribución de
probabilidad
Divisor
Tipo B / rect
3
Incertidumbre estándar
[unidades]
0.7%
0.4%
3
Tipo B / rect
Tipo B
3
0.1%
3
ucombinada = 3.43 %
Uexpandida = 7 %
71
IfEP Workshop: Proficiency Testing - Background and general aspects
Why proficiency testing?
 PT schemes are just one important tool in quality control
and quality management.
 Participating in Proficiency Tests
 Use of certified reference material
 Validation of methods
QM
72