Accreditation & Validation

Joris Van Loco Scientific Institute of Public Health Food Section IPH

Method Validation

 Is method validation • • analyzing 6 samples ?

Calculating the bias, repeatability, reproducibility,… of a method ?

• • Knowing the detection limits of the method ?

knowing the uncertainty associated with a method?

• satisfying ISO 17025 assessors?

What is Method Validation?

 Method validation is the process of proving that an analytical method is acceptable for its intended purpose

Why is Method Validation Necessary?

 To prove what we claim is true   To increase the value of test results To justify customer’s trust  To trace criminals Examples • • • To value goods for trade purposes To support health care To check the quality of drinking water

When and How should Methods be Validated

   New method development Revision of established methods When established methods are implemented in new laboratories   Interlaboratory Comparison Single lab validation • Full Validation • Implementation Validation  Method performance parameters are determined using • equipment that is: • Within specification • Working correctly • Adequately calibrated • Competent operators

Validation and Quality Control

 In house validation • • • (Bias), recovery Repeatability Within lab reproducibility  Internal QC • Control charts Long term within lab reproducibility  Proficiency Testing • Bias (trueness)  Collaborative trial • Reproducibility • Bias (trueness)

Method Validation

      Accuracy • • Trueness (CRM) Recovery (spikes) Precision • • Repeatability (Within) reproducibility Selectivity (& Specificity) Detection capability • LOD, LOQ, CC  , CC  Linearity – calibration range Robustness • Applicability – stability

Method Validation

Performance Characteristics 2002/657/CE

Qualitative methods S C Quantitative S methods C Detection Capability CCß + + + + Decision Limit CC



- + - + Trueness/ Recovery - - - + Precision - - + + Selectivity/ Specificity Applicability/ Ruggedness/ Stability + + + + + + + +

S: Screening methods C: Confirmatory methods

Linearity

 Purpose • to evaluate the linear response of your instrument   How • Evaluating your calibration model • • • Mandels fitting test Lack-of-Fit Residuals 0,14 0,12 0,1

Simple regression: confidence and prediction interval

0,08 0,06 0,04 Conclusion • Linear model • 0,02 0 -0,02 0 0,5 <> other (i.e. quadratic) regression model 1 1,5 2 2,5

concentration (ng/ml)

3 3,5 4 4,5

Linearity

12 8 4 0 -4 0 -8 -12

Linear relationship

20 40

X

60 80 100  Residual plots (e i )

e i



Y i



Y

ˆ

i

• with

Y

ˆ

i



a



bX i

 Statistical tests • • Lack-of-fit Mandel’s fitting test

Coefficient of correlation (r)

 Is NOT a suitable measure for linearity 80 70 60 50 40 30 20 10 0 0,9975 0,998 0,9985 0,999 0,9995

correlation coefficient (r)

1 Pb Cd

Matrix Effect

   Purpose • To evaluate whether you have a concentration dependent systematic error due the matrix • i.e. ion suppression How • comparison of standard curve with matrix matched standard curve Conclusion • Standard solutions, spiked extracts or spiked samples for the calibration line.

t

(

b

)  0,4 0,35 0,3 0,25 0,2 0,15 0,1 0,05 0 -0,05 0

Sp b s



b a

 

X i

,

s

1 

X s

 2   

X i

,

a

1 

X a

 2

Sp

 (

n a

 2 )

n s S

2

s

 

n a

(

n a

 4  2 )

S a

2 50 100 150 200 250 addition standard

Detection Limits

DIN 32645 Detection limit from blanks from calibration data Funk IUPAC dynamic model Coleman recursive formula explicit formula

x D  s L b t 1 m  1 n x D  s x 0  t f;α  1 m y c b s t 1 n n x 2 Q x  0  x  2 n   x i  x  2 x D s y a  t  1 n  y c  y  2 a 2  n   x i  x  2 S c  t x D A  s a     t x D  DL a   σ B σ 0  t x D    σ A A   A  1 n x 2 S xx   1 2  t  J 2  4HK  1 2 2H B  t t   1 n    K I  2t A σ 0  K I I   1 n σ A 2  t A  x D  x  2   1 2 S xx     x 2 S xx   1 2 F   B 2  1  1 n     S xx G    xx H  A 2  S  1 xx J K   2

Detection Limits

A) DIN 32645 Detection limit

x = D y - a k b = s x0  t f;a  1 m + 1 n x 2 + Q x

by fast estimation:

x  n;α  s x0

Capability limit Determination limit

x = x C NG + s x0  t f;β  1 m + 1 n x 2 + Q x x DT x0  t f;α  1 m + 1 n +  Q x  2

by fast estimation

x   n;α  s x0

Factor for fast estimation

Φ n;α = t f;α  1+ 1 n

Detection Limits

B) Funk Detection limit dynamic model

y c b s y t 1 n  0  x  2 n   x i  x  2 x D s y  t  a 1 n  y c  y  2 a 2  n   x i  x  2

Determination limit dynamic model

x c  s y  t  a 1 n x 2 n   x i  x  2 x DT  y h a  b  s y a  t  1 n y h b 2 s y t 1 n  y h  y  2 a 2  n   x i  x  2  x c  x  2 n   x i  x  2

Detection Limits

C) IUPAC Detection limit

S c  t s I x D     t A    K I  2t A σ 0  K I  σ B σ 0 t σ A A σ A A  2

Detection limits “How to”

   Choose a definition and stick to it • Describe the equation used in the validation file • Problems • statistics <> practical limitations • statistics <> ID-criteria Practical LOD • Analyzing samples with decreasing concentration • Minimum concentration which fulfills the identification criteria = practical limit of detection • Repeat the experiment S/N • i.e. LOD=3xS/N

Quantitation Limit (L

Q

)

 The quantification limit is the minimum signal (concentration or amount) the can be quantified. • the residual standard deviation (RSD) is included in the definition.

L Q

 1

RSD Q



Q

• The IUPAC default value for RSD Q = 0.1 (or 10%).  L Q =10s Q .



- and



-error

   -error • risk of erroneously rejecting H 0 • i.e. risk of the conviction of an innocent  -error • risk of erroneously accepting H 0 • i.e. risk of the non conviction of a criminal

Detection Capability

Case of a permitted limit (MRL)

MRL CC  +1.64

 MRL CC +1.64

 sample   =   5% Signal or Concentration

Determination of CC



and CC

 1,5

CC



CC

 1,2

y c

0,9 0,6 0,3 0 0,5

1,12 1,25

0,7

CC

 0,9

MRL

1,1 Conc (µg/kg)

CC

 1,3 1,5

Detection Capability

Case of a permitted limit (MRL) y c CC

 

a



y c



b



bx MRL a



t df

, 1  

Sy

1  1

I

.

J

  

x MRL



x i

 

x x

  2 2

y c



a



bCC

 

t df

, 1  

Sy

1  1

I

.

J

 

CC

  

x i

 

x x

  2 2

Detection Capability

Case of no established permitted limit or banned substance

X blank CC 

+2.33



blank

CC e +1.64

 sampl    1% ≠   5% Signal or Concentration

Presence of Heteroscedasticity

Nitroimidazoles in plasma (MNZ-OH)  Residuals plot • “<“ - shape   Plot of S vs conc • Linear relationship between S and concentration  Heteroscedasticity  Impact on CCa and CCb • CCa and CCb are incorrectly calculated • • Sblank ↓  CCa ↓ CCb ↓ or ↑

RESIDUALS PLOT OF THE LINEAR MODEL

0,8 0,6 0,4 0,2 0 -0,2 0 -0,4 -0,6 -0,8 1 2 3 4 5

Concentration

0, 5 0, 45 0, 4 0, 35 0, 3 0, 25 0, 2 0, 15 0, 1 0, 05 0 0

MNZ-OH

1 2 3

Conc

y = 0,0831x + 0,0023 R 2 = 0,9924 4 5 6 6

Other examples

• Nitroimidazoles in plasma

MNZ

0, 12 0, 1 0, 08 0, 06 0, 04 0, 02 0 0 0, 5 -0, 5 1

Conc

1, 5 y = 0,0389x + 0,0146 R 2 = 0,7416 2 2, 5 3

RNZ

0, 4 0, 35 0, 3 0, 25 0, 2 0, 15 0, 1 0, 05 0 -0, 05 0 0, 5 1 1, 5 y = 0,0764x - 0,0172 R 2 = 0,7899 2 2, 5

Conc

3 3, 5 4 4, 5 5 • Nitrofurans in honey

AMOZ

y = 0,0426x + 0,0105 R 2 = 0,9768 0, 14 0, 12 0, 1 0, 08 0, 06 0, 04 0, 02 0 -0, 02 0 -0, 5 0, 5 1

Conc

1, 5 2 2, 5 3 • Corticosteroids in liver

Triamcinolone Acetonide

0, 8 0, 7 0, 6 0, 5 0, 4 0, 3 0, 2 0, 1 0 -0, 1 0 -0, 2 0, 5 1 1, 5 2 2, 5

Conc

3 y = 0,1975x - 0,1607 R 2 = 0,8451 3, 5 4 4, 5 5

Weighted regression equations for CC



and CC



CC

 

t df

, 1  

b S

2

CC

      

J

.

1 

w i



J

.



w i



x

2

w x i



x w

 2   

S y

2

CC

  

df

,  , 

b S

2

CC

      

J

.

1 

w i



J

.



w i



x

2

w x i



x w

 2   

S y

2 • Solved by iteration

Conclusion detection capability

 Many definitions of detection limits • • detection limit ( ≈ CC  _banned substances) determination limit ( ≈ CC  _banned substances) • Quantition limit  Complicated statistics  KISS • demonstrate with real (spiked) samples at low concentration level  practical limit of detection

Selectivity/Specificity

 Identity : Signal to be attributed to the analyte • GLC (change column/polarity), GC/MS, Infra-red  Selectivity : The ability of the method to determine accurately the analyte of interest in the presence of other components in a sample matrix under the stated conditions of the test.

 Specificity is a state of perfect selectivity

Selectivity

 • • • The procedure to establish selectivity: • • Analyze samples and reference materials Assess the ability of the methods to confirm identity and measure the analyte Choose the more appropriate method.

Analyze samples Examine the effect of interferences

Selectivity: Verification of the identification criteria (2002/657/EC)

   MS – criteria • 3 or 4 identification points • 1 precursor and 2 transition ions • Relative ion intensities LC – criteria • Relative retention time (RRT): +/- 2.5 % (LC) UV – criteria • • Spectrum match +/- 3 nm   CC  is concentration at or above the calculated CC  for which the ID criteria are fulfilled in 95% CC  of the cases.

is concentration at or above the calculated CC  for which the ID criteria are fulfilled in 50% of the cases.

Ruggedness and Robustness

 Intra-laboratory study to check changes due to environmental and/or operating conditions • Usually it is part of method development • Deliberate changes in • • • • • Temperature Reagents ( e.g. different batches) Extraction time Composition in the sample etc

Precision – ISO 5725 1-6 (1994)

   

Expresses the closeness of agreement (dispersion level, relative standard deviation) between a series of measurements from multiple sampling of the same homogeneous sample (independent assays) under prescribed conditions.

Irrespective of whether mean is a correct representation of the true value.

Gives information on

random errors

Evaluated at three levels:  repeatability  intermediate precision (within laboratory)  reproducibility (between laboratory)

Precision (cont.) – ISO 5725 1-6 (1994)

 

Repeatability

: precision under conditions where the results of independent assays are obtained by the same analytical procedure, on identical samples, in the same lab, by the same operator, using the same equipment and during short interval of time

Intermediate precision

: ISO recognizes M-factor different intermediate precision conditions or 3) (M = 1, 2  M = 1 : only 1 of 3 factors (operator, equipment, time) is different  M= 2 or 3 : 2 or all 3 factors differ between determinations

Precision (cont.) – ISO 5725 1-6 (1994)

 

Reproducibility

 : precision under conditions where results obtained:

by same analytical procedure



on identical sample



in different laboratories, different operators, different equipment

Reproducibility established by interlaboratory study (

standardisation

of an analytical procedure)  Intermediate precision Repeatability Reproducibility

Evaluation of Precision

 10 samples for each conc.under r,R, within lab R • Standard Deviation  

i n

  1 (

x n

  1

x

) 2

RSD



s x

 100  Determination in pairs under r,R, within lab R • Std. Dev. between two single determinations s r S R S Rw •

s

  (

a i



b i

) 2 2

d

a-b, the difference between the values, d, the number of pairs

Repeatability (r) and within-lab reproducibility (Rw)

ANOVA table for a single factor balanced design with 3 replicate samples on the same day.

Source Sum of squares df Mean Squares Expected mean squares day SS days n days - 1 SS MS days days / (n = days – 1) σ repl ² + 3σ days ² replicate Total SS repl SS T n T – n days n T – 1 SS repl MS repl / (n T = – n days ) SS T = MS T 1) / (n T – σ repl ² repeatability (Sr²) and within-lab reproducibility variances (S Rw ²)

S r ² = S repl ² S Rw ² = S r ² + S days ²

The S repl ²and S days ² can be obtained from mean squares as (n repl S repl ² = MS repl S days ² = (MS days – MS repl ) / 3 = 3):

Repeatability and reproducibility

    The value of 2.8?

• Variance of difference between 2 replicate measurements is 2s² •  Confidence interval at 95% level on the difference is 0 ± 1.96 √2 s  ± 1.96 x 1.41 s r = ± 2.8 s r 95% probability that difference between duplicate determinations will not exceed 2.8 s r r = limit of the repeatability r = 2.8 s r R = limit of the reproducibility R = 2.8 S R

Precision criteria 2002/657/CE

-100 -140 -20 -60

Horwitz: RSD

R

(%) = 2

(1-0.5logC) RSD R

% 2 0 , 5 log

C

%

Dioxins 100 Aflatoxines 60 Drugs narcotics in food pesticide residu's 20

C

Determination of Trueness

 Using Certified Reference Materials  Using RM or In-house materials  Using Reference methods • • Single sample Many samples  Via Interlaboratory study

Trueness, extraction yield (recovery) and apparent recovery

   Trueness as bias means the closeness of agreement between the average value obtained from a large series of test results and an accepted reference value. Trueness is usually expressed Recovery (extraction yield): yield of a preconcentration or extraction stage of an analytical process for an analyte divided by amount of analyte in the original sample.

Apparent recovery : observed value derived from an analytical procedure by means of a calibration graph divided by reference value.

Trueness criteria 2002/657/CE

 When no such CRMs are available, it is acceptable that trueness of measurements is assessed through recovery of additions of known amounts of the analyte(s) to a blank matrix. Data corrected with the mean recovery are only acceptable when they fall within the ranges