Transcript slides
Charles Y. Tan, PhD
USP Statistics Expert Committee
Outline
Introduction of <1210>
Key topics
Accuracy and Precision
Linearity
LOD, LOQ, range
Summary
USP <1210>
United States Pharmacopeia
General Chapters
<1210> Statistical Tools for Method Validation
Current status: a draft is published in Pharmacopeial
Forum 40(5) [Sept-Oct 2014]
Seek public comments
Purpose of <1210>
A companion chapter to <1225> Validation of
Compendial Procedures
USP <1225> and ICH Q2(R1)
USP <1033> Biological Assay Validation
Statistical tools
TOST, statistical equivalence
Statistical power, experimental design
tolerance intervals, prediction intervals
Risk assessment, Bayesian analysis
AIC for calibration model selection
Recent Framework
Life cycle perspective
procedure design
performance qualification / validation
ongoing performance verification
ATP: Analytical Target Profile
Pre-specified acceptance criteria
Assume established
Validation: confirmatory step
Statistical interpretation of “validation”
Performance Characteristics
Different statistical treatments
Tier 1: accuracy and precision
Statistical “proof” ATP is met
Equivalence test / TOST
Sample size / power, DOE
Tier 2: linearity, LOD
Relaxed evidential standard, estimation
Sample size / power optional
USP General Chapter <1210>
Statistical Tools for Method Validation
Accuracy and Precision
Separate Assessment Of Accuracy And Precision
Confidence interval within acceptance criteria from ATP
Combined Validation Of Accuracy And Precision
γ-expectation tolerance interval: 100γ% prediction
interval for a future observation,
Pr (-λ ≤ Y ≤ λ) ≥ γ
γ-content tolerance interval: 100γ% confidence of all
future observations
Bayesian tolerance interval
Experimental Condition
Yij = μ + Ci + Eij
Ci: experimental condition
combination of ruggedness factors: analyst, equipment,
or day
DOE: experience the full domain of operating
conditions
As independent as possible
Eij: replication within each condition
One-way analysis (w/ random factor): why?
Separate Assessment
Closed form formulas:
Accuracy: classic confidence interval for bias
Precision: confidence interval for total variability under
one-way layout (Graybill and Wang)
Power and sample size calculation
Statement of the parameters: bias, variance
Eg. CI of bias: [-0.4%, 1.1%], within ±5% (ATP)
Eg. CI of total variability: ≤2.4%, within 3% (ATP)
Implicit risk level: 95% confidence intervals
Combine Accuracy and Precision
Statement of observation(s)
Closed form formulas, but a bit more complicate
99%-expectation tolerance interval: eg. [-4.3%, 5.0%] within
±10% (ATP)
99%-content tolerance interval: eg. [-5.9%, 6.6%] within ±15%
(ATP)
Bayesian tolerance interval
“the aid of an experienced statistician is recommended”
Simpler Alternative: directly assess the risk with the λ
given in ATP
Pr (-λ ≤ deviation from truth ≤ λ|data)
Scale of Analysis
Pooling variances is central to stat analysis
Variance estimates with df=2 are highly unstable
Need to pool across samples, levels
Variance at mass or concentration scale/unit
Increase with level
Solutions:
Normalize with constants, eg. Label claim
Normalizing by observed averages makes stat analysis too
complicated
Log transformation
%NSD and %RSD
Linearity
Internal performance characteristic
External view: accuracy and precision
Transparency => credibility
Appropriateness of standard curve fitting
A model
A range
Better than the alternatives (all models are
approximations)
Proportional: model: Y = β1X + ε
Straight line: Y = β0 + β1X + ε
Quadratic model: Y = β0 + β1X + β2X2 + ε
Current Practices
Pearson correlation coefficient
Anscombe's quartet
Lack-of-fit F test
independent replicate
Mandel’s F-test, the quality coefficient, and the Mark–
Workman test
Test of significance
Evidential standard: low since it gives the benefit of doubt to
the model you want
Good precision may be “penalized” with a high false rejection
rate
Poor precision is “rewarded” with false confirmation of the
simpler and more convenient model
Anscombe's Quartet
Two New Proposals
Equivalence test, TOST, in concentration units
Define maximum allowable bias due to calibration in ATP
Construct 90% confidence interval for the bias comparing the
proposed model to a slightly more flexible model
Closed form formula, complex
Evidential standard: could be high, depend on allowable bias
Akaike Information Criterion, AICc
Compare the AICc of the proposed model to a slightly more
flexible model (smaller wins)
Very simple calculations
Evidential standard: most likely among candidates
Different Burden of Proof
Hypothesis Testing: Neyman-Pearson
Frame the issue: null versus alternative hypotheses
Goal: reject the null hypothesis
Null hypothesis: protected regardless of amount of data
Decision standard: beyond reasonable doubt
Legal analogy: criminal trial
Information Criteria: Kullback-Leibler
Frame the issue: a set of candidate models
Goal: find the best approximation to the truth
Best: most parsimonious model given the data at hand
Decision standard: most likely among candidates
Legal analogy: civil trial
Stepping-stone or tactical questions: information criteria are apt
alternatives to hypothesis tests
IUPAC/ISO LOD (RC and RD)
IUPAC/ISO LOD
LOD: Using Prediction Bounds
Range and LOQ
Range
suitable level of precision and accuracy
Both upper and lower limits
LOQ (LLOQ)
acceptable precision and accuracy
lower limit
LOQ versus LOD
Only one is needed for each use
LOQ for quantitative tests
LOD for qualitative limit tests
LOQ calculation in ICH Q2: candidate starting values
Summary
A draft of USP <1210> is published, seeking public
comments
A step in the right direction?
More than a bag of tools
Implement modern validation concepts with a
statistical structural
More tools development needed
More statisticians involvement needed in
pharmacopeia and ICH development