Transcript Overview – Courses - STT

Content
Analytical Quality Specifications ("Goals")
Introduction
Concepts
• Clinical concepts
• Questionnaires to clinicians
• Goals from biology
• Goals from experts
• State-of-the-art
Comparison of goals
Comparison "state-of-the-art" with goals
Analytical goals – Translation into practice
Goals – future vision
Outlook
References
Statistics & graphics for the laboratory
35
Introduction
Introduction
The analytical quality triangle
The establishment of analytical performance goals (quality specifications) is an
important part of the analytical quality triangle, which should guarantee medically
relevant quality of laboratory tests.
Clinical
Biology
Expert
State-of-art
Chemistry
Instrument
Quality
specification
Quality
creation
Quality
management
Planning
Control
Assurance
Improvement
In the words of ISO 15196 – Analytical goals (discontinued) this is expressed as:
"Well-defined analytical performance goals for clinical laboratories have practical,
regulatory, and commercial implications. The goals help to determine the day-today practices of clinical laboratories. In turn, these goals should shape, if not drive
the plans of IVD manufacturers to produce devices that can meet the analytical
needs. In a proper synergy, regulatory agencies should base their expectations of
laboratory quality on well-conceived analytical goals".
Unfortunately, the clinical community has not been able to establish a
consensus on goals for analytical quality. A major problem is the involvement of
many groups with different interests: "analytical performance goals shall be
reviewed by the laboratory director to verify that they a. meet local medical needs;
b. meet applicable regulatory requirements; c. are economically feasible; d. are
technically achievable" (ISO 15196).
Nevertheless, what has been achieved is a consensus on a hierarchy of
models to be used for the establishment of analytical goals.
Statistics & graphics for the laboratory
36
Introduction
Introduction
Hierarchy of models for the establishment of analytical goals
• Clinical concepts
• Concepts based on biological variation
• Expert opinion
• Regulations
• "State-of-the-art"
Consensus Statement (Stockholm 1999).Scand J Clin Lab Invest 1999;59:585.
In the following, these concepts will be reviewed with the emphasis on their
underlying statistical basis. For easier understanding, a list of abbreviations of
biological and analytical components of variance is given below.
Abbreviations for biological variation
• SDW-S/CVW-S (or CVw) = Within-subject
• SDB-S/CVB-S (or CVb) = Between-subject
• SDG/CVG (or CVg) = "Group"
= SQRT(CV2W-S + CV2B-S)
= ¼ of the reference interval
• SDBiol/CVBiol : if the component (within, etc.) is not specified
CAVE: SDB-S often is understood as SDG
Abbreviations for analytical variation/error
• SDA/CVA (or CVa) = Analytical
• TEA = Total allowable analytical error
Statistics & graphics for the laboratory
37
Clinical concepts
Clinical concepts
Unifactorial health/disease models (1 analyte/1 disease)
For example:
• HbA1c for long-term control of diabetes.
2 Situations
• Separate distributions (bimodal) of the analyte in sick and healthy
• One distribution (unimodal) of the analyte in sick and healthy
Unimodal situation
Bimodal situation
The bimodal situation
Prerequisites
The “gold-standard”
The true status of each population has to be established by other means than the
test being subject to evaluation, namely, a so-called “gold standard”.
Defining a decision point (“cut-off” value)
A decision point (sick/healthy) must be defined. Note that this point must not lie at
the crossing of the two distributions. Dependent on the importance of false
negatives or false positives, it can be moved towards increased sensitivity or
specificity.
Note: For monitoring, a medically significant change has to be defined.
Classification of results
With respect to the gold standard, test outcome is classified as “true positive”,
“false positive”, “true negative”, or “false negative”.
Statistics & graphics for the laboratory
38
Clinical concepts
Clinical concepts – Bimodal situation (ctd.)
Analytical error and test outcome
• At a given clinical situation, the influence of analytical quality on test
sensitivity and specificity is investigated by simulation studies.
Simulation of systematic error
• The introduction of a systematic error in the direction of the diseased
population increases the false positive results. The introduction of a
systematic error in the direction of the healthy population would increase the
false negative results.
Simulation of random error
• The introduction of random analytical error, generally, deteriorates test
accuracy.
Simulation of systematic error
Simulation of random error
Bias  diseased > FP, FN
Bias  healthy > FP, FN
General deterioration
of test accuracy
See also: Hyltoft Petersen P. Scand J Clin Lab Invest 1999;59:517-22
Clinical concepts – Results
From considerations about maximum tolerable deterioration of the intrinsic clinical
quality of a test, maximum values for analytical inaccuracy and imprecision are
derived by simulation studies.
The derived specifications
• are restricted to the specific clinical situation addressed
• depend on the opinion of the clinicians involved (e.g., definition of the “cut-off”
value)
• depend on the parameter that was used for optimization (sensitivity, specificity,
cost/benefit considerations, etc).
Statistics & graphics for the laboratory
39
Clinical concepts
Clinical concepts
Unimodal – Decision limits
Serum-Cholesterol
Uni-modal distribution
The whole population
Classification
Effect of analytical quality
Number of patients
Serum cholesterol in men – Effect of bias
Klee et al. Scand J Clin Lab Invest 1999;59:509-12 (wrong numbers: ±1, 3, 10%).
(200 mg/dl = 5.17 mmol/l; 240 mg/dl = 6.2 mmol/l)
On the basis of the hospital population at Mayo, the % misclassifications due to
bias were investigated.
250
— 200 mg/dl
– – – ± 2%
• • • • • ± 6%
• – • – ± 20%
200
150
100
50
0
50
100
150
200
250
300
350
Cholesterol, mg/dl
A bias of ±1% caused a -6.5% to +5.8% change in the classification of patients for
cardiac disease. A bias of of ±3% caused a -18.4% to +16.7% change in the
classification. From these data, a maximum bias of 1% was proposed for
cholesterol.
Statistics & graphics for the laboratory
40
Clinical concepts
Clinical concepts
Unimodal – Decision limits
Serum cholesterol in men – Effect of bias
Similar investigations have been made by Hyltoft Petersen et al. (Clin Chim Acta
1997;260:189-206). However, they made a Gaussian model of the distribution and
read the effect of bias from the cumulated distribution. Moreover, they mirrored the
distribution and centered it at the decision point, for an easier visual assessment.
% at or above given CHOL
% with CHOL >6.2 as function of bias
Increase
+1 mmol/l
Cumulated
distribution
-1 mmol/l
±3%
6.2
S-CHOL (mmol/l)
Bias (mmol/l)
A bias of +3%, for example, incresed the positives from 48% to 62% (= +14%).
However, no limit was recommended in that publication.
Effect of interferences
Assume a "low prevalence situation" where 0.2% of patients are found above a
certain cutoff (= 2 of 1000). If the test shows 0.2% interferences that cause
otherwise normal to be above the cutoff, 4 are tested positive (100% increase).
For a prevalence of 2% (= 20 of 1000), 22 will test positive with a test with 0.2%
interference (10% increase).
 In a “high prevalence” situation, interference may be less a problem.
Statistics & graphics for the laboratory
41
Clinical concepts
Clinical concepts
Examples from literature
• Haemoglobin A1c for long term diabetic control [1]
• Theophylline in serum for therapeutic drug monitoring [2]
• Creatine kinase isoenzyme in the diagnosis of acute myocardial infarction [3]
• Blood thyroid-stimulating hormone in screening for congenital hypothyroidism [3]
• Cholesterol as a measure for risk of coronary heart disease [3]
• AFP, prenatal screening for neural tube defects [5]
• Reviews: Hyltoft Petersen & Hørder [3, 4] and a special issue of the Upsala Journal
of Medical Sciences [5].
Summary clinical concepts
• Their establishment is very complicated
• They offer several values, dependent on the model used to derive them (e.g.,
optimisation of specificity, sensitivity, cost/benefit, etc.)
• Only a few analytes have been tackled
• They are extremely valuable
Statistics & graphics for the laboratory
42
Clinical concepts
Clinical concepts
References
1 Larsen ML, Fraser CG, Petersen PH. A comparison of analytical goals for
haemoglobin A1c assays derived using different strategies. Ann Clin Biochem
1991;28:272-8.
2 Jenny RW. Analytical goals for determination of theophylline concentration in serum.
Clin Chem 1991;37:154-8.
3 Petersen PH, Hørder M. Ways of assessing quality goals for diagnostic tests in
clinical situations. Arch Pathol Lab Med 1988;112:435-43.
4 Hyltoft Petersen P, Hørder M. Influence of analytical quality on test results. Scand J
Clin Lab Invest 1992;52 Suppl 208:65-87.
5 de Verdier C-H, Groth T, Hyltoft Petersen P, editors. Medical need for quality
specifications in clinical laboratories. Upsala J Med Sci 1993;98:189-491.
References specific to TSH
1 Klee GG. Clinical interpretation of reference intervals and reference limits. A plea for
assay harmonization. Clin Chem Lab Med 2004;42:752-7.
2 Fatourechi V, Klee GG, Grebe SK, Bahn RS, Brennan MD, Hay ID, McIver B, Morris
JC 3rd. Effects of reducing the upper limit of normal TSH values. JAMA
2003;290:3195-6.
3 Fatourechi V, et al. Factors influencing clinical decisions to initiate thyroxine therapy
for patients with mildly increased serum thyrotropin (5.1-10.0 mIU/L). Mayo Clin Proc
2003;78:554-60.
4 Klee GG, Schryver PG, Kisabeth RM. Analytic bias specifications based on the
analysis of effects on performance of medical guidelines. Scand J Clin Lab Invest
1999;59:509-12.
5 Hay ID, Klee GG. Linking medical needs and performance goals: clinical and
laboratory perspectives on thyroid disease. Clin Chem 1993;39:1519-24.
6 Ward G, McKinnon L, Badrick T, Hickman PE. Heterophilic antibodies remain a
problem for the immunoassay laboratory. Am J Clin Pathol 1997;108:417-21.
7 Despre´s N, Grant AM. Antibody interference in thyroid assays: a potential for
clinical misinformation. Clin Chem 1998;44:440–54.
8 Ismail AAA et al. Wrong Biochemistry Results: Two Case Reports and
Observational Study in 5310 Patients on Potentially Misleading Thyroid-stimulating
Hormone and Gonadotropin Immunoassay Results. Clin Chem 2002;48:2023–29.
Statistics & graphics for the laboratory
43
Clinical concepts
Questionnaires to clinicians
Method
The clinicians or patients (self-monitoring) are presented for case histories or
situations in the form of paper vignettes.
The physician / patient is often asked about a critical difference (CD, or Dmed)
necessary to initiate/change an action.
Case story/situation
-Relevant problem
-Frequently encountered
-Laboratory test of especially importance
-Setting ”easily” described
-Conversational writing style; ”quoting” patients.
Example
Suppose you find in a patient under treatment the first laboratory
result given in the table. You follow the course of this patient
and/or the effect of treatment. Please circle the value that would
represent a significant change, in these cases improvement.
Result
1
2
3
4
5
Sodium
(mmol/l)
119
120
121
122
123
Potassium
(mmol/l)
2.4
2.5
2.6
2.7
2.8
Glucose
(mg/dl)
250
240
230
220
210
Protein
(g/l)
48
50
52
54
56
Elion Gerritzen WE. Analytical precision in clinical chemistry
and medical decisions. Am J Clin Pathol 1980;73:183-95.
Dependent on the question, the CD can comprise
• Pre-analytical variation
• Imprecision under defined conditions
• Within-subject variation (CVW-S)
• Group biological variation (CVG)
• Bias
Statistics & graphics for the laboratory
44
Questionnaires to clinicians
Probability considerations ("z")
Probability/certainty that patients (physicians) attribute to the answer of the
question depends on:
-Semantic of the question and - of course - the clinical situation
-Willingnes to accept false positive/negative actions
-”One sided” or ”two sided” situations
Establishment – Summary
• Production of adequate questionnaires
• Extraction of a medical action limit: Dmed
• Transformation of Dmed into an analytical specification for CV, bias or TE
Calculations
Monitoring: Dmed = Bias+z•2•SQRT[CVA2+CVW-S2]
 CVA = SQRT[{(Dmed-Bias)/z•2}2-CVW-S2]
Diagnosis: Dmed = Bias+1.65•SQRT[CVA2+CVG2]
 CVA = SQRT[{(Dmed-Bias)/1.65}2-CVG2]
Note: z = 1.65 (1.96) for "better/worse" ("change")
Calculation example
Imagine that you have measured your blood glucose level to be 8.0 mmol/l
A. To what value do you think your blood glucose must increase to before you
would be sure that it represents a true increase? ___
B. To what value do you think your blood glucose must decrease to before you
would be sure that it represents a true decrease? ___
Answers
Increase from
8.0 mmol/l
Decrease from
8.0 mmol/l
25th percentile
25%
25%
50th
30%
38%
percentile
Calculations
CVA = SQRT[{(D med-Bias)/z•2}2-CVW-S2 ]
D med = 25% increase from 8 mmol/l
Bias = 0
z = 1.65
CVW-S = 5%
 CVA ~ 10%
Statistics & graphics for the laboratory
45
Questionnaires to clinicians
Medical decision limits investigated
Medical decision limits (Dmed) related to biological variation (CVW-S) and analytical
state-of-the-art (CVA) (the data of Linnet and Skendzel were used for the
calculation)
When CVA = 0, is Dmed/CVW-S = 2.77
When CVA/CVW-S = 0.5, is Dmed/CVW-S = 3.10
When CVA/CVW-S = 1.5, is Dmed/CVW-S = 5.0
Analyte
CVA/CVW-S
D med/CVW-S
CVW-S
Sodium
1.5
4.5
0.8
Creatinine
1.4
5.9
4.4
Calcium
1.3
5.4
1.8
Glucose
0.8
5.1
4.4
Potassium
0.5
2.7
4.4
Bilirubin
0.5
1.9
23
Triglycerides
0.2
1
26
We observe incongruent Dmed values:
Dmed/CVW-S should be ~3.5 for glucose
Dmed/CVW-S should be ~2.8 for triglycerides (biological variation underestimated?)
Summary – Goals from Questionnaires to Clinicians
• Their establishment is complicated
• Usually, they reflect the analytical quality the clinicians got offered
• Sometimes they are based on incongruent Dmed values
• Often, biological variation is not adequately considered, with the consequence
that the derived values are mostly too generous
• They are very useful when they were established in the correct manner
Statistics & graphics for the laboratory
46
Questionnaires to clinicians
References
• Barnett RN. Medical significance of laboratory results. Am J Clin Pathol
1968;50:671-6.
• Barrett AE, Cameron SJ, Fraser CG, Penberthy LA, Shand KL. A clinical view of
analytical goals in clinical biochemistry. J Clin Pathol 1979;32:893–6.
• Elion Gerritzen WE. Analytical precision in clinical chemistry and medical
decisions. Am J Clin Pathol 1980;73:183-95.
• Skendzel LP, Barnett RN, Platt R. Medical useful criteria for analyte performance
of laboratory tests. Am J Clin Pathol 1985;83:200-5.
• Thue G, Sandberg S, Fugelli P. Clinical assessment of haemoglobin values by
general practitioners related to analytical and biological variation. Scand J Clin Lab
Invest 1991;51:453-9.
See also
• Gilbert RK. Progress and analytical goals in clinical chemistry. Am J Clin Pathol
1975;63:960-73.
• Koch DD, Oryall JJ, Quam EF, et al. Selection of medically useful quality-control
procedures for individual tests done in a multitest analytical system. Clin Chem
1990;36:230-3.
Statistics & graphics for the laboratory
47
Goals for "multipurpose" analytes
Goals for "multipurpose" analytes
Potassium test in a general laboratory
Urgency situations
• Digitalis intoxication
• Heart arrhytmias
• Acute renal failure
• Diarrhea
Therapy decisions, treatment with
• Angiotensin-converting enzyme blocker
• Digitalis
• Diuretics
Monitoring illness
• Diabetes mellitus
• Chronic renal failure
• Gastrointestinal diseases
Conclusions from the potassium example
It is NOT useful to set different goals for all the different situations.
Usually, 1 test is used for all applications in the laboratory.
Take the most stringent goal, then all situations are covered
• The proposed goals are independent of the magnitude of potassium present in a
specimen
• Always gaussian distribution and 5% test-level is considered
Weakness of the approach
• "One size NEVER fits all"
• If there are better goals, use them
Proposal: Establish goals from biological variation
Statistics & graphics for the laboratory
48
Goals from biology
Goals from biology
General considerations for analytical goals
Monitoring:
• Stable operation and low imprecision compared to the within-subject
biological variation.
Diagnostic Testing:
• Sufficient accuracy to allow the use of common reference intervals.
Goals from biology – The most important concepts
Tonks (1967)
Empirical concept
The total allowable analytical error (TEA) should not exceed one quarter of the
reference interval (RI).
TEA  1/4[RI]/[mean of RI] x 100%.
Statistical background
Reference interval = 4 • CVG  TEA  CVG
Cotlove, Harris & Williams (1970)
Relation of the analytical error (SDA) to a biological standard deviation (SDBiol)
Statistical background (in CV-terms)
CVA = 0.5 • CVBiol,
CVA adds only 12% to the
total test variability (CVT)
CVT = [(0.5 CVBiol)2 + CVBiol2]1/2
CVT = 1.12 CVBiol
Increase of
variation (%)
SDA  0.5 SDBiol
150
100
50
12%
0
0.0 0.5 1.0 1.5 2.0
CVA/CVBiol
Relationship between concepts
Tonks & Cotlove, Harris & Williams are quite similar
If bias is neglected, TEA defined by Tonks equals 2 SDA
The reference interval represents the mean ± 2 SDG = 4 SDG
Tonks formula can then be written as:
2 SDA  0.25 x 4 x SDG, equals SDA  0.5 SDG
Statistics & graphics for the laboratory
49
Goals from biology
Harris (1988)
Bias (B) and imprecision were both taken into account.
The original equation:
• SDA  0.5 SDW-S (SDG)
Was thus changed into:
• [SDA2 + B2]1/2  0.5 SDW-S (SDG).
Gowans et al. (1988)
Statistical background
Confidence interval (CI) of IFCC recommendations for establishment of a
reference-interval: n = 120
... translated into increase of SDG by SDA
m-1.96*s
m+1.96*s
Sample
size
10,000
1,000
N = 120
100
10
90 % CI
90 % CI
Statistics & graphics for the laboratory
50
Goals from biology
Gowans et al. (1988)
Statistical background > Imprecision
Confidence interval n = 120 translated into increase of SDG by SDA:
SDT = SQRT[SDG2 + SDA2]: =translated into imprecision
SDG alone, n = 
2.5% outside
SDT, n = 
4.6% outside
-5 -4 -3 -2 -1
0
1
2
3
4
5
90% CI, n = 120
Statistical background > Bias
Confidence interval n = 120 translated into bias
 0.25 CVG
Bias, n = 
4.6% outside
No bias, n = 
2.5% outside
-5 -4 -3 -2 -1
0
1
2
3
4 5
 CVG
90% CI, n = 120
Statistical background: IFCC reference-interval, n = 120
Consequence: Confidence interval for sample size
• n = 120, maximum 4.6% outside each limit
• n = , maximum 2.5% outside each limit
Model
• Allocate the difference n = 120 versus n = 
to analytical error
Outcome of the concept
• B  0.25 SDG (without imprecision)
• [SDA  0.6 SDG (without bias)] (original publication, meanwhile adapted)
• SDA  0.52 SDG (without bias; from "diagnostic curve": see later!)
Statistics & graphics for the laboratory
51
Goals from biology
Klee (1993)
Statistical background: Influence of imprecision and bias of a test on its
clinical specificity
Definition of an analytical "error budget": SQRT[SDA2+Bias2]
Limit: 50% increase in the false-positive rate for classifying healthy subjects
(unimodal distribution,
± 2s decision limit)
3.41% versus
2.28% outside
Outcome of the concept:
Error budget: 0.45 SDG
Allocated as
• 0.36 SDG for bias, &
-5 -4 -3 -2 -1 0 1 2 3 4 5
• 0.18 SDG for SDA
Monitoring goals from biology
The reference change value (RCV) (Least "medical significant difference")
(Harris & Yasaka Clin Chem 1983;29:25-30)
Calculations
Smallest medically significant difference (Dmed) that analytically can be detected for
two consecutive measurements with P = 0.05 and SDA = 0:
• Dmed = 1.96 x 2 x SDW-S = 2.77 x SDW-S
When D SE > 0 and SDA > 0, the equation can be rewritten (Larsen et al. 1991)
• Dmed = 2.77 x (SDA2 + SDW-S2)1/2 + D SE Eq. 1
When D SE = 0 and SDA = 0.5 SDW-S follows:
• Dmed = 2.77 (0.25 SDW-S2 + SDW-S2)1/2
• Dmed = 3.10 SDW-S
When SDA = 0 and Dmed = 3.10 SDW-S, Equation 1 can be rearranged:
• 2.77 x SDW-S + D SE = 3.10 SDW-S
Resulting in:
• D SE  0.33 SDW-S
Summary
 SDA  0.5 SDW-S
 Dmed = 2.77 SDW-S if SDA = 0
 Dmed = 3.10 SDW-S if SDA = 0.5 SDW-S
 D SE  0.33 SDW-S if SDA = 0
Statistics & graphics for the laboratory
52
Goals from biology
When SE and RE are present (complex calculations!)
Reference change (Dmed) concept (left figure):
relationship between allowable drift and CVA in fractions of CVW-S (DSE/ CVW-S and
CVA/CVW-S ).The figure shows the case Dmed = 3.10 CVW-S
Common reference interval concept (right figure):
relationship between allowable bias and CVA in fractions of CVG (SE/CVG and
CVA/CVG). Case where 4.6% are outside each reference limit.
Diagnosis
0.5
0.4
0.4
SE/CVG
SE/CVW-S
Monitoring
0.5
0.3
0.2
0.1
0.3
0.2
0.1
0
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0
0.1
CVA/CVW-S
0.2
0.3
0.4
0.5
0.6
CVA/CVG
Goals – Monitoring & Diagnosis: Which is most stringent?
Monitoring and diagnosis use different biological CV
• Assume CVb (between) = 1.75 • CVw (within)  CVg ~2 CVw
• Transform the CVg scale in diagnosis to CVw
Diagnosis transformed to
monitoring (CVb = 1.75 * CVw)
0,6
0,6
0,5
0,5
0,4
0,4
SE/CVw
SE/CVw
Monitoring
0,3
0,2
0,3
0,2
0,1
0,1
0
0
0
0,25
0,5
0,75
CVa/CVw
1
0
0,25
0,5
0,75
CVa/CVw
1
 Monitoring gives more stringent goals
Statistics & graphics for the laboratory
53
Goals from biology
Missing data for SDW-S and SDB-S
Assumption
• SDB-S = 1.75 • SDW-S, then SDG = SQRT(1.752 + 12) = 2 • SDW-S
With
• RI = 4 • SDG (RI: reference-interval)
Follows
• RI = 4 • 2 • SDW-S = 8 • SDW-S
> SDW-S = RI/8
Assumption
• SDA ~ 0,5 • SDW-S
Follows
• SDA ~ 1/15 RI,
Conclusion for SDA
• When data for SDW-S and SDB-S are missing, take RI/15 to comply with the
Harris concept
Statistics & graphics for the laboratory
54
Goals from biology
Summary
Goals from biology – Summary
Imprecision
(SDA)
Bias, Shift/Drift
(SE, or DSE)
Total Error
(TE)
—
—
0.5 SDBio
0.5 SDBio$
—
—
—
—
[SDA2+SE2]1/2
0.5 SDBio
Gowans et al.
0.5 SDBio
0.52 SDBio
0.33 SDBio
0.25 SDBio
"Graphics"
"Graphics"
Klee
0.18 SDBio
0.36 SDBio
0.45 SDBio
Tonks
Cotlove et al.
Harris
$Dependent on the concept, SDBio is within-, between-, or group
biological variation.
Conclusion: most values for CVA are ~0.5 CVBiol; most values for bias are ~0.25
or 0.33 CVBiol. Values for total error vary, another concept will be shown later (see
Ricos concept).
Goals from biology – Most often used
For monitoring as:
• SDA  0.5 SDW-S (in the absence of unidirectional systematic changes)
• DSE  0.33 SDW-S (when imprecision is negligible)
For diagnostic testing as:
• B  0.25 SDG (when the imprecision is negligible), or
• SDA  0.52 SDG (when bias is negligible; from "diagnostic curve")
Statistics & graphics for the laboratory
55
Goals from biology
TE from biology
The Ricos et al. concept$ (TE = SE + z • RE)
$Ricós C, Baadenhuijsen H, Libeer J-C, Hyltoft Petersen P, Stöckl D, Thienpont
LM, Fraser CG. Eur J Clin Chem Clin Biochem 1996;34:159-65.
TE = 0.25 • CVg + 1.65 • [0.5 • CVw]
CVg = group biological variation (can be calculated from CVw and CVb;
alternatively = ¼ of the reference interval)
CVw = within-subject biological variation
CVb = between-subject biological variation
Note: in www.westgard.com CVg = CVb
Remark: The Ricos et al. concept is a simplified, practical concept and a hybrid
between monitoring and diagnosis.
The Ricos et al. concept combines concepts for monitoring & diagnosis by use of
the respective RE/SE goals with SE = 0 and RE = 0.
Diagnosis
0,5
0,4
0,4
SE/CVg
SE/CVw
Monitoring
0,5
0,3
0,2
0,1
0,3
0,2
0,1
0
0
0
0,1 0,2
0,3 0,4 0,5 0,6
0
0,1 0,2 0,3 0,4 0,5 0,6
CVa/CVg
CVa/CVw
Remark
TEa is not constant in the biological concepts; it depends on the SE/RE-ratio:
Select a realistic SE/RE-ratio. Choose, in the monitoring case the point SE = RE
 TE = 0.66 CVw
TERicos = 0.25•CVg (=2•CVw) +1.65•[0.5•CVw] = 1.3 • CVw
TE monitoring (k: 1.65 to 2, SE/RE<1)
1,2
SE/CVw
1
The Ricos concept
gives too big TE-values.
TE
0,8
SE/REline
TE-line
0,6
0,4
Realistic
0,2
0
0
0,1
0,2
0,3
CVa/CVw
0,4
0,5
Statistics & graphics for the laboratory
56
Goals from biology
TE from biology
BUT: Some are too stringent for current technology
Biology gives extreme low values for some analytes:
TEa sodium: Ricos: 0.9% (http://www.westgard.com/biodatabase1.htm; but note:
CVg on that site = CVb)
Mon
SE=0
"0,5"
Analyte
Sodium
Chloride
Calcium
Protein
Albumin
Diag
RE=0
"0,25"
Ricos
"1,65"
CVw CVb CVg REmax SEmax TEmax
0.7
1.2
1.9
2.7
3.1
1.0
1.5
2.8
4.0
4.2
1.2
1.9
3.4
4.8
5.2
0.4
0.6
1.0
1.4
1.6
0.3
0.5
0.8
1.2
1.3
0.9
1.5
2.4
3.4
3.9
 Apply a bottom-line#
#Stöckl D. Desirable Performance criteria … based on biological analyte variation
- hindrances to reaching some and reasons to surpass some. Clin Chem
1993;39:913-4.
Bottom-line (for stable process!)
CV: 1%
Bias: 1,5%
TE: 3,2%
Consider to surpass some
Think about using more stringent goals than derived from biology  the goals are
a compromise; account for IQC; some goals are surpassed, by far, by current
technique.
• Stöckl D. Desirable Performance criteria … based on biological analyte variation
- hindrances to reaching some and reasons to surpass some. Clin Chem
1993;39:913-4.
• Tonks: 10% upper limit for TE
Mon
SE=0
"0,5"
Analyte
Urea
Triglycerides
CK
ALT
Bilirubin
Iron, total
Diag
RE=0
"0,25"
Ricos
"1,65"
CVw CVb CVg REmax SEmax TEmax
12.3
20.9
22.8
24.3
25.6
26.5
18.3
37.2
40.0
41.6
30.5
23.2
22.0
42.7
46.0
48.2
39.8
35.2
6.2
10.5
11.4
12.2
12.8
13.3
5.5
10.7
11.5
12.0
10.0
8.8
15.7
27.9
30.3
32.1
31.1
30.7
Statistics & graphics for the laboratory
57
Goals from biology
TE from biology
NOTE
When data for CVw and CVb are not available,
"Ricos":
take 1/6th of the reference interval! (assumption CVb = 1.75 • CVw)
Monitoring ("Most stringent"):
take 1/10th of the reference interval!
Conclusions
Desirable analytical quality should be based on within- and between-subject
biological variation because:
• the model is simple to understand and apply
• there are many data on biological variation
• the within- and between-subject biological variations are nearly constant over
geography
• when these criteria are fulfilled, the analytical quality will satisfy most clinical
needs
• the model has a great practical and educative value.
But
Sometimes they are too narrow (e.g., electrolytes), sometimes they are too broad
(e.g., enzymes).
They do not consider that different requirements might be necessary for different
concentrations.
Current approaches give too generous values.
Statistics & graphics for the laboratory
58
Goals from experts
Goals from "Experts"
Statistical background
• Usually, empiric
Establishment
• EQA-performance
• "State-of-the-art" performance
Statistics & graphics for the laboratory
59
Comparison of goals
State-of-the-art versus State-of-the-art
Comparison of imprecision data (“mean claims”) of the instrument
generation 2001 from 4/5 manufacturers.
Manufacturer Clinical chemistry Immunoassays
Beckman
Synchron LX20
Access
Bayer
---
Advia Centaur
Abbott
Aeroset
Architect
Roche
Modular
Elecsys
Ortho
Vitros 700
Vitros ECi
Analyte
Level
Na
mmol/l#
120
145
160
total-CV (%)
Best Worst
0.5
1.1
0.7
1.2
0.8
1.2
Analyte
Level
Mg
mmol/l
0.6
0.9
2
total-CV (%)
Best Worst
3
4
2
4.4
1
3.5
Cl
mmol/l
85
100
120
0.8
0.8
0.6
1.4
1.5
1.4
Crea
mg/dl
0.9
1.5
6
1.1
1
1.1
5
3.6
3
Ca
mmol/l
2
2.4
3.4
1.3
1.2
1.2
1.8
2
2.3
K
mmol/l
3
4.5
7
1.1
1
0.6
2
2
1.3
Prot
g/dl
4
5.5
7
1.1
1
1.1
2
2
2.5
Gluc
mg/dl
90
150
300
1.2
1.5
1.2
2.2
2.2
2.3
Alb
g/dl
2.5
3
4.5
0.9
0.8
0.6
2.3
2
1.7
Chol
mg/dl
100
150
250
1.5
1.6
0.8
1.8
1.8
2
ALP
U/l
100
250
600
3
1.1
2.5
4.5
3
3
Analyte
Level
Urea
mg/dl
20
50
80
total-CV (%)
Best Worst
1.8
6.1
1.8
2.1
1.6
1.9
#Note: SI Units, except
when all used conventional
Analyte
Level
total-CV (%)
Best Worst
1.3
5.7
1.7
3.2
0.9
2.7
LDH
U/l
150
500
1000
Phos
mg/dl
2.5
3.5
8
1.1
1.1
0.5
2.7
2.1
2.4
Trigly
mg/dl
80
150
250
1.1
0.8
1.6
2.2
1.6
1.8
UA
mg/dl
3
5
8
1.9
1.4
0.8
2.3
1.6
1.9
CK
U/l
100
300
600
2
1.3
0.9
5
4.6
4
ACP
U/l
5
10
30
3.9
3
1.1
4.9
5.3
2.7
ALT
U/l
30
150
400
2.2
1.3
0.7
6
2.9
2.3
Amyl
U/l
60
350
500
1
1
0.8
3.5
1.6
1.6
Bili
mg/dl
0.8
4
20
3.4
1.4
1.2
5.6
4.3
3.8
AST
U/l
30
150
250
1.6
0.8
0.7
4.5
2.4
2.4
Fe
µg/dl
80
120
250
1
1.2
0.7
3.5
2
2.1
Statistics & graphics for the laboratory
60
Comparison of goals
State-of-the-art versus State-of-the-art
Analyte
Level
total-CV (%)
Best Worst
2
7
2
4
3
5
Analyte
Level
TSH
µIU/ml#
0.1
5
30
TT4
nmol/l
50
100
200
5
2
3
fT4
pmol/l
6
15
40
TT3
nmol/l
fT3
pmol/l
Prog
nmol/l
3
40
85
7
6
8
E2
pmol/l
150
500
2000
6
4
3
20
8
6
4
3
4
10
7
6
Testo
nmol/l
2
15
40
7
3
3
8
6
6
1
2
5
3
2
2
8
5
5
LH
mIU/ml
5
50
100
3
2
2
9
4
5
3
10
20
4
3
3
15
9
8
FSH
mIU/ml
5
30
80
4
2
3
7
5
6
PRL
mIU/l
60
400
2000
4
3
4
8
5
6
Analyte
Level
B12
pg/ml
200
500
1000
#Note: SI Units, except
when all used conventional
total-CV (%)
Best Worst
3
6
2
5
3
6
total-CV (%)
Best Worst
6
13
3
8
3
8
Analyte
Level
total-CV (%)
Best Worst
8
12
4
8
4
10
AFP
IU/ml
10
100
500
CEA
ng/ml
5
30
80
3
3
3
6
5
3
Folate
ng/ml
2
7
20
9
6
5
10
8
7
ßhCG
mIU/ml
10
200
1000
5
3
3
7
6
5
Ferritin
ng/ml
10
150
450
4
3
3
5
6
6
PSA (t)
ng/ml
0.5
4
35
2
3
3
6
5
4
CA 19-9
IU/ml
30
100
250
4
3
4
9
5
4
CA 125
IU/ml
30
100
400
4
3
3
5
4
5
• There is a big difference between best and worst state-of-the-art
• Note: “Best/worst” do not refer to the complete system. E2 may be best on 1
system, but the same system may have a poor fT4 test
• For clinical chemistry, total CV’s of the best systems are generally <2%, with a
minimum of 0.5%
• For immunochemistry, total CV’s of the best systems are generally <5%, with a
minimum of 2%
Statistics & graphics for the laboratory
61
Comparison of goals
Questionnaire & Biology
The table compares analytical goals from questionnaires [1], with analytical goals
from biology [2]. It shows that goals from questionnaires may be very generous
(for example, Calcium: CV = 4.8%; Cholesterol: CV = 12.3%). A weakness in that
publication was that clinicians may have neglected the biological variation of the
analytes (see also p. 46).
Analyte
CVA[1]
CVA[2]
Calcium
4.8
0.9
Cholesterol
12.3
2.7
Glucose
11.2
2.2
Potassium
4.8
2.4
Urea
12.2
6.3
[1] Skendzel LP, Barnett RN, Platt R. Medical useful criteria for analyte
performance of laboratory tests. Am J Clin Pathol 1985;83:200-5 (Questionnaire).
[2] Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Proposed quality
specifications for the imprecision and inaccuracy of analytical systems for clinical
chemistry. Eur J Clin Chem Clin Biochem 1992;30:311-7 (Biology).
Statistics & graphics for the laboratory
62
Comparison of goals
European EQA (% limits from 13 countries) – "Expert goals"
Analytes arranged by increasing biological variation.
From: Ricos et al. Eur J Clin Chem Clin Biochem 1996;34:159-65.
The table shows an extreme variation in EQA-goals.
Proposed goals – Is there a consensus?
TEa of serum sodium measurement
Source
Biological variation
(Ricos data, “Westgard page”)
TEa (%)
0.9
¼ of the reference-interval
2.2
American EQA (CLIA)
(in % mean of RI)
2.9
German EQA
5
TEa of serum glucose measurement
Source
TEa (%)
Boyd & Bruns
(CC, 2001;47:209)
Biological variation
(Ricos data, “Westgard page”)
2.7
6.3
¼ of the reference-interval
10
American EQA (CLIA)
(in % mean of RI)
10
German EQA
15
Glucometers ("Upper A-line")
20
Unfortunately, there is no consensus in goal-setting!
Statistics & graphics for the laboratory
63
Comparison of goals
Overview
Clinical
Biological
Regulation
Stateof-the-art
Availability
Very few
General
General
General
Applicability
General
Restricted to
(note: "too
specific
wide/too
situations
strict")
General
General
Agreement
Fair
Fair
Big diversity
Some
diversity
Remark
For special
centers
(endocrine
labs)
Useful as
goals in
method
selection
Most useful Actually
for staying in available
business
quality
Goals for analytical quality –Résumé
Most obvious for daily practice
• The “own” state-of-the-art
• Legislation/EQA
Most relevant for the patient
• ”Clinical”
• ”Biological”
Most realistic as goal
• The “better” state-of-the-art
• ”Monitoring”/”diagnosis”
Relevance beyond establishing analytical goals
Clinical concept
Important for test selection (“a-priori quality”)
Questionnaires to clinicians
Important for the use of a test in practice
Biology
• Determines the treshold for action (Dmed monitoring)
• Determines the “gray” zone in diagnosis
Statistics & graphics for the laboratory
64
Comparison "state-of-the-art" with goals
State-of-the-art/"Harris" (CVa  0.5  CVW-S)
Some remarks at the beginning
Currently insufficiently known:
• Bias from comparison with a reference method
• SE from calibration tolerance/IQC limits
• The global effect of unspecificity/interference
Usually, SE goals cannot be investigated and TE goals, only, can be
compared with 2 • CV
 Comparisons are restricted to RE
TE goals from biology: "1 number for all"
 Comparisons are restricted to CV's in the reference interval
CVa (tot)
Analyte
Level
Goal
Best
Na, mmol/l
145
0.4
0.7
Cl, mmol/l
100
0.6
0.8
Ca, mmol/l
2.4
1
1.2
Prot, g/dl
5.5
1.4
1
Alb, g/dl
3
1.6
0.8
Mg, mmol/l
0.9
1.8
2
Crea, mg/dl
1.5
2.2
1
K, mmol/l
4.5
2.4
1
Glu, mg/dl
150
2.5
1.5
Chol, mg/dl
150
3
1.6
ALP, U/l
250
3.2
1.1
CVa (tot)
Analyte
Level
Goal
Best
LDH, U/l
500
3.3
1.7
Phos, mg/dl
3.5
4.3
1.1
UA, mg/dl
5
4.3
1.4
ACP, U/l
10
4.5
3
Amyl, U/l
350
4.8
1
AST, U/l
150
6
0.8
Urea, mg/dl
50
6.2
1.8
Trigl, mg/dl
150
10.5
0.8
CK, U/l
300
11.4
1.3
ALT, U/l
150
12.2
1.3
4
12.8
1.4
120
13.3
1.2
Bili, mg/dl
Fe, µg/dl
Statistics & graphics for the laboratory
65
Comparison "state-of-the-art" with goals
State-of-the-art/"Harris" (CVa  0.5  CVW-S)
CVa (tot)
Analyte
Level
Goal
Best
TT4, nmol/l
100
3
2
PRL, mIU/l
400
3.5
3
fT4, pmol/l
15
3.8
3
fT3, pmol/l
10
4
3
TT3, nmol/l
2
4.4
2
Testo, nmol/l
15
4.4
3
CEA, ng/ml
30
4.7
3
FSH, mIU/ml
30
5.1
2
CA 125, IU/ml
100
6.8
3
PSA (t), ng/ml
4
7
3
50
7.3
2
LH, mIU/ml
CVa (tot)
Analyte
Level
Goal
Best
Ferritin, ng/ml
150
7.5
3
TSH, µIU/ml#
5
9.9
3
E2, pmol/l
500
11.3
4
CA 19-9, IU/ml
100
12.3
3
Prog, nmol/l
40
---
3
AFP, IU/ml
100
---
2
ßhCG, mIU/ml
200
---
3
B12, pg/ml
500
---
4
7
---
6
Folate, ng/ml
Other way of presentation
We can look at the contribution of the analytical variance to the total variance.
For CVa  0.5  CVW-S this translates to CVA2  20% of CVtot2;
CVtot2 = [CVA2 + CVW-S2] = [0.25 CVW-S2 + CVW-S2] = 1.25 CVW-S2.
Statistics & graphics for the laboratory
66
Comparison "state-of-the-art" with goals
Contribution of analytical variance to total
CVA2 in % of [CVA2 + CVW-S2]; Basis: Best CVA and CVW-S
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Sodium
Chloride
Calcium
Magnesium
PRL
fT4
fT3
Protein
Testosterone
Acid Phosph.
TT4
CEA
Glucose
Cholesterol
Album in
LDH
Creatinine
TT3
CA 125
PSA (t)
Potassium
Ferritin
FSH
Estradiol
Alk. Phosph.
Uric acid
TSH
Urea
LH
Phosphate
CA 19-9
a-Am ylase
AST
CK
Bilirubin
ALT
Iron, total
Triglycerides
"Goal"
CV A = 0.5 CV W
CVA2 in % of [CVA2 + CVG2]; Basis: Best CVA and CVG
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Sodium
Chloride
Calcium
Magnesium
Acid Phosph.
fT4
Protein
Glucose
Album in
Ferritin
TT4
Potassium
Testosterone
Estradiol
TT3
LDH
Cholesterol
TSH
Phosphate
Urea
Creatinine
Uric acid
LH
CA 125
FSH
CEA
PRL
Alk. Phosph.
PSA (t)
AST
Bilirubin
Iron, total
a-Am ylase
CA 19-9
CK
ALT
Triglycerides
"Goal"
CV A = 0.52 CVG
State-of-the-art/"Harris" (CVa  0.5  CVW-S)
1st summary and remarks
• Bias (SE), and consequently TE, cannot be assessed reliably
The optimistic point-of-view
For imprecision: the best state-of-the-art seems to be close to the currently
used biological goals: Exceptions: Na, Cl, Ca & Mg
BUT: GOALS are for the laboratory!
 Consequences for the manufacturer?
What if bias, interferences, etc. are considered?
Statistics & graphics for the laboratory
67
Analytical goals – Translation into practice
Analytical goals – Translation into practice
GOALS are for the laboratory: TE = SE + 1.65 x RE!
Translation requirements for the manufacturer
• Give room for the laboratory
• Leave room for realistic IQC procedures
 As manufacturer,
consume only a part of the goal! : 1/2 to 1/4?
 What about goals expressed as total error?
How to fractionate (allocate) TE in SE/RE components?
To answer these questions, we will address
-Finding the "manufacturer line"
-TE-calculation with 1- and 2-sided random error
-Distributions at different SE/RE ratios (position of an "operating point")
-"Fractionation" of TE at various ratios of SE/RE
For this purpose, we will make use of the Westgard total error charts.
Construction of TE-charts (Example TE = 10%)
10
Bias (%)
8
k = 1.65
k=2
k=3
k=4
k=6
6
4
2
0
0
2
4
6
CV (%)
TE charts are constructed by use of the formula TE = SE + k x RE. The SE
component is plotted on the y-axis, the RE component on the x-axis.
First, SE is calculated for RE = 0;  SE = 10%
Second, RE is calculated with SE = 0;  RE = 10%/k; for k = 2  RE = 5%.
Then, the 2 points are connected. All points on the line follow the respective
equation 10% = SE + k x RE.
Laboratory line: k = 1.65!
Statistics & graphics for the laboratory
68
Analytical goals – Translation into practice
Finding the "manufacturer line"
How big should “k” be?
Give room for IQC
We look into the Westgard TE-charts used for IQC: "OPSpecs-charts"
OPSpecs Chart TEa 10% with 90% AQA (SE)
Allowable bias (%)
12
k = 1.65
10
8
The laboratory line
IQC multirule
6
IQC 1,2.5s(n=4)
&k=4
4
IQC 1,2.5s(n=2)
2
The manufacturer line
0
0
1
2
3
4
5
6
7
Allowable CV (%)
From IQC OPSpecs: “k” should (at least) be 4
(Note: nearly all IQC-lines are at the left of k = 4)
Statistics & graphics for the laboratory
69
Analytical goals – Translation into practice
TE-lines: the 1-sided/2-sided problem
Populations at different operating points (TE = 10%)
Move operating point along the 1.65-line (1-sided 95% TE limit)
Populations with high SE: 1-sided, 5% out of +10% TE-limit
Populations with low SE: 2-sided, 10% out of ±10% TE-limit
The 1-sided case (high SE) changes gradually to the 2-sided case
Populations at different operating points (TE = 10%)
The 2-sided
case becomes relevant at SE  RE
0.8
0.7
1,65s
0.6
RE = 0,5%
0.5
RE = 2%
0.4
0.3
RE = 6,1%
0.2
TE
0.1
0
-5
0
5
10
15
 TE curves at fixed “out-of-limit-percentage”, principally,
should respect 1- & 2-sided probabilities.
10
1,65 s
SE (%)
8
6
SE = RE
2s
4
2
0
0
2
4
6
8
RE (%)
At SE  RE (SE/RE  1) the probabilities become practically 1-sided
Statistics & graphics for the laboratory
70
Analytical goals – Translation into practice
Distributions at different SE/RE ratios
Chart with the TE-lines: TE = SE + k • RE (k = 1.65/2/3/4)
TE concept: TE = SE + k*RE (k=1.65/2/3/4)
Move along different
SE/RE ratios and look at the populations at the intersection
Moving along different SE/RE ratios
points of the SE/RE-lines with the TE-lines at different k-values.
12
10
1,65 s
2s
3s
4s
SE/RE = 16
SE/RE = 2
SE/RE = 1
SE/RE = 0,5
SE/RE = 1/16
8
6
4
2
0
0
2
4
6
8
SE/RE = 1/16 Robust situation
An excellent method
(k = 4) allows additional SE
Moving along SE/RE = 1/16
0.250
0.200
1,65 s
0.150
2s
3s
0.100
4s
TE
0.050
0.000
-5
0
5
10
15
SE/RE = 16 Labile situation
An excellent method (k = 4) does
not allow much more SE
Moving along SE/RE = 16
0.800
0.600
1,65 s
2s
0.400
3s
4s
0.200
TE
0.000
-5
0
5
10
15
Statistics & graphics for the laboratory
71
Analytical goals – Translation into practice
Observations from the Westgard TE-chart
Position of an operating point
Operating points at big SE/RE ratios describe “labile” analytical procedures
-The difference between excellent and poor method (Westgard
terminology) is marginal
Operating points at small SE/RE ratios describe “robust” analytical procedures.
Method decision chart
The distinction between excellent to poor methods is meaningless at high SE/RE
Method Decision Chart: Kalium - TE = 8%
ratios.
8
7
Marginal
Bias (%)
6
5
4
Poor
3
Oper.
point
Good
2
Excellent
1
0
0
1
2
3
4
CV (%)
"Fractionation" of TE at various ratios of SE/RE
Calculation of TE along SE/RE = 16 & 1/16 at different k-values
TE = SE + 1.65 • RE (SE/RE = 16)
TE = SE + 2 • RE (SE/RE = 1/16)
Further observations from the Westgard TE-grid (TE = 10%)
Along:
The SE/RE = 16 line
The SE/RE = 1/16 line
k
TE
k
TE
1.65
10
2
10
3
9.3
3
6.7
4
8.8
4
5.1
 Fractionation of TE occurs only at “medium to low” SE/RE ratios
Statistics & graphics for the laboratory
72
Analytical goals – Translation into practice
“Allocation and fractionation” of TE
Bias (%)
Method Decision Chart
Direction of
error fractionation
-k
-k
-k
-k
Moving along SE/RE = 1
= 1.65
=2
=3
=4
2%
TEgoal
1.65 s
4s
3.8%
-5
CV (%)
0
5
SE (%)
10
15
TE goal 10%
Direction [0;0] to [SE=RE]; move from k = 1.65 to k = 4
• k = 1.65: SE = 3.8%; RE = 3.8%
 at k = 4: SE = 2%; RE = 2%  TE (k = 4) = 5.3%
Direction [0;0] to [SE=0;RE]; move from k = 2 to k = 4
• k = 2: SE = 0%; RE = 5%
 at k = 4: SE = 0%; RE = 2.5%  TE (k = 4) = 5%
Movement in the Method Decision Chart at a SE/RE = 1 ratio from the 1.65s-line
to the 4s-line
TE drops to 53% of original
Movement in the Method Decision Chart at SE = 0 from the 2s-line to the 4s-line
 TE drops to 50% of original
The fraction of the manufacturer
Start from the lab TE for monitoring (#) and use
TE = SE + 1.65 • RE
Move in the Method Decision Chart at a SE/RE = 1 ratio from the 1.65s-line to the
4s-line [for SE = 0 from the 2s-line to the 4s-line]
 TE manufacturer 50% of TE lab
Take as bottom-line (Stöckl, Clin Chem):
CV ~ 0.5%, SE ~ 1%,
TE ~ 1.8% (k = 1.65)
Laboratory bottom-line (TE~2 • manufacturer):
CV ~ 1%, SE ~ 1.5%, TE ~ 3.2%
(#) When the biological variation is unknown, use
TE = 1/10 RI ( TE monitoring at RE = SE).
Statistics & graphics for the laboratory
73
Goals – future vision
Laboratory goals from biology – future vision
• Always consider SE & RE
• Choose a TE-value
in relation to a certain SE/RE-ratio
• Use “bottom-line” goals
RE = 1%, SE = 1.5%, TE = 3.2%
• [Use maximum values]
RECALL
“Sometimes too small, sometimes too broad”
Be practical:
Tonks/Stöckl: Use upper-/lower limits
Tonks: 10% upper limit for TE
Stöckl: 1% (0.5%) lower limit for bias (imprecision)
Tonks DB. A study of accuracy and precision of clinical chemistry determinations in
170 canadian laboratories. Clin Chem 1963;9:217-33.
Stöckl D. Desirable Performance criteria for quantitative measurements in medical
laboratories based on biological analyte variation - hindrances to reaching some
and reasons to surpass some. Clin Chem 1993;39:913-4.
Current and future goals compared
Laboratory
Current: CVa  0.5  CVW-S ("Harris"; without bias)
Future: CVa  0.25  CVW-S ("Advanced laboratory goal"; with bias)
Manufacturer
Current: CVa  0.5  CVW-S ("Harris"; without bias)
Future: 50% of "Advanced laboratory goal"
CVa  0.125  CVW-S ("Advanced manufacturer goal")
Future = ¼ of current!
Statistics & graphics for the laboratory
74
Goals – future vision
CVA – Manufacturer "advanced"/"State-of-the-art"
Contribution of analytical uncertainty to total (compare with p. 67!)
UA2 in % of [UA2 + CVW-S2]
Laboratory UA (incl. Bias, etc) = 4 x best CVA and CVW-S
CV A = 0.5 CVW
Sodium
Chloride
Calcium
Magnesium
PRL
fT4
fT3
Protein
Testosterone
Acid Phosph.
TT4
CEA
Glucose
Cholesterol
Album in
LDH
Creatinine
TT3
CA 125
PSA (t)
Potassium
Ferritin
FSH
Estradiol
Alk. Phosph.
Uric acid
TSH
Urea
LH
Phosphate
CA 19-9
a-Am ylase
AST
CK
Bilirubin
ALT
Iron, total
Triglycerides
"Goal"
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
UA2 in % of [UA2 + CVG2]
Laboratory UA (incl. Bias, etc) = 4 x best CVA and CVG
CV A = 0.52 CV G
Sodium
Chloride
Calcium
Magnesium
Acid Phosph.
fT4
Protein
Glucose
Album in
Ferritin
TT4
Potassium
Testosterone
Estradiol
TT3
LDH
Cholesterol
TSH
Phosphate
Urea
Creatinine
Uric acid
LH
CA 125
FSH
CEA
PRL
Alk. Phosph.
PSA (t)
AST
Bilirubin
Iron, total
a-Am ylase
CA 19-9
CK
ALT
Triglycerides
"Goal"
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
2nd summary and remarks
• Assumption: bias, interferences, etc
• Assumption: the uncertainty of the laboratory is 4 x the best manufacturer
performance
The pessimistic point-of-view
There are very many problem analytes according to the biological concept, in
particular, in the monitoring situation!
Statistics & graphics for the laboratory
75
Outlook
Analytical quality – still an issue?
We believe YES
Quality compared to goals
Stöckl D. Modern quality management misunderstood? Clin Chem 1998;44:1066-7.
Others believe NO
Quality compared to own process specifications
Quality evaluated against “harm” but not against usefulness
-Blumenthal D. The errors of our ways. Clin Chem 1997;43:1305.
-Plebani M, Carraro P. Mistakes in a stat laboratory. Clin Chem 1997;43:1348-51.
-Witte DL et al. Errors, mistakes, blunders, and outliers, or unacceptable results:
how many? Clin Chem 1997;43:1352-6.
Analytical quality – The future?
• Market transparency – independent of goals
-Why should I pay the same for different quality
(compare with reference methods)
• Quality assessed against usefulness (“does not harm” is not sufficient anymore)
-Evidence-based
• New analytes
• Reconsider old/develop new diagnostic strategies (#)
(#) Hammond HC. Applying the value-of-information paradigm to laboratory
management. Clin Lab Manage Rev 1996;10:98-106. (measuring, or not, glucose for
the early identification of diabetes)
Priority list for improvement
Imprecision
-medium priority (between day)
Bias/traceability
-high priority for some analytes
(be careful during “restandardisation”)
[Diagnostic]accuracy on individual sample
-highest priority (see hCG example, Lancet 2000;355:712)
Statistics & graphics for the laboratory
76
Outlook
Analytical quality – the future?
Remember, the influence on TE of:
• Bias (traceability)/SE
• Sample related effects (overall matrix)
• Specificity (cross-reactions)
• Common interferences (lipemia, etc.)
• Effects of drugs
• Auto-/heterophilic antibodies
• Genetic variants
Insufficiently known
 Important for the future:quality for the individual sample (patient!)
Remarks
• Know the different concepts for deriving analytical quality specifications (=
goals), but be able to use them in the appropriate manner
• Make a difference between “desirable” and required quality
• Take critically part in establishing (inter)national specifications
• Stay informed about “state-of-the-art” quality
Analytical goals – Which ones to use?
Document your own quality
Compare it with the “state-of-the-art”
• Identify therefrom “problem analytes”
Compare it with international goals
• Identify therefrom “problem analytes” or “problem goals”
Identify “problem analytes” by communication with the laboratories/clinicians
Identify “possible” problem analytes by comparison with “biological“ specifications:
biological variation, ref.-intervals
Verify “possible” problem analytes by comparison with “clinical“ specifications and
“bottom-line” goals
Check whether you want to improve the quality of some tests, independent of
proposed goals
Statistics & graphics for the laboratory
77
References
Data on biological variation
• Ross JW. Evaluation of precision. In: Werner M, ed. CRC Handbook of Clinical
Chemistry. Boca Raton, Fla: CRC Press: 1982;1:391-422.
• Fraser CG. The application of theoretical goals based on biological variation data in
proficiency testing. Arch Pathol Lab Med 1988;112:404-15.
• Fraser CG. Biological variation in clinical chemistry. An update: collated data, 19881991. Arch Pathol Lab Med 1992;116:916-23.
• Fraser CG. Databases for facilitating work on setting quality specifications: biological
variation. Upsala J Med Sci 1993:98:415-6.
• Sebastian-Gambaro MA, Liron-Hernandez FJ, Fuentes-Arderiu X. Intra- and interindividual biological variability data bank. Eur J Clin Chem Clin Biochem 1997;35:845-52.
Selected references
Analytical goals
• Stöckl D, Baadenhuijsen H, Fraser CG, Libeer J-C, Hyltoft Petersen P, Ricós C.
Desirable routine analytical goals for quantities assayed in serum (Discussion paper from
the members of the EQA Working Group A on analytical goals in laboratory medicine).
Eur J Clin Chem Clin Biochem 1995;33:157-69.
• Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Proposed quality specifications for
the imprecision and inaccuracy of analytical systems for clinical chemistry. Eur J Clin
Chem Clin Biochem 1992;30:311-7.
• Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Quality specifications. In: Haeckel
R, editor. Evaluation methods in laboratory medicine. Weinheim: VCH
Verlagsgesellschaft, 1993:87-99.
• Ross JW. Evaluation of precision. In: Werner M, ed. CRC Handbook of Clinical
Chemistry. Boca Raton, Fla: CRC Press: 1982;1:391-422.
• Fraser CG. The application of theoretical goals based on biological variation data in
proficiency testing. Arch Pathol Lab Med 1988;112:404-15.
• Fraser CG. Biological variation in clinical chemistry. An update: collated data, 19881991. Arch Pathol Lab Med 1992;116:916-23.
• Fraser CG. Databases for facilitating work on setting quality specifi-cations: biological
variation. Upsala J Med Sci 1993:98:415-6.
• Sebastian-Gambaro MA, Liron-Hernandez FJ, Fuentes-Arderiu X. Intra- and interindividual biological variability data bank. Eur J Clin Chem Clin Biochem 1997;35:845-52.
• Tonks DB. A study of accuracy and precision of clinical chemistry determinations in 170
canadian laboratories. Clin Chem 1963;9:217-33.
• Cotlove E, Harris EK, Williams GZ. Components of variation in long term studies of
serum constituents in normal subjects. III. Physiological and medical implications. Clin
Chem 1970;16:1028-32.
• Harris EK. Statistical principles underlying analytical goal-setting in clinical chemistry.
Am J Clin Pathol 1979;72:374-82.
• Elevitch FR, editor: Analytical goals in clinical chemistry. Proceedings of the CAP Aspen
Conference; 1976; Skokie (IL). Skokie (IL): College of American Pathologists, 1977.
• Proceedings of the subcommittee on analytical goals in clinical chemistry, World
Association of Societies of Pathology, London, Analytical goals in clinical chemistry: Their
relationship to medical care. Am J Clin Pathol 1979;71:624-30.
• Harris EK. Proposed goals for analytical precision and accuracy in single point testing.
Arch Path Lab Med 1988;112:416-20.
Statistics & graphics for the laboratory
78
References
Selected references
• Ross JW. A theoretical basis for clinically relevant proficiency testing evaluation limits:
Sensitivity analysis of the effect of inherent test variability on acceptable method error.
Arch Path Lab Med 1988;112:421-34.
• Ross JW, Fraser MD. Analytical goals developed from inherent error of medical tests.
Clin Chem 1993;39:1481-94.
• Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Horder M. Analytical goals for the
acceptance of common reference intervals for laboratories throughout a geographical
area. Scand J Clin Lab Invest 1988;48:757-64.
• Klee GG. Tolerance limits for short-term analytical bias and analytical imprecision
derived from clinical assay specificity. Clin Chem 1993;39:1514-8.
• Harris EK, Yasaka T. On the calculation of a "reference change" for comparing two
consecutive measurements. Clin Chem 1983;29:25-30.
• Hyltoft Petersen P, Fraser CG, Westgard JO, Lytken Larsen M. Analytical goal-setting
for monitoring patients when two analytical methods are used. Clin Chem 1992;38:225660.
• Queralto JM, Boyd JC, Harris EK. On the calculation of reference change values, with
examples from a long-term study. Clin Chem 1993;39:1398-1403.
Goals from "experts"
• Richtlinien der Bundesärztekammer zur Qualitätssicherung in medizinischen
Laboratorien. Dtsch Ärztebl 1988;85:A699-A712.
• Department of Health and Human Services. Medicare, Medicaid and CLIA programs;
regulations implementing the Clinical Laboratory Improvement Amendments of 1988
(CLIA). Fed Regist, Feb 28, 1992;57:7002-186.
• Belgisch Staatsblad - Moniteur Belge. 01.04.1996. N. 96-716.
• Ricós C, Baadenhuijsen H, Libeer J-C, Hyltoft Petersen P, Stöckl D, Thienpont LM,
Fraser CG. External quality assessment: currently used criteria for evaluating
performance in European countries, and criteria for future harmonization. Eur J Clin Chem
Clin Biochem 1996;34:159-65.
Reference intervals
• Harris EK, Boyd JC. Statistical bases of reference values in laboratory medicine.
Washington: AACC Press, 1995:384pp.
• Burtis CA, Ashwood ER, eds. Tietz Textbook of Clinical Chemistry. 2nd ed. Philadelphia
(PA): W.B. Saunders Co, 1994.
• Heil W, Schuckließ F, Zawta B. Reference ranges for adults and children. Klinikum
Wuppertal-Barmen, Universität Witten-Herdecke und Boehringer Mannheim GmbH: 1995.
• Soldin SJ, Hicks JM, Gunter KC, Brugnara C (editors). Pediatric reference ranges. 2nd
edition. Washington: AACC Press, 1997:181pp.
• Perkins SL, Livesey JF, Belcher J. Reference Intervals for 21 clinical chemistry analytes
in arterial and venous umbilical cord blood. Clin Chem 1993;39:1041-4.
• Tietz NW, Shuey DF, Wekstein DR. Laboratory values in fit aging individuals sexagenerians through centenarians (special report). Clin Chem 1992;38:1167-85.
• Caroli S, Alimonti A, Coni E, Petrucci F, Senofonte O, Violante N. The assessment of
reference values for elements in human biological tissues and fluids: a systematic review.
Crit Rev Anal Chem 1994;24:363-98.
Statistics & graphics for the laboratory
79
Notes
Notes
Statistics & graphics for the laboratory
80
Notes
Notes
Statistics & graphics for the laboratory
81