Transcript Document

OUTLINE OF PRESENTATION
1. Systematic(bias)error in epidemiological
studies
2. Validity and testing for validity
3. Reliability and testing for reliability
4. Group task for today
1. Bias is a systematic error in the design, conduct, or
analysis of a study that results in errors when
calculating measures of association between risk
factors and outcomes
2. It is thus always one-sided, either underestimating or
over-estimating risk
3. It is not usually a subjective error, and is often
unavoidable, but must be recognised as a limitation of
the study to allow for correct interpretation
Types of bias in epidemiological studies
Selection bias: In selection bias study participants are selected in one locality
of convenience and are therefore not representative of the target population
e.g. Selection of a sample for high prevalence of alcoholic foetal syndrome in
western cape is not representative of prevalence of alcoholic foetal syndrome
in SA.
Information bias /Recall bias:
Recall bias result because those affected by a disease often recall their
exposures better than those who are unaffected e.g. Those who were exposed
to polio virus will remember better than those who were unexposed
Non-response bias:
leads to a lack of information on non-responders. Thus if the non-response
rate is high, the study may be severely biased. Sometimes called volunteer
bias.
4.
Misclassification bias:
Validity of a test
The validity of a test is defined as its ability to distinguish between those who have the
disease(Sensitivity )and those who do not have the disease(Specificity).
Factors affecting validity:
1. Incorrect calibration and or uncalibration of the measuring device e.g.
Sphygmomanometer(internal validity)
2. Unstructured interviews or questionnaires in a study (internal validity
3. Unrepresentativeness of the sample from the population(external validity.
Components of validity
Validity of test has therefore two components namely sensitivity and specificity.
Sensitivity
Sensitivity of a test is defined as the ability of a test to identify correctly those who
have the disease
Specificity.
Specificity of a test is defined as the ability of a test to identify correctly those who do
not have the disease.
True characteristic in the population
Test
Result
Have the
disease
Positive True positive(TP)
Have the disease
and test positive
Negative
Do not have
the disease
False Positive(FP)
Do not have the disease
but test positive
False negative(FN)
Have the but
True Negative(TN)
Test negative
and test negative
Sensitivity = TP
TP +FN
Do not have the
Specificity = TN
TN+FP
Types of validity tests
Gold standard test
Screening test
Gold standard test refers to a diagnostic test or benchmark that is the best
available under reasonable conditions. It is referred to as the most accurate test
possible without restrictions
A hypothetical ideal of gold standard test has a
1. sensitivity of 100% with respect to the presence of the disease (it identifies all
individuals who have the disease; it does not have any false-negative results)
Sensitivity =
a
a +c
OR
TP
TP +FN
Also a hypothetical ideal of gold standard test has a
2. specificity of 100% with respect to the absence of disease (it
identifies all individuals who do not have the disease.it does
not have any false-positive results).
Specificity = d
d +b
OR
TN
TP +FP
Example:
Suppose we have a hypothetical population of
1000 people, the gold standard will yield a
sensitivity of 500 with respect to the presence
of disease with no false negative test results and
500 with respect to the absence of disease with
no false positive test results.
See below as shown.
True characteristics in the population
Test
Results
Have the
disease
Positive
Negative
Total
500
0
500
Sensitivity= 500
500+0
= 100%
Do not
have the disease
0
500
500
Specificity= 500
500+0
= 100%
In practice, there are sometimes no true "gold standard" tests.
SCREENING VALIDITY TEST
Screenings tests are tests that look for diseases before you have
symptoms. Screening tests can detect diseases early and thus start
treatment
Detectable
Preclinical
phase
Clinical
phase
Clinical
outcome
No disease
Symptoms
first
appear
Biologic
onset of
disease
Disease
detectable
by screening
disease
diagnostic
test
Therapy
given
• Hypothetical example:
Suppose we have a hypothetical population of
1000 people of whom 100 have a certain
disease and 900 do not. In this instance we use a
screening test to distinguish persons with the
disease from those who do not.
The results obtain by applying the test to this
population of 1000 people are shown below
True characteristics in the population
Test
Results
Have the
disease
Positive
80
100
Negative
20
800
Total
100
900
Sensitivity= 80
80 +20
= 80%
Do not
have the disease
Specificity= 800
800 +100
= 89%
• The question we now ask is how good is the screening test
compared to the gold standard.
First, the test indicates that of the 100 people with the
disease,80 were correctly identified as positive and a positive
identification was in 20 and thus the sensitivity of the test
which is defined as the proportion of diseased people were
correctly identified as positive by the test is 80/100 or 80%
Secondly, the test indicates that of the 900 people who did
not have the ,the test correctly identified 800 people as
negative and thus the specificity f the test, which is defined as
the proportion of non diseased people who are correctly
identified as negative by the test is therefore 800/900 or 89%
Why is the issue of false positive important?
The issue of false positive is important because:
1. All people who screened positive are brought
back for sophisticated and more expensive test
which becomes a burden on the health care
system.
2. Another is the anxiety and worry induced in
persons who have been told that they tested
positive and this labelling is not completely
erased even if the results of subsequent
evaluation are negative.
Why is the issue of false negative important?
The issue of false positive is important because:
1. If for example, the disease is a type of cancer that is
curable only in its early stages, a false negative result
could represent a virtual death sentence.
2. Thus the result depends on the nature of the severity
of the disease being screened for effectiveness of
available intervention measure and whether the
effeteness is greater if the intervention is
administered early in the natural history of the
disease
Predictive value of a test
We have so far asked the question "How good is the test(gold std) at
identifying people with the disease and people without the disease”.
Put in another way "If we screen a population, what proportion of
people who have the disease will be correctly identified?”
These question is clearly for public health consideration.
Positive predictive value of a test
In the clinical setting, a different question for the clinician would be "If
the test results are positive in this patient, what is the probability
that this patient has the disease "This is called the positive predictive
value(PPV) In other words the question is “what proportion of patients
who test positive actually have the disease in question?
Negative predictive value of a test
Also a parallel question can be asked about
negative test results as follows
“If the test results is negative ,what is the
probability that this patient does not have the
disease "This is called negative predictive
value(NPV) of the test.
Calculation of both PPV and NPV
PPV of a test is calculated by dividing the
number of true positive(TP) by the total number
of people who tested positive, i.e. TP divide by
TP plus FP
NPV of a test is calculated by dividing the
number of true negatives(TN) by the total
number of people who tested negative i.e. TN
divide by TN plus FN
True characteristics in the population
Test
Results
Have the
disease
Do not
have the disease
Positive
80
100
Negative
20
800
Totals
180
PPV=80/180=44%
820 NPV=800/820=90%
A Positive predictive value of 44% means that of the 180 tested
only 80 have the disease
A Negative predictive of 90% means that of the 820 tested, only
800 do not have the disease
Reliability(Repeatability) of a diagnostic or screening tests
Reliability is the ability of a measure(both the subject and the observer to
produce a repeatable result .
This does not mean that the result is accurate, it just means that if the test
is repeated, a similar result will be obtained.
Clearly ,regardless of the sensitivity and specificity of test, if the test
results cannot be reproduced, the value of and usefulness of the test are
minimal
The factors contributing to the variability or non reliability of a test are
listed.
Intra-subject variation( variation within individual subject0
Intra-observer variation (variation in reading of test result by same person
Inter-subject variation(variation of test results between or among
individual
Intra-observer variation(variation of test results between observers
Number of
patients
1
2
3
4
5
6
7
8
9
10
n=10
Observer 1
Observer 2
0
0
1
0
1
0
1
0
1
1
0
1
0
0
0
0
1
0
0
1
0=Negative
1=Positive
Positive for test 1 and 2Patients 7 and 10= 2 positive’s
Negative for 1 and 2Patient 1, 4, 6, 8 =4 negative’s
Positive for 1 and negative for 2Patient 3, 5, and 9=3 positives
Negative for 1 and positive for 2 Patient 2, =1 negative
Observer 1
Positive Negative Total
Observer 2 Positive
2
1
3
Negative
3
4
7
Total
5
5
10
Calculate the expected the total expected proportion of agreement (Pe)
Calculate the expected the total expected proportion of agreement(Po)
The (kappa) statistic
• Use the 2x2 table on the next slide for calculating the kappa statistic
(strength of agreement):
 The total expected proportion of agreement, Pe,
is given by :Pe = [ (a+c) * (a+b)] + [(b+d) *(c+d)]
N
N
N
N
 The total observed proportion of agreement, Po is
given by: Po= (a+d)
N
 The (kappa) statistic is given by: Po- Pe
1- Pe
where Po-Pe represents the proportion of the observed
agreement in excess of chance, and 1-Pe represent the
maximum proportion of agreement in excess of chance.
•
•
•
•
•
•
Value of kappa
<0.2 Poor
0.21-0.4
0.41-0.6
0.61-0.8
0.81-1.0
Strength of agreement
Poor
Fair
Moderate
Good
Very good
•
•
•
Read chapters ………………………….
Elect 2 students to act as Doctor X and Doctor Y
The rest of the students will be patients, half of which will go to Dr X, the other
half to Dr Y
• The doctors will interview the patients, and decide whether they are eating a
healthy diet or should be referred to a nutritionist
• The recommendation for each patient will be recorded on the data collection
sheet by each doctor.
• Then the doctors will swop patients and repeat the interviews.
• No contamination must take place, i.e. the doctors and patients should not share
with each other anything that was said by their counterpart in
the previous interview.
• The 2 doctors will make 10 copies of their data collection sheets and
distribute these to the 10 class groups.
• The groups will summarise the data using the data summary sheet.
• Each group will calculate the kappa statistic, and the results will be presented to
the class.