Transcript Summary of latest TB indicators

Operational research:
methods and examples
Udo Buchholz,
WHO/Stop TB/TME
1
What is operational research?
(OR)
• Definitions found on the internet:
– "Mathematical common sense"
– "Systematic study, by observation and
experiment, of the working of a system,
e.g. health services, with a view to
improvement"
– "Using scientific methods to attack a
complex problem or system"
2
In the beginning there was ...
a question
"Why in the world is it that 30% of
our patients on treatment default?"
NTP manager in the morning
3
Description of defaulters in
Russia1
• Profession: unemployed: 26%, labourers
21%, students of vocational schools 19%,
disabled 7%
• Education: incomplete secondary
education: 70%
• Residence: homeless 5%, >5km away from
treatment site 26%
• Behavioural risk factors: alcoholism 44%
1Data are from W. Jakubowiak, Russia
4
Are these variables risk factors for default?
– use of patient cohort for cohort study
9%
8%
7%
6%
p-value<0.001
5%
100%
90%
80%
70%
60%
p-value<0.001
50%
40%
30%
20%
10%
0%
4%
3%
2%
1%
0%
Alcoholics
No alcohol addiction
Ex-prisoner
No prison history
5
Social support system
• Examples from different oblasts:
– Food incentives
– Hygienic kits
– Free transportation
– Psychological support
– ....
6
Adherence with social support
100%
90%
80%
70%
60%
p-value=0.2
50%
40%
30%
20%
10%
0%
Ex-prisoner
No prison history
7
More examples
• "Defaulting from anti-tuberculous
treatment in a teaching hospital in Rio de
Janeiro, Brazil" (IJTLD 2004)
• "A concurrent comparison of home and
sanatorium treatment of PTB in South
India" (BWHO 1959)
• " 'Lost' smear positive PTB cases: where
are they and why did we lose them?"
(IJTLD 2005)
8
Determinants of a study
•
•
•
•
•
Problem or question
Data available
Funding and staff available
Political or hierarchical support
 Type of study
9
Which scientific methods can we use?
- Type of studies
• Descriptive studies
– Analysis of surveillance data
– Ecological study (correlational)
– Cross-sectional survey
• Analytical studies
– Observational (case-control study, cohort
study)
– Experimental
• Other
– E.g. capture-recapture study
10
Example: Surveillance data reveal large provincial
differences of ss+ TB/all PTB
ss+/all pulmonary
Smear-positive diagnosis by province, Syria
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
A Ar D
H
I
K
L
M OS Q
R RD S SW T
Z
11
No. of slides/patient is correlated with
proportion of ss+/PTB
Smear-positive diagnosis: Syria
100
ss+/all pulmonary
90
80
70
60
50
40
30
20
1.6
1.8
2.0
2.2
2.4
2.6
slides/patient
2.8
3.0
3.2
3.4
12
Ecological comparison
(correlational)
• Correlation of aggregated or group
data
• Association on the individual level is
unknown and may be different
• Many relationships on global level
are strictly speaking of ecological
nature
13
Example of an "ecological" comparison: The prevalence of HIV in
TB patients (y-axis) against the prevalence of HIV in adults (x-axis).
Measured prevalence of HIV in TB patients (%)
80
MAL
ZIM
ETH
BOT
60
KEN
SOA
LES
CAF
IVC
BUU
TAN
HAI
BFA
IVC
40
COD
KEN CAE
RWA
MOZ
BFA
NIE
GHA
DJI
20 DJI
CNG
CAM
0
0
10
20
30
Estimated prevalence of HIV in adults (%)
40
14
Cross-sectional survey
• Collection of representative data
• Based on sampling size calculations,
sampling frame and sampling
scheme
– Simple random sample
– Systematic sampling
– Cluster sample (design effect!)
15
Surveys are frequently used in
TB epidemiology
• Sampling universe is the population:
– Prevalence surveys
– Tuberculin skin test surveys
• Sampling universe is "all TB patients"
– Proportion of diagnosed new TB patients with
HIV test
• Sampling universe is the number of
culture positive TB patients
– Drug resistance surveys
16
Analytical studies
• Are used to identify risk factors or other
forms of "exposure" and their association
with an outcome, e.g. death, default, etc.
• Make use of a comparison group
• Hypotheses are tested
• Null hypothesis: "There is no association
of exposure and outcome" or:
"Exposure and outcome are independent"
•  We then calculate the probability that
this is true based on the data
17
Case control study
• Starts with a group of cases, i.e. with a
certain outcome, that is consistent with a
case definition
• The case definition must be specific in
regards to time, place and person
• E.g. "a person with smear positive TB
diagnosed in Geneva city in 2004"
• Then select a group of persons without the
outcome from the same population, here for
example the general population
• From the case definition it follows: "a
person without TB living in Geneva in 2004"
18
Case control study: ascertainment
of exposure status
• After identification of cases and
controls the exposure status
preceding the outcome is
investigated
• E.g.: income (high versus low)
• Thus, the directionality is usually
retrospective
19
Selection of controls
• Imagine the cohort from which the
cases would have arisen
• Or: Would the control have been a
case if he/she had had the outcome
in question?
• Example: cases of rare kidney
disease in the Mayo clinic
20
Typical control options
•
•
•
•
•
Friend controls
Neighbourhood controls
Physician controls
Hospital controls
Population-based controls
• Consider:
– Selection bias
– Feasibility
21
2 x 2 table (CCS (1))
Ss+ TB No ss+ TB
Low income
High income
50
950
5
1995
1000
2000
3000
• 50/1000 ss+ TB cases (5%) were poor, but
only 5 of 2000 (0.25%) among the non-TB
persons
•  Ss+ TB patients were 20 times more
likely than the general population to be
poor, however ...
22
2 x 2 table (CCS (2))
Ss+ TB No ss+ TB
Low income
High income
50
950
5
1995
1000
2000
3000
• The chances of ss+TB patients to be poor is
expressed as the odds = probability of poverty / prob
of rich
= 50/1000 / 950/1000 = 0.053
• The odds of non TB persons for poverty is therefore:
5/2000 / 1995/2000 = 0.00251
• The ratio of the two odds (the odds ratio (OR)) is:
0.053/0.00251 = 21
23
Use of case control studies
•
•
•
•
When type of outcome is rare
We can examine >1 exposure
Usually relatively quick and inexpensive
Disadvantages:
– Not useful for rare exposures
– Because exposure is in the past: watch out for
recall bias
– Selection of cases and controls often not
straightforward (selection bias)
24
Cohort study
• Starts with a group of people or a
population that can be divided in two
groups based on a defined exposure
which some have and some don't
• The groups are then followed-up and an
outcome is counted
• A case definition is still important
• The directionality is usually forward, but
can also be backwards (retrospective
cohort study)
25
2 x 2 table (cohort study)
Low income TB
patient
High income
TB patient
Adverse
outcome
Treatment
success
20
80
100
10
190
200
300
• We follow 100 low income TB patients and 200 high income
TB patients up for adverse outcomes
• It turns out that 20 of 100 (20%) poor have a bad outcome
versus 10 of 200 (5%) of the rich.
• Thus, the poor are 4 times more likely to have an adverse
treatment outcome.
• Measure of association is the risk ratio (RR) = 0.2/0.05 = 4
26
Use of cohort studies
• When exposure is rare
• We can examine >1 outcome
• The outcome measure for the strata is an incidence
rate or (cumulative) risk and the overall point
estimate the rate ratio or risk ratio (RR)
• Disadvantages:
– Not suitable for rare outcomes
– Not ideal for outcomes in the far future (unless you have
much time or lots of scientific altruism)
– Watch out for loss to follow-up (they may represent a certain
category of patients)
27
The TB quarterly "cohort"
• Pro- or retrospective cohort study
Pro-
Alhocol addiction
Default?/Cure?
No addiction
Default?/Cure?
Retro-
• (Nested) case-control study
Alhocol addiction
No addiction
Information may be available
from start of treatment
Cases Controls
Default Cured
28
2 x 2 table
ill
exposed
Not ill
exposed
(unprotected)
DISEASED
exposed
(unprotected)
HEALTHY
not exposed
not exposed
(protected)
DISEASED
not exposed
(protected)
HEALTHY
29
Cohort study
ill
healthy
exposed
not
exposed
time
30
Case control study
ill
healthy
exposed
not
exposed
time
31
Analytical study:
experimental / intervention study
• Prospective
• Use of a cohort
• Exposure is usually an intervention, a
drug or vaccine
• Patients are ideally randomized which
guarantees minimisation of bias
• Example: IPT intervention study in South
African gold miners; recruitment in
random sequence; comparison before /
after IPT phase
32
Steps for a OR protocol (1)
• Starts with a problem or question: e.g. "Why is
there no decline in urban TB in Japan?"
• Gathering of information:
– Analyse exhaustively routinely collected (surveillance)
data and disaggregate also by province etc
– Talk with stakeholders
– Investigation of the literature
– Contact other countries
• Develop a hypothesis
• Depending on money and staff available:
generate a protocol; but this can also be used to
generate money and staff
33
Steps for a OR protocol (2)
• Writing of the protocol:
–
–
–
–
–
–
–
–
–
–
You can structure it similar to a scientific paper
Introduction/rationale
Objective
Methods (study type, sample size, case definitions used,
inclusion/exclusion criteria, training, data collection,
data entry (double entry?, data validation), quality
control, lab methods, method of analysis)
Ethical considerations
Results: shell tables, expected figures
Timeline
Budget
Appendices (questionnaire, maps, consent form...)
Good idea to do a pilot: feasibility, cost, first crude data
verify sample size assumptions
34
Now it is up to you
35