Transcript Systematic Reviews and Meta-analysis

Critical Appraisal of:
Systematic Review: Bisphosphanates and
Osteonecrosis of the Jaw
Basil Al-Saigh
August 2006
What is Systematic Review?
Systematic Review :


Based on protocol

Involves critical appraisal

Synthesis of the data
Meta-analysis:


Statistical combination of data
Systematic vs. Narrative Reviews
Feature
Narrative
Systematic
Question
Broad; not stated
Focused
Search
Unknown
Comprehensive
Article selection Not stated
Criterion-based
Uniformly applied
Appraisal
Variable
Rigorous; critical
Inferences
May be evidence-based
Evidence-based
Advantages
Breadth; traditional;
integrative
Resolve discord
Why do a Systematic Review?

To examine the quality of the evidence

To summarize the treatment benefit in a given therapeutic area –
provide the best estimate of its direction and magnitude

To resolve discordance between trials

To investigate reasons for the discordance
Formulate the question

Who are the subjects?

Diagnostic criteria :


Population :


The end point was jaw disease in association with use of
bisphosphanates
Patients with cancers such as MM and metastatic CA taking IV
Bisphosphanates were the target population.
Setting :

Case series and reports analyzed retrospectively using internet
searches on MEDLINE and PubMed.
Formulate the question

What is the intervention?

Nature, Timing, Route and Placebo :

Since these are case reports and series, the above does
not apply when appraising this review

Otherwise, must be taken into account
Formulate the question

What are the outcomes?

Mortality :


No case reports or series reported mortality
Morbidity :

Defined as osteonecrosis of the Jaw, irrespective of location
(mandible vs maxilla) in patients taking bisphosphanates
Formulate the question


Conflict of interest?

No funding received for this review

One reviewer receives grants from Novartis
What type of studies?

Case reports and case series

Included if showed acceptable documentation of disease and use of
bisphosphanates irrespective of including information of sex of
patient, site of lesion and type of bisphosphanate used
Locate Studies

Systematic approach to avoid selection and publication bias :

Several authors published more than 1 paper describing the
same patients with osteonecrosis related to bisphosphanate
use

Such authors were contacted for confirmation and when this
occurred, only date from the larger more recent publication
were used
Locate Studies

Use of multiple sources :

Electronic: MEDLINE, PubMed


Could have also used Embase, Cinahl
Manual :

Other references obtained from citations from retrieved articles.

Otherwise, the same search words were used to search
abstracts from meetings of the American Society of Clinical
Oncology
Locate Studies

Document the search strategy :

Case studies and reports with search words that crossed :





Osteonecrosis – Avascular Necrosis – Phosphorous Necrosis –
Bisphosphanates – Diphosphanates
Jaw diseases – Myeloma – Breast Cancer – Metastatic Cancer
Search words do not include Osteoporosis.
Under-representation of the effect of oral BPn on this condition?
Uses of Reference Librarians :

No mention of having used that
Select Studies

Consider language restriction :

English and foreign language articles used

Increased the number of case reports and series

Since this is a review of case reports, sample size is important
for any meaningful initial conclusions
Select Studies

Follow explicit inclusion and exclusion criteria :

Inclusion criteria defined in the search words

No search word for Osteoporosis

Acceptable reports are those that documented disease AND
use of the BPn, regardless of whether the sex of the patient,
the site of lesion of the specific BPn used
Select Studies

Consensus of two reviewers examining the study abstracts :


3 reviewers sited
Rules to settle disagreements :

None sited
Assess Study Validity
Two reviewers independently review all studies and rate them re :
Allocation concealment – no one guesses randomization
Double blinding – neither patient nor investigator knows the
treatment received
Adequate follow-up – avoid attrition bias
Blind outcome assessment – avoid detection bias
Intention-to-treat analysis
Not applicable here, since these were reviews of case reports and
series
Analyze the Results

Consider the clinical homogeneity

Consider the statistical homogeneity

Cochrane’s q test for heterogeneity

If sufficiently homogeneous, conduct a Meta-analysis
Meta-analysis

Statistical combination of the results of multiple studies, weighed
by the inverse of the variance of the estimate in each study
(larger sample sizes receive more weight)
Sensitivity Analysis

How sensitive is the review to the specific methods used?

Change the inclusion criteria :


Add Search word of Osteoporosis?
Exclude foreign language/unpublished/poor quality studies :

No mention of who did the translation of foreign studies

Important in this review, as it would increase sample size
and give more meaningful conclusions
Interpretation and Recommendation

Scientific :

Limitations of the review & Identify areas of deficient
knowledge requiring further research :

Are alternate dosing strategies helpful in preventing this
condition yet providing the therapeutic effect of the drug?

What is the role of monitoring bone turnover in patients at
risk?
Interpretation and Recommendation

Need for a staging system to guide treatment?

Criteria for dx of early changes that precede and predict
bone exposure are desirable

Studies needed to elucidate additional RF for the
development of this condition

RCT’s to determine the most effective treatment protocols
for patients with this condition
Interpretation and Recommendation

Clinical :

Place results in the context of other evidence

Illustrate how the results might be used clinically

Incorporate into clinical recommendations and guidelines
Interpretation and Recommendation

Management recommendations :

Review of Expert panel recommendations of 2004, reviewers
own experience and that of colleagues



Group 1 – About to begin IV BPn therapy / Patients with
OP and taking oral medications
Group 2 – Patients receiving BPn therapy with no ON of
the Jaw
Group 3 – Patients with ON of the Jaw
Interpretation and Recommendation

Recommendations on the discontinuation of BPn therapy :

From the context of evidence of the long half life of BPn, d/c of
therapy might not be beneficial after development of ON of the
Jaw

From the context of anecdotal reports of resolution of ON of
the Jaw after several months of therapy cessation, d/c of the
medication may be beneficial a few weeks before and after
dental surgery as well as after development of ON of the Jaw
Interpretation and Recommendation

Many patients have OP

Review relevant because it highlights a dose dependant effect of
BPn use and development of ON the Jaw

Helps to reassure patients

Highlights RF ex. Dental procedures to guide management

Precipitates multiple questions for research avenues
Critical Appraisal of:
Systematic Review: Bisphosphanates and
Osteonecrosis of the Jaw
Basil Al-Saigh
August 2006