Transcript Forming Guideline Recommendations

Interpreting Evidence and
Developing Recommendations:
Recommendation Considerations
Consensus Overview
Recommendations

QUANTITY, CONSISTENCY and QUALITY
How does the quality of evidence impact its
importance and value?
What is “enough” evidence.
Is their alignment between clinical significant and
statistical significance?
What evidence is most “important”?

HARMS vs. BENEFITS:
How do you weigh the good with the bad?
How do you communicate this with stakeholders?
Recommendations

ACCEPTABILITY & GENERALIZABILITY:
Evidence and its interpretation needs to meaningful and
relevant to the context.
How are preferences and values integrated into the
interpretation?

RESOURCES
Do you introduce costs into the debate?
Do you have the expertise and strategy with
which to do this?
Recommendations

One wants to express the evidence & its
interpretation by the panel into a concise
and meaningful statement of action.
Elements of a Recommendation

Intent or purpose of the recommendation
action
 Defining features of relevant patients
 Statement of recommended action
 Caveats or qualifying statements
Language: GRADE Revisited
GRADE
• Strong recommendation
•benefits clearly outweigh risks/hassle/cost
•risk/hassle/cost clearly outweighs benefit
• Weak recommendation
•weak evidence
•close balance between up and downsides
Language: GRADE Revisited
GRADE Recommendation Schema
• Do
• Probably do
• Probably don’t do
• Don’t do
Language: GRADE Revisited

variability in patient preference
◦ strong, almost all same choice (> 90%)
◦ weak, choice varies appreciably

interaction with patient
◦ strong, just inform patient
◦ weak, ensure choice reflects values

use of decision aid
◦ strong, don’t bother
◦ weak, use the aid

quality of care criterion
◦ strong, consider
◦ weak, don’t consider
Language: GRADE Revisited
potential challenges

can this approach capture the nuances and
complexities inherent in an evidentiary
base?

what are the implications for policy and
practice when comparing consistent
application across cancer diagnoses or
across different diseases?
Language: Alternatives
Use language
Information on characteristics of the evidence –
populations, treatments, etc.
Information on quality of the evidence - blinding,
allocation, drop out, etc.
Provide explicit links between the evidence and the
recommendations
Language: Alternatives
Verbs
is recommended/is recommended as the standard
• is an option/is a reasonable option
• could be considered/should be considered
• could be offered/should be offered
•
Potential Challenges
•
•
what do these words mean in practice?
are they perceived in the same manner?
Example
Epidermal Growth Factor Receptor (EGFR)
Targeted Therapy in Stage III and IV
Head and Neck Cancer: A Clinical Practice
Guideline
The addition of cetuximab to radiotherapy in patients
with locally advanced HNSCC
 increased overall survival (median 49.0 months
vs. 29.3 months; hazard ratio [HR] 0.74, 95% CI, 0.570.97, p=0.03) in favour of combined
 progression-free survival (median 17.1 months vs.
12.4 months; [HR] 0.70, 95% CI, 0.54-0.90; p=0.006)
in favour of combined
 locoregional control (median 24.4 months vs 14.9
months; [HR] 0.68, 95% CI, 0.52-0.89; p=0.005) in
favour of combined
 objective response rate (74% vs 64%; odds ratio
[OR] for response 0.57, 95% CI, 0.36-0.90; p=0.02) in
favour of combine
 Cetuximab did not increase common head and neck
cancer adverse effects which can occur during
radiotherapy. The most common and significant side
effects (grades 3-5) in response to cetuximab were
acneiform rash (17% vs 1%, p<0.001), and infusion reaction
(3% vs 0%, p=0.01). Overall quality of life was neither
improved of worsened by the addition of cetuximab to
radiotherapy.
 Chemoradiotherapy (platimum-based) is the current
standard of care for patients with locally advanced HNSCC
and to date there is no evidence comparing cetuximab plus
radiotherapy to chemoradiotherapy or whether the addition
of cetuximab to chemoradiotherapy is of benefit to these
patients.
WHAT DO YOU THINK THE RECOMMENDATIONS
OUGHT TO BE?
HOW WOULD YOU GRADE THESE
RECOMMENDATIONS?
DO YOU THINK GRADING THE RECOMMENDATIONS
IS USEFUL?
Recommendation
The addition of cetuximab to radical radiotherapy is
recommended in patients with locally advanced, nonmetastatic HNSCC who are medically unsuitable for
concurrent platinum-based chemotherapy or over the age of
70 to improve overall survival, progression free survival and
time to local recurrence.
Qualifying statement: Platinum-based chemoradiotherapy
remains the current standard of care for treatment of locally
advanced HNSCC.
Consensus
What it is:
 a transparent and explicit method of reaching
agreement among key stakeholders and/or
experts
 need for all projects because evidence is not
black and white – requires interpretation and
agreement
 becomes key when there is little evidence or
evidence is of poor quality
Formal Consensus
Why use a formal consensus method?
 Decisions need to be made - In the absence of
clear evidence, expert opinion may provide the best
(and only) guidance
 As part of an adaptation process
 Process needs to be rigorous, transparent,
credible and defensible
 Several available methods (Delphi, nominal group
technique (NGT), consensus development
conference, others)
Murphy et al. 1998. Consensus development methods, and their use in clinical guideline
development. Health Technology Assessment Programme. UK
19
Consensus Methods: when & how to use
~ good question – lack of evidence ~
1.
2.
3.
4.
5.
Start with a question.
WG generate postulates (recommendations).
Consensus group rates & provides feedback.
WG modify postulates in response.
Feedback to the consensus group
• original postulates, group feedback from
Round 1, modified postulates
6. Repeat step 3
7. WG final review, draft guideline
Protocol:
Modified
Delphi
Process
Considerations
•
•
•
•
•
•
members of consensus group?
anchors for rating?
how to define consensus?
how many cycles?
consensus vs. group think
packaging
Consensus
What about other, more “typical” circumstances
 How will you arrive at a final opinion?
◦ naturally
◦ vote

How will you report on majority and minority
opinions? Are minority opinions important?
Questions?
Feedback?