Transcript Coverage with Evidence Development

Comparative Effectiveness Research:
Background, Priorities, Methods
Sean Tunis MD, MSc
July 10, 2009
1
http://www.cbo.gov/ftpdocs/89xx/doc8972/02-15-GeogHealth.pdf
Quality of Evidence for Guideline
Recommendations in CV disease
Robert Califf, IOM Meeting on Evidence-based Medicine, December 2007
Great Expectations
 Promoted from CBO director to OMB director
 “better information about the costs and
benefits of different treatment options,
combined with new incentive structures
reflecting the information….is essential to
putting the country on a sounder long-term
fiscal path.”
4
The nature of “C” evidence
 The paradox


“Available evidence is limited or poor quality”
18,000 RCTs are published every year
 The gaps, as seen by decision makers





Patients are highly selected
Research settings are not typical of community
Missing or incorrect comparators
Physiologic or surrogate outcomes, not function
Results are not available when decisions made
6
Committee’s Definition of CER
The generation and synthesis of evidence that
compares the benefits and harms of
alternative methods to prevent, diagnose,
treat, and monitor a clinical condition or to
improve the delivery of care. The purpose of
CER is to assist consumers, clinicians,
purchasers, and policy makers to make
informed decisions that will improve health
care at both the individual and population
levels.
7
Recommendation 6
The CER Program should fully involve
consumers, patients, and caregivers in key
aspects of CER, including strategic
planning, priority setting, research
proposal development, peer review, and
dissemination.
8
Engaging stakeholders
• Recommendation 1: Conduct a
systematic assessment of best practices
for effective engagement of decision
makers
• Recommendation 2: As a condition of
receiving federal funding for any CER
study, the investigators must form a
stakeholder advisory committee whose
function is based on findings of #1
Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact  June 9, 2009
9
AJRR 9/19 Meeting Participants
•Sean Tunis (CMTP)
•Mark McClellan (Brookings)
•Tony Rankin (AAOS)
•John Callaghan (AAOS/Iowa)
•Kevin Bozic (AAOS/UCSF)
•David Lewallen (AAOS/Mayo)
•Jeannie Kennedy (AAOS)
•Kathryn Sale (AAOS)
•Shami Feinglass (CMS)
•Nancy Foster (AHA)
•Mike Rapp (CMS)
•Teresa Lee (Advamed)
•Penny Mohr (CMS)
•Jeff Kang (Cigna)
•Tom Gross (FDA)
•Bob McDonough (Aetna)
•Susan Gardner (FDA)
•Joshua Benner (Brookings
•Danica Marinac-Dabic (FDA)
•David Brogan (Brookings
•Mark Melkerson (FDA)
•Michael Gluck (CMTP)
•Art Sedrakyan (AHRQ)
•Randee Kastner (CMTP)
•Jean Slutsky (AHRQ)
1
Categories of CER Methods
• Systematic reviews of existing research
• Decision modeling, with or without cost
information
• Retrospective analysis of existing clinical or
administrative data
• Prospective non-experimental studies,
including registries
• Experimental studies, including randomized
clinical trials (RCTs)
Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact  June 9, 2009
11
All methods have a role
• Inevitable tradeoff between internal validity
and feasibility, generalizability, cost, time
• The nature of the research question, and the
decision maker will influence best practices
• Experimental studies will have a crucial role
in CER, and there is need for improving
design and implementation
• Non-experimental methods hold great
promise, particularly as methods are refined
and data infrastructure is improved
Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact  June 9, 2009
12
Debate on CCTA Coverage
• Payers/researchers
»
»
»
want RCT with death/AMI outcome
20k patients, 2+ years of follow-up
Non-experimental options rejected
• Vendors / clinicians
»
existing evidence adequate for coverage
• Medicare final decision (March 2008)
»
»
»
No adequately designed studies show improved outcomes
“We believe large, well-designed prospective trials needed”
Broad coverage by local contractors retained
• NHLBI currently reviewing 3 large RCTs
• No progress on shared evidentiary framework for CER
Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact  June 9, 2009
13
Recommendation 7
The CER Program should devote sufficient
resources to research and innovation in
the methods of CER, including the
development of methodological guidance
for CER study design such as the
appropriate use of observational data and
more informative, practical, and efficient
clinical trials.
14
Baucus-Conrad PCOR Legislation
• “Establish and maintain methodological
standards for comparative clinical
effectiveness research on major categories of
interventions to prevent, diagnose, or treat a
clinical condition or improve the delivery of
care”
15
Effectiveness Guidance Documents
• Analogous to FDA-guidance
• Targeted to product developers, clinical
researchers
»
Recommendations for design of clinical studies to
generate evidence that will provide “reasonable
confidence” of improved health outcomes
• Multi-stakeholder advisory group, iterative draft
and comment process
• EGDs under development by CMTP
»
»
»
»
»
Gene expression profiling for breast cancer
Treatments for chronic wounds
Cardiac imaging
Integrative medicine
Radiation oncology
Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact  June 9, 2009
16
Pragmatic Pharmaceutical Trials
• Optimize design of phase III trials to be more
informative for post-FDA decision makers
• Clarify patient, clinician, payer evidence needs
• Identify critical regulatory, methodological
financial, operational barriers
• Develop PPCT guidance document
• Industry, FDA, CMS, NICE, PBAC, CDR,
Consumer’s Union, Medco, BSBCA, others
17
Contact Info
•
•
•
•
[email protected]
www.cmtpnet.org
443-759-3116 (D)
410-963-8876 (M)
Final Portfolio: 100 CER Priority Topics
TABLE 5-1 Recommended Research Priorities by Research Area
Category
Health Care Delivery Systems*
Racial and Ethnic Disparities
Cardiovascular and Peripheral Vascular
Geriatrics
Functional Limitations and Disabilities
Neurologic Disorders
Psychiatric Disorders
Pediatrics
Primary
Secondary
Research Area Research Area Total
23
27
50
3
26
29
8
13
21
2
19
21
2
20
22
6
11
17
7
10
17
1
15
16
* Although this category was described as “Safety and Quality of
Health Care” in the web-based questionnaire, the category was relabeled by the committee as “Health Care Delivery System” to be
more accurate.
19
TABLE 5-3 Committee’s Recommended Research
Priorities by Types of Intervention
Types of Interventions
Systems of Care
Pharmacological Treatment
Standard of Care
Behavioral Treatment
Prevention
Procedures
Provider Patient Relationships
Treatment Pathways
Testing, Monitoring, and Evaluation
Devices
Alternative Treatment
Other
Total
Number of Topics
43
36
33
29
24
23
20
19
17
13
9
18
284
NOTE: The total exceeds the total number of priority topics as respondents were allowed to select
multiple interventions to be compared for each topic.
20
Recommendation 5
The HHS Secretary should establish a
mechanism—such as a coordinating
advisory body— with the mandate to
strategize, organize, monitor, evaluate and
report on the implementation and impact of
the CER Program.
21
Recommendation 6
The CER Program should fully involve
consumers, patients, and caregivers in key
aspects of CER, including strategic
planning, priority setting, research
proposal development, peer review, and
dissemination.
22
Recommendation 6 cont’d
– The CER Program should develop strategies
to reach out to, engage, support, educate, and,
as necessary prepare consumers, patients,
and caregivers for leadership roles in these
activities.
– The CER Program should also encourage
participation in CER in order to create a
representative evidence base that could help
identify health disparities and inform decisions
by patients in special population groups.
23
Recommendation 8
The CER Program should help to develop
large-scale, clinical and administrative
data networks to facilitate better use of
data and more efficient ways to collect
new data to inform CER.
– The CER Program should ensure that CER
researchers and institutions consistently
adhere to best practices to protect privacy and
maintain security.
24
Recommendation 8 cont’d
– The CER Program should support the
development of methodologies for linking
patient-level data from multiple sources.
– The CER Program should encourage data
holders to participate in CER and provide
incentives for cooperation and maintaining
data quality.
25
AHRQ Systematic Review:
Tx of Clinically Localized Prostate Cancer
 Limited evidence on relative safety and effectiveness
of major treatment options

prostatectomy, brachytherapy, radiation, active surveillance
 New technologies rapidly spreading without data

robotic surgery, proton beam
 Rigorous trials needed to compare treatment options,
especially for side effects
CMS Efforts to Improve
Evidence
 Category B IDE regulation (1996)
 Cover routine costs of clinical trials (2000)
 Coverage with evidence development (2003)
 Promote pragmatic clinical trials (2003)
 Priorities for Sec 1013 of MMA (2004)
 MCAC becomes MedCAC (2005)
 Ad hoc efforts to work with NIH
27
Tools and Strategies for CER
 Methodological framework
 Pragmatic clinical trials
 Coverage with evidence development
28
CMTP Mission
 Collaborative projects to promote generation
of new evidence that it is more informative to
patients, consumer clinicians, and payers.

Fewer C’s, more B’s and A’s
 We don’t do CER studies; we develop
methods, policies, and collaborations to make
CER happen
29
CMTP Basics
 Started Jan 2006 within HealthTech in SF

CHCF and Blue Shield California Foundation
 Incorporated as 501c3 in Jan 2008 (Maryland)

5 member governing board; 14 on advisory
board
 Funding


Founding members: Blue Shield California,
Kaiser, United, Aetna NPC, Pfizer, Amgen, JNJ
Additional funds from foundations, government,
professional societies
 Staffing: 12 FTEs
Project Categories
 Priorities for evidence development
 Facilitated research


Total joint replacement registry
Proton beam vs IMRT in prostate cancer
 Methods development / guidance


Gene expression, chronic wounds, cardiac
imaging
“Pragmatic” Phase III pharmaceutical trials
 Applied policy

Commercial payer CED
MedPAC project on Medicare CER and CED
31

Coverage with Evidence
Development
 Links payment to requirement for
prospective data collection
 Intent is to guide clinical research to
address questions of interest to Medicare

Medicare must approve study design
 Goal to support evidence and rapid access

Lower evidence threshold with commitment
to generate better info later
32
Examples of Medicare CED
 Lung volume reduction surgery
 FDG-PET for suspected dementia
 Implantable defibrillators
 Off-label use of drugs for colorectal cancer
 FDG-PET for oncology
 Home testing for sleep apnea
 Daily dialysis
 Artificial heart
33
CED Lesson Learned
 Timing: when coverage under review, may be too
late for CED
 Methods



Difficult to design studies in coverage context
Registries provide broader access; ?? validity
RCTs viewed as equivalent of non-coverage

Large simple trials may help, but few examples
 Payers view as benefit expansion; Vendors opposite
 Unclear how best to fund clinical and research costs
Banff Workshop on CED
 Presentations from Scotland, UK, Australia,
US, Ontario, British Columbia, Alberta
 Everyone trying to solve the same policy
problem – balancing access and evidence
 Encountering similar pressures and
compromises, repeating the same mistakes
 Some hints about viable approaches
 Will begin work on CED Field Guide for
Payers and Policy Makers
35
Medicare Review of CCTA
 EPC report from Duke (April 2006)

Limited evidence of clinical utility in any
population
 MedCAC mtg (May 2006)

“Uncertain confidence about existing
evidence”
 Broad local coverage of CCTA
 Medicare draft policy in 12/07 proposed CED
for CCTA in “adequate” studies
Debate on CCTA Coverage
 Payers/researchers



want RCT with death/AMI outcome
20k patients, 2+ years of follow-up
Non-experimental options rejected
 Vendors / clinicians

existing evidence adequate for coverage
 Medicare final decision (March 2008)



No adequately designed studies show improved outcomes
“We believe large, well-designed prospective trials needed”
Broad coverage by local contractors retained
 NHLBI currently reviewing 3 large RCTs
Cardiac Imaging Think Tank
 Discussed alternative methods for evaluating
cardiac imaging



Co-sponsors: ICER, ACC, ACR, SCCT, ASNC
private payers, CMS, vendors, clinical
researchers, consumers, AHRQ, VA, etc.
Goal: balance validity with feasibility
 Current status
 Meeting summary completed
 Effectiveness Guidance Document on clinical
utility of non-invasive cardiac imaging
Effectiveness Guidance Documents
 Analogous to FDA-guidance
 Targeted to product developers, clinical researchers


Recommendations for design of clinical studies to generate
evidence that is adequate for decision making
“reasonable confidence” of improved health outcomes
 Started from insights from systematic reviews
 Multi-stakeholder advisory group, iterative draft and
comment process
 Ongoing work




Gene expression profiling for breast cancer
Treatments for chronic wounds
Cardiac imaging
Radiation oncology
Tools and Strategies for CER
 Coverage with evidence development
 Methodological guidance
 Pragmatic clinical trials
40
The blank pragmatic–explanatory continuum indicator summary (PRECIS) “wheel.”
PRECIS Domains Illustrating the Extremes of Explanatory and
Pragmatic Approaches to Each Domain
Domain
Pragmatic Trial
Explanatory Trial
Participants
Participant Eligibility Criteria All participants who have the
condition of interest are
enrolled, regardless of their
anticipated risk,
responsiveness, comorbidities, or past
compliance.
Step-wise selection criteria are
applied that: (a) restrict study
individuals to just those
previously shown to be at
highest risk of unfavorable
outcomes, (b) further restrict
these high risk individuals to
just those who are thought
likely to be highly responsive
to the experimental
intervention, and (c) include
just those high risk, highly
responsive study individuals
who demonstrate high
compliance with pre-trial
appointment-keeping and a
mock intervention.
PRECIS Domains Illustrating the Extremes of Explanatory and
Pragmatic Approaches to Each Domain
Domain
Pragmatic Trial
Explanatory Trial
The primary outcome is an
objectively measured,
clinically meaningful outcome
to the study participants. The
outcome does not rely on
central adjudication and is one
that can be assessed under
usual conditions: for example,
special tests or training are not
required.
The outcome is known to be a
direct and immediate
consequence of the
intervention. The outcome is
often clinically meaningful,
but may sometimes (early
dose-finding trials for
example) be a surrogate
marker of another downstream
outcome of interest. It may
also require specialized
training or testing not
normally used to determine
outcome status or central
adjudication.
Follow-up and Outcomes
Primary Trial Outcome
Pragmatic Clinical Trials
Initiative
 Optimize design of phase III trials to be more
informative for post-FDA decision makers
 Clarify patient, clinician, payer evidence needs
 Identify critical regulatory, methodological
financial, operational barriers
 Develop PPCT guidance document
 Industry, FDA, CMS, NICE, PBAC, CDR,
Consumer’s Union, Medco, BSBCA, others
46
47
Contact Info
 [email protected]
 www.cmtpnet.org
 443-759-3116 (D)
 410-963-8876 (M)