Transcript Melinda Karp - Consumer-Purchaser Disclosure Project

Measuring and Reporting
Patients’ Experiences with
Their Doctors
Process, Politics and Public
Reports in Massachusetts
Melinda Karp
MHQP Director of Programs
July 12, 2007
Today’s Objectives
 Describe evolution of MHQP agenda for
measuring and reporting patient experiences-key methods questions in moving from research
to large scale implementation
 Describe stakeholder perspectives and decision
points around key reporting issues
 Discuss how MHQP data is being used and key
ongoing challenges
The Headlines from October, 1994…
…Led to the Creation of MHQP in 1995
• Provider Organizations
– MA Hospital Association
– MA Medical Society
– 2 MHQP Physician Council
representatives
• Government Agencies
– MA EOHHS
– CMS Region 1
• Employers
– Analogue Devices
• Health Plans
– Blue Cross Blue Shield of
Massachusetts
– Fallon Community Health Plan
– Harvard Pilgrim Health Care
– Health New England
– Neighborhood Health Plan
– Tufts Health Plan
• Consumers
– Exec. Director HCFA
– Exec. Director NE Serve
• Academic
– Harris Berman, MD, Board Chair
The Evolution of MHQP’s Patient
Experience Measurement Agenda
2002-2003
Demonstration project in partnership with The
Health Institute (Funded by Commonwealth
and RWJF)
Gained interest and acceptance of survey among
key stakeholders
Demonstrated relevance of physician level data
and feasibility of collaborative approach
Ambulatory Care Experiences Survey (ACES)
developed for the project has figured importantly in
development of C-G CAHPS
The Evolution of MHQP’s Patient
Experience Measurement Agenda
2004-2005
 Development of viable business model for
implementing statewide patient experience survey
 Approved cost sharing methodology for health plans and
physician organizations to finance survey.
2005-2006
 Fielding and reporting of first statewide survey
 Survey field period, July-September 2005
 Focus on primary care practice sites
 497 practices
 92% of registered primary care physicians (over 4000 MDs)
 50 item instrument covering 8 domains
 Internal release of results to physician practices, November
2005
 Public release of results, March 2006
Next Steps for the MHQP Patient
Experience Measurement Agenda
2007-2008
 Fielding of Specialist Care Survey (2007)
 Repeat Primary Care Survey (2007)
 Assess the impact of reporting efforts for
physician and health plan stakeholders
 Engagement around QI activities
– Participation in Commonwealth Fund grant to study highest
performing practices
– Physician Foundation Grant to develop and pilot integrated
clinical-patient experience QI curriculum
 Pilot Medicaid Survey (2008)
“1st Generation” Questions: Moving
MD-Level Measurement into Practice
• What sample size is needed for highly reliable estimate of
patients’ experiences with a physician?
• Is there enough performance variability to justify measurement?
• How much of the measurement variance is accounted for by
physicians as opposed to other elements of the system (practice
site, network organization, plan)?
• What is the risk of misclassification under varying reporting
frameworks?
Allocation of Explainable Variance:
Doctor-Patient Interactions
100
80
62
60
Doctor
74
77
70
84
Site
Network
40
Plan
20
38
25
22
29
16
0
Source: Safran et al. JGIM 2006.
Allocation of Explainable Variance:
Organizational/Structural Features of Care
100
80
39
36
23
Doctor
60
Site
40
45
56
77
Network
Plan
20
16
0
Organizational
Access
8
Visit-based
Continuity
Integration
Source: Safran et al. JGIM 2006.
Summary and Implications
• Reliable information can be obtained with sufficient sample size
– Data obtained using C/G CAHPS approach yields data with MDand site-level reliability >0.70
– For site-level reliability, number of MDs per site influences required
sample sizes
• There is enough variability to justify physician-level reporting
• Risk of misclassification of performance can be minimized
– Can be held to <5% by:
• Limiting number of performance categories
• Creating buffer (“zone of uncertainty”) around performance
cutpoints
• Trade-offs are likely around data quality standards (e.g.,
acceptable “risk”) vs. data completeness
Setting the Stage for Public
Reporting: Key Issues for Physicians
• What measures get reported
– Criteria for reporting measures publicly
– Use of “super” composites to rate overall performance
– Level of detail presented to consumers—composites vs. item
level
• How measures get reported
–
–
–
–
–
Absolute vs. relative scoring
Use of “subjective” performance labels—Excellent, Good, etc.
Determining performance categories
Minimizing the risk of misclassification
Recognizing high achievers
• Setting the context for the user
Consumer Perspectives:
Focus Group Feedback On…
• Labeling measure composites
• Providing item level detail
• Describing how care is delivered in MA
and where the MHQP data fits
• Creating trust for the user
– Transparency about project funding, methods
– Endorsement from the physician/health care
community--AMA, MMS or State Health Department
Consumer Perspectives:
Focus Group Feedback On…
• Using quality information
–
–
–
–
Picking a new doctor
Evaluating a current doctor
Recommending a doctor for family and friends
Seeing how the doctors in their area were performing overall
• Useful information to support use of the
performance data
– Useful tools to help pick a doctor
– Information about the doctor and the practice
– Links to other websites
Consumer Perspectives:
Focus Group Feedback On…
• Talking about quality with doctors
– Skeptical
– Worried
“Wouldn’t have any effect or worse, might be negative”
• Benefits of the information
– Information = Empowerment
“Having options and the ability to make a choice when finding a
doctor”
“Taking charge of my health because now I can make decisions
based on information that I didn’t have before”
– Improving the quality of care
“I would hope they [doctors] would use this as feedback to improve
their practices”
Integrating Stakeholder Perspectives
“We must be willing to learn the lesson that
cooperation may imply compromise, but if it
brings a world advance it is a gain for each
individual nation”.
Eleanor Roosevelt (1884 – 1962)
“All government -- indeed, every human benefit and
enjoyment, every virtue and every prudent act -- is
founded on compromise and barter”.
Edmund Burke (1729 - 1797)
Key Decisions for Public Reports
• Search Approaches:
– search by Physician Name
– proximity search using region or zip code
– Search from list of practice sites, medical groups
• Front-end presentation of how care is
delivered and context for report
• Umbrella categories for measures but no
data roll-up
Key Decisions for Public Reporting
• 4 categories of relative performance
– ½ star “buffer zones” rounded to next performance category for
public reporting
• Criteria for including a measure
– 50% of practices needed “A” level reliability for measure to be
included
– A practice needed 3 eligible measures to be included
• No “subjective” labels attached to performance
• Drill down to item level results
• Measure specific messages about quality
Visit the
MHQP website
at
www.mhqp.org
Visit the MHQP
website at
www.mhqp.org
One More Stakeholder: The Media
What the headlines could have been…
The Headlines from March 9, 2006
The Headlines from March 9, 2006
Worcester Telegram & Gazette
How is MHQP Data Being Used?
• Current uses
–
–
Reporting to physicians for quality improvement, compensation
Direct to consumer online reporting
• Links from MA state website to MHQP reports
• Links from health plan websites to MHQP reports
• Links from several provider organization websites to MHQP reports
• Emerging uses
–
–
–
–
Support MA transparency agenda in State Health Care Reform
Law (Section 16)
Physician certification
Links to MHQP reports by employers
Health plan recognition programs, P4P, and product design
Ongoing Challenges
• Creating a sustainable financing model for
continued measurement and reporting
– Collaboration alone is not enough—aligning stakeholder
agendas and incentives is critical
• Continuing to meet the rapidly evolving
information needs of the marketplace while
maintaining the collaborative
– physician needs for quality improvement
– health plan/employer needs to develop innovative insurance
products/incentives
– Consumer needs to guide decision making
For more information about MHQP…
Melinda Karp, Director of Programs
[email protected]
617-972-9056
www.mhqp.org