Melinda Karp - Consumer-Purchaser Disclosure Project
Download
Report
Transcript Melinda Karp - Consumer-Purchaser Disclosure Project
Measuring and Reporting
Patients’ Experiences with
Their Doctors
Process, Politics and Public
Reports in Massachusetts
Melinda Karp
MHQP Director of Programs
July 12, 2007
Today’s Objectives
Describe evolution of MHQP agenda for
measuring and reporting patient experiences-key methods questions in moving from research
to large scale implementation
Describe stakeholder perspectives and decision
points around key reporting issues
Discuss how MHQP data is being used and key
ongoing challenges
The Headlines from October, 1994…
…Led to the Creation of MHQP in 1995
• Provider Organizations
– MA Hospital Association
– MA Medical Society
– 2 MHQP Physician Council
representatives
• Government Agencies
– MA EOHHS
– CMS Region 1
• Employers
– Analogue Devices
• Health Plans
– Blue Cross Blue Shield of
Massachusetts
– Fallon Community Health Plan
– Harvard Pilgrim Health Care
– Health New England
– Neighborhood Health Plan
– Tufts Health Plan
• Consumers
– Exec. Director HCFA
– Exec. Director NE Serve
• Academic
– Harris Berman, MD, Board Chair
The Evolution of MHQP’s Patient
Experience Measurement Agenda
2002-2003
Demonstration project in partnership with The
Health Institute (Funded by Commonwealth
and RWJF)
Gained interest and acceptance of survey among
key stakeholders
Demonstrated relevance of physician level data
and feasibility of collaborative approach
Ambulatory Care Experiences Survey (ACES)
developed for the project has figured importantly in
development of C-G CAHPS
The Evolution of MHQP’s Patient
Experience Measurement Agenda
2004-2005
Development of viable business model for
implementing statewide patient experience survey
Approved cost sharing methodology for health plans and
physician organizations to finance survey.
2005-2006
Fielding and reporting of first statewide survey
Survey field period, July-September 2005
Focus on primary care practice sites
497 practices
92% of registered primary care physicians (over 4000 MDs)
50 item instrument covering 8 domains
Internal release of results to physician practices, November
2005
Public release of results, March 2006
Next Steps for the MHQP Patient
Experience Measurement Agenda
2007-2008
Fielding of Specialist Care Survey (2007)
Repeat Primary Care Survey (2007)
Assess the impact of reporting efforts for
physician and health plan stakeholders
Engagement around QI activities
– Participation in Commonwealth Fund grant to study highest
performing practices
– Physician Foundation Grant to develop and pilot integrated
clinical-patient experience QI curriculum
Pilot Medicaid Survey (2008)
“1st Generation” Questions: Moving
MD-Level Measurement into Practice
• What sample size is needed for highly reliable estimate of
patients’ experiences with a physician?
• Is there enough performance variability to justify measurement?
• How much of the measurement variance is accounted for by
physicians as opposed to other elements of the system (practice
site, network organization, plan)?
• What is the risk of misclassification under varying reporting
frameworks?
Allocation of Explainable Variance:
Doctor-Patient Interactions
100
80
62
60
Doctor
74
77
70
84
Site
Network
40
Plan
20
38
25
22
29
16
0
Source: Safran et al. JGIM 2006.
Allocation of Explainable Variance:
Organizational/Structural Features of Care
100
80
39
36
23
Doctor
60
Site
40
45
56
77
Network
Plan
20
16
0
Organizational
Access
8
Visit-based
Continuity
Integration
Source: Safran et al. JGIM 2006.
Summary and Implications
• Reliable information can be obtained with sufficient sample size
– Data obtained using C/G CAHPS approach yields data with MDand site-level reliability >0.70
– For site-level reliability, number of MDs per site influences required
sample sizes
• There is enough variability to justify physician-level reporting
• Risk of misclassification of performance can be minimized
– Can be held to <5% by:
• Limiting number of performance categories
• Creating buffer (“zone of uncertainty”) around performance
cutpoints
• Trade-offs are likely around data quality standards (e.g.,
acceptable “risk”) vs. data completeness
Setting the Stage for Public
Reporting: Key Issues for Physicians
• What measures get reported
– Criteria for reporting measures publicly
– Use of “super” composites to rate overall performance
– Level of detail presented to consumers—composites vs. item
level
• How measures get reported
–
–
–
–
–
Absolute vs. relative scoring
Use of “subjective” performance labels—Excellent, Good, etc.
Determining performance categories
Minimizing the risk of misclassification
Recognizing high achievers
• Setting the context for the user
Consumer Perspectives:
Focus Group Feedback On…
• Labeling measure composites
• Providing item level detail
• Describing how care is delivered in MA
and where the MHQP data fits
• Creating trust for the user
– Transparency about project funding, methods
– Endorsement from the physician/health care
community--AMA, MMS or State Health Department
Consumer Perspectives:
Focus Group Feedback On…
• Using quality information
–
–
–
–
Picking a new doctor
Evaluating a current doctor
Recommending a doctor for family and friends
Seeing how the doctors in their area were performing overall
• Useful information to support use of the
performance data
– Useful tools to help pick a doctor
– Information about the doctor and the practice
– Links to other websites
Consumer Perspectives:
Focus Group Feedback On…
• Talking about quality with doctors
– Skeptical
– Worried
“Wouldn’t have any effect or worse, might be negative”
• Benefits of the information
– Information = Empowerment
“Having options and the ability to make a choice when finding a
doctor”
“Taking charge of my health because now I can make decisions
based on information that I didn’t have before”
– Improving the quality of care
“I would hope they [doctors] would use this as feedback to improve
their practices”
Integrating Stakeholder Perspectives
“We must be willing to learn the lesson that
cooperation may imply compromise, but if it
brings a world advance it is a gain for each
individual nation”.
Eleanor Roosevelt (1884 – 1962)
“All government -- indeed, every human benefit and
enjoyment, every virtue and every prudent act -- is
founded on compromise and barter”.
Edmund Burke (1729 - 1797)
Key Decisions for Public Reports
• Search Approaches:
– search by Physician Name
– proximity search using region or zip code
– Search from list of practice sites, medical groups
• Front-end presentation of how care is
delivered and context for report
• Umbrella categories for measures but no
data roll-up
Key Decisions for Public Reporting
• 4 categories of relative performance
– ½ star “buffer zones” rounded to next performance category for
public reporting
• Criteria for including a measure
– 50% of practices needed “A” level reliability for measure to be
included
– A practice needed 3 eligible measures to be included
• No “subjective” labels attached to performance
• Drill down to item level results
• Measure specific messages about quality
Visit the
MHQP website
at
www.mhqp.org
Visit the MHQP
website at
www.mhqp.org
One More Stakeholder: The Media
What the headlines could have been…
The Headlines from March 9, 2006
The Headlines from March 9, 2006
Worcester Telegram & Gazette
How is MHQP Data Being Used?
• Current uses
–
–
Reporting to physicians for quality improvement, compensation
Direct to consumer online reporting
• Links from MA state website to MHQP reports
• Links from health plan websites to MHQP reports
• Links from several provider organization websites to MHQP reports
• Emerging uses
–
–
–
–
Support MA transparency agenda in State Health Care Reform
Law (Section 16)
Physician certification
Links to MHQP reports by employers
Health plan recognition programs, P4P, and product design
Ongoing Challenges
• Creating a sustainable financing model for
continued measurement and reporting
– Collaboration alone is not enough—aligning stakeholder
agendas and incentives is critical
• Continuing to meet the rapidly evolving
information needs of the marketplace while
maintaining the collaborative
– physician needs for quality improvement
– health plan/employer needs to develop innovative insurance
products/incentives
– Consumer needs to guide decision making
For more information about MHQP…
Melinda Karp, Director of Programs
[email protected]
617-972-9056
www.mhqp.org