Transcript Developing Performance Measures

Developing
Performance
Measures
Portland Oregon TGA
Objectives:
Share Portland TGA performance
measures
 Describe the process of developing one
measure and refining it
 Share challenges and lessons learned

Portland TGA Performance Measures
Purpose/
Objective
Data Source
Decision
Support
Outcomes
Reports
Decision
Support
Clinical
Outcomes
Measurable Items


At least one service-specific outcome;
Client access to primary HIV medical care (% of clients with a HIV medical visit in last 6
mos).


Stable/improved CD4 counts;
Care provided according to clinical standards (% clients with PPDs; HAART; PCP/MAC.
prophylaxis; syphilis & Hep C screening; Pap test);
Maintenance of participation in medical care.

Client
Information
Systems
Monthly Service
Utilization
Reports
Client
Information
Systems
Quarterly
Service
Utilization
Reports



Annual Client
Services Data
Review
Delivery
System
Design
Delivery
System
Design
Client
Satisfaction
Survey
Contractor
Narrative
Reports
Contractor
Narrative
Reports
Workshop
evaluations
Patient Self
Management
Annually
Annually
Monthly service utilization reports include all with service category specific items:
o Unduplicated number of clients served YTD;
o Number of clients served monthly;
o Number of service units provided.
Monthly
Reports separated by service category, including:
o Unduplicated number of clients served;
o Number of service units provided;
o Total dollar amount spent.
Quarterly

The proportion of clients served and the proportion of services provided equals or exceeds
racial/ethnic minority representation in the epidemic.
The proportion of clients served and the proportion of services provided equals or exceeds
female representation in the epidemic.
Clients have demonstrated need (as defined by other client characteristics: %FPL, permanent
Annually
housing status, insurance status, etc.).
New clients are served in core service areas.
An increased proportion of clients are receiving medical care services in the EMA (Unmet
Need estimate).
Continuity of client care across care continuum.


Increase/decrease in mean scores across 10 client satisfaction items.
Themes for suggested service changes.


At least one quality service improvement (QSI) measure per contractor
Progress in implementation of CCM within each local contractor is reported.


Number of PSMP workshops held.
Increases in self efficacy

Client
Information
Systems
Timeframe



Annually
Measuring Client Access to Primary HIV
Medical Care
Original Measure (2001): Does client have an HIV
medical provider?


+
Easily reported by all
service providers
Valuable for providers
- coordination of care

Non-specific (Y/N)
 Does
not measure
actual activity


Client self report
Duplicated
Measuring Client Access to Primary HIV Medical
Care
Next Measure (2006): Did you see your medical provider
in the last 4 months, 5-6 months, > 6 months?




+
More activity specific,
client getting care
Flexibility in collection
methodology depending
upon provider
Began collecting at client
level within database
1st step in a process

Different levels of
confidence in the data





Check box annual survey
Collecting date of last visit
Client self report for some
providers
Medical chart abstraction
for others
Not all providers
submitting data at client
level
OUTCOME MEASURES
WITH OUTCOME MEASURES, IT IS IMPORTANT TO DOCUMENT THE NECESSARY DETAIL
BEHIND THE MEASURE IN ORDER TO CREATE ACCURATE, RELIABLE AND CONSISTENT
MEASURES. DOCUMENTING THE CONTENTS OF A PERFORMANCE MEASURE ALSO
PROMOTES RELIABILITY AND VALIDITY OF THE DATA BY PROVIDING A FRAMEWORK
FOR CONSISTENT MEASUREMENT OVER TIME AND BETWEEN DIFFERENT PROVIDERS.
PLEASE USE THIS TEMPLATE TO DOCUMENT THE DETAILS FOR EACH OF YOUR
OUTCOME MEASURES.
MEASURE DESCRIBED:
Specific outcome measure description
DATA REQUIREMENTS:
What is the specific information being collected (i.e. dates, scales,
etc.)?
Will this information be collected from the client or another source?
DATA SOURCE:
How are you collecting this information – i.e. on intake form, in
interview - and what is the specific question being asked? (If a survey
is being used, please submit this survey.)
When and how often is this information being collected?
CALCULATION METHOD:
How will this outcome measure be calculated from the data being
collected?
DATA LIMITATIONS:
Please let us know of any issues with data that may compromise
measure, such as missing data or use of estimates, etc.
Measuring Client Access to Primary HIV
Medical Care
Current Measure: Was your last medical visit 4
months ago , 5-6 months ago , > 6 months ago?

Shifted from range to specific visit dates

All core service providers submitting client level data and required to
use best or good data collection criteria

Best - data sources that pull directly from client medical record accepted
first, followed by provider pulling from medical record
 Good – client self report of last medical visit date collected regularly
from all clients

Support Services – may use best or good criteria, acceptable is
minimum requirement

Acceptable – client self report from a sample, reported in aggregate
Challenges
Maintaining reasonable expectation of
service providers
 Using medical outcomes is reasonable as
an outcome measure for the system. Is it
equally applicable across providers?
 Data sharing – inability to share data
increases data collection and reporting
burden HIV

Lessons Learned
Programmatic lessons
 Keep clients at the center – performance measures are
aimed at improving the quality of the care and evaluation
programs, not at proving a thesis
 Don’t let measurement guide the program - the results
should guide the program but not the measurability
 Work with providers to establish measures - provide
technical assistance and make compromises
Technical lessons
 Be realistic about time involved in collecting data for
certain measures
 Critical to have a client level data base
 Providers are experts at providing care – we need to
provide expertise in analysis
Contact Information
Marisa McLaughlin, Research Analyst
[email protected]
503-988-3030 x25705

Margy Robinson, HIV Care Services Mgr.
[email protected]
503-988-3030 x25681
