Transcript Data Quality and Data Use - PATH: Driving transformative

Indicators
for ACSM
Objectives
•
Explain the role of indicators in monitoring and
evaluation for ACSM.
•
Describe the characteristics of well-defined indicators.
•
Demonstrate how to develop indicator descriptions.
Creating a Framework
RESULTS
•
•
•
•
•
•
•
•
•
•
• # of times radio
spots aired
•
• # of subway ads
developed
•
•
•
•
What Is an Indicator?
•
•
Clues, signs, or markers.
Used to:
1. Track inputs, activities, outputs, and outcomes.
2. Measure progress toward the goal and objectives.
Crow and the Pitcher!
Indicators are Part of the M&E Plan
Data
Quality
Data
Collection
Evaluation
Strategy
Indicators
Framework
Data Use
and
Reporting
M&E
Plan
Budget
Examples
Framework Component
Indicator
INPUT: IPCC training (curriculum)
100 copies of training manuals, slide
sets, and handouts on IPCC
ACTIVITY: Lobbying
Meeting with the finance minister and
NTP director
OUTPUT: Patient coalition meeting
Number of patients attending the
meeting
OUTCOME: Increased funding
Percentage of NTP budget covered by
the Ministry of Health
OUTCOME: Increased support for
community-based DOTS
* Policy change to allow volunteer
health workers to serve as treatment
supporters
OUTCOME: Increased political support
for TB control
* Parliament declares TB a national
emergency
Indicators: Important M&E Evidence
Goal
What am I trying to achieve?
Objective What are the major steps I need
to take to reach my goal?
Activities What am I going to do to reach
my objective?
Inputs
What resources do I need to
complete each activity?
Outputs
What will the immediate product
of my activities be?
Outcomes What do I hope will happen as a
result of my activities?
Impact
What do I think these activities
will contribute to my goal?
Indicators MONITOR progress:
Have I done what I said I would
do?
Indicators EVALUATE results:
Did my work have the desired
effect and contribute to my
goal?
Outputs versus Output Indicators
•
Output is the immediate result of an activity:
 # of people trained
 # of people with TB symptoms going to health facility for evaluation
 REPORT WHAT YOU ACTUALLY ACCOMPLISHED
•
Indicators for outputs measure your result against a target
value:
 # of people actually trained versus how many you wanted to train
 # of people with TB symptoms going for evaluation versus how

many you thought would go
ASSESS WHETHER YOU REACHED YOUR TARGET VALUE
NTP Goal: Reduce morbidity and mortality due to
TB in Country X.
Indicator:
Number of TB deaths
per 100,000 per year
NTP Objective: Increase case detection rate from 42% to
60% by 2015.
Indicator:
Case detection rate
ASCM Objective: Raise knowledge of TB symptoms and TB
services to increase the number of people in City X seeking care
for TB symptoms at DOTS centers by 30% by December 2013 .
Indicator:
# of people requesting
screening at the City X DOTS
center compared to baseline
Inputs
Activity
Outputs
Outcome
Impact
Funding
Develop
and
produce
subway ads
# ads
produced
Increased
knowledge
of TB and
DOTS centers
Increased #
of TB cases
detected
List of subway
routes
Indicator:
Lists
developed
(Yes/No)
Indicator:
Ads produced
according to
schedule (Yes/No)
# subway
trains with
ads
Indicator:
Number of
ads and
subway trains
versus target
Indicators:
• % of people who know
two TB symptoms and
have heard of the DOTS
center
• # of people requesting
screening at City X
DOTS center
Indicator:
Difference in case
detection rate
between
intervention and
control cities
Main Steps of Creating Indicators
1. Select an indicator
2. Test against criteria
3. Write a description
Characteristics of Good Indicators
Valid
Reliable
Does it tell us what we really want to know or could it measure
something else?
Is the description clear? Will everyone interpret and calculate this
the same way?
Activity
Specific
Does this tell us only about our activity? Could any other factor
influence this indicator?
Feasible
Can we realistically collect and analyze this? Do we have a
workable data source and enough money and people to collect it?
Do the results mean the same thing in different geographic areas
Comparable
at different times?
Is This Indicator Valid?
•
•
Does it tell us what we really want to know?
Could it actually measure something else?
Example:
% of Ministry of Health budget dedicated to TB as an
indicator of government commitment to TB control.
Improved indicator:
% of NTP budget covered by government (compared to %
covered by donors).
Is This Indicator Reliable?
•
Will everyone interpret or calculate
same way?
it the
Example: Number of partners actively participating in
advocacy coalition.
Improved indicator: Number of partners who attend at
least 75% of coalition meetings.
Example: Smear conversion rate.
Is This Indicator Activity Specific?
•
Does it tell us about our activity only? Could any
other factor (ACSM or not) influence this
indicator?
Example: % of TB screening clients receiving a smear
test as an indicator that providers from our training are
referring TB suspects properly.
Improved indicator: % of TB screening clients with
documented referral for smear microscopy.
Is This Indicator Feasible?
•
Do we have a realistic data source? Do we
have enough money and staff?
Example: % of population with correct knowledge
about TB symptoms, annually (indicator of a successful
communication activity).
Alternative indicator: Number of people with possible
TB symptoms presenting for diagnosis at specified
DOTS clinics.
Comparable
•
Do the results mean the same thing in different
geographic areas at different times?
Example: Number of nongovernmental organizations in
each region mobilized to participate in World TB Day.
Alternative indicator: % of nongovernmental
organizations in each region mobilized to participate in
World TB Day.
Indicator Descriptions
•
•
•
What is the complete definition?
What is the data source for this indicator?
How do we calculate the value of the indicator?
 Numerator, denominator
 Qualitative criteria
Indicator Description
Indicator: Percentage of district DOTS nurses receiving IPCC training
Definition
Percentage of DOTS nurses
who attend the full training
and receive a completion
certificate.
Data Source
IPCC training
attendance sheet
and certificate list
submitted to NTP.
Calculation
Number of nurses completing
the training.
Total number of DOTS nurses
in the district.
Questions?