Transcript Pretoria Meeting MERLevel 1

OVC Monitoring, Evaluation and
Reporting Strategy (“the MER”)
Feb 20-21, 2014
Pretoria, South Africa
OVC HQ Technical Working Group
©USAID/ Tash McCarroll
OVC MER Strategy – Moving Towards
Outcomes
• New OVC MER strategy is a critical and positive shift toward a
comprehensive, outcomes based strategy for showing impact
• USG investments in OVC programs substantial, but impact mostly
undocumented.
• Challenges to understanding impact: lack of standardized
measures and tools tailored to OVC population & child well-being
• OVC TWG & Measure developed set of core quantitative child
outcomes and caregiver/ household outcomes tools
– Toolkit formed strong foundation to build a new set of indicators and
global strategy when OGAC commissioned a new MER strategy.
– Pilot-tested in Zambia and Nigeria; Translations to French in progress
2
Specific Shifts in Data Collected
• Emphasize program effectiveness & outcomes
• Reflects 2012 PEPFAR OVC Guidancedevelopmental milestones across ages/stages
• Track coverage: active beneficiaries for OVC
• Track contribution to Continuum of Response
• Elevate core indicators from OVC Survey
toolkit
OVC Indicators in the MER
MER
Code
NGI Code
OVC_SERV
New. Replacing
OVC_ACC
Indicator Description
Reporting
Level
C1.1.D <18
Number of active beneficiaries served by PEPFAR OVC
programs for children and families affected by HIV/AIDS
1
N/A –new
indicator
Number of active beneficiaries receiving support from
PEPFAR OVC programs to access HIV services
1
FY 14
FY 15
Targets
and
results
Targets
and
results
√
√
√
OVC Outcome Indicators Package (special studies)
OVC_HIVST
Percent of children whose primary caregiver knows the
child’s HIV status
OVC_NUT
Percent of children <5 years of age who are undernourished
OVC_SICK
Percent of children too sick to participate in daily activities
OVC_BCERT
OVC_SCHATT
N/A- new
indicators
Percent of children who have a birth certificate
Percent of children regularly attending school
OVC_PRGS
Percent of children who progressed in school during the last
year
OVC_STIM
Percent of children <5 years of age who recently engage in
stimulating activities with any household member over 15
years of age
OVC_CP
Child Protection Indicator
% eligible households able to access money to meet
important family needs
2
Budget and
Plan
√
No
Targets
required
REPLACED C1.1.D- OVC_SERV
Number of active beneficiaries served by PEPFAR OVC
programs for children and families affected by HIV/AIDS
Indicator REPLACES Umbrella CARE starting in FY14
Implications:
– Overall intent is comparable to old indicator: Measures coverage
– More specific: uniquely OVC-related vs. overlap with clinical care
– Defines “active beneficiary” as receiving quarterly contact-aligns
with criteria needed to meet direct support definition
– Clarification of who is counted is anticipated to improve quality
of data
OVC Level 1 Indicators-OVC_SERV
Indicator code:
OVC_SERV
Number of active beneficiaries served by PEPFAR OVC
1 programs for children and families affected by HIV/AIDS
NGI Mapping:
C1.1.D <18 disaggregation continuing – significant modification in definition; trend
analysis will be impacted
Both Direct Service Delivery (DSD) and Technical Assistance-only (TA-only) targets
and results should be reported to HQ
PEPFAR Support
Target/Result Type:
Numerator:
Denominator:
Disaggregation(s):
Data Source:
Data Collection
Frequency:
Number of active beneficiaries served by PEPFAR programs for children and
families affected by HIV/AIDS
1 N/A
1 Sex: Male, Female
1 Age: <1, 1-4, 5-9, 10-14, 15-17, 18+
Registers, organization records, or other program monitoring tools
Data should be collected continuously as part of service delivery. Data analysis and
review should be done semi-annually, in time for PEPFAR reporting cycles to monitor
progress towards achieving the targets and to identify and correct any data quality
issues.
PEPFAR MER Introduction 2013_11_21
6
1
OVC Level 1 Indicators
Indicator code:
OVC_SERV
Number of active beneficiaries served by PEPFAR OVC
1 programs for children and families affected by HIV/AIDS
Method of Measurement:
The indicator is generated by counting the number of active beneficiaries (children or caregivers) who received at least
one PEPFAR service from facilities and/or community -based organizations (see definition of an ‘active beneficiary’
below). To reduce the burden of counting on organizations providing services, registers with names of children who meet
the criteria for “active beneficiary” are sufficient for generating the number included in this indicator. The types of OVC
programs required to report on this indicator are outlined in the 2012 PEPFAR OVC Guidance and are those designed to
address critical care needs.
Explanation of Numerator:
Active beneficiary is defined as an individual who has received program services in the last three months and who is
scheduled to receive program services at least once every three months, as outlined in program guidelines or standards of
practice. New beneficiaries who only registered in the last quarter will be counted as active even if they have not yet
received services. Partners will report on the number of beneficiaries on their “active” registries. Partners will not be
required to count the number of individuals receiving services at each reporting period.
PEPFAR MER Introduction 2013_11_21
7
OVC Level 1 Indicator: OVC_ACC
Number of active beneficiaries receiving support from PEPFAR
OVC programs to access HIV services
• This is a direct output indicator that measures the number of
children and/or caregivers who are referred to and enabled to
access HIV-related services. The data obtained from this
indicator can inform cross-program planning including PMTCT,
pediatric and adult HIV testing, and treatment programs.
• Learning year-2014; Report starting in FY15
8
LEVEL 2 INDICATORS
PEPFAR MER Introduction 2013_11_21
9
Reminder: Distilling core indicators
• Our starting point: 6+1 domains of PEPFAR OVC
programming
• Step 1: Extensive literature search
• Step 2: Gaps (HES, PSS) filled through targeted research
 Result: >600 child/HH wellbeing
questions/indicators
• Step 3: Analysis against 8 criteria
 Result: shorter list of questions for discussion
10
Highlights of MER Outcomes Package:
Data Collection for Improved Quality
• Biennial (every two years)
• Greater Age Disaggregation – all indicators
• Utilize trained data collections external to
service delivery
– Relieve burden of front line service providers
– Build on capacity of local research institutions
11
OVC OUTCOMES PACKAGE – LEVEL 2
Outcomes Package- Quality*
• % <15 whose primary caregiver knows their HIV status
• % children <5 who are undernourished
• % children who are too sick to participate in daily activities
• % children who have a birth certificate
• % children regularly attending school
• % children who progressed in school during the last year
• % eligible children <5 years who engaged in stimulating activities with household
members over 15 years of age
• Child Protection indicator (in development)
• % eligible households able to access money to meet important family needs
* See Survey Toolkit: http://www.cpc.unc.edu/measure/our-work/ovc/ovc-program-evaluation-tool-kit
Added MER Level 2 indicators (not in
current version of Survey Toolkit)
• Percent of children whose primary caregiver
knows the child’s HIV status
• Child Protection Indicator (in development)
PEPFAR MER Introduction 2013_11_21
13
STAY TUNED FOR…
• OVC SPECIFIC MER SUPPLEMENTAL:
TECHNICAL GUIDANCE FOR PLANNING AND
IMPLEMENTING SPECIAL STUDIES
(Forthcoming, May 2014)
• * See Survey Toolkit:
http://www.cpc.unc.edu/measure/ourwork/ovc/ovc-program-evaluation-tool-kit
14
Indicator Reference Sheet (Level 2)
Indicator: NC.1
Percent of < 15 whose primary caregiver knows their HIV status
Indicator Type
Outcome
Numerator:
Number of active beneficiaries <15 years of age whose primary caregiver knows the child’s
HIV status
Denominator:
Number of active beneficiaries surveyed
Required:
Disaggregation(s):
Sex: male, female
Age group: <1, 1-4, 5-9, 10-14 years, 15-17
Data Source:
Special Study
Data Collection
Frequency:
Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.1
Indicator Type
Percent of children <5 years of age who are undernourished
Outcome
Numerator:
Number of active beneficiaries <5 years of age with physical growth below international
growth standards
Denominator:
Number of active beneficiaries <5 years of age surveyed
Required:
Disaggregation(s):
Sex: male, female
Age group: 6-11 months, 1-4 years of age
Data Source:
Special Study
Data Collection
Frequency:
Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.4
Indicator Type
Purpose:
Percent of children too sick to participate in daily activities
Outcome
Numerator:
Number of active beneficiaries who are too sick to participate in daily activities
Denominator:
Number of active beneficiaries surveyed
Required:
Disaggregation(s):
Sex: male, female
Age group: <5 years, 5-9 years, 10-14, 15-17 years
Data Source:
Special Study
Data Collection Frequency: Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.9
Percent of children who have a birth certificate
Indicator Type
Outcome
Numerator:
Number of active beneficiaries who have a birth certificate
Denominator:
Number of active beneficiaries surveyed
Required:
Disaggregation(s):
Sex: male, female
Age group: <1, 1-4, 5-9, 10-14, 15-17
Data Source:
Special Study
Data Collection Frequency:
Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.11
Indicator Type
Percent of children regularly attending school
Outcome
Numerator:
Number of active beneficiaries aged 5-17 years of age regularly attending school
Denominator:
Number of active beneficiaries aged 5-17 years surveyed
Required:
Sex: male, female
Disaggregation(s):
Age group: 5-9 years, 10-14 years, 15-17 years
Optional:
Further disaggregation by age, particularly within the 5-9 age group may be useful at the
national level as often children do not begin school until the age of 6 or 7.
Data Source:
Data Collection
Frequency:
Special Study
Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.12
Indicator type
Percent of children who progressed in school during the last year
Outcome
Numerator:
Number of active beneficiaries aged 5-17 years of age who progressed in school during the last year
Denominator:
Number of active beneficiaries surveyed who were enrolled in school during the previous year
Sex: male, female
Age group: 5-9 years, 10-14 years, 15-17 years
Optional:
Disaggregation(s):
Further disaggregation by age, particularly within the 5-9 age group may be useful at the national level as often
children do not begin school until the age of 6 or 7. It may also be useful to disaggregate by primary and secondary
school.
Data Source:
Special Study
Data Collection
Frequency:
Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.13
Indicator Type
Percent of children <5 years of age who recently engage in stimulating activities with
any household member over 15 years of age
Outcome
Numerator:
Number of active beneficiaries <5 years who have engaged in stimulating activities with
any household member over 15 years of age during the last 3 days, as reported by an
adult
Denominator:
Number of active beneficiaries <5 surveyed
Disaggregation(s):
By sex: Male, Female
By age group: 0-11 months, 12-23 months, 2-4 years
Data Source:
Special Study
Data Collection Frequency:
Biennially
Indicator Reference Sheet (Level 2)
Indicator: HW.2
Percent of households able to access money to meet important family needs
Indicator Type
Outcome
Numerator:
Number of active beneficiary households able to access money to meet important family
needs
Denominator:
Number of households surveyed that contain at least one active beneficiary
Disaggregation(s):
Optional:
Location: urban, rural
Data Source:
Special Study
Data Collection
Frequency:
Biennially
Indicator Reference Sheet (Level 2)
Indicator: CW.14
Child Protection Indicator (to be defined)
Indicator Type
Outcome
Numerator:
Denominator:
Disaggregation(s):
Data Source:
Special Study
Data Collection Frequency:
Biennially
LET’S KEEP DISCUSSING!
24