Monitoring & Evaluation (M&E) - an integral component of NRHM PIPs

1

Parameters of Health Care Awareness Available Accessible - equity Affordable Quality

2

Monitoring & Evaluation

Requirements under the Logical Framework Approach of the National PIP

3

Agreed Indicators … (1)

13 Process Indicators – 6 monthly review 1.

% of ANM positions filled 2.

% of states and districts having full time program manager for RCH with financial and administrative powers delegated 3.

% of sampled state and district program managers aware of their responsibilities 4

1. % of sampled state and district program managers whose performance was reviewed during the past six months 5. % of districts not having at least one month stocks of (a)Measles vaccine, (b)Oral Contraceptive Pills and (c) Gloves

Agreed Indicators … (2)

13 Process Indicators – 6 monthly review (cont’d)

6.

7.

8.

9.

10.

11.

12.

13.

% of districts reporting quarterly financial performance in time % of district plans with specific activities to reach vulnerable communities % of sampled districts that were able to implement M&E triangulation involving communities % of sampled outreach sessions where guidelines for AD syringe use and safe disposal are followed % of sampled FRUs following agreed infection control and health care waste disposal procedures % of 24 hrs PHCs conducting minimum of 10 deliveries/month % of upgraded FRUs offering 24 hr. emergency obstetric care services % of sampled health facilities offering RTI/STI facilities as per agreed proto cols 6

Agreed Indicators

… (3) Output Indicators from Mid & End-line Surveys

The states are to set levels of achievement based on their own assessments.

1. Contraceptive prevalence rate 2. % eligible couples using any spacing method for > 6 months 3. % of women delivered during past one year who received 100 IFA tablets 4. % deliveries conducted by skilled providers (doctors, nurses or ANMs) 5. % of 24 hrs PHCs conducting minimum of 10 deliveries/month 7

Agreed Indicators … (4)

6. % of upgraded FRUs offering 24 hr. emergency obstetric care services 7. % of 12-23 months children fully immunized 8. % of mothers and newborn children visited within 2 weeks of delivery by a trained community level health provider/AWW or health staff (ANM/Nurse/Doctor) 9. % of children suffering from diarrhea during past 2 weeks received Oral Rehydration Solution 10. Polio free status achieved since

Details of numerator and denominator and sources of information given in Appendix IV of National PIP

MIES System Model in NRHM Program Planning & Service Delivery Validation Monitoring Evaluation Quality Assessment

9

M&E Activities: –(1) A - Surveys 3.1 Improve DLHS-3 (Mid-line Survey) to reflect Equity & Access 3.4 Equity Indicators for large national surveys like NFHS Actions & Current Status NFHS-III:

– Key results out, Detailed results in August, 2007

DLHS-III:

– Pilot Work to begin in July, 2007

Annual Health Survey (AHS)

– being conceived with the RGI – District Health Profile 10

3.2

3.5 3.6

M&E Activities: –(3) B. MIS Guidelines for local data to prepare & monitor plans CNAA Manual to be revised with field test Simplified MIES proforma Actions & Current Status

• • • •

MIES Format developed and sent to State Govts in August, 2006.

Key Indicators for Health Programmes integrated and sent to States in November, 2006 Data on revised formats have been received from 13 States

o Andaman & Nicobar Isles, Chhattisgarh, Delhi, Goa, Gujarat, Himachal Pradesh, Jharkhand, Madhya Pradesh, Maharashtra, Manipur, Meghalaya, Puducherry, Rajasthan

User Guidelines for MIES Format finalised.

o

Contains information on use of MIES data for local needs

11

M&E Activities: –(5) B. MIS 3.7 Analytical reporting for outputs Actions & Current Status.

• A web based information and communication system to be implemented in two phases.

– The first phase would essentially be a Bulletin Board for the NRHM/RCH. – The second phase is to take care of the data ware housing requirement of MIES.

• In-house software being developed for routine MIS reports

12

M&E Activities:–(6) B. MIS 3.9 Concurrent evaluation studies through common format Actions & Current Status Evaluation presently being done through RETs

– Max 168 (7x2x12) districts per annum – Needs reinforcement and realignment

PRCs undertake adhoc evaluation studies Several Institutions approached for Concurrent Evaluation of NRHM

• Workshop held in Feb, 2007 • A Committee formed to evaluate the proposals • Framework for Evaluation being worked out 13

M&E Activities:–(7) B. MIS 3.10 Quality Assurance Surveys Actions & Current Status Monitoring tools, check lists, operational manual for QA finalized Six States identified for piloting of QA and launched in December, 2006

- UP, Uttaranchal, West Bengal, Assam , Maharashtra, Karnataka (1 Dist each, 2 in UP) - Duration of QA piloting would be of 24 months 14

M&E Activities:–(8) B. MIS 3.11 Programme Management Reviews (Assessment of Management Capacity) Actions & Current Status

Study by IIM-A completed and Tools prepared and circulated to States Based on feedback, the Tools were refined and up-scaled country-wide in January, 2007.

Each State to canvas the Tool and report findings annually 15

M&E Activities:–(9) B. MIS 3.12 Community Monitoring & Triangulation Actions & Current Status Plan of action

1.

2.

Experts on Community Monitoring approached for a Concept Note on Triangulation Based on the feedback from the experts, a Triangulation framework would be prepared and piloted 16

Status of MIES System Model Program Planning & Service Delivery

Working Group on M&E

Validation Community Monitoring & Triangulation MIES Monitoring Format & guidelines Evaluation

•Surveys, RETs •Concurrent evaluation

Quality Assessment Pilot on Programme Mgmt Review

Launched 19

The ten Steps to building a performance – based M&E System

1.

2.

3.

Conducting a Readiness Assessment Identifying & agreeing on Performance Outcomes to Monitor and Evaluate Selecting Key Indicators to Monitor Outcomes 4.

5.

6.

7.

8.

Baseline Data on Indicators to Monitor Outcomes Planning for Improvement-Setting Realistic Targets A Monitoring and Evaluation System (HMIS, Web based ???) The Role of Evaluation Reporting the Findings 9.

Using Own/Other’s Findings 10.

Sustaining the M&E System 20

M&E Plan:

MDG goals achievement plan 1.

– – – Map information sources and flows Performance data – Monthly, Qtrly, Annual Surveys - NFHS, DLHS, NSSO, Other agencies Evaluation Studies – RET, PRC, NGOs, Communities 2.

• • • • • Map M&E requirements Household surveys IT infrastructure mapping Hardware, Software, Network, HMIS application, People – existing skills vs training needs

State, District, Blocks

Printing of Registers at SC, PHC, CHC etc Map institutions (public & private) – by service, strength etc Address vulnerable groups in IDHAP 3.

4.

• • Report on MIES format only Don’t change the reporting format Form 9 No longer required Plan for sustainability of M&E 21

What we expect from the State Governments …(1)

 Prepare a

State-Logical Framework Approach

(LFA) on the lines of the National PIP to monitor the M&E Activities    Reporting Monthly Performance data on

Revised MIES format

.

Assess management capacity - Programme Management Tool Undertake Household Surveys – to be updated annually  Revising and fresh printing of the primary registers from the Sub-centre level onwards – capture data for SC/ST   Training of manpower on data collection, flow and relevance Upgrading/modifying the hardware and software for States having an HMIS.  For States not having an HMIS efforts are to be made for establishing one The State Govts may revisit and revise their State PIPs to take the above aspects into consideration .

22

What we expect from the State Governments …(2) Challenges

     Getting the primary registers from the Sub-centre level onwards in position to capture data for vulnerable groups Institutionalising the data flow mechanism for timley and quality data Leveraging the advances in IT for expediting data flow Capturing information from private sector Have an inclusive Distt Action Plan to cover M&E requirements at the Distt level and aggregated at State level 23

Triangulation – an emerging concept

Monitoring/ Performance data

Concurrent Evaluation

Triangulation Survey Data/ Evaluation studies Community Monitoring

24