Republic of South Africa The Presidency Department of Performance Monitoring and Evaluation

Diagnostic assessment of M&E human resource capacity and training provision

20 September 2013 Presentation at the SAMEA Conference This research was done in collaboration with MZABALAZO Advisory Services

Diagnostic assessment of M&E human resource capacity and training provision

   

Research questions

 What is the current state of M&E capacity within M&E units of government departments?

 What training is available to enable government departments to acquire or improve their M&E competence

Scope of the research

 M&E units in national and provincial departments and in Office of the Premier

Sample

 43 departments (13 national,30 provincial departments and six Office of Premier, 98 questionnaires)

Methodology

 literature review, self reflective questionnaires, interviews and focus group discussions The Presidency: Department of Performance Monitoring and Evaluation 2

Key concepts that guided the research

 Six dimensions  M&E Leadership  Integrating planning, monitoring and evaluation  Monitoring as a management function  Monitoring and reporting on implementation of programmes  Evaluation and learning  Information and knowledge management of M&E evidence (Each dimension has descriptive statements of competency) The Presidency: Department of Performance Monitoring and Evaluation 3

Monitoring and reporting on implementation of programmes and/or projects Statements of competency  Ability to systematically collect data from a range of sources to monitor programmes and/or projects  Ability to use quantitative and qualitative approaches in monitoring programmes and/or projects  Ability to write clear, concise and focussed reports on performance of programmes and/or projects that is credible, useful and actionable  Ability to verify the quality of data to support reports  Ability to analyse and interpret data to track performance  Ability to clearly articulate, communicate, and disseminate findings to staff and management at different levels  Ability to use monitoring reports to take corrective actions The Presidency: Department of Performance Monitoring and Evaluation 4

Monitoring as a management function

Adequate skills and knowledge in monitoring for management

Managing service delivery improvement 62% 70% 78% Managing human resource management and development 58% 90% 86% Premier's Office Provincial National 89% Managing finances 49% 65% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% The Presidency: Department of Performance Monitoring and Evaluation 5

Information and knowledge management

Adequate skills and knowledge in information and knowledge management

Information dissemination and sharing 76% 100% 97% Organisational learning 52% 63% 79% Premier's Office Provincial National 81% Data systems 53% 69% 0% 20% 40% 60% 80% 100% 120% The Presidency: Department of Performance Monitoring and Evaluation 6

Evaluation and learning

Improvement Report writing and communications Managing evaluations Planning evaluations Using research practice in evaluations Evaluative discipline and practice

Adequate skills and knowledge in evaluation and learning

60% 56% 64% 74% 80% 81% 41% 44% 60% 19% 42% 52% 59% 68% 78% 56% 64% 78% Premier's Office Provincial National The Presidency: Department of Performance Monitoring and Evaluation 7

Key findings on the demand side

 There is a need to acknowledge the different types and purposes of M&E function in relation to the different frameworks in government  Provincial departments and OTPs weak in leading M&E  Effective implementation M&E is weakened by officials not having adequate knowledge and skills in  data systems  Evaluation and learning  Monitoring HRMD, finance and service delivery improvement (this collaborates with MPAT and AG findings) The Presidency: Department of Performance Monitoring and Evaluation 8

Supply of M&E

        Higher Education M&E training programmes are offered at NQF levels 5 to 9 – undergraduate and postgraduate programmes. PALAMA short courses are at level 5 and 6 (15 credit programme) The majority of training programmes are accredited Most of the programmes are offered via full or partial contact. The online delivery of training is limited. 70% of M&E training is accessed via short learning programmes Potential niche – M&E post graduate certificate (Hons) to professionalise practitioners from various disciplines DPME Evaluation courses and training to departments in the evaluation plan – “learning by doing” approach Courses are also offered by development agencies The Presidency: Department of Performance Monitoring and Evaluation 9

Key recommendations

1

.

M&E training needs   Providers of training need to respond to demand in following areas: Understanding policy frameworks – need to scale up advocacy of new M&E policy to create common understanding across government  Using log frame approach as a planning tool in projects and programmes – designing evaluation at the planning phase of programmes and projects  Setting up and managing data systems  Strengthening monitoring as a management function  Managing evaluations and learning  Training needs to target provincial departments, Office of Premiers and municipalities.

The Presidency: Department of Performance Monitoring and Evaluation 10

Recommendations

2.

It’s just not training  Need for an enabling institutional environment (we are working in an inter-connected complex system - need for collaboration and co ordination within, between and across spheres of governemt)  good governance and accountability (e.g. oversight bodies and citizens)  Aligning budgets with strategic outcomes (NT)  leadership in departments promoting M&E as a strategic issue  organisational design (DPSA)  integrated business process, and clear service delivery plan (DPSA/DGOG/ Sector Departments )  integrating M&E into the planning process (DPME, NT/Sector Depts/Depts)  appointment of right people to the job (DPSA/Departments)  defining jobs clearly and managing individual performance (DPSA/Departments)  Nurturing a performance culture in using evidence for planning, tracking performance, reporting, decision making and improvement The Presidency: Department of Performance Monitoring and Evaluation 11

Recommendations

3. M&E Competency Framework        There is general agreement and support for the Evaluation competency framework developed by DPME The emerging monitoring and reporting dimensions used in this research could be used to develop monitoring competencies Leadership in monitoring Integrating planning, monitoring and reporting Monitoring as a management function Monitoring and reporting on projects and programmes Information and knowledge management Credible evaluations is dependent on quality data, There is a need to keep an eye on the collection of routine administrative monitoring data during a project cycle in government; and data needs to be managed by an effective data system . The Presidency: Department of Performance Monitoring and Evaluation 12