Transcript Demystifying Evaluation in WIPO – Best Practices from Initial Evaluations November 2012

Demystifying Evaluation in WIPO –
Best Practices from Initial Evaluations
November 2012
PERFORMANCE MANAGEMENT (RBM) AND EVALUATIONS
A&M = facilitator for organizational performance management, including
performance assessment
Evaluations are
dependent on the
results/measurements
frameworks defined at
the planning stage
Evaluations are
complementary to the
assessments of
organizational performance
Both performance assessment and evaluations identify
opportunities for improvement (including lessons learned) which
feeds into the subsequent planning cycle
2
WIPO’s Results Based Management (RBM) Framework
WIPO’s Planning
Framework
Results Chain
Medium-Term
Strategic Plan
(MTSP) *
Involvement of
Member States
Reporting
External /
Internal
External
factors
Impact
2010-2015
Reporting on
the MTSP to
Member States
Strategic Outcomes
&
Strategic Outcome
Indicators
Program and
Budget
Program and
Budget
Program and
Budget
2010/11
2012/13
2014/15
Expected
Results &
Performance
Indicators
Expected
Results &
Performance
Indicators
Expected
Results &
Performance
Indicators
Results
Annual
Work
plan
Annual
Work
plan
Annual
Work
plan
Annual
Work
plan
Annual
Work
plan
Annual
Work
plan
2010
2011
2012
2013
2014
2015
Activities
Activities
Activities
Activities
Activities
Activities
Individual
Staff
Objectives
Individual
Staff
Objectives
Individual
Staff
Objectives
Individual
Staff
Objectives
Individual
Staff
Objectives
Individual
Staff
Objectives
2010
2011
2012
2013
2014
2015
* MTSP and ) and the comments from Member
States as reflected in the report of the
Assemblies 2010 and its annex
Attribution
Involvement of
and Approval by
Member States
Internal
Management
Reporting
Outputs
Staff
Performance
Program
Performance
Reports
Internal
factors
PMSDS
3
THE A&M EXPERIENCE
 Limited experience with independent evaluations in the A&M Sector
(as opposed to audits)
 Confined to the Validations of the Program Performance Reports
(PPR)
 The Validations of the PPRs are not strictly speaking evaluations but
follows a process and scope which can be called evaluative in nature
4
USEFULNESS OF PPR VALIDATIONS
 The independent Validations of the PPR are very appreciated by
Member States
 It serves as a useful tool in the Secretariat’s performance dialogue
with the Member States
 The independent Validation exercise is a “best practice” within the
UN system (WIPO the only organization where Program Managers’
performance assessment is independently verified)
 Useful feedback for the further strengthening of the implementation
of RBM
5
MAIN SUCCESS CRITERIA
 Ownership - of findings, conclusions and recommendations is essential
for learning and improvement
 requires a “participatory” approach during planning (ToR),
conduct of the evaluation (consultation with main stakeholders)
presenting and disseminating evaluation results
 Knowledge - of the subject matter to be evaluated is essential in order
to ensure validity of findings
 Constructive approach -
glass is “half full” versus “half empty”
 Timing – Program Managers’ performance assessment & validation
exercise are conducted in parallel leading to validation of “interim”
performance data (A&M is pre-validating, providing quality assurance and
checks of historical consistency)
INCREASES THE UTILITY OF EVALUATIONS
(ACCOUNTABILITY AND LEARNING)
6
EXPECTATIONS FOR FUTURE EVALUATION WORK
 Continue enhancing the utility of the PPR validation exercises
 Strengthening the planning of evaluations to ensure their
complementarity to organizational performance assessments
 evaluations to provide more in-depth analysis of
“what works well” and “what does not work well” and “why”
 Ensuring that lessons learned get fed into the next planning cycle
(closing the feedback loop)
7