Transcript Document
Independent Evaluation in IFC Presentation to Staff of Islamic Development Bank May 3, 2009 Marvin Taylor-Dormond 1 Constructing an Evaluation Framework • Why do we evaluate? 2 Why do we evaluate? • Accountability • Learning • Allocation of resources / Better results 3 IEG: independent reporting Board Director-General, Evaluation (DGE) President World Bank Group Multilateral Inv. Guarantee Agency World Bank IEG-WB IEG-IFC 4 IEG-MIGA IEG-IFC structure Quality, Knowledge, Partnerships, Communication Macro/Aggregate Evaluation Micro Evaluation 5 IEG-IFC structure Micro Evaluation 6 Scope of micro evaluation • Independent project-level evaluation of IFC’s investment and advisory operations • Accountability for IFC’s operations at the project level • Provides the foundation for IEG’s macro evaluations 7 Evaluation of investment projects (XPSR system) Self-evaluation – IFC Staff • IFC staff prepare Expanded Project Supervision Reports (XPSRs) Independent Evaluation - IEG-IFC • • • • • • Sample selection Guidance Independently validated XPSR ratings Collect and disseminate lessons Desk-based or field-based validations Mini-XPSRs / Project Evaluation Summaries 8 Evaluation of advisory services (PCR evaluation) Self-evaluation – IFC Staff • IFC staff prepare Project Completion Reports (PCRs) Independent Evaluation - IEG-IFC • Independent evaluation / validation of PCRs • Desk review of adherence to guidelines in PCR ratings • Desk and field reviews for validation of results 9 Measuring development Investment Operations Advisory Services Project Business Success Economic Sustainability Environmental & Social Strategic Relevance Output Achievement Development Development Outcome Effectiveness Outcome Achievement Impact Achievement Private Sector Devpmnt Efficiency 10 Investment project evaluation: Ratings XPSR example: Real Sector Unsuccessful Mostly Mostly Highly unsuccessful successful unsuccessful Business success Economic sustainability Environmental & social effects Private sector development Highly successful 1. Project’s Development Outcome Project’s impacts on: Successful Unsatisfactory Partly Unsatisfactory Satisfactory Excellent 11 IEG-IFC structure Macro/Aggregate Evaluation 12 Scope of macro evaluation • Focus on and inform IFC’s strategies, policies & procedures • Provide accountability • Country, sector, & thematic evaluations • Forward-looking 13 Studies •Country and regional •Thematic •Sectoral •Corporate •Process 14 Five Outcome Drivers Five drivers explain twothirds of project outcomes 15 Frontier strategy targets high-risk countries Country risk at approval: Low High 70% Success rates were HIGHER for projects in countries that were HIGH risk at approval 60% Success rate 50% 40% 30% 20% 10% 0% Development outcome IFC investment outcome Development outcome and investment Development Success rate, 2005-2007 (%) outcome 16 Improvement in business climates key to success Country risk changes from Approval Evaluation Got better Worsened 90% 90% 80% 80% 70% 70% Success rate Success rate Worsened 60% 50% 40% Got better 60% 50% 40% 30% 30% 20% 20% 10% 10% 0% 0% Investment Outcome Development outcome Development Success rate, 2005-2007 (%) 17 A low-quality sponsor can jeopardize project success • IEG defines sponsors in terms of experience, financial capacity, commitment, and reputation • Low sponsor quality was prevalent in about 3540% of IFC commitments between 2002 and 2008 • Sponsor quality can vary considerably by sector, and even within the same sector 18 Better work quality leads to better outcomes Work quality comprises: • Screening, appraisal and structuring • Supervision & administration • Role & contribution throughout investment cycle With high work quality: • Development success increased from 18% to 73% • Investment success increased from 52% to 74% 19 Tracking Recommendations IEG recommendation and IFC Management response recorded IEG and IFC agree on appropriate indicators Step One Dissemination Internal Publication External Publication Step Two Management Action Tracking Annual Updates 20 IFC updates status and level of adoption (LOA) IEG updates its own Assessment of the status and LOA IEG-IFC structure Knowledge, Dissemination, Communication 21 What we do • Disseminate lessons learned from IFC’s experience among staff for better operational results • Build knowledge on private sector development issues • Disseminate IEG evaluations to public-atlarge for accountability and awareness • Develop and foster partnerships with evaluators in other development institutions 22 Products we offer Findings & Annual Evaluation Briefs Review Portfolio presentations BBLs Publications Intranet Intranet Training Training Studies & Studies Publications Studies Staff use evaluations Quick Quick Takes Takes Top Top Lessons Lessons (country, (country, sector, sector, subject) subject) Lessons Lessons Lessons from Retrieval Retrieval operations: Network e-LRN (LRN) (LRN) Project level evaluations 23 Learning by doing Easy search Best Newest lessons lessons E-LRN Home page Featured lessons Most commented lessons 24 Reports disclosed to date… • Independent Evaluation of IFC’s Development Results 2007 & 2008 • Evaluation of IFC’s Private Enterprise Partnership Program – Europe and Central Asia • Evaluation of IFC’s Assistance to Micro, Small and Medium Enterprises • Ukraine, Indonesia and Nigeria Country Impact Reviews • Evaluation of WBG’s Experience with Guarantees 25 Two recent reports… BROE: Adequacy, coverage, and quality of IFC’s development results measurement systems Health: An assessment of the WBG’s support for health, nutrition, and population since1997 26 Accountability & Dissemination 27 Learn from experience … “I could tell you of projects that failed because lessons were ignored or missed” 75% of respondents the June 2005 IFC Knowledge Survey 28 IEG can help get better results Call, e-mail, stop by! www.ifc.org/ieg IEG Help Desk 1 (202) 458-2299 [email protected] 29 Independent Evaluation in IFC Presentation to Staff of Islamic Development Bank May 3, 2009 Marvin Taylor-Dormond 30