Transcript Slide 1

The World Bank’s Engagement
with Earmarked Funding:
The Case of Global Funds
Chris Gerrard
Lead Evaluation Officer, IEG
April 17, 2012
Outline of the Presentation
►The use of Trust Funds at the World Bank
►The Bank’s involvement in Global and Regional
Partnership Programs (GRPPs)
►IEG findings on GRPP evaluations
►Specific findings on the Five-Year Evaluation of the
Global Fund to Fight AIDS, Tuberculosis and Malaria
►Some Conclusions
Trust Funds Are Now a Significant Pillar of the
Global Aid Architecture (2010)
Contributions to Bank-Administered Trust Funds
Have Now Surpassed Contributions to IDA
WB TF Disbursements were $8.3 Billion
in FY2011 -- by Type of Trust Fund . . .
. . . And by Type of Program
The Bank Is Involved in About 120 Global and
Regional Partnership Programs (GRPPs)
►About 85 global programs and 35 regional programs, which
together spent around $7.0 billion in fiscal year 2011.
►These are programmatic partnerships in which:
• The partners dedicate resources (financial, technical, staff,
reputational, etc.) toward achieving agreed objectives over time.
• The activities of the program are global, regional, or multicountry (not single country) in scope.
• The partners establish a new organization with shared
governance and a management unit to deliver these activities.
►40 percent are located in the Bank, 35 percent in other
international/partner organizations, and 25 percent are freestanding independent legal entities.
The Bank Plays Multiple Roles
in GRPPs . . .
. . . But Is Not a Major Source of Funding
Most GRPPs Are Knowledge and
Advocacy Networks . . .
Location of the Secretariat
Principal Activities
Knowledge Networks
In the
World Bank
In Another
Partner
Independent
Legal Entity
Total
11
31
11
53
20
4
6
30
1
1
2
Technical Assistance
For NPGs
For GPGs/RPGs
Financing Country-Level Investments
For NPGs
3
For GPGs/RPGs
13
3
6
22
Financing Global
Investments for GPGs
1
2
4
7
Total
48
41
28
117
3
But “Investment” Programs Spend Most
of the Money (US$ Millions)
Location of Secretariat
Principal Activities
Knowledge Networks
In the
World Bank
In Another
Partner
Independent
Legal Entity
Total
52
68
52
171
184
3
94
282
14
241
255
Technical Assistance
For NPGs
For GPGs/RPGs
Financing CountryLevel Investments
For NPGs
291
For GPGs/RPGs
970
124
4,045
5,139
Financing Global
Investments for GPGs
580
66
180
826
2,077
274
4,612
6,963
Total
291
Most of the Growth Has Been in “Global Funds”
-- FIF-Funded GRPPs Financing Investments
Program
CGIAR
OCP/APOC
Governing Body
Implementing
Agency
Secretariat
Services
Yes
Chair
--
Yes
Yes
Voting member
--
--
Official observer
Yes
Yes
Voting member
--
--
Start Date
DGF
Grants
1971
1974/1995
GEF
1991
GAVI
2000
LDCF/SCCF
2001
Official observer
Yes
Yes
Global Fund
2002
Non-voting member
--
--
GPE (EFA-FTI)
2002
Voting member
Yes
Yes
AF
2008
--
Yes
Yes
CIFs
2008
Non-voting member
Yes
Yes
GAFSP
2010
Non-voting member
Yes
Yes
2001-07
GRPPs Present Challenges to the
World Bank . . .
►They challenge the Bank’s traditional financial accountability mechanisms. Their legal and governance arrangements do not always confer clarity on how collective
responsibility for results is supposed to work in practice.
►Bank is dedicating more and more senior management
time to the governance of these programs.
►GRPPs and the Bank’s country teams plan their countrylevel activities in different ways. Investment programs, in
particular, have the potential for collaboration -- or
competition -- with the Bank’s country operations.
. . . and for Evaluation
►Programs have shared governance and accountability
for results – to achieve something collectively that
individual partners could not achieve by acting alone.
►Programs evolve over time – do not usually have a
fixed end-point.
►Programs operate at multiple levels – global, regional,
national, and local.
►Programs are diverse in size, age, sectoral focus,
objectives, and activities (knowledge, technical
assistance, investments).
Most GRPPs Have Been Evaluated, But
Transparency & Accessibility Have Been Weak
►Of the 93 programs that are five years older or more,
77 have had evaluations.
►However, only 30 evaluation reports have been posted
on the programs’ external Web sites and only 12
programs have posted a “management response” to the
evaluation.
►By way of comparison, 57 programs have posted their
charters on their external Web sites.
IEG’s Findings on GRPP Evaluations – Based
on 20 Global Program Reviews (GPRs)
►IEG annually reviews 3-4 global or regional partnership
programs (GRPPs) in which the World Bank is a partner
►Completed 20 such reviews since 2006
►That of the Global Fund is the most recent – published
on February 8, 2012
►These are not full-fledged evaluations. They are reviews
of the programs:
• Based on an external evaluation, typically commissioned by the
governing body of the program, and
• Focusing on the Bank’s engagement with the program.
The 20 GPRs Have Covered Most Types
and Locations of GRPPs
Location of the Secretariat
Principal Activities
In the
World Bank
Knowledge,
advocacy &
standard-setting
networks
CGAP
IAASTD
MAPS
Providing countrylevel technical
assistance
Cities Alliance
MDTF-EITI
PRHCBP
TFSCB
Financing
investments
(global or
country-level)
In Another Partner
Organization
Independent Legal
Entity
ADEA (AfDB)
GISP (CABI-Nairobi)
ILC (IFAD)
PARIS21 (OECD)
ProVention (IFRC)
GDN (New Delhi)
GFHR (Geneva)
GWP (Stockholm)
CEPF (CI)
Stop TB (WHO)
Development Gateway
(Washington, DC)
Global Fund (Geneva)
MMV (Geneva)
Evaluation Independence
►14 of 20 evaluations were effectively independent
throughout the evaluation process:
• Organizationally and behaviorally independent of program
management
• Protected from outside interference
• Avoided conflicts of interests
►Independence of 2 evaluations was compromised
during one stage of the evaluation process:
commissioning, team selection, oversight, or review
►Independence of 4 evaluations was compromised at
more than one stage
Evaluation Quality
►The quality of the evaluations was satisfactory in 8 of
20 cases in terms of evaluation scope, design, methods,
and feedback
►The evaluations had moderate shortcomings in 9 cases,
and significant shortcomings in 3 cases
►Most common issues adversely affecting quality were:
•
•
•
•
An unclear terms of reference
Insufficient budget and time
Weak M&E frameworks for the program
Lax evaluation methodology and tools
Evaluation Impacts on the Programs
►Evaluations have had notable impacts on the programs.
►Most common impacts have been:
• Half the programs modified their governance
arrangements in some way, such as establishing an executive
committee.
• Almost half the programs revised their strategies.
• About one-quarter improved their M&E systems.
Specific Findings on the Five-Year
Evaluation of the Global Fund, 2009
► The Five-Year Evaluation was organizationally and behaviorally
independent of Global Fund management. It was protected from
outside interference and the potential conflicts of interest that arose
were appropriately identified and managed.
► Study Area 1, on the organizational efficiency and effectiveness of the
Global Fund, and Study Area 2, on its partner environment at the
global and country levels, were formative evaluations that have had
major impacts on the Global Fund’s organizational and institutional
arrangements.
► Study Area 3 was a summative evaluation of the collective efforts to
reduce the burden of the three diseases. This was a contribution
analysis, not an impact evaluation. It could not, by design, assess the
independent contribution of the Global Fund to country-level results.
► The FYE did not achieve two major objectives — developing the
“determinants” of good grant performance in Study Area 2, and
building evaluation capacity in Study Area 3.
Validating the Major Findings of the FYE
1. Additionality and sustainability
2. Performance of Country Coordinating Mechanisms
(CCMs)
3. Effectiveness of country-level partnerships
4. Application of Performance-Based Funding (PBF)
5. Access and coverage of service delivery
6. Equity in country-level governance and delivery
7. Impact on domestic health systems
8. Institutional risk management
9. Global-level governance
Additionality
ODA and Other Official Flows for HIV/AIDS, TB and Malaria
Sustainability
FYE
► Reliance on external funds and inadequate investments in
long-term domestic capacity raised concerns about the
long-term sustainability of recipient countries’ diseasecontrol programs.
IEG
► The low-income countries that IEG visited were
becoming increasingly dependent on Global Fund support
for antiretroviral treatment of people living with AIDS.
► There are increasing concerns at the global level that other
donors’ support for treatment may be less forthcoming in
the future.
Country Coordinating Mechanisms
FYE
► The CCMs were successful in mobilizing domestic and
international partners for submission of grant proposals
to the Global Fund and in enabling civil society
organizations and affected communities to participate in
the proposal preparation process
► But that CCMs were ill-equipped to provide adequate
oversight of grant implementation.
IEG
► Little improvement since 2007 in the capacity of CCMs to
oversee the implementation of Global Fund grants from
the country perspective, because they generally lacked the
authority and the resources to do so effectively.
Country-Level Partnerships
FYE
► Country-level partnerships with external partner agencies were
based mostly on good will and voluntary collaboration rather
than on negotiated commitments with clearly articulated roles
and responsibilities.
► They represented more of a “friendship model” than a genuine
“partnership model.”
IEG
► Partnerships with other development agencies (including the
World Bank and bilateral donors) have generally improved since
2007 in terms of other partners’ providing technical assistance
in support of Global Fund activities.
► However, country-level stakeholders still see the Global Fund as
a largely separate development agency with its own distinct
modalities that are not well integrated into the existing donor
coordination mechanisms in the countries.
Performance-Based Funding (PBF)
FYE
► The Global Fund had attempted to implement PBF on a
scale unprecedented in the international health arena.
► However, this “focus on results” remained a work in
progress and had evolved into a complex and burdensome
system that focused more on project inputs and outputs
than on development outcomes and impacts.
IEG
► PBF was working reasonably well in three countries
(Burkina Faso, Cambodia, and the Russian Federation) in
terms of monitoring outputs and coverage in relation to
the key performance indicators in the grant agreements.
► PBF was not working well in the other three countries
visited (Tanzania, Nepal, and Brazil).
Grant-Level Monitoring
FYE
► The Global Fund has very detailed and well-documented
requirements for grant-level monitoring, which are tied to
its PBF approach.
IEG
► The Global Fund does not have a system for end-of-grant
evaluations; its grant-level M&E system is designed more
to facilitate grant disbursements than to contribute to an
overall assessment of the program.
► While the FYE was an independent and quality evaluation,
it was constrained by the absence of an M&E framework
for the cumulative assessment of grant performance; it
had to rely on other approaches, such as the country-level
impact assessments in Study Area 3.
Conclusions
►Global Funds are growing for a number of reasons:
• To provide global and regional public goods, particularly in
environment and health.
• To respond to post-conflict and post-disaster situations
• Dissatisfaction with traditional aid mechanisms.
• The involvement of new actors and constituencies in
development.
• The existence of new information and communication
technologies that facilitate collective action.
• Collective decisions since 2000 to concentrate resources on
achieving selected MDGs.
GRPP Governance Has Become More
Complicated
No. of
programs
International/
regional
organizations
Donor
countries
Private
foundations
Shareholder
models
24
24
(WB 24)
17
5
Stakeholder
models
90
72
(WB 60)
62
22
Total
114
96
(WB 84)
79
27
Low- and
middleincome
countries
Civil
society
organizations
Commercial
private
sector
2
1
55
49
27
55
51
29
Harmonization, Alignment and Country
Ownership
►Global funds are facilitating donor coordination at the point
at which donors contribute to the trust fund and serve on
governing bodies, but this does not automatically translate
into a similar degree of coordination at the country level.
►Development partners need to provide greater technical
support to strengthen the ability of governments to
effectively coordinate donor efforts around agreed national
strategies.
►The long-term sustainability of the benefits of global fund
investments depends on the complementary activities of
donor partners and strengthening the capacity of recipient
countries.
Managing for Results
►Financial management of donor trust funds has been
largely successful from the donor’s perspective, but
programmatic oversight has been weaker – whether GRPPs,
or Bank-managed trust funds.
►There exist tensions between (a) strengthening countrylevel M&E systems, say, for health and the environment,
and (b) demonstrating accountability for the use of donor
funds.
►There is still a need to address global public policy issues
such as:
• Intellectual property rights in agriculture
• Communicable diseases and drug resistance
• Climate change
IEG Recommendations to
World Bank Management
►Develop a separate and strengthened framework to
guide its acceptance and management of FIFs
►The Bank should have an explicit engagement strategy
for each GRPP in which it is involved, including
• The expected roles of the Bank in the program at both the
global and country levels
• How the program’s activities are expected to be linked with
the Bank’s country operations
• How the risks to the Bank’s participation will be identified
and managed.
►Adopt a three-pillar structure for non-FIF trust funds
Adopt a Three Pillar Structure for IBRD/IDA
Trust Funds
For trust funds other than FIFs, the Bank in consultation
with stakeholders, should adopt a three pillar structure.
Bank Administered Trust Funds
IBRD/IDA Trust Funds
Financial
Intermediary Funds
Multi-Country
Single Country
Programs
Umbrella
Facilities
Global and Regional
Partnership Programs
Possible Umbrella Facility Arrangements
Strategy/ Business Plan
Umbrella Facility A
Results Framework
Reporting
Window 1
Window 2
Window 3
Window 4
Strategy/ Business Plan
Umbrella Facility B
Results Framework
Reporting
Window 1
Window 2
Thank You
All our Global Program Reviews
are available at:
www.globalevalutions.org