The World Bank’s Engagement

with Earmarked Funding: The Case of Global FundsChris GerrardLead Evaluation Officer, IEGApril 17, 2012

Outline of the Presentation

► The use of Trust Funds at the World Bank► The Bank’s involvement in Global and

Regional Partnership Programs (GRPPs)► IEG findings on GRPP evaluations► Specific findings on the Five-Year

Evaluation of the Global Fund to Fight AIDS, Tuberculosis and Malaria

► Some Conclusions

Trust Funds Are Now a Significant Pillar of the Global Aid Architecture (2010)

Contributions to Bank-Administered Trust Funds Have Now Surpassed Contributions to IDA

WB TF Disbursements were $8.3 Billion in FY2011 -- by Type of Trust Fund . . .

. . . And by Type of Program

The Bank Is Involved in About 120 Global and Regional Partnership Programs (GRPPs)► About 85 global programs and 35 regional programs,

which together spent around $7.0 billion in fiscal year 2011.

► These are programmatic partnerships in which:• The partners dedicate resources (financial, technical,

staff, reputational, etc.) toward achieving agreed objectives over time.

• The activities of the program are global, regional, or multi-country (not single country) in scope.

• The partners establish a new organization with shared governance and a management unit to deliver these activities.

► 40 percent are located in the Bank, 35 percent in other international/partner organizations, and 25 percent are free-standing independent legal entities.

The Bank Plays Multiple Roles in GRPPs . . .

. . . But Is Not a Major Source of Funding

Most GRPPs Are Knowledge and Advocacy Networks . . .

Principal Activities

Location of the Secretariat

TotalIn the

World BankIn Another

PartnerIndependent Legal Entity

Knowledge Networks 11 31 11 53

Technical Assistance

For NPGs 20 4 6 30 For GPGs/RPGs 1 1 2

Financing Country-Level Investments

For NPGs 3 3 For GPGs/RPGs 13 3 6 22Financing Global Investments for GPGs 1 2 4 7

Total 48 41 28 117

But “Investment” Programs Spend Most of the Money (US$ Millions)

Principal Activities

Location of Secretariat

TotalIn the

World BankIn Another

PartnerIndependent Legal Entity

Knowledge Networks 52 68 52 171

Technical Assistance

For NPGs 184 3 94 282 For GPGs/RPGs 14 241 255Financing Country-Level Investments

For NPGs 291 291 For GPGs/RPGs 970 124 4,045 5,139Financing Global Investments for GPGs 580 66 180 826

Total 2,077 274 4,612 6,963

Most of the Growth Has Been in “Global Funds” -- FIF-Funded GRPPs Financing Investments

Program Start Date DGF Grants Governing Body

Imple-menting Agency

Secre-tariat

Services

CGIAR 1971 Yes Chair -- Yes

OCP/APOC 1974/1995 Yes Voting member -- --

GEF 1991 Official observer Yes Yes

GAVI 2000 2001-07 Voting member -- --

LDCF/SCCF 2001 Official observer Yes Yes

Global Fund 2002 Non-voting member -- --

GPE (EFA-FTI) 2002 Voting member Yes Yes

AF 2008 -- Yes Yes

CIFs 2008 Non-voting member Yes Yes

GAFSP 2010 Non-voting member Yes Yes

GRPPs Present Challenges to the World Bank . . .► They challenge the Bank’s traditional financial

accoun-tability mechanisms. Their legal and governance arrange-ments do not always confer clarity on how collective responsibility for results is supposed to work in practice.

► Bank is dedicating more and more senior management time to the governance of these programs.

► GRPPs and the Bank’s country teams plan their country-level activities in different ways. Investment programs, in particular, have the potential for collaboration -- or competition -- with the Bank’s country operations.

. . . and for Evaluation

► Programs have shared governance and accountability for results – to achieve something collectively that individual partners could not achieve by acting alone.

► Programs evolve over time – do not usually have a fixed end-point.

► Programs operate at multiple levels – global, regional, national, and local.

► Programs are diverse in size, age, sectoral focus, objectives, and activities (knowledge, technical assistance, investments).

Most GRPPs Have Been Evaluated, But Transparency & Accessibility Have Been Weak► Of the 93 programs that are five years

older or more, 77 have had evaluations.► However, only 30 evaluation reports have

been posted on the programs’ external Web sites and only 12 programs have posted a “management response” to the evaluation.

► By way of comparison, 57 programs have posted their charters on their external Web sites.

IEG’s Findings on GRPP Evaluations – Based on 20 Global Program Reviews (GPRs)► IEG annually reviews 3-4 global or regional

partnership programs (GRPPs) in which the World Bank is a partner

► Completed 20 such reviews since 2006► That of the Global Fund is the most recent –

published on February 8, 2012► These are not full-fledged evaluations. They are

reviews of the programs:• Based on an external evaluation, typically

commissioned by the governing body of the program, and

• Focusing on the Bank’s engagement with the program.

The 20 GPRs Have Covered Most Types and Locations of GRPPs

Principal Activities

Location of the Secretariat

In the World Bank

In Another Partner Organization

Independent Legal Entity

Knowledge, advocacy & standard-setting networks

CGAPIAASTDMAPS

ADEA (AfDB)GISP (CABI-Nairobi)ILC (IFAD)PARIS21 (OECD)ProVention (IFRC)

GDN (New Delhi)GFHR (Geneva)GWP (Stockholm)

Providing country-level technical assistance

Cities AllianceMDTF-EITIPRHCBPTFSCB

Financing investments (global or country-level)

CEPF (CI)Stop TB (WHO)

Development Gateway (Washington, DC)Global Fund (Geneva)MMV (Geneva)

Evaluation Independence

► 14 of 20 evaluations were effectively independent throughout the evaluation process:• Organizationally and behaviorally independent of

program management• Protected from outside interference• Avoided conflicts of interests

► Independence of 2 evaluations was compromised during one stage of the evaluation process: commissioning, team selection, oversight, or review

► Independence of 4 evaluations was compromised at more than one stage

Evaluation Quality

► The quality of the evaluations was satisfactory in 8 of 20 cases in terms of evaluation scope, design, methods, and feedback

► The evaluations had moderate shortcomings in 9 cases, and significant shortcomings in 3 cases

► Most common issues adversely affecting quality were:• An unclear terms of reference• Insufficient budget and time• Weak M&E frameworks for the program• Lax evaluation methodology and tools

Evaluation Impacts on the Programs

► Evaluations have had notable impacts on the programs.

► Most common impacts have been:• Half the programs modified their

governance arrangements in some way, such as establishing an executive committee.

• Almost half the programs revised their strategies.

• About one-quarter improved their M&E systems.

Specific Findings on the Five-Year Evaluation of the Global Fund, 2009► The Five-Year Evaluation was organizationally and

behaviorally independent of Global Fund management. It was protected from outside interference and the potential conflicts of interest that arose were appropriately identified and managed.

► Study Area 1, on the organizational efficiency and effectiveness of the Global Fund, and Study Area 2, on its partner environment at the global and country levels, were formative evaluations that have had major impacts on the Global Fund’s organizational and institutional arrangements.

► Study Area 3 was a summative evaluation of the collective efforts to reduce the burden of the three diseases. This was a contribution analysis, not an impact evaluation. It could not, by design, assess the independent contribution of the Global Fund to country-level results.

► The FYE did not achieve two major objectives — developing the “determinants” of good grant performance in Study Area 2, and building evaluation capacity in Study Area 3.

Validating the Major Findings of the FYE

1. Additionality and sustainability2. Performance of Country Coordinating

Mechanisms (CCMs)3. Effectiveness of country-level partnerships4. Application of Performance-Based Funding

(PBF)5. Access and coverage of service delivery6. Equity in country-level governance and delivery7. Impact on domestic health systems8. Institutional risk management9. Global-level governance

Additionality

ODA and Other Official Flows for HIV/AIDS, TB and Malaria

Sustainability

FYE ► Reliance on external funds and inadequate investments in long-term domestic capacity raised concerns about the long-term sustainability of recipient countries’ disease-control programs.

► The low-income countries that IEG visited were becoming increasingly dependent on Global Fund support for antiretroviral treatment of people living with AIDS.

► There are increasing concerns at the global level that other donors’ support for treatment may be less forthcoming in the future.

IEG

Country Coordinating MechanismsFYE ► The CCMs were successful in mobilizing

domestic and international partners for submission of grant proposals to the Global Fund and in enabling civil society organizations and affected communities to participate in the proposal preparation process

► But that CCMs were ill-equipped to provide adequate oversight of grant implementation.

► Little improvement since 2007 in the capacity of CCMs to oversee the implementation of Global Fund grants from the country perspective, because they generally lacked the authority and the resources to do so effectively.

IEG

Country-Level Partnerships

FYE ► Country-level partnerships with external partner agencies were based mostly on good will and voluntary collaboration rather than on negotiated commitments with clearly articulated roles and responsibilities.

► They represented more of a “friendship model” than a genuine “partnership model.”

► Partnerships with other development agencies (including the World Bank and bilateral donors) have generally improved since 2007 in terms of other partners’ providing technical assistance in support of Global Fund activities.

► However, country-level stakeholders still see the Global Fund as a largely separate development agency with its own distinct modalities that are not well integrated into the existing donor coordination mechanisms in the countries.

IEG

Performance-Based Funding (PBF)FYE ► The Global Fund had attempted to implement PBF

on a scale unprecedented in the international health arena.

► However, this “focus on results” remained a work in progress and had evolved into a complex and burdensome system that focused more on project inputs and outputs than on development outcomes and impacts.

► PBF was working reasonably well in three countries (Burkina Faso, Cambodia, and the Russian Federation) in terms of monitoring outputs and coverage in relation to the key performance indicators in the grant agreements.

► PBF was not working well in the other three countries visited (Tanzania, Nepal, and Brazil).

IEG

Grant-Level Monitoring

FYE ► The Global Fund has very detailed and well-documented requirements for grant-level monitoring, which are tied to its PBF approach.

► The Global Fund does not have a system for end-of-grant evaluations; its grant-level M&E system is designed more to facilitate grant disbursements than to contribute to an overall assessment of the program.

► While the FYE was an independent and quality evaluation, it was constrained by the absence of an M&E framework for the cumulative assessment of grant performance; it had to rely on other approaches, such as the country-level impact assessments in Study Area 3.

IEG

Conclusions

►Global Funds are growing for a number of reasons:• To provide global and regional public goods,

particularly in environment and health.• To respond to post-conflict and post-disaster situations• Dissatisfaction with traditional aid mechanisms.• The involvement of new actors and constituencies in

development.• The existence of new information and communication

technologies that facilitate collective action. • Collective decisions since 2000 to concentrate

resources on achieving selected MDGs.

GRPP Governance Has Become More Complicated

No. of programs

Inter-national/ regional organi-zations

Donor countries

Private foun-

dations

Low- and middle- income

countries

Civil society organi-zations

Commer-cial

private sector

Share-holder models

24 24(WB 24) 17 5 2 1

Stake-holder models

90 72(WB 60) 62 22 55 49 27

Total 114 96(WB 84) 79 27 55 51 29

Harmonization, Alignment and Country Ownership► Global funds are facilitating donor coordination

at the point at which donors contribute to the trust fund and serve on governing bodies, but this does not automatically translate into a similar degree of coordination at the country level.

► Development partners need to provide greater technical support to strengthen the ability of governments to effectively coordinate donor efforts around agreed national strategies.

► The long-term sustainability of the benefits of global fund investments depends on the complementary activities of donor partners and strengthening the capacity of recipient countries.

Managing for Results

► Financial management of donor trust funds has been largely successful from the donor’s perspective, but programmatic oversight has been weaker – whether GRPPs, or Bank-managed trust funds.

► There exist tensions between (a) strengthening country-level M&E systems, say, for health and the environment, and (b) demonstrating accountability for the use of donor funds.

► There is still a need to address global public policy issues such as:• Intellectual property rights in agriculture • Communicable diseases and drug resistance• Climate change

IEG Recommendations to World Bank Management► Develop a separate and strengthened

framework to guide its acceptance and management of FIFs

► The Bank should have an explicit engagement strategy for each GRPP in which it is involved, including• The expected roles of the Bank in the program at

both the global and country levels• How the program’s activities are expected to be

linked with the Bank’s country operations• How the risks to the Bank’s participation will be

identified and managed.

► Adopt a three-pillar structure for non-FIF trust funds

Adopt a Three Pillar Structure for IBRD/IDA Trust Funds

For trust funds other than FIFs, the Bank in consultation with stakeholders, should adopt a three pillar structure.

IBRD/IDA Trust Funds Financial Intermediary Funds

Single CountryPrograms

UmbrellaFacilities

Global and Regional Partnership Programs

Bank Administered Trust Funds

Multi-Country

Possible Umbrella Facility Arrangements

Umbrella Facility A

Window 2Window 1 Window 3 Window 4

Strategy/ Business Plan

Results Framework

Reporting

Umbrella Facility B

Window 1 Window 2

Strategy/ Business Plan

Results Framework

Reporting

Thank You

All our Global Program Reviews are available at:

www.globalevalutions.org