Consortium of British Humanitarian Agencies (CBHA) mid-term review

cbha mid-term review Andy Featherstone

February 2011

Final report

cbha mid-term review

2

contents

executive Summary

acronyms

3

5

1. IntroductIon and Methodology 6

2.

IncreasIng access to Fast, eFFIcIent and eFFectIve FundIng For Front-lIne huManItarIan Work

7

3. IncreasIng nuMbers oF coMpetent natIonal and InternatIonal Managers and leaders

15

4. IncreasIng agency surge capacIty to respond approprIately to neW eMergencIes

19

5. strengthenIng huManItarIan logIstIcs systeMs 21

6. assessIng eFFectIveness & learnIng FroM experIence 23

7. More than the suM oF Its parts strengthenIng the capacIty oF the 3rd pIllar

24

8. suMMary oF conclusIons and recoMMendatIons 26

annexes 27

This Mid-Term Review was commissioned by the Consortium of British Humanitarian Agencies (CBHA) and funded by the Department for International Development (DFiD). It was written by Andy Featherstone, and supported by Action Against Hunger (UK) as part of the Learn ing & Evaluation project (Output 2, Indicator 2, Milestone 1). The recommenda-tions made in this report will contribute to future workshops and learning papers scheduled for 2011 & 2012.

Cover photograph: acF-UK

3cbha mid-term review

Introduction & Methodology1. Introduction: The CBHA (Consortium of British Humanitarian agencies) is a ground breaking initiative for strength-ening the co-ordination and capacity of the NGO sector to deliver appropriate, high quality, and quicker humanitar-ian assistance to populations affected by disaster. It consists of fifteen UK NGOs who have committed to imple-menting 5 objectives which support this purpose.

Methodology: The purpose of the review is to strengthen CBHA planning and sustainability by providing clear and concise lessons about the implementation of its four streams of work to date. It will build on existing data collected through CBHA monitoring, evaluation and learning activities1.The goals of the review are to provide a solid founda-tion of evidence upon which the Consortium will formulate strategy and to inform the CBHAs first annual report to DFID (Department for International Development).

Limitations: The terms of the reference for the review were clear but ambitious. Because the review was being conducted at a relatively early stage in the Consortiums life (11-months into its 24-month lifespan), information and evidence for some of the outputs was limited; although in others, particularly the Emergency Response Fund (ERF) and the capacity building component, there was significant evidence available. Some changes were made to the ToR as it was not always possible to accurately identify impact or return on investment within such a short space of time.

Structure of the Report: This report seeks to document and analyse the evidence across the 5 objectives of the CBHA. Section 1 gives an introduction to the CBHA and provides an overview of the methodology. Section 2 through section 6 reviews each of the 5 outputs of the CBHA and seeks to document the baseline data, review the achievements made against the logical framework, assess the contribution made towards strengthening humani-tarian capacity and makes recommendations for the second year of the programme. Section 7 focuses attention at the purpose level of the programme and reviews progress made towards building capacity for humanitarian response and strengthening collaboration. Section 8 provides a summary conclusion and lists the recommenda-tions.

Increasing access to Fast, efficient and effective Funding for Front-line humanitarian Work2. The establishment of a funding facility to underwrite emergency assessments and rapid response to humanitarian crises is the largest component of the CBHA and accounts for 50% of the DFID grant. With governance arrange-ments and a process for allocating grants having been established in March 2010, funding has been allocated to responses in 4 countries (Kyrgyzstan, Myanmar, Pakistan and Sri Lanka) and over 50% of the funds have been disbursed. 14 of the 15 Consortium members have made successful applications for funding and a comprehen-sive L&E framework has collected significant evidence of the ERF facilitating timely and effective humanitarian re-sponse. While a workshop2 has contributed to learning on the ERF the boundaries of the CBHAs funding objective have also been expanded through 2 innovative responses to meeting the overwhelming needs of those affected by floods in Pakistan. In addition to the ERF grant, DFID channelled an additional 1m of funds to Consortium members for a 3-month response and in November a group of 6 CBH agencies embarked on an 8-month 20 mil-lion DFID-funded recovery response.

Increasing numbers of competent national and International Managers and leaders3. The cornerstone of the capacity building objective is the humanitarian competency framework which was devel-oped and jointly signed off by all 15-agencies in August. This has played a key role in the development of two complimentary pieces of work; the Humanitarian Leadership Development Programme (HLDP) which has already commenced its first cycle in Nairobi and the Humanitarian Staff Development Project which will start its first set of learning programmes in the Horn of Africa later this month. While it is too early to speak of impact both are well-supported by the CBHA members and have the potential to fill important capacity gaps. A learning and evaluation framework has been articulated which has the potential to provide a sound basis for learning.

Increasing agency surge capacity to respond appropriately to new emergencies4. Surge grants of 110,000 to each member for the two-year programme have provided important capacity at a time when internal resources have been squeezed as a result of budget constraints and significant humanitarian need. Surge capacity is essential for meeting the objectives of the CBHA to strengthen timely and effective humanitar-

executIve suMMary

The full ToR is reproduced in Annex 1.1. The outputs of the meeting are documented in CBHA (2010) Learning Paper 1: Emergency Relief Fund (ERF), December 2010 available at 2. http://www.thecbha.org/media/website/file/CBHA_Learning_Paper_1.pdf


4

ian response and evidence suggests that the funds have had considerable impact. While the ERF addresses the problem of agencies accessing timely funding, the surge grants have gone some way to filling agency capacity constraints. The significant value of the grants to Consortium members and the diverse approaches that have been adopted suggest that there is significant potential in this objective for strengthening learning and collaboration in the future.

strengthening humanitarian logistics systems5. The scoping documents submitted by participating Consortium members showed great interest in the potential outputs of the Helios project and there continues to be considerable enthusiasm for the collaboration upon which the success of the project depends. However it has taken time for agencies to develop an adequate knowledge of the system to be able to make judgements on its potential value. For those which continue to prioritise their en-gagement with Helios and which continue to commit resources (particularly staff time) to this CBHA objective there is a need to review what can be achieved by project close and to ensure adequate resourcing of the programme for the year ahead.

learning & evaluation6. A sound start has been made in collecting evidence to underpin the results achieved by the CBHA - this is par-ticularly evident for the ERF objective which has collected an extremely rich data set that provides important les-sons on the timeliness, appropriateness and effectiveness of the rapid response funding facility. L&E across the other three objectives of the CBHA lag behind that of the ERF although there is now an L&E plan in place for the Capacity Building Objective which sets out a coherent framework and establishes questions around which learn-ing will be focused. The surge capacity objective and Helios objective do not have dedicated L&E plans although progress is documented through annual reports. Despite sound progress having been made interviews suggest that the L&E function is under-resourced and there is a risk that ambition may exceed the capacity of the team to deliver against it.

More than the sum of its parts strengthening the capacity of the 37. rd pillarWhile the programme objectives provide a good fit with the overall goal of the CBHA there was clearly a degree of opportunism in their selection but this should be considered a strength rather than a weakness. With the excep-tion of the ERF the CBHA has drawn its inspiration from the practice of its members and has selected projects which are credible, have a track record of success or potential to deliver it and which have a champion. While the ERF has its influences outside of the Consortium members its development is clearly rooted in existing practice (the DEC and UN Emergency Response Fund both have similar goals) albeit with a focus on facilitating timely response which it has achieved with some success. This review has provided early evidence of an important con-tribution made to humanitarian response.

While capacity is the end, collaboration is the means to that end and for the CBHA to realise its transformative agenda it will need to galvanise unprecedented levels of trust across its members and beyond. The review has documented the promising start that has been made towards achieving this but there is considerable work still to do and as it approaches its second year the most important task will be to learn from its early successes and to seek to build the trust that will be required for it to meet its ambitions for having broader impact in the sector.

summary conclusion & recommendations8. The review has found that considerable progress has been made across the 5 objectives of the CBHA and that the Consortium has made an important contribution to strengthening humanitarian capacity and delivering humanitar-ian response. The majority of CBHA activities have happened according to plan and in most cases agencies have achieved or gone beyond their stated objectives. Reporting across the CBHA agencies has also generally been of a high standard. This is no small achievement for such a young interagency entity and is indicative of the value that agencies are placing on documenting results and demonstrating impact, a commitment that came out clearly during the review.


aa Action )Aid

acF Action Against Hunger

aar After Action Review

alnap Active Learning Network on Accountability & Performance in Humanitarian Action

ca Christian Aid

caFod Catholic Overseas Development Agency

cap (UN) Consolidated Appeal Process

cbha Consortium of British Humanitarian Agencies

cerF Central Emergency Response Fund

dac Development Assistance Committee

dec Disasters Emergencies Committee

dFId Department for International Development

ecb Emergency Capacity Building Project

erF Emergency Response Fund

elhra Enhanced Learning & Research for Humanitarian Agencies

Fata Federally Administered Tribal Areas

haI Help Age International

herr Humanitarian Emergency Response Review

hldp Humanitarian Leadership and Development Programme and Trainee Scheme

Irc International Rescue Committee

IrW Islamic Relief Worldwide

l&e Learning & Evaluation

MerlIn Medical Emergency Relief International

ngo Non-governmental Organisation

nsdp National Staff Development Programme

odI Overseas Development Institute

pia People in Aid

pMu Programme Management Unit

rte Real Time Evaluation

sc Save the Children

un United Nations

WahrF West Africa Humanitarian Response Fund

Wv World Vision

acronyMs


6

1.1 Introduction to the cbhaThe CBHA is a ground breaking initiative for strengthening the coordination and capacity of the NGO sector to deliver appropriate, high quality, and quicker humanitarian assistance to populations affected by disaster. It consists of fifteen UK NGOs who have committed to implementing activities which support this objective.3 The CBHA has received initial two-year funding of 8,000,000 from DFID for five streams of work. The longer term vision is to work with more donors to increase the size of the emergency response fund and be able to provide more assistance to more communities, to save lives and help them recover from disasters.

IntroductIon and Methodology1.

The CBHA has been enabled by a contemporary recognition in British NGOs that rapid transformation is required. These 15 organisations embrace the network paradigm of the age and seem to maxim-ise the potential of a complex network of civil society relationships that transcend traditional political boundaries. The CBHA is a new form of virtual organisation consisting of an astonishing array of several thousand organisational relationships that offers great potential in scaling up the challenges of the age. At minimum the pathways in this network offer a rapid and highly efficient method of getting resources to the front line of humanitarian action in the most vulnerable areas of the world. The CBHA directly ad-dresses long-term NGO weaknesses while increasing its capacity: in human resource systems, in the ability to adopt technological innovations and tools, and in how NGOs collaborate with each other.4

box 1: the emerging potential of the cbha

1.2 purpose & objectives of the cbhaThe purpose of the CBHA is to reduce suffering, mortality and morbidity in conflicts and natural disasters by strength-ening the coordination and capacity of the third pillar the NGO sector - to deliver appropriate, higher quality, more effective and quicker humanitarian responses over the current decade, 2010-2020.

To make this objective tangible and achievable a number of programme outputs have been developed in a programme logical framework:

Consortium member agencies have immediate access to emergency response funding enabling them to mount focused and effective field operations in the immediate aftermath of a disaster or other crisis;

Increased numbers and competencies of leaders through developing a competency framework and delivering it through both national and international programmes;

Consortium member agencies have available standing capacity to respond quickly and effectively to emergencies;

Improve the ability for organisations to respond more efficiently in humanitarian and relief work through the adop-tion of information technology and best practices in the control of their supply chains;

Evidence is available to assess the effectiveness of the programme in the first two-years and it is integrated in future strategic planning of the CBHA.

Each of these objectives has been given a percentage impact weighting in the logical framework which gives an indica-tion of the importance attached to it; the ERF funding and capacity building objectives are given the highest weighting, that of 30%. The L&E objective is given the next greatest weighting at 20% despite having the smallest budget alloca-tion5. The surge capacity objective has a weighting of 15% and the Helios objective has a weighting of 5%.

1.3 purpose of the reviewThe purpose of the review is to strengthen CBHA planning and sustainability by providing clear and concise lessons about the implementation of its four streams of work to date. It will build on existing data collected through CBHA

Members of the CBHA include ActionAid, Action Against hunger, CARE International UK, CAFOD, Christian Aid, Concern Worldwide UK, HelpAge International, 3. International Rescue Committee UK, Islamic Relief Worldwide, Merlin, Oxfam GB, Plan International UK, Save the Children UK, Tearfund and World Vision UK.Taken from the CBHAs submission to DFIDs Humanitarian Emergency Response Review (HERR), December 2010.4. The L&E component accounts for 0.5% of the CBHA budget however this amount increases significantly when L&E activities in each of the 4 CBHA objectives 5. are added to this figure.


monitoring, evaluation and learning activities6. The goals of the review are to provide a solid foundation of evidence upon which the Consortium will formulate strategy and to inform the CBHAs first annual report to DFID.

1.4 MethodologyThe review was undertaken over a 15-day period and focused on two methods of enquiry; a desk-based study of rel-evant CBHA documents including the CBHA Baseline, reports, learning papers, real-time evaluations (RTE) and strate-gic documents and interviews with the 8 staff employed to implement activities (from Save the Children, ActionAid, the Helios Foundation and Action Against Hunger). While the reviewer sought to assess progress made against all aspects of the logical framework, greater priority was given to those objectives which carried the highest impact weighting.

1.5 limitationsThe terms of the reference for the review were clear but ambitious. By CBHAs own admission the baseline document is imperfect and can at best be considered a scoping document. Because the review was conducted at a relatively early stage in the Consortiums life information and evidence for some of the work streams were limited, although in others, particularly the ERF, there was significant evidence available. Some changes were made to the ToR as it was not always possible to accurately identify impact or estimate return on investment. Instead the review sought to assess the con-tribution which each of the work streams made to strengthening humanitarian response. While the logical framework provides a summary of the objectives and the milestones required to achieve them the lack of an overarching strategy for the CBHA makes it difficult to review the whole as more than the sum of its parts.

Objective: The ERF mechanism disburses flexible pre-positioned funds to NGOs in order to mount field operations in the immediate aftermath of a disaster or other crisis in advance of funding becoming available through the usual existing channels (whether from DFID, other donors or the UN). The aim of the fund is to facilitate rapid response and enable those who are best placed among members of the Consortium and/or their affiliates to respond immediately in the event of a rapid onset disaster, a sudden deterioration in a chronic emergency or early warning indicators in a slow onset disaster.

2.1 reviewing the baseline and Identifying gaps in agency humanitarian response capacityEvidence suggests there has been an escalation in the loss of lives and destruction of livelihoods as a result of hu-manitarian crises and natural disasters over recent years. Save the Children, in their state of the system report7 provide compelling evidence of the increase in the number of crises and predict that this trend will continue due to a cocktail of inter-connecting factors which include the effects of climate change, continued population growth and rapid urbani-sation. The increase in global need has seen the resources of governments and humanitarian agencies stretched and there is a growing perception that the humanitarian system is failing to provide adequate timely and effective assist-ance to people affected by crisis. The scale of need and perceived weaknesses in the response to the Haiti earthquake and Pakistan floods have only served to strengthen this view.

In providing access to timely funds the CBHA is seeking to directly address such criticisms and contribute to swifter humanitarian assistance. The baseline document explores some of the greatest impediments to timely decision-mak-ing and resource mobilisation for member agencies. It highlights the significant time it can take agencies to access external funding streams for rapid onset crises and documents the limited access that many agencies have to internal funds. While the majority of agencies make a decision whether or not to respond within 72-hours, it can take up to 10-days for programmes to commence. For slow onset and chronic crises delays in programme implementation can be even longer.

While it is difficult to draw conclusions about how agency capacity has been affected by the global recession, particularly during a period when the scale of humanitarian need has provoked significant public generosity and has necessitated the launch of the UNs largest ever Consolidated Appeal, many of the CBHA agencies have seen a reduction in their incomes. The baseline suggests there is significant potential for the ERF to have a positive impact on response with the majority of those surveyed considering the ERF necessary for them to expand their humanitarian response capacity.

The full ToR is reproduced in Annex 1.6. Bookstein A & Dempsey B (2010) At a Crossroads, Humanitarianism in the Next Decade Save the Children. 7.

IncreasIng access to Fast, eFFIcIent and eFFectIve 2. FundIng For Front-lIne huManItarIan Work


8

While the baseline is imperfect its focus on exploring the differential access agencies have to timely funds provides an accurate reflection of one of the key impediments to timely and effective response and as such acknowledges some of the key issues that the ERF seeks to address.

2.2 achievements against the logical Framework

The Establishment of the ERF and the Scope of its DisbursementsThe establishment of a funding facility to underwrite emergency assessments and rapid response to humanitarian crises is the largest component of the CBHA. Fifty per cent of the DFID grant (4 million) has been allocated to this fund for the purpose of providing predictable, flexible seed money in advance of other emergency funding becoming available. Swift progress has been made in establishing the fund, agreeing a governance structure, articulating and disseminating a set of guidelines for decision-making and outlining reporting requirements. These appear to be well-understood by member-agencies and have been used throughout the CBHAs first year of operations.

To date alerts have been called by CBHA members for a total of 11 emergencies of which 7 have been followed up with conference calls and ERF funds have been allocated to 4 responses (details of the alerts, teleconferences and alloca-tions are provided in Annex 15). CBH agencies have received funds to conduct assessments and launch humanitarian response to the Kyrgyzstan conflict in June 2010 (162,212), for the Pakistan floods in August 2010 (750,000 and 996,000), for the Myanmar cyclone in October 2010 (450,000) and the Sri Lanka floods in January 2011 (285,000).

Timeliness of Decision-making Processes and Fund AllocationAs it seeks to address the failure of agencies to access rapid funding for assessment and response the timeliness of ERF funds is a key indicator of success and one in which it has performed extremely well. The response to Cyclone Giri in Myanmar provides a good example and one that is representative of other ERF responses. The Cyclone made landfall on 23 September, Christian Aid raised an alert on 25 September, a teleconference was held the next day and on 27 September a decision was taken to allocate 450,000 to CBHA members for an emergency response. The funds were transferred to four agencies whose proposals had been successfully peer-reviewed on 29 September and relief activities commenced the same day. All programmes were completed by 29 October as planned.

While the timeliness of the decision-making process and grant allocation is excellent once an alert has been called, a review of ERF allocation dates against significant media reports and a comparison against actions taken by other donors provides interesting evidence.

Sri Lanka: On 13 January the BBC reported that 1-million people had been affected and it was on 14 January that the CBHA released over 285,000 to 5 members of the Consortium. The UN Flash Appeal was launched four days later on 18 January.

Pakistan: The BBC reported that 1,100 had been killed by the floods and over 1.5 million had been affected on 02 August. The following day the ERF released 750,000 to 8 members to fund responses in Federally Administered Tribal Areas (FATA) and Khyber Pakhtunkhwa Provinces. The UK-based NGO, Muslim Aid launched an Appeal for Pakistan on 30 July (at which time assessments were taking place) and CBHA member Islamic Relief launched an appeal on 31 July (by which time they had already started relief distributions). The DEC launched their appeal on 03 August.

Kyrgyzstan: The CBHA provided ERF funding to three agencies between 18 and 23 June8 for people affected by the conflict in Kyrgyzstan and those displaced to Uzbekistan. The UN Central Emergency Response Fund (CERF) allocated $1 million to humanitarian response to the conflict on 29 April; ICRC were providing assistance to war-wounded by mid-May but it was not until the middle of June that security was adequate for NGOs to conduct any significant work so in this respect, the CBHA funds were timely.

While the data presented is far from exhaustive it offers some insights about ERF decision-making; there is an in-teresting link between BBC reports and appeals being called underlining the inter-linkage between the media and humanitarian funding decisions; the Kyrgyzstan example shows the added complexity of making timely responses to complex emergencies and the impact that operational security has on this; the launching of the DEC appeal so close to the CBHA appeal for Pakistan suggests the importance of understanding the donor landscape and the potential for complementarity or duplication between different funding streams. The limited time available for the review meant that it wasnt possible to conduct a more detailed comparative analysis of the timeliness of different funding streams (which

The reason for the staggered timeline was because the situation was still unclear and agencies were waiting to receive assessments from the field. As a result of 8. this the CBHA board agreed to set two deadlines. After the Kyrgyzstan response the ERF sub committee and board agreed not to use multiple deadlines again.


in addition to the DEC also includes the UN CERF, DFID WAHRF and myriad others) and this may be an area that the CBHA may want to explore in more detail during the evaluation.

Feedback documented during the ERF learning review9 and interviews with CBHA members highlighted broad satisfac-tion with the timeliness of the fund once an alert had been called but concern was raised about the high transaction costs incurred during decision-making processes and about the almost exclusive use of the fund to date for response to natural disasters (Kyrgyzstan being the exception). While the prevalence and severity of floods and cyclones over the last year have played an important part in this, members felt that it was important to ensure that the ERF responded to the broadest range of crises and there was particular interest in exploring how it could respond appropriately to com-plex and chronic crises which are often the hardest responses to fundraise for. In suggesting this there was recognition that the present ways of working the 30-day response time and the limited size of the grants made might not be appropriate to such contexts which may limit its effectiveness.

A number of suggestions were made about how to reduce transaction costs and how to strengthen the impartiality of decision-making processes during the ERF learning review; it was felt that a pre-classification of countries may assist in guiding early decisions about when to raise an alert which could draw on evidence such as government capacity, internal response resources and the vulnerability of the population to crisis. There was also considerable support for establishing a framework or set of principles to guide decision-making during the ERF teleconferences which could draw on evidence about the scale and magnitude of the crisis, whether the host country had requested international assistance and the in-country capacity to respond to the crisis. While these criteria nominally exist as part of a basket of discussion points for the teleconference agenda, it was felt that their systematic use to drive decision-making could potentially reduce transaction costs and strengthen impartiality of the process.

How Have the Funds Been Spent: A Profile of Fund-use by CBH AgenciesTo date the ERF has disbursed over 2.64 million to its members (this includes the 996,000 DFID accountable grant in addition to the 4 million fund for the second flood response in Pakistan10) which equates to a 53% spend against the budget. This is an encouraging start for a new funding mechanism and the significant number of proposals that have been submitted by CBH agencies for its use goes some way to proving agency demand.

The table in annex 12 provides an analysis of funding by agency and a breakdown of the budget by chapter and shows that 14 of the 15 agencies have received ERF funding11. Oxfam is the only member which has not received funding as it considers its own internal funds sufficient for the purposes of assessment and initial response and consequently has made no applica-tions. For the agencies who have received ERF funds there has been a relatively equitable division with 5 of the 15 Consortium agencies having received between 200-300,000. The mean total allocation for the 13 beneficiary members is 181,40012. Interviews with CBH agencies and material from the ERF Learning Workshop provide mixed feedback about the size of the disbursements; for some the ERF allocations play a significant part in their decision-making about whether to respond to a crisis, but for the larger agencies the motivation may be less about accessing the fund now and more about strategic engage-ment in the CBHA for the future. This is not to diminish the value of the grants or the impact they have which the results show are significant - but it is important to note the mixed motivations that exist within the diverse group of agencies.

The CBHA has gone to considerable lengths to ensure the transparency of its decision-making processes for ERF al-locations. The Programme Handbook outlines a process for members to apply for up to 200,000 based on a clear eligibility criteria13 and decisions for all ERF proposals are based on a peer review mechanism. Data made available to the reviewer for each of the ERF funding decisions included agency proposals, a criteria matrix to facilitate decision-making and a table detailing the funding allocation and outlining reporting requirements. The ERF learning review held in November 2010 underlined the importance of the peer review process in building trust between agencies. Interviews also highlighted the impact that the process has had on driving up the quality of programme proposals with a signifi-cant improvement having been made since the initial set of funding applications for the Kyrgyzstan response. While part of this is doubtless due to the growing familiarity that agencies have with the proposal format and decision-making process, review participants were particularly candid about the internal reflection that many had undertaken about the quality of their proposals as a result of the peer review process.

An analysis of the financial data for the ERF allocations (which is summarised in annex 11) shows that the majority of funds (78%) were used for used to purchase supplies (relief items) which were distributed to project participants. The

The outputs of the meeting are documented in CBHA (2010) Learning Paper 1: Emergency Relief Fund (ERF), December 2010 available at 9. http://www.thecbha.org/media/website/file/CBHA_Learning_Paper_1.pdf The initial fund was 4m. The additional 1m for the 2nd Pakistan flood response brings the total to 5m.10. Care received its first ERF allocation for Sri Lanka data for which isnt included in the table.11. These calculations do not include Sir Lanka as budget figures were not made available to the reviewer.12. The criteria requires that agencies have existing programmes or partners in close proximity to the disaster zone; that they have the ability to mount a surge 13. operation; and, they have the ability to start the operation within 7-days of the telecon and complete it within 30-days


10

other main budget chapters (which include logistics, personnel, personnel support and project management support) account for between 4% and 6% of total agency expenditure and project equipment accounted for only 0.2% of the total disbursement which is in line with the guidance provided in the Programme Handbook.

In terms of fund-use, the picture is relatively uniform across the different members of the Consortium; the vast majority used their ERF allocations to support rapid response. In one case only have the funds been used to establish an agency presence in a country. For all other responses reports show that agencies already had a presence or partners in the country, often in close proximity to where the assistance was required and funds were typically used to purchase relief items, to support beneficiary identification and to cover the costs of distribution. While timely decision-making is an important precursor to rapid deployment of staff, the cost analysis provides evidence that the CBH agencies have also been effective in providing tangible assistance to people who require it which is a very positive outcome.

Details of approximate costs per beneficiary for each agency and each response are presented in annex 7. While there are risks inherent in dissecting these figures in isolation of the operational context which has an important bearing on costs there are some trends which are worthy of comment and analysis; the three responses where the costs per beneficiary are greatest are those where considerable staff costs were included in the budget; the three interventions with the lowest cost per beneficiary were in the health sector; agencies that target specific parts of a community (e.g. the elderly) are likely to have greater costs per beneficiary. While efficiency is one of the criteria on which ERF funding decisions are based it may be helpful to explore the issue of costs per beneficiary more closely as part of this.

ERF Monitoring and EvaluationThe ERF currently has two different monitoring and evaluation systems; a Real Time Evaluation (RTE) which should be conducted within 30 days of the disbursement of funds14 and the ERF Grant Reporting Form which is required 90 days after the funds have been allocated. The former is designed to foment real time learning about the first 30 days of the response, providing a platform for improved longer-term programming. The Grant Reporting Form is designed to report against the original goals and activities set out in the ERF application. Together the two represent a rich source of information and provide an extensive evidence base about the use of the funds and the effects they have had on the project participants.

While the reports certainly provide important information, agencies which have produced both a monitoring report and an RTE spoke of the need to achieve a balance between ensuring accountability and achieving the broader goal of facilitating rapid response15. Concern was also raised about the risk of duplication both internally (between the report form and the RTE) and externally (with other donors reporting requirements). The implementation of the RTE in par-ticular is cumbersome and in large and/or fast-moving emergencies the date that the report is due may coincide with programmes reaching critical mass, a time where agency resources are often stretched to the limits. The limited size of the Programme Management Unit (PMU) and the heavy workload of the team also place considerable limitations on the capacity of the CBHA to process the data and there is a risk of information overload. The ERF learning review high-lighted the lack of familiarity of staff with the concept of RTE. There is also a concern that the RTEs may provide false positives as they are conducted as an internal exercise (and possibly by the same team members who are providing the assistance) and the risk of bias from such a methodology should not be ruled out. These issues are summarised well by one agencys RTE submission.

It has been fed back that training needs to be carried out around how best to use this tool [RTE] so it doesnt become another form to be filled out for HQ; more staff need to understand the purpose and potential benefits of this format. It was felt that this tool could be more useful, particularly for bringing about more immediate alterations to projects, if staff were able to engage with the purpose of RTEs at an earlier stage.

The use of the DAC criteria16 to underpin the ERF reporting framework may also be inefficient. While it makes for a comprehensive report some of the criteria may not be the most appropriate for a response fund of such short dura-tion and theres also a strong likelihood that field staff may lack familiarity with some of the criteria which may limit the usefulness of the data17.

The RTE was initially a compulsory reporting requirement but a decision was subsequently made to make them optional while the format was being formalised.14. A review of agency RTE submissions shows the duration of the exercise has been between 4- and 20-days.15. The DAC definition of evaluation is, an assessment, as systematic and objective as possible of an on-going or completed project, programme or policy, its 16. design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, developmental efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision-making process of both recipients and donors. (OECD (1991), Principles for Evaluation of Development Assistance Paris).See Active Learning Network on Accountability and Performance in Humanitarian Action (2006) Evaluating Humanitarian Action Using the OECD-DAC Criteria 17. An ALNAP Guide for Humanitarian Agencies, ODI, London.

11


2.3 assessing the contribution Made to strengthening humanitarian response capacity

A Significant Contribution to Timely Humanitarian ResponseThe CBHA has collected a wealth of monitoring and evaluative data on the funding mechanism and the projects which have been funded and a review of this material provides evidence of the contribution which the ERF has made to strengthening response capacity. Without exception, all of the review forms submitted by CBH agencies reported a positive impact of the ERF funds. Of the 16 ERF reports reviewed, there is a high degree of correlation in the feedback provided18. The feedback that is reported most frequently is the impact of the funds on the timely provision of relief; the existence of the ERF and the speed with which decisions were made allowed agencies to make swift commitments to establish operations and provide relief. Agencies with limited resources (financial and surge capacity) were often almost completely reliant on the ERF to launch a rapid humanitarian response which underlines the value of the fund.

Merlin is a medium-sized NGO, and as such does not have large amounts of unrestricted funds. Therefore, the availability of immediate funding in the early days of the response enabled initial activities to take place, that wouldnt have been possible without this support. MERLIN, Pakistan I ERF report form

Knowing this funding was in the pipeline enabled CAFOD to be decisive in the nature and scale of our response, ensuring an immediate response in the very early stages of the flood disaster. CAFOD, Pakistan I ERF report Form

For many Consortium members the timely establishment of agency presence was considered to be critical to their longevity as it permitted staff to initiate in-country fundraising activities. While this is not a specific objective of the ERF its impact is significant as it suggests that the ERF can contribute to the establishment of longer-term emergency operations.

Importance was also attached to the protection benefits that the early arrival of an NGO presence has on communities affect-ed by conflict and the impact that the timely arrival of specialist NGOs can have on specific vulnerable groups. With the cluster system of coordination now firmly established and often quickly activated after a crisis, early engagement of CBH agencies in clusters can potentially have a multiplier-affect as HelpAge International reported in its Kyrgyzstan ERF report form;

It is important also to mention that the presence of international actors immediately on the field also had a calm-ing and reassuring effect on the population as a whole and the presence of HelpAge immediately at the cluster group meetings ensured that the particular needs of older people were highlighted with other actors.HelpAge International, Kyrgyzstan ERF report form

Despite criticism of the transaction costs required for ERF activation a number of agencies provided positive feedback on the added value of the teleconferences which informed agency analysis and interventions and were felt to assist in avoiding duplication in the field;

...the coordination in the UK was (and continues to be) very helpful in avoiding overlap amongst at least all the CBHA members (collectively making a significant contribution to the overall response. Islamic Relief Worldwide, Pakistan I ERF report form

While there is no doubt that the ERF has facilitated timely response there was insufficient time during the review to ex-plore how each agency would have responded had CBHA funds not existed the baseline suggests that for those who have their own resources and for whom the ERF provides a complimentary funding stream rather than constituting the primary one, a response would have been launched irrespective of ERF funding. It is noteworthy that in two instances the existence of DFIDs WAHRF has played a part in the decision to not allocate ERF funds (for flooding in Niger and for the Liberia/Ivory Coast refugee influx) yet ERF funds were allocated to Pakistan a day before the launch of the UK DEC appeal which has considerable overlap in agency membership as the CBHA. Such issues will be important to explore in more detail during the programme evaluation.

But these are small criticisms and for a mechanism that has been established for such a short time the achievements of the ERF are certainly significant and in testing the ERF mechanism the CBHA has gone some way to demonstrating the important multiplier effect that swift access to funds can achieve.

18. Each of the 16 comments on the impact of the CBHA on humanitarian response are available in Annex 6.


12

Emerging Impact on People Affected by Crisis19

RTEs were conducted for the responses in Kyrgyzstan and Pakistan (I &II) and provide extensive evidence about the achievements of the interventions and the perceived impact they had on people affected by the crisis. A review of the documents suggests that the humanitarian response activities of CBHA agencies have provided appropriate and nec-essary assistance to communities in a timely manner. The quote from an RTE submitted by World Vision in support of its Pakistan I ERF allocation is indicative of many of the submissions reviewed;

Virtually all community members involved in focus groups (approximately 90%) considered the assistance re-ceived as relevant. Men and women without distinction expressed that the relief aid delivered had helped them address their immediate needs in a critical time when their houses and other assets had been either washed away or damaged by the floods. Respondents confirmed that clean drinking water, shelter, food and essential NFIs were among their most urgent requirements. The general perception was that the assistance delivered to the affectees was appropriate help to the right people at right time.

Each of the RTEs provided positive feedback from project participants on the quality of the items which were reported as either of sufficient quality or of a higher quality than those that they were replacing and as a result expectations were most often met or exceeded by CBHA agencies who undertook an RTE. There are no reports of items that were inappropriate or that were of poor or inferior quality.

Overall there is a very positive response to the identification of beneficiaries which suggests that strong processes of beneficiary selection were the norm. Where time permitted most agencies adopted a participatory approach and sought to work with members of the community (activists, leaders & women) to identify vulnerable groups to target assistance. In a few cases it was reported that specific sections of the community had not received prior information about the distribution (women in Pakistan, villagers in Kyrgyzstan) but the RTEs suggested that despite this they were incorporated into distributions and received relief items.

In the case of Pakistan the comparatively small size of the ERF grants made in the context of such a considerable humanitarian catastrophe meant that coverage was generally low although most agencies tended to focus on specific areas which meant that a reasonable local coverage was attained.

While the involvement of the community and sound needs assessment strengthened the sustainability of interventions, that the assistance was provided early on after the crises, was time-bound and was limited in its scope due to the comparatively small budget available limited the long-term sustainability of the programme as many agencies reported in the RTE. There is a justifiable challenge here about whether a 30-day funding facility should seek agencies to report against the sustainability criterion. That said several agencies used their own resources to follow-up ERF-funded inter-ventions with longer-term support which considerably strengthened the sustainability of their programmes.

The response in Kyrgyzstan provides an interesting insight into the very limited expectations that communities had of receiving assistance and consequently the considerable value they placed on items that they received. This tends to add weight to the case for ensuring the use of the ERF in lower-profile emergencies where agencies stand less chance of receiving funding and those in need of assistance stand less chance of obtaining it. In such contexts the added value of an ERF allocation could be amplified.

Results from the Kyrgyzstan RTE also add weight to the impact that agency presence has on the safety and security of communities at risk of violence with ACFs RTE commenting on the importance which people attached to their presence, which some rated as more significant than the material assistance they received. The importance of this should not be underestimated particularly given that such outcomes are often omitted from traditional methods of reporting.

Each of the RTEs reported a considerable number of follow-up actions although the majority didnt have a specific person assigned to undertake the tasks and the report allows no follow-up so there are no indications as to whether the issues were addressed by the respective agencies.

Beyond 30 Days: The Use of the ERF Model for ad hoc CollaborationWhile the CBHA has sought to achieve the outputs outlined in its logical framework it has also been opportunistic in how it has worked with the Consortium members to respond to humanitarian needs. Pakistan in particular has provided two opportunities to deviate from its established ways of working and explore alternative means of funding members to provide services to people affected by crisis.

As a result of the limited time available for the review, the significant volume of data that exists and the lack of prior analysis undertaken by the CBHA, the 19. review was restricted to a cursory analysis of the RTEs.

13


The first of these opportunities came in August 2010 when DFID provided the CBHA with an additional grant of 996,000 with a stipulation that each Consortium member who draws from the fund should match it from other fund-ing streams. Rather than provide a 30-day response facility the programmes were delivered over a 90-day period and reporting requirements were amended to reflect this. Apart from these changes, the peer review process for allocating funds and the reporting mechanisms (submission of a mandatory report and voluntary RTE) were applied as stipulated in the Programme handbook.

A more significant opportunity arose with the decision for a group of CBH agencies to participate in an 8-month DFID-funded Pakistan recovery response. This response saw 6 agencies (ACF, Care, Concern, IRC, Oxfam, and SC) apply as an ad hoc Consortium for a 21m grant. This programme emerged at DFIDs request as a new way for promoting wide-scale recovery in the rural economy of three flood-affected provinces of Pakistan. The PMU brokered the formation of this Consortium after the CBHA Board decided to pursue this unanticipated opportunity to significantly and appropriately assist Pakistans flood-affected population. This was an important initiative for the CBHA because of the large scale of humanitarian need in Pakistan and because it was an early experiment in front-line collaboration.

Two bodies of evidence will inform future Consortium strategy; the ability of the CBHA to deliver results at scale in the dynamic context of Pakistans flood recovery, and lessons from this process of context-specific collaboration. While its still too early to include a review of programme achievements there has been much discussion about the strategic implications of the use of the Consortium in this way and the process through which collaboration was brokered. That the CBHA mantle was super-imposed on top of an existing in-country NGO alliance which had al-ready commenced negotiations with DFID is a divisive issue. For many of the Consortium members who had already commenced negotiations with DFID the CBHAs intervention was considered an unnecessary duplication; from other members of the Consortium who couldnt join the alliance once the CBHA had gotten involved there were claims of partiality.

While the Pakistan recovery programme has the potential to sign-post an expanded role for the CBHA in the future the experience has also revealed a dissonance between those who view the CBHA as adding value in the UK and those who consider that it may have a wider, global remit. With the CBHA having entered its final year of funding it will be important for the PMU to progress discussions about its future role with an important emphasis on learning from the Pakistan venture.

2.4 emerging lessons and recommendations

Strengthening ERF Decision-making ProcessesSummary of Evidence & Emerging Lessons: The use of the CBHA by DFID to support humanitarian response reduces transaction costs for DFID, however collaboration is not cost-free and the costs saved by the donor are passed onto the CBHA and its members. Consortium members have raised concerns about the way decisions are made and the heavy transaction costs required by the process. The peer review method of allocating ERF funds enhances transpar-ency, drives up the quality of proposals and can also help to avoid duplication of programmes however the process of deciding when to call an alert and moving through the decision-making process is time-consuming.

recommendation 1: The ERF sub-committee should revise ERF procedures with the aim of reducing member trans-action costs and ensuring principled decision-making. In the first instance, decision-making must truly reflect the humanitarian principles that Consortium members espouse and these should form the basis of a framework that can be transparently applied throughout the decision-making process (the alert note should speak to these/the telecon agenda should reflect these/the amount of ERF funding allocated should be determined by these20).

recommendation 2: The development of such a framework will potentially strengthen trust in the process and may permit greater delegation of duties (proactive preparation of the alert, leadership of the teleconference, decision about the amount of funds to be allocated to the response) to the PMU which would reduce transaction costs for Consor-tium members and would significantly increase the efficiency of the process21.

recommendation 3: An analysis of the funding landscape for proposed CBHA responses should play a more sig-nificant role in decision-making processes during the alert phase. Efforts should be made to draw on the widest pool

While the development of the framework is beyond the scope of this review it is recommended that the following should comprise key reference documents; 20. The ECB Good Enough Guide (www.ecbproject.org/geg), the ECB Common Accountability Framework (www.alnap.org/pool/files/22_accountability_frame-work.pdf) and the Good Humanitarian Donorship Principles (http://www.goodhumanitariandonorship.org/gns/ principles-good-practice-ghd/overview.aspx).This recommendation will be dependent on there being sufficient capacity within PMU.21.


14

of donors for which information is available including DEC, UN CERF, DEC WAHRF and other rapid response funding channels. There is also a strong argument for members disclosing their own unrestricted funding for rapid response in order for the ERF to more objectively quantify the added value to its members.

Testing the Full Mandate of the ERFsummary of evidence & emerging lessons: The CBHA has facilitated timely and effective first phase response to rapid onset crises through the 30-day ERF funding mechanism in Kyrgyzstan, Myanmar, Pakistan and Sri Lanka. De-spite the predilection for supporting responses to new emergencies (particularly natural disasters), the ERF also has a role to play in responding to complex emergencies, slow onset crises and humanitarian spikes in chronic crises and it will only be through establishing a body of evidence that the pilot will be best placed to demonstrate its full potential and to meet the expectations of Consortium members. There has been considerable advocacy on the need to ensure chronic or neglected crises receive appropriate support from donors and there is an argument for the CBHA to seek an opportunity to test the relevance of the ERF to such contexts.

recommendation 4: The use of a framework to drive decision-making may result in ERF allocations being made to a more representative cross-section of humanitarian crises. However if this isnt the case there is an argument for the CBHA to seek to diversify the contexts within which ERF funds are allocated to provide the best foundation for learning from the pilot.

Beyond 30-days: Innovation in Humanitarian Collaborationsummary of evidence & emerging lessons: The CBH agencies have benefitted from the additional 1m match funding grant and have provided timely and effective assistance to flood-affected communities in Punjab and Sindh Provinces of Pakistan. The 20 million Pakistan recovery programme that followed was a pragmatic and innovative response to the overwhelming needs of those affected by the floods and has provided rich lessons. While it was procedurally complex to oversee and was divisive within the Consortium the programme that has resulted has the potential to provide a highly appropriate response and may also provide an important insight into the future role of the CBHA beyond its current funding horizon. Key questions remain however about the legitimacy of the CBHA outside of its UK base and there is work to do to galvanise the full support of its members if similar responses are to be made in the future.

recommendation 5: While there is an appetite from many within the CBHA to expand the role of the Consortium beyond its UK base this view is not universally held. If formal field-based CBHA collaboration is shown to have added value22 and there is a willingness to explore such opportunities in the future then it will be important to learn the les-sons of Pakistan and to facilitate a discussion between the members about how future collaboration can best be managed.

Achieving a Balance Between Collecting Evidence and Facilitating Actionsummary of evidence & emerging lessons: The reporting framework which underpins the ERF has generated a rich collection of qualitative and quantitative data which is highly relevant to decision-making about the future of the CBHA but the reporting burden is significant and the limited capacity the PMU has to analyse the data challenges the utility of the exercise. In particular the use of the DAC criteria as a guide for reporting templates may not be the most appropriate framework for the task and may risk compromising the data collected.

recommendation 6: The lack of familiarity of some field staff with the concept and practice of RTE, the limitations of agency staff evaluating their own work, the significant work involved in conducting an RTE and the limited capac-ity that the CBHA has to process the results all suggest that the costs of conducting such exercises outweigh the benefits. It is recommended that this methodology is used only where there is a clear learning objective (for particular types of response or innovative collaborations).

recommendation 7: In order to best articulate the achievements and learn from the responses of the ERF there would be considerable benefit in identifying a core set of questions which draws directly from the objectives of the ERF (timeliness and effectiveness) and provides evidence to support the purpose of the CBHA (appropriateness of the ERF, added value of the ERF) and in designing a single reporting format which is framed around these.

The Pakistan recovery programme is yet to finish and an evaluation of the impact of the programme and effectiveness of the collaboration should play an 22. important part in decision-making about similar work in the future.

15


Objective: The aim of the capacity building objective is to increase the numbers and skills of leaders in the sector by developing a competency framework and delivering it through national and international programmes. It is planned to achieve this through a collaborative approach to ensure economies of scale and greater coherence between actors using different response models. The CBHA aims to build on existing work through flexible participation of its member-ship in a suite of activities grouped under two streams which together reflect the approach of most agencies towards building the capacity of staff and other key actors.

3.1 reviewing the baseline and Identifying gaps in agency humanitarian response capacityThe baseline gives a mixed picture when it comes to humanitarian recruitment practices. While 11 of the 15 member agencies used competency frameworks for various HR processes including their interviews, training and staff devel-opment practices, there is not a uniform approach between agencies and practices differed greatly from agency to agency. There is also significant variation in the approach that agencies take towards humanitarian staff development and the numbers who benefit from such training. Competency frameworks and staff development strategies within CBH agencies have tended to develop over time, often in an ad hoc fashion and there was broad interest in the con-cept of a shared humanitarian competency framework which agencies could jointly draw on and which could galvanise stronger collaboration in the sector.

Agreed competencies between agencies could lead to improved standards and coherence of programmes. CBHA Baseline document, January 2011

While the baseline indicated that there was broad consensus about the value of a shared approach, it also highlighted a number of concerns which included the need for a professional approach (rather than it being an HR exercise) and a coherent output which would adequately cover all aspects of humanitarian work.

The dearth of talented humanitarian leadership has long been acknowledged across the sector and was highlighted in the ALNAP-authored document, The State of the Humanitarian System, which expressed considerable concern about the strength of leadership and coordination in emergency responses.

Of all its challenges, international humanitarian action was seen to suffer most from lack of effective leadership and coordination, according to its constituents responses, which were consistent across regions and agency affiliations.23

While some steps are now being taken to address this gap24 these are in their infancy and are yet to have significant impact. Agencies within the CBHA have embarked on a range of individual efforts to recruit, develop and retain leaders but to date most of these have been in-house initiatives. A joint effort between the 15 CBH agencies was considered to have both internal value and would potentially provide a resource to the sector more broadly.

In addition to a dearth of leaders there have also been long-held concerns about a lack of capacity within the humani-tarian sector more broadly, a situation which ALNAP et al articulate well and which is echoed by the findings of the ECBs25 National Staff Development Programme (NSDP);

Humanitarian assistance remains one which is characterised often as pre-professional, uneven in management competency, inefficient, inequitable, and often at odds with long term recovery and development.26

ECB research concluded that the pool of qualified and experienced international personnel is always smaller than the level of demand from humanitarian organizations, especially at the time of an emergency response.27

Active Learning Network on Accountability and Performance (ALNAP) (2009) The State of the Humanitarian System: Assessing Performance and progress - A 23. Pilot Study, Overseas Development Institute.Most notably through the establishment of a joint initiative between ALNAP, the Disaster Resilience Leadership Academy of Tulane University, the Humanitar-24. ian Futures Programme, and People in Aid to systematically explore and improve leadership within the humanitarian sector. More on this joint initiative can be found at http://www.alnap.org/initiatives/current/leadership.aspx;.The Emergency Capacity Building Project (ECB) is a consultative membership of seven large humanitarian agencies. The goal of the ECB Project is to improve the 25. speed, quality, and effectiveness of the humanitarian community in saving lives, improving the welfare, and protecting the rights of people in emergency situations.Active Learning Network on Accountability and Performance in Humanitarian Action (ALNAP), Humanitarian Futures Programme (HFP), Disaster Management 26. Leadership Academy (DMLA) (2010a) Humanitarian and Disaster Management Leadership: A Collaborative Research and Development Initiative.Damerell J (2008) ECB National Staff Development Programme (NSDP), External Strategic Review, People In Aid. 27.

IncreasIng the nuMber oF coMpetent natIonal and 3. InternatIonal Managers and leaders


16

The inherent unpredictability of emergencies limits the practicality and cost-effectiveness of recruiting and retaining sufficient numbers of operational staff with emergency management and leadership capabilities. This, together with the limitations on the timely employment of staff with relevant skillsets at times of rapid humanitarian scale-up presents a concern for both national and international NGOs. The baseline correctly identifies that NGOs have a variety of meth-odologies to develop staff but it is the lack of a consistent approach to scale which not only undermines the quality of staff development programmes but is also inefficient.

3.2 achievements Made against the logical Framework

Objective 1: Development of Shared Competency & Leadership Behaviour FrameworksUnder the leadership of ActionAid a common humanitarian and leadership competency framework was finalised in August 2010 and signed-off by all 15 CBHA agencies. The process involved an initial review of current research and competency frameworks across the sector and was developed through a series of workshops where CBHA staff and external resource people came together to seek consensus on the humanitarian core competencies.

This process was highly collaborative commencing with a review of competency frameworks by People in Aid (PiA) who were commissioned to map out the existing frameworks from the CBH agencies, other NGOs, the UN and review initial research conducted by the Enhanced Learning & Research for Humanitarian Agencies (ELRHA) initiative. The relevant commonalities were identified and drawn together for the proposed framework and an initial draft document was produced and feedback was elicited from the Consortium members. Two regional workshops were held in Nairobi and London to bring together key personnel from each agency to develop an agreed set of core humanitarian compe-tency and leadership behaviours. These were written up in a final report which was then adopted by the CBHA Board in early August.

As part of the objective PiA has conducted a mapping study of people development programmes and learning devel-opment courses being used by CBH agencies. These were then matched against the competency framework as part of the process of developing an overarching curriculum for the CBHA capacity building programme which provided the foundations for objective 2 and 3.

Objective 2: Development of Humanitarian Skills and Leadership Competencies of Existing Staff at National & Re-gional LevelThe aim of the second objective under the capacity building output is to address the skills gap in the sector by develop-ing two generic staff development programmes focusing on core humanitarian skills and leadership and management skills. It anticipates that these generic programmes can then be adapted to different countries and staff as appropriate across the sector.

The motivation for developing the programme came from an acknowledgement that it is most often national staff who takes on the greatest responsibility for delivering humanitarian response yet at the same time it is this same group who have least access to acquiring the skills and knowledge necessary to perform these roles. Two learning programmes have been developed which aim to deliver core skills through a mix of training styles (coaching, action learning, self-learning, group work, and practical course work) in humanitarian core competencies and humanitarian management and leadership. It is anticipated that once learning materials have been developed they will be piloted in a number of different contexts before being translated and being made available as a resource for the wider humanitarian community.

A rapid staff capacity assessment was conducted in four locations comprising Bangladesh (5-8 December 2010), In-donesia (2-5 November 2010), Bolivia (11-15 October 2010) and the Horn of Africa (13-30 September 2010) to identify staff for the proposed staff development programme and to generate feedback about different learning approaches. The assessment was benchmarked against the competency framework developed in objective 1 of the capacity build-ing objective and provided valuable input into the design and contextualisation of the curriculum content for the activities to follow. It also affirmed the relevance of the work achieved to date and built momentum for the pilot staff development programmes scheduled for 2011.

Concurrently with the assessments, and drawing on feedback from the events, work was started on developing the learning tools with significant progress being achieved towards two staff development initiatives; the Humanitarian Staff Development Project and the Humanitarian Management and Leadership programme. To date PiA has been com-missioned to oversee the development of course material and facilitator and participant resources are in the process of being developed for both the programmes. The aim is to have 20 people participating in each of two programmes in four countries making a total of 160 participants.

17


Objective 3: The Humanitarian Leadership Development Programme Trainee Scheme (HLDP)The aim of the HLDP is to bring new leadership talent into the humanitarian sector and is aimed at those who are new to humanitarian work (although they may have worked in CBH agencies or other organisations in non-humanitarian roles) or are new to leadership roles. It seeks to provide the framework and financial resources to enable CBH agencies to provide a targeted development opportunity for their most talented staff with the greatest leadership potential. The HLDP is based around two programme streams; international (with London as the hub) and national/regional (with Nairobi as the hub)28. Both streams are based on the same operational model but will focus on different geographical areas. The goal of the objective is to drive up minimum standards of staff leadership development across the sector and allow the CBH agencies to tailor courses to their own specific needs with a strong focus on experiential learning and coaching.

A management team has been recruited to lead on programme delivery and are based in London and Nairobi; learn-ing tools have been developed which incorporate the humanitarian core competencies and are delivered through an Emergencies Foundation Course; and a rigorous system of self-assessment has been designed to provide feedback on the progress made towards achieving the core competencies by course participants. While the model is relatively expensive (estimated to cost 15,000 per participant for a total of 45-particpants which includes the costs of the man-agement team but doesnt include costs incurred by the hosting agency), much of the budget goes towards paying for the training events and while alternative approaches which focus purely on internships were considered they were rejected as they didnt offer the same opportunity to provide targeted skills development within each participant. The commitment to delivering a high-quality training product and the ambition to seek accreditation for the HLDP neces-sitated that a professional approach be taken and provided justification for the cost.

The first of three cycles of training has commenced in Nairobi, with 15 trainees having been selected from over 2,200 applications. All 15 were accepted and are now based at 11 of the 15 CBH agencies across four countries in the Horn of Africa (Kenya, Somalia, Sudan and Ethiopia). Each of the trainees has now been integrated into their host agency and has completed work plans to guide their activities. 15 coaches have received 2-days training during a Nairobi coach-ing workshop and have already completed 2-3 sessions with each of their trainees. At time of writing, preparations are underway for the Advanced Field Training which is scheduled to start in early February.

Preparations have also started for the international cycle of the HLDP with recruitments already underway with nine agencies having agreed to play a hosting role.

3.3 reflections on the process, emerging lessons & early Impact With the staff development programmes under objective 2 still to commence and the HLDP under objective 3 being in process there is very limited evidence available to assess the contribution of the CBHAs capacity building objective to strengthening capacity in the sector and while the potential benefit is great it largely remains yet to be proven. For this reason this section will seek to reflect on the process in order to contribute to learning. Where there is evidence of early impact this will be highlighted.

Early Adoption of the Humanitarian Competency FrameworkWith the competency framework having been successfully completed and signed-off in mid-2010 there have been a number of agencies that have applied it to specific parts of their work in addition to it providing an important foundation for the other two capacity building components:

IRW has used the framework for the recruitment and selection of its emergency roster which has been developed by the CBHA surge capacity-funded post. To date 36 people have been selected for the global and regional roster. Beyond its use for recruitment the humanitarian competencies framework has also provided the foundation for IRWs training programme for its roster staff. The week-long training course is mandatory for all members and uses the competencies for the development of leadership skills and team building exercises.

Care International has used the framework to design a new training course on humanitarian operations aimed at emergency coordinators and other deployable staff. It has been used to develop a self-assessment tool for partici-pants and a manager feedback tool which will provide the basis for the development of individual learning plans.

RedR UK has also taken steps to incorporate the competency framework into its training programme and has recently aligned its courses in Managing People and Projects and Security Management and a credit-rated (but not compe-tency-based) course in the Essentials of Humanitarian Practice with it. They will also shortly be launching a Diploma in Humanitarian Practice which will comprise six courses that they also aim to align to the CBHA competencies.

The focus on the Horn of Africa was to facilitate linkages with objective 2 of the Capacity Building objective. 28.


18

Building on the Baseline: Developing a Robust Learning & Evaluation FrameworkWhile the relatively recent commencement of the 1st cycle of the HLDP means that it is too early for evidence to have been documented the process of participant assessment and programme evaluation is noteworthy as it potentially provides a com-prehensive data set for evaluation. At the start of the programme each participant must complete a self-assessment against the humanitarian core competencies and then on four occasions throughout the life of the 12-month cycle these are revisited and updated. As well as providing important data about participant progress from which broad evidence about the success of the programme can be extrapolated the data may also be used to monitor host agency implementation and in so doing can perform a quality control function which will be essential given the diversity of CBHA agencies and the potential ambition to seek accreditation for the programme once it has been tested.

An overview of the strategies for learning from the second objective has also been prepared which outlines a comprehensive approach which includes baseline monitoring prior to candidates participation in the programme (proposed techniques include a baseline assessment, self-assessment questionnaire and a learning development plan) and assessment throughout imple-mentation (proposed techniques include the use of learning logs, coaching, observation notes, peer learning and feedback) which will provide a rich evidence-base.

recommendation 8: The support provided to two different but complimentary staff development programmes and the com-prehensive approach adopted by each to monitoring throughout the programme life-cycle provides an important opportunity for the CBHA to conduct a comparative analysis of the approaches.

The Benefits of Collaboration: Using Agency Experience to Strengthen Collective PracticeA particular strength of the CBHA is the opportunity that exists for each member to benefit from a large knowledge pool and this has been used to good effect in designing the capacity building objective:

The development of the humanitarian competency framework involved considerable participation from across all 15-agen-cies and many others external to the CBHA and the final product was signed-off by all members. It built important owner-ship across a significant platform of agencies and as such there is the potential for considerable uptake across the sector, which is already beginning to be realised.

The skills development component has similarly built on an existing knowledge-base and has benefitted from cross-agency par-ticipation from Consortium members. The programme has its roots in the successful ECB NSDP29, the strategic external review for which considered that it was breaking the mould on interagency collaboration and co-operation and considered the pilot programme invaluable both for those who participated and those who implemented the programme. The CBHA simulation conducted in advance of the programme found that the majority of the participants were either very ready or somewhat ready to prepare and respond to an emergency situation which provides some insight into the potential value of the programme.

The first, Nairobi-based cycle of the HLDP has seen 11 of the 15 agencies offer to host trainees with the second, UK-based cycle likely to see a similar number of hosts. In addition to the depth of experience this has bought to the HLDP the use of facilitators from across the 15 agencies has bought a level of diversity which has contributed significantly to the approach-es adopted. The training was adapted from a successful Save the Children apprenticeship and training programme30.

Recognising the Potential Value of Staff InteractionsAnecdotal evidence from the ECBs NSDP suggests that it had important unanticipated impact which is of significant relevance to the CBHAs own capacity building activities. While the learning has not been formally documented it was found that the inter-personal and inter-agency links made through participation in the programme sustained themselves long after the programme had ended and had an important impact during response to humanitarian crises where many of the participants found themselves working together again. Participation in the staff development workshops provided a double impact building knowledge and competencies but also strengthening trust between people and organisations. While personality-based humanitarian system co-ordination can be a cause for concern, where such networks have a basis on professional learning there is significant value in exploring the benefits of these. Although evidence of this remains anecdotal it was highlighted as a lesson from the Bangladesh and Horn of Africa simulation31 and certainly provides an interesting hypothesis for the CBHA to explore during implementation.

recommendation 9: An unintended and largely undocumented impact of capacity building programmes has been the trust that has developed between humanitarian staff which has had a positive impact on interagency collaboration. It is recom-mended that the CBHA exploit this opportunity and develop relevant methodologies to evaluate this.

The baseline simulation summary findings document reported that establishing networks and relationships prior to deploying in emergency situations would 29. save time during real events and enable quicker decision-making, as a key lesson from the simulation exercises.Relevant ECB materials on the NSDP are available at 30. http://www.ecbproject.org/resources/staff. The specific objective of the NSDP pilot project was to devise a replicable model which could increase the number of agency staff at the national level capable of leading and managing emergency programs. See Damerell J (2008) ECB National Staff Development Programme (NSDP), External Strategic Review, People In Aid.The HLDP was borne out of Save the Childrens successful implementation of their Humanitarian Skills Development programme. Evidence from the imple-31. mentation of SCs Child Protection Trainee Scheme also contributed to the development of the programme.

19


Objective: The member agencies of the CBHA have a diverse approach to managing their surge capacity which in-cludes the use of standing staff, internal deployments, staff partner and training and external rosters. Whilst the ERF builds on existing abilities to respond it also provides the Consortium members with an increased opportunity to do so which places demands on existing surge capacity. Member agencies of CBHA are not all equally able to support and sustain the increased and improved level of rapid response and high level interagency co-ordination that access to the ERF will permit. Even those that already have good capacity will benefit from an increased capacity to respond at a time when demands are growing and public fundraising is unstable.

4.1 reviewing the baseline and Identifying gaps in agency humanitarian response capacitySurge capacity in the humanitarian sector may be defined as

the ability of an agency to scale-up quickly and effectively to meet increased demand to stabilise or alleviate suffering in any given population.31

As the definition suggests, surge capacity is far more than a human resource function and Houghton and Emmens (2007) describe a framework of inter-related components which include resources (human, financial, materials and equipment), mechanisms (planning and resourcing), and enabling factors (preparedness, flexibility, durability, sustain-ability) which to work effectively must be underpinned by an agencies philosophy, mandate and structure. The focus of the CBHA output is on standing capacity rather than on the wider framework described above.

While the Issue of surge capacity isnt dealt with in depth by the initial baseline report it does provide a snapshot of agency preparedness and provides evidence of existing surge mechanisms across the majority of CBH agencies. It also notes that finding staff with adequate technical and language skills is considered to be an impediment to humani-tarian response which would lead one to place a value on core surge capacity although this link is not made in the document.

The baseline information provided by the 15-agency surge capacity proposals provides far greater detail about existing practice amongst Consortium members and highlights their diverse capacities. A review of these provides a cogent summary of the many impediments to adequately resourcing in-house surge capacity which include limited human and financial resources and the complexity of balancing conflicting organisational priorities. These are expressed well in an extract from an agency surge application.

Over the past 15 years [the organisation] has responded to many disasters but the timeliness, scale, and to some extent, quality has been inconsistent due to variable readiness, experience and capacity at field office level. Further, the timeliness and consistency of support to rapid onset emergencies by [the organisations] Disaster Management Unit is not assured due to conflicting work priorities and a limited deployment budget. There is a need across the network to increase understanding of disasters as well as enable [the organisations] country offices to establish and improve rapid response capability. The country offices are the key initiators of disaster response; their capability is built on their long term presence in the countries of operation.

Beyond the insights offered by the baseline there is evidence to suggest that first phase response is increasingly de-pendent on implementation and support from surge capacity staff for both operational and non-operational organisa-tions. All the CBH agencies have surge capacity in one form or another and a review of the development of this capacity shows that for many agencies there has been a steady growth in the size and scope of this. The larger, operational agencies in particular now have significant experience in strategically managing globally mobile surge staff.

That is not to say that surge capacity is a panacea and at best it is a short-term fix (the longer-term solution is in recruit-ing competent individuals and in-house training for core staff which is dealt with under the capacity building objec-tive). The roles are often demanding and consequently staff retention for some of the more demanding posts can be comparatively short which can be a drain on human resourcing and can also represent a significant loss of institutional memory as front-line surge staff can build up a lot of knowledge and experience very quickly.

Houghton R & Emmens B (2007) A Review of Surge Capacity and Surge Capacity Mechanisms within International NGOs, People in Aid. 31.

IncreasIng agency surge capacIty to respond 4. approprIately to neW eMergencIes


20

4.2 achievements Made against the logical FrameworkThe input of the CBHA took the form of a 110,000 grant for each of its members to use over a two-year period. At the most basic level the anticipation was that it would fund a rapid response post for each agency for the period of the DFID grant. The fact that there was no stipulation made as to how to use the funds has led to a refreshing diversity of approaches.

All 15-agencies submitted proposals for the grant. A review of the proposals shows the breadth of the surge capacity function within CBH agencies and a range of different approaches used to strengthen humanitarian response capacity (outlined in annex 10). Where agencies had existing surge personnel the funds have been used to compliment this, often to fill specific gaps (such as in shelter or food security). Four agencies highlighted constraints in their ability to recruit or retain emergency pool teams and are consequently using the funds to assist them to reach planned surge capacity levels; other agencies have invested the funds in UK-based posts which are either internationally mobile or which will be used to coordinate the work of surge teams or (in one case) to develop organisational policy on surge capacity. Training was also a common theme across the proposals particularly (although not exclusively) for agencies who were non- or semi-operational and who delivered much of their humanitarian assistance through partners.

Results submitted by CBH agencies during the review indicate that considerable progress has been made towards the achievement of agency objectives (see annex 9 for a summary of results). Of the report forms reviewed all of the agen-cies had successfully recruited the additional posts funded by the surge grant and where a training objective had been funded, most agencies reported that initial training outputs have been achieved. This in itself is an impressive result and suggests that the application process benefited from a strong internal process of problem identification and there has been a demonstrable commitment to maximising the added value of the funds throughout the year.

The agency report forms also identified a number of complementarities between the surge capacity objective and the other objectives of the CBHA. In particular there has been a strong link between the surge posts and the delivery of ERF programmes which was anticipated at project identification but there have also been other complementarities; in at least one case (IRW) the competency framework developed under the capacity building objective was used in the recruitment of the surge capacity post. It is this cross-fertilisation between CBHA outputs that adds significant value at the organisational level.

4.3 assessing the contribution Made towards strengthening humanitarian response capacityEvidence from the first year of implementation suggests the grants have made an important contribution to humanitar-ian response and capacity building at both a strategic level within agencies and at a field level in a range of humanitar-ian responses. With the posts now fully integrated into members organisations and activities well underway there is great potential for their impact to be significantly increased in the second year of implementatio

Documents

Consortium of British Humanitarian Agencies (CBHA) mid-term review