FUND FOR EVALUATION IN YOUTH EMPLOYMENT:

Building the Evidence Base for What Works

Information Note + Call for Applications

DEADLINE: 15 August 2012

Application Form available here

P

ho

to S

ve

n T

orf

inn

2

ABOUT YEN

YEN (Youth Employment Network) is an interagency partnership of the UN, ILO and the World

Bank set up after the Millennium Summit in 2001 to find new and durable solutions to the

youth employment challenge. The network is a global platform whose goal is to prioritize

youth employment on the development agenda and to exchange knowledge on effective

policies and programmes to improve employment opportunities for youth. A major

concentration of YEN’s work is supporting rigorous evaluations of innovative youth

employment programmes.

ABOUT YEF

In 2010, YEN initiated a partnership with the ILO to strengthen the promotion of youth

entrepreneurship in East Africa. Sponsored by the Danish-led Africa Commission, the Youth

Entrepreneurship Facility (YEF) is a comprehensive 5-year initiative to promote

entrepreneurship as well as jobs and business creation among youth. YEF advocates for better

knowledge on what works - and why - to improve labour market outcomes of young

entrepreneurs through evaluation clinics and technical and financial support for evaluations.

BACKGROUND

There is a severe lack of rigorous evaluations of youth employment programmes, especially in

developing countries, which has led to a large evidence gap of not only “what works” in the

delivery of youth employment programming but more precisely “how and why it is working”.

An analysis of the Youth Employment Inventory reveals that only 23% of youth employment

programmes show evidence of net impact. Improving the evidence base for designing and

implementing youth employment schemes will enhance delivery effectiveness while

producing reliable information for policy makers allowing them to make informed decisions

on resource allocation and programming priorities.

In 2009, YEN organized two learning events called “YEN Evaluation Clinics”. The objective of

these Clinics was to teach youth employment practitioners the basic techniques for

conducting rigorous impact evaluations and recommend strategies to improve and strengthen

3

evaluation plans. The Clinics revealed that three major bottlenecks restrict the growth of the

evidence base for youth employment:

1) The limited knowledge and skills among youth employment practitioners and policy

makers to allow them to initiate and undertake impact evaluations, a limitation that

starts with insufficient monitoring of existing programmes.

2) The insufficient production of impact evaluations which limits recognition on the

value of monitoring and evaluation (M&E), especially in developing countries.

3) The lack of communication and dissemination of new and existing evidence and

serious limitations on how to use evidence to improve and restructure programmes.

To address these challenges, YEN initiated “The Fund for Evaluation in Youth

Employment”.

The Fund started in 2010 in East Africa with support from the Africa Commission, the Jacobs

Foundation, and Swedish Sida. The Fund quickly expanded operations in the Middle East and

North Africa with additional support from Silatech, the World Bank, and the International

Initiative for Impact Evaluation (3ie). The design of the Fund relies heavily on experiences

accrued from the World Bank’s Strategic Impact Evaluation Fund (SIEF) and the 3ie.

THEMATIC SCOPE

The Fund’s fourth call for applications will concentrate on youth entrepreneurship

programmes in Sub-Saharan Africa. Recently, entrepreneurship programmes have been

receiving increasing attention from governments, donors and multilateral agencies as an

alternative job creation strategy. Given the limited absorptive capacities of existing formal

labour markets in the developing world, promotion of youth entrepreneurship and self-

employment is one of the few feasible options to create employment opportunities both in

the informal and formal economy. Nevertheless, the evidence to support positive impact of

entrepreneurship schemes is extremely weak.

Impact evaluations of these programs should aim at answering the following questions:

1. What are the most effective strategies or combination of strategies for improving

business development, income, and employability amongst current or promising young

entrepreneurs?

4

2. What are cost effective ways to deliver entrepreneurship interventions for youth in

Sub-Saharan Africa?

FUND DESIGN

Intervention strategy

The Fund for Evaluation in Youth Employment employs an

iterative strategy of LEARNING-APPLYING-SHARING. The three

interlinked but distinct components attempt to cover the

entire value chain of evidence-based advocacy: (1) building

the capacity of practitioners and policy makers to evaluate

their programmes, (2) commissioning a series of evaluations

which contribute to the evidence base, and (3) sharing results

to improve youth employment policies and programmes.

1 - Learning component: YEN Evaluation Clinic

“Evaluation Clinics” are intensive 3-4 day trainings that focus on knowledge sharing and

learning between youth employment practitioners with the goal of improving current

evaluation practices. The Clinics provide access to specialized knowledge and support via

hands-on consultations from evaluation experts while providing a platform to initiate and

improve evaluation designs. Learning is facilitated through the application of evaluation

concepts and techniques to live case studies. Live case studies are selected through an

application process and will be eligible to receive financial support in the form of small

grants from the Fund. Tangible outputs of the Clinics are draft evaluation concept notes for

each live case study.

In addition, youth employment policy makers and practitioners can improve their knowledge

on evaluations by participating in YEN’s online Evaluation Groupsite and periodic webinars

teaching key evaluation concepts.

2 - Application component: Technical and Financial support

Project teams that submit a sound evaluation plan following participation in the Evaluation

Clinic will be eligible to receive a small grant to cover part of the costs associated with

designing and conducting their evaluation. Conducting impact evaluations can be expensive

5

depending on the availability of evaluation expertize, the sample size and the type and

amount of data collection needed. The Fund’s contribution to the evaluation will range from

US$5,000 to US$80,000, which means projects will have to bring in co-funding or fundraise to

cover the evaluation full costs. While not strictly limited to it, seed funding received from

YEN should contribute to the data collection efforts and in some particular cases the funds

will support evaluation design or other implementation costs.

3 - Sharing component: Dissemination of results and lessons learned

Better evidence has limited practical value if it is not understood and used to improve policy

and programmes. Experience has proven that producing reliable evidence is only the first

step towards changing to a culture of evidence based decision making. The challenge will be

taking lessons learned from impact evaluations and communicating them in an effective,

timely and appropriate manner to practitioners and policy makers.

A major strength of YEN is its access to policymakers responsible for designing and delivering

national policy in youth employment. YEN’s Lead Country Network and the local public and

private partnerships created by YEF in Kenya, Uganda, and Tanzania will be an important

avenue to disseminate results of impact evaluations. YEN’s close partnership with the World

Bank will provide a platform for wide communication of evidence. In addition, policy makers

and practitioners will be reached web-based dissemination through a wide range of web-

based fora including YEN’s website, the global Youth Employment Inventory, the YEN

Evaluation Clinic Groupsite and the new YEN Marketplace.

SELECTION PROCESS

Timeline and specifications

� Launch of Request for Applications 9 July 2012 � Deadline to receive applications 15 August 2012 � Evaluation Clinic 24-28 September 2012 in Tanzania1

� All applications must be submitted using the online application form found at

http://fs17.formsite.com/youth/2012/index.html

1 The precise location still has to be defined.

6

� For questions related to the Fund and the application contact: Silvia Paruzzolo, Evaluation Specialist Youth Employment Network 4, route des Morillons CH-1211 Genève 22 Switzerland +41227997824 Yenetwork@ilo.org

Applicants will go through a two stage selection process:

• Stage 1 Shortlist of evaluation clinic participants:

Interested applicants will submit application forms which will provide details about the

evaluation plans, but also about the project, the organization and its M&E system and

capacity. Qualified applicants will be shortlisted by a selection committee formed by YEN and

other M&E experts. The shortlisted applicants will be invited for an interview with the YEN

team in order to be selected to participate in the next Evaluation Clinic. Winners will be

invited to participate as live case studies and receive hands-on advice and consultation on

the M&E system and impact evaluation design for their programs.

• Stage 2 Shortlist of evaluation plans:

Clinics will serve as a platform for evaluation experts to work with project teams to define

and/or improve evaluation plans. Within two months after participation in the Clinic,

evaluation teams will submit detailed concept notes on the evaluation and, where possible, a

cost-benefit analysis plan. Concept notes will be reviewed and rated to receive final grant

approval for those projects deemed ready for evaluation. Selected organizations will receive

funding between December 2012 and February 2013.

Eligibility

The following will be eligible to apply and participate in the selection process:

1. Project teams

Members of organizations responsible for the implementation of the project described in the

application will be eligible to apply and participate in the selection process.

7

2. Theme

Organizations implementing interventions in the youth entrepreneurship field whose

objectives involve creating self-employment opportunities, developing entrepreneurial skills

and competences, and increasing access to finance. Evaluations focusing on young women

and rural youth are encouraged.

3. Potential for knowledge generation

Successful applicants will be those proposing solutions to the youth entrepreneurship

challenge that can be evaluated and generate knowledge of what works or doesn’t work in

the field of youth entrepreneurship. In order for a project’s impact to be evaluated and have

potential to generate rigorous evidence it must be based on a clear implementation “model”

and the implementing organization must have an established way of operating plus enough

capacity and institutional buy-in to implement an evaluation plan. Therefore applicants must

demonstrate a clear project implementation approach and at least 2 year of experience in

project implementation. Organizations must also have an annual total operating budget of at

least USD$ 250,000.

4. Eligible Countries

Proposed projects to undergo the evaluation must be based in the following countries: Sub-

Saharan Africa: Angola, Benin, Botswana, Burkina Faso, Burundi, Cameroon, Cape Verde,

Central African Republic, Chad, Comoros, Congo, Côte d'Ivoire, Democratic Republic of the

Congo, Equatorial Guinea, Eritrea, Ethiopia, Gabon, Gambia, Ghana, Guinea, Guinea-Bissau,

Kenya, Lesotho, Liberia, Madagascar, Malawi, Mali, Mauritius, Mozambique, Namibia, Niger,

Nigeria, Rwanda, Sao Tome and Principe, Senegal, Seychelles, Sierra Leone, Somalia, South

Africa, Swaziland, Togo, Uganda, United Republic of Tanzania, Zambia and Zimbabwe.

5. Working Language

The working language will be English.

6. Institutions

Applications will be accepted from the following types of organizations, as long as they are

an implementing organization:

• NGOs / CSOs / CBOs

• Training organisations

• bilateral or multilateral agencies

8

• Public-private partnerships

• Social entreprises or social entrepreneurs

7. Exclusion

The Fund will not support the following types of organizations or interventions:

• Programs promoting religious or ideological doctrine

• Land, site acquisition or facilities construction projects

• Programs grant making to other organizations or individuals

• Individuals, either through scholarships or other forms of financial support

• Political campaigns

Selection criteria Priority will be given to applications that demonstrate the following characteristics:

1. Innovativeness of project

Organizations that propose new and creative solutions to the youth entrepreneurship

challenge will be given high marks. Innovative project ideas could include social enterprises

or market-based approaches to development, linking young entrepreneurs to value chains,

unique delivery models, environmental or technology based solutions and youth inclusive and

gender-responsive ideas. Extra weight will be given to applications that demonstrate ability

to reach disadvantaged youth populations, especially young women and rural youth.

2. Capacity and Investment in M&E

Successful applicants will demonstrate at minimum some level of institutional investment and

basic skills and experience in M&E and familiarity with key concepts (such as results chain/log

frameworks, indicators, qualitative and quantitative data collection methods). Experience

using different types of impact evaluation designs (experimental and non-experimental

evaluation) would be desirable. See Annex for a discussion of YEN’s evaluation principles.

3. Potential for knowledge generation and replication

The Fund seeks to identify organizations and projects with a strong commitment and the

potential to generate new evidence of what works in the field of youth entrepreneurship.

Successful applications will provide evidence of effective implementation and also potential

for replication of the project idea by the implementing organization itself or other

organizations in the field.

9

4. Quality of collaborating project team

The background, professional experience and education of the project team who will

implement the evaluation plan will be taken into consideration. Their demonstrated

motivation, dedication and interest in the topic of M&E will also be taken into consideration.

Benefits to selected organizations Shortlisted Applications

• Expenses paid for one participant in the Tanzania Evaluation Clinic in

September 2012

• Intensive training on M&E methodologies and techniques

• Assignment of an evaluation expert who will be focused on the organization’s

evaluation live case during the clinic

• Exposure to the other clinic participant’s experiences and lessons during the

work on their live cases

• Access to knowledge sharing portal and a youth entrepreneurship community

of practice

Shortlisted Evaluation Plans

• All of the above plus seed funding

10

APPLICATION FORM:

FUND FOR EVALUATION IN YOUTH EMPLOYMENT

ONLY ONLINE APPLICATIONS WILL BE ACCEPTED: http://fs17.formsite.com/youth/2012/index.html

Deadline: 15 August 2012

PERSONAL DETAIL

1. First Name

2. Last Name

3. Title

4. Organization

5. Address 1

6. Country

7. City

8. Postal Code

9. Business Phone

10. Mobile phone

11. Email Address

ORGANIZATION DETAIL

1. Legal Name

2. Address 1

3. City

4. Country

5. State

6. Postal Code

7. Business Phone

8. Email Address

9. Year founded

10. Organization's mission statement

11. Organization type

12. Leadership - provide the name, title and brief biography of the organization's top

leader

FINANCIAL AND OPERATIONAL INFORMATION

1. Operating budget (total) in 2011 in USD$

2. Three top donors –

What are the names of your 3 top current donors and what are their funding

commitments?

3. Largest grant –

11

Amount in USD$, number of years and name of donor

4. Organizational chart –

Upload an organizational chart which shows the structure of the organization and the

positions of the key staff

5. Location of program delivery –

Where does your organization operate its programs? Indicate towns/cities, regions and

countries

PROJECT INFORMATION

1. Project summary

Please provide a very short summary of your project (100 words). Include the

following information: objectives, services delivered, partner institutions, stage of

project and timeline.

2. What is the total budget for the project?

3. How many beneficiaries does the project serve or intends to serve over the life of

the project?

4. What is the primary age of the participants?

5. What is the primary gender of participants?

6. The Youth Employment

Challenge Describe the constraint to youth entrepreneurship that your project is

trying to address. What are the main reasons this problem exists? What are the issues

that contribute to the challenge of starting and expanding businesses for youth in your

area of operation?

7. The Innovation

Describe the approach your organization takes in promoting youth entrepreneurship.

What are the key innovative design features of your project How is your approach

different from that of other projects implemented in Sub-Saharan Africa? Why is it

better than other approaches in creating business opportunities for youth?

8. The Results

Describe the results that your project has already achieved. Are you able to provide

figures on project outcomes such as businesses supported, jobs created or incomes

increased? How were these outcomes measured?

9. The future

Describe your organization’s plan for scaling up or replicating your innovative

approach. How does being able to evaluate your program help you achieve your long

term goals for the project and the organization as a whole?

10. Example of your work

This space is provided for you to give a short story or personal case study which

exemplifies what you feel is your organizations’ impact on youth. Please provide as an

attachment which can be in the form of a DOC, PDF, video, audio or presentation.

11. Or submit a link to a youtube video, photos, powerpoint presentation, facebook or

linked-in group.

12

MONITORING AND EVALUATION (M&E) DETAILS

1. M&E system

Describe the M&E system currently used by your organization. What are the main M&E

methodologies used? What are the main output and outcome indicators? How do you

choose which indicators will be measured and how?

2. Impact Evaluation

Does your organization plan to run an impact evaluation in the future? Or have you run

one in the past? If so, please briefly describe the reasons for wanting to do an impact

evaluation and the methodology you would like to use.

3. Data collection

How and how often does your organization collect data?

4. Roles and responsibilities

Who is responsible for M&E in your organization? What are his/her main tasks or

duties? Do you partner with any external organization on M&E? If yes, who and for

which tasks?

5. Challenges

Describe the challenges you face in developing and using your M&E system. For

example, are you challenged by time or money resources?

ATTACHMENTS

1. Proof of legal status of the applicant organization.

2. Letter from the applicant organization’s executive director, CEO, or board,

expressing its commitment to the execution and participation in this process

3. The CVs of one or two key members of the M&E team, i.e. who would be tasked to

implement the M&E efforts developed at the clinic (these could be from the

applicant organization and/or the partner organization, if any, supporting the M&E

efforts)

4. The CVs of one or two key members of the M&E team, i.e. who would be tasked to

implement the M&E efforts developed at the clinic (these could be from the

applicant organization and/or the partner organization, if any, supporting the M&E

efforts)

13

ANNEX: YEN EVALUATION PRINCIPLES

Definitions:

Impact evaluation assesses the changes that can be attributed to a particular intervention,

such as a project, programme or policy. In contrast to outcome monitoring, which tacks and

examines whether targets have been achieved, an impact evaluation is structured to answer

the question: how would participants’ well-being have changed if the intervention had not

been undertaken? This involves a counterfactual analysis, that is, “a comparison between

what actually happened and what would have happened in the absence of the intervention.”

Cost benefit analysis involves weighing the total expected costs against the total expected

benefits of one or more actions in order to gauge the efficiency of the intervention.

Evaluation Design:

• A range of methodological options exist to build a counterfactual. These options include

both experimental and quasi experimental designs. Further information is provided at

http://go.worldbank.org/7M4NUSKE10.

• The most interesting impact evaluations provide both qualitative and quantitative

evidence (i.e. use a mixed method approach to data collection). Qualitative data

collection techniques include focus groups and key informant interviews.

• The basis of a quality impact evaluation revolves around four key inputs: (1) a logical

results chain that connects inputs to outcomes and specifies indicators for measurement,

(2) a clearly defined evaluation question which serves as the research hypothesis, (3) an

appropriate strategy for identifying the counterfactual and (4) appropriate techniques for

collecting robust data. The four key inputs will serve as the basis for the learning agenda

and the development of the evaluation plans during the Evaluation Clinics.

Costs

Experience suggests that the costs of conducting impact evaluations can be high. The initial

design and data collection make up for the lion share of an evaluation budget; data collection

costs can easily reach 80% of the total evaluation budget. The Fund for Evaluation in Youth

14

Employment seeks to provide technical and financial support so that also organizations with

smaller budgets can produce evidence of what works. Good quality impact evaluations can

also be designed and kick-started with smaller budgets since additional fundraising is more

likely to be successful when a good design is in place.

Communicating results

• As important to the production of new evidence, how evaluations will communicate the

results of their evaluation should be clearly addressed. This should include dissemination

via events, workshop, and websites. It is also envisioned that each completed evaluation

will produce “a brief”, which is a simple, easy to read report on the major outcomes,

challenges, and lessons learned from the evaluation.

• Engaging policy makers is an important part of dissemination. Project teams should make

attempts to involve policy and government stakeholders in conducting evaluations thereby

ensuring an “authorizing environment” has been created.