Terms of ReferenceFinal Project Evaluation

Solidar Suisse

1. General Information:

Project Title: Institutionalizing School Based Disaster Risk Management in Urban Context of KPType of Contract: Consultant Individual Consultant Institutional Consultancy Mode: National □ International Mode of Selection: Competitive □ Single Source Duration of Contract: 30 days Deadline for the submission of Proposal: 31 October 2018

2. Organization: Solidar Suisse:Solidar Suisse is an international humanitarian organization based in Zurich, Switzerland, has been working since 1936 in the sector of Humanitarian AID, International Corporation, and Social Affairs. Solidar strives for decent work, democratic participation, and social justice worldwide. Solidar started its operation in Pakistan after 2010 super flood and extended humanitarian assistance with focus on emergency relief items, shelters, WASH, sustainable livelihoods and disaster risk management to over 500,000 beneficiaries in Khyber Pakhtunkhawa and Punjab provinces. Solidar also piloted a community led approach to bring back out of school children especially child laborer to formal education and running scale up phase to strengthen non-formal education centers in Lahore improve access to quality primary education.

3. Background:

Solidar Suisse with the collaboration of HELVETAS Swiss Inter-cooperation and financial assistance of ECHO is running a school safety project in Peshawar since June 2017. The project aims to take reduced vulnerabilities of 25'000 students & staff of 75 urban schools for improving KP's action-oriented policy environment on DRR, prevention and preparedness and by strengthening preparedness and response capacities of schools. The major outcomes of the project are

SBDRM mainstreamed in institutional framework of the Education department of KP SBDRM operationalized by applying and increasing disaster preparedness and

response capacities in 75 schools of urban Peshawar Coordination among other DRR actors including ECHO partners ensured for effective

results

The duration of Project is 19 months with its closing date as 31st December 2018. The targeted area (union councils) of project is as under;

1. Union councils Ghari Qamardin2. Union Council Ander Sher3. Union Council Khaliesa 14. Union Council Pakha Ghulham 5. Union Councils Lala Kalay 6. Union Council Nahaqi

Identified/targeted union councils are highly prone to both natural and human induced hazards and are regularly affected which eventually affects negatively on its local economy and the rehabilitation capacity of the communities. The Project focused on enhancing capacities of education managers in district Peshawar including Elementary and Secondary Education department, Provincial Institute of Teacher’s Education, Private school regulatory authorities and teachers from 75 schools i.e. 50 Public schools and 25 private schools. Under this project special considerations were given to build upon what had already been achieved and not to re-invent the wheel and nullify all the efforts done during the recent ECHO funded projects focusing on school safety and SBDRM. Project ensured that program service delivery was provided always in close coordination with target beneficiaries, internal DRR programs within Solidar and Helvetas, other ECHO partners and relevant Government organizations.

The project developed a strong partnership with the technical staff and policy makers in the government with a sharp inclining interest curve to mainstream SBDRM measures and commitments. Sustainability of project efforts and continuation of mainstreaming SBDRM in the government and private schools was of utmost importance and each activity was thought out around this parameter. The major achievement was the strong ownership of the government and its readiness and preparedness to mainstream SBDRM, all major ingredients in this regard were made available - including reactivation of DRR Steering Committee and working groups.

4. Objective/purpose of the consultancy services

This evaluation will be conducted by an independent evaluator. Under the direction of the Country Representative and in cooperation with Program Coordinator and DRR Expert, Project Manager and MEAL Officer, the consultant will be responsible for conducting the final evaluation of the project.The final evaluation is intended to assess the overall impact of the project activities and compile major lesson learned by the project. The main purpose of the end-of-project evaluation is to assess the overall achievement of the project goal toward its targeted outcome as proposed in logical framework. The key objectives of the evaluation are:

a. To evaluate the overall project by assessing the achievement of the project goals while focusing on the effectiveness, relevance, efficiency, sustainability, and impact of the project outcome achieved.

b. To compile and document key lessons learned during the project implementation and identifying promising practices for knowledge purpose.

c. To identify key areas for continued advocacy, sustainability/replication of the project and other future intervention related to project activities.

5. Scope of the Evaluation The evaluation will cover all the three expected outcomes of the project. It will be conducted in project area agreed upon by the evaluator/consultant and Solidar. It will cover the period from the date of effectivity of the project (June 2017) to the time of this evaluation (November - December 2018).The evaluation will assess project outcomes at all levels: within the targeted government institutions and schools in all targeted Union councils of district Peshawar. The evaluation will cover all areas of implementation, including activities delivered by the partner organizations. The evaluation will assess the impact of the project on target beneficiaries, including primary beneficiaries and secondary beneficiaries.

6. Users of evaluation Following are the users of this evaluation study.

Headquarters of partner organizations in Switzerland to contribute to their knowledge about their interventions in Pakistan, its potential application to other contexts and mobilizing further resources for program sustainability/replication.

Country Offices of partner organizations in Pakistan to inform their current progress on the programs and future programming trends and interventions.

Government officials who are responsible for up-take of project intervention. Target School communities to take ownership and contribute further in school

rehabilitation and development work. For donor organization to see how the project has performed amidst various challenges

7. The Evaluation Framework The evaluator will follow the lead of the ToRs and project log frame as basic framework to structure the report. Furthermore, the evaluator will use Core Humanitarian Standard for system wide standards and Sphere Minimum Standards, where applicable, for sectorial standards and comprehensive school safety framework for the project approach.

8. Main Evaluation Questions and Sampling

The key questions that need to be answered by this evaluation include the following, which are divided into five categories of analysis. The OECD/DAC evaluation criteria– effectiveness, relevance, efficiency, sustainability, and impact – will be thoroughly applied in this evaluation.

8.1 Evaluation Criteria Mandatory Evaluation Questions

Criteria1 Questions1. Effectiveness 1) To what extent were the intended project goal, outcomes and outputs

achieved and how?2) To what extent did the project reach the targeted beneficiaries at the project goal and outcome levels? How many beneficiaries have been reached?3) To what extent has this project generated positive (or negative) changes in the lives of School communities and institutions? Why? What are the key changes in the routines of the beneficiaries? Please describe those changes.4) What internal and external factors contributed to achievement and/or failure of the intended project goal, outcome and outputs? How?

5. Relevance To what extent was the project strategy and activities implemented were relevant (policy and strategy) in responding to the need of schools?

To what extent do achieved results (project goal, outcomes and outputs) continue to be relevant to the needs of schools?

7. Efficiency 1) How efficiently and timely has this project been implemented and managed in accordance with the Project Documents? Specifically have resources been used well and strategies to implementation been appropriate.2) What gains have been made in relation to strengthening the resilience of the targeted School communities a) How cost efficient was the project?

9. Sustainability How are the achieved results, especially the positive changes generated by the project in the lives of targeted School communities at the project goal level, going to be sustained after this project ends?

1 Criteria and questions are given in the TORs

Criteria Questions10. Impact What are the unintended consequences (positive and negative) resulted from the

project?11. Knowled

ge creationWhat are the key lessons learnt that can be shared with other practitioners on strengthening the School’s resilience related to project interventions?Are there any promising practices? If yes, what are they and how can these promising practices be replicated in other projects and/or in other areas that have similar interventions?What are the general strengths and weaknesses of the project?What unexpected outcomes and impact has the project had?What advocacy and implementation priorities still require action and commitment from the other stakeholders?What are proposed further possible intervention in the project area?.

12. How project made a difference.

What were the major challenges and gaps in the designing, planning and implementation of the project?What have been the most effective methodologies and approaches used to bring about changes in peoples’ lives? What has worked, what has not and why?How effective has project management been (please address risk management, monitoring, financial and administrative systems)What are the key lessons learned from this project? Have the lessons been shared with others?Based on this study, do SOLIDAR need to expand its area of intervention in the same area with more activities or need to replicate the same project in nearby vicinity?

8.2 Proposed Sampling Size Below is the sampling size proposed for this study? These sampling size can be further improved/adjusted by the consultant after mutual agreement with Solidar based on the time frame and knowledge of the consultant.

Type of Beneficiary

Total Size Sample Size in %

In Number (Approx.)

Details

Schools 75 Schools 20% 15 Schools(10 Government and 5 private schools)

At least 50 individuals per school (Student, teachers and members of parent teacher councils) – Consultant can propose the best sampling size further for this upon mutual agreement.

School Disaster Management Committee (SDMCs)

75 SDMCs/PTCs

20% SMCs

15 SDMCs/PTCs Where each SDMC/PTC has up to 10 members.

Each SMC include 10 members. The sample of SMCs will include 50% male members and 50% female members

11. Key deliverables of evaluator and timeframe

The inception report is required within three days of signing contract. The final report is expected latest by 15th of December 2018.

Sr.# Deliverables Description of Expected Deliverables Deadline of eachdeliverable

1 Inception report An inception report must be prepared by the evaluator before going into the technical mission and full data collection stage. It must detail the evaluator’s understanding of what is being evaluated and why, showing how each evaluation question will be answered by Way of: proposed methods, proposed sources of data and data collection/analysis procedures. The inception report must include a proposed schedule of tasks, activities and deliverables, designating a team member with the lead responsibility for each task or product and budget. It will be discussed and agreed with the project team and MEAL Officer.

2 Documents’ Review

Literature review of all the key project documents. It also includes inception visit if needed.

3 Development of data collection instruments/tools.

All questionnaires and other tools developed to collect information should be shared and finalized prior to evaluation.

4 Data collection from Field (Peshawar)

Complete data collections, meetings, interviews etc. from the field, beneficiaries and other stakeholders (Layyah):

5 Draft Report and Pictures

Presentation regarding findings in the draft report could be added for a quick review. Evaluators must submit draft report for review to Solidar for review to ensure that the evaluation meets the required quality criteria.High Quality Pictures (with written consent form the person, duly signed) in report should also be submitted in original form separately along with report and the title of each picture should include short caption.

10 December 2018

6 Submission of Final Evaluation Report and Data sharing

Relevant comments from key stakeholders must be well integrated into the final version, and the final report must meet all requirements.All materials produced by the study including hard copy of the report(s) and raw data transcripts in soft form

15 December 2018

12. Evaluation ethicsThe evaluation must be conducted in accordance with the principles outlined in the UN Evaluation Group (UNEG) ‘Ethical Guidelines for Evaluation’ http://www.unevaluation.org/ethicalguidelines. It is imperative for the evaluator to:

Guarantee the safety of respondents and the research team. Apply protocols to ensure anonymity and confidentiality of respondents. Ensure any

other evaluation team members are briefed on ethical issues.

Provide referrals to local services and sources of support for women that might ask for them.

Ensure compliance with legal codes governing areas such as provisions to collect and report data, particularly permissions needed to interview or obtain information about children and youth.

Store securely the collected information.

13. ReportingReport (of no more than 25 pages, excluding annexes) to contain: Executive Summary; Background Information on the context; Purpose of the evaluation and the methods chosen; Outcome of using the methods; At least three case studies; Results and discussion; Conclusions; Recommendations (immediate and long term); Appendices. The report should be written in English and presented to Solidar Suisse in hard copy and electronic formats, with the electronic document in a format compatible with MS Word.Outline for Final evaluation report is attached as” Annex 1”

14. Reporting and SupervisionCountry Representative, Project Manager and MEAL Official (Solidar Suisse)

15. Qualification/level requirements of Consultant In-depth and strong knowledge about DRM and education sector. At least Master’s degree in Social, Development Studies, or related field preferences

will be given to MPhil or Phd. At least 10 years’ experience of development sector, with similar projects and in

conducting Program/project Evaluations Proven knowledge of education and development sector and research skills

particularly in the fields of Monitoring, Evaluations, Accountability and Learning along with the knowledge of DRR, SBDRM and various local issues in the context of Khyber Pakhtunkhwa province and Pakistan.

Excellent report writing and presentation skills in English. Able to represent data in tabular, graphical and narrative form.

16. Time Frame The assignment will begin during second half of November and conclude on 15th December

2018.

17. Content of the Proposal:We invite interested organizations to submit the following application documents:

a) Expression of interest addressing expertise and track record – max 2 pages;b) Proposed methodology and rationale, detailed budget, timeline/work plan;c)CVs; andd) Three examples of previous similar work along with final version of the report

The financial proposal must be submitted indicating PKR currency with a detailed break up of all the activities budgeted for, showing unit, unit cost and duration.

18. Logistics and ProceduresThe project will facilitate the consultant for local travel with in Peshawar for data collection, visiting schools and conducting meetings with other stakeholders. The Consultant would be required to make all other travel and logistical arrangements using their own resources. Solidar wouldn’t be able to extend any support for making such arrangement. The cost must be included in the overall budget. Separate bills for travel or any other expenses on original wouldn’t be entertained. Our staff would only assist in access to beneficiaries or authorities involved in the evaluation.

The Consultant would be responsible for compliance with all tax and social security obligations resulting from this contract and has to cover for required insurance coverage on his/her own account and expense. The Consultant may become legally obliged to contribute to the public pension insurance of his/her country of residence. It is understood that such contributions are not payable by Solidar and are to be borne by the Consultant him/herself.

19. Submission Procedure:The technical and financial proposals should be sent at (info@solidar-switzerland.org.pk) by or before October 31, 2018. Only short-listed applicants will be contacted. Solidar reserves the right to reject any or all proposals and is not bound to any legal claim in this regard. No telephone inquiries will be entertained.

Queries can be sent at info@solidar-switzerland.org.pk

Annex 1: Outline for Final Evaluation Report Structure

Section Should Contain Title page andopening pages

Key information, including:• Evaluation intervention being evaluated• Timeframe of the evaluation and date of the report• Locations (country, region, etc.) of the evaluation – may include maps• Evaluators’ names and/or organizational affiliations• Name of the commissioning organization commissioning the evaluation• Table of contents which also lists tables, graphs, figures and annexes• List of acronyms

Executive summary

A stand-alone section (usually 1-3 pages) that includes:• Overview of the humanitarian action being evaluated• Evaluation objectives and intended audience• Evaluation methods• Most important findings and conclusions, following the sequence in which these are presented in the main report• Main recommendations

The Project and its development context

Project start and its durationImplementation statusProblems that the project seeks to addressImmediate and development objectives of the projectMain stakeholdersResults expected

1. Introduction The scope and purpose of the evaluation, intended audience, team composition, and structure of the report.The overarching evaluation questions.Were there any changes to the evaluation questions proposed in the ToR?How was scoping done?

2. Methods A description of the main methods used, their appropriateness and why they were chosen.The nature and scope of involvement of the affected population.Key constraints to carrying out the evaluation (e.g., lack of time, constrained access to affected population, lack of baseline data), and their effect.Any biases in the evaluation process or evaluation team and how thesewere mitigated.International standards used as reference points in the evaluation, e.g. Sphere, CHS, and any conceptual frameworks used referred to Section Framing evaluation

3. Context Contextual analysis of the disaster to which the intervention is responding, e.g. affected area and population, key events.

4. Main sections Organized by evaluation criteria, by evaluation questions or other framework appropriate to the evaluation and its intended users, these chapters present the evidence and findings.

TipPresenting the conclusions and recommendations at the end of each section emphasizes (and helps to ensure) they are grounded in the

relevant evidence, findings, and conclusions.5. Lesson Learned Best and worst practices in addressing issues relating to relevance,

performance and success

6. Conclusions Flow logically from and reflects the report’s central findings.Provide a clear and defensible basis for value judgements.Provide insights pertinent to the intervention that has been evaluated and to the purpose of the evaluation.

7. Recommendations

Should be:(a) Clear, relevant and implementable, reflecting any constraints to follow-up(b) Follow on from the main conclusions and reflect consultation with keystakeholders(c) Presented in priority order, with a timeframe for implementation, suggestingwhere responsibility for follow-up should lie(d) Limited in number.

Annexes Annexes usually include:• TOR• List of persons interviewed, and sites visited• List of documents consulted, and secondary data used• More details on the methods, such as data-collection instruments• Evaluators’ biographical data and/or justification of team composition• Evaluation matrix• Chronology of the issue or action being evaluated Other annexes could address topics on which a detailed discussion would be outof place in the main report, or present results from specific methods (such as asummary of the responses to an online survey).

TipUse annexes for supporting elements and for detail that wouldclutter the main report.

TipReport will be circulated mainly in electronic form, considerpresenting annexes as a separate document to make the reportshorter and less intimidating when first opened.