The World Bank · Document of The World Bank Report No: ICR00004241 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-48750) ON A CREDIT IN THE AMOUNT OF SDR 44.9 MILLION (US$70 MILLION

Document of The World Bank

Report No: ICR00004241

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-48750)

ON A

CREDIT

IN THE AMOUNT OF SDR 44.9 MILLION (US$70 MILLION EQUIVALENT)

TO THE

REPUBLIC OF GHANA

FOR A

GHANA SKILLS AND TECHNOLOGY DEVELOPMENT PROJECT

June 29, 2017

Education Global Practice GED13 Africa

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective January 31, 2011)

Currency Unit = Ghana Cedi (GH₵) Cedi 1.55 = US$1.00

US$1.00 = 0.64023 SDR

FISCAL YEAR

ABBREVIATIONS AND ACRONYMS

AfDB African Development Bank CAS Country Assistance Strategy CBT COTVET

Competency-Based Training Council for Technical and Vocational Education and Training

CPS Country Partnership Strategy CSIR Center for Scientific and Industrial Research DANIDA Danish International Development Agency EDSeP Education Sector Project EOI Expression of Interest ESMF Environmental and Social Management Framework FM Financial Management GAEC Ghana Atomic Energy Commission GDP Gross Domestic Product GETFund Ghana Education Trust Fund GNP Gross National Product GPRS II Growth and Poverty Reduction Strategy II GSTDP Ghana Skills and Technology Development Project GTUC Ghana Technical University College GTZ Gesellschaft für Internationale Zusammenarbeit (German International

Cooperation Agency) ICT Information and Communications Technology IDA International Development Association IDP Institutional Development Plan IFR Interim Financial Report ISR Implementation Status and Results Reports JICA Japan International Cooperation Agency KPI Key Performance Indicator M&E Monitoring & Evaluation MDG Millennium Development Goal MEST Ministry of Environment, Science, and Technology MESW Ministry of Employment and Social Welfare MIS Management Information System MOE Ministry of Education

MOFEP Ministry of Finance and Economic Planning MSME Micro, Small, and Medium Enterprises NCB National Competitive Bidding NER New Education Reform NGO Non-Governmental Organization NPV Net Present Value PDO Project Development Objective PEC Project Executive Committee PIM Project Implementation Manual PSC Project Steering Committee PSDS II Private Sector Development Strategy II PSU Project Support Unit R&D Research and Development S&T Science and Technology SBD Standard Bidding Document SDF Skills Development Fund SDR Special Drawing Rights SIL Specific Investment Loan SME Small & Medium Enterprise STI Science, Technology, and Innovation STIP Science, Technology, and Innovation Policy TA Technical Assistance TI Technical Institute TVET Technical and Vocational Education and Training UNDP United Nations Development Program

Senior Global Practice Director: Jaime Saavedra Chanduvi

Practice Manager: Halil Dundar

Project Team Leader: Eunice Yaa Brimfah Ackwerh

ICR Team Leader: Eunice Yaa Brimfah Ackwerh

i

GHANA Ghana Skills and Technology Development Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 16 4. Assessment of Risk to Development Outcome ......................................................... 26 5. Assessment of Bank and Borrower Performance ..................................................... 28 6. Lessons Learned ....................................................................................................... 31 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 32 Annex 1. Project Costs and Financing .......................................................................... 33 Annex 2. Outputs by Component ................................................................................. 35 Annex 3. Economic and Financial Analysis ................................................................. 50 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 65 Annex 5. Beneficiary Survey Results ........................................................................... 67 Annex 6. Stakeholder Workshop Report and Results ................................................... 70 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 71 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 79 Annex 9. List of Supporting Documents ...................................................................... 80

MAP

ii

A. Basic Information

Country: Ghana Project Name: Ghana Skills and Technology Development Project

Project ID: P118112 L/C/TF Number(s): IDA-48750

ICR Date: 06/29/2017 ICR Type: Core ICR

Lending Instrument: SIL Borrower: REPUBLIC OF GHANA

Original Total Commitment:

XDR 44.90M Disbursed Amount: XDR 43.84M

Revised Amount: XDR 43.84M

Environmental Category: B

Implementing Agencies: Council for Technical and Vocational Education and Training (COTVET)

Cofinanciers and Other External Partners: DANIDA B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 12/14/2009 Effectiveness: 06/29/2011 11/23/2011

Appraisal: 12/07/2010 Restructuring(s): 06/10/2016

Approval: 03/29/2011 Mid-term Review: 12/02/2013 12/12/2013

Closing: 06/30/2016 12/31/2016 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Moderately Satisfactory Overall Borrower Performance:


iii

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual

Major Sector/Sector

Information and Communications Technologies 2 2

Public Administration - Information and Communications Technologies

2 2

Education 54 54

Workforce Development/Skills 39 39

Tertiary Education 4 4

Public Administration - Education 11 11

Industry, Trade and Services 44 44

Other Industry, Trade and Services 44 44

Major Theme/Theme/Sub Theme

Finance 10 10

Financial Infrastructure and Access 10 10

MSME Finance 10 10

Human Development and Gender 60 60

Education 40 40

Access to Education 10 10

Science and Technology 10 10

Standards, Curriculum and Textbooks 10 10

Teachers 10 10

Labor Market Policy and Programs 20 20

Active Labor Market Programs 10 10

Labor Market Institutions 10 10

Private Sector Development 30 30

Business Enabling Environment 17 17

iv

Innovation and Technology Policy 17 17

Enterprise Development 13 13

MSME Development 13 13 E. Bank Staff

Positions At ICR At Approval

Regional Vice President: Makhtar Diop Obiageli Katryn Ezekwesili

Country Director: Henry Kerali Ishac Diwan

Practice Manager: Halil Dundar Peter Materu (Acting)

Task Team Leader(s): Eunice Yaa Brimfah Ackwerh Peter Darvas

ICR Team Leader: Eunice Yaa Brimfah Ackwerh

Senior Global Practice Director:

Jaime Saavedra Chanduvi Ritva Reinikka

ICR Primary Author: Laura Stonestreet McDonald F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The PDO was “to improve demand-driven skills development and increase adoption of new technologies in selected economic sectors.” The PDO in the Financing Agreement was as follows: “The objective of the Project is to improve demand-driven skills development and to increase adoption of new technologies in selected economic sectors of the Recipient.” Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised. (a) PDO Indicator(s) Section 3 and Annex 2 of the ICR provide a comprehensive assessment on the level of achievement under each PDO- and intermediate-level indicator. These sections provide detailed and disaggregated data for all PDO-level indicators.

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Increase in labor productivity by participating firms disaggregated by economic sector and size (small, medium, large).

v

Value quantitative or Qualitative)

0 60 N/A 426

Date achieved 03/06/2011 6/30/2016 12/31/2016 Comments (incl. % achievement)

Target exceeded. There was an increase of 426 percent over the 2013-2016 period. The year-on-year increases in productivity are more modest and are reported in Annex 2.

Indicator 2: Increase in investment by participating enterprises in skills and technology development (disaggregated by economic sectors; size (small, medium, large); skills and technology)


0 40 N/A 129


Target exceeded. There was an increase of 129 percent over the 2014-2016 period.

Indicator 3: Satisfaction with skills by trainees (disaggregated by sex, region) Value quantitative or Qualitative)

0 70 N/A 98


Target exceeded.

Indicator 4: Satisfaction with skills by participating firms disaggregated by economic sectors; region; size (small, medium, large)


0 70 N/A 97.5


Target exceeded.

Indicator 5: Direct project beneficiaries (% of whom are female) Value quantitative or Qualitative)

0 24,000 (50) N/A 105,085 (31.5)


Target exceeded.

vi

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: National Skills Strategy developed and adopted Value quantitative or Qualitative)

No Yes N/A Partially achieved


Target partially achieved. National TVET Strategy developed and approved by the Council, though it has not yet been approved by the Cabinet.

Indicator 2: COTVET management information system for skills development is functional


No Yes N/A Partially achieved

Date achieved 03/06/2011 6/30/2016 12/31/2016

Comments (incl. % achievement)

Target partially achieved. A simple management information system (MIS) has been set up within COTVET. The Government is currently working with development partners to further strengthen this system to make it more robust and comprehensive.

Indicator 3: Guidelines for quality assurance systems including certification are developed and adopted


No Yes N/A Yes

Date achieved 03/06/2011 6/30/2016 12/31/2016


Target achieved. Under the Project, 23 manuals to guide TVET institutions in the development, delivery and management of TVET programs (using CBT) were developed, certification processes have been established and quality assurance mechanisms have been put in place.

Indicator 4: Institutional Development Plans for training providers developed and implemented


0 10 N/A 10


Target achieved.

Indicator 5: STI Directorate established with full complement of staff

vii


No Yes N/A Yes


Target achieved.

Indicator 6: New collaborations between supported technology providers and private sector for adaptation and diffusion of technology


0 20 N/A 36


Target exceeded.

Indicator 7: Information System available for STI Directorate use Value quantitative or Qualitative)

No Yes N/A No


Target not achieved.

Indicator 8: Employers rating of competencies of trained employees Value quantitative or Qualitative)

0 60 N/A 97.5


Target exceeded.

Indicator 9: Number of Industry/service provider collaborations Value quantitative or Qualitative)

0 200 N/A 361


Target exceeded.

Indicator 10: Increased proportion of those trained that become certified (disaggregated by gender and age)

Value quantitative or

0 50 N/A 0

viii

Qualitative) Date achieved 03/06/2011 6/30/2016 12/31/2016


Target not achieved. A large number of individuals received competency-based training prior to the formal establishment of the National TVET Qualifications Framework. As such, the certifications for completion of the training (received by 90 percent of trainees) could not be formally recognized as adhering to the NTVETQ criteria.

Indicator 11: New training courses and new partnerships established Value quantitative or Qualitative)

0 20 N/A 122, 46


Target exceeded.

Indicator 12: New technologies adopted by participating firms Value quantitative or Qualitative)

0 120 N/A 70


Target partially achieved with 70 developed by participating firms.

Indicator 13: Collaborations between enterprises and technology centers (total for all centers receiving project support)


0 60 N/A 549


Target exceeded.

Indicator 14: Grant Agreements for SDF sub projects signed


0 440 N/A 646


Target exceeded.

Indicator 15: Sub-projects satisfactorily completed

ix


0 440 N/A 617


Target exceeded.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 09/21/2011 Satisfactory Satisfactory 0.00 2 06/12/2012 Moderately Satisfactory Moderately Satisfactory 7.40 3 01/08/2013 Moderately Satisfactory Moderately Satisfactory 8.19

4 10/13/2013 Moderately Satisfactory Moderately

Unsatisfactory 13.61

5 04/29/2014 Moderately Satisfactory Moderately

Unsatisfactory 22.10

6 12/22/2014 Satisfactory Moderately Satisfactory 33.22 7 06/29/2015 Satisfactory Moderately Satisfactory 50.38 8 12/31/2015 Satisfactory Moderately Satisfactory 54.04 9 06/22/2016 Satisfactory Moderately Satisfactory 58.58

10 12/30/2016 Moderately Satisfactory Moderately Satisfactory 63.05 H. Restructuring (if any)

Restructuring Date(s)

Board approved PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring &

Key Changes made

06/10/2016 MS MS 58.6

Extension of closing date by six months and a reallocation of funding across categories.

x

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1. Country Context. At the time of project appraisal, Ghana was one of the top three economic performers in Sub-Saharan African and was striving to become a middle-income country1 by 2015. Over the preceding decade, Ghana had shown impressive gains in terms of economic growth and poverty reduction. From 2000 to 2009, real Gross Domestic Product (GDP) growth had averaged over 5.7 percent annually and was projected to reach 4.5 percent in 2010 and 20 percent in 2011. Poverty had also decreased since the early 1990s – from 52 percent to 28.6 percent in 2005/2006.2 Economic growth was largely driven by key exports including cocoa, rubber, gold, timber and on remittances. Moreover, the services sector (wholesale, retail, construction, and hospitality), as well as agriculture, were also key drivers of Ghana’s economic growth. 2. Sector Context. Ghana faced a number of pressing barriers to labor productivity which continued to limit the performance of firms and enterprises across economic sectors. In addition to these barriers, the education, technical and vocational education and training (TVET) and science, technology, and innovation (STI) sectors were not well-aligned with labor market demand–limiting the country’s short-, medium- and long-term growth prospects. At the time of project appraisal, Ghana’s productivity was lower than other African countries which had a similar economic standing (e.g., Mauritius and Botswana) and the large majority of employment was in the informal sector of the economy. There was a clear need to simultaneously equip the labor force with market-relevant skills while also promoting the development, adoption and diffusion of new technologies as a way to increase productivity, competitiveness and, in turn, promote equitable economic growth. 3. At the time of project appraisal, most TVET was provided by the formal sector – with 200 public formal training institutions (26 of which were technical institutes - TIs) and around 430 private institutions. Of a total of around 105,000 students, around one-third were enrolled in public formal TI3 while the other two-thirds were enrolled in private (mostly church-owned) TI. The majority of vocational training occurred through informal apprenticeships which were not subsidized by the Government but rather financed by the workers themselves. 4. Inadequate and low quality training, resulting in a lack of labor market-relevant skills, were pressing issues. In addition to low qualifications of the labor force, employers were struggling with a shortage of skilled workers which was resulting in a large number

1 At the time, this was defined as having a GDP per capita of US$1,000. 2 Ghana Statistical Service, 2007, cited in Rolleston, 2010. 3 Of the approximately 35,000 students enrolled in public formal TI, around 17,500 students were enrolled in the 26 TIs (under the auspices of the MOE) while the same number of students were enrolled across the other 174 public formal TIs (operating under the auspices of various governmental agencies, e.g., Ministry of Employment, Trade, Local Government and Agriculture).

2

of hard-to-fill vacancies. This was, in large part, a result of limitations in the existing formal training system and/or lack of adequate/formal training among workers. Most of the training provided by formal TIs was not demand-driven – as it was not linked – and, therefore, not influenced by or responsive to labor market needs. The curricula, in addition to being outdated, were not tailored to the specific needs of high growth sectors. Moreover, most staff from TVET training institutes did not possess hands-on industry-relevant skills necessary to prepare their students for the labor market and learning facilities and equipment were typically outdated. Further, the certification system for apprenticeships was not standardized nor was prior learning recognized in certification. A limited number of TVET providers had experience with competency-based training (CBT) although the types of training courses provided were limited in scope and did not cover key sectors of the country’s economy. In sum, TVET providers were not focused on improving trainees’ employability and in addition to confirming this, an in-depth study of the skills sector in Ghana concluded that the TVET system was fragmented, inefficient, and was not well-aligned with labor market demand. 5. In order to strengthen the quality and relevance of training programs and improve the employability of trainees, the Government sought to further strengthen the provision of training by strengthening and improving the overall quality of the entire TVET system and its relevance to the labor market. Specifically, the Government would address these issues through: (i) the provision of targeted financing to strategic sectors of the economy; (ii) the promotion of coordination across public and private providers; (iii) the development and use of standards, quality assurance mechanisms and qualification and certification systems; (iv) the involvement of the private sector in the design and delivery of training programs; and (v) the integration of CBT in various vocational fields. 6. In addition to needed improvements in the provision of TVET, further investment was needed in the development, transfer and use of new technologies. At the time of project preparation, Ghana’s STI system was largely underfunded and the majority of resources available were absorbed by recurrent expenditures (salaries and costs to operate facilities).4 As such, limited discretionary resources were available for strengthening the quality and relevance of the STI system. Funding decisions were made with little input from science and technology (S&T) and research institutes, universities and/or end users (e.g., private sector, farmers, and informal enterprises). As a result, there was limited competition and, in turn, limited pressure on these institutes to further develop and utilize new technologies as a way to increase productivity and drive economic growth. At the time, only a small portion of resources used for research and development (R&D) came from the private sector and there was no funding instrument or incentive structure specifically devoted to developing and upgrading new technologies and innovations for use in commercial industries.

4 The STI system received US$49 million or 1.1 percent of Ghana’s budget in 2008 representing 0.3 percent of GDP with most of the funding going to public research institutes. A significant portion of research institute budgets (from the Government) went to salaries and facilities with less than 15 percent available for research or equipment.

3

7. In 2004, a draft TVET Policy Framework for Ghana had been developed while a government white paper around the same time led to the introduction of the New Education Reform (NER) and the establishment of the National Education Reform Implementation Committee (NERIC). This reform program aimed to address important inefficiencies in the education system – including TVET. Improving the quality and relevance of post-basic education was also a core feature of the NER. A number of development partners (DPs) aimed to contribute to the Government’s efforts in increasing the efficiency, relevance and quality of skills development programs in Ghana, most notably DANIDA, AfDB and GTZ. Key government strategies including the Medium-Term Development Framework (MTDF) and Ghana’s Growth and Poverty Reduction Strategy II (GPRS II) also highlighted the need to build human capital to increase competitiveness, productivity, and economic growth.

8. Prior to project preparation, the Government had taken a number of steps to address challenges related to TVET and to promote and incentivize STI development, including the establishment of the Council for Technical and Vocational Education and Training (COTVET) in 20065 and the Ministry of Environment, Science and Technology (MEST) in 2009. In addition to revising the Education Strategic Plan (ESP) (2010-2020), the Government had finalized a Science, Technology, and Innovation Policy (STIP) in 2010 –which aimed to harmonize TVET and STI policies. Through a combination of reforms, targeted policies, and strategic investments and partnerships, the Government aimed to improve economic competitiveness, increase productivity, promote employment and foster economic growth. 9. Project Context. Through the Ghana Skills and Technology Development Project (GSTDP) (P118112) (funded by an International Development Association (IDA) credit in the amount of SDR 44.9 million, or US$70 million equivalent), the Bank aimed to support the Government’s strategic objectives to strengthen the TVET system and to promote STI development, transfer and uptake. It was also aligned with the country’s Private Sector Development Strategy II (PSDS II) which aimed to promote innovation, and improve training and business development services. Using a Sector Investment Loan (SIL) instrument, the Bank would support the Government in further developing informal and formal TVET systems which were demand-driven. In the formal sector, the GSTDP aimed to improve quality of training and industry-readiness of graduates at universities, polytechnics, TVET institutes/schools as well as the employability of those seeking work and employees themselves. In the informal sector, the Project aimed to support workers and firms, with added support from industry and business associations, where capacity was limited. In addition to these important aims, the GSTDP also sought to support the development and adoption of new technologies in priority economic sectors. In addition to being built on a large body of analytical work, the GSTDP incorporated key elements of

5 COTVET was established by the Government in 2006 as an intergovernmental agency reporting to the ministries responsible for TVET. Specifically, it is a national body set up by an Act of Parliament of the Republic of Ghana to coordinate and oversee all aspects of technical and vocational education and training in Ghana.

4

Component 3 (Tertiary Education Innovation 6 ) of the IDA-funded Education Sector Project (EdSeP) (P050620) which was approved in March of 2004 and closed in October 2011. 10. The GSTDP which was approved on March 29, 2011 by the Bank’s Board of Executive Directors and became effective approximately eight months later on November 23, 2011.

1.2 Original Project Development Objectives (PDO) and Key Indicators 11. According to the project appraisal document (PAD) the project development objective (PDO) was “to improve demand-driven skills development and increase adoption of new technologies in selected economic sectors.” The PDO in the Financing Agreement is worded slightly differently, with the PDO being “to improve demand-driven skills development and to increase adoption of new technologies in selected economic sectors of the Recipient.” 12. The five PDO-level indicators were: (i) percentage increase in labor productivity by participating firms (disaggregated by economic sector and size (small, medium and large)); (ii) percentage increase in investment by participating enterprises in skills and technology development (by economic sector, size, skills and technology); (iii) percentage of trainees satisfied with skills (sex and region); (iv) percentage of participating firms satisfied with skills (by economic sector, region and size); and (v) number of beneficiaries (percentage of which were female).

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 13. There were no revisions to the PDO or key indicators.

1.4 Main Beneficiaries 14. The targeted beneficiaries included employers, employees, industry associations, public and private training institutes, universities, polytechnics, and other agencies engaged in technology development. In the short and medium-term, the GSTDP aimed to benefit employers and employees of designated economic sectors who would gain additional skilled employment, higher earnings, and improved productivity. In the

6 The SDF incorporated elements of the Teaching and Learning Innovation Fund (TALIF) supported under Component 3 of the EdSeP. As described in the project summary, Component 3 activities were funded through a demand-driven TALIF. The Fund's overall objectives were: (i) to raise the quality of teaching and learning performance; (ii) to sharpen the relevance, competences and skill content of tertiary education; (iii) to improve the efficiency by which institutions operate their academic programs; and (iv) through the combined efforts of these, augmented by the development of distance education capabilities and earmarked assistance to the University for Development Studies (UDS) in the disadvantaged northern region, to open up greater access to tertiary-level academic programs.

5

medium- to long-term, it aimed to benefit a larger group of economic actors through better access to modern training and technology services. 1.5 Original Components 15. Component 1: Institutional Strengthening of Skills Development (Estimated Cost at Appraisal: US$4 million; Revised Allocation: US$3.45 million; Actual Expenditure: US$3.42 million). The objective of the Component was to support the development of a national skills strategy to strengthen Government institutional capacity to manage the sector (in planning, coordination, quality assurance) and modernize TVET service delivery towards improved quality, relevance, accountability and effectiveness in skills development. The Component was comprised of two Sub-components: Sub-component 1.1 which aimed to support the development of COTVET’s technical capacity, strategic systems and policies; and Sub-component 1.2 which aimed to provide targeted support to a select number of TVET providers. 16. Component 2: Institutional Strengthening of Science and Technology Development (Estimated Cost at Appraisal: US$4 million; Revised Allocation: US$3.26 million; Actual Expenditure: US$3.19 million). This Component aimed to strengthen the planning, management, and coordination of national STI policies and programs in order to make efficient use of resources and complement the national economic development. It also aimed to support technology development and diffusion that would be more responsive to the needs of the economy by enhancing interactions between selected research institutes, university departments, and their external clients (e.g., private sector), thereby encouraging domestic innovation. This Component was comprised of two sub-components. Sub-component 2.1 aimed to improve the planning, management and coordination of policies by stablishing a well-functioning STI Directorate within MEST capable of supporting evidence-based STI policies and implementing priority activities in the national STI Development Plan (2011-2015). Sub-component 2.2 aimed to improve the capacities and incentives of selected research institutes, universities, and technology providers to develop, adapt and diffuse technologies to private sector enterprises on a demand-driven basis. 17. Component 3: Financing of Skills and Technology Development Programs through the Skills Development Fund (Estimated Cost at Appraisal: US$50 million; Revised Allocation: US$49.27 million; Actual Expenditure: US$47.96 million). The objective of this component was to finance skills and technology development programs in prioritized economic sectors through a demand-driven Skills Development Fund (SDF) managed by COTVET. Component 3 was comprised of two Sub-components. Sub-component 3.1 aimed to provide a demand-driven response to the challenges facing productive sectors of the economy by supporting grantees through an SDF with four separate funding windows – formal sector, informal sector, training and technical institutions, and S&T training service providers. Sub-component 3.2 aimed to support SDF outreach and management by funding extensive outreach activities, training and technology needs assessments, partnerships with intermediaries, and the cost of management, administration and coordination for the SDF.

6

18. Component 4: Project Management and Monitoring and Evaluation (Estimated Cost at Appraisal: US$5 million; Revised Allocation: US$5.67 million; Actual Expenditure: US$5.62 million). This Component aimed to provide effective implementation of the project by establishing a project support unit embedded within COTVET and reporting to the Project Steering Committee (PSC). The PSU was responsible for overseeing and managing donor-supported activities in the skills sector. This was to ensure harmonization among DP-supported activities. For M&E, COTVET was to update data to facilitate accurate reporting on the key progress indicators identified in the results framework (RF). Most of the data for monitoring project outcomes would come from regular project reports, key surveys and a labor market observatory. These data would be supplemented by project preparatory studies in key economic sectors and a baseline survey undertaken during the first year of implementation. The Project would also contract with an independent monitoring firm to support M&E activities, including an impact evaluation.

1.6 Revised Components 19. The components were not revised under the Project.

1.7 Other significant changes 20. A Level II restructuring of the Project was approved on June 10, 2016. This restructuring extended the project closing date from June 30, 2016 to December 31, 2016 and introduced a reallocation of funding across categories.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry Project Preparation 21. Given the broad project aims, efforts were made to ensure that preparation was participatory and involved key stakeholders, including: various government ministries (education; environment, science and technology; employment, etc.); academic and research institutions; and private sector representatives. Mission teams were quite large and were comprised of various S&T and TVET specialists, as well as experts in the design and implementation of competitive skills funds. Several DPs including the AfDB, Japan International Cooperation Agency (JICA) and German Technical Cooperation Agency (GTZ) also participated in preparation missions. DANIDA committed US$10 million to support the SDF. 22. The Bank worked closely with the Government and DPs to develop key aspects of the design and implementation arrangements of the GSTDP, including the establishment of the PSC and the SDF Secretariat. The Bank, in collaboration with the Government, also made a concerted effort to seek private sector inputs into the GSTDP design. The

7

Government played an active role in project preparation establishing an inter-governmental technical working group comprised of representatives from relevant ministries to support project preparation. In addition to proposed counterpart funding in the amount of US$5 million (in-kind contribution), the Government planned to create the STI Directorate, under the MEST7, which was expected to benefit from and fulfill an important role in the implementation of the GSTDP. 23. The GSTDP was conceived as a response to a growing focus of the Bank and Government on increasing Ghana’s economic competitiveness. A number of analytical studies were undertaken during project preparation which informed the overall design of the Project and choice of activities to be supported under each of the GSTDP’s components. These studies included, among others, an in-depth study on the supply of and demand for skills in Ghana, and research on the status, issues, policies and training needs of the “prioritized economic sectors for project intervention” including, among others, ICT, livestock, horticulture, and construction. These sector-specific studies provided valuable information in defining the GSTDP’s overarching objectives as well as its envisioned outputs and outcomes. The Bank’s own internal reviews, including the concept review, decision review and the quality enhancement review (QER) which was held in September 2010 – were thorough and provided useful guidance to the team in further fine-tuning the link between project-supported activities and the envisioned outputs and outcomes. The team took on the QER’s recommendations to utilize the SDF as the main vehicle to finance skills and technology development within strategic economic sectors that showed potential for increasing productivity, earnings and economic growth in Ghana. 24. Project preparation was thorough and well documented, with the status of preparation, and decisions and discussions related to activities and implementation arrangements detailed in aide-memoires and other relevant project documents and manuals. Despite these positive attributes, additional efforts might have been made to provide additional clarity – particularly in terms of capacity-building of the COTVET – not on the proposed outcomes (i.e., a stronger COTVET with specific capabilities) but the specific activities that would be supported under the GSTDP to ensure this was achieved. In light of the expansion of the SDF, and the administrative support needed in its management (particularly in M&E), additional attention might have been afforded to how these capacity needs would be met, while also strengthening the COTVET. 25. Given that COTVET was a nascent entity only recently established and comprised of four staff, the Bank’s pragmatic choice to establish a separate PSU was sound, and fully supported by DANIDA and other DPs working in the sector. The original design included a plan to slowly build the capacity of COTVET to lead efforts in the TVET sector as part of its mandate as this was part of the Government’s larger strategic agenda for managing and coordinating efforts in the TVET sector.

7 The Ministry’s name at preparation was the Ministry of Environment, Science and Technology (MEST) and was subsequently change to the Ministry of Environment, Science, Technology and Innovation (MESTI).

8

Design 26. The objectives of the Project were strongly aligned with the Government’s strategic agenda as laid out in the Medium-term National Development Policy Framework (2010-2013), the Country Partnership Strategy (CPS) and were aligned with and reflective of objectives of other DPs in supporting Ghana’s TVET and STI sectors.8 Activities included in the Project design could be expected together to contribute to achievement of the PDO, among them: capacity-building among key government agencies, TVET and research/S&T institutes; improvements in strategy and policy to guide skills development and to strengthen linkages between industry and S&T institutes; development of STI and TVET strategies and supportive systems; and full establishment of a competitive SDF. 27. Lessons learned were integrated into project design, including, the importance of: using a competitive grant funding program to ensure efficiency in the selection and financing of sub-projects; ensuring strong participation of stakeholders in project design; and building institutional capacity. The GSTDP’s support to the SDF was also appropriate given experience up until that time not only with competitive skills funds in Ghana but also in other settings. Project manuals, including the SDF and implementation manuals, were thorough including detailed information and specifics on implementation arrangements and roles and responsibilities of various stakeholders. However, additional M&E templates and data collection instruments and methods could have been more clearly described in these key operational documents – though these were subsequently developed in the course of project implementation. 28. The Project’s focus on strategic priority economic sectors selected on a demand-driven basis was sound choice given the Project’s emphasis on increasing productivity and competitiveness among targeted beneficiaries. Based on the thorough analytical work carried out during appraisal, a set of potential sectors which could be supported under the SDF were identified (including ICT, construction and housing, tourism and hospitality, livestock and horticulture). Despite this, given the demand-driven nature of the SDF financing modality, additional sectors which showed high potential and from which there were demand (applicants) could also be supported. 29. The GSTDP’s implementation arrangements were largely sound with critical bodies established to manage and coordinate various activities (PSC, SDF committee, among others). In hindsight, however, while the comprehensive design of the GSTDP was appropriate in light of the Government’s own agenda – because it involved multiple entities and two ministries which were not accustomed to collaborating on activities (COTVET under the Ministry of Education and the STI Directorate under the MEST) – the design might have ensured a very strong mechanism was in place to guide project implementation.

8 These aspects were also well-aligned with Government strategies for the TVET/skills and STI sectors and the country’s broader agenda for economic growth and development, highlighted in core documentation and strategies – including the 2004 TVET strategy.

9

30. The scope of the GSTDP – which included strategy and policy development, their concrete application, and a competitive skills fund (the SDF) with additional innovations – was reflective of a far-reaching PDO. While the specific activities and objectives for each Component and Sub-component were well-defined and relevant to the full attainment of the PDO, additional clarity on the intended sequencing of activities could have been made more explicit in the PAD. Additionally, while risks and mitigation measures were properly identified, additional considerations might have been given to the resources (both human and financial) that would be needed to implement such a far-reaching Project.

31. The Project’s Results Framework (RF) was generally sound and capable of capturing progress across each component and sub-component – including main outputs and outcomes envisioned under the Project. While the RF and the M&E arrangements were generally robust, additional details on M&E methods and additional PDO-level indicators to directly capture progress under Components 1 and 2 might have been incorporated into the RF – though a number of indicators which were highly relevant in assessing progress in these domains were included at the intermediate-level. In addition, the larger M&E system established under the Project provided a detail and comprehensive assessment of the activities which were supported under Components 1 and 2 as well as under the SDF (which accounted for the largest portion of project financing). The M&E system which was established and significantly strengthened under the Project was quite comprehensive and provided a comprehensive dataset to measure the achievements observed across all Components and Sub-components (though some were not formally included in the RF). Quality at Entry 32. The objectives and design were highly relevant to the Government’s strategic agenda in the skills and technology development sectors. Project preparation was highly participatory, and the choice of project-supported interventions were guided by analytical work while aspects of the implementation arrangements were guided by lessons learned. The project design and preparation benefitted from a thorough QER. While the PAD was very thorough, the linkages between Components 1 and 2 and the overall objectives of the Project and additional performance measures could have been incorporated into the RF at the PDO-level to better capture progress on these Components. While M&E was generally strong – and ultimately strengthened during project implementation— additional efforts could have been made at appraisal to better develop and finalize the data collection instruments and M&E arrangements at appraisal. In light of the above, the Quality of Entry is rated Moderately Satisfactory. 2.2 Implementation 33. The Project was approved on March 29, 2011 and became effective on November 23, 2011. Its effectiveness was delayed as additional time was needed to finalize the Project Implementation Manual and to receive the Government’s formal issuance of legal opinion on the Project. Despite this short delay, progress had already been made on a

10

number of activities including: the set-up of key committees: development of the SDF manual; and preparation for the SDF’s first call for proposals. 34. The GSTDP experienced some initial challenges in the first year of implementation. Specifically, the Executive Director of COTVET left his position (his contract was not renewed) and key project staff including the Project Coordinator, FM Specialist and SDF Manager also left their positions. As a result of the PSU’s limited capacity in the initial phase of the Project, the first call for proposals under the SDF was delayed as was the recruitment of the necessary firm/s to guide the implementation of Components 1 and 2 of the Project. The first call, undertaken beginning in July 2011, received very few applicants and awarded only six grants. Following a technical review of the Project, the following improvements were made to the SDF: (i) the SDF grant approval process (including the application form, application flow, changes to the grant agreement) was modified, including the approval timeline; (ii) a large number of project intermediaries were hired and trained to assist prospective applicants in the application process; and (iii) non-performing evaluators were replaced and new technical evaluators were hired and provided intensive training. Further, (iv) efforts were made to ensure equity in access to grants (particularly among the informal sector and in the northern regions of the country) by undertaking targeted outreach campaigns. 35. By July of 2012, the pace of implementation had picked up as a result of a strengthened PSU supported by additional procurement staff, an M&E specialist and a few grant administrators as well as improved working conditions (with new offices in a separate building). The second call for proposals had been launched in September of 2012 and 72 grants were awarded. By December 2012, disbursement under the GSTDP was still low – reaching only 12 percent of the total credit amount, primarily reflective of spending under Component 3. During this time, an increased number of consultants were hired within the PSU – in large part to administer and manage aspects of SDF implementation – which required significant time and resources. 36. During this period and in the time leading up to the mid-term review (MTR), activities under Components 1 and 2 continued to experience delays – related to identifying and hiring relevant firms to provide TA targeted at institutional strengthening. There was also growing concern related to the limited technical capacity of the STI Directorate. While relevant TVET manuals and policies had been developed – there was concern related to their formal adoption and implementation on the ground – and the criteria for selection of TVET institutes to benefit under Sub-Component 1.1 had not been agreed upon. Limited progress had been made in the set-up of the management information systems (MIS) in both COTVET and MESTI as planned under Components 1 and 2. Elections which took place in January of 2013 resulted in some changes in the Government and to the composition of the PSC. 37. During the period which followed, in addition to the strengthening of the PSU, a comprehensive M&E system was developed. This M&E system included a number of data collection instruments jointly developed by the Bank and the Government. This comprehensive M&E system tracked all PDO- and intermediate-level indicators included

11

in the Project’s RF as well as additional performance measures that the Government and Bank had agreed would be useful in further capturing the Project’s implementation progress and overall development impact. While delays under Components 1 and 2 persisted, important progress had been observed in the implementation of the SDF. The third call for proposals launched in June 2013 received approximately 450 grant applications with 168 grants awarded. Disbursement had reached 20 percent in October of 2013. By December of 2013, the SDF had launched its fourth call. At this time, the total number of grants awarded under SDF had exceeded the original estimates forecasted in the PAD and available evidence pointed to the significant positive impact of the SDF in terms of increased productivity, increased investments, and employment creation, among participating firms. Further, data revealed that trainees and employers were largely satisfied with the support and capacity-building provided. 38. The MTR which took place in December of 2013, benefitted from the participation of a number of technical experts and also benefitted from the participation of various DPs, especially DANIDA. Both the Bank and the Government issued in-depth mid-term reports which were discussed during the mission and subsequently used to guide implementation during the latter years of the Project. 39. Following the MTR, strong performance on the SDF continued and implementation of activities under Components 1 and 2 gained significant momentum. Key aspects of project management – including FM, procurement and M&E – continued to improve. At the same time, additional management and administrative support was needed– in part driven by the M&E requirements related to the ever-increasing number of beneficiaries under the SDF, especially under Window 2 (informal sector). By the fall of 2014, there were 40 consultants working within the PSU (18 of which were funded by the GSTDP). Increased communication and support from the PSU, were followed by progress on activities related to COTVET’s capacity-building, hiring of TA to support select TVET institutions in drafting institutional development plans (IDPs), strengthening of the STI Directorate, and further strengthening linkages and incentives between the five research institutes (benefitting from grants under Sub-component 1.2) and industry partners. By the fall of 2014, disbursement had reached 41 percent and the SDF had launched its fifth call for proposals. DANIDA undertook an independent verification study which confirmed and validated the results which had been reported by the Project’s M&E system. 40. At around this time, on the basis of findings from the MTR and higher than anticipated costs associated with managing the SDF (due to the extensive M&E required and the higher than anticipated number of beneficiaries), the possibility of restructuring was discussed in order to: (i) introduce minor changes to the project design; (ii) formally adopt additional performance measures to gauge project performance under Components 1 and 2; and (iii) reallocate resources towards the SDF given the high demand for support under this competitive fund and the higher than anticipated cost of monitoring such a large and geographically diverse set of grantees.

41. Significant momentum was observed under Components 1 and 2 of the Project following the MTR. Specifically, Cadena was recruited to provide TA to the Project,

12

including support to: the development of relevant guidelines and manuals; the development of the National Strategy; and institutions in the development of IDPs. If this had been done earlier in the Project life, further and continued support to TVET institutions (for example, under Sub-component 1.2) in the implementation of their IDPs would have been possible. It may also have provided the opportunity for COTVET to further solidify its role in the TVET sector – where, in turn, the PSU (initially envisaged as a small unit within COTVET) continued to play a lead role in project implementation. Finally, delays in the development of the COTVET MIS – prevented the full establishment of a robust and comprehensive database on TVET provision throughout the country. 42. In the fall of 2014, the Government requested a restructuring to include a reallocation of financing to cover, in part, increased operating costs under the SDF (M&E and project administration) as well as a number of studies. The Bank requested modifications to the proposed reallocation in the spring of 2015, which was followed by another restructuring request from the Government in November 2015 – which was only approved by the Bank in June 2016. During this delay in restructuring, some activities were significantly delayed due to insufficient financing in the relevant disbursement categories. In addition to extending the project closing date by six months from June 30, 2016 to December 31, 2016, this Level 2 restructuring introduced a reallocation of funds across categories, with additional financing provided to cover management costs and operating expenditures of the SDF. This restructuring supported the continued effective implementation of the Project, in particular the functioning of the SDF and relevant M&E activities, while it also provided the GSTDP additional time to consolidate important gains that had been achieved under Components 1 and 2. By project closing on December 31, 2016, 96 percent of the credit (or SDR 43.17) had been disbursed.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 43. M&E Design. The M&E system and framework were generally sound. A number of staff were hired to oversee data collection, monitoring and reporting including one M&E specialist, two M&E officers, and one M&E assistant, located within the PSU (within COTVET). Despite this support – and refresher training courses for M&E staff – M&E required significantly more time and more field presence than had been anticipated at appraisal. The RF was strong with intermediate results indicators well-aligned with project-supported activities, capable of measuring progress under the Project and relatively easier to measure than the PDO-level indicators. These indicators were classified by Component and primarily focused on key inputs and outputs – important building blocks to achievement of the PDO. Despite these strengths, and although intermediate results indicators were included to capture progress made under sub-objective 2 (to increase adoption of new technologies), additional performance measures could have been incorporated into the RF at the PDO-level to better capture progress on Components 1 and 2. 44. M&E Implementation. While the Bank and Government considered the RF to be strong, in light of the large number of indicators and the important role of these measures in tracking progress under the Project, it was agreed that M&E would need to be further

13

strengthened. During a technical mission in June 2013, intensive support was provided to ensure that each measure was well-defined; specify the method for calculation of each indicator; develop relevant M&E templates for data collection; and to solidify accurate and timely reporting. The Government also undertook an important initiative in developing an M&E manual which would “enable effective management and accountable implementation” of the project-supported activities and would “provide a broad macro as well as micro picture of the project, clarify and confirm expected outcomes and illuminate the processes that have enabled or impeded progress towards expected outcomes.” This document produced by the PSU provides a detailed and comprehensive overview of the framework, data sources and data collection methods, roles and responsibilities of those involved in M&E at different levels. It also lays out the monitoring strategy, data verification processes and approaches to be used for data analysis while including a number of additional indicators which the Government determined would be useful in assessing the Project’s overall impact (e.g., number of productive partnerships). Though the labor market observatory was not established as had been anticipated in the PAD, the M&E system did capture productivity, earnings and investment as well as the number of jobs which were created as a result of funding under the SDF. 45. M&E related to the SDF relied on 6 M&E staff and 22 field monitors, and in addition to regular data collection and monthly monitoring there were random site visits and spot checks, which were capable of signaling any bottlenecks in implementation to the PSU. This stronger system provided the necessary evidence to systematically document implementation progress and progress made towards achievement of the PDO. These data were also used to further strengthen the Project’s implementation. For instance, the M&E system found that SDF grantees were largely concentrated in the Greater Accra region and, therefore, outreach efforts were adapted to focus more on the Northern areas. 46. In light of the above, M&E design, implementation and utilization is rated Substantial under the Project.

2.4 Safeguard and Fiduciary Compliance Safeguards 47. The Project was classified as a Category B project with the environmental impacts of activities expected to be limited since most activities involved TA or training. The social impact was expected to be largely positive – with firm growth and development driving economic growth, increased employment, and in turn, poverty reduction. Consistent with Bank procedure, an Environmental and Social Management Framework (ESMF) was developed and disclosed in country and at the Bank Infoshop on January 25, 2011. The project design also incorporated measures to limit any negative environmental or social impact of the Project, for example, the SDF manual included a negative list of activities that could not be supported under the competitive fund. During the early stages of the Project life, limited safeguards issues were encountered and often the rating on compliance was missing from ISRs. Later in the project life, however, there was increased emphasis on safeguards with training provided under the project for S&T providers under

14

Components 2 and 3 and the safeguards specialist recommended screening of sub-projects under the SDF (supported by the project’s safeguards specialist through site visits) which were undertaken by grantees. 48. Training in safeguards was also provided for lead focal persons in COTVET, PSU and MESTI. Assessments undertaken during the project life, revealed that the Project was compliant with safeguards requirements and no major impact (either positive or negative) were observed. The only major concern raised related to safeguards pertained to potential support for the construction of a building to house the STI Directorate in 2015. Once it was determined that this activity would trigger OP 4.12 (Involuntary Resettlement), it was agreed that the Project would only support minor renovations to the MESTI’s (STI Directorate) existing offices.

Financial Management

49. Following an assessment of financial management (FM) capacity during project preparation, it was agreed that an FM consultant familiar with Bank procedures would be hired to work closely with COTVET to establish robust FM systems and to provide training and TA to accounting staff. Further, core staff to support FM under the GSTDP included an FM Specialist, three FM officers, and a few finance assistants (junior and senior accountants). At the project level, efforts were made to provide limited training in FM to project grantees as well as to field-level monitors, M&E staff, and SDF intermediaries. While the initial accounting system was deemed adequate by the Bank, a new system (purchased and installed under the Project) was rolled out in 2012 which provided additional tools to systematically monitor financial transactions. Throughout the project life, as needed, additional FM staff were hired. Generally, FM performance was adequate and complied with key FM covenants of reporting. Some limitations, including incomplete and delayed IFRs, were identified and addressed in a timely manner. All external audits under the Project were unqualified though some were slightly delayed. 50. An FM review in June of 2015 identified ineligible expenditures. Though these did not pertain to GSTDP financial resources, it pertained to COTVET’s resources and, therefore, was an issue of concern to the Bank. The Government and COTVET worked closely with the Bank to resolve this issue – providing all of the necessary information related to the ineligible expenditures in a timely manner with the amount of the ineligible expenditures reimbursed in full by early September 2016. The Bank put mitigation measures in place to prevent this from happening during the remaining life of the Project. 51. In light of the above, Financial Management performance under the Project is rated Moderately Satisfactory.

Procurement

52. An assessment of procurement capacity during project preparation determined that COTVET was in compliance with procurement law. However, in light of COTVET’s limited procurement experience and the volume of procurement to be processed under the

15

GSTDP, it was agreed that the PSU would need additional procurement staff. In addition to the head of the Procurement Unit (procurement specialist in the PSU), two procurement assistants and two procurement officers were hired (with these positions funded either by the Bank or AfDB). These staff had relevant procurement experience working for other organizations. 53. While procurement performance was generally adequate under the GSTDP, due to limited capacity, there were some challenges related to record-keeping and also in determining or tracking which DPs were funding specific procurements. In addition to documenting these issues, the Bank provided recommendations to address these issues (e.g., a consolidated procurement plan) as well as training on the Bank’s procurement processes and procedures. The Bank and the PSU worked together to address identified bottlenecks as they emerged. For example, when it was determined that the rigorous procurement procedures were difficult to follow for some of the SDF grantees in the informal sector, it was agreed that the Project could apply “commercial best practices”. To this end, efforts were made to familiarize informal sector firms with basic record-keeping and with the concepts of value for money – where they were encouraged to solicit a few quotations for goods and services to be procured and to select the offer with the lower price and best value. 54. Although initially there were significant delays in recruiting the firms to provide the TA required to guide the implementation of Components 1 and 2 (including the selection and hiring of the TA firm, Cadena, which represented the largest procurement contract under the Project), these were ultimately addressed. Some delays in procurement were also, in part, further postponed when the Bank did not provide its no objection to specific procurement activities in a timely manner. Though some of these instances were likely the result of the Bank’s desire to provide a very thorough review – it posed some delays in implementation of activities. Despite these initial delays, overall procurement performance improved following the MTR and remained generally strong until project closing. 55. In light of the above, procurement is rated Moderately Satisfactory under the Project.

2.5 Post-completion Operation/Next Phase 56. A number of DPs have invested in supporting the skills development and technology development agendas of the Government moving forward, including AfDB and DANIDA. DANIDA has committed approximately US$14.3 million to supporting the SDF 2, which commenced in July of 2016 and is expected to continue through 2020. This follow-on operation builds on the SDF supported under the GSTDP, ensuring consolidation of some of the GSTDP’s achievements and applying lessons learned from the SDF implementation (providing support for proposal development, establishing a robust system for evaluating proposals and selecting grantees, and use of a strong M&E system to gauge project performance and to take corrective actions as necessary to further improve its overall impact). Further, findings from a value for money audit undertaken by DANIDA were used in its revised design. SDF 2 relies heavily on an online platform and an even

16

more streamlined SDF manual. This new SDF aims to continue building the capacity of COTVET to carry out its mandate in the sector. With support under the GSTDP and now with more staff and its current strong leadership, COTVET is well placed to guide reforms and to coordinate efforts in the sector as well as to formalize the use of the instruments that have been created under its purview.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation 57. The GSTDP’s objectives were strongly aligned with and highly relevant to the country’s needs and Government priorities for skills development and STI. Specifically, the GSTDP’s focus on promoting demand-driven skills development and increasing the adoption of new technologies as a way to increase productivity, earnings, and economic growth was relevant and aligned with the Government’s strategic agenda and sought outcomes under Medium-Term National Development Policy Framework (2010-2013). It was also relevant to the CPS FY2013-2016 which focused on increasing skills development, creating jobs and promoting the country’s economic growth. These objectives remain highly relevant today as evidenced in the draft National Plan on Education (2016-2030). The Project aimed to address the pressing barriers to labor productivity which were continuing to limit the performance of firms and enterprises across economic sectors. 58. The relevance of the Project’s design was sound as it supported efforts to: i) address supply and demand aspects of skills and technology, in formal and informal sectors; (ii) stimulate private sector development; (iii) build institutional capacity of key agencies including the COTVET and the STI Directorate; (iv) provide support to improve the capacities of TVET institutions and research institutes/S&T providers; and (v) support a competitive and demand-driven SDF. The design drew on analytical work and integrated lessons learned in Ghana and through other investments in the TVET sector which aimed to improve the quality of skill training programs, their relevance to the labor market, and to improve competitiveness through an increased reliance on new technologies. Further, experience in the design and implementation of competitive skills funds – similar to the SDF – was used to establish appropriate and transparent processes for screening and selecting fund grantees. The Project’s emphasis on capacity-building of COTVET and STI Directorate was sound in light of their respective mandates and the Government’s vision for these two entities and sectors moving forward. The Project’s scope was impressive as it included most of the elements essential to a well-functioning and demand-driven TVET system. Despite these positive features of the design, the activities under Components 1 and 2 were quite ambitious as they aimed to both reform the TVET and STI systems and to provide targeted support to a select number of institutions. Greater emphasis on and attention to their sequencing, increased selectivity of activities to be supported under these Components, and explicit linkages between these Components, their Sub-components, and the overarching project objectives, would have been beneficial. 59. In light of the above, the relevance of the GSTDP’s objectives is rated High and the relevance of its design is rated Substantial.

17

3.2 Achievement of Project Development Objectives 60. The following section provides an overview of achievement under the Project by sub-objective. Sub-objective 1: To improve demand-driven skills development 61. Through Component 1, the Project sought to increase the Government’s institutional capacities in planning, coordination, quality assurance to improve the quality and relevance of skills training programs. Specifically, through Sub-component 1.1, the GSTDP strengthened the capacity of the COTVET which was appropriate in light of its institutional mandate. 62. Fostering and supporting demand-driven skills development would require an entity which was efficient and capable of ensuring not only the provision of high quality TVET, but also capable of ensuring that the direction and provision of training was well-aligned and, therefore, informed by industry needs. As such, COTVET would need to possess the technical capacity and tools to guide TVET programming. It would also need to have a strong ability to guide TVET programs to be more responsive and useful in meeting industries’ needs for skills. In addition to supporting COTVET in acquiring the necessary tools to carry out this leadership and coordination role under Sub-Component 1.1, the Project also strengthened the capacity of COTVET to work alongside TVET institutions under Sub-component 1.2.

63. Progress towards achievement of these objectives—which were supportive of the overarching objectives of the Project –was as follows: The National TVET Strategic Plan (2015-2025) was developed and approved by the council though not adopted by the Cabinet (partially achieving the end-of-project target); and an MIS managed by COTVET was established which is currently being used as a data repository and registry for TVET (though it remains to be scaled-up nationwide, partially achieving the end-of-project target). While the establishment of the system signified an important achievement under the Project, further development of the system is being supported by the Government which will allow COTVET to collect additional data on key aspects of TVET programs. Further, under Sub-component 1.1, 23 manuals and guides for development and implementation of TVET programs using CBT were developed to steer a standardized assessment and certification of TVET delivery and are readily available (achieving the end-of-project target) while a quality assurance manual was also developed for the institutionalization of quality assurance in TVET institutions in Ghana. 64. Support was also provided by both Cadena and COTVET to 10 TVET institutions9 in preparing their institutional development plans (IDPs) which touched on key aspects of TVET institutional planning and management – including financing and revenue generation, decision-making, and steps to improve the quality and relevance of TVET

9 Wa Technical, Kumasi Technical, Kpando Technical, Kumasi VTI, Yamfo VTI, Assin Fosu VTI, Sunyani CDVTI, OIC, G-Takoradi, ICCES Agomeda Center and Adidome Farm Institute.

18

programs. The TA was a collaborative process supported by Cadena and COTVET and the relevant TVET provider. Specifically, the TA supported these institutions in identifying weaknesses of the training environment and analyzing trends in key aspects of the institution (staffing, students, quality of training). This information was used to work with institutions in determining strategic priorities and identifying actions to achieve them. The IDPs developed under the Project were adopted and implemented (achieving end-of-project target), to the extent possible given available financial resources though each of the 10 TVET institutes also received some additional support from the GSTDP for quality upgrades including basic tools and equipment, and minor renovation works. Despite the delays in hiring the technical firm (Cadena) to support these activities, all plans were completed and are being used to guide the institutions moving forward. 65. In addition, the following was also achieved towards improving the capacity of the COTVET as well as the capacity of other TVET providers to design and implement relevant (demand-driven) training programs: manuals were developed for TVET M&E; a legislative instrument (L.I 2195) was adopted that provides a regulatory framework for the TVET sub-sector; a guide for the implementation of recognition of prior learning (RPL) was developed with the involvement of key industry stakeholders and a manual to operationalize the National TVET Qualifications Framework (NTVETQF) was developed. Further, in the context of the National Education Sector Annual Review (NESAR) and other international education fora, COTVET presented this framework to relevant stakeholders. In addition, various trainings were provided, to: 56 managers of TVET institutes and representatives of some TVET agencies on institutional development planning; 36 TVET instructors in the implementation of Recognition of Prior Learning in TVET institutions; COTVET staff and management of the 10 selected institutions (under Sub-component 1.2) to effectively use and manage the MIS to create and disseminate timely and accurate results/reports; 30 TVET managers in leadership skills, FM, and time management; 170 TVET institute managers and instructors on the implementation of quality assurance in TVET delivery; and 30 supervisors from 12 TVET agencies on M&E. 66. Table 1 shows the level of achievement on each intermediate results indicator in the RF pertaining to Component 1.

Table 1: Level of Achievement of Intermediate Results Indicators (Component 1)

Indicator Target Achievement National Skills Strategy developed and adopted

Yes Target partially

achieved COTVET management information system for skills development is functional

Yes Target partially

achieved Guidelines for quality assurance systems including certification are developed and adopted

Yes Target achieved

Institutional Development Plans for training providers developed and implemented

10 Target achieved

19

Sub-objective 2: Increase the adoption of new technologies in selected economic sectors 67. As a core element to increasing the adoption of new technologies in selected economic sectors, the GSTDP aimed to strengthen the STI Directorate in becoming the key entity for coordinating and leading STI in Ghana – as laid out in the STIP. In addition to strengthening the capacities of the new STI Directorate to fulfil its role in the planning, management and coordination of STI policies (under Sub-component 2.1), the GSTDP would also support the strengthening of S&T Providers (Sub-component 2.2). 68. The following progress was made on each of the intermediate results indicators pertaining to Component 2: STI Directorate was established will full complement of staff (achieving end-of-project target); and 36 new collaborations were established between supported technology providers and private sector for adaptation and diffusion of technology. However, an information system to be used to provide policy makers with up to date information on financing, performance and impact of technology providers and the wider STI system which had been envisaged in the original project design was not developed (though training in MIS was provided) (not achieving the end-of-project target). Nonetheless, a national baseline survey was undertaken to establish a benchmark for assessing progress in STI in Ghana with data collected from eight research institutions.

69. In addition, under this Component, five technology centers in research/science and technology institutes – Ghana Atomic Energy Commission (GAEC), Kumasi Polytechnic, University of Ghana, Center for Scientific and Industrial Research (CSIR) and Ghana Technology University College (GTUC)– were selected to receive grants of US$500,000 each. Within each of these centers – technology development, innovation, transfer and marketing centers were established; 50 technologies were profiled for transfer and commercialization/marketing; 200 researchers were trained in technology transfer and commercialization/marketing, intellectual property processes and industry engagement techniques; and 10 industry-research partnerships were established and 10 industry-research seminars organized. In addition, technology transfer guidelines and strategies were developed by three of the centers and business plans developed by two of the centers and guidelines and strategies for technology transfer were developed by three of the centers (CSIR, GAEC and Kumasi Polytechnic) while one institution developed a revolving fund where student researchers could apply for funding. While the implementation of this sub-component was delayed, important progress was made in strengthening the relationship between the private sector/industry and these research centers.

70. Table 2 shows the level of achievement on each of the intermediate results indicators in the RF pertaining to Component 2.

20

Table 2: Level of Achievement on Intermediate Results Indicators (Component 2)

Indicator Target Achievement STI Directorate established with full complement of staff

Yes Target achieved

New collaborations between supported technology providers and private sector for adaptation and diffusion of technology

20 Target exceeded with 36 collaborations developed

Information System available for STI Directorate use

Yes Target not achieved

Contribution of the SDF to achievement of sub-objectives 1 and 2 71. The level of achievement on each of the PDO-level indicators shows the higher-order impact of the GSTDP achieved through its support to market-relevant (demand-driven) skills development and adoption of new technologies. Specifically, these indicators provide evidence of increases in overall productivity in key economic sectors in Ghana – critical to the competitiveness of its industries and economic growth. 72. A total of 646 sub-projects were supported under the SDF (exceeding the end-of-project target of 440). Of these, 617 have been successfully completed (also exceeding the end-of-project target of 440). Under the SDF, there were five calls for proposals/awards as follows: Call 1 (2011– 2012); Call 2 (2012 – 2013); Call 3 (2013); Call 4 (2014 – 2015); and Call 5 (2015 – 2016). On the basis of a thorough review and approval process as detailed in Annex 2, grantees were selected. 73. The large majority of SDF grantees worked in the informal sector. In many instances the grant recipient under Window 2 (informal sector) was an association, with the “firm” referring to the master craftsmen and the number of participants (beneficiaries under this window) referring to the master craftsmen and their colleagues/apprentices who also received training under the grant. The largest number of grants were provided in the transport and storage industry, followed by the tourism and hospitality, and service sectors. Matching funds in the amount of GH₵ 20.81 were invested by SDF grantees (either a monetary or in-kind contribution). 74. The total number of beneficiaries greatly exceeded the end-of-project target by a significant amount reaching more than 105,085 beneficiaries (receiving training under the SDF) relative to an end-of-project target of 24,000 beneficiaries (see Table 3 for projected and actual number of beneficiaries with disaggregation by gender). The number of ap The proportion of female beneficiaries of around 31.5 percent was lower than the initial end-of-project target of 50 percent. The proportion of females varied during project implementation and was around 47 percent in the last year of the Project. However, the provision of training to a large number of individuals in the Road and Transport Sectors (a largely male-dominated industry) is likely responsible for the decrease in the proportion of females out of the total number of beneficiaries by Project closing.

21

75. Initially, grantees were concentrated in the Greater Accra area and, to ensure a more equitable distribution of funding, increased efforts were undertaken to raise public awareness of the SDF as laid out in the SDF Communications and Outreach Plan developed under the Project. In addition to maintaining a strong media presence (through TV, radio and print media), as part of awareness raising of the SDF and its successes, a Results Forum was held with participants including grantees, members of the business community, members of Parliament, DPs and the media. In addition to this, a number of SDF fairs and business fora were held which were widely attended and intermediaries hired by the SDF in addition to supporting potential applicants in developing proposals, also aimed to improve public awareness and overall demand for SDF funding.

Table 3: Expected and Actual Number of Trainees by window and gender

Window Expected Number of Trainees

Actual Number of Trainees

% of actual against

Expected

Actual Number of males Trained

Actual Number

of females Trained

% Female

1 2,535 8,352 329.47 5,457 2,941 35.21

2 2,535 88,856 3505.17 60,771 28,085 31.61

3 2,535 3,110 136.65 2,521 614 19.74

4a 2,984 2,847 95.41 2,260 604 21.22

4b 440 1,566 355.91 904 662 42.27

Total 11,029 105,085 952.81 72,027 33,146 31.54 Increased Productivity among Targeted Beneficiaries 76. Analysis of available data provides evidence that the productivity (as measured by output per labor hour and the value placed on goods and services produced by SDF grantees) increased significantly during the life of the Project. The following tables provide an overview (by firm sector and firm size) of productivity gains. In order to best capture the full impact over time, a random sample of a quarter of the beneficiaries from Calls 1 and 2 (and representing each window) were tracked year-on-year and the productivity calculations are based on these data. Increases in productivity are observed across all windows for most years (see Tables 4 – 7 below). 77. Table 4 provides a summary of productivity gains observed by window as measured by output per labor hour (in GH₵)10. As can be seen from the table, productivity (in terms of output per labor hour) increased for most windows across the life of the Project. These increases in productivity may be in part explained by the positive impact of the GSTDP on increasing financial and human resources (capacity) through the SDF. Table 5 shows the aggregate increase in productivity from 2013 to 2016.

10 This measure is used for productivity calculations under the SDF. For additional details on the precise figures used to arrive at the aggregate values, see Annex 2.

22

Table 4: Summary of Productivity by window (2013-2016) (in GH₵)

Window 2013 2014 2015 2016

1 11.01 16.30 99.73 211.54

2 5.21 4.37 4.1 2.54

3 18.15 108.46 87.09 285.75

4a 20.92 35.03 4.76 3.29

4b 2.26 5.28 64.58 100.56

78. The year-on-year increases in productivity across all sectors was as follows: 1) 2013-2014: 104.28 percent; 2) 2014-2015: 67.38 percent; and 3) 2015-2016: 53.89 percent.11 Annex 2 provides additional details on the productivity gains observed between 2013 and 2016 by firm size and sector.

Table 5: Overall increase in productivity between 2013 and 2016 under the SDF

Productivity (in GH₵) % change

Dec. 2013 Dec. 2016

12.69 66.78 426.20% Increased investment among participating firms 79. An increase in investment by participating enterprises in skills and technology development was also observed during the Project – demonstrating the extent to which SDF beneficiaries themselves invested additional resources in either skills training or innovative technology. At the inception of the Project, the firms reported little investment in skills or technology innovations as part of their businesses. There was, however, significant increase in such investment over the life of the Project, by the end of the Project the total investments from participating firms through matching grants was equal to GH₵ 20.81 million with contributions by window as follows: Window 1: GH₵ 2.57 million; Window 2: GH₵ 6.74 million; Window 3: GH₵ 1.61 million; Window 4a: GH₵ 9.09 million; and Window 4b: GH₵ 0.79 million. As such, the increase in investment between 2014 and 2016 was 129 percent. For disaggregated investment data by firm size and sector, see Annex 2. Additionally, based on data collected on employment generation from 2014, 2015 and 2016, it is estimated that firms (formal and informal sectors) benefitting from the SDF hired around 50,000 people from 2013 – 2016 though this cannot be directly and/or solely attributed to Project support. 80. The satisfaction of trainees and employers with skills acquired and provided was also very positive. This assessment of satisfaction of trainees used a self-report measure

11 Year-on-year increase in productivity calculations are based on the rate of change in productivity between the two years under consideration: [(final year value – prior year value)/prior year value] x 100

23

on their level of satisfaction in terms of the training’s relevance in four competency areas (practical, behavioral, technical and communications) using a 5-point Likert scale (very dissatisfied, dissatisfied, somewhat dissatisfied, somewhat satisfied and very satisfied). There was also a measure to gauge employees’ (i.e., trainees’) satisfaction with the completeness (sufficiency) of the training provided. While trainees were tracked under each call, a random sample (N =312) was tracked across the life of the Project. Overall, 98 percent of trainees sampled (both male and female) were somewhat or very satisfied12 with the training provided. Females were only slightly more likely to indicate that they were overall very satisfied than males (54.7 percent compared to 52.9 percent). Given that the large majority of respondents were satisfied with the training, there was limited variability across regions in Ghana. However, it is important that when we look level of satisfaction, the individuals residing in Greater Accra, Central and Volta were more likely to be very satisfied than individuals in other regions, particularly those in Northern, Eastern, Upper East and West regions. 81. The satisfaction of employers with employees’ skills was also gauged using the same 5-point Likert scale with level of satisfaction being a composite measure of satisfaction with trainees’ skills across the four competency areas mentioned above. Similar findings were observed in terms of employer satisfaction with trainees’ acquired skills. Across all regions, around 97.5 percent of employers reported being very satisfied with their employees’ competencies. A review of the data shows some variation by region and by economic sector. Specifically, the proportion of employers being very satisfied with their employees’ competencies being higher in Greater Accra (in and around the capital city) than in the Northern and Eastern areas. Further, it shows that the level of satisfaction among employers as a whole was greater in the construction sector than in the services and manufacturing sectors. Moreover, employers in smaller firms expressed more satisfaction with their employees’ skills than those in larger firms with 100 percent of small firms being very satisfied and only 60 percent of larger ones reported being very satisfied.

82. Most of the intermediate-results indicators were exceeded: 361 industry/provider collaborations were established under Window 1 (exceeding end-of-project target of 200); 122 new training courses were developed and 46 partnerships were established (exceeding end-of-project target of 20); and 549 collaborations were established between enterprises and technology centers (total for all centers receiving project support) under Window 4 (exceeding end-of-project target of 60). However, under the SDF fewer technologies (70) were adopted by participating firms (not achieving end-of-project target of 120) and an indicator related to certification was not achieved Since a large number of individuals received competency-based training prior to the formal establishment of the National TVET Qualifications Framework whereby the certifications for completion of the training (received by 90 percent of trainees) could not be formally recognized as adhering to the NTVETQF criteria.

12 Percentage satisfied refers to those individuals who answered that they were very or somewhat satisfied with the relevance and completeness of training divided by the number of people included in the sample.

24

3.3 Efficiency 83. Initial implementation delays during the first two years of the GSTDP including in building the capacity of the PSU; setting up the M&E systems to monitor the SDF grants and baseline data collection; contracting critical TA required for COTVET capacity strengthening; and setting up the MIS under Components 1 and 2 led to reduced overall implementation efficiency. As a result of these initial delays, the Project closing date needed to be extended by six months. As of project closing, approximately US$1.5 million remained undisbursed and was, therefore, cancelled. These combined results reduced the overall implementation efficiency of the Project. 84. Despite these challenges, the data show important efficiency gains related to: increases in labor productivity of participating firms; increases in investment by participating enterprises in skills and technology development; satisfaction with market-relevant skills by trainees and participating firms; new technologies adopted by participating firms; and the development of collaborations between enterprises and technology centers. In addition, as mentioned previously, the Project was also able to leverage additional resources through matching grants. 85. Original estimates of the Net Present Value (NPV) and the Economic Rate of Return (ERR) were calculated at appraisal. The NPV was estimated at US$60.6 million and the ERR was estimated at 43 percent. Using the original assumptions13, the same calculations were undertaken with data collected during project implementation (as presented in the table below). Although the NPV and ERR are positive, they are significantly lower than the projections calculated at appraisal. The reason behind this is unclear but may perhaps be a function of the overly optimistic projections made at appraisal.

Table 6. Calculations for the NPV and ERR

Present Value of Flows (for Firm output)

Benefits (US$m) 67.53 Costs (US$m) 43.88 Net Present Value ($USm) 23.65 Economic Rate of Return 18.5%

86. In addition, the SDF appears to be associated with increased employment. Based on data collected on employment generation from 2014, 2015 and 2016, firms (formal and informal sectors) benefitting from the SDF hired around 50,000 people from 2013 - 2016. Though this cannot be solely and/or directly attributed to the Project. Direct financial benefit to the Government will also be in the form of corporate and income taxes – with the net fiscal benefits of the GSTDP to the Government estimated at US$24.18 million. It

13 Grant support will increase efficiency of the beneficiaries and yield economic output double the amount of support provided by the grant; A lag time of one year was used for firm output; and for a 12 percent discount rate and a 12-year project horizon

25

is assumed that on top of the repayment costs, the sustainability of the Project will require additional support costs equal to about 5 percent of the Project per year for 10 years (2016-2026) following the initial investment phase. Despite some of the important efficiency gains realized under the Project, and the fact that many of the outputs and outcomes of the Project were achieved by project closing, implementation efficiency was modest, operating costs for the Project (specifically, the PSU) were higher than what had been anticipated at appraisal, and some project resources remain unspent and needed to be cancelled. 87. Notwithstanding the positive NPV and ERR, and other efficiency gains realized under the Project, given the delays in implementation under Components 1 and 2 as well as the cost overruns under Sub-Components 3.2 and Component 4, the overall efficiency under the Project is rated Modest.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 88. The project design was aligned with the Government’s objectives for the skills development and STI sectors, and through activities supported under each of the Project’s components, had an important impact on strengthening a system which supported demand-driven skills development and introducing new technologies. Further, the Project further strengthened linkages between research institutes and the private sector. Important progress was observed under the GSTDP with the large majority of outputs and outcomes achieved (and in some cases exceeded) by project closing. As such, the design included both a competitive fund as well as the key elements to further strengthen the TVET and STI sectors. 89. While the Project did achieve critical outputs and outcomes, there were some limitations in the design – including an ambitious timeline to effectively implement all of the activities specified under Components 1 and 2 and insufficient performance measures formally included in the RF to more fully gauge the overall impact of activities supported under these Components. The focus on capacity-building of COTVET and strengthening of linkages between research institutions and industry, however, was critical in ensuring that skills development became more demand-driven and in ensuring that new technologies were introduced to industries, in the longer term. 90. Based on the composite score of the sub-ratings for Relevance of Objectives and Design, Efficacy and Efficiency, the Overall Outcome Rating for the Project is Moderately Satisfactory.

Table 7: Overall Outcome Rating

Relevance of Objectives and Design

Efficacy Efficiency Overall Rating

High/Substantial Substantial Modest Moderately Satisfactory

26

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 91. Significant efforts were made to ensure access of poor and vulnerable populations in Ghana to the SDF. In particular, when monitoring under the SDF revealed that a smaller number of grants were being provided to the northern regions (Northern, Upper East and Upper West – tend to have higher poverty rates) was much lower than other regions, specific efforts were undertaken to undertake outreach efforts in these regions. Additionally, the SDF funded a large number of intermediaries to properly reach and provide support to those applicants requiring assistance in developing proposals which met the criteria for SDF funding. (b) Institutional Change/Strengthening 92. Institutional strengthening was an important element in the GSTDP design. With the support of GSTDP – through direct capacity-building support (TA) and other project-supported activities – both COTVET and MESTI have increased their capacity to fulfill their institutional mandates in their respective sectors. For COTVET, in addition to developing and being guided by a comprehensive TVET strategy, several manuals have been developed, trainings provided for and by COTVET, and fruitful collaborations have been established with a number TVET institutions (in the design of their IDPs). In addition to its expansion in size since project inception, COTVET has developed a five-year costed strategic plan to guide its efforts in the sector and since project closing has continued to maintain its visibility and involvement in donor-supported initiatives. Capacity within the STI Directorate (of MESTI) has also been strengthened under the GSTDP. Through direct project support and its close collaboration with various research institutes and technology providers, the MESTI has developed its capacity to promote collaborations between research institutes/S&T and industry and to promote transfer and adoption of new technologies among various industries. (c) Other Unintended Outcomes and Impacts (positive or negative) Not applicable.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Please refer to Annex 5.

4. Assessment of Risk to Development Outcome Rating: Moderate 93. Institutional capacity was strengthened as a result of the training provided under the Project, specifically with regard to the development of the National TVET Strategic Plan (2015-2025) and the STIP which provide a framework for the Government to

27

undertake solid and viable reforms within the TVET and STI sectors. Today, the COTVET has strong leadership, additional staff, and as mentioned above, a five-year costed strategic plan to achieve a number of its corporate goals – including, among others, to ensure quality assurance and sustainable financing of TVET. The STI Directorate is staffed with personnel with requisite skills to more effectively carry out its mandated responsibilities. COTVET is also taking steps, including the establishment of inter-governmental TVET working group, to ensure that all relevant ministries are included in the continued development of policies and activities in the TVET sub-sector. The Government today has expressed a high level of commitment to ensuring that the provision of skills is demand-driven and guided by skills needs for the 21st century. Maintaining gains observed under the Project and continuing to promote the adoption and transfer of new technologies, and to ensure stronger linkages between the private sector and TVET providers, continue to be core aims of the Government in the TVET and STI sectors. 94. The SDF, on the basis of available data –has been highly successful and has shown very positive results. Together, this intervention has reaped important gains in productivity – and grantees continue to benefit from the use and transfer of new technologies and stronger partnerships in skills development. Many grantees have provided matching funds which further highlights their commitment to sustaining and scaling up the achievement which were realized under this Project. This further reduces the risk to the development outcome moving forward as firms have shown not only the ability but also the propensity to provide additional investments to improve skills development and productivity within their firms. 95. SDF 2 – supported by DANIDA with funding in the amount of approximately US$14.3 million – began in July of 2016 and continues to build on the important achievements realized under the GSTDP. While this is an important extension of the SDF and supports the COTVET, and applies similar arrangements, it does not rely on all of the PSU staff which supported the GSTDP. However, other DP-funded activities, continue to rely on the PSU for management of operations in the sector. Further, as result of capacity-building support to COTVET, it is better equipped to guide and coordinate the TVET sector moving forward. As such, the institutional and human capacity built under the GSTDP will continue to be maintained in core domains, including FM, procurement, project management, grant administration, and M&E, and in the area of skills development and technology development, transfer and adoption. 96. Given the participatory nature of preparation and input from diverse stakeholders in the design, the buy-in and support for the PDO within the Government and among a number of partners is high moving forward. Indeed, support from various DPs including AfDB and DANIDA is expected to continue to be maintained in the future. DANIDA is continuing to provide support to the skills and STI sectors through its contribution to SDF II while AfDB continues to provide support through its Development of Skills for Industry Project (DSIP) (US$125 million).

97. On the basis of the above, the Risk to Development Outcome is rated Moderate.

28

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 98. The Project’s design and objectives were strongly aligned with the Government’s TVET and STI strategies. The Bank team worked closely with the Government, DPs and research institutes and private sector partners to design the GSTDP in order to ensure proper alignment and buy-in from the various stakeholders operating in Ghana. The original design was informed by a series of analytical studies and lessons learned from Ghana and from other relevant sectors –benefitting from extensive technical support from experts across all of the relevant thematic areas (TVET, S&T, private sector, competitiveness, etc.) Efforts were made to strengthen buy-in from the Ministry of Education as well as the Ministries of Environment, Science and Technology, and Employment, among others, with the aim of ensuring that their technical and strategic inputs were incorporated into the Project’s final design. It appears that the efforts to include a number of different ministries in the initial phases of the Project design may have led to increased collaboration among them in the area of skills development and STI transfer and adoption. 99. The design of the GSTDP focused on reviewing and revising sector strategies and policies as well as providing targeted support to both government institutions and other beneficiaries (TVETs supported under Sub-component 1.2, research institutes supported under Sub-component 2.2 and Grantees supported – under Component 3). At the same time, the linkages between the various Components and Sub-components could have been made more explicit in the PAD and additional performance measures could have been included at the PDO-level to more fully assess the Project’s performance under Components 1 and 2 though the Project’s RF included relevant indicators at the intermediate level to measure the level of achievement across these Components. 100. In light of the above, Bank performance at ensuring Quality at Entry is rated Moderately Satisfactory. (b) Quality of Supervision Rating: Moderately Satisfactory 101. Implementation support missions were carried out in a timely manner and benefitted from relevant sector experts. Field missions often included visits to SDF beneficiaries. The ISRs provided a candid overview of key challenges confronted by the Project including: difficulties in establishing the PSU; identifying and hiring TA for the effective implementation of Component 1 and 2; and initial delays in establishing a functional M&E system capable of capturing progress across each of the GSTDP’s components. While the Bank was effective in identifying these key challenges to project

29

implementation and was candid in its reporting in ISRs, initially it was unable to assist the Government in finding solutions to these issues. It was only subsequently when the team was further strengthened, that effective solutions to these challenges were agreed and taken on board. 102. The MTR and related documentation were thorough, providing a comprehensive overview of the achievements of the Project as well as some of the bottlenecks, as described above, which were hindering the timely implementation of some key project-supported activities. Around the same time, the Bank provided targeted technical support to strengthen the larger M&E framework – ensuring that each measure was well-defined and that the methods for calculation of indicators were clear; developing relevant M&E templates for data collection; ensuring accurate and timely reporting. 103. During the MTR, the Bank had highlighted the need to consider adjusting aspects of the Project’s design – making links between the components more explicit and incorporating additional PDO-level indicators to capture progress under Components 1 and 2. While these revisions were not formally introduced through a restructuring of the Project, with the support of the Bank, the Government did make efforts to strengthen the linkages between the Components and Sub-components and continued to strengthen M&E to allow for a more systematic and thorough assessment of achievements observed under the Project. In addition to the Washington, DC-based TTL in the earlier part of the Project, having a dedicated group of staff members, including a senior education specialist and fiduciary specialists in Accra – helped the Bank provide ongoing support the client on a day-to-day basis. Though the Bank’s performance across all relevant areas (FM, procurement, safeguards and M&E) was generally strong, more attention should have been given to consistently providing disaggregated data in ISRs. 104. Despite some of the initial challenges described above, the Bank’s performance was generally strong and supported the Government’s efforts to reform the TVET and STI sectors and to successfully implement the demand-driven SDF. 105. In light of the above, Bank performance during Supervision is rated Moderately Satisfactory. (c) Justification of Rating for Overall Bank Performance 106. Based on the above, Overall Bank Performance is rated Moderately Satisfactory. Rating: Moderately Satisfactory

5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 107. The Government’s support of the Project was evidenced in a number of activities during the preparation phase, including: the establishment of the interdepartmental group

30

to support preparation of the project; the assignment of various staff from different ministries to play key roles in project design and implementation; and the continued and active participation in project preparation during each mission. While important progress was made during preparation – in large part as a result of Government commitment – the Project only became effective eight months after approval by the Bank since the legal opinion had not been issued (standard condition of effectiveness) and the PIM had not been finalized. The Government worked collaboratively with the Bank to define the activities which would be most beneficial for strengthening the capacity of both COTVET and MESTI in order to equip these entities with necessary tools to lead efforts in the TVET and STI sectors. The Government is committed to maintaining these achievements and to scaling up its support to both the skills development agenda and the harnessing of new technologies as a way to improve productivity and growth in Ghana. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 108. The PSU effectively managed core aspects of project implementation. While initially there were challenges in recruiting the relevant staff to carry out its role, the PSU had been significantly strengthened and was by mid-term, fully functional. In addition to the GSTDP, the PSU also managed other donor-supported projects. The intricate system developed and revised as needed to manage all aspects of the SDF implementation (launch of call for proposals, technical support to proposal development, transparent process for selection of grantees, ongoing support to sub-projects, establishment and management of robust M&E system, etc.) played an important role in the Project’s achievements. 109. COTVET worked closely with the Bank to support key aspects related to project implementation (including audits, IFRs, etc.) and, though with some delay, effectively carried out planned activities under the Project, including development and drafting of manuals, providing relevant training in key areas (quality assurance and recognition of prior learning), supporting select institutes in drafting IDPs, and establishing and managing an MIS. With its strong leadership today – it has undertaken continued efforts to heighten its visibility and to fulfill its mandate in the TVET sector as a guide and coordinating body. It has done this through the development of a corporate strategy, the establishment of an inter-ministerial TVET working groups, making its tools and guides readily available to interested parties, and encouraging TVET institutes to register within its MIS. (c) Justification of Rating for Overall Borrower Performance 110. Based on the above, the rating for Overall Borrower Performance is Moderately Satisfactory. Rating: Moderately Satisfactory

31

6. Lessons Learned 111. Close collaboration with Government, other partners and key stakeholders is critical in ensuring buy-in to project design – particularly in terms of those activities supporting sector reforms. The Bank worked closely with the Government throughout preparation to ensure that the Project design and objectives were tailored to the needs and larger Government strategies for the TVET and STI sectors. 112. While there were delays in activities related to capacity-building of the TVET and STI Directorate, the decision to incorporate such support into project design as requested by the Government, is likely to reap important dividends in introducing and guiding sustainable change in these sectors. Despite the delays observed under Components 1 and 2, which supported among other activities, capacity-building of key government institutions in the TVET and STI sectors, these entities (COTVET and STI Directorate) are now well established, have gained important visibility, and are coordinating efforts in their respective sectors. Moving forward these entities will be important and strong proponents of continued reforms in these areas and activities such as the SDF. 113. While the establishment and design of a robust M&E system is important early on – targeted technical support during implementation can be useful in addressing any limitations and ensuring adequate information to measure progress made under the Project. Though initially there were challenges with the M&E system’s ability to fully document the outputs and outcomes achieved under the operation, with targeted technical support provided by the Bank, the M&E system was significantly strengthened – and capable of collecting comprehensive dataset on key inputs, outputs and outcomes under the Project. 114. Appropriate sequencing of activities and linkages between project components and sub-components should be explicit in the PAD. The GSTDP design was very comprehensive, however, the linkages between project sub-components (under Components 1 and 2), and their sequencing, was not clearly articulated in the PAD. Further, such clarifications in the PAD are important in confirming between the Bank and Government a shared understanding of project activities, their sequencing, and objectives. 115. Competitive skills funds, such as the SDF, can guide and transform the provision and delivery of training with important dividends for industries and TVET providers alike. Available data and information obtained through the course of this evaluation, have demonstrated the important impact of the SDF in terms of an increased productivity, generation of resources and the establishment of mutually beneficial partnerships.

32

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers (c) Other partners and stakeholders

33

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components/ Sub-components

Appraisal Estimate (USD

millions)

Revised Allocation

(2016)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Component 1: Institutional Strengthening of Skills Development

4 3.45 3.42 86

1.1 Development of COTVET technical capacity, strategic systems, and policies

2 2.28 2.29 115

1.2 Support to TVET Providers 2 1.16 1.13 57 Component 2: Institutional Strengthening of Science and Technology Development

4 3.26 3.19 80

2.1 Strengthening national STI planning, management and coordination

2 0.56 0.51 26

2.2 Strengthening science and technology providers

2 2.70 2.68 134

Component 3: Financing of Skills and Technology Development through the Skills Development Fund (SDF)

50 49.27 47.96 96

3.1 Skills Development Fund 45 42.6 41.33 92

3.2 SDF Outreach and Management 5 6.67 6.63 133

Component 4: Project Management 5 5.67 5.62 112

Total Baseline Cost 63 61.65 60.18 96

Physical Contingencies 3 0.00 0.00 -

Price Contingencies 4 0.00 0.00 -

Total Project Costs 0.00 0.00 Front-end fee PPF 0.00 0.35 0.36 - Front-end fee IBRD 0.00 0.00 .00 - Total Financing Required 70* 62 60.53 86

*The original credit amount was US$70 million equivalent, by the end of the project, the US equivalent had become US$62 million due to exchange rate losses over the life of the Project. Of this amount US$1.5 million was undisbursed/cancelled. In light of the significant exchange rate losses, the last column showing the percentage at appraisal, does not highlight the higher actual expenditure as a proportion of what was estimated at appraisal – as the original estimated amount was US$70 million and the overall expenditure was much lower overall (US$60.53 million).

34

(b) Financing

Source of Funds Type of

Cofinancing Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 5.00 0.00 - International Development Association (IDA)

70.00 60.53 86

*The Government support was primarily through in-kind contributions (e.g., staff time, office space, etc.)

35

Annex 2: Outputs by Component 1. The PDO of the GSTDP was “to improve demand-driven skills development and increase adoption of new technologies in selected economic sectors.”14 It had the four following Components: Institutional Strengthening of Skills Development; Institutional Strengthening of Science and Technology Development; Financing of Skills and Technology Development Programs through the Skills Development Fund; and Project Management and Monitoring and Evaluation. 2. The GSTDP was financed by an IDA credit in the amount of SDR 44.9 million (US$70 million equivalent), was approved on March 29, 2011, became effective on November 23, 2011 and closed on December 31, 2016. 3. The following provides an overview of the outputs and outcomes achieved under the GSTDP. Component 1 Institutional Strengthening of Skills Development 4. The objective of Component 1 was to build systems and strengthen the capacities of the Government and the training providers in planning, coordination, quality assurance and service delivery towards improved quality, relevance, accountability and effectiveness in skills development. As a result, the roles and responsibilities between Government and training providers were to be realigned to make sure skills development was less fragmented, and more demand-driven with both being more capable of achieving and demonstrating better performance in promoting relevance, efficiency and accountability in skills development. This Component was comprised of two sub-components – Sub-component 1.1: Development of COTVET technical capacity, strategic systems and policies and Sub-component 1.2: Support to TVET Institutions – through which the Component aims would be achieved. Sub-component 1.1: Development of COTVET technical capacity, strategic systems and policies

5. This Sub-component aimed to support efforts to strengthen Government systems and capacities towards a more demand-driven, efficient and effective TVET system. Under this sub-component institutional development and capacity-building activities would focus on the COTVET, set up in 2006 by law as an intergovernmental agency responsible for, among other things, TVET policy, quality assurance, and monitoring. These responsibilities were in line with the changing role and capabilities of the Government in a more demand-driven TVET system.

6. Rather than being a provider of publicly financed and regulator of all TVET programs (public and private), COVET would have the capacity to: (i) provide overall

14 The PDO in the Financing Agreement was as follows: “to improve demand-driven skills development and to increase adoption of new technologies in selected economic sectors of the Recipient.

36

vision, plans, budgeting, financing guidelines, policies, incentives to stimulate demand and to improve cost-effective responses by public and private providers to this demand; (ii) provide on-time information for decisions, training providers, beneficiaries and stakeholders; and (iii) assure quality across sectors (public and private) and types and levels of training through standards, qualification/certification guidelines and services.

7. The new governance system for skills development and COTVET’s capacities were to be developed within a larger program which benefitted from support through a number of initiatives, including projects supported by JICA, DANIDA, and the Korean Government and planned support from AfDB and GTZ. COTVET would also be responsible for harmonization of these different activities and efforts towards TVET reform.

8. The following progress was made on each of the intermediate results indicators related to Sub-Component 1.1: The National Skills Strategy was developed by 2015 and approved by the Council (though not adopted by Cabinet, partially achieving the end-of-project target); and the COTVET MIS had been established which can be used as a data repository for TVET (though it was not fully developed – with linkages with 23 institutions; partially achieving end-of-project target). This system would be critical in allowing COTVET to fulfill its mandate to report on all TVET institutions across ministries and sectors and to provide up-to-date information of the financing, performance and impact of TVET in the country. Further, under this sub-component, in addition to further building the Council’s website, 23 manuals and guides15 for development and implementation of TVET programs using CBT, certification processes were established and quality assurance mechanisms have been put in place (achieving end-of-project target).

9. In addition to this, the following was accomplished in an effort to strengthen COTVET and TVET systems and policies: legislation (L.I 2195) was adopted which provides a regulatory framework for the TVET sector; a guide for the implementation of recognition of prior learning (RPL) was developed with key industry stakeholders; and manuals for TVET M&E and to operationalize the National TVET Qualifications Framework (NTVETQF) were developed. COTVET has also developed equivalencies for all qualifications that existed prior to the implementation of this framework. Further, in the context of the National Education Sector Annual Review (NESAR) and other international education fora, COTVET has presented this framework to relevant stakeholders.

Sub-component 1.2: Support to TVET institutions

10. Support through Sub-component 1.2 aimed to improve accountability, and institutional effectiveness of formal public and private non-profit TVET providers in priority economic sectors to deliver demand-driven training. This was to support TVET providers in modifying the training they provided and related services to better fit within a demand-driven system. This required changes in their mandates, management and reliance on input-based financing from Government to becoming more dynamic and innovative

15 Some of the manuals included: Guide to Certification and Approval of Occupational Standards and Qualifications.

37

providers of skills development services with real-time linkages to the skills market, a capacity to focus on outcomes (such as changes in labor productivity and employability) and responsibility for improving the effectiveness of services provided.

11. TA provided under this Component by a technical firm (Cadena) and COTVET supported 10 TVET institutions16 in preparing their institutional development plans (IDPs) which touched on key aspects of TVET institutions—including financing (and revenue generation), planning, decision-making, and steps to improve the quality and relevance of their TVET programs. These IDPs and their implementation aimed to lead to improved TVET services in terms of their economic and labor market relevance (better placement of trainees, improved training programs, other services related to skills development including on-the-job programs, guidance systems, practical training solutions) and/or cost-efficiency as well as accountability. The IDPs developed under the Project were adopted and implemented (achieving the end-of-project target) and each of the 10 TVET institutes also received some additional support from the GSTDP for quality upgrades including basic tools and equipment, and minor renovation works. In a context of limited resources, however, the TVET institutions have not been able to fully implement all of the activities included in the IDPs although progress has been made on a number of activities included within the plans.

12. A number of relevant trainings were also provided under this Component, including to: 56 managers of TVET institutes and representatives of some TVET agencies on institutional development planning; 36 TVET instructors in the implementation of recognition of prior learning in TVET institutions; 30 TVET managers in leadership skills, FM, and time management; 170 TVET institute managers and instructors on the implementation of quality assurance in TVET delivery; 30 supervisors from 12 TVET agencies on M&E; and COTVET staff and management of the 10 selected institutions to effectively use and manage the MIS to create and disseminate timely and accurate results/reports.

Component 2 Institutional Strengthening of Science and Technology Development 13. The Ministry of Environment, Science, Technology and Innovation (MESTI)17 is responsible for management and implementation of the Government’s STIP. 18 The Ministry established an STI Directorate within MEST with the mandate to plan, manage, and coordinate implementation of the country’s STI policies and programs. 19 This Component of the GSSTDP aimed to: (i) strengthen the planning, management, and

16 Wa Technical, Kumasi Technical, Kpando Technical, Kumasi VTI, Yamfo VTI, AssinFosu VTI, Sunyani CDVTI, OIC, G-Takoradi, ICCES Agomeda Center and Adidome Farm Institute. 17 At project appraisal, its name was the Ministry of Environment, Science and Technology (MEST). 18 A national STI Policy was completed in 2010 under the guidance of MEST and with the participation of a broad set of Ghana’s STI stakeholders including from research bodies, universities and polytechnics, the private sector, government, NGOs, and others. 19 The mandate conferred STI Directorate with the following responsibilities: (i) formulation of national STI policies across the broad spectrum of national activities related to STI; (ii) coordination, harmonization, and monitoring of the implementation of national STI policies and programs; and (iii) provision of technical advice on STI policies and programs to the Ministry leadership and to the wider national government.

38

coordination of national STI policies and programs in order to make efficient use of existing resources and complement the national economic development plan; and (ii) support efforts to make technology development and diffusion more responsive to the needs of the economy by enhancing interactions between selected research institutes, university departments, and their external clients, primarily the private sector, thereby encouraging domestic innovation. Specifically, Component 2 aimed to support aspects of the Government’s STIP which had as its medium-term objective: “to restructure the entire science and technology set-up, infrastructure and programs in order to make them more responsive to national needs and priorities in all sectors of the economy with special attention to restructuring of the national science and technology advisory system, improving basic and applied research infrastructure, revitalizing the teaching of science and mathematics in the education system, promoting the training of a critical mass of middle-level technical personnel to address the provision of basic needs, acquisition of skills in high technology areas and their integration into known technologies, and promoting mastery of known technologies and their application in industry”. Sub-component 2.1: Strengthening national STI planning, management, and coordination

14. This Sub-component aimed to improve the planning, management, and coordination of STI policies by supporting the establishment of a well-functioning STI Directorate within MEST capable of making evidence-based STI policies and implementing priority activities in the STI Development Plan (2011-2015). Specifically, support under this sub-component would be provided to: (i) develop the technical capacity of the STI Directorate staff necessary to make evidence-based STI policies, coordinate implementation of the national STI policy with other government agencies and the private sector, and monitor STI programs and policies; (ii) develop information systems used to provide policy makers with up to date information on the financing, performance, and impact of technology providers and the wider STI system; and (iii) ensure the STI Directorate successfully implements selected activities under the STI Development Plan.

Sub-component 2.2: Strengthening science and technology providers

15. This Sub-component aimed to improve the governance and incentives of selected research institutes, universities, and technology providers to develop, adapt and diffuse technologies to private sector enterprises on a demand-driven basis. At the time of project appraisal, the linkages between technology providers and industry in the country was very weak. A number of research outcomes remained on the shelves of the universities and research institutes and were not accessible to industry. There was often a mismatch between the activities of technology providers and national industrial development goals and objectives. Universities and research institutes were often focused on fundamental research at the expense of technology adaptation and diffusion. This sub-component was designed to support the STIP’s objective to establish an effective national innovation system which includes projects for “promoting joint research between research and development (R&D) institutions (including the universities) and industry” as well as self-financing technology transfer centers at polytechnics, universities and research institutions to support the adoption of new technology and research findings. This sub-component was

39

meant to complement the funding available to technology providers under Window 4 of the SDF.

16. The following progress was made on each of the intermediate-results indicators pertaining to Component 2: STI Directorate was established will full complement of staff20 (achieving end-of-project target); and 36 new collaborations were established between supported technology providers and private sector for adaptation and diffusion of technology. However, an MIS to be used to provide policy makers with up-to-date information on financing, performance and impact of technology providers and the wider STI system which had been envisaged in the original project design was not developed (though training in MIS was provided). Nonetheless, a national baseline survey was undertaken to establish a benchmark for assessing progress in STI in Ghana with data collected from eight research institutions. 17. In addition, under this Component, five technology centers in research/science and technology institutes – Ghana Atomic Energy Commission (GAEC), Kumasi Polytechnic, University of Ghana, Center for Scientific and Industrial Research (CSIR) and Ghana Technology University College (GTUC) – were selected to receive grants of US$500,000 each in order to develop, adapt, and diffuse technology, innovations, and related knowledge to the Ghanaian public and private sectors and to support post graduate students and researchers into demand–driven, industry led technology development. These institutes demonstrated a desire to undertake market-oriented reforms and were selected in collaboration with MEST and other key stakeholders.

18. Within each of these centers -- technology development, innovation, transfer and marketing centers were established; 50 technologies were profiled for transfer and commercialization/marketing; 200 researchers were trained in technology transfer and commercialization/marketing, intellectual property processes and industry engagement techniques; 10 industry-research partnerships were established; and 10 industry-research seminars organized. In addition, technology transfer guidelines and strategies were developed by three of the centers and business plans developed by two of the centers and guidelines and strategies for technology transfer were developed by three of the centers (CSIR, GAEC and Kumasi Polytechnic) while one institution developed a revolving fund where student researchers could apply for funding.

19. Some of the technologies developed under this Sub-component included, among others: the use of greenhouse technology for the production of export-oriented chili pepper (GAEC); the use of integrated web-based applications as an efficient and low cost way of locating individuals in emergency situations and allowing users to store geospatial information of frequently visited places (GTUC); and the development of an efficient fish smoker to smoke relatively large quantities of fish within a comparatively shorter period

20 Under the Project, the Directorate was strengthened with six staff including a Director, Deputy Director, 3 program officers and a secretary and has renovated and fully equipped office space. In addition, these staff were provided with short-term training.

40

of time, under hygienic conditions and provides safety to the fish monger (Kumasi Polytechnic).

20. While the implementation of this sub-component was delayed, in turn, preventing the establishment of an MIS under the Project, the national baseline survey undertaken established a benchmark for assessing progress in STI in Ghana with data collected from eight research institutions. Further, important progress was made in strengthening the relationship between the private sector/industry and these research centers. Component 3: Financing of Skills and Technology Development Programs through the Skills Development Fund 21. The objective of Component 3 was to finance skills and technology development programs in prioritized economic sectors through a demand-driven skills development fund (SDF) managed by COTVET. Component 3 was comprised of two Sub-components. Sub-component 3.1 aimed to provide a demand-driven response to the challenges facing productive sectors of the economy by supporting grantees through an SDF with four separate funding windows – formal sector, informal sector, training and technical institutions, and S&T training service providers. Sub-component 3.2 aimed to support SDF outreach and management by funding extensive outreach activities, training and technology needs assessments, partnerships with intermediaries, and the cost of management, administration and coordination for the SDF. Sub-Component 3.1: Skills Development Fund 22. The SDF aimed to support key priorities in skills and technology development, specifically: demand driven skills and technology development; a focus on enterprise development, stimulating demand and cost-sharing employers and training providers; and policy and incentive-based Government financing rather than Government being a service provider. It aimed to put in place a more transparent and competitive system and help set up a framework for harmonized donor support. Sub-Component 3.2: SDF Outreach and Management 23. This Sub-component aimed to support extensive outreach activities, training and technology needs assessments, partnerships with intermediaries, and the cost of management, administration and coordination for the SDF. Through this Sub-Component, the Project aimed to support public awareness, community outreach and training programs to inform employers, training and technology providers and industry associations about the potential benefits, good practices and application procedures. 24. A total of 646 sub-projects were supported under the SDF (exceeding the end-of-project target of 440). Of these, 617 have been satisfactorily completed21 (also exceeding

21 Satisfactorily completed is determined on the basis of two criteria: implementation of planned activities and the closure of the sub-project.

41

the end-of-project target of 440). Under the SDF, there were five calls for proposals/awards as follows: Call 1 (2011 – 2012); Call 2 (2012 – 2013); Call 3 (2013); Call 4 (2014-2015); and Call 5 (2015-2016). Tables 1 and 2 provide an overview of SDF grantees by call and window. The large majority of SDF beneficiaries were in the informal sector. In many instances the grant recipient under Window 2 (informal sector) was an association, with the “firm” referring to the master craftsmen and the number of participants (beneficiaries/trainees under this window) referring to the master craftsmen and their colleagues/apprentices who also received training under the grant. The largest number of grants were provided in the transport and storage industry, followed by the tourism and hospitality, and service sectors. 25. Most of the intermediate results indicators were exceeded: 361 industry/provider collaborations were established under Window 1 (exceeding end-of-project target of 200); 122 new training courses were developed and 46 partnerships were established (exceeding end-of-project target of 20); and 549 collaborations were established between enterprises and technology centers (total for all centers receiving project support) under Window 4 (exceeding end-of-project target of 60). However, under the SDF fewer technologies (70) were developed by participating firms (not achieving end-of-project target of 120) and an indicator related to certification was not achieved since most training participants had received training prior to the formal establishment of the NTVETQF and, therefore, their certificates of completion of training could not be retroactively formally considered certified. 26. Significant outreach was undertaken to raise awareness of the SDF – including its objectives, eligibility criteria, and application process. This was done through a strong media presence (through TV, radio and print media), as well as various public fora, including SDF fairs. The SDF also relied on intermediaries to improve public awareness and overall demand for SDF funding while also supporting potential applicants in developing proposals. When it was discovered through project monitoring that a smaller number of grants were being awarded in the northern areas of the country, increased efforts were undertaken to raise public awareness of the SDF in these areas of the country. While it was expected that the number of beneficiaries under the Project would reach 24,000, this target was significantly exceeded reaching 105,085 beneficiaries. At the same time the proportion of female beneficiaries was around 31 percent, lower than the end-of-project target of 50 percent. The proportion of females varied during project implementation and was around 47 percent in the last year of the Project. However, the provision of training to a large number of individuals in the Road and Transport Sectors (a largely male-dominated industry) is likely responsible for the decrease in the proportion of females out of the total number of beneficiaries by Project closing. Table 1 provides an overview of expected vs. actual number of trainees (disaggregated by gender) and Table 2 provides an overview of trainees by window.

42

Table 1: Expected and Actual Number of Trainees by SDF Window

Window Expected Number of Trainees

Actual Number of Trainees

% of Actual Against

Expected

Actual Number of Males Trained

Actual Number

of Females Trained

% Female

1 2,535 8,352 329.47 5,457 2,941 35.21

2 2,535 88,856 3,505.17 60,771 28,085 31.61

3 2,535 3,110 136.65 2,521 614 19.74

4a 2,984 2,847 95.41 2,260 604 21.22

4b 440 1,566 355.91 904 662 42.27

Total 11,029 105,085 952.81 72,027 33,146 31.54

Table 2: Overview of SDF grant allocations and beneficiaries Window

1 Formal Sector

Training

2 Informal

Sector Training

3 Training

Innovation Grants

4a Technology

Development Matching

4b Technology Partnership

Grants

Total

Number of Grants 81 469 27 33 7 617

Total Expenditure (US$)

8.56 16.63 6.87 9.68 3.69 45.44

% of SDF budget 16% 29% 11% 35% 9% 100% Number of Beneficiaries (Trainees)

8,352 88,856 3,110 2,847 1,566 105,085

Number of Women Beneficiaries

2,941 28,085 614 604 662 33,146

Percentage of Women Beneficiaries (%)

35.21 31.61 19.74 21.22 42.27 31.54

SDF Processes 27. Under the SDF, there were five calls for proposals/awards to selected grantees. Funding was provided through each of its windows (1, 2, 3, 4a and 4b) – which had specific targets. While Window 1 aimed to provide support to the formal sector, Window 2 provided financing to the informal sector. Window 3 supported training innovations and Window 4 financed science and technology. Under each window, matching grants were provided by SDF grantees (with some variation in terms of the type of contribution, including monetary and in-kind). 28. The selection procedures were thorough and detailed. Specifically, when a call for proposals was launched, it was widely publicized using various media (TV, radio programs, newspapers, brochures/flyers). An applicant would be advised to visit the SDF website to access an application. The applicant then had the opportunity to contact an intermediary (paid by the SDF) for assistance in preparing their grant proposals. Each proposal was then evaluated by two independent technical consultants. Applicants who were unsuccessful at this stage were notified and provided an explanation as to why their application did not

43

pass the technical proposal. At the same time, SDF staff undertook site visits to those applicants who did receive a passing score. Their report would then be sent to the SDF manager. If the applicants passed this phase, they were then recommended by the SDF manager to a 9-member SDF approval committee for consideration. The committee would then either approve or reject proposals, and in either case, inform the applicant of the reason for their decision. In some cases, conditional approval would be granted based on changes requested by the evaluators. The names of grantees were posted on the COTVET website. Increased Productivity among Targeted Beneficiaries 29. Analysis of available data provides evidence that the productivity (as measured by output per labor hour and the value placed on goods and services produced by SDF grantees) increased significantly during the life of the Project. The following tables provide an overview of productivity gains. In order to best capture the full impact over time, a random sample of a quarter of the beneficiaries from Calls 1 and 2 (and representing each window) were tracked year-on-year and the productivity calculations are based on these data. Increases in productivity are observed across all windows for most years (see Tables 4 – 6 below). 30. Table 3 provides a summary of productivity gains observed by window as measured by output per labor hour (in GH₵)22. As can be seen from the table, productivity (in terms of output per labor hour) increased for most windows across the life of the Project. These increases in productivity may be in part explained by the positive impact of the GSTDP on increasing financial and human resources (capacity) through the SDF. Tables 4 through 7 show the figures used to arrive at the values reported below.

Table 3: Summary of Productivity by window (2013-2016) Output per labor hour (GH₵)

Window 2013 2014 2015 2016

1 11.01 16.13 99.73 211.54

2 5.21 4.37 4.1 2.54

3 18.15 108.46 87.09 285.75

4a 20.92 35.03 4.76 3.29

4b 2.26 5.28 64.58 100.56

22 This measure is used for productivity calculations under the SDF.

44

Table 4: Productivity at Baseline (December 2013)

Window Sum of Value of Output

Produced (GH₵)

Sum of Number of workers (in

the production

department)

Sum of Number of days workers worked

Sum of Number of hours worked

each day

Sum of Total

number of days by workers on the

production plant

Sum of Total

number of hour by workers

on production

plant

Productivity (Output/ Labour Hour)

% increase

over baseline (2013)

1 63,063.15 144 10 15 720 5730 11.01 -

2 56,320.12 232 44 67 1,310 10,820 5.21 263.23

3 268,023.83 388 26 37 1,960 14,770 18.15 148.32

4a 39,167.31 39 12 16 234 1,872 20.92 851.98

4b 1,153.85 14 10 15 70 510 2.26 (95.80)

Total 427,728.25 817 102 150 4,294 33,702 12.69 323.21

Table 5: Productivity (December 2014)


Produced Per week (GH₵)


the production

department)



each day

Sum of Total

number of days

worked by workers on the

production plant

Sum of Total

number of hour by workers

on production

plant

Productivity (Output/ Labour Hour)

% increase over Dec

2013

1 120,006.13 180 17 23 924 7,362 16.30 48.11

2 67,862.08 282 63 104 1,626 15,540 4.37 (16.10)

3 529,304.39 159 20 29 795 4,880 108.46 497.71

4a 52,406.08 32 17 24 187 1,496 35.03 67.43

4b 2,692.31 14 10 15 70 510 5.28 133.33

Total 772,270.99 667 127 195 3602 29788 25.93 104.28

45



Produced Per week (GH₵)


the production

department)



each day

Sum of Total number of

days worked by workers on the production

plant

Sum of Total

number of hour by

workers on production

plant

Productivity

(Output/ Labour Hour)

% increase over Dec

2014

1 994,934.0

8 237 17 26 1,235 9,976 99.73 511.83

2 73,164.21 674 46 62 2,452 17,864 4.10 (6.21)

3 327,180.0

0 130 18 23 487 3,757 87.09 (19.71)

4a 3,600.00 12 7 9 84 756 4.76 (86.41)

4b 15,500.00 6 5 8 30 240 64.58 1123.39 Grand Total

1,414,378.28 1,059 93 128 4,288 32,593 43.40 67.38


Window Sum of

Value of Output

produced per week (GH₵)


the production

department)



each day

Sum of Total number of

days worked by workers on the production

plant

Sum of Total

number of hours

worked by workers on

the production

plant

2016 Product-

ivity

% increase over 2015

1 1,803,981.

79 216 22 33 1,060 8,528 211.54 112.10

2 66,909.41 597 133 181 3,562 2,6360 2.54 (38.02)

3 797,232.28 89 41 46 480 2,790 285.75 228.12

4.1 8,296.50 42 30 50 252 2,520 3.29 (30.86)

4.2 24,135.57 6 5 8 30 240 100.56 55.71 Grand Total

2,700,555.575 950 231 318 5,384 40,438 66.78 53.89

31. As presented in the tables above, the year-on-year increases in productivity across all sectors was as follows: 1) 2013-2014: 104.28 percent; 2) 2014-2015: 67.38 percent; and 3) 2015-2016: 53.89 percent. 23 Table 8 shows the aggregate increase in productivity from 2013 to 2016 with the total (aggregate) increase in productivity between 2013 and 2016 being 426.20 percent.

23 Year-on-year increase in productivity calculations are based on the rate of change in productivity between the two years under consideration: [(final year value – prior year value)/prior year value] x 100

46

Table 8: Overall increase in productivity between 2013 and 2016 under the SDF

Productivity (in GH₵) % change

Dec. 2013 Dec. 2016

12.69 66.78 426.20

32. The following tables show changes in productivity between 2013 and 2016 by sector and firm size. The year-on-year increases in productivity were observed across most sectors over the life of the Project – when looking at the rate of change between productivity in 2013 (first year for which baseline data were available) and the end-line (December 2016) there is some variation by sector and firm size. Although it is unclear why some sectors (and medium sized firms) experienced negative productivity gains between those two years, field visits and discussions with the Government during the preparation of the ICR revealed that these productivity decreases may be related to exogenous shocks. Despite some of this variability, productivity increases overall for SDF beneficiaries was both positive and substantial between 2013 and 2016.

Table 9: Overview of Increase in Productivity among SDF grantees (by Sector) (in GH₵)

Firm Sector December

2013 December

2016 % change

Fisheries 5.07 10.47 106.27%

ICT 3.30 100.56 2950.47%

Livestock and Horticulture 9.69 3.94 -59.31%

Manufacturing 36.32 375.71 934.34% Manufacturing (Garment & Textiles) 9.61 8.24 -14.24%

Services 9.82 85.43 769.75%

Tourism and Hospitality 21.02 0.10 -99.52%

Table 10: Overview of Increase in Productivity among SDF grantees (by Size) (in GH₵)

Firm Size December

2013 December

2016 % change

Small 8.37 44.93 436.68%

Medium 31.50 1.66 -94.72%

Large 9.67 112.74 1,065.85% *Companies with 0 to 9 employees are classified as small; 9 – 25 as medium; and more than 25 as large

47

Increased investment among participating firms 33. An increase in investment by participating enterprises in skills and technology development was also observed during the Project – demonstrating the extent to which SDF beneficiaries themselves invested additional resources in either skills training or innovative technology. At the inception of the Project, the firms reported little investment in skills or technology innovations as part of their businesses. There was, however, significant increase in such investment over the life of the Project through matching grants. Table 11 below provides an overview of increased investment by window over the life of the Project, with the total amount of investment at end-line reaching GH₵ 20.81 million. Tables 12 and 13 show investment data by firm size and sector. As Table 12 shows the majority of investments through matching grants were made under calls 2, 4 and 5, with medium sized firms contributing approximately 68 percent of the total investments made during the life of the Project. In terms of investment by sector, as shown in Table 13, ICT, Livestock and Horticulture, Manufacturing, Oil and Gas, and Service responsible for 86 percent (or GH₵ 17.9 million) of the 20.8 million total invested.

Table 11: Investment by Participating Enterprises (2014-2016) by window

(in GH₵)

Call Window

Total 1 2 3 4a 4b

1 49,361 17,044 - - 39,529 105,934

2 1,935,536 307,679 866,393 1,443,114 606,942 5,159,666

3 372,702 966,158 684,076 760,664 103,304 2,886,907

4 110,175 5,452,998 59,300 2,069,741 43,542 7,735,758

5 104,771 - 7,227 4,818,580 - 4,930,578

Overall 2014 2,488,892 1,307,140 861,166 3,056,769 1,357,809 9,071,778

Total investment 2,572,546 6,743,880 1,616,997 9,092,101 793,318 20,818,844 % increase from baseline 3.4% 415.9% 87.8% 197.4% -41.6% 129.5%

Table 12: Investment by Participating Enterprises (2014-2016) by firm size (in GH₵)

Call

Firm size 1 2 3 4 5 Total

Large 39,529 1,226,728 1,322,575 233,283 39,838 2,861,954

Medium 53,971 3,555,324 1,492,033 7,220,226 1,840,736 14,162,291

Small 12,434 377,614 72,298 282,248 3,050,004 3,794,599

Total 105,934 5,159,666 2,886,907 7,735,758 4,930,578 20,818,844

48

Table 13: Investment by Participating Enterprises (2014-2016) by sector (in GH₵)

Sector Call

TOTAL 1 2 3 4 5

Garment and Textiles (Manufacturing) 49,361 806,721 4,670 27,720 0 888,472

ICT 0 738,872 107,548 0 874,814 1,721,234

Livestock and Horticulture 17,044 337,177 136,730 1,312,907 3,660,530 5,464,389

Manufacturing 39,529 583,833 881,313 917,378 395,234 2,817,287

Construction and Housing 0 15,732 409,950 0 0 425,682

Fisheries 0 175,758 0 118,393 0 294,151

Jewelry 0 772,176 0 0 0 772,176

Oil and Gas 0 0 0 4,915,219 4,915,219

Service 0 1,643,529 1,229,262 198,964 0 3,071,756

Tourism and Hospitality 0 85,866 117,433 245,175 448,474

Education 0 0 0 0 0 -

Health and Social Work 0 0 0 0 0 -

Transport and Storage 0 0 0 0 0 -

TOTAL 105,934 5,159,666 2,886,907 7,735,758 4,930,578 20,818,844

34. Additionally, based on data collected on employment generation from 2014, 2015 and 2016, it is estimated that firms (formal and informal sectors) benefitting from the SDF hired around 50,000 people from 2013 to 2016. 35. The satisfaction of trainees and employers with skills acquired and provided was also very positive. This assessment of satisfaction of trainees used a self-report measure on their level of satisfaction in terms of the training’s relevance in four competency areas (practical, behavioral, technical and communications) using a 5-point Likert scale (very dissatisfied, dissatisfied, somewhat dissatisfied, somewhat satisfied and very satisfied). There was also a measure to gauge employees’ (i.e., trainees’) satisfaction with the completeness (sufficiency) of the training provided. While trainees were tracked under each call, a random sample (N =312) was tracked across the life of the Project. 36. Overall, 98 percent of trainees sampled (both male and female) were somewhat or very satisfied24 with the training provided. Females were only slightly more likely to indicate that they were overall very satisfied than males (54.7 percent compared to 52.9 percent). Given that the large majority of respondents were satisfied with the training, there was limited variability across regions in Ghana. However, it is important that when we

24 Percentage satisfied refers to those individuals who answered that they were very or somewhat satisfied with the relevance and completeness of training divided by the number of people included in the sample.

49

look level of satisfaction, the individuals residing in Greater Accra, Central and Volta were more likely to be very satisfied than individuals in other regions, particularly those in Northern, Eastern, Upper East and West regions. 37. The satisfaction of employers with employees’ skills was also gauged using the same 5-point Likert scale with level of satisfaction being a composite measure of satisfaction with the employees’ skills across the four competency areas mentioned above. Similar findings were observed in terms of employer satisfaction with trainees’ acquired skills. Across all regions, around 97.5 percent of employers reported being very satisfied with their employees’ competencies with some variation by region and sector. A review of the data shows some variation by region and by economic sector. Specifically, the proportion of employers being very satisfied with their employees’ competencies being higher in Greater Accra (in and around the capital city) than in the Northern and Eastern areas. Further, it shows that the level of satisfaction among employers as a whole was greater in the construction sector than in the services and manufacturing sectors. Employers from smaller firms expressed more satisfaction with their employees’ skills than those in larger firms, with 100 percent of small firms being very satisfied and only 60 percent of larger ones. Component 4: Project Management and Monitoring and Evaluation 38. The PSU was able to effectively fulfill its role including (i) managing and reporting on project implementation, (ii) provide monitoring and evaluation of project activities, (iii) acting as liaison with the World Bank; and (iv) assisting COTVET and all participating ministries for effective coordination and M&E, as well as the implementation of an information and communications strategy.Under the Project, fiduciary performance was moderately satisfactory. At the project level, efforts were made to provide limited training in FM to project grantees as well as to field-level monitors, M&E staff, and SDF intermediaries. Further, the Project was compliant with key FM covenants of reporting. Some limitations, including incomplete and delayed IFRs, were identified and addressed in a timely manner. The PSU worked closely with the Government and the Bank to resolve the issue of ineligible expenditures identified in a timely manner. With regards to procurement while initially there were some challenges identified in record-keeping and tracking procurement under different funding sources, these issues were addressed and overall procurement performance improved following the MTR and remained generally strong until project closing.

50

Annex 3. Economic and Financial Analysis Introduction

1. The economic and financial analysis contained in this annex includes an overview of how the GSTDP stimulated skills and technology based development in key economic sectors. This Annex includes the following sections: (i) background; (ii) returns to education and training; (iii) an analysis of baseline data; (iv) beneficiary analysis; (iv) trend analysis of firm output and labor productivity; (v) individual earnings; and (vi) employment generation; (vii) a financial analysis; and (viii) an analysis of financial efficiency. Background 2. At the time of project appraisal, Ghana confronted a range of pressing challenges inhibiting labor productivity. These challenges, in turn, limited the performance of firms and enterprises across a range of economic sectors. At the same time, the education, TVET, and STI sectors were poorly aligned with market demand – limiting the country’s ability to improve productivity and growth prospects. At the time of appraisal, Ghana’s productivity was lower than other African countries with similar economic standing (e.g., Mauritius and Botswana) and a large majority of employment was concentrated in the informal sector. These conditions clearly defined the need to more effectively equip the labor force with market-relevant skills while simultaneously promoting the development, adoption and diffusion of new technologies to increase productivity, competitiveness and, in turn, equitable economic growth. Expected Gains 3. In line with the analysis contained in the PAD, expected gains associated with the GSTDP, and activities undertaken through the SDF specifically, were the improved employability and earnings capacity of beneficiaries, and labor productivity, output and job creation on the part of firms. Returns to Education 4. Returns to education measure increases in earnings associated with an additional year of schooling. As a consequence, returns to education are useful for measuring how skills, acquired through education, or associated with a particular cycle of education, pay off in the medium term as reflected in increased wages. Table 1 collates recent estimates of returns to education across three cycles of education – basic, secondary and tertiary – across a range of sub-Saharan African countries. The two studies that include Ghana estimate that an additional year of primary, secondary and tertiary education in Ghana are associated with increases of between 2 to 10 percent, 7 to 15 percent, and 11 to 31 percent, for each cycle of education respectively, in an individual’s earnings. Available data suggests that the magnitude of economic returns accruing to an individual, increases with each subsequent level of education.

51

Table 1: Returns to Education: Comparison across countries

Papers Country/Countries Primary (%)

Secondary (%)

Tertiary (%)

Bigsten et al (2000)

Cameroon, Ghana, Kenya, Zambia, and Zimbabwe

2-7 7-19 11-31

Schultz (2004) Ghana, Cote d’Ivoire, Kenya, South African, Nigeria, Burkina Faso

3-10 10-15 -

Lassibille and Tan (2005)

Rwanda 18.8 (ns) 22.6 (ns) 21.2-29.1

Soderborn et al (2006)

Kenya and Tanzania

1-7 9-12 30-33

Kabayarara amd Teal (2008)

Tanzania ns 22 (ns) 20.5-24.9

Note: ns: Not statistically significant. Source: S. Gundersen / International Journal of Educational Development 50 (2016) 74–89.

5. Table 2 presents estimates of rates of return to higher education for selected countries in Africa. The mincer coefficient represents the marginal effect of a higher education qualification on earnings. For example, in Ghana, individuals with a higher education qualification demonstrate, on average, earnings 78 percent higher than the average of individuals without a higher education qualification. The internal rate of return (IRR) increases significantly once it is adjusted for employment. In many developing countries, the likelihood that an individual will access employment varies significantly by level of education. As a consequence, the IRR adjustment takes this effect into account. The data presented in Tables 1 and 2 suggest that one would expect improved educational attainment in Ghana to be accompanied by an increase in earnings. In the context of this Project, it is also useful to investigate returns to training specifically in addition to estimates of improved earnings associated with levels of formal education.

52

Table 2: Returns to Higher Education: A cross-country comparison

Country Mincer (%) IRR (%) IRR (Employment Adjusted) (%)

Burundi 95 29 42

Egypt 32 10 15

Ghana 78 26 49

Mali 44 9 13

Nigeria 96 13 16

Rwanda 158 33 29

South Africa

85 21 51

Sudan 90 16 25

Tanzania 141 26 31

Togo 70 31 42

Tunisia 81 22 27

Uganda 87 22 18

Source: “Graduate employment and the returns to higher education in Africa” by Barouni and Broecke, 2013.

Returns to Training 6. The value of skills, and skills improvement, can be estimated by measuring the association between training and individual earnings. There is a significant body of literature relating to returns to education, however far less attention has been paid to returns to training. This is primarily due to the relative dearth of data tracking skills development and its impact on earnings. 7. A literature review described in the PAD provides estimates for effects of training on wages and productivity. Extrapolating from the analysis contained in the literature, the PAD estimated that training had the potential to increase output in Ghana by a magnitude of between 50 to 127 percent (Biggs et al.; World Bank, 1995). It was also illustrated that wages on average increase by 20 percent in developing countries (Middleton et al., World Bank, 1993; Rosholm et al., 2007). 8. Figure 1 illustrates results from the 2013 World Bank Enterprise Survey as they pertain to Ghana. Just over 15 percent of Ghanaian firms surveyed (15.3 percent) reported that an inadequately educated workforce was a major constraint to their operations, a significant increase on the 4.5 percent of firms who answered similarly to the same question in 2007. The increase in the number of firms highlighting a poorly educated workforce as a constraint to their operations constitutes evidence of market failures in the labor market. Larger firms offer more formal training, and tend to have higher percentage of unskilled production workers. Forms operating in the services sector, on average, offer more formal training in comparison to firms in the manufacturing sector.

53

Figure 1: Ghana Workforce Summary 2013

Source: World Bank Enterprise Survey 2013

Economic Analysis 9. Due to the nature of the PDO, a cost benefit analysis of the Project should take into account two broad areas of benefit: the increased employability and earnings capacity of the beneficiaries, as well as positive externalities associated with more highly skilled workers and their improved capacity to absorb new technology. These factors can be measured in the form of productivity and job creation at the firm level. 10. This analysis will focus primarily on the SDF, which accounted for 80 percent of project funding, and follows the structure of the economic analysis presented in the PAD. This allows for comparisons between the initial estimations of NPV and ERR, and those associated with the Project. 11. The key indicators of interest are the effects of SDF related activities on employment, earnings, as well as firm output growth and productivity. The analysis will provide updated beneficiary numbers and economics impacts of the competitive fund, which supported the sectors tabulated in Table 3. The financial analysis re-estimates the net benefits of the project along with computing unit cost analysis comparing training efficiency across sectors.

40.135.8

44.7

31.0

56.360.1

25.8 25.8

0

22.6

28.7

45.5

15.3 14.5 16.1 14.5 14.5

23.9

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

Ghana GhanaManufacturing

Ghana Services Ghana small firm(5‐19)

Ghana Mediumfirm (20‐99)

Ghana Large firm(100+)

Ghana Sectoral Comparison 2013

Percent of firms offering formal training

% of unskilled production workers

% firms inadequately educated workforce as a major constraint

54

Table 3: GSTDP Economic Sectors

Economic Sectors supported by the GSTDP

% of total SDF Budget

Transportation 33 Hospitality and Tourism 28 Construction and Housing 10 Agriculture 9 Trade and Vehicle Repair 8 Beauty and Tailoring 5 Electronics and Electricals 2 Health Care 1 Financial Intermediation 1 Manufacturing 1 NGOs 1

12. The section below presents baseline data from the GSTDP Sector Baseline Survey report, and the section that follows presents the number and characteristics of SDF beneficiaries. Thereafter, the analysis focuses on three key indicators: Firm Output/ Productivity, Individual Earnings and Employment Generation in relation to the SDF.

Baseline Data 13. Figure 2 presents data published in the 2013 World Bank Ghana Enterprise survey. At that time, Annual productivity growth was estimated at 4.9 percent, the rate of growth of employment was estimated at 5.7 percent, and real growth in annual sales was estimated at 9.2 percent. Improved economic performance is associated with an increase in the size of firms, and at that time, the manufacturing sector was on average demonstrating better performance compared to the services sector.

Figure 2: Performance of Firms in Ghana: Ghana Enterprise Survey 2013

Source: World Bank Ghana Enterprise Survey 2013

9.2

10.2

8.2

6.4

13.7

19.4

5.7

6.3

5.0

4.8

7.3

7.6

4.9

6.4

3.2

3.0

7.0

17.1

0.0 5.0 10.0 15.0 20.0 25.0

Ghana

Ghana Manufacturing

Ghana Services

Ghana Small Firms

Ghana Medium Firms

Ghana Large Firms

Performance of Firms Ghana 2013

Annual labor productivity growth (%) Annual employment growth (%)

Real annual sales growth (%)

55

14. Table 4 presents baseline data from the GSTDP Baseline Sector Report for labor productivity by sector. In this context, labor productivity is defined as the value of output over labor hours. The data show that there was significant variation in the level of labor productivity across sectors, with agriculture reporting the lowest measure of labor productivity (GH₵ 0.38/hour), while livestock reporting the highest (GH₵ 3.88/hour), largely a consequence of the low level of labor intensiveness required in the livestock sector.

Table 4: Baseline Productivity Data by Sector

Sec

tors

Agriculture Livestock Hospitality and

Tourism ICT

Construction and Housing

Horticulture

Man

-hou

rs p

er d

ay

Val

ue o

f pr

oduc

t/ser

vice

Man

-hou

rs p

er d

ay

Val

ue o

f pr

oduc

t/ser

vice

Man

-hou

rs p

er d

ay

Val

ue o

f pr

oduc

t/ser

vice

Man

-hou

rs p

er d

ay

Val

ue o

f pr

oduc

t/ser

vice

Man

-hou

rs p

er d

ay

Val

ue o

f pr

oduc

t/ser

vice

Man

-hou

rs p

er d

ay

Val

ue o

f pr

oduc

t/ser

vice

Hours

GH₵

Hours

GH₵ Hours GH₵ Hours GH₵ Hours GH₵ Hours GH₵

Values 13.5

2 5.13 14 50.97 186.72 350.50 44.86 55.67 56.00 95.90 13.52 5.52

Val

ues

of

pro

du

ce/s

ervi

ce

per

man

-hou

rs

GH₵ 0.38 GH₵ 3.66 GH₵ 1.88 GH₵ 1.24 GH₵ 1.71 GH₵ 0.41

Base Case Figures for Fund Beneficiaries 15. The SDF solicited five rounds of proposals through the course of project implementation. As presented in Table 5, the SDF supported a total of 646 sub-projects through four ‘windows’: Window 1 (Formal Sector); Window 2 (Informal Sector); Window 3 (Training Innovations); Window 4a (Technology Development Matching Grants) and Window 4b (Technology Centres). Roughly one-third of SDF (32 percent) was extended to ten trade associations and non-governmental organizations (NGOs) covering ten economic sectors. Table 5 provides information on allocations, expenditures and beneficiaries under the SDF.

56

Table 5: Data on allocations of grants under the SDF Window

1 Formal Sector

Training

2 Informal

Sector Training

3 Training

Innovation Grants

4a Technology

Development Matching

4b Technology Partnership

Grants

Total

Number of Grants (Calls 1 to 5) 81 469 27 33 7 617 Total Window Expenditure (US$)

8,562,624 16,633,318 6,870,820 9,678,134 3,691,602 45,436,498

% of SDF budget 16% 29% 11% 35% 9% 100% Total Number of Grantee Firms 3,038 88,264 66 44 13 91,425 Number of Beneficiaries 8,352 88,856 3,110 2,847 1,566 105,085 Number of Women Beneficiaries

2,941 28,085 614 604 662 33,146

Percentage of Women Beneficiaries (%)

35.21 31.61 19.74 21.22 42.27 31.54

Firm Output and Labor Productivity 16. At project appraisal, it was assumed that support in the form of SDF grants would increase efficiency within beneficiary firms and yield economic output equivalent to double the value of the value of the grant provided to the firm. Figure 3 illustrates labor productivity at beneficiary firms (measured by output per labor hour) increased by 53.89 percent in 2016. This measure of improved labor productivity is considerably higher than that measured by the 2013 Ghana Enterprise Survey, which indicated labor productivity growth of an average of 4.9 percent annually. 17. This analysis draws on the most comprehensive and detailed data collected and monitored over the life of the Project from calls 1 and 2, with a random sample of firms drawn from each of the windows. Labor productivity for participating firms was disaggregated by economic sector and firm size (small, medium, large).

57

Figure 3: Percentage increase in Labor Productivity (Output per Labor Hour)

Source: GSTDP M&E Data

18. In the PAD, the NPV was estimated at US$60.6 million with an ERR of 43 percent. These figures can now be re-estimated using data collected throughout the span of project implementation, and compare this analysis to the estimates envisioned in the PAD.

19. Assumptions informing the original estimated NPV were as follows:

Grant support would increase efficiency within beneficiary firms and yield economic output equivalent to double the value of support extended to the firm through the grant.

A lag time of one year was used for firm output. A 12 percent discount rate across a 12-year project horizon.

20. Annual total firm output was calculated from estimated disbursement figures. This can now be re-estimated using the figures provided in Table 6 as a base in light of the following assumption:

Grant support increased productivity over baseline by 53.89 percent.

Table 6 Calculations for the NPV of Firm Output Present Value of Flows

(for Firm output) Benefits (US$m) 67.53 Costs (US$m) 43.88 Net Present Value ($USm) 23.65 Economic Rate of Return 18.5%

21. As a consequence, the NPV of firm output can be estimated at US$23.65 million.

0.00%

68.14%

104.28%

67.38%

53.89%

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

Jul‐13

Sep‐13

Nov‐13

Jan‐14

Mar‐14

May‐14

Jul‐14

Sep‐14

Nov‐14

Jan‐15

Mar‐15

May‐15

Jul‐15

Sep‐15

Nov‐15

Jan‐16

Mar‐16

May‐16

Jul‐16

Sep‐16

Nov‐16

Increase in Labour Productivity

% increase in labour productivity over baseline target

58

Individual Earnings 22. The GSTDP’s objective was to increase the supply of skills in response to demand in key sectors. Project design was informed by the belief that improving the workers’ skills would improve productivity. In turn, the wage rate is equal to marginal productivity. 23. The following estimates were made at project appraisal:

Wages would increase by 20 percent; Trainees that were placed in apprenticeships were expected to earn 50 percent more

than non-trainers; Baseline annual real earnings for the private formal sector were estimated at

US$7,700, and for the private informal sector at US$4,900; and It was estimated that 4,000 workers would benefit from formal training, and a

further 10,000 from informal training.

24. Analysis of data collected through the life of the project indicates that when calculating the NPV of individual earnings the following now holds true:

8,352 individuals benefited from formal training, 88,856 benefited from informal training;

Wage earnings increased by 4 percent per annum on average for Calls 1 and 2 (controlling for inflation); and

Using real discount rate of 12 percent over a 20-year project horizon.

25. A beneficiary assessment was completed in February 2017. Of the 38 project beneficiaries who participated in the survey, 32 (84.21 percent) reported that the Project had improved their income. The average income of beneficiaries prior to enrolling in GSTDP doubled after receiving support from the project, rising to GH₵ 1,500 per month. The survey revealed that average monthly income for non-beneficiaries at the time of the beneficiary assessment was approximately GH₵ 700, equivalent to less than half of the monthly income of project beneficiaries. However, it is still difficult to attribute improved earnings solely to the GSTDP, and there may be other significant differences between the beneficiaries and non-beneficiaries. It is also important to note that these data are based on self-report measures.

26. GSTDP M&E data earnings information is presented in Table 7. Using a sample from Calls 1 and 2 from which productivity earnings increased on average by 4 percent per annum (controlling for inflation).

59

Table 7: Average Monthly Earnings (GHS)

Formal and Informal Sector Training 2013 2014 2015 2016

Fisheries 488.24 597.06 861.76 991.18

ICT 1,170.00 1,600.00 2,050.00 2,440.00

Livestock & Horticulture 354.11 533.04 495.64 1,354.95

Manufacturing 1,340.00 1,360.00 1,820.00 1,890.00

Manufacturing (Garment & Textile) 412.00 448.00 565.00 825.00

Service 1,543.98 1,832.71 2,118.14 2,343.73

Tourism & Hospitality 1,670.00 2,000.00 2,360.00 2,705.00

Average 1,058.42 1,279.85 1,509.67 1,872.22

% change in earnings - 21% 18% 24%

Inflation rate (GDP Deflator) WB 15.58% 17% 18% -

% change in earnings corrected for inflation - 5% 1% 6%

Table 8: Summary NPV Individual Workers Wage Gain Annual

Present Value of Earnings (USD)

NPV Individual worker’s wage gain assuming works for 30 years after training (USD)

Total Project beneficiaries’ net increase in earnings (USD)

Estimate in the PAD

a. Baseline (Formal)

$6,776.00

Number of beneficiaries

8,352

Returns to training 4% increase in earnings per annum

USD 2,263.79

18,907,174.08 43,660,000

b. Baseline (informal)

$4,312.00

Number of beneficiaries

88,856

Returns to training 4% increase in earnings per annum

USD 1,440.59

128,005,065 173,670,000

Total 146,912,239.10 217,330,000 Using real discount rate of 12% over a 20-year project horizon

60

Employment Generation 27. By improving the quality and relevance of skills, as well as by upgrading and absorbing new technology, firms can increase output and hire more workers. 28. The PAD estimated the following:

Employment increase per firm is premised on the projection that 60 percent of firms in the formal sector would add one worker per grant received.

In the informal sector, it projected that 20 percent of firms would add one worker per grant received.

Based on these assumptions, it was estimated that support to firms would result in the creation of 722 new jobs in the formal and informal sectors.

29. Data on employment generation was collected for the years 2014, 2015 and 2016. For each call, firms were randomly selected, and as a consequence we can assume that outcomes are generalizable to the general population of grantees. As collated in Table 9, we can assume that in 2016, an additional 17,995 employees were hired as a result of the SDF, 40 percent of whom are female. As of 2016, annual employment growth as a result of the SDF (Number of additional staff hired as a result of SDF / total current number of staff) was 18 percent against baseline data from the World Bank Enterprise Survey which estimates annual employment growth of 5.4 percent over the same period.

30. In 2014 and 2015, formal employers were more likely to employ additional staff as a result of the SDF compared to their peers in the informal sector. In 2016, the sectoral trend was reversed with 33 percent of informal employers taking on additional workers compared to 30 percent of formal employers surveyed. Despite the fact that overall formal firms hired more staff than informal firms as a result of SDF supported activities, formal firms hired proportionately fewer women than informal firms. In 2016, 5 percent of new employees hired by formal firms were female, compared to 60 percent of those hired by informal firms.

Table 9: Employment Generation Sample Descriptive 2014 2015 2016 F I Total F I Total F I Total Number of firms in sample 96 250 361 59 219 278 99 216 315

% of firms that hired additional staff as a result of SDF

28% 14% 17% 29% 20% 22% 30% 33% 31%

Number of additional staff hired (as a result of SDF)

420 266 686 225 108 333 223 316 539

Average number of staff hired (of firms that did hire) 1DP

16 8 11 13 2 5 9 4 5.5

% of female additional hires 25% 56% 37% 13% 48% 24% 5% 60% 40% Annual Employment Growth (as a result of SDF)

20% 14% 17% 11% 11% 11% 12% 26% 18%

Estimate of staff hired in total 10,906 firms *

20,394 11,997 17,995

Assuming that figures can be scaled up to GSTDP population; *F = formal, I = informal; Source: GSTDP M&E data

61

Financial Analysis 31. The financial analysis presented in this annex is presented from the perspective of the government: it identifies the project related inflows and outflows to and from the government, linked to investments in skills and technology development. The NPV and the ERR are estimated along with a discussion of internal, making use of a unit cost analysis.

Costs and Benefits 32. The Project had a total cost of US$70 million25, of which US$47.96 was allocated to Component 3, with US$41.33 million allocated to the fund itself. IDA financed 100 percent of direct investment costs associated with the Project. Table 10 presents the Project’s disbursement schedule. 33. The PAD assumed that in addition to repayment costs associated with the Project, the government will need to dedicate funds equivalent to 5 percent of the Project’s total cost each year for ten years (2016-2026), to ensure that the positive effects of the Project are sustained and maximized. 34. The Project has demonstrated the value of improved skills and technology to firm output and productivity, private returns accruing to individuals and positive spill-overs accruing to public institutions. As a consequence, the Project has the potential to act as a catalyst for future allied interventions. Donor partner support for related interventions is expected to continue and will sustain and augment the achievements of the Project. DANIDA’s new SDF II project initiated in July 2016 will continue to support skills development and spur employment over the course of the next three years, while also emphasizing and sustaining the achievements of the GSTDP.

Table 10: Actual IDA Disbursements Actual Disbursements (US$ million)

Year Q3 2011 Q3 2012 Q3 2013 Q3 2014 Q3 2015 Q3 2016 Q2 2017 Amount disbursed IDA

0 5.32 3.54 12.95 14.88 6.48 0.71

Amount Cumulative

0 5.32 8.86 21.81 36.69 43.17 43.88

Percentage of Total Budget 0% 12% 20% 49% 82% 96% 97.62%

35. Table 11 presents key costs and benefits of SDF-supported interventions. Premised on the analysis presented in the economic analysis, we expect the following:

1) An increase in labor productivity between 2015 and 2016 of 53.89 percent. 2) An increase in individual earnings of 4 percent per annum.

25 As a result of exchange rate losses, the total US$ amount available was around US$62 million.

62

3) Annual Employment growth of 18 percent (additional staff hired as a result of SDF / total current staff, as of 2015).

36. Moreover, 2015 M&E data suggest that participating firms increased their investment in skills and the development of technology by, on average, 136 percent.

Table 11: Key Costs and Benefits of the SDF Window Total

Number of Project Grants

Total Window Cost

(US$)

Total number of

firms

Total number of Project

Beneficiaries (Trainees)

1 81 8,562,624 3,038 8,352 2 469 16,633,318 88,264 88,856 3 27 6,870,820 66 3,110 4a 33 9,678,134 44 2,847 4b 7 3,691,602 13 1,566

TOTAL 617 45,436,498 91,425 105,085

Net Benefit of the SDF 37. Direct benefit accruing to the Government as a consequence of project interventions will be in the form of corporate and income taxes. Net fiscal benefits derived from the project are estimated in Table 12. It is assumed that on top of the repayment costs, the sustainability of the project will require additional support equivalent to approximately 5 percent of the project per year for ten years (2016-2026) following the initial investment phase. 38. The PAD estimated the following: total benefits valued at US$67.3 million and a net benefit of US$41.8 million. Revised estimates compute a total benefit of US$49.98 million and a net benefit of US$24.18 million. The downward revision of these estimates is attributable to a productivity increase of 50 percent compared to the 100 percent productivity increase assumed in the PAD.

63

Table 12: Calculations for Net Benefit of the SDF Benefits (US$M) Tax Rate Tax Revenue

for GoG NPV of firm output 23.65 25% 5.91 NPV of individual earnings

146.9 30% 44.97

Total Benefit 49.98 Net Benefit 24.18

Efficiency of Implementation 39. The economic justification for investment in TVET remains strong. The GSTDP aimed to improve the efficiency and relevance of skills development and technology transfer. Improved skills and more effective use of technology can contribute to further efficiency gains in and the improved competitiveness of Ghanaian firms, both in the formal and informal sectors. The broader social implications of higher earnings potential, improved efficiency and competitiveness include new job opportunities and reduced inequality. 40. Initial implementation delays incurred during the first two years of GSTDP implementation – specifically with regards to capacity-building in the PSU; establishing M&E systems to monitor SDF grants and baseline data collection; retaining TA for COTVET capacity strengthening; and the establishment of MIS under Components 1 and 2 – contributed to reduced implementation efficiency. 41. Data collected through the GSTDP M&E framework demonstrates promising results regarding efficiency gains related to: improved labor productivity in participating firms; increases in investment in skills and technology development on the part of participating enterprises; satisfaction with market-relevant skills by trainees and participating firms; and new technologies adopted by participating firms. 42. Skills training is a key cost driver of the SDF, as shown by Table 13. The unit cost of training an individual under the SDF was US$486.58.

64

Table 13: Unit Costs of Training by Window

SDF Training Costs Window Total Commitments

(US$) Number of Trainees

Number of males

Trained

Number of

Females Trained

Unit Cost (US$)

1 8,037,927.24 8,216 5,295 2,953 978.33

2 16,579,606.47 88,940 60,826 28,059 186.41

3 7,241,036.34 2,990 2,437 578 2,421.75

4a 14,384,363.67 2,806 2,233 590 5,126.29

4b 4,785,256.03 1,566 904 662 3,055.72

Total 51,028,189.75 104,872 71,809 33,082 486.58 43. Notwithstanding the positive NPV and ERR, and other efficiency gains realized under the Project, given the delays in implementation under Components 1 and 2 as well as the cost overruns under Sub-Components 3.2 and Component 4, the overall efficiency under the Project is rated Modest.

65

Annex 4. Bank Lending and Implementation Support/Supervision Processes

a) Task Team members

Names Title Unit Responsibility/

Specialty Lending Eunice Yaa Brimfah Ackwerh

Senior Education Specialist GED13 Education

Beatrix Allah-Mensah Senior Operations Officer AFCW1 Operations Norosoa Andrianaivo Operations Analyst GHN07 Operations Peter Darvas Senior Economist GED13 Economics Robert Wallace DeGraft-Hanson

Sr Financial Management Specialist

GGO31 Financial Management

Michael Ehst Senior Private Sector Specialist

GTCID Private Sector

Manush A. Hristov Senior Counsel LEGES Legal

Anders Jensen Senior Monitoring & Evaluation

GENGE M&E

Charles John Maguire Consultant GEDDR Petra Righetti E T Consultant AFTEE - HIS Rajiv Sondhi Senior Finance Officer WFALA Finance Katherine Deaton Steel Energy Specialist AFTG2 - HIS Energy

John W. Fraser Stewart Senior Natural Resources Mgmt. Specialist

GCCIA - HIS Natural Resources Management

Anubha Verma Consultant GEDDR Education Alfred Jay Watkins Consultant GFMDR

Supervision/ICR Eunice Yaa Brimfah Ackwerh

Senior Education Specialist GED13 Education

Janet Omobolanle Adebo Program Assistant GED13 Operations Beatrix Allah-Mensah Senior Operations Officer AFCW1 Operations Norosoa Andrianaivo Operations Analyst GHN07 Operations Charles John Aryee Ashong Senior Procurement Specialist GGO01 Procurement

Demba Balde Senior Social Development Specialist

GSU01 Social Development

Elizabeth Brower Consultant Economics Peter Darvas Senior Economist GED13 Economics Robert Wallace DeGraft-Hanson

Senior Financial Management Specialist

GGO31 Financial Management

Michael Ehst Senior Private Sector Specialist GTCID Private Sector Linda English Consultant Education Martin Fodor Senior Environmental Specialist GEN2B Environment Kennedy Fosu Communications Officer AFREC Communications Franklin Kuma Kwasi Gavu Consultant GEN01 Safeguards Manush A. Hristov Senior Counsel LEGES Legal Michael Gboyega Ilesanmi Social Development Specialist GSU01 Social Development

66

Anders Jensen Senior Monitoring & Evaluation

GENGE M&E

Charles John Maguire Consultant GEDDR Gloria Malia Mahama Social Development Specialist GSU01 Social Development Laura McDonald Operations Officer GED13 Operations Deborah Mikesell Senior Education Specialist GED13 Education Petra Righetti E T Consultant AFTEE - HIS Rajiv Sondhi Senior Finance Officer WFALA Finance Katherine Deaton Steel Energy Specialist AFTG2 - HIS Energy

John W. Fraser Stewart Senior Natural Resources Mgmt. Specialist

GCCIA - HIS Natural Resources Management

Anita Bimunka Takura Tingbani

Environmental Specialist GEN01 Environment

Stephen Kwaku Tettevi Team Assistant AFCW1 Operations Bernardo da Cruz Vasconcellos

Education Specialist GED07 Education

Anubha Verma Consultant GEDDR Education Alfred Jay Watkins Consultant GFMDR

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending 48 354.5

Total: 48 354.5 Supervision/ICR 114.76 456.6

Total: 114.76 457.1

Total: 162.76 811.7

67

Annex 5. Beneficiary Survey Results 1. This summary is adapted from the Executive Summary of a Final report of a Beneficiary Assessment of the Ghana Skills and Technology Development Project submitted in April 2017. Introduction 2. The main objective of this Beneficiary Assessment Study (BAS) was to critically assess the immediate and potential medium term and long term impacts of the project on beneficiaries. The period for the Beneficiary Assessment spanned over a three-month period, between September and December 2016. The evaluation involved employers, employees, industry associations, public and private training institutes, universities, polytechnics, informal sector operators and other agencies engaged in technology development. It also covered some secondary beneficiaries whose benefits were as a result of spill over impacts from the primary beneficiaries. The impact of the Project was assessed along the key criteria of relevance, effectiveness and efficiency. Performance of Project 3. This was assessed through three criteria: relevance of the objectives, effectiveness and efficiency. The criteria focused on the quality of the GSTDP objectives “Doing the right things” and the extent that the right objectives were achieved at reasonable cost “Doing things right”. The section below on project performance answers the question – “Were the right things done right?” The following provides an overview of beneficiary feedback in each of these three areas related to the GSTDP performance. Relevance 4. The Technical and Vocational Education and Training institutions and technology providers’ staff/contact persons who were interviewed rated the Project as highly relevant to the skills and technology development of the youth and therefore relevant to the national poverty reduction agenda and to the needs of both the urban and rural poor. Some were of the view that the Project’s goal and objectives, which aim to reduce urban and rural poverty through skills and technology development, are consistent with the Government’s development objectives which may be the driving force behind the GSTDP as stated in Ghana Poverty Reduction Strategy (GPRS) I and II and the Ghana Shared Growth and Development Agenda (GSGDA, 2010-2013). The interviewees were quick to add that at the institutional level, the objectives of the GSTDP are in line with the institutional agenda that aims at building capacities of their staff to be continuously relevant in the fast changing technological era. That is, the GSTDP has helped bring skills and technology development to the doorstep of these institutions. It was found that the GSTDP activities were highly relevant to the institutions’ priorities as outlined in their development plans and to the needs of their trainees.

68

Effectiveness 5. The views expressed by the respondents about the effectiveness of the GSTDP indicate that the project has been effective in progressing towards attainment of the project objectives and goal by reaching out to most of them (individual beneficiaries and institutions responsible for skills development and technological impact). Efficiency 6. The assessment discovered that COTVET has been efficient and successful in the implementation of the GSTDP since the economic or financial returns from the various types of skills and technological development enterprises that the Project institutions and individual beneficiaries are engaged in has been laudable. Analysis of Outputs and Impact Assessment 7. The assessment revealed that the Project has had great impact on Project beneficiaries by increasing their level of incomes. The increase in incomes has helped some beneficiaries to acquire business assets (lands, purchased new equipment), household assets (put up houses, bought radios and vehicles, etc.) and invested in their children’s education. 8. Human and social capital was observed to have also increased. The numerous skills training programmes organized by the Project for the beneficiaries on group development, enterprise skills, business management and marketing have translated into improvement in social lives of beneficiaries. Again, the respondents claimed that through the Project, they have been able to increase their self-esteem. Additionally, the Project has helped to establish social networking among beneficiaries which has enabled them to boost businesses. Performance of partners 9. The primary supporting partners that worked with the Project were COTVET, TVET, and the Ministry of Environment, Science, Technology and Innovation. The Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis indicated that the implementing and supporting agencies had the technical expertise to deliver on the roles they were assigned, especially in the area of technology transfer. Generally, it can be said that the collaborating partners of the project performed creditably well due to their ability to over achieve set targets. Sustainability 10. The interaction with the project staff and some beneficiaries indicated that both parties are convinced that the GSTDP is sustainable. This they explained is based on the fact that beneficiaries have undergone training and capacity building programmes that have made them competent to retool groups and individuals by improving on their skills and

69

developing their technological knowhow. More importantly, beneficiaries have been supplied with tools and kits to boost their operations. It was also intimated that the continuous monitoring of operations of these institutions and beneficiaries by officers of GSTDP will help to ensure the sustainability of their operations. Recommendations 11. Some of the key recommendations from the Beneficiary Assessment included: a) The role of master craftsmen must be given the right recognition in the implementation

of the Project in order to motivate them to train more apprentices. To achieve this, incentives such as waiver of taxes as well as linkages to sources of supply of quality equipment and tools at competitive prices should be considered. Also, attachments of master craftsmen to technical institutions to upgrade their skills could also be introduced to help them impart improved knowledge to their apprentices. The support given to graduate apprentices in the form of start-up kits is innovative and must be expanded. However, careful selection of beneficiaries will be necessary to ensure that maximum impact is made on skills training and technological development at the local level.

b) While the project has done very well coordinating its activities from the PSU of the COTVET in Accra, due to the nationwide coverage of the project, it may require that regional or zonal coordinating units are set up to enhance effective Project coordination. This is based on experiences from other projects that have had national coverage.

c) The Project has gone a long way in reducing poverty by helping the entrepreneurial poor beneficiaries to acquire relevant employable and demand-driven skills that has enabled them to establish their own businesses. Incomes of beneficiaries have been significantly enhanced by the Project and this has translated into investments in education, household assets, and improvement in health. However, the Project can make better impact on poverty reduction if training given to beneficiaries is supported by easy access to credit.

d) The targeting of beneficiaries, as observed from the survey, left out the poor in rural

settlements because most of the technical skills and technological training and adoption were concentrated in the urban centers. The Project can enhance its targeting processes to reach the very poor and vulnerable by improving the dissemination of information to the remote areas or villages that might not have received information about GSTDP but contains the very poor people.

70

Annex 6. Stakeholder Workshop Report and Results Not applicable.

71

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The following is a summary of the Government’s Project Completion Report (PCR). The full PCR has been archived in WBdocs. Background The Ghana Skills and Technology Development Project (GSTDP) is part of the skills reform initiative programme of the Government of Ghana that has been funded by the World Bank over a period of five (5) years from March 2011 to December 2016. The Council for Technical and Vocational Education and Training (COTVET) is the implementing agency and leads the process of implementation jointly with the Ministry of Environment, Science and Technology and Innovation (MESTI) and in close collaboration with the various technical and vocational education and training (TVET) providers, Science, Technology and Innovation (STI) providers both in the public and private sectors, as well as the formal and informal sectors of the economy. The primary objective of the GSTDP was to stimulate skills and technology based development in key economic sectors through demand-driven improvements in the quality of formal and informal training and development and adoption of new technologies in some selected priority sectors. Project Objectives Project Development Objective

The Project aims to improve demand-driven skills development and increase adoption of new technologies in selected economic sectors.

Specific Project Objectives

Specific objectives under the project are as follows: (i) Increase in labor productivity of participating firms disaggregated by

economic sector and size (small, medium and large) (ii) Increase in investment by participating enterprises in skills and technology

development (disaggregated by economic sectors; size – small, medium, large; skills and technology

(iii) Satisfaction with skills by trainees disaggregated by sex and region (iv) Satisfaction with skills by participating firms (v) Direct project beneficiaries of which (%) are female as measured by the

number of people trained The interventions of the Project were delivered through four technically integrated and mutually complementary components and these are:

72

Component 1: Institutional Strengthening of Skills Development Component 2: Institutional Strengthening of Science & Technology Development Component 3: Financing of Skills and Technology Development Programmes

through the Skills Development Fund Component 4: Project Management and Monitoring and Evaluation

The Project coverage included all the ten regions of Ghana. The primary target groups of the Project’s demand-driven services were made up of employers, employees, industry associations, public and private training institutes, universities, polytechnics, informal sector operators and other agencies engaged in technology development. The Project also partnered institutions, particularly the Ministry of Environment, Science, Technology and Innovation (MESTI), COTVET and TVET. Performance of Project (PP) This was assessed through three criteria: relevance of the objectives, effectiveness and efficiency. The criteria focused on the quality of the GSTDP objectives “Doing the right things” and the extent that the right objectives were achieved at reasonable cost “Doing things right”. PP answers the question – “Were the right things done right?” Below is a brief explanation of the three criteria under the PP. Relevance The Technical and Vocational Education and Training institutions and technology providers’ staff/contact persons who were interviewed rated the Project as highly relevant to the skills and technology development of the youth and therefore relevant to the national poverty reduction agenda and to the needs of both the urban and rural poor. Some were of the view that the Project’s goal and objectives, which aim to reduce urban and rural poverty through skills and technology development, are consistent with the Government’s development objectives which may be the driving force behind the GSTDP as stated in Ghana Poverty Reduction Strategy (GPRS) I and II and the Ghana Shared Growth and Development Agenda (GSGDA, 2010-2013). The interviewees were quick to add that at the institutional level, the objectives of the GSTDP are in line with the institutional agenda that aims at building capacities of their staff to be continuously relevant in the fast changing technological era. That is, the GSTDP has helped bring skills and technology development to the doorstep of these institutions. It was found that the GSTDP activities were highly relevant to the institutions’ priorities as outlined in their development plans and to the needs of their trainees. Effectiveness The views expressed by the respondents about the effectiveness of the GSTDP indicate that the project has been effective in progressing towards attainment of the project objectives and goal by reaching out to most of them (individual beneficiaries and institutions responsible for skills development and technological impact).

73

Efficiency The assessment discovered that COTVET has been efficient and successful in the implementation of the GSTDP since the economic or financial returns from the various types of skills and technological development enterprises that the Project institutions and individual beneficiaries are engaged in has been laudable. Project Achievements Achievements under Component 1 Below are some achievements:

A legislative instrument, L.I 2195 has been developed for the regulation, development and implementation of TVET in the country.

A total of 23 manuals and guides have been developed to guide TVET institutions in the development, delivery and management of TVET programmes in competency based mode.

A manual has been developed for the operationalization of Recognition of Prior (RPL) in the TVET sub sector.

A quality assurance manual has been developed for the institutionalization of quality assurance in TVET institutions in Ghana.

A TVET Management Information System (MIS) has been developed to serve as a database repository for TVET in Ghana

A total of 56 TVET managers and senior supervisors have been trained in the development of institutional development plans (IDPs).

A total of 36 TVET instructors have been trained in the implementation of Recognition of Prior Learning in TVET institutions.

A total of 10 technical and vocational institutes have been supported to develop their IDPs for implementation.

A total of 54 TVET managers have been trained in TVET MIS, data uploading, data manipulation and report generation.

Achievements under Component 2 The following are some major achievements under component 2.2

5 Technology Development, Innovation, Transfer and Marketing Centres/Offices established in UG, CSIR, GAEC, GTUC and K-Poly

50 Technologies Profiled for Transfer and Commercialization/Marketing 200 Researchers Trained in Technology Transfer and

Commercialization/Marketing, Intellectual Property processes and Industry engagement techniques

40 Technologies researched and developed for transfer 10 Industry-Research Seminars organized 10 Industry-Research Partnerships established

74

8 Business plan developed by CSIR and GAEC Guidelines and Strategies for Technology Transfer developed by CSIR, GAEC

and Kumasi Poly Achievements under Component 3

The total number of projects being supported by SDF [from Calls#1-5] currently stands at 646 with a cumulative approved grant value of US$51.68 million.

Total SDF disbursements as at end of July stood at US$ 41.33 million. The total number of trainees for Calls #1 – 5 is 103,683 compared to the expected

24,000 at programme design Of the 646 sub-projects approved:

- 630 have been completed and closed out - 16 were cancelled – contracts signed but no monies disbursed for lack of interest

from grantees Achievements under Component 4 Since project inception, the PSU has created and strengthened the Department with competent staff to lead in the M&E process. The current staff strength of the M&E Division comprises a Head of Department, an M&E Specialist, one M&E Officer, one M&E Assistant, three National Service Persons, and twenty-two field based SDF Project Monitors. M&E frameworks have been developed and baseline data have been collected for all donor funded projects under the PSU including GSTDP and DSIP. The GSTDP by its undeniably broad scope required an elaborate framework and robust M&E system to track the ever increasing activities and deliverables that characterized the implementation of the grant projects under the Component 3 (Skills Development Fund) of the Project. The major monitoring and evaluation accomplishments over the period under review areas itemized Performance of Project Partners Performance Assessment of Executing Agency/Project Management Unit As project implementation progressed, the project team demonstrated improved capacity for effective delivery of relevant services to beneficiaries. In particular, the capacity of staff to prepare terms of reference, RFPs and bidding documents for various assignments, as well as develop the capacity of implementing agency staff, coordinate, supervise and monitor programs was considerably enhanced. Annual work plans and budgets were approved by the Bank, without much difficulty, a testament to the project team’s capacity to execute its mandate. The project team has met all the scheduled reporting requirements for the period. In addition, the project has met its reporting obligations in respect of the submission of audit reports to the Bank. So far certified audit reports have been submitted for Years 2011 to 2016.

75

Performance Assessment of the World Bank The project team expressed satisfaction with the operational support provided by the World Bank team over the implementation period. The provision of timely and qualitative operational support to the project team had, in great measure, aided in project quality control. In reviewing terms of reference, request for proposals, bidding documents draft contracts and reports of consulting firms contracted to undertake various assignments, the World Bank team, very often, provided useful technical inputs that enhanced the quality of these documents. Sustainability/Future plans of the project Analyses of sustainability of GSDTP project took into consideration an assessment of whether project interventions have a potential for being sustained after project closure. For this purpose assessment of management and organization of various sub-projects and financial considerations for operation and maintenance of sub-projects has been done. Overall, indications are that the gains established under the project are likely to be sustained after project closure Implementation Challenges In spite of the successful management of grant processes, there are a number of key challenges that plague the PSU. The following are some of the challenges: Delays in approval and disbursement of funds. Many of the Project beneficiaries, complained about delays in approval and funds disbursement. In some instances planned training schedules were not followed; delays and shifting of training programmes were common occurrences. Lack of clear-cut understanding of what the project was about. Most beneficiaries were not aware of what the project was about and their graduated apprentices didn’t know about the project and how it impacted on their training. There were instances where some beneficiaries in some regions denied ever receiving any support of any kind in terms of training or otherwise but only confirmed after several probing and the mention of ‘COTVET project’ before they remembered. Low number of Technology Centre proposals. Most of the technological institutions and centres in Ghana that were expected to have submitted proposals for grant support failed to put in proposals. However, many of the project beneficiary institutions that were technologically inclined managed to achieve most of the set targets assigned to them. Rising costs. Grant operational expenses were higher than anticipated especially during the first three years of the operation of the Fund. This is the outcome of a deliberate attempt at addressing a core issue of trust in matching grant management in Ghana. Ghana has had a checkered history of Matching Grant Facility management, plagued by abuses by the private sector, on the back of very weak systems of control on inbound operations and even

76

weaker oversight of post disbursement activities. This resulted in decrease of trust in the potential of such an intervention to bring about any improvement in the outcome of the Project, namely productivity improvement and job creation. A new pathway had to be charted, and COTVET spared no expense in deploying very robust grant operation system with demonstrably strong system of controls and sufficient staff strength and other resources to enforce such controls to assure the private sector of fairness, of speedy turnaround time. Post Approval Implementation Support. Firstly, the rapid rate of grant awards required an increase in the number of Project Monitors to collect baseline and other information to help track the results of the intervention. However, the Monitors were not experienced and/or did not have the mandate to offer implementation support to the grantees. This assignment had to be carried out by the Grant Officers at the PSU, who were primarily focused on inbound grant operational activities. Thus, additional personnel were deployed to support post approval activities and work to ensure that grant projects are executed on schedule and on budget. Inadequate Technology Centre Applicants. The results obtained indicated that access to technology results in significant improvement in productivity of grantee firms. However, the number of technology Centre applicants not increased in tandem with training assistance. Quality of Technical and Financial Proposals. The quality of technical and financial proposals was a major challenge to the SDF. The SDF Committee offered provisional approval to grant proposals that met the minimum quality standard. The applicants then had to address a myriad of issues precedent to final approval. This exercise was tedious and consumed significant resources of the PSU in hand- holding support. To save the time, the PSU had to continue offering training to Project Intermediaries and applicants to thoroughly address questions especially those related to sustainability. Timely execution of grant projects. All the projects that have been completed were significantly delayed, some by as many months as the original duration of the project. These delays then required more monitoring hours from the PSU. The delays were mainly due to the inability (and in some cases unwillingness) of the grantees to submit to the financial management and procurement documentation requirements of the PSU. New improved methods of training deployed in 2015 addressed this bottleneck. Lessons Learnt

Key lessons learnt in the Implementation of the Project are:

It is not enough to offer training without offering complementary services such as

credit facilities and other support like the provision of start-up kits to project beneficiaries in order to facilitate higher adoption rate.

When coordinated effort is made at the national and district level to capacitate (skills training) and capitalize (credit) from Micro and Small Enterprises (MSEs),

77

the relevant demand-driven skills that beneficiaries derived from the project could play an important role in delivering them from lack of skills and protracted poverty.

The project monitoring and evaluation unit should be resourced and tasked to undertake monitoring activities on the individual beneficiaries throughout the period.

The project needed an inter-sectoral unit to help coordinate activities across sectors and bring operators of the various sectors with common agenda together to help solve complex problems.

COTVET/GSTDP/SDF should seek counseling for any business they decide to put up a structure for because they are some technical things that will be needed in the building which will not be incorporated in it when the business is not asked

Recommendations: 1. Technology Promotion and Apprenticeship Support. The role of master craftsmen must be given the right recognition in the implementation of the Project in order to motivate them to train more apprentices. To achieve this, incentives such as waiver of taxes as well as linkages to sources of supply of quality equipment and tools at competitive prices should be considered. Also, attachments of master craftsmen to technical institutions to upgrade their skills could also be introduced to help them impart improved knowledge to their apprentices. The support given to graduate apprentices in the form of start-up kits is innovative and must be expanded. However, careful selection of beneficiaries will be necessary to ensure that maximum impact is made on skills training and technological development at the local level. 2. Project Coordination. The project has done very well coordinating its activities from the Project Support Unit (PSU) of the Council for Technical & Vocational Education and Training (COTVET) in Accra. However, due to the nationwide coverage of the project, it may require that regional or zonal coordinating units are set up to enhance effective Project coordination. This is based on experiences from other projects that have had national coverage.

3. Poverty reduction. The Project has gone a long way in reducing poverty by helping the entrepreneurial poor beneficiaries to acquire relevant employable and demand-driven skills that has enabled them to establish their own businesses. Incomes of beneficiaries have been significantly enhanced by the Project and this has translated into investments in education, household assets, and improvement in health. However, the Project can make better impact on poverty reduction if training given to beneficiaries is supported by easy access to credit. 4. The targeting of beneficiaries. As observed from the survey, the project left out the poor in rural settlements because most of the technical skills and technological training and adoption were concentrated in the urban centers. The Project can enhance its targeting processes to reach the very poor and vulnerable by improving the dissemination of information to the remote areas or villages that might not have received information about GSTDP but contains the very poor people.

78

Conclusions The overall conclusion is that, the Project was able to achieve most of its set objectives by being able to provide technical assistance and skills training to key institutions and individual beneficiaries in the selected GSTDP priority economic sectors under the various components and windows. This is so because the Project has been able to equip the beneficiaries with demand-driven employable skills and enhanced their income generating capacities that have helped to alleviate poverty and improved upon their living conditions.

79

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Comments from AfDB were incorporated into this version.

80

Annex 9. List of Supporting Documents Aide Memoires from Missions (2009-2017). COTVET. A five-year Corporate Strategic Plan for the Council for Technical and Vocational Education and Training. COTVET. (March 2017). Ghana Skills and Technology Development Project. Implementation Completion Report. COTVET/PSU. (March 2016). Ghana Skills and Technology Development Project. Annual Report. COTVET/PSU. (July 2014). GSTDP Monitoring and Evaluation Framework. Project Appraisal Document. Final report of a Beneficiary Assessment of the Ghana Skills and Technology Development Project (April 2014). Implementation Status and Results Report (2011-2016) National TVET Strategic Plan (2015-2025). Technical Vocational Education and Training. Prepared by TRANSTEC consultants. Quality Enhancement Review documentation (September 13, 2010. Skills Development Fund (2011-2016). Results Book World Bank. Project Appraisal Document on a Proposed Credit in the amount of SDR 44.9 million (US$70 million equivalent) to the Republic of Ghana for a Ghana Skills and Technology Development Project. March 3, 2011.

81

MAP