Document of The World Bank

FOR OFFICIAL USE ONLY

Report No: 35850 - MX

IMPLEMENTATION COMPLETION REPORT(SCL-43490)

ON A

LOAN

IN THE AMOUNT OF US$300.00 MILLION

TO NACIONAL FINANCIERA, S.N.C.

GUARANTEED BY THE UNITED MEXICAN STATES

FOR A

KNOWLEDGE AND INNOVATION PROJECT (Loan 4349-ME).

June 27, 2006

Finance, Private Sector and Infrastructure DepartmentColombia and Mexico Country Management UnitLatin America and the Caribbean Regional Office

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 22, 2006)

Currency Unit = Mexican Peso MXN 1 = US$ 0.087

US$ 1 = 11.45 MXN

FISCAL YEARJanuary 1 December 31

ABBREVIATIONS AND ACRONYMS

CONACYT National Council for Science and Technology (Consejo Nacional de Ciencia y Tecnología) CAS Country Assistance Strategy

IBRD International Bank for Reconstruction and Development KIP Knowledge and Innovation Project M & E Monitoring and Evaluation

NAFIN Financial National (Nacional Financiera) a Mexican development bank NPV Net Present Value

PCU Project Coordination UnitR&D Research and DevelopmentRETO Network for Efficiency and Organized Work (Red de Eficiencia y Trabajo Organizado)SME Small and Medium EnterpriseS&T Scientific and Technological

Vice President: Pamela CoxSector Director:

Country Director:Makhtar DiopIsabel M. Guerrero

Sector Manager:Country Sector Leader:

Susan G. GoldmarkAnna Wellenstein

Task Team Leader/Task Manager: Esperanza Lasagabaster

MEXICOKNOWLEDGE AND INNOVATION PROJECT

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 75. Major Factors Affecting Implementation and Outcome 216. Sustainability 227. Bank and Borrower Performance 238. Lessons Learned 249. Partner Comments 2510. Additional Information 27Annex 1. Key Performance Indicators/Log Frame Matrix 28Annex 2. Project Costs and Financing 30Annex 3. Economic Costs and Benefits 33Annex 4. Bank Inputs 34Annex 5. Ratings for Achievement of Objectives/Outputs of Components 36Annex 6. Ratings of Bank and Borrower Performance 37Annex 7. List of Supporting Documents 38

Project ID: P044531 Project Name: MEXICO: KNOWLEDGE AND INNOVATION

Team Leader: Esperanza Lasagabaster TL Unit: LCSFRICR Type: Core ICR Report Date: June 27, 2006

1. Project DataName: MEXICO: KNOWLEDGE AND INNOVATION L/C/TF Number: SCL-43490

Country/Department: MEXICO Region: Latin America and the Caribbean Region

Sector/subsector: Central government administration (53%); Tertiary education (22%); Micro- and SME finance (13%); General industry and trade sector (12%)

Theme: Technology diffusion (P); Small and medium enterprise support (P); Education for the knowledge economy (P); Other financial and private sector development (P)

KEY DATES Original Revised/ActualPCD: 09/19/1997 Effective: 01/04/1999 03/02/1999

Appraisal: 03/11/1998 MTR: 06/13/2001 06/13/2001Approval: 06/16/1998 Closing: 12/31/2003 12/31/2005

Borrower/Implementing Agency: GOM/CONACYTOther Partners:

STAFF Current At AppraisalVice President: Pamela Cox Shahid Javed BurkiCountry Director: Isabel M. Guerrero Olivier LafourcadeSector Manager: Susan G. Goldmark Krishna Challa (Acting)Team Leader at ICR: Esperanza Lasagabaster Daniel CrisafulliICR Primary Author: C. Cesar Yammal; Esperanza

Lasagabaster

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: HL

Institutional Development Impact: SU

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry: U

Project at Risk at Any Time: No

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:

The objective of this project was to promote the generation, diffusion, and application of knowledge for innovation in support of economic and social development. Emphasis was placed on stimulation of linkages and effective diffusion of knowledge for innovation through:

Supporting excellence in scientific and technological (S&T) research, increasing the availability lof S&T human capital, and instituting an integrated strategy for development of fields of science of strategic importance to Mexico’s economic and social development;Supporting increased firm-level productivity through provision of decentralized, demand-driven ltechnological services for small and medium sized enterprises (SMEs) and creation of a pilot private sector-led venture capital scheme; andFacilitating linkages between private firms, universities and research institutions through lfinancial support for joint activity and technical assistance to bridge institutions.

3.2 Revised Objective:

Although there were two amendments to the loan agreement (section 3.4), the original objective remained unchanged.

3.3 Original Components:

The project consisted of three components: (A) the Science and Technology Research Component; (B) the Industry-University Linkage Component; and (C) the Enterprise Technology Enhancement Component. The costs and Bank financing of the original and revised components are summarized in Table 3.1. See also Table 2.C in Annex 2.

Table 3.1 Original and revised financing, Knowledge and Innovation Project (US$M)

Original design First amendment

Second amendment

At closing

Component Total Costs

Bank financing Bank financing Bank financing Bank financing

A. Science and technology research 285 135 135 250 263

B. Industry-University linkage 156 63 63 13 11

C. Enterprise technology enhancement 191 73 73 25 18

D. Unallocated 30 30 20 0 0

E. Institutional strengthening -- -- 10 12 8

Total 663 300 300 300 300

- 2 -

Project Component A. Science and Technology Research (original cost: US$285.00 million, IBRD US$135.00 million)

This component was to improve the quality, efficiency, and relevance of scientific and technological research in Mexico through (i) consolidation and expansion of the peer-review evaluation system; (ii) decentralization of decision-making and research, and better incentives for human resource formation in research projects; and (iii) identification and promotion of new fields of research of high social and economic priority to Mexico. There were three subcomponents:

A1. Support for research projects. This subcomponent was to promote quality in research, consolidate and improve peer review, and participatory planning; encourage intensive human resource training; and prioritize timely integration of young researchers in the system.

A2. Field development. This subcomponent was to support research areas which had not developed under the traditional research support structure. The subcomponent was to (i) finance the formation of new research groups and networks; and (ii) give direct support to research projects.

A3. Special Programs. This subcomponent was to support a series of special programs to improve (i) monitoring and evaluation and (ii) institutional capacity of CONACYT.

Project Component B. Industry-University Linkage(original cost: US$156.44 million, IBRD US$62.50 million)

This component aimed at (i) increasing enterprise investment in science and technology by strengthening relationships with academic and research institutions; (ii) improving the impact of academic institutions on firm-level innovation and productivity; (iii) promoting the creation of public goods through increased investment in R&D; and (iv) facilitating the interaction between the academic and private sectors. There were three subcomponents:

B1. Restructuring of Public S&T Institutes. This subcomponent sought to enhance client orientation and effectiveness of Public S&T institutes in support of industry, while increasing their cost recovery.

B2. Joint Industry-Academic Projects. The subcomponent sought to enhance industry-academic bonds by providing support for joint activities in applied research; product and process design, development, and improvement; and technology adaptation and diffusion.

B3. Technical Assistance to Universities. Support would seek to strengthen university outreach offices.

Project Component C. Enterprise Technology Enhancement (original cost: US$191.33 million, IBRD US$72.50 million)

The objective of this component was to directly impact the productivity and competitiveness of firms, particularly small and medium enterprises (SMEs). It comprised four subcomponents:

C1. Technology Modernization Program. This subcomponent sought to foster technology modernization of SMEs and develop an active market in technology services.

- 3 -

C2. Regional/Sector Technology Centers. This subcomponent aimed at improving local access to technology services, primarily for SMEs, through selective support to private technology centers.

C3. Special pilot studies of emerging importance. The project was to finance pilot projects in support of emerging policy priorities in the area of innovation.

C4. Pilot Venture Capital Fund. The Fund sought to fill a financing gap by providing equity investment in start-up, technology based enterprises.

Assessment of project objectives and original components:

The project’s development objective was fully consistent with Mexico’s national strategy and the Bank’s Country Assistance Strategy (CAS) of achieving growth with stability by helping accelerate firm level productivity through the restructuring and development of a more efficient system of knowledge and innovation. The alarming findings that average output per worker had fallen by more than 20 percent between 1981 and 1994 gave high priority to policy measures seeking to spur higher growth in total factor productivity. Science and technology investments, in particular private investments, remained very low and the efficiency of the innovation system in channeling these scarce resources into productive applications was weak. Thus, increasing the effectiveness and raising total investments in science and technology became critical to raising the productivity of Mexican firms.

In addition, the project supported the CAS goals of modernization of the state by seeking to increase the effectiveness and transparency of public programs. It was also supportive of the CAS goal of social development by fostering research and technological development in fields with an important social impact.

Project design reflected its ambitious objective and the government's vision for a more coherent and complete national innovation system. The design was comprehensive, covering three critical areas along a supply-demand framework for innovation and technology: (i) promoting and improving the quality of supply (Component A: Science and Technology Research); (ii) articulating the demand (Component C: Enterprise Technology Enhancement); and equally important (iii) fostering stronger linkages between the supply and demand for innovation and technology products and services (Component B: Industry-University Linkage). Sectoral work conducted since the mid-1990s and during project preparation had found important gaps and the need for interventions in all three areas.

The project rightly identified priority areas for intervention but overestimated CONACYT’s institutional capacity with regards to technology development activities, which constituted new areas for the agency. The presence of too many subcomponents together with limited capacity tended to focus the institution’s attention in activities that were better aligned with its traditional mission, to the detriment of others that may have been perceived as more challenging. This problem was compounded by the fact that the project did not increase overall resources for CONACYT, rather it involved a reallocation of the portfolio towards new areas of intervention. Also, project design did not pay sufficient attention to dissemination activities. These challenges slowed down the implementation of components B and C.

- 4 -

3.4 Revised Components:

There were two amendments to the loan agreement and one extension of the closing date for 24 months until December 31, 2005. The amendments to the project were well justified, addressing various implementation issues and building on the positive changes and momentum brought about by the legal reform of the national innovation system in 2002. The extension allowed the completion of project activities which had been delayed due to low disbursement during the project’s first two years of implementation. See Table 3.1 and Table A.2.3, Annex 2, for details on reallocated amounts as a consequence of loan amendments.

Amendment 1

On September 18, 2002 the loan agreement was amended to modify the pari-passu and introduce technical changes encompassing the following: (i) the percentage of research grant amounts (Component A) financed by the Bank increased from 50% to 75%; (ii) research grants under the framework of the Millennium Science Initiative (described in section 4.2, Outputs by component) and grants for biotechnology were formally incorporated into Component A; (iii) the threshold and the aggregate totals for the use of National Competitive Bidding procedures in procurement of goods was raised; and (iv) the creation, responsibilities, and financing of the Project Coordination Unit (PCU) were formalized.

As a consequence of this amendment, a new Component D (Institutional Strengthening) was added to support the operation of the PCU and the implementation of a Government Resource Planning system at CONACYT (section 4.2). Actual cost and IBRD financing for Component D were US$9.6 million and US$7.7 million, respectively. Most of this funding was allocated to the development of the new resource planning system with very positive results in terms of transparency and services to beneficiaries.

Amendment 2

In June 2002, Mexico enacted a new legal framework for the science and technology sector through the following laws: (i) the Law for Science and Technology (Ley de Ciencia y Tecnología), and (ii) Law for CONACYT (Ley Orgánica del CONACYT). The new laws led to a deep restructuring in the allocation of public resources for science and technology and resulted in a major operational change with the creation of the Sectoral Funds (Fondos Sectoriales), which are funds jointly operated by CONACYT and the corresponding line ministry (agriculture, health, economy, energy, etc.).

A second amendment to the project was approved on June 6, 2003, to adapt the project to the legal reforms of 2002 and recognize the low disbursements experienced within Components B and C. (See Table 3.1 and Table A.2.3 in Annex 2).

A. The Science and Technology Research Component had performed highly satisfactorily during the two years prior to the amendment and disbursed 76 percent of its initial funding of US$135 million. It was therefore agreed (i) to scale up the investment in research support, particularly in the Millennium Science research initiatives and (ii) to add a new sub-component, Scholarships for Graduate Studies, that would strengthen Mexico's capacity to generate new knowledge and would support the project objectives. As a result, the total loan amount available to support

- 5 -

research projects increased by US$50.8 million to US$175.8 million, and US$71.6 million was allocated to scholarships (see Table A.2.3, Annex 2). These increases were made through resources freed from components B and C since the total loan envelope remained constant at US$300 million.

B. The Industry-University Linkage Component had only disbursed 4 percent of its funding by 2003. The new Science and Technology Law (2002) generated new momentum for strengthening the public S&T institutes. It mandated the creation of external committees with private sector participation for each center, which provided the necessary strategic leadership to reorient them. With a more supportive incentive framework, the restructuring of the public S&T institutes could be completed in a more cost-effective manner and with less technical assistance resources than initially planned. The Joint Industry-Academic Projects sub-component was phased out due to its low demand. It was estimated that the component could thus be completed using US$12.7 million from the Bank loan, instead of US$62.5 million as originally planned.

C. The Enterprise Technology Development Component had disbursed only 17 percent of allocated resources at the time of the second amendment. In addition, the goal of the Technology Modernization subcomponent was being attained through the new Sectoral Fund for Economic Development, created as a result of the Law for Science and Technology. The new fund, which was co-financed by the Ministry of Economy and CONACYT, took over and scaled-up financing of technology development projects in SMEs. It was therefore decided to conclude activities under this sub-component and reallocate them to component A.

3.5 Quality at Entry:

Overall project quality at entry is rated moderately unsatisfactory. The project addressed the World Bank’s Country Assistance Strategy (CAS) objectives of growth with stability, modernization of the State, and social development. Further, the Government had indicated strong support for the project during the June 1997 Country Strategy and Implementation Review and continued to advocate accelerated project preparation. CONACYT’s senior management had undertaken a series of initiatives showing their strong interest and commitment to the project, including the pilot restructuring of four public research centers.

At the technical level, the project drew on (i) seven years of experience implementing the Mexico Science and Technology Infrastructure Project (Loan 3475-ME); (ii) numerous analytical studies of the Mexican Innovation System undertaken by CONACYT and the Bank during 1997-98, including reports financed by the PHRD grant approved for project preparation; and (iii) contributions to and assessment of project design by leading academics and experienced practitioners. Multiple workshops were also held with external stakeholders including the private sector, academics and young researchers.

In sum, the project had very strong government support and the main sector issues were correctly identified. Project preparation, however, was not extensive enough and consultations with internal stakeholders were less than sufficient. The project was prepared in an expeditious manner but at the expense of a more thorough assessment of CONACYT’s institutional capacity, especially in those areas that concerned technology transfer to the private sector. CONACYT’s organizational culture was not adequately understood and project design did not address change management issues. The new strategy was not sufficiently communicated to staff, who should have also participated more

- 6 -

actively in the design of new policy instruments. Although the establishment of the Directorate for Technology Development predated the project and a new and dynamic director was brought on board to lead it, the directorate was very small, did not have requisite human resources, and lacked the experience needed for rapid implementation of Component C. Amounts allocated to Components B and C were overestimated given CONACYT's capacity at the time. Also, project design should have placed greater emphasis on dissemination efforts and on the relevancy of performance indicators.

A combination of lack of clarity on implementation arrangements, rigidities in procurement processes, inadequate internal consultations, and insufficient capacity of the nascent Directorate for Technology Development led to delays in project implementation. Many of these issues were solved during implementation with the creation of the Project Coordination Unit, strengthening of the Directorate for Technology Development, and modifications to procurement processes. In addition, the legal reforms approved in 2002 consolidated the project’s vision for a more coherent national innovation system and provided momentum to some areas that were moving slowly, in particular, the restructuring of public S&T institutes and activities supporting technology transfer to the private sector.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:

The project’s overall performance was moderately satisfactory with highly satisfactory outcomes in the promotion of excellence in S&T research, the increased availability of S&T human capital, and the development of new research fields; the successful launch of a pilot private sector-led venture capital scheme; and modest achievements in the promotion of greater linkages between private firms, universities, and research institutions. Achievements with regard to the promotion of increased firm-level productivity through provision of decentralized, demand-driven technological services for SMEs were very limited. In addition, the project contributed to the development and implementation of a solid legal framework to support Mexico's national innovation system. These achievements are further discussed below. (See also Annex I with further detail on performance indicators.)

The objective of supporting excellence in scientific and technological (S&T) research, increasing the availability of S&T human capital, and instituting an integrated strategy for development of fields of science was achieved in a highly satisfactory manner. The project has made very substantial contributions to Mexico’s knowledge base. It has helped to deepen and consolidate reforms of science research funding, which were initiated under the previous Science and Technology Infrastructure Loan. In line with international best practices, a transparent and merit-based selection process for research projects is now in place and accepted by the scientific community. Also, the project has promoted research in fields that were traditionally overlooked and has greatly expanded the level of advanced human capital. Furthermore, the legal reforms to Mexico’s science and technology system (2002) have provided an integrated and strategic framework for the development of fields of science.

The above is further evidenced by the following key sectoral outcomes. (See Annex 1.)

• Increased scientific production. Mexico's scientific publications raised significantly both in absolute and relative numbers during the project period. In absolute terms, the number of

- 7 -

international scientific articles published per year increased by 46 percent from 4,031 articles in 1998 to 5,885 articles in 2004. As a result, Mexico’s share of worldwide scientific publications grew from 0.5 percent in 1998 to 0.76 percent in 2004.

• Improved formation of highly skilled human capital and its insertion in the private sector. The project had a very positive impact on the overall stock of highly skilled human capital in Mexico. In 2004, 1,717 doctoral students completed their degrees, a jump of over 100% from 833 graduates in 1998. And, the number of doctoral graduates per million inhabitants increased by 78% in between 1998 and 2004.

• Furthermore, the total number of researchers in Mexico climbed by 55 percent during the period 1998 and 2004, with numbers in the private sector growing at faster pace than in the public sector. As a result, the proportion of researchers in the private sector to total researchers almost doubled during the project life (from 17% in 1998 to 33% in 2004). The technological capacity of the private sector was therefore significantly strengthened.

There are clear links between the project, which provided 10 percent of CONACYT's funding during its implementation period, and the aforementioned sectoral outcomes. CONACYT is the single most important financial source for research projects in Mexico and funds 54 percent of post-graduate scholarships, producing a similar level of PhD graduates, many of which become researchers. The project's Young Researchers' Program (section 4.2.A) has also facilitated the transition to a research career.

Achievements with regard to the objective of supporting increased firm-level productivity through provision of decentralized, demand-driven technological services for small and medium sized enterprises (SMEs) and creation of a pilot private sector-led venture capital scheme were moderately unsatisfactory. A pilot venture capital scheme has been successfully established. The fund (with a US$6 million public contribution) has been able to leverage US$20 million from the private sector, providing resources for start up technology companies, all of which are still on-going and expanding. However, technological services supported by the project (Technology Modernization Fund) only reached a limited number of SMEs and a productivity analysis was not conducted. That said, preliminary findings of the Sectoral Fund for Economic Development, which superseded the Technology Modernization Fund, suggest that the fund is achieving the objectives of its predecessor, and a full impact evaluation and productivity analysis will be available within the next few months.

The objective of facilitating linkages between private firms, universities and research institutions through financial support for joint activity and technical assistance to bridge institutions was moderately achieved. The Public S&T Institutes were successfully restructured and are now well integrated within Mexico’s national innovation system. They have expanded their linkages to the private sector, as evidenced by increased level of self-generated income, which rose on average by 87 percent over the life of the project. But, less than sufficient linkages between private firms and universities were developed in the context of the project. Only a third of the estimated industry-academic projects were supported. Those supported were quite successful. Sixteen new products were commercialized or processes put into operation, and seven patents were filed.

In addition, the project contributed to the overall objective of “promoting the generation, diffusion and application of knowledge for innovation in support of economic and social development” by facilitating the development and

- 8 -

subsequent implementation of a coherent legal framework for Mexico's science and technology system. The policy work conducted during project preparation and early project implementation contributed to the two legal reforms of 2002 (section 3.4). The new laws led to a fundamental restructuring of how resources are allocated in science and technology and provided the basis for a more articulated national innovation system. They resulted in a redistribution of responsibilities and better coordination among relevant government entities. New sectoral funds were created that: (i) increased the relevance of research through a vastly improved definition of priorities; (ii) instituted transparent and peer review-based selection processes; (iii) emphasized human capital formation; and (iv) increased the overall amount of resources available for research. Prior to these changes, innovation related funds were often assigned to line ministries without proper coordination and were not subject to adequate guidelines to foster a transparent allocation of public resources.

Finally, CONACYT’s mission has been broadened and the institution has embraced a holistic approach towards science, technology and innovation, outcomes that the project supported during preparation and implementation.

4.2 Outputs by components: A. Science and Technology Research(actual cost: US$445.88 million, or 156 percent of appraisal estimates; IBRD US$263.45 million)

The implementation of this component was highly satisfactory. The subcomponent supported the successful implementation of a large number of research projects and promoted research in new fields and traditionally neglected areas. Most importantly, the quality and relevance of research projects substantially improved through the introduction of more efficient and transparent project selection procedures and innovative funding mechanisms and initiatives. In addition, a substantial number of human resources at the graduate level were formed through scholarships and research project support, having created strong foundations for the generation of new knowledge.

Research Projects (actual cost: US$359.41 million, IBRD: US$176.97 million).

The component supported CONACYT’s research projects approved during the period 1998-2003, a considerable volume of Research and Development (R&D) that can be viewed as an “engine” of the Mexican Science and Technology system. At the institutional level, the project enhanced and expanded the use of the peer reviewer system, whereby high-level expert committees were formed in order to foster a professional and transparent selection of research projects. Highly recognized international experts were invited to participate in these evaluation committees. As the project advanced, Bank missions could appreciate the peer review system being used more fully and effectively. These new selection mechanisms led to a more efficient and relevant allocation of public funds for research and enhanced the quality of the research portfolio in Mexico. CONACYT also established a Technical Secretariat to strengthen processes for evaluating research proposals. The Secretariat comprised a number of experts that supported the evaluation committees and coordinated peer reviewers. In addition to project evaluation, the Secretariat (later absorbed by the Directorate for Science) helped to improve the monitoring and ex-post impact evaluation of funded projects. Moreover, accountability for research results improved through follow-up seminars and workshops to evaluate research output. The size of the average grant was increased permitting the

- 9 -

funding of new projects that were formerly ineligible and reducing the administrative burden imposed on researchers.

An electronic management system was developed to process research grant proposals, streamlining the selection process, decreasing ad hoc reviewer response times, and allowing for the enlargement of the pool of potential reviewers (especially those residing outside Mexico). A total of five manuals, addressing various aspects of the research process (inter alia proposal submission, selection, monitoring and evaluation), were developed and widely distributed to the scientific community.

Although there is a considerable time lag between the approval and the outcomes of research projects, preliminary results are very encouraging showing an upward trend that seems to be consolidating. The number of research projects approved by CONACYT rose from an average of 598 projects per year in 1998/1999 to an average of 657 projects in the period 2002/2004. Table 4.2 shows the outcomes of projects approved during the period 1998-2000 with a significant rise in the number of patents and published articles in indexed publications. The outcomes for later projects are still being compiled and will appear in future reports.

Table 4.2. Outcomes of Research Project Approved during 1998-2000

Year of Project Approval Indicator

1998 1999 2000

Number of approved projects1 604 592 615

Number of graduate degrees produced2, 3 1,386 1,773 2,118

Number of post doctoral researchers involved 23 22 49

Number of published articles in indexed publications4 2,793 4,867 4,288

Number of patents 10 27 25

Number of repatriated or retained researchers4 239 246 309

Notes: 1. Source: CONACYT. Numbers for the years 2001-03 were not included as only preliminary

data was available. However there is a clear upward trend 2. Number of specialization, master, and doctoral students that obtained their degrees while

participating in supported projects 3. Source: SABPI Database 4. Source: CONACYT’s Directorate for Science

Source: SIPPIC Database

A critical “by-product” of these projects has been the formation of highly skilled human capital. The number of “specialization” (one-year academic programs), master, and doctoral students that obtained their degrees while participating in supported projects jumped from 1,386 in 1998 to 2,118 in 2000, a notable 53 percent increase. There was a marked increase in the number of post-doctoral researchers involved in the projects, which is crucial to the development of the research system. Also, it provided more opportunities for recent PhD graduates. Finally, the number of publications and patents increased by more than 50 percent compared to the baseline.

- 10 -

In addition to funding “traditional” research projects, the project fostered new areas and modes of research, including (i) biotechnology projects, (ii) the Millennium Science Initiative (to promote centers of research excellence), (iii) projects in new fields of research and (iv) the Young Researchers program. These initiatives supported research in areas of crucial socio-economic importance that had traditionally been neglected in favor of the politically stronger, but perhaps not equally relevant, disciplines of research. In all cases, funding was significant and multi-year (2-4 years) permitting a major research impact. An international committee was formed to evaluate proposals in the above areas in conformity with international best practices, assuring greater objectivity and professional standards. The Millennium Science and new field development initiatives were later absorbed into the regular calls for proposals for research projects.

The call for proposals for biotechnology concentrated in the areas of agriculture and plants, health and molecular medicine, environment and bioremediation, and biodiversity. Twelve out of 48 proposals were selected and funded at a level close to US$3 million each. The Millennium Science Initiative program provided financing of US$1-1.5 million for each of the following four projects, selected out of 120 pre-proposals:

• Signaling, plasticity, and neurogeneration in the central nervous system;• Access to digital services and information for large communities of users;• High energy physics; and• Physical-chemical studies of new nano-structured materials.

Table 4.3 enumerates projects supported under New Fields of Research with an approximate funding of US$2 million each. Four projects were selected in each of the two calls for proposals, out of 163 and 43 pre-proposals respectively. The projects emphasized research in applied molecular biology with a potentially high socio-economic impact.

- 11 -

Table 4.3. Field Development Projects

Project Host Institutions

Development of Genomics in Mexico: Rhizobium Etli’s genome as a model system National Autonomous University, Morelos Campus

Medicinal Molecular Program in Gene Therapy National Nutrition Institute Salvador Zubirán

Biomolecular Materials Autonomous University San Luis Potosí and Sonora University

Radio-astronomic Interpherometry National Autonomous University, Morelia Campus

Material Science and Engineering applied to Petroleum Exploration and Exploitation in Mexico National Autonomous University

Biotechnology for the Genetic Improvement of Maize National Polytechnic Institute

Development of Vaccines and Analog Receptors against Emerging Infections National Nutrition Institute Salvador Zubirán

Cell Engineering and Biodiversity: optimization of cellular processes to increase the production of molecules of commercial interest National Autonomous University

The Young Researchers program has proven a highly effective tool to assist the best young Mexican researchers returning home from abroad or graduating from a domestic Ph.D. program. This program allowed for the return or retention of between 200 and 300 young researchers per year (Table 4.2). In the case of recent Ph.D. graduates, the project offered equipment and materials to launch a research project of their own, while the host university committed to providing staff positions for them. Hence, young researchers were offered attractive professional opportunities to return to Mexico.

Scholarships for Graduate Studies (actual cost: US$86.47 million, IBRD: US$86.47 million).

The subcomponent was successful in increasing the stock of advanced human capital (Table 4.4). Graduate studies were mainly conducted at the doctoral and master levels, and also included studies at the specialization level. CONACYT was responsible for supporting about 54 percent of post-graduate scholarships in Mexico in 2005. The share of support to PhD students rose from an average of 39 percent of all scholarships in 1998/1999 to an average of 46 percent in the period 2002/2004. This is a positive trend since doctoral level programs influence national skill production the most, generally producing high knowledge spill-over but marginal private returns.

- 12 -

Table 4.4. Indicators related to supported Scholarships for Graduate Studies

Indicator 1998 1999 2000 2001 2002 2003 2004 2005

Graduate students benefited1, 2 9,085 9,950 10,897 11,934 12,371 13,484 16,816 19,243

Doctoral graduates/ million inhabitants3 8.7 8.5 11.0 10.6 12.2 13.8 15.5 n.a.

Notes: 1. From 2001 on, the series only registers active scholarship recipients and excludes scholarships that

were cancelled or temporarily inactive. 2. Source: General Science and Technology Report, 2004. CONACYT 3. Source: Survey of doctoral graduates, 2005. CONACYT

In addition, the project supported the formulation and implementation of a new strategy for the formation of human capital. The main elements of the strategy were: (i) to strengthen linkages with market demand for graduates, particularly the private sector; (ii) focus on efficiency and improved graduation rates which reached 90%; and (iii) insert graduates in the labor market, with a tracking information system being implemented.

In 2004, CONACYT undertook an evaluation of the scholarship program including a beneficiary survey. The evaluation found that the scholarship program is essential for the production of doctoral graduates in Mexico since 90 percent of the beneficiaries reported that they would not have pursued further studies without financial assistance. Also, the survey revealed a high degree of satisfaction with the program’s requirements (83 percent) and selection methods (78 percent), but an acceptable degree of satisfaction with the amount (65 percent) and the length of financial support (62 percent). These responses are indicative of a well-functioning program that provides adequate but not generous financial support.

B. Industry-University Linkage (actual cost: US$13.89 million, or 9 percent of appraisal estimates; IBRD: US$11.04 million)

The implementation of this component was moderately satisfactory. The main accomplishment was the restructuring of the Public S&T Institutes. In addition there were two other subcomponents, Technological Assistance to Universities (a small activity) and Joint Industry-Academic Projects, which provided funding for collaborative projects led by companies. The last subcomponent resulted in a few successful collaborations between academic centers and the private sector, but its overall reach was very modest partly due to insufficient dissemination, the lack of an innovative culture in companies, and scarce tradition of working with public research centers. Despite these limitations, it provided some useful lessons and helped to create the institutional capacity within CONACYT to launch subsequent initiatives to continue strengthening linkages between research centers and the private sector (e.g., consortia programs).

Restructuring of Public S&T Institutes (actual cost: US$10.11 million, IBRD: US$7.28 million)

- 13 -

The restructuring of public S&T institutes (CONACYT research centers) was achieved. The centers improved the technological relevance of their research and were able to increase the level of self-generated income. The program commenced in 1997 with four pilot technology centers carrying out a strategic review and a business planning exercise. Gradually, the rest of the institutes developed a business plan and underwent a restructuring with significant progress in the organizational, financial, human resource and commercial areas which resulted in productivity gains. They also received critical research equipment. The restructuring process accelerated with the 2002 legal reforms, which granted operational autonomy to the institutes and mandated the creation of an external Board of Directors for each center with local and international representatives from the productive sector. The new committees brought strategic leadership to the centers and their restructuring process. The head of the public S&T institutes are now appointed by the Board, replacing the traditional practice of largely "political" appointees who were not always suitable for the job. As a result, the restructuring of these centers was achieved with less resources than initially programmed.

The restructuring of the institutes was instrumental in increasing the relevance of research and outreach to users. In particular, it promoted a change of behavior towards industry needs through the provision of new services, collaborative R&D projects and placement of highly qualified graduates at private firms, who became critical linkages between the centers and the companies. Very importantly, the quality of the scientific staff at the institutes improved. The share of researchers participating in the National Researchers System increased from 43 percent in 1998 to 52 percent in 2003 (Table 4.5). Scientific production rose from 1,356 scientific articles in 1998 to 1,636 articles in 2003, an increase of 21 percent.

Table 4.5 Public S&T Institutes: Evolution of Key Indicators

Indicator 1998 1999 2000 2001 2002 2003

Number of S&T institutes 28 28 28 29 27 27

Scientific staff in the National Researchers System1

(as percent of total scientific staff) 43.1 44.2 31.7 32.5 38.4 51.5

Number of scientific articles published 1,356 1,237 1,220 1,235 1,365 1,636

Source: Institutes’ Annual Reports, 1998-2003 Notes: The National Researchers System provides financial incentives to the best researchers in Mexico, selected through competitive mechanisms.

During the project period, self-generated resources of the publicly funded S&T centers grew on average by 87 percent, constituting 30 percent of their total funding by the end of the project, up from 20 percent in 1998. At the 10 scientific centers, self-generating resources increased by 68 percent during the 1998-2005 period. On average, nine technological centers almost doubled the resources generated through provision of paid services to industry--a strong indicator of enhanced client orientation. These results were largely achieved through improved strategic planning and management practices, which the project supported.

Joint Industry-Academic Projects (actual cost: US$2.68 million, IBRD: US$2.68 million).

- 14 -

The PAIDEC program (Programa de Apoyo a Proyectos Conjuntos de Investigación y Desarrollo) supported 33 company-led applied research and experimental development projects in 32 companies covering product and process development. CONACYT had estimated to support 95 projects through this 3-year long program, but insufficient promotion slowed demand. Despite the small number of beneficiaries, PAIDEC’s results were positive. An independent study that evaluated the project’s technology development component (Gorraez, 2005) found that more than half of the beneficiaries had not entered into collaborative projects with research institutions before. According to the study, 74 percent of the surveyed companies reported a positive experience of collaboration with research institutions. Many of the beneficiaries later continued collaborative work on their own.

As a result of the program, sixteen new products were commercialized or new processes were put in operation, and seven patents were filed. In addition, beneficiary companies hired 12 professionals with advanced degrees, an indicator of the value attached to advanced knowledge and human capital by participating firms.

Technical Assistance to Universities (actual cost: US$1.11 million, IBRD: US$1.08 million).

Out of 129 proposals received, the PROVINC Program (Programa de Apoyo a la Vinculación con la Academia) supported 37 technology transfer units at universities and CONACYT centers in the form of technical assistance and investment in information technology to promote outreach to the private sector. By the end of the project, 32 units continued operations (the remaining units were either decentralized to individual schools or discontinued due to the lack of university funding). The interventions were cost effective at an investment level of US$20,000 per university for two years. All units developed and implemented a business plan with PROVINC support. The unit services with highest private sector demand were training/continued education, consulting and laboratory services.

By the end of the program, 9 technology transfer units had filed a total of 24 patents. Also, 8 units had successfully implemented a PAIDEC funded project demonstrating that they could serve as a bridge for collaborative research with industry.

C. Enterprise Technology Enhancement (actual cost: US$58.64 million, or 31 percent of appraisal estimates; IBRD: US$17.8 million)

The implementation of this component was moderately unsatisfactory. The component launched a series of new initiatives to encourage small and medium enterprises (SMEs) to “catch up” technologically and increase productivity. Interventions addressed both demand needs (the Technology Modernization Program) and improvements in the supply of technological services (Regional/Sector Technology Centers). Overall, these two subcomponents offered some positive demonstration effects; the best technological centers, for example, can be models for similar centers to be established throughout the nation. These subcomponents, however, operated on a small scale and were not able to affect national outcomes. The Technology Modernization Program was subsequently replaced by the Sectoral Fund for Economic Development. Preliminary data suggests that the fund is reaching the objectives of the previous program. In addition, a venture capital scheme was piloted filling a crucial financing gap for start up technology companies.

- 15 -

Technology Modernization Program (actual cost: US$29.67 million, IBRD: US$14.22 million)

The Technological Modernization Program increased technological capacity of participating firms but was able to reach a limited number of SMEs. The program provided matching grants for projects implemented by 887 SMEs, 64 percent of which were small companies. During the period 1998-2001, 1,008 projects were approved, of which 780 were fully implemented: 59 percent of SMEs implemented one project, 37 percent two projects, and 4 percent implemented three projects. With the help of consultants, the enterprises produced “initial technology assessments” that served not only to design program supported projects, but also to stimulate and guide future projects with the companies’ own resources. The average grant amount was US$20,000 with firms contributing an equivalent amount. The three most important activities supported by the project were ISO certifications, process optimization, and reengineering (Figure 4.1). According to the survey carried out by Gorraez (2005), beneficiaries reported that the most important results were productivity improvements followed by increased sales and number of clients.

02468

101214161820

Perc

enta

ge

Quality

Proces

s optim

ization

Reengin

eering

Proces

s contr

ol

Best Pr

actices

Produc

tivity

IT

Others

Activity

Figure 4.1 Technology Modernization Projects: Distribution by Type of Activity

The bulk of participating SMEs (78 percent) belonged to the manufacturing sector (Figure 4.2). Within this sector, demand was concentrated in the four most dynamic sub-sectors (metal/mechanic, chemical, food processing and textiles), which were also facing strong external competition.

- 16 -

02468

1012141618

Perc

enta

ge

Metal/ m

echani

cChem

ical

Food

proces

sText

iles

Other m

anufac

turing

Agri/ fo

rest/

fish

Commerce

Service

s

Logist

icsOthe

rs

Sector

Figure 4.2 Technology Modernization Projects: Distribution by Economic Sector

The program faced the challenge of ensuring a wide geographical outreach. Broadly speaking, the geographic distribution of beneficiaries was correlated with the intensity of manufacturing activity at the state level: 59 percent of completed projects corresponded to the four states with the highest density of manufacturing activity, contributing 47 percent of the GDP (Figure 4.3). Perhaps, another factor explaining the geographic concentration of beneficiaries was that an insufficient number of program promoters was identified and trained. By the end of the program, there were 17 active promoters with clear limitations to cover 31 states plus the Federal District.

0

5

10

15

20

25

30

Perc

enta

ge

Jalisco

Edo. de

Mexic

o

Distrito

Federa

l

Guanaju

ato

Quereta

ro

Nuevo L

eon Others

State

Figure 4.3 Technology Modernization Projects: Geographical Distribution

The component could have been more effective and could have reached a far larger number of

- 17 -

firms with a more proactive dissemination program and higher grant amounts. These limitations were corrected when the fund was absorbed and scaled-up by the Sectoral Fund for Economic Development, jointly administered by CONACYT and the Ministry of Economy.

During the period 2002-2005, the Economic Sectoral Fund supported a total number of 267 projects, 43 per cent of which corresponded to small enterprises. Financing amounted to 654 million pesos or close to 62 million dollars. Of these, 44.5 percent of projects invested in the development of new products and services; 34.8 in new processes or technologies; and the remainder in the development of technology centers. Each million pesos invested by the Fund has leveraged an additional investment of 1.8 million pesos by private firms, has resulted in new sales of 20 million pesos, and the creation of 7.7 new jobs, twenty-five percent of which are highly specialized technical jobs. These findings are very promising, and the government is preparing itself to conduct a more-in depth evaluation of the program's additionality over the next few months.

Regional/Sector Technology Centers (actual cost: US$2.28 million, IBRD: US$2.28 million)

This subcomponent created six sector technology centers and strengthened four existing centers. The ten centers are currently operational, while seven of them are already established as sustainable operations with continued increased revenues. All of them are privately owned with a total of 146 entrepreneurs and firms having participated in their establishment. In 2004, they extended technological services to about 1,000 firms. The centers focus on specific economic sectors, namely food industry; agriculture; chemical/petrochemical; furniture; telecommunications; metal mechanic and automobile; and pharmaceutical. Five centers are laboratories for tests, analyses, and certification; two centers concentrate in agricultural research; another one performs quality systems evaluations and certification; another functions mainly as a broker for technological services; and another supports process and product technology development.

Table 4.6 Regional and Sector Technology Centers supported by the Project

Foundry and Machinery Institute of Jalisco Biotechnology Unit of Valle de Zamora

Agricultural Association of Río Fuerte del Sur BAMBUVER

LAAIF GOMCO Group Research and Development Center GIRSA

Cheese Producers Menonita Mexican Institute of Norms and Certification

Council for the Quality Promotion of Milk and Derivatives Laboratories LATTICE

Benefiting from project support, two centers achieved quality systems certification, four centers have their processes certified, and two additional centers partially certified. Finally, two centers generated patents: GIRSA with four patents and the Foundry and Machinery Institute of Jalisco with one patent.

These centers have been successful in providing critical technological services to enterprises, mainly

- 18 -

SMEs, but looking forward they need to expand their geographical outreach and/or new centers need to be created. Three centers are located in Jalisco, two in Distrito Federal and one in each of the following states: Estado de México, Chihuahua, Veracruz, Michoacán and Sinaloa.

Pilot Venture Capital Fund (actual cost: US$26.65 million, IBRD: US$1.27 million).

The venture capital fund was established after a lengthy, but worthwhile process, as a partnership between CONACYT and NAFIN. CONACYT funds up to 20 percent of a given venture and provides technical evaluation of start-ups requesting financing. NAFIN administers the fund and convenes private sector investors (for example, angel investors and private venture firms) to leverage public funds. Three companies were supported in 2004 and four companies in 2005 leveraging private sector investments of about ten million dollars each year. These enterprises operate in the fields of information and communication technologies, electronics, and aeronautics.

This technology venture capital fund builds on NAFIN's long trajectory with venture capital funds. The fund incorporates best practices: (i) an exhaustive due-diligence process with the participation of highly experienced business advisors; (ii) active involvement of angel investors in the boards of beneficiary firms; and (iii) training on corporate governance for entrepreneurs, which is resulting in increasingly professional management teams and board of directors. The fund’s operational guidelines detail rules for participation in company ownership and exit options. Typically, public investments in beneficiary firms are in the range of US$ 250,000-500,000, about 4-8 percent of total shares, and proportional voting rights. Exit options include the re-purchase of shares within a three-year period, valuated using discounted cash flows or comparisons to similar companies, at nominal value plus about a 25 percent annual rate of return.

The fund combines CONACYT’s technological knowledge and NAFIN’s experience with venture capital funds. It has adopted best practices, attracted much interest by angel investors, and appears to be filling a crucial financing gap for new technological firms. The amount of private sector resources leveraged is a positive indication of the fund’s performance. A more complete evaluation will be provided in the future once several companies have been spun off.

D. Institutional Strengthening (actual cost: US$9.62 million, IBRD: US$7.71 million)

The implementation of this component was satisfactory.Most importantly, this component supported a major reengineering and integration of CONACYT's information technology systems, an initiative known as RETO (Red de Eficiencia y Trabajo Organizado or Network for Efficiency and Organized Work). RETO has been crucial in increasing the cohesiveness of CONACYT’s programs across departments, which translates into a better allocation of resources and improved services to its beneficiaries. See section 4.5 for details on RETO’s institutional impact. It also supported the establishment of a project coordination unit (PCU) in 2002 which greatly facilitated project implementation and contributed to the resolution of the procurement issues referred to in section 5. The PCU was established at CONACYT within the Finance and Administration Directorate, and staffed by consultants who worked very closely with all of CONACYT's directorates.

4.3 Net Present Value/Economic rate of return:

- 19 -

Estimating precise rates of return for R&D investments undertaken in the project is difficult, but overall project results indicate that activities were cost-effective. The impact of available resources was maximized through the implementation of increasingly efficient and higher quality activities. The empirical literature also provides evidence that R&D investments result in higher social returns than investments in physical capital (for example, Lederman and Maloney (2003) and Griliches (1992)).

On the Science and Technology component, improved procedures supported by the project point to substantial gains in efficiency, quality, and relevancy of research. Efficiency has been improved through better selection of research projects by applying a more stringent and transparent peer reviewer system. This together with an increase in the average grant amount and extension of the implementation period has resulted in a higher concentration of resources in fewer but better projects and has reduced the administrative cost imposed on researchers, who used to spend substantial time applying for numerous small grants. On the Restructuring of Public S&T Centers (Component B), their greater reliance on private financing points to the higher relevancy of their research. Also, the production of scientific articles and number of patents resulting from research grants as well as from the restructuring of S&T centers increased. Benefitting from the governance reforms, the restructuring of these institutes was achieved in a more cost effective manner than initially planned. However, it is difficult to estimate the precise value and rate of return of the above activities due to commercialization lags and other challenges in quantifying scientific output.

The Scholarship for Graduate Studies program (Component A) was also very cost-effective, producing a large number of graduates at the doctoral level with a very high completion rate (90 percent). And, as supported by the economic literature, formation of advanced human capital is a key driver of technical progress (De Ferranti et al. 2003).

On the Technology Modernization Program (component C), ex-post investment recovery rates of technological investments by private firms are estimated in the range of 80-100 percent and more than 60 percent have continued to implement technological investments after the project. But, an impact analysis on enterprise productivity was not implemented. CONACYT is working on a new impact methodology for the Sectoral Fund for Economic Development that superseded this program. On the venture capital fund, a US$6 million public contribution has been successful in raising an additional US$20 million in private financing; all beneficiary firms are still on-going and their revenues are rising. A further evaluation of the rate of return will be available when firms start to spin off.

4.4 Financial rate of return:

Not applicable.

4.5 Institutional development impact:

Institutional development impact was substantial. In particular, the program was able to enhance substantially CONACYT’s institutional capacity. It helped to strengthen its key operational directorates, especially the Directorate for Technology, which evolved from a very small team to a full-fledge Directorate. The Directorate for Technology has now acquired the requisite capacity to implement a substantial component to promote business innovation under the follow-up Innovation for Competitiveness Loan (Credit 72960). As a result of the project, the Directorate for

- 20 -

Science is applying best international standards for the management of science and research programs under its purview.

The introduction of the RETO management information system enhanced CONACYT’s overall operational capabilities. RETO achieved the reengineering, standardization and integration of CONACYT’s key business processes, which correspond to the different stages of the project cycle, from call for proposals, selection, and monitoring and evaluation. All processes can be accessed through an internet portal. CONACYT also implemented an ISO 9001-2000 quality management system obtaining the corresponding certification by the end of 2004. Improvements in efficiency, organizational capacity, and customer service are further evidenced by the nomination of CONACYT as a finalist for the INTRAGOB award in 2005, which is a competition on quality management among government agencies.

The quality of management and research capabilities at the public S&T centers improved substantially resulting in higher client responsiveness and greater access to private sector financing. Moreover, several of these centers are now ready to participate in the consortia program (intended to link firms and public research centers) that will be implemented as part of the aforementioned follow-up operation. The regional/sectoral technology centers supported by the program are offering highly needed technological services to about 1,000 firms per year, and the best ones could serve as models for similar centers that should be established around the country.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:

There were no major external factors that affected project implementation and outcomes, barring procurement arrangements. The inadequacy of procurement arrangements for the Research Projects sub-component became an obstacle to smooth project implementation. Initially, the project followed decentralized procurement guidelines but these proved ineffective as discussed below (section 5.3). There was an attempt to overcome these difficulties by defining new procurement rules that consolidated purchases of scientific equipment into packages, as it is commonly used with standardized items. Scientific equipment, however, is usually highly customized to the specificities of particular research disciplines and is often provided by few providers or even a sole provider. The “packaging” of purchases implied that researchers frequently needed to modify their equipment request and select a second or third best option to fit the package and wait for a long period while the process was being conducted, only to often find out that there were no bidders. These implementation rigidities slowed down activities and caused dissatisfaction among beneficiaries.

5.2 Factors generally subject to government control:

Successive governments showed a strong support for the project during its preparation and implementation phases. Moreover, the 2002 legal reforms to the S&T system were fully consonant with the project’s objectives, seeking to establish a more effective national innovation system, increase the overall amount of resources available for S&T, and promote their more efficient use (section 3.4). They helped to accelerate some of the project activities that were lagging behind. The project was subsequently amended to be better aligned with this important and positive legal reform.

5.3 Factors generally subject to implementing agency control:

- 21 -

Project design called for decentralized procurement by the beneficiaries, which required extensive training and oversight, especially of universities participating in the Science and Technology Research Component. CONACYT, however, was not ready for these responsibilities and did not provide sufficient training and monitoring. Compliance with the project’s procurement rules turned out to be weak. This led to the issuance of new procurement rules, which also proved inadequate for the project’s needs and slowed implementation of the Science and Technology Research Component, as discussed in 5.1.

Initial procurement delays were compounded by the dispersion of implementation responsibilities among the different CONACYT directorates with technical oversight for project components. As a consequence, insufficient attention was paid to administrative and general management duties. This situation was substantially changed at the beginning of 2002 with the creation of the PCU, which assumed responsibility for overall project management in close coordination with the technical directorates.

5.4 Costs and financing:

The project was implemented with a total cost of US$528 million, equivalent to 80% of the appraisal estimate of US$663 million (Table A.2.1, Annex 2), while the disbursement of the loan amount totaled US$300 million, exactly as planned. Taking into consideration the size and complexity of the project, the final cost is very close to the appraisal estimate. However, there were important reallocations between components, which were justified, responding to changes in the environment in which the project was implemented and to the relative demand for project activities (see section 3.4).

6. Sustainability

6.1 Rationale for sustainability rating:

Sustainability of the project is highly likely. Institutional changes supported by the project are fully consolidated, and in many cases they are being further enhanced. The new practices for granting support for research projects are now fully institutionalized and recognized by CONACYT and the academic community. CONACYT continues to support scholarships programs, increasingly upgrading its practices to target them to graduate programs of excellent quality and high relevance for the private sector. CONACYT is utilizing the RETO system to its full potential and is planning to further upgrade it in the context of the follow up operation.

The public S&T centers are exploring opportunities for further collaboration with industry through the new consortia program. The Technology Modernization Program was superseded by the Sectoral Fund for Economic Development, which is jointly administered by CONACYT and the Ministry of Economy. The new sectoral fund benefited from the lessons learned during the implementation of its predecessor program. Seven of the Regional/Sector Technology Centers have already reached sustainability, while their revenue continues to expand. Several of these efforts are being supported in the follow up operation--Innovation for Competitiveness Program.

6.2 Transition arrangement to regular operations:n/a.

- 22 -

7. Bank and Borrower Performance

Bank7.1 Lending:

Lending performance is rated moderately unsatisfactory. As noted earlier, project design responded to strong government demand and was fully consistent with the Bank’s CAS. It included lessons learned from the previous operation (Mexico Science and Technology Infrastructure Project) and took full advantage of sectoral work undertaken prior and in parallel to project preparation. It also benefited from a wide range of consultations with external stakeholders.

Preparation, however, did not allow sufficient time to undertake a careful assessment of CONACYT’s capacity to implement reforms and new instruments and thus overestimated resources for Components B and C. Also, consultations with internal stakeholders on the change process were insufficient. More efforts should have been made to understanding change management and creating capacity within CONACYT’s technical directorates prior to launching the new instruments.

7.2 Supervision:

The supervision of the project was moderately satisfactory. The Bank team comprised a multisectoral team (including participation from the private sector and human development networks) and highly recognized international specialists. Counterparts acknowledged on numerous occasions the valuable technical assistance provided by the team. But, changes in task managers affected supervision at times. Attention from Bank management was adequate and responsive to implementation challenges and supportive of amendments to address pitfalls or to adapt the project to the new Science and Technology Law (2002). Bank management too provided adequate resources for project supervision.

7.3 Overall Bank performance:

Bank performance is rated moderately satisfactory. The project was fully supportive of the CAS goals and design benefitted from a strong body of analytical work, but project preparation did not pay adequate attention to institutional capacity issues. During supervision, the Bank worked proactively with CONACYT in identifying solutions to various bottlenecks, although greater continuity in task managers would have been desirable. Overall, the Bank's dialogue during preparation and implementation contributed to improvements in the design and implementation of CONACYT's instruments and public policies to support innovation.

Borrower7.4 Preparation:

The performance of the government during project preparation is rated moderately unsatisfactory. The government and CONACYT’s leadership were strongly committed to the project objectives to enhance the relevancy and effectiveness of S&T resources and to foster stronger bonds between research institutions and the productive sector. They advocated for expeditious project preparation and actively engaged with the Bank team. Limited attention, however, was paid to the capacity of the respective technical teams at CONACYT and their understanding of the reform process. Management’s vision was not sufficiently communicated to staff, who became concerned about the

- 23 -

new policy directions and their impact on CONACYT’s traditional areas of intervention. Staff was not adequately prepared for the reform, which slowed down the launch of new policy instruments.

7.5 Government implementation performance:

The implementation performance of the government is rated satisfactory. Project implementation spanned two government administrations, both of which gave high priority to the project. Moreover, the enactment of the new Science and Technology Law (2002) testifies to the highest level of political commitment to support reforms in the S&T system. As noted above, the law brought about a profound change on how resources are allocated for science and technology and created the basis for improved coordination within the national innovation system. Also, it helped to accelerate some of the reforms envisioned in the project.

7.6 Implementing Agency:

The overall performance of the implementing agency is rated moderately satisfactory. The project spanned several management teams, which consistently showed very strong commitment to the project. A number of pitfalls, however, emerged in the early years of implementation with responsibilities dispersed throughout the institution and with inadequate attention paid to change management. There was no close monitoring of procurement activities by beneficiaries early on.

Management responded to these challenges by creating a strong and very professional Project Coordination Unit in 2002. As stated in section 4.2.D, the unit was instrumental in helping to address procurement difficulties, ensuring closer project oversight, and helping to realize synergies among various project components. The Unit became an important liaison between the Bank and the various technical directorates. It also implemented several institutional building activities, in particular the establishment of an integrated management information system (RETO). Throughout project implementation, various technical directorates were strengthened as well.

7.7 Overall Borrower performance:

Overall Borrower performance is rated moderately satisfactory.The project benefitted from high commitment by the government and CONACYT's leadership during both project preparation and implementation. But, change management was not adequately conceived which slowed down activities during the early years of project implementation until various gaps were addressed.

8. Lessons Learned

• Building up human resource and research capacity is a long-term endeavor that requires concerted public efforts. Mexico is reaping the benefits of continued investment in these areas, positioning itself as a Latin American leader in terms of production of advanced human capital. The S&T capacity building strategy has been reinforced by CONACYT's co-funding of scholarships and research grants with state governments (Mixed Funds) and line ministries (Sectoral Funds). This is an example to be considered by other countries, as a way to leverage the use of public resources, increase policy coherence and maximize development impact.

• Effective support programs for private sector technology development need adequate time for demand to be properly articulated. The private sector does not have sufficient awareness and understanding of

- 24 -

technology and R&D issues, which slows implementation of these programs. And, it frequently lacks confidence in government programs, especially if R&D support in the past has been volatile. Confidence needs to be rebuilt which requires time. The programs should also allow resources to help to finance the formulation of new technological projects due to private sector weaknesses in technological areas.

• Related to the above, the importance of dissemination activities cannot be underestimated. The successful implementation of new S&T programs requires well-designed dissemination programs that are able to reach targeted beneficiaries. Dissemination activities, however, were perceived as secondary and not pursued actively enough hindering the potential impact of the Technology Modernization component and the Industry-Academy Linkage subcomponents. The mechanisms for promoting private demand under the Technology Modernization Component were also cumbersome and inadequate to reach a large number of companies.

• Project design should be simple and flexible to facilitate implementation and allow the project to adapt to changing conditions and emerging opportunities. Operations benefit from having a simple structure with a limited number of components and subcomponents, each with a clear and strategic focus. The Knowledge and Innovation Project had too many different subcomponents, some of which were very small, complicating implementation. The new Science and Technology Law involved institutional changes which created opportunities to enhance the allocation of S&T resources. Mid-way through implementation, the Knowledge and Innovation Project was adapted to take into account the new framework.

• Project design needs to define clear performance indicators which are closely connected to project objectives and interventions. The Knowledge and Innovation Project indicators were not clear and comprehensive enough at the outset, and these had to be discussed during implementation. Moreover, performance indicators of policy interventions in the area of innovation are often hard to identify since most of these interventions have a long time lag before results and positive spillovers can be fully observed. Further analysis in this area is necessary.

• Effective change management is crucial to foster successful organizational change. While project design included a careful assessment of fiduciary arrangements and related capacity, change management aspects and the organizational culture were not properly taken into account. Adequate communication and engagement of staff in the development of new policies, processes and products are critical to facilitate institutional and organizational change and prevent resistance from internal stakeholders.

• Procurement rules need to be customized to project needs. Procurement of scientific equipment is much more specialized than other types of procurement. Rules for procuring scientific tools should be flexible enough to permit the acquisition of the most suitable pieces of equipment, while maintaining transparency and adequate ex-post controls. A pilot with the Universidad Nacional Autónoma de México allowed for direct purchases of small equipment and scientific inputs costing less than US$5,000, a flexibility that was needed due to the trial and error nature of experimentation, often requiring to buy different consumables depending on the results of a given experiment.

9. Partner Comments

(a) Borrower/implementing agency:

- 25 -

CONACYT (Implementing agency)

From the standpoint of CONACYT, the implementation of the Knowledge and Innovation Project is considered satisfactory, taking into account all its facets and a comprehensive view of its development and implementation.

Financing from multilateral agencies is part of the Mexican government’s financing strategy. CONACYT has been part of this strategy during approximately the last 14 years, through the implementation of projects seeking to strengthen the country's scientific, technological and innovation capacity, with the purpose of supporting Mexico's economic and social development.

The World Bank's contribution to the science and technology sector has meant, beyond its financial support, a permanent source of technical assistance through its monitoring, evaluation, and advisory activities, both at the macro level (development/implementation of sectoral policies) and at the micro level (the diversity of programs and activities that have been developed throughout these years). There was always a constructive and proactive dialogue among the different actors affecting the implementation of these projects, which has motivated a constant reflection on how the state's intervention can be targeted to develop a national system of innovation that responds to society's needs as a whole.

The Knowledge and Innovation Project provided important lessons, which became a valuable input during the preparation of the Innovation for Competitiveness Program (January 2006). The main lessons learned are described in the previous section (8. Lessons Learned). However, we consider appropriate to mention the following:

Flexibility in project design and implementation. Mexico's circumstances as a country in transition ltoward a more advanced development stage mean that its systems (legal and regulatory, budgetary and financial, public policies, technical knowledge, etc.) are evolving, a situation that would potentially impact the performance of current projects and those to be implemented in the coming years.

Bank's recognition of Mexico's own systems. Despite being a subject of general discussion (mainly on lfiduciary aspects) between Bank management and the Mexican government, we highlight the Bank's vision towards CONACYT, which acknowledged activities and/or programs implemented in different areas and complemented these with recommendations on best international practices.

Guidelines and norms. Linked to the previous points, there is a need to use mechanisms that lrespond to the particular circumstances and needs of the project, both of CONACYT and its final beneficiaries. In particular, the guidelines for procurement of equipment under the Support for research component reflected a poor understanding of the needs of its final beneficiaries during both project preparation and the definition of a solution at mid-term.

Need to move forward on financial management and audit systems. Similarly, these are subjects in which lthe dialogue between the government of Mexico and the Bank continues to advance. The direct participation of the implementing agencies in this process, however, is very important, especially in the specific design of projects and programs. Also, it is very important to establish

- 26 -

a policy of continuous review and flexibility in order to adapt the project to changing needs.

Comments from the Ministry of Finance (Secretaría de Hacienda y Crédito Público, SHCP)

The last paragraph (fifth line) in the section on Assessment of Project Objectives and Original Components, notes that the problem (too many subcomponents) was caused because the project did not increase the resources available for CONACYT, but on the contrary implied a reallocation of the portfolio toward new activities. The SHCP notes that this comment is not reasonable since the absence of additional financing should have been foreseen from the start. The government´s policy clearly establishes that Bank loans do not represent additional resources to the executing entities.

The section on Bank supervision does not indicate that the Bank team underwent three changes of task manager, including one that the government knew little. This partly affected Bank supervision since missions were not conducted as regularly as desired.

Comments from Nacional Financiera

We have reviewed the document and consider that its content accurately reflects the development of the project. In this regard, it is important to highlight the need that projects be evaluated early on to prevent its design from conflicting with the norms/plans to utilize external financing resources and to ensure that institutional capacity of the executing agency does not become an obstacle to using external resources. Moreover, the document reflects the flexibility that IBRD manifested in due course to modify procurement procedures to accommodate the specific requirements of the scientific projects that were supported with project financing.

(b) Cofinanciers:n/a.

(c) Other partners (NGOs/private sector):n/a.

10. Additional Information

n/a.

- 27 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Table A.1 Outcome / Impact Indicators:

Indicators were sharpened during the life of the project and are more specific than those included in Annex I of the PAD, which had not defined any specific output targets. Also, a broader set of outcome indicators was measured during project implementation and at project completion since the initial PAD outcome indicator was not comprehensive and could only capture accomplishments related to some of the project objectives (mainly component 3 and partially on component 2). A broader set of indicators, for example, was necessary to measure project outcomes related to the project objectives of generating new knowledge for innovation (supporting excellence in S&T research and increasing the availability of S&T human capital).

Indicator/Matrix Baseline (12/31/1998)

Actual/Latest Estimate (12/31/2004)

Target (12/31/2005)

Investment in R&D 0.38% of GDP 0.45% of GDP 0.47% of GDP

Private R&D investment 0.09% of GDP 0.15% of GDP 0.17 of GDP

Private R&D investment – percentage of total national investment 24% 34% 37%

International articles 4,031 5,885 6,300

Mexico’s share of scientific articles in the world production 0.57% 0.76% 0.81%

Availability of Human capital: Doctoral graduates 833 1717 2000

Total number of researchers 22,190 34,485 36,700

Total number of researchers per 1000 employed 0.5 0.81 0.85

Number of researchers in the private sector

4,117 (17% of all) 11,458 (33% of all) 12,680

Impact on enterprise productivity of Technology Modernization Fund (TMF) --

Impact analysis on enterprise productivity for TMF not conducted. Such analysis will be available within the next few months for Sectoral Fund for Economic Development which replaced TMF.

--

Note: It is difficult to ascertain how much of the rise in private R&D responds to project interventions, but the project’s impact on private R&D is likely to have been modest.

- 28 -

Output Indicators

Indicador/Matrix Baseline (12/31/1998)

Latest Estimate* (12/31/2004)

Component A. Science and Technology Research

Number of approved projects 604 615**

Number of graduate degrees produced 1,386 2,118**

Number of post doctoral researchers involved 23 49**

Number of published articles in indexed publications 2,793 4,288**

Number of patents 10 25**

Number of repatriated or retained researchers 239 309**

Graduate students benefited with scholarships 9,085 16,816

Component B. Industry-University Linkage

Number of Joint Industry-University Projects supported by CONACYT 0 33

Percentage of staff from public S&T Institutes belonging to the National Researchers System 43.1 51.5

Number of scientific articles published by staff from public S&T institutes 1,356 1,636

Additional number of patents filed by companies 0 7

Additional number of patents filed by technology transfer offices 0 24

Component C. Enterprise Technology Enhancement

Number of SMEs supported by the Technology Modernization Program 0 887

Technological upgrading projects fully implemented by participating firms 0 780

Number of firms serviced by Regional/ Sector Technology Centers 0 1,000

Additional number of patents filed by Regional/ Sector Technology Centers 0 5

Number of new ventures supported by the Pilot Venture capital Fund 0 7***

*Actual values correspond to 2004 or latest year available. **Number refers to 2000. Projects approved in 2004 were 663 ***Value refers to 2005

- 29 -

Annex 2. Project Costs and Financing

Table A.2.1 Project Costs by Component (in US$ million equivalent)

Appraisal Estimate

Actual/Latest Estimate

Percentage of Appraisal Component

US$ million US$ million percentage

A. Science and Technology Research 285.00 445.88 156

A.1-3 Support for Research Projects and Field Development 285.00 359.41 126

A.4. Scholarships for Graduate Studies 0.00 86.47 n.a.

B. Industry - University Linkage 156.44 13.89 9

B.1. Restructuring of Public S&T Institutes 66.00 10.11 15

B.2. Joint Industry-Academic Projects 71.24 2.68 4

B.3. Technical Assistance to Universities 19.20 1.11 6

C. Enterprise Technology Enhancement 191.33 58.64 31

C.1. Technology Modernization Program 67.49 29.67 44

C.2. Regional/ Sectoral Technology Centers 70.84 2.28 3

C.3-4 Pilot Venture Capital Fund and Special Pilot Studies 53.00 26.69 50

D. Institutional Strengthening 0.00 9.62 n.a.

E. Unallocated 30.00 0.00 0

Total Project Costs 662.77 528.02 80

Note: n.a.: not applicable

- 30 -

Table A.2.2 Project Costs by Financing Source (in US$ million equivalent)

Appraisal Estimate Actual/Latest Estimate % of Appraisal Component

Bank Govt. Private Sector Bank Govt. Private

Sector Bank

A. Science and Technology Research

135.0 150.0 263. 182.4 195%

B. Industry - University Linkage 62.5 48.7 45.2 11.0 2.9 18%

C. Enterprise Technology Enhancement

72.5 10.6 108.3 17.8 6.3 34.5 25%

D. Institutional Strengthening 7.7 1.9

E. Unallocated 30.0 0.0

Totals 300.0 209.3 153.5 300.0 193.5 34.5 100%

Table A.2.3 Reallocation of Loan Proceeds (US$ million equivalent)

Component Amendment Sep. 18, 2002

Amendment June 6, 2003

Reallocation July 7, 2005

Closing Account Dec.

6, 2005

A. Science and Technology Research 135.00 250.37 263.45 263.45

A.1-3 Support for Research Projects and Field Development 135.00 178.80 176.97 176.97

A.4. Scholarships for Graduate Studies 0.00 71.57 86.47 86.47

B. Industry - University Linkage 62.50 12.70 11.04 11.04

B.1. Restructuring of Public S&T Institutes 26.00 7.37 7.28 7.28

B.2. Joint Industry-Academic Projects 31.50 3.90 2.68 2.68

B.3. Technical Assistance to Universities 5.00 1.43 1.08 1.08

C. Enterprise Technology Enhancement 72.50 25.30 17.80 17.80

C.1. Technology Modernization Program 30.00 16.95 14.22 14.22

C.2. Regional/ Sectoral Technology Centers 35.00 3.31 2.28 2.28

C.3-4 Pilot Venture Capital Fund and Special Pilot Studies 7.50 5.04 1.31 1.31

D. Institutional Strengthening 10.00 11.63 7.71 7.71

E. Unallocated 20.00 0.00 0.00 0.00

Totals 300.00 300.00 300.00 300.00

- 31 -

Table A.2.4 Project Costs by Procurement Arrangements (US$ million equivalent)

Appraisal Estimates Actual Costs Expenditure Category

ICB NCB Other Totals ICB NCB Other N.B.F Totals

Goods 36.8 6.8 1.4 45.0 8.7 1.2 2.2 0.0 12.2

Services (Consultants, training , promotion)

0.0 0.0 53.2 53.2 0.0 0.0 3.6 1.6 5.2

Grants and Scholarships 0.0 0.0 564.6 564.6 7.7 2.1 431.4 63.4 504.6

Project Administration (Operating Costs)

3.6 0.0 2.5 0.0 6.1

Unallocated 0.0 0.0 30.0 30.0 0.0 0.0 0.0 0.0 0.0

Totals 36.8 6.8 619.2 662.8 20.0 3.4 439.7 65.0 528.0

Table A.2.5 Loan Amounts by Procurement Arrangements (US$ million equivalent)

Appraisal Estimates Actual Costs Expenditure Category

ICB NCB Other Totals ICB NCB Other Totals

Goods 2.8 3.8 1.4 8.0 7.4 1.1 1.9 10.4

Services (Consultants, training , promotion) 0.0 0.0 35.5 35.5 0.0 0.0 3.6 3.6

Grants and Scholarships 0.0 0.0 256.5 256.5 5.8 1.6 274.0 281.3

Project Administration (Operating Costs) 2.8 0.0 1.9 4.7

Unallocated 0.0 0.0 30.0 30.0 0.0 0.0 0.0 0.0

Totals 2.8 3.8 293.4 300.0 15.9 2.6 281.5 300.0

- 32 -

Annex 3. Economic Costs and Benefits

Estimating precise rates of return for R&D investments undertaken in the project is difficult, but overall project results indicate that activities were cost-effective. The impact of available resources was maximized through the implementation of increasingly efficient activities. The empirical literature also provides evidence that R&D investments result in higher social returns than investments in physical capital (for example, Lederman and Maloney (2003) and Griliches (1992)).

On the Science and Technology component, improved procedures supported by the project point to substantial gains in efficiency, quality, and relevancy of research. Efficiency has been improved through better selection of research projects by applying a more stringent and transparent peer reviewer system. This together with an increase on the average grant amount and extension of the implementation period has resulted in a higher concentration of resources in fewer but better projects and has reduced the administrative cost imposed on researchers, who used to spend substantial time applying for numerous small grants. On the Restructuring of Public S&T Centers (Component B), their greater reliance on private financing points to the higher relevancy of their research. Also, the production of scientific articles and number of patents resulting from research grants as well as from the restructuring of S&T centers increased. Benefitting from the governance reforms, the restructuring of these institutes was achieved in a more cost effective manner than initially planned. However, it is difficult to estimate the precise value and rate of return of the above activities due to commercialization lags and other challenges in quantifying scientific output.

The Scholarship for Graduate Studies program (Component A) was also very cost-effective, producing a large number of graduates at the doctoral level with a very high completion rate (90 percent). And, as supported by the economic literature, formation of advanced human capital is a key driver of technical progress (De Ferranti et. al. 2003).

On the Technology Modernization Program (component C), ex-post investment recovery rates of technological investments by private firms are estimated in the range of 80-100 percent and more than 60 percent have continued to implement technological investments after the project. But, an impact analysis on enterprise productivity was not conducted. As noted earlier, CONACYT is working on a new impact methodology for the Sectoral Fund for Economic Development that superseded this program. On the venture capital fund, a US$6 million public contribution has been successful in raising an additional US$20 million in private financing; all beneficiary firms are still on-going and their revenues are rising. A further evaluation of the rate of return will be available when firms start to spin off.

- 33 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/Preparation10/14/1997 5 TEAM LEADER (1);

TECHNOLOGY POLICY SPEC (1); SCIENCE POLICY SPEC (1); CONSULTANT (2)

Appraisal/Negotiation04/14/1998 7 TEAM LEADER (1);

TECHNOLOGY POLICY SPEC (1); SCIENCE POLICY SPEC (1); CONSULTANT (2); PROCUREMENT SPEC (1) AND FINANCIAL MANAGEMENT SPEC (1)

04/28/1998 8 TEAM LEADER (1); TECHNOLOGY POLICY SPEC (1); SCIENCE POLICY SPEC (1); CONSULTANT (2); PROCUREMENT SPEC (1) AND FINANCIAL MANAGEMENT SPEC (1); COUNSEL (1)

Supervision05/24/1999 7 TEAM LEADER (1);

TECHNOLOGY POLICY SPEC (1); SCIENCE POLICY SPECIAL (3); UJIVERSITY-INDUSTRY LI (1); MATCHING GRANT SPECIAL (1)

S S

11/18/1999 2 TEAM LEADER (1); TECHNOLOGY SPECIALIST (1)

S S

06/26/2000 3 SPECIALIST (2); ECONOMIST (1)

S S

06/13/2001 4 TEAM LEADER (1); SPECIALIST (1); CONSULTANT (1); PROCUREMENT SPECIALIST (1)

S S

01/28/2002 4 TTL (1); TECHNOLOGY DEVELOPMENT (1); PROCUREMENT SPECIALIST (1); PROJECT ASSISTANT (1)

S S

10/25/2002 3 TASK MANAGER (1); S S

- 34 -

ECONOMIST (1); CONSULTANT (1)

01/19/2003 5 TASK TEAM LEADER (1); ECONOMIST (1); CONSULTANTS (2); SECTOR LEADER (1)

08/30/2003 6 TASK MANAGER (1); ECONOMIST (1); SR. EDUCATION SP (1); LEAD ECONOMIST (1); CONSULTANT (ESF) (1); CONSULTANT (OECD) (1)

S S

04/26/2004 4 TASK TEAM LEADER (1); ECONOMIST (1); CONSULTANT (1); FINANCIAL MANAGEMENT SPECIALIST (1)

S S

08/04/2004 5 TASK TEAM LEADER (1); ECONOMIST (1); CONSULTANT (1); FINANCIAL MANAGEMENT SPECIALIST (1); EDUCATION SPECIALIST (1)

S S

04/15/2005 3 1 TASK TEAM LEADER; (1) FM SPECIALIST; (1) EDUCATION SPECIALIST

S S

ICR09/26/2005 5 TASK TEAM LEADER (1);

ECONOMIST (1); PROCUREMENT SP (1); SPECIALIST IN SCIENCE AND TECHNOLOGY (1); FINANCIAL MANAGEMENT SPEC. (1)

S S

PS: There is no more available information in Bank's systems on preparation missions.

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/Preparation N/A in SAP 54.9Appraisal/Negotiation 9.65 65.7Supervision 95.08 516.36ICR 9.05 33.88Total 113.78 670.84

- 35 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

- 36 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 37 -

Annex 7. List of Supporting Documents

CONACYT. Operational Manual, Mexico: Knowledge for Innovation Project, Mexico, D.F.

CONACYT. March 31, 1999. Annual Self-evaluation Report, Mexico, D.F.

CONACYT. 2001. Challenges on scientific and technological capacity and competitiveness. Diagnostic 2001-2006 (Retos actuales en materia de capacidad científica y tecnológica, y competitividad. Diagnóstico 2001-2006).

CONACYT. 2004. 2001-2003 Report and Perspectives for 2004 (Informe 2001-2003 y Perspectivas para el 2004).

CONACYT, 2004, 2005. General Report on Science and Technology (Informe General del Estado de la Ciencia y Tecnología).

Gorraez, Irma. 2005. Evaluation of the Technology Components, Knowledge and Innovation Project.

The World Bank. May 22, 1998. Project Appraisal Document, Mexico: Knowledge and Innovation Project; Report Number 17896, Washington, D.C.

The World Bank. July 14, 1998. Mission Note, Washington, D.C.

The World Bank. December 30, 1998. Implementation Completion Report, Mexico: Science and Technology Infrastructure Project (Loan 3475-ME); Report Number 18763, Washington, D.C.

The World Bank. June 2, 1999. Mexico: Private Sector Development Strategy and Operations, Washington, D.C.

The World Bank. June 26 - 30, 2000. Project Supervision Mission Aide-Memoire, Washington, D.C.

The World Bank. June 13 - 21, 2001. Project Supervision Mission Report and Aide-Memoire, Washington, D.C.

The World Bank. January 28 – February 1, 2002. Project Supervision Mission Report and Aide-Memoire, Washington, D.C.

The World Bank. October 16 - 24, 2002. Project Supervision Mission Report and Aide-Memoire, Washington, D.C.

The World Bank. January 19-21, 2003. Project Mission Report, Washington, D.C.

The World Bank. August 25 - 29, 2003. Project Supervision Mission Aide-Memoire,

- 38 -

Washington, D.C.

The World Bank. October 30 - 31, 2003. Project Supervision Mission Report, Washington, D.C.

The World Bank. April 26 – 29, 2004. Project Supervision Mission Report and Aide-Memoire, Washington, D.C.

The World Bank. August 24 – 31, 2004. Project Supervision Mission Report and Aide-Memoire, Washington, D.C.

The World Bank. August 26 – September 2, 2004. Project Identification Mission Report and Aide-Memoire, Washington, D.C.

The World Bank. September 26 – 30, 2005. Project Mission Aide-Memoire, Washington, D.C.

- 39 -

- 40 -