Independent Development infoDev) Global Trust …...4.6 Knowledge Sharing and Collaboration 97 4.7 Relationship to the World Bank 99 4.8 Sustainability 100 4.9 Conclusions & Issues

Independent Eva lua t ion o f the In format ion for Deve lopment ( i n foDev) Globa l Trus t Funded Program

Vo l u m e I I – C a s e S t u d i e s

M a r c h 2 0 0 7

V o l u m e I I – C a s e S t u d i e s

March 2007

© UNIVERSALIA 1288 p:\intl\1288 evaluation of info dev\final report\volume ii case studies\final\volume ii case studies final_04kr.doc

i

A c r o n y m s

ANRT Agence Nationale de Réglementation des Télécommunications

BPO Business Process Outsourcing

CICT Centre for ICT

CIDA Canadian International Development Agency

CITPO Communications and Information Technology Policy division of the International Finance Corporation

EOI Expression of Interest

GDP Gross Domestic Product

GeSCI Global e-schools and Communities Initiative

GICT Global Information and Communications Technologies Group

GIPC Ghana Investment Promotion Centre

GSR Global Symposium of Regulators

GTZ Deutsche Gesellschaft fur Technische Zusammenarbeit

ICT Information and Communication Technologies

ICT4D Information and Communication Technologies for Development

ICT-ITES Information and Communication Technologies and IT Enabled Services

IDRC International Development Research Center

IDSC infoDev Incubator Support Center

infoDev Information for Development

ITES Information Technology Enabled Services

KIAsia Kenan Institute Asia

M&E Monitoring and Evaluation

MDG Millennium Development Goal

MEIA Monitoring, Evaluation and Impact assessment

MIGA Multilateral Investment Guarantee Agency

RFP Request for Proposals

SME Small and Medium Enterprises

TAP Technical Advisory Panel

TDB Telecommunication Development Bureau

TMG Telecommunications Management Group

TOR Terms of Reference


March 2007


ii

A c r o n y m s

TWBI Tianjin Women Business Incubator

UNESCO United Nations Educational, Scientific and Cultural Organization

USAID United States Agency for International Development

WSIS World Summit on Information Society

YBIC Yangling Business Incubator Centre


March 2007


iii

C o n t e n t s

1. Case Study on Ghana ICT Study 67

1.1 Introduction 67 1.2 Background 67

1.2.1 Context 67 1.2.2 Rationale 67

1.3 Key Steps in Developing the Ghana Study 68 1.3.1 Design 68 1.3.2 Implementation 69

1.4 Management of the Initiative 70 1.5 Relevance to Stakeholders 70 1.6 Monitoring and Knowledge Sharing 71 1.7 Conclusion and Areas for Improvement 72

2. Case Study on the Incubator Initiative 73


2.2.1 What are Business Incubators? 73 2.2.2 infoDev’s Rationale for the Initiative 74

2.3 Design 74 2.4 Grant Portfolio 75 2.5 Implementation and Management of the Initiative 77

2.5.1 Network Building 78 2.6 Knowledge Sharing 79 2.7 Effects 80 2.8 Relevance to infoDev’s Key Stakeholders 81 2.9 Maintaining the Momentum 82 2.10 Conclusions and Areas for Improvement 83

3. Case Study on ICT Regulation Toolkit 85


3.2.1 Context 85 3.2.2 Rationale 86

3.3 Design 86 3.3.1 Content 86 3.3.2 Design of Modules 87


March 2007


iv

3.4 Implementation and Management of the Initiative 88 3.5 Relevance to stakeholders 90

4. Case Study on Work in the Education Sector 91

4.1 Introduction 91 4.2 ICTs and Education 91

4.2.1 infoDev’s Rationale 91 4.2.2 Identifying Needs for infoDev’s Work 92 4.2.3 Relevance to Key Stakeholders 94

4.3 Implementation 95 4.4 Knowledge Products 96 4.5 Effectiveness 97 4.6 Knowledge Sharing and Collaboration 97 4.7 Relationship to the World Bank 99 4.8 Sustainability 100 4.9 Conclusions & Issues 102

E x h i b i t s

Exhibit 2.1 Distribution of grants by region 75 Exhibit 2.2 Examples of infoDev supported projects 76 Exhibit 3.1 Contractees for the ICT Regulation Toolkit Modules 87 Exhibit 3.2 Expected Deliverables 89 Exhibit 4.1 Mapping out Mainstreaming ICTs: M&E Handbook as an example 93


March 2007


67

1 . C a s e S t u d y o n G h a n a I C T S t u d y

1 . 1 I n t r o d u c t i o n

This case study presents an account of how and why the study ‘Improving Business Competitiveness and Increasing Economic growth in Ghana: The role of ICT-ITES’ was developed.

The methodology used to complete the case study includes a review of key documents, including the concept note launching the study, the request for an Expression of Interest used to select consultants, a draft version of the study as well as copies of similar studies previously undertaken by infoDev. Face-to-face and telephone interviews were also conducted with key stakeholders involved in the development of the study.

The case study focuses on the actual processes that led to the development of report as well as its perceived relevance by stakeholders.

1 . 2 B a c k g r o u n d

1 . 2 . 1 C o n t e x t

Often seen as a model for political and economic reform in Africa, Ghana has been one of the most stable countries in West and Central Africa in recent years. Since the early 1980s, the country has experienced relatively strong economic growth, with real GDP growth reaching 5.8 percent at the end of 2004. While agriculture continues to be the main contributor to Ghana’s economy, with a doubling of exports in the cocoa sector, recent data indicates that the service sector is emerging as the second largest source of employment in the country. The various sub-sectors that are part of the larger service area include, among others, financial and banking services, and a nascent Information Technology Enabled Services (ITES) and Business Process Outsourcing (BPO) segment, which together provide some 2000 jobs.

A careful analysis of Ghana’s ITES-BPO segment indicates that Ghana holds key strengths that could trigger significant growth in this sector, such as a large pool of potentially good English speaking workers, a competitive labour differential to US and UK companies, increased availability of telecom and internet services, as well as strong government support for such economic activity. Despite some structural constraints, growth in the ITES-BPO sector has the potential of bringing important benefits in terms of job creation, increased Foreign Direct Investment flows as well as increased employment for women. Investments to attract international players and improve ICT infrastructure are important factors when leveraging the recent developments in this sector.

1 . 2 . 2 R a t i o n a l e

Given this context, the government of Ghana has engaged in a sustained dialogue with the World Bank regarding the growth of the business sector and the potential role that ICT-enabled businesses could play. In August 2006, a US$40 million World Bank loan to the government of Ghana was approved for the implementation of the selected components of the Government’s Information and Communication Technologies (ICT) for Accelerated Development Policy, also know as the e-Ghana project. The e-Ghana Project is composed of three key components, all of which are geared towards greater private sector participation: (i) Creating the Enabling Environment necessary for the growth of the sector; (ii) Supporting local ICT businesses and IT Enabled Services (ITES); and (iii) Promoting e-Government applications and Government communications.


March 2007


68

Prior to the implementation of the ICT and ITES components, both the government of Ghana and the Communications and Information Technology Policy (CITPO) division of the Global Information and Communication Technologies department (GICT), the World Bank Unit responsible for overseeing the loan, agreed that a preliminary study on Ghana’s business environment and opportunities in ICT was necessary to inform any investment decisions that would be made. infoDev was perceived to be an appropriate partner for the preliminary study because of its track record in conducting similar types of studies in the Caribbean and Tanzania. CITPO, generally concerned with operational and management matters, sought a partner that would have complementary expertise in developing knowledge products.

The rationale for infoDev’s involvement in the study was that it provided an opportunity to develop a framework to evaluate key drivers for an IT-enabled service sector in Ghana, which in turn would help refine their framework on growth and competitiveness and inform other studies. The study was also perceived as a means to engage with country stakeholders from the very outset of the project. This had not previously been the case with other studies developed as part of infoDev’s Competitiveness and Growth programming, as country stakeholders became involved in the study only later in the process.

1 . 3 K e y S t e p s i n D e v e l o p i n g t h e G h a n a S t u d y

1 . 3 . 1 D e s i g n

The first step in the development of the Ghana study was the drafting of a concept note, in early 2006, by infoDev staff involved in competitiveness and growth programming. It was then shared with the rest of infoDev’s team for review. The purpose of the concept note was to provide a short explanation of the subject matter to be analyzed, its relevance to the ICT4D and to infoDev’s programs as well its linkages to donors’ specific needs and interests. At this stage, infoDev’s team saw the need to break out of the typical ‘ICT hype’ and over-optimistic analysis in order to look carefully at the fundamental development factors in the study. It was important for the team to avoid the weaknesses of previous e-strategies, developed in mid-1990s by developing countries and donors, which were not necessarily rooted in an accurate contextual analysis and therefore not realistic in terms of their recommendations.

All stakeholders involved in the study, including infoDev, the government of Ghana, CITPO, the Multilateral Investment Guarantee Agency (MIGA) and the consultants, Hewitt Associates, agreed on the importance of first looking at Ghana’s key growth factors, and, based upon them, assessing the extent to which ICTs could play a contributory role in enhancing the economy’s competitiveness, bearing in mind the government’s overarching poverty reduction objective. Based upon this orientation, the study was to include the following components:

• A review of the Ghanaian economy’s position within the international business environment and the opportunities/challenges for increasing exports in ITES and in agri-processing and other sectors;

• A review of international best practice and Ghana’s experience to date in harnessing ICTs for building competitiveness and growth;

• A needs assessment with respect to the regulatory environment, human resources, ICT infrastructure and other constraints in Ghana; and

• Recommendations for action to address constraints and support the realization of Ghana’s potential in emerging sectors.


March 2007


69

While ensuring that the study would inform the implementation of the e-Ghana project, infoDev also wanted to ensure that the study’s design would provide a specific framework to assess Business Process Outsourcing (BPO) in a way that could be used by other donors. The possibility of extracting lessons from the study for a larger learning community remained an objective throughout the project.

1 . 3 . 2 I m p l e m e n t a t i o n

Given that prior discussions had already taken place with the CITPO and infoDev, the initial concept note for the study was reshaped into ToRs within approximately 3 weeks, a much shorter period of time than had been required for similar previous studies at infoDev. Ghana’s Ministry of Communication, MIGA and CITPO all collaborated with infoDev to ensure that what they perceived as key dimensions of the study would be included in the ToRs. Once this step was completed, an Expression of Interest was developed according to World Bank procurement guidelines. Because the project’s budget was under US$200,000, the procurement process was completed in just one phase in which infoDev invited various firms to submit their consultants’ qualifications. The key criteria in selecting Hewitt Associates, and more specifically their Indian consulting office, was the firm’s ability in providing not only a sound analysis of the business environment but also the firm’s expertise in making recommendations specific to Business Process Outsourcing.

The selection committee was composed of all the financial stakeholders of the study. Sources of funding for the US$190,000 budget were the following: the government of Ghana (US$85,000), MIGA (US$25,000) and infoDev (US$80,000). infoDev also allocated additional resources for the design and review process of the study and for its eventual dissemination. Resources earmarked to the study’s management equaled approximately 8 weeks of infoDev staff time.

Hewitt Associates, following the methodology that had previously been tested in other consulting assignments, undertook the core of the research. The methodology was based on the 5-drivers model that presents infrastructure, people, the legal environment, the financial environment and the industry cluster as the key factors influencing ICT enabled business development in a particular region. To complete the study, Hewitt Associates undertook a thorough review of competitiveness studies and ICT reports, documenting ICT initiatives and conducting wide consultations, including on-site visits as well as several face-to-face and email interviews with a variety of policy makers and stakeholders in Ghana, representatives of development agencies, foreign investors and other actors involved in ICT4D.

infoDev’s key responsibilities related to sharing background information with the consultants, reviewing the consultants’ drafts and providing comments, liaising with the government of Ghana, CITPO and MIGA and eventually, upon completion of the report, editing it in order to extract the general framework for assessing BPOs. It is worth mentioning that CITPO’s staff also participated in the discussions with consultants to provide some insight on the Ghanaian context.

An important step in the development of the study was the organization of a workshop in Ghana to share emerging findings and to stimulate discussions with various stakeholders on those findings. infoDev made significant efforts to invite experts in ICT-enabled services from India and South Africa, thus bringing additional credibility to the project and in turn encouraging a stronger commitment from the government of Ghana. The workshop was also an important opportunity to open a frank dialogue between Ghana’s government and the private sector, and provided insights on the state of public/private relations in the country.


March 2007


70

1 . 4 M a n a g e m e n t o f t h e I n i t i a t i v e

While a key challenge for the infoDev team in managing the study was to ensure optimal quality assurance, engaging the Government of Ghana in the review process was felt to be equally important from the viewpoint of fostering their ownership of the process. On the one hand, infoDev had the responsibility of meeting CITPO’s and MIGA’s requirements in terms of coherence between the study and the e-Ghana project components; on the other, infoDev also had to maintain transparent and sustained relationships with the government of Ghana, the key financial contributor to the project, in order to ensure its full participation in the review process.

At the beginning of the project, the Ministry of Communication was involved in selecting the consulting firm. Subsequently, for each version of the draft report, the Government of Ghana, through the Ministry of Communications, was invited to participate directly in the review process in order to share their views on the emerging findings presented by Hewitt Associates. In the process, the Ministry of Communications posted each new version of the report on their official website so as to engage as many stakeholders as possible and to ensure optimal transparency, especially with respect to actors from the country’s private sector.

The ‘relationship intensive’ or participatory nature of the study also provided a key opportunity to develop knowledge products with greater country involvement and thereby to increase the relevance of the study to end-users at the country level. Respondents from infoDev mentioned that this approach was an innovative way to test whether fostering closer relationships with the country can bring positive returns once a large project is implemented at a later stage. CITPO and MIGA representatives also mentioned that involving operational partners was the best way to ensure the study’s utility. On their part, Hewitt Associates stressed the effectiveness of infoDev staff in providing proper insights and feedback to the study and managing the on-going review process with various stakeholders. As for the Ministry of Communication, it expressed great satisfaction in having the opportunity to help shape the study and its recommendations, as well as to meet the needs of its key stakeholders. It was noted that infoDev handled the management of the study very professionally and responsively.

To maintain dialogue with multiple stakeholders, however, demanded more involvement and time from infoDev staff, which was a challenge given the limited human resources that were initially planned for the project’s management.

1 . 5 R e l e v a n c e t o S t a k e h o l d e r s

Stakeholders interviewed have agreed that the study developed was most relevant because it fed directly into the design of the ICT business/ITES component of the e-Ghana project. The study provides a detailed comparative analysis of Ghana’s positioning in the ITES sector, and will be of use to both the government of Ghana, in terms of the supply-side investment decisions, as well as to the World Bank’s CITPO branch in terms of monitoring the implementation of the project. From MIGA’s perspective, the study responds to their needs in that it can inform investments opportunities, mechanisms to promote investment in Ghana as well as specific channels through which MIGA can provide technical assistance, such as the Ghana Investment Promotion Centre (GIPC).


March 2007


71

Ghana’s Ministry of Communication emphasized that while the study provides comprehensive baseline data on the BPO industry in the country and recommendations tailored to the national context, there is a need for on-going support from infoDev to facilitate and assist the policy formulation that will be based it. One suggestion was for infoDev to support a group of local experts within the Ministry of Communication that would work to create the right enabling environment for BPO in Ghana. The extent to which infoDev is in a position to provide this type of guidance post-study remains to be assessed by the team.

Looking forward, infoDev respondents stressed that the demand for this particular type of study is more likely to come from the World Bank, with the support of national governments, given the scope of its operations in supporting private sector development. Supporting this view, World Bank staff involved in other infoDev country studies (Tanzania, Eastern Caribbean) noted a ‘natural’ collaboration given that infoDev staff is quite knowledgeable of World Bank country teams' needs.

infoDev respondents emphasized that the assessment framework used in the study could serve as a reference for donor agencies reflecting upon whether outsourcing should be supported in their recipient countries. Others noted that the study could be of use to many donors because of the detailed information it provides on Ghana’s economy.

With respect to relevance to infoDev in general, the team has envisaged ways in which the material from the study can be linked to other infoDev projects. For example, stakeholders from the incubator project sponsored in Ghana have been invited to participate in the workshop in order to learn about ITES opportunities and share their own experiences. In addition, some of the data presented in the study has served to inform the development of infoDev’s study on IT parks, due to be completed as part of its work in Growth and Competitiveness.

1 . 6 M o n i t o r i n g a n d K n o w l e d g e S h a r i n g

An important component of the Hewitt Associates’ study is the presentation of a roadmap for Ghana to strengthen and enlarge the share of its ITES-BPO market. In addition to highlighting specific market opportunities and niche areas in the country, the study proposes strategic recommendations to the government of Ghana and other industry stakeholders in order to address identified supply-side constraints. Such recommendations include, among others, enhancing employability and basic industry readiness skills, increasing PC penetration in education, developing local, middle management and technical capabilities for the industry and retaining skilled workers within it.

The extent to which such recommendations will inform stakeholders’ decision-making for the implementation of the e-Ghana project and in turn contribute to increased growth in the ITES-BPO sector remains to be seen, given that the project is still in its incipient stages. In its current format, the study does include an M&E framework, providing quantitative indicators that will help track the economic results emerging from the e-Ghana project. Qualitative monitoring of the actual process through which recommendations are shaped into decisions and actions, however, had not been planned for at the time of writing the case study. Respondents from infoDev and partners expressed an interest in monitoring this process but stressed that it would require additional resources not planned for at the study’s inception.

With respect to the dissemination of the study, it was intended from the very beginning of the project that the study would be shared with key stakeholders in Ghana as well as partners at the World Bank. For dissemination outside this circle, infoDev has planned to post the study on its website, thus permitting maximum general access. Going forward, the infoDev team is planning to develop an ITES-BPO assessment framework based on the study’s lessons learned and emphasizing this framework to encourage its use among other donors.


March 2007


72

1 . 7 C o n c l u s i o n a n d A r e a s f o r I m p r o v e m e n t

The present case study clearly indicates that infoDev has been effective in managing a multi-stakeholders work collaboration and communication in order to ensure optimal relevance of a knowledge product. infoDev was also described as very professional in its provision of quality assurance. Overall, infoDev has played a unique role of brokering knowledge that meets the specific needs of a country in the context of major capital investments.

Data collected suggests that management of the project was relatively time consuming for the infoDev staff, and this aspect may have been underestimated in the beginning. It was also noted that resources for monitoring the use of the study after its completion have not been sufficiently allocated. Given that knowledge generation is infoDev’s core business, this is an area that should be looked at more carefully in the future.

Respondents KEY INFORMANTS

Kerry McNamara, infoDev

Seth Ayers, infoDev

Navin Kumar, Hewitt Associate

Kwaku Ofosu-Adarkwa, Chief Director, Ministry of Communications of Ghana

Cecile Niang, CITPO, World Bank

Eduardo Hernandez, MIGA, World Bank

Muhamet Fall, MIGA, World Bank


March 2007


73

2 . C a s e S t u d y o n t h e I n c u b a t o r I n i t i a t i v e


This case study presents an account of the implementation and management processes of the Incubator Initiative as well as its relevance to donors and to long-term sustainable development.

The methodology used to complete the case study includes a review of key documents, including the concept note initiating the study, the Request for Proposal used to select participants, infoDev’s Directory of Incubators Project, the initiative’s mid-term review as well as the preliminary draft of its impact assessment. Face-to-face and telephone interviews were also conducted with key stakeholders involved in the management of the initiative as well as with donors involved in the initiative.


2 . 2 . 1 W h a t a r e B u s i n e s s I n c u b a t o r s ?

Business incubators are defined as ‘organizations that support the entrepreneurial process, helping to increase survival rates for innovative startup companies. Entrepreneurs with feasible projects are offered a menu of support resources and services, including: provision of physical space, management coaching, help in developing an effective business plan, administrative services, technical support, business networking, legal advice, and advice on sources of financing. A business incubator's main goal is to produce successful firms that are financially viable.’1

The concept of business incubation began in the US in the 1960s when a consortium of colleges, universities and academic health centers opened the University City Science Centre in Philadelphia, Pennsylvania to support the establishment and growth of early-stage companies in the research parks facility. During that period, incubators focused on providing low cost space and a small set of shared administrative services such as secretarial services and meeting rooms. Over time, data collected on business incubation performance indicated clear effects in terms of job creation, capital investment and economic wealth to their communities.2 With the emergence of positive results, countries from around the world gradually supported the idea of business incubation as a viable way to stimulate and broaden the horizons of local economies.

The economic assumption behind business incubation is that it will create jobs and therefore have an impact on local communities. In the context of developing countries, it is argued that the employment impact of incubation is likely to contribute to poverty reduction by providing a regular and sustainable level of income to individuals. Researchers have also noted indirect effects from business incubators on the enabling environment through policies related to small and medium business development and the availability of finance.

1 infoDev’s website: 2 ITWeb website: http://www.itweb.co.za/sections/business/


March 2007


74

2 . 2 . 2 i n f o D e v ’ s R a t i o n a l e f o r t h e I n i t i a t i v e

During infoDev’s early years, a technical advisory panel (TAP) composed of IT specialists provided regular guidance to its overall work and suggested emerging subject matters in the sphere of ICT for development that could be of interest to the organization. Around 1999, the TAP began a discussion related to the use of ICTs for capacity building purposes with Small and Medium Enterprises (SMEs). infoDev judged working with intermediary organizations such as a business incubator’s various stakeholders as a good strategy to reach out to a larger number of SMEs.

2 . 3 D e s i g n

Following Japan’s declared commitment to support ICT4D, infoDev saw the country as a potential partner and thereby approached the Government of Japan to propose a financial partnership that would allow the development of an incubator initiative.

The design of the initiative began with the drafting of a concept note that infoDev submitted to Japan’s Ministry of Finance, which had expressed an interest in business incubation. This note was the object of an on-going dialogue between the two parties, lasting several months, in order to come to an agreement on the nature and the specifics of the grant-making scheme that would support business incubators. While the concept note initially focused on the establishment of a network of incubators for small and medium ICT enterprises, its emphasis was gradually shifted to the various institutional mechanisms through which ICTs can support business development. The focus of the initiative they agreed upon explains why the grant-making scheme was open not only to ICT incubators but to a diverse array of organizations including science and cyber parks, national associations, foundations and not-for-profit organizations, universities and research institutions that could play an intermediary role.

As mentioned in the infoDev Directory of Business Incubators, the initiative was designed to foster entrepreneurship and private sector development in developing countries through the establishment of a network of incubators. The initiative would encourage the effective use of ICTs in order to increase the operations and improve overall performance of SMEs supported by the incubators.

More specifically, the incubator initiative was designed to provide support through the following means:

• Provide the necessary financial and technical assistance to help incubators scale up their operations;

• Facilitate the development of new business incubators in areas not currently served;

• Encourage innovative uses of ICTs and entrepreneurship based on local needs; and

• Establishing a network of knowledge sharing among business incubators.


March 2007


75

It was agreed that the initiative would unfold in various phases, which would allow, firstly, the consolidation of knowledge and best practices on the business incubation and then move to the actual grant-making disbursement to various projects. infoDev had planned to provide, depending on the recipient organizations, three types of grants: operational, planning, and capacity building3. The idea was for the actual technical assistance and capacity building to be provided by local or international experts selected by the grantees. infoDev strongly encouraged grant recipients to use their own network and resources to select providers of technical assistance rather than to rely on infoDev to provide this information.

A common understanding of the initiative’s objectives and grant-making specifics allowed both the Government of Japan and infoDev to sign a grant agreement in 2002 for the value of US$17 million over four years. Because of its direct linkages with private sector development, the incubator initiative was situated within infoDev’s entrepreneurship and competitiveness program.

2 . 4 G r a n t P o r t f o l i o

infoDev’s support to incubators varies significantly across geographic regions. The region receiving most of the grants, both in terms of the number grants and amount of financial support, is Asia (22 grants), followed by Latin America and the Caribbean (15 grants), Eastern Europe and Central Asia (14), Africa (13 grants) and the Middle East (8 grants). However, looking at the trend over the years, it is important to note that there have been an increased numbers of grants to support incubator projects in Africa.

Exhibit 2.1 Distribution of grants by region

REGION FIRST ROUND

(2003) SECOND

ROUND (2004) THIRD ROUND

(2006) TOTAL NUMBER

OF GRANTS TOTAL AMOUNT OF GRANT

FUNDING ($US)

Africa 1 7 5 13 2.020.000

Asia (East and South)

6 7 9 22 3.970.000

Eastern Europe and Central Asia

3 6 5 14 2.670.000

Latin America and the Caribbean

6 4 5 15 3.350.000

Middle East 1 2 5 8 1.475.000

Total 17 26 29 72 13.485.000

3 – Operational grants are aimed at consolidating the operational and self-sustaining capacity, efficiency, performance and outreach of business incubators through the building and more effective use of ICT skills and ICT-enabled business incubation services. – Capacity building grants are aimed at providing financial and technical assistance to new or start-up business incubators and related programs for developing and piloting viable incubation models and processes. – Planning grants are aimed at providing financial support for the planning of self-sustaining new business incubators in developing countries


March 2007


76

An analysis of infoDev’s Directory of Incubators also provides information on the distribution of grants by category (operational, planning, and capacity building). The Directory indicates, as illustrated below, that capacity building grants are quite uniformly distributed across geographic regions, except for Eastern Europe and Central Asia. As for the other two kinds of grant, the distribution is more heterogeneous, operational grants being provided in particular to Asia and Latin America, while planning grants have been more focused on Asia and Africa. Eastern Europe and Central Asia have received no planning grant for several years.

As mentioned earlier, infoDev has supported a wide range of recipient organizations, which in turn have developed different types of business incubation projects, depending on their niche and expertise. Exhibit 4.2 provides three examples of incubator projects that well illustrate the diversity of infoDev’s incubator portfolio.

Exhibit 2.2 Examples of infoDev supported projects

EXAMPLES OF infoDev SUPPORTED PROJECTS

Thailand: Planning Grant and Capacity Building Grant to Kenan Institute Asia (KIAsia). Amount of each grant: US $100,000. Years: 2004 and 2006.

This grant is an example of a planning grant given to a foundation, namely the Kenan Institute Asia, with the aim of assisting entrepreneurs to realize their business ideas, especially in rural communities where residents often lack funds and basic business know-how. In order to attain this goal, KIAsia has developed a partnership - KIAsia BI- with another institution, the Office of Vocational Education Commission. The grant has been used to support a wide range of activities led by KIAsia BI, such as attending training courses to build up internal capacity (in particular the “train the trainer” course); developing a network with other organizations in Thailand and abroad; developing a business plan for the incubator; and incorporating ICT use into a business model which targets rural communities. In 2006, the Kenan Institute Asia received an additional grant for capacity building for the support of a program to upgrade the business-incubation services in several Asian countries through tailored interventions.

China: Operational Grant to Tianjin Women Business Incubator (TWBI) and Yangling Business Incubator Centre (YBIC). Grant amount: US $500,000. Year: 2003.

infoDev’s grant to China’s Ministry of Finance supports both TWBI and YBIC in their activities aimed at assisting entrepreneurs. While TWBI has the status of a non-profit mixed incubator with a gender profile (it assists female entrepreneurs), YBIC is a business incubator specialized in agriculture, animal husbandry and biotechnology commercialization.

The grant made possible a wide range of activities, including building up a broad network of alliances; training programs; capacity building and enterprise development; support for clients’ enterprises; dissemination of new technologies; development of a portal, and project evaluation.

Ghana: Capacity Building Grant to BusyInternet Ghana Limited. Grant amount: US $300,000. Year: 2004.

The funded entity is a private Ghanaian company with the mandate of promoting social and economic development in Africa through information and communication technology applications. In particular, BusyInternet Ghana Limited provides resources for businesses and organizations to execute their core competencies.

infoDev’s grant was used to launch an 18-month program providing technical assistance to four companies previously selected; to plan activities with the aim to increasing publicity on the concept of incubating small businesses using ICT; and to start consultations with the Ministry of Communications on the establishment of the government’s technology incubator at the Ghana Multimedia Centre.


March 2007


77

2 . 5 I m p l e m e n t a t i o n a n d M a n a g e m e n t o f t h e I n i t i a t i v e

In the first phase of the initiative, infoDev contracted Anprotec (National Association of Promoters of Innovative Ventures, Brazil) to develop the infoDev Incubator Support Center (IDISC). Meant to serve as a clearinghouse of knowledge and best practices, IDISC enabled practitioners in the field to address operational problems in their environment. For the three years during which it was online (2002-2005), IDISC was both a repository of studies, best practices, toolkits and methodologies and a service provider offering training and online advisory services. According to stakeholders interviewed in the course of the impact assessment of the initiative (see section 1.6), the IDISC was never developed as a fully-fledged online tool as initially planned, which limited its use among several grantees. Stakeholders noted a great need for knowledge support and therefore a certain level of unmet expectations vis-à-vis the IDISC, because of the limited period it was accessible online.

The second phase of the initiative began in 2003, when infoDev requested proposals from organizations that had at least two years of experience in business incubation. At this stage, infoDev selected 17 projects4, to which it disbursed operational grants in support of already-existing incubators in developing countries. The operational grants were designed to consolidate self-sustained capacity, efficiency and performance of business incubators. Before disbursing the grants, and in compliance with World Bank practice, infoDev had to go through vetting processes in order to receive comments and authorization from country desks regarding the grant transfer to recipients in their respective countries. Such a process, which can take up to four months, was also necessary for the subsequent Request for Proposals (RFP) in 2004 and 2006.

At this stage, the initiative was administered by one person within the infoDev team who received support from approximately 10 task managers working either at the Global Information and Communications Technologies Group (GICT) of the World Bank, its country desks, or from within infoDev itself. Task managers were responsible for supervising the implementation of an individual incubator project in one country and ensuring that all planned deliverables were implemented and reported on. infoDev had the overall responsibility of coordinating and supervising the initiative, which, in some cases, meant providing guidance to grant recipients for the submission of various deliverables such as inception reports.

Later, in 20045, the project’s third phase was launched with a second call for proposals; it was open to all organizations, regardless of their depth of experience in business incubation. In addition to operational grants, the third phase introduced planning grants that were designed to lend support to the planning of new and start-up incubators. It had been decided at the design stage of the initiative that the second round of grants would follow quickly after the first one in order for start-up incubators from more needy regions, especially those in Africa, to benefit from the financial support for a substantial period of time.

4 In 2003, infoDev received 80 proposals and funded 17 grants 5 In 2004, infoDev received 155 proposals and funded 26 grants


March 2007


78

As the initiative portfolio increased to 43 projects in 2004, and then 72 projects in 20066 in 50 countries, it became necessary to centralize its management and transfer the supervising tasks from World Bank task managers to infoDev staff. Given the increasing number of projects in each region, from a management perspective it was a sound decision to manage the projects by region, with a dedicated infoDev staff member assigned to the task. Regional coordinators’ key responsibilities were to task manage individual grants and the network of grants in a specific region, which implied supervising and providing guidance for the various deliverables. In addition, coordinators were tasked with creating linkages between the regional projects and leveraging potential synergies. A further responsibility revolved around extracting lessons learned from the regions, as recommended in the 2004 mid-term review. (The knowledge-sharing component of the initiative will be described in greater detail below.)

The key challenge related the management of the initiative was identified as the limited resources attached to it. As stressed by several stakeholders interviewed in the course of the impact evaluation, the human resources assigned to the initiative were insufficient, because of regional coordinators’ multiple competing tasks, which in turn caused delays in disbursing grants or providing feedback on or approving deliverables submitted by grantees. In was also noted that in some cases, the coordinator’s limited experience in business incubation were obstacles to providing the coaching demanded by grantees to push their project forward. As an alternative to providing internal guidance, infoDev hired internationally recognized consultants with strong regional expertise to provide technical assistance and direct mentoring to individual incubators, as and when the need arose.

While the shift to regional coordinators did allow for much greater synergies and learning opportunities between incubators from a region, respondents mentioned that limited time and resources were also an obstacle to documenting lessons learned for each region and to sharing knowledge among regional coordinators throughout the initiative.

2 . 5 . 1 N e t w o r k B u i l d i n g

In parallel to task managing the grant-making process, the infoDev team was actively engaged in harnessing potential flows of information and creating synergies among incubators worldwide. Having to support a wide variety of incubator projects, often complex and very different from one another, the infoDev team saw the necessity to develop regional networks of incubators, which would bring a wealth of knowledge and in time play a coaching role increasingly challenging for infoDev to assume. To achieve this objective, regional coordinators were responsible for planning fora and regional workshops as well as facilitating the emergence and functioning of regional networks.

infoDev’s Global Forum on Business Incubation, held in New Delhi in 2004, was the first occasion for the incubator project’s representatives from all regions to gather in a joint event for mutual support and knowledge sharing. Building on this experience, infoDev supported the development of regional networks by organizing regional workshops in 2005 and 2006, which gathered participants from four regions: Asia, North African and the Middle East, Central Asia and Eastern Europe as well as Latin America and the Caribbean. The purpose of these fora was to share their experiences and lessons learned, build new relationships and consolidate linkages between networks.

6 In 2006, infoDev received 142 proposals and funded 29 grants.


March 2007


79

All regional workshops brought together the private sector, including business incubators and ICT-enabled enterprises, as well as policy makers from the public sector. Such fora provided participants with an opportunity to explore and discuss the challenges faced by entrepreneurs and business incubators, the role of the public sector, potential strategies to strengthen innovation and entrepreneurship as well as capacity development needs of business incubators and ICT-enabled entrepreneurs.

Given infoDev’s limited capacity to provide effective mentoring and coaching to incubators, both in terms of resources available and expertise, it opted for the recruitment of regional facilitators in all regions, except Asia, starting in the summer of 2006. Regional facilitators were selected based on their knowledge of the cultural, political and economic context of the regions as well as on their years of experience in small business/incubation matters. The primary role of facilitators is to assist the regional network grantees take ownership of the network and to foster its operations.

2 . 6 K n o w l e d g e S h a r i n g

A preliminary step in distilling lessons from the initiative was infoDev’s decision to support the conduct of a mid-term review by an independent team in 2004. The purpose of the review was to assess the relevance of the initiative and to provide insight as to how the initiative should be pursued during its remaining life span. Among the many issues explored in the mid-term review are the common constraints faced by incubators, issues to be addressed to improve the implementation of the initiative, the practical issues facing IDISC as well as sustainability prospects for the incubator projects. As part of its key recommendations, the mid-term review underlined the need for infoDev to: 1) strengthen the evaluation and knowledge sharing components of the initiative; 2) develop regional networks of incubators; and 3) provide mentoring and consultation to grantees. As illustrated in this case study, infoDev has addressed all three recommendations.

Though the monitoring of incubators’ performance was not integrated into the initiative from its inception, infoDev saw the necessity of capturing key lessons from the initiative in order to ensure a greater learning process. By 2006, infoDev had fully developed the evaluation component of the initiative comprising an analysis of the Incubator Initiative, 10 Incubator Case Studies, and the implementation of a Monitoring, Evaluation and Impact assessment (MEIA) which would permit learning lessons from the initiatives. The MEIA would, it was hoped, facilitate presenting recommendations for the future.

infoDev contracted a consulting consortium7 to conduct the initiative’s key evaluation: ‘Innovation and Entrepreneurship in Developing countries: Impact Assessment and Lessons learned from infoDev’s Global Network of Business Incubators.’ The process began in April 2006, when the evaluation team first met with infoDev in Washington, and was completed in November 2006, when the team presented key findings and recommendations from the evaluation to a wide variety of stakeholders at the Global Forum on Business Incubators. It is worth mentioning that a video on business incubators visited during the evaluation will also be produced in order to present a more concrete and lively picture of the type of work conducted in the field.

7 The consortium comprised of OTF Group, Louise Walker Consulting, CREEDA projects and UK Business Incubation.


March 2007


80

Several respondents suggested that an earlier start of the monitoring and evaluation process would have been advisable in order to capture the emerging lessons learned as well as to collect baseline data mid-way through the initiative. Even so, the many evaluation products are likely to bring a wealth of knowledge on the various aspects of the initiative, including the incubation environment, financing challenges, the use of ICTs in incubation and their effects on local communities. Such lessons will in turn inform the team’s work in planning and developing a sustainability plan to ensure that the initiative remains strong beyond the life of Japanese support. However, limited resources and competing priorities were significant obstacles to an early implementation of a Monitoring and Evaluation (M&E) system.

2 . 7 E f f e c t s

In April 2006, a consortium of consultants was mandated to conduct the Impact Assessment and Lessons Learned from infoDev’s Global Network of Business Incubators. At the time of writing this case study, the consortium had just completed the report for the impact assessment. As illustrated below, the Incubator Initiative has generated several positive results.

As well summarized and explained in the Impact Assessment, the study’s central insight is that business incubation indeed fosters ICT-enabled innovation and entrepreneurship in developing countries by addressing weaknesses in the business environment for its clients, including ICT access and capacity. It also generates important ‘externalities’ by creating a platform for leaders to influence the broader business environment for high-growth SMEs. (…) infoDev has played a role in business incubation’s effectiveness by being an important part of the overall resource mix helping many of its grantee organizations to succeed. These findings strengthen the Initiative’s central hypothesis that business incubation can be a mechanism for helping entrepreneurs to leverage ICT to foster innovation and entrepreneurship, and ultimately, socioeconomic impact. 8

Before presenting the types of effects and results that business incubators have had, it is important to mention that there are no universal models of business incubation and no ‘one size fits all’ model of successful business incubation. It is the development of targeted services adapted to particular weaknesses in the environment that serves as the common denominator of effective incubators. Other common features of the various incubators reviewed include, among others, the importance of mentoring and networking services, the presence of innovative ways proposed to help clients overcome barriers to SME financing, as well as on-going encouragement of pro-innovation attitudes among clients, such as high interpersonal trust and tolerance for risk.

The impact assessment reports a wide range of outcomes and impacts, which are context dependent. It is clearly indicated that, with varying degrees, grantees have been successful in:

• Launching businesses – More than 75% of grantees report that their clients complete significant milestones, including graduating as a self sustaining business or selling product or service

• Creating jobs – A majority of grantees indicated that their clients had created jobs in their own businesses, the numbers depending on the context

• Taxes – Several grantees noted that their clients are contributing to the economy through the payment various regular taxes

8 Innovation and Entrepreneurship in Developing Countries: Impact Assessment and Lessons Learned from infoDev’s Global Network of Business Incubators, Final Report, p.4.


March 2007


81

• Social impact – It was reported that some clients of incubators generated tangible social impact, including job creation among marginalized and low income group

Despite significant challenges, the impact assessment also indicates that business incubation helps to reduce barriers to innovation and entrepreneurship in the broader business environment. Many grantees are able to influence the larger business environment for technology entrepreneurs from various angles such as increasing access to SME financing and improving macro environment for SME financing and advocating for policy and legal reforms that will benefit entrepreneurs. The study also shows that incubators have had a significant impact on their environment by influencing the culture of innovation and entrepreneurship beyond their client base, by having a role model impact that inspires the creation of similar organizations and by aligning the participation of various stakeholders from the private, academic and public sector.

Another positive effect that business incubation has had is to develop leaders in innovation and entrepreneurship and creating a platform for their impact. Indeed, the impact assessment notes that the success of incubators is often linked directly to skills, vision and commitment of leaders and leadership teams.

Finally, another important finding from the impact assessment, and one of great interest for this case study, is that donors, and more specifically infoDev can play a catalytic role in improving the efficiency and effectiveness of business incubators. As a contributor in the overall mix of resources available to incubators, infoDev helped grantees and their clients develop their capacity to leverage ICT mostly by providing funds for infrastructure such as telecommunications, Internet and LAN as well as training to build skills. infoDev involvement was also key in enabling knowledge sharing through regional or global workshops, in building networks, which was judged extremely valuable by grantees, and in strengthening the credibility of many grantees as well.

Bust most importantly, infoDev has been filling a gap in service provision by providing funds for services such as business pre-planning, planning, and networking. According to the impact assessment, ‘Almost 85% of grantees agreed with the statement that without infoDev’s support, they could not have undertaken their proposed activities, and almost a third indicate that there are no organizations that are similar to the grantee organization in their local business environment. (…) This data suggests that infoDev is helping grantees to fill a gap in service provision in grantee business environments, an observation confirmed frequently in interviews.’

The above findings on the outcomes of the Incubator Initiative are fully documented in the report developed by the consultants’ consortium, entitled ‘Innovation and Entrepreneurship in Developing countries: Impact Assessment and Lessons learned from infoDev’s Global Network of Business Incubators.’

2 . 8 R e l e v a n c e t o i n f o D e v ’ s K e y S t a k e h o l d e r s

Donors interviewed admitted to a certain level of skepticism when infoDev first proposed the incubator initiative. Some feared that it would make too great a demand on their human resources, which would in turn be detrimental to infoDev’s other programs. Others questioned the extent to which the initiative was relevant to infoDev’s overall mandate as a knowledge organization.

Nonetheless, most donors and technical partners interviewed noted that the initiative per se was an interesting one and relevant in terms of SME development. Among the most interested agencies, some have taken steps to inform partner agencies and organizations in their country on the implementation of the initiative in order to create new linkages with incubator networks. For example, in Germany, the Ministry of Technical cooperation (GTZ) facilitated communications between one of its partner organizations experienced with business incubator training facilities,


March 2007


82

InWent Capacity Building International, and infoDev. InWent had the opportunity to collaborate with one of the regional networks and is currently planning to publicize its experience at the upcoming Global Forum in Hyderabad. In Canada, the Canadian International Development Agency (CIDA) involved the International Development Research Center (IDRC) and Industry Canada in discussions on business incubation with infoDev.

While not all donors considered the initiative as fitting within their agency’s priorities, some viewed the business incubation as an innovative way of supporting private sector development and thus decided to provide financial support to specific grantee incubators. Though there is no specific mechanism in place to track the extent to which additional funds have been leveraged for the incubators, respondents did share some anecdotal evidence. For example, Finland’s Ministry of Foreign Affairs provided infoDev with earmarked funds to support the development of incubators in Asia. Given Finland’s interest in SME development, it viewed participation in the initiative as an effective way to gain knowledge of concrete experiences on the ground. Another European agency, the European Commission, decided to support three new incubators in the Caribbean. In India, the government disbursed a grant to match the infoDev’s financial contribution. In the case of Vietnam, both USAID and Microsoft provided additional funding, while the World Bank provide approximately US$5 million to an existing incubator project in Georgia.

Technical partners interviewed also had a positive response to the incubator initiative. For example, a respondent from the Centre for the Development of Enterprises, funded by the European Commission, noted that that given the challenges facing SME development (especially ICT-enabled ones) in Sub-Saharan Africa, infoDev’s support for start-up incubators was most relevant. Likewise, representatives from India’s National Science and Technical Entrepreneurship Development Board of India noted that infoDev’s sustained focus on sharing best practices and knowledge on business incubation was beneficial to developing countries, where the environment is not always friendly to emerging small and medium businesses.

The main contributor to the initiative, Japan, will not provide additional funding to the initiative, but remains committed to using the lessons learned from the incubator initiative to encourage the use of ICTs for development among various stakeholders. Representatives from the Japanese government are planning to showcase the incubator model as a way to demonstrate how relatively small grants can have a multiplier effect in a local economy. Already, the Government of Japan has approached the Bank of Tokyo to ask for their help in creating linkages between existing incubators and potential financial supporters from the private sector in the country.

2 . 9 M a i n t a i n i n g t h e M o m e n t u m

While infoDev staff expressed confidence in the types of results that the initiative would yield in the short term, many underlined the need to make on-going efforts in order to ensure the project’s sustainability. A first step in ensuring that the initiative’s momentum is maintained is infoDev’s decision to temporarily host the IDSIC support center online, in order to respond to the on-going demand from incubators. A the time of writing, infoDev was holding discussions with various incubator partner organizations regarding the possibility of shifting the management of the site to partners in recipient countries.

infoDev staff involved in the initiative raised a number of points that they considered to be critical to the project’s sustainability beyond the Japanese grant, which will come to an end in 2007. Offering on-going support to the business incubators’ regional networks was underlined as key to maintain their dynamism. To do so, its staff suggested that infoDev become an advocate of regional networks to eventually leverage financial support from donors. The extent to which donors would be interested in doing so is not, however, clear at this point.


March 2007


83

The integration of lessons learned from the incubator initiative into new infoDev products or studies was also stressed as a key factor in formulating sustainable new programmes. Staff agreed that it is vital to ensure that knowledge gained through a specific project serves infoDev’s work from a long-term perspective. Some donors supported the view that infoDev should position itself as a center of excellence on business incubation for ICT-enabled services. The infoDev team is already building on its knowledge of incubators and feeding it into various knowledge products, including a study on financial gaps facing ICT-enabled businesses as well as a document compiling best practices for the development of IT parks as well as the SIDS Toolkit.

With respect to the notion of 'Scalability' that infoDev wishes to foster, lessons learned from the incubator initiative will present new incubation models that entrepreneurs can adapt to their local context, with the objective of fostering innovation and employment.

The sustainability issues highlighted here are clearly echoed in the impact assessment report, which recommends that donors, policymakers and infoDev pay attention on:

• Greater focus on knowledge generation and diffusion around the drivers of success in business incubation, entrepreneurship and innovation in developing countries

• Greater focus on ‘incubating incubators’, customized support for individual organizations and the leaders working to launch and scale them

• Linking business incubation more explicitly to a broader portfolio of investments in strengthening the business environment for ICT-enabled entrepreneurs

2 . 1 0 C o n c l u s i o n s a n d A r e a s f o r I m p r o v e m e n t

Data collected indicates that infoDev has been successful in task managing the Incubation initiative in a manner that ensures the optimal use of grants provided to various organizations. When requested, infoDev was effective in identifying external resources that were able, on the one hand, to provide technical assistance to specific grant recipients, and on the other hand, foster the development of regional networks. Data emerging from the impact assessment conducted by a consultants’ consoritum in turn shows that infoDev’s support contributed to the leveraged use of ICTs, greater increased businesses and employment, and more exchanges of ideas and information among incubators. infoDev’s dissemination of information on the initiative also helped generate an interest among specific donors and additional financial support.

The key area for improvement identified in this case study is related to the ‘late’ integration of monitoring and evaluation into the initiative as well as the limited opportunities that internal coordinators have had to share experience and knowledge.

Respondents: KEY INFORMANTS

Vivek Chaudhry, Task Team leader, infoDev

Ellen Olafsen, regional coordinators, infoDev

Samia Melhem, regional coordinator, infoDev

Kerry McNamara, regional coordinator, infoDev

Seth Ayers, regional coordinator, infoDev

Joan Hubbard, regional coordinator, infoDev

Anne Morris, consultant’s consortium


March 2007


84

KEY INFORMANTS

Louise Walker, consultant’s consortium

Romeo Bertolini & Monika Muylkens, Federal Ministry for Economic Cooperation, GTZ, Germany

Jyrki Pulkinen, Finish Cooperation

Aliou Abdoullahi, Centre for the development of Enterprises

Jérome Adam, Agence Française de Développement

Richard Fuchs, International Development Research Center

Bengt Oberger, SIDA

Marianne Medori, Ministry of Foreign Affairs

HK Mittal, National Science and Technical Entrepreneurship Development Board of India


March 2007


85

3 . C a s e S t u d y o n I C T R e g u l a t i o n To o l k i t


This case study offers an overview of the evolution of the ICT Regulation Toolkit Initiative, the factors motivating its development and implementation, and the challenges associated with its implementation and sustainability.

The methodology used to complete this case study includes a review of the key documents, the request for Expression of interest used to select the consultants, and an examination of the modules available online at the time of writing (i.e. Module 3: Authorization of Telecommunications Services and Module 6: Legal and Institutional Framework). Face-to-face and telephone interviews also were conducted with the key stakeholders involved in the development of this product.


3 . 2 . 1 C o n t e x t

In November 2000, infoDev published, in cooperation with the International Telecommunication Union (ITU), the Telecommunications Regulation Handbook. The objective behind infoDev’s support for this project was twofold. First, infoDev sought to foster the development of regulatory environments that would accelerate connectivity and access to information services in emerging and industrialised economies in the light of the globalisation of the telecommunication sector and growing shift toward the privatisation and liberalisation of domestic telecommunication markets. Second, it wanted to provide regulators, and other stakeholders in the regulatory process, with an easy-to-use reference source that dealt with the legal and regulatory methods used to implement market-based reform initiatives and which provided practical examples of the practices used by regulators throughout the world.

The Telecommunications Regulation Handbook is divided into six modules along with a host of relevant appendixes. The six modules are:

1) Overview of telecommunications regulation

2) Licensing telecommunications services

3) Interconnection

4) Price regulation

5) Competition policy

6) Universal service

This product proved to be one of infoDev’s most successful pursuits, and has become a mainstay of telecommunication regulation training. This is in no small measure due to the fact that this product was made freely available online and, subsequently, was translated in six languages – Arabic, Chinese, English, French, Spanish, and Russian.


March 2007


86

3 . 2 . 2 R a t i o n a l e

The online ICT Regulation Toolkit is a second generation product that is poised to potentially reinforce infoDev’s good standing in the telecommunication regulation domain. As with its predecessor, a key objective of this initiative is to provide regulators, and other stakeholders in the regulatory process, with an easy-to-use reference source dealing with the legal and regulatory methods used to implement market-based telecommunication reform initiatives and which provides practical examples of the practices used by regulators throughout the world.

By 2003, it was apparent that the Handbook would need to be updated and expanded to account for the rapid pace of technological innovation if it was to remain relevant for its target audience. This recognition also coincided with the appointment of Mostafa Terrab as the new Program Manager for infoDev. Before joining infoDev Mr. Terrab had been the head of Morocco’s telecommunication watchdog, Agence Nationale de Réglementation des Télécommunications (ANRT). As such, he had keen insight into the limited information that regulators in developing countries had at their disposal, but which they required to make effective regulatory decisions.

Mr. Terrab is credited for having come up with the idea of creating an updated, expanded version of the Handbook that would be based on a web-based interface and which would function as a ‘living’ resource. He also took the lead on approaching the International Telecommunication Union (ITU) Telecommunication Development Bureau (TDB) to seek their participation in this initiative.9 Partnering with the ITU-TDB was essential to granting credibility to this initiative. Under the terms of the partnership arrangement, the ITU-TDB contributed minimally to the funding to the initiative, but would provide documentation and play a role in reviewing the content developed for this initiative. It also was to play a role in informing the broader telecommunication community that this project and in marketing the final product.

3 . 3 D e s i g n

3 . 3 . 1 C o n t e n t

The next step in the development of this initiative consisted of selecting the areas that would be covered in the online Toolkit. In the early planning stages, the members of infoDev’s and the GICT staff who were involved in this initiative took their lead from the Handbook and asked ‘what more do we need?” The actual process of selecting modules for inclusion involved a number of iterative stages in which lists of modules were proposed, discussed and revised. A complicating factor at this time was the realization that proceeding in accordance with the structure of the Handbook would have meant having separate modules for a number of inter-related issues.

In the end, it was agreed that seven modules would be authored during 2005-2006. The modules selected were:

1) Overview on Regulation

2) Competition and Price Regulation

3) Authorization of Telecommunication Services

4) Universal Service

9 By the time the relationship between infoDev and ITU-TDB formalized work on the first module to be

developed already had commenced.


March 2007


87

5) Radio Spectrum Management

6) Legal and Institutional Framework

7) New Technologies and Impact on Regulation

infoDev subsequently presented an action plan for this initiatives to the donors who then approved the online ICT Regulation Toolkit under the Access for All theme.

3 . 3 . 2 D e s i g n o f M o d u l e s

The distinguishing feature of the online Toolkit is that is it is not designed as a book, but rather as an interactive product. To this end, the online modules are intended to be – or least were originally envisaged as being – an online ‘living’ resources that would attract feedback and be regularly updated. In the words of an individual closely associated with the development of the online Toolkit, “The modules have not been designed as a piece of a book but as an interactive product”

The objective was to create a series of modules, each of which would offer a high level summary of the topic at hand and which would provide access to addition online content through hyperlinks. This was accomplished by adopting a layered structure for each module. The first layer consist s of a summary discussion of the particular topic. The second contains detailed discussions of specific sub-topics falling under the umbrella of the main topic. In the third layer practice notes as well as commentary on, and summaries of, on reference documents are provided. The fourth layer consists of hyperlinks to specific reference documents.

Exhibit 3.1 Contractees for the ICT Regulation Toolkit Modules

MODULE CONTRACTEE LOCATION

1. Regulating the telecommunications sector – overview

Not Assigned

2. Competition and price regulation

NERA Economic Consulting Boston, MA, United States

3. Authorization an telecommunication services

McCarthy Tétrault LLP Toronto, ON, Canada

4. Universal service Not Assigned – expected by January 2008

5. Radio Spectrum Management McLean Foster & Company Almont, ON, Canada

6. Legal and institutional framework

Telecommunications Management Group, Inc. (TMG)

Arlington, VA, United States

7. New technologies and impact on regulation

Centre for ICT (CICT), Technical University of Denmark

Lyngby, Denmark

In terms of designing the online Toolkit, one of the key issues that needed to be dealt with in the early stages of this initiative was the way in which it would be contracted. The original Handbook had been contracted with the Canadian-based law firm of McCarthy Tétrault. Among other things, sourcing the Handbook to a single contractor had helped to ensure that the final product was coherent and rooted in one perspective. However, the size of budgetary allocations for the individual modules (approximately US$195,000) precluded the possibility of contracting the online


March 2007


88

Toolkit project to one entity given the procurement regulations of the World Bank.10 As a result, a strategy of competitive sourcing has been employed for each of the modules developed to date.11 The organizations contracted to develop the modules are all based in North America and Europe (see Table 1)

The organizations who were awarded contracts for developing the individual modules were required to deliver three products: (i) a concept paper examining key and emerging issues with regard to the respective module; (ii) a web-based electronic toolkit based on the concept paper and which provided a ‘non-linear’ learning experience; and (iii) training materials in the form of a PowerPoint presentation providing a summarized perspective of the issues covered in the module.

The first module to be completed and by posted online was module 3 – Authorization of Telecommunications Services. It went live in August 2005 and was showcased at the ITU Global Symposium of Regulators (GSR) and at the WSIS in the autumn of 2005. A second module, module 6 – Legal and institutional framework, went live in July 2006. With the exception of the module 4- Universal service, are expected to be online by the end of 2006.

3 . 4 I m p l e m e n t a t i o n a n d M a n a g e m e n t o f t h e I n i t i a t i v e

There are a number of challenges with which infoDev and the contractees have had to contend during the life cycle of this initiative. The key issues centre around coordinating the complex nature of this undertaking. In terms of the program’s management a number of the interviewees commented on the lack of consistent leadership and coordination. For example, in a span of six months within 2006 the initiative has had three different program managers. To this end, all of the interviewees alluded to a pervasive sense of confusion with regard to had overall leadership for the initiative and a desire to have seen a more ‘hands-on’ form of leadership.

The competitive sourcing process also contributed to perceptions of a lack of coordination. Two factors were at play in this regard. First, this process meant that different organizations were contracted to produce the individual modules. While the process of competitive sourcing has the advantage of providing a diversity of views, it gives rise to the potential for a lack of coherence (i.e. conflicting perspectives being present)12 and risks fostering inconsistencies in the presentation and content across modules. Indeed, all of the people interviewed for this case study asserted that, although all the modules produced to date are of good quality, clear discrepancies exist in terms of content and perspective. These inconsistencies appear to have been reinforced by the fact that there was little-to-no collaboration between the task managers for the respective modules. In essence, the toolkit initiative appears to have been managed as seven different projects.

At a technical level two additional challenges manifest themselves. Both are linked to the web-based nature of the modules. At one level, there appears to have been communication problems between the individual contractees and the organization responsible for providing the web platform tools for the online modules and for hosting the development site for the online Toolkit. This served to create delays in the preparation of the modules. At another level, the review of the online modules was complicated by the fact that hard copies of the content have not been provided to the

10 Had the procurement for the online Toolkit proceeded on the basis of a single contract, the total budget

would have been in the range of US$ 1.4 million. 11 At the time of writing the Universal service module has not yet been sourced. It is expected that it will be

completed by January 2008. 12 As one interviewee put it, “A university in Denmark doesn’t think the same as advocacy lawyers in D.C.”


March 2007


89

reviewers for all of the modules developed to date, and the authors of the different modules have used the web-platform tools in different ways. This, combined with the sheer quantity of material included in some of the modules has made the review process a considerable task for task managers and for the the external reviewers.13

Exhibit 3.2 Expected Deliverables

ORIGINAL DELIVERABLES MODIFIED DELIVERABLES

1. Workshop/Presentation – First draft of the study and blue-print of online-module as well as proposed, additional content material for online module.

1. Workshop/Presentation – First draft of the study and blue-print of online-module as well as proposed, additional content material for online module.

2. Workshop/Presentation – submission of final draft of study

2. Workshop/Presentation – submission of final draft of study

3. Final version of study and the online module as well as presentation of draft training/presentation material

3. Final version of study and the online module as well as presentation of draft training/presentation material

4. Presentation of final study, online module, and training material

4. Presentation of final study, online module, and training material

5.

• Development of two days worth of training material (in addition to one day worth of training material initially specified);

• Development of 50 summaries of the studies that serve as the basis for the online module

• Two additional capacity building/training courses held and prepared by the contractee.

One of the issues that the contractees have had to contend with is the changing requirements in the development of the modules. In January 2006 a retreat was convened by infoDev. The participants included representatives from infoDev, the ITU-TDB, the GICT, contractees for modules 2, 5, 6, and 7. At the retreat the contracted organizations presented their respective concept papers, a preliminary outline of the web interfaces they were developing and their action plans. A key outcome of this meeting, and feedback from the previous showcasing events in Tunis, was recognition that, as one interviewee put it, “some things were missing.” This led to amendments being made to the contracts requiring an additional deliverable and the project deadline for the four modules being extended to December 2006 (See Table 2).14

The nature of the relationship between infoDev and the ITU also has presented certain challenges. On the one hand, the ITU’s involvement in this initiative was critical in terms of it garnering credibility, gaining access to the world’s telecommunication regulators, and for marketing the final product. On the other hand, the ITU has not contributed, in any substantive sense, to funding the online Toolkit initiative. Given that the ITU and infoDev are meant to be equal partners in the development of this undertaking, the question that arises is whether infoDev will be able to effectively leverage its brand and avoid losing itself in this relationship.

13 For example, some of the modules are in the range of 200-300 pages in length, whereas others are much

shorter. 14 In the light of this amendment, the contractees’ remuneration was increased by approximately US$20,000.


March 2007


90

3 . 5 R e l e v a n c e t o s t a k e h o l d e r s

At the time of writing the online Toolkit initiative is on-going. Therefore, it is too early to comment on its potential relevance to telecommunication regulators and other stakeholders in the regulatory process. However, on the basis of the interviews conducted with those involved in this project, there appears to be a broad consensus on the factors that, ultimately, will need to be resolved if this project is to enjoy a similar level of success as the Handbook.

The first is ensuring the relevance of the material contained in the modules vis-à-vis the rapid changes taking place in the domains covered. At issue here is the question of who and/or how the modules will be ‘fed’ or updated. Once the contractees deliver the online module, they assume no responsibility for maintaining the modules. Although limited discussions regarding the creation of a mechanism for coordinating and updating the modules have reportedly taken place, no decisions about this matter have been made.

A second, and related, issue is focuses on ensuring greater coherence and consistency across the modules. Resolving likely, would entail hiring a common editor across each of the modules who also could play a role in keeping the contents dynamic and up to date. While this matter also is apparently under consideration no decision has been made.

The third issue relates to accessibility in terms of the language(s) in which the modules are based. A key factor underpinning the effectiveness and widespread dissemination of the Handbook was the fact that it was translated into six languages. Whether or not the modules will be available in several languages and, if so, who will assume responsibility for undertaking this task has yet to be addressed.

A final area of area of uncertainty centers on the receptivity of the intended audience to an entirely web-based platform. Some of the interviewees expressed reservations about the extent to which regulators in developing countries will be willing, or whether they have the means, to access toolkits that are exclusively web-based. At issue here is the lack of a paper-based equivalent for the modules. Although a 50 page executive summary is to be made available for each module, these summaries are not the functional equivalents of book chapters. This has served to raise questions about the extent to which this project is sufficiently aligned with the interests of its intended target audience.

The common denominator across each of the above issues is the resources that are available. In the light of existing resource constraints the focus has been on first getting the product online and, subsequently, tackling the issue of maintenance.

If we don’t feed the toolkits and keep them ‘living’, this initiative won’t deliver!

It would be nice if it functioned like Wikipedia

We give you several tools to get an end product. Not one recipe

There is a risk that once the toolkits are online they could be lost – they are being set up for easy use and access but if it just stops there, they won’t have a shelf-life like the Regulation Handbook


March 2007


91

4 . C a s e S t u d y o n W o r k i n t h e E d u c a t i o n S e c t o r


This case study presents an account of infoDev’s work in the education sector, its relevance and sustainability. The study is based upon: (1) a review of key infoDev documents, products and/or publications; and (2) interviews with key stakeholders who know of infoDev’s work in the education sector. Interviews were conducted over the telephone, and used the Monitoring and Evaluation Handbook as the entry point for talking with people about infoDev. Doing so helped focus the discussion on specific infoDev education products15.

4 . 2 I C T s a n d E d u c a t i o n

4 . 2 . 1 i n f o D e v ’ s R a t i o n a l e

There are currently 867 million illiterate adults in the world today, 64 per cent of them women. Underlying these statistics is the reality that the quality of education in developing countries is inadequate. Moreover, forecasts by UNESCO suggest that an additional 15 to 35 million educated and trained teachers will be needed over the next decade if all countries are to achieve the UN’s Millennium Development Goal (MDG) of universal primary education by 2015.16

15 It should be noted that, for the purpose of this review, the M&E Handbook was selected as an example of infoDev’s work in the education sector in order to focus the discussion around infoDev’s products and services, how stakeholders use such products and services, and how relevant these product/services are to donors, policymakers and practitioners working in the education sector. The reviewers do recognize, however, that infoDev’s work in the education sector does not rely solely on this one deliverable but is one element in a larger set of resources. 16 UN Chronicle, Online Edition, accessed 6 Oct., 2006.

Some advantages of ICTs in education include:

Enables teaching-learning materials to be presented in multiple media for multi-channel learning, fosters enquiry, exploration and the acquisition of different concepts through different paths of learning;

Engages and motivates students in the learning process by employing complementary use of images, sounds, movements, animations and simulations;

Provides opportunities to students to build competencies at their own pace and allows students to utilize their newly acquired knowledge to solve problems and formulate or resolve new ones;

Provides worldwide access to knowledge resources – the most cost-effective means of bringing the world to the classroom. Via the internet, ICTs offer a platform for students to communicate with their peers & colleagues without any geographical constraints.

One World South Asia, March 16, 2005

ICT in education initiatives contribute to MDGs by:

Increasing access through distance learning

Enabling a knowledge network for students

Training teachers

Broadening the availability of quality education materials

Enhancing the efficiency and effectiveness of educational administration and policy

ICT and Education, World Bank


March 2007


92

Information and communication technologies (ICTs) have been integrated and used in education curricula in OECD countries, with an emphasis on using computers in classrooms to provide the skills and training needed for a knowledge/information-based economy. ICTs have also been instrumental in bringing both distance education and e-learning into prominence, where e-universities were developed to help educate more people with fewer teachers and classrooms. The same model has been applied in developing countries, where resources such as teachers, schools and textbooks are scarce and costly, and as a result, there has been an increase in demand for ICTs in developing countries. But these actions and activities are, for the most part, based on the assumption that ICTs in education have a positive cost-effective impact. Yet few people have tested this assumption.

4 . 2 . 2 I d e n t i f y i n g N e e d s f o r i n f o D e v ’ s W o r k

Over the past three years, infoDev has made the transition from being a grant-making institution to an organization that sponsors applied research and analysis projects to identify best practices in the use of information and communication technologies for development. Initially, the idea was to develop a strategy for infoDev’s work in a “mainstreaming” sector that could then be adapted and adopted for use in other sectors. For a variety of reasons, including a specific grant from the European Commission to carry out work in the education sector as well as a request from the World Bank’s Education Sector Anchor Unit, infoDev decided to first explore the idea of working in the education sector by researching issues related to ICT use in education initiatives.

The first activity related to this strategy was a “knowledge mapping” exercise that mapped out what is known and what is not known when it comes to ICT use in education in developing countries. Conducting a “knowledge mapping” exercise is infoDev’s standard practice when initiating a project in a new sector. The idea is to “take stock…about ICT use in a particular sector, both to provide a useful, easy-to-digest synthetic distillation of the ‘most important stuff’ that has been learned in the field over the previous decade, and to help assess where infoDev’s subsequent activities related to this sector may be most useful and strategic”. This work was co-sponsored by infoDev and the Education Sector at the World Bank. As a basis for this study, infoDev looked at ICT components in World Bank education projects.


March 2007


93

Exhibit 4.1 Mapping out Mainstreaming ICTs: M&E Handbook as an example

Knowledge Mapping Exercise:identifying key issues for

investigation

Identification of Work: prioritization of issues

Development of Knowledge Products as a tool for

development and advocacy

Dissemination of Knowledge Products

Creation of Partnerships/Collaboration that may provide new opportunities

Operationalization of research: NEPAD Project as an example of how to

use/invest in M&E for ICT/education projects

M&E in ICT/Education

M&E Handbook

WSISe-Learning Africa

World Bank HD Week

UNESCO Toolkit

Finding 1: Mapping out the use of ICTs in education initiatives provided a clear set of information gaps from which a work program for the education sector could be identified.

The knowledge mapping exercise was used to identify priority areas for infoDev’s activities in the education sector. The first priority, based on the results of the initial exercise and feedback from donors and other partners, was to commission a study looking at the monitoring and evaluation of ICT/education initiatives. It was felt that this would help policymakers and donors alike to consider whether investing in ICT/education initiatives, and investing in M&E of these initiatives, might be advisable and if so, how it might best be accomplished.

“The mapping exercise – this was the seed, the foundation of infoDev’s work. It started to get at the question, the issue of how to deal with the big debate: do they invest money in ICTs for education? Where? How? This work would start to get at the question of how to invest, and where to invest”.


March 2007


94

Finding 2: To some respondents, it was not clear who or which organizations other than infoDev and the World Bank participated in selecting the priority areas to be developed.

A few respondents felt that it was not clear who, or which organization, was making the decisions in terms of infoDev’s priority areas. It was felt that the process used to identify key issues and questions for further research and analysis was not necessarily transparent to themselves or their organization. For some, this transparency issue stems, in part, from infoDev’s relationship to the Bank. Respondents felt that the extent to which the agenda is driven by on-the –ground exigencies or by the needs of the World Bank was not clear.

Finding 3: Most respondents viewed the monitoring and evaluation of ICT use in education as an important issue that infoDev is addressing in its work.

Overall, the stakeholders that we spoke with supported infoDev’s choice to prioritize M&E in the use of ICTs in education. In our interviews, stakeholders mentioned the importance of having data on the impact of ICT use in education projects/initiatives. Within the development community there is a larger debate concerning the use of ICTs in education – whether developing countries with limited resources and budgets should be investing in ICTs for education, and if so, where and how. Respondents stated that by engaging in the knowledge mapping exercise, and by focusing on M&E, infoDev was beginning to address the issue of ICT impact in education, and gathering valuable data to this end.

4 . 2 . 3 R e l e v a n c e t o K e y S t a k e h o l d e r s

Finding 4: The information and products that infoDev provides are considered to be unique by many in the ICT/education field. In this regard, infoDev has created a niche for itself and has a role to play in the education sector.

The transition from a grant making organization to a research one has provided infoDev the opportunity to find its niche in the area of ICTs and education. In doing so, it has created a unique role for itself in the field by providing information and data at a global or strategic level. Many respondents felt that this is a valuable role that other organizations are unable to fill for a variety of reasons, including more broadly defined mandates, lack of resources and so forth. As such, other organizations look to infoDev to provide them with the kind of specialized information that they need for their own work with their partners and member states.

“I find infoDev to be a great resource. But there are challenges about the way they work – their primary client is the Bank and in the end, they have to answer to the Bank. But the Bank’s interest is not always the primary interest to [our organization]. This posits the question: if they do actionable research – actionable to whom?”

“Although I’m aware of a more consultative approach for the survey study, my impression is that, as far as the M&E Handbook goes, there was no front-end marketing to find out the need or interest”.

“Unique because the information they provide is more at a strategic or global perspective.”

“The Handbook is absolutely necessary – countries need the justification to spend the money, especially developing countries. They need to justify the huge expenditures and they can only do that through systematic reporting.”

“We do not have the budget for looking at areas of specialization. We look to infoDev to provide that kind of specialized information.”


March 2007


95

Moreover, many of the interviewees felt that infoDev is not only providing this important and valuable information but that they are committed to issues of ICT4D in education and that they have a “sincere interest in bettering the education in developing countries”. These respondents felt that because of this sincere interest, infoDev garnered a lot of respect by others working in the education sector. This respect added to infoDev’s credibility and thus helped staff to establish partnerships for collaborations with donors, and other bilateral and multilateral agencies.

4 . 3 I m p l e m e n t a t i o n

During the mapping exercise, the monitoring and evaluation of ICT use in education projects was identified as a priority research issue. A concept note was produced by infoDev’s education staff, which was then supervised and approved by the Task Team Leader for the Mainstreaming ICTs work area. The note was then evaluated and approved by other senior infoDev staff. Once the concept note was approved, Terms of Reference (TORs) were developed for the Expression of Interest (EOI) submitted by infoDev to procure external consultants to carry out the study. infoDev staff worked with and consulted their colleagues at the Education Advisory Service to develop the ToRs and EOI. The main objective of this study, as outlined by the EOI was:

to develop a Handbook for policymakers and donor staff working in the education sector in developing countries to recommend best practices for conducting monitoring and evaluation studies of the impact on student achievement of educational initiatives that utilize ICTs.17

Upon submitting the draft EOI, it then “has to be cleared by World Bank Procurement…and World Bank Legal”. Many respondents mentioned the challenges of the World Bank procurement system; one interviewee said “it took a year for me to get a grant for a project worth $100,000. Procurement and Legal wanted to get the grant right from the beginning, there was so much back and forth, how many days for this and so forth, and it was such a small amount of money. It just wasn’t worth it for me”.

infoDev staff stated that once approved, the implementation and management of the study has been challenging for them since much of their time is spent on administration activities that are necessary to keep the project moving forward. The M&E activity was implemented as a collaborative effort among a number of well-known people in the ICT/education field, organized under the auspices of LEARN International. The Handbook was produced in draft format for distribution at the World Summit on Information Society (WSIS) held in Tunis, Tunisia, in November 2005.

The Task Manager, under the supervision of the Task Team Leader, manages the studies in the education sector. Generally, infoDev manages by deliverables. In the case of the M&E Handbook, the Task Manager received a pre-draft version and provided extensive comments to the consultants.

17 Request for Expression of Interest: How Should We Measure the Impact of ICT Use in Education?, March 2005.


March 2007


96

Finding 5: infoDev staff indicate that internal management procedures are very time consuming.

infoDev contracted external consultants to carry out studies and develop other products, which required creating a targeted internal grant. infoDev staff sees this as very time-consuming, as it requires them to “outsource to external consultants for product development. I spend much of my time doing administrative tasks – for example, administrative tasks related to internal financial management and disbursement processes”.

Finding 6: The procedures and systems that infoDev uses for grants delay the disbursement process.

According to at least one infoDev staff member, “the disbursement process is rather painful.” Theoretically, once a deliverable is received and approved, a payment can be made in four days. The problem for grantees is that it is not clear to them what they need to do to get paid. As one staff member stated, “there is typically a delay…as the processes for getting money from infoDev are not clearly spelled out in the grant agreement. For some reason, we have been told that we are not to circulate to grantees formal standardized instructions outlining what they need to do to get paid”. As such, the internal system for disbursing funds happens in a non-systematic, ad hoc way.

4 . 4 K n o w l e d g e P r o d u c t s

Finding 7: Key stakeholders who have used the Monitoring and Evaluation Handbook find it to be a useful product even though it is in draft form.

Most respondents were aware of the M&E Handbook, as well as infoDev’s other products. Those who have used the Handbook find it to be a highly useful tool in their own work. At least two respondents used the Handbook to develop indicators for their projects and for their organizations. Others interviewees shared the Handbook with the Ministries of Education that they work closely with, and have praised it for leading these ministries to consider the potential outcomes of using ICTs in education initiatives.

Finding 8: Although stakeholders indicated that the tools related to ICTs in education are considered useful, they also felt that infoDev does not provide tools or information that are easily operationalized at the ground level.

Providing the research and analysis on specific issues is the first step to addressing the needs of those who have asked for such information. The next step is to have them operationalize that information. Most respondents suggested that infoDev does not have the resources or the capacity to operationalize the toolkits they have produced. Operationalizing such toolkits would include capacity building activities for users, which may include, among others, donors, development practitioners and policymakers. Throughout the interviews, many of these people hinted that infoDev would need more people if they wanted to use and apply the toolkits they produce. Other options might include infoDev establishing partnerships with organizations that specialize in working with others to operationalize or implement these kinds of activities. This may provide a more effective and efficient route for infoDev to apply such knowledge.

“The Handbook is a very interesting piece of academic work. The challenge for infoDev is to transform the book into something people can use”.

“In my experience, people are looking for something less theoretical. In this sense, the book needs to be organized in incremental terms of how the information is presented”.


March 2007


97

One example of infoDev establishing a partnership to operationalize its research is illustrated through its work on the NEPAD e-schools initiative. Recently, infoDev teamed up with the Commonwealth of Learning to use the M&E Handbook to “develop a monitoring and evaluation framework and implementation plan” for the NEPAD e-schools demo project. The NEPAD e-schools initiative is a multi-country, multi-stakeholder, Africa-wide initiative that “intends to impart skills to young Africans in primary and secondary schools and to use ICT to improve the provision of education in schools”.18 From infoDev’s perspective, this activity is an “on the ground” example of how users can use the Handbook to guide them on how to use and invest in M&E in ICT/education projects.19

4 . 5 E f f e c t i v e n e s s

Finding 9: infoDev is recognized for its knowledge and expertise in Monitoring and Evaluation of ICTs in education.

Many of those we interviewed felt that infoDev is becoming recognized for their expertise and knowledge in monitoring and evaluation, and more particularly, for their knowledge of M&E of ICTs in education. As a particular area of specialization, it is considered by most to be a very important and useful field of expertise and one that needs to have leadership. An unintentional result of this work then is that infoDev has established itself within the education sector as a leader in the area of M&E of ICTs in education.

4 . 6 K n o w l e d g e S h a r i n g a n d C o l l a b o r a t i o n

Finding 10: Others perceive infoDev as being very collaborative and willing to establish partnerships with major players for future projects. Through these partnerships and collaborations, infoDev is able to respond to ground-level needs.

Many in the ICT/education field see infoDev’s M&E study as being an important piece of work. Sharing the findings of this study and the resulting Handbook has helped infoDev to establish itself in the field, helping to build their credibility and legitimacy in the education sector and establishing vital links with other players in the sector, such as UNESCO’s Bangkok Office, the Commonwealth of Learning, and GeSCI, to name just a few. Such links have led to several collaborative efforts that have produced other products, including toolkits.

18 Activity File. Monitoring and Evaluation of the NEPAD e-Schools Demo Project: What are the issues, challenges and effective implementation models for introducing ICTs in African schools? infoDev website, accessed November 28, 2006. 19 infoDev, Personal Communication, November 22 and 28, 2006.

“We don’t have M&E expertise at [our organization] and we have to look at all aspects of a project - strategic planning, implementation, M&E planning and so forth, so we collaborate with infoDev to bring that M&E expertise to [our organization]”.

“It’s about bringing the international expertise to develop M&E frameworks and planning for it, and then building the local capacity to do it”.


March 2007


98

infoDev’s collaboration with UNESCO is instructive. After a series of workshops were held, policymakers approached UNESCO with a request to develop a toolkit to assist them in their work. After seeing their M&E Handbook, UNESCO invited infoDev to participate in the project as a competent player in the education sector. UNESCO and infoDev have been working for the past two years on a toolkit – “ICT in Education Toolkit for Policymakers, Planners and Practitioners.” It is meant to be a comprehensive standard training tool for ministries of education and their donor counterparts to plan and assess ICT in education initiatives. More recently, infoDev funded a training workshop on the ICT Toolkit for Policymakers, held jointly with UNESCO. Fourteen Ministers of Education spent one day working with both infoDev and UNESCO on the toolkit, and were so impressed by this training that they changed the agenda for their next meeting to focus specifically on this issue.

According to infoDev staff, participating in the production of the toolkit has enabled infoDev to anchor their education knowledge products in the sector within a larger, holistic planning context and process.

These kinds of collaborations also help infoDev to respond to the needs of their clients. At least one respondent mentioned how infoDev consistently works in a collaborative way with others in the field to get input into the studies and to “ensure that they are responding to ground-level needs”.

Finding 11: infoDev has designed ways to share information and knowledge with others on a regular and consistent basis.

Most stakeholders felt that infoDev is an organization that is easy to work with and one that shares its information and products enthusiastically with others. A number of World Bank staff that work closely with infoDev share its resources and products with their task managers in order to help them keep abreast of what is going on in the education sector at a global level. Other stakeholders mentioned that they share information and other resources with infoDev regularly through newsletters as well as through personal communication with staff. Most felt that this consistent communication and knowledge sharing was both important and valuable because they are working in a rapidly growing and expanding field where knowledge and information is generated on a continual basis and it is difficult to follow and keep up.

Finding 12: infoDev’s current strategy and funding cycle for completing studies does not allow for marketing and disseminating the studies quickly and widely upon completion.

The Handbook is available both on infoDev’s website (full text) as well as in hard copy format. It has, however, remained in draft format since it was distributed at the World Summit on the Information Society in Tunis. It has been circulated on the web; it has received attention through a special panel discussion at WSIS-Tunis, and was presented at eLearning Africa in May, 2006. infoDev staff have also carried out various informal marketing efforts.

“infoDev sends out a newsletter, as do we – we also have a newsletter – and we share information. For example, if they have a product or tool that they want to share they let us know.”

“infoDev does not have the money or the resources to finish the product in terms of doing a final edit. We are now just doing a final edit of the Handbook but it has taken a long time to get to this point.”

“I am reluctant to “put it out there” because it’s not final, it’s not a finished product and I see that as a real problem”.

“The products are generally of high quality and very useful; it’s the delivery of the products that’s the limitation. infoDev doesn’t have a mechanism to deliver their products.”


March 2007


99

Some staff at infoDev consider this to be an issue of inadequate resources available for finalizing their products (final edits, etc.). However, one staff member also made the following comments regarding the planning process for disseminating the Handbook:

“It is important to note that the Handbook has not yet received a strong marketing/dissemination push by infoDev, as the priority for infoDev’s work in the education sector through June 2006 was completing studies, rather than publishing or disseminating them. In this way, the studies could be taken together and could possibly form the basis for a ‘curriculum’ related to ICT use in education, rather than as one-off studies not related to each other.”

The disconnect between completing studies and disseminating the findings and results is an issue common to many research-oriented organizations. However, donors, policymakers, practitioners and others need the data to make informed decisions about investing in ICTs in education. It is difficult, if not impossible, to influence change without providing information to those who need it. This is particularly pertinent if one of the target audiences for infoDev’s education work are the policymakers in developing countries who are looking for current information about ICT use in education. Several recent studies show that if research organizations hope to use their research to influence policymakers (and thus policy) communication and dissemination activities need to be on-going throughout the project cycle, rather than at the end20.

4 . 7 R e l a t i o n s h i p t o t h e W o r l d B a n k

Finding 13: Many of the stakeholders we interviewed felt that infoDev’s relationship with the World Bank was both positive and negative, since it gave them credibility but it also created a lot of administrative bureaucracy.

Many respondents stated that they saw infoDev as not being independent of the World Bank. However, several of these interviewees went on to express that they felt this was both a positive and a negative factor for infoDev. It was suggested that infoDev learn to leverage its relationship with the Bank, since the Bank’s brand can be a source of strength for infoDev, permitting it to gain access and credibility. At the same time, it was felt that infoDev should be independent enough of the Bank that it can lighten the bureaucratic load for both staff and consultants.

Using the Bank’s procedures also sends a message to external partners that because infoDev works with it, that they are not entirely independent of the Bank and that they must answer to the Bank’s accounting staff.

20For more discussion on the issue of research-to-policy links, see Canada’s International Development Research Centre (IDRC) at http://www.idrc.ca/evaluation and the UK’s Overseas Development Institute (ODI) at http://www.odi.org.uk/rapid for recently completed studies and ongoing projects.

”The relationship with the World Bank is useful for infoDev because it gives them credibility. The Bank has enormous sway among developing countries, and among other donors. It gives them the credibility it needs to stand on its work.”

“The World Bank’s licensing arrangements for publications was seen as a limitation for those who were interested in the M&E Handbook – too much bureaucracy to get permission to copy and use the material.”

“One of the difficulties for infoDev is that it’s a question of identities – what is the relationship between the Bank and infoDev? It’s not always clear.”


March 2007


100

Finding 14: Some interviewees felt that the personal relationships they had established with infoDev staff were critical for them to carry out the paperwork necessary for working with infoDev.

Some of the interviewees also mentioned the administrative difficulties they encountered because infoDev uses the World Bank’s model for their procedures for proposals, EoIs and so forth. Because infoDev uses the World Bank’s model, the proposals are then subject to the Bank’s processes for project disbursements and accounting. At this point, “individuals become critical – staff at infoDev can help you through the paperwork”. At least one interviewee we spoke with found the heavy burden of administration at infoDev “shocking”: “I just wanted to make some revisions for our toolkit. It was just a small contract, but the administration of doing these revisions was very burdensome”.

4 . 8 S u s t a i n a b i l i t y

Finding 15: Monitoring and Evaluation in ICT in education is a long-term issue. It is unclear whether infoDev sees such an activity as part of its strategic function.

The knowledge mapping exercise provided the evidence base to show that more data and more information regarding the impact of ICTs in education were needed. As a result, infoDev produced its M&E Handbook. The intent of this Handbook is to provide information to the users, in this case donors and policymakers, that would help them “to make informed policy-related decisions due to insufficient knowledge of the associated costs, benefits, mechanisms and trade-offs.” 21 Developing one output in the area of M&E in the use of ICTs is not a strategic response to a long-term issue.

The knowledge mapping exercise also pointed to eight other areas that need to be investigated. Many of these other areas are also long-term issues, such as for example, “policy issues” and “teachers, teaching and ICTs”. As with the M&E issue, one or two outputs dedicated to these issues is not a strategic response. As such, it might be prudent for infoDev to make some strategic decisions in terms of the areas it wants to work in within the education sector.

Finding 16: Many of the targeted users of the Monitoring and Evaluation Handbook want to operationalize the research. It is unclear whether infoDev has the resources or has engaged in the strategic thinking to operationalize the research they support.

Many stakeholders found the M&E Handbook to be too theoretical in nature, and as such difficult to operationalize on the ground. It is not clear whether (1) infoDev sees operationalizing their research as part of their strategic function, and (2) whether they have the human and financial resources available to provide this kind of activity in support of their research. Once the toolkits and handbooks have been produced and disseminated widely, the next logical step is to work with the users to operationalize the information provided. This may entail building the capacity of the users, and working with them on a continual basis to provide the technical assistance they need throughout the duration of their project/programs.

21 infoDev website, accessed 22 October 2006.


March 2007


101

As we have learned in this study, infoDev has recently teamed up with both UNESCO and the Commonwealth of Learning in two separate activities to work with users to operationalize the information provided in the M&E Handbook (i.e., NEPAD project ) and the policymaker toolkit (workshops with UNESCO). However, it is not clear to us whether or not these are one-off activities or whether infoDev has engaged in any strategic thinking around this idea of operationalizing their research on a continual basis. Such activities may include the provision of continual technical assistance, resources, capacity building events and other interventions to the users, stakeholders and/or partners. It is not clear to us whether or not infoDev has the resources and/or capacity needed to implement such a work program.

Finding 17: Stakeholders we talked to felt that the products and services infoDev supplies are a global public good that should be paid for through donors and/or grant money.

Products and services that can be enjoyed by countless people, and are available for free are considered to be global public goods. infoDev’s education products are one such type of global public good: toolkits and handbooks that are available to many on the internet for free. At least two respondents we talked to felt that infoDev’s products should remain as “global public goods”, since, as one respondent noted, few people will pay a user fee for such information. When probed further, he went on to suggest that grant money must be found to pay for these products since they are so valuable to so many people. Another respondent felt that if the donors are requesting the information then they should be the ones to pay for it. Either way, the important point here, is that the products and services that infoDev supplies is valued by the users and should be paid for through donor and/or grant money.

Finding 18: infoDev needs to clarify its strategic direction and mandate, as well as identify its primary clients to key stakeholders.

A few respondents felt that infoDev needs to clarify its mandate and strategic directions with its key stakeholders. They know that infoDev works with ICTs more broadly than just in the education sector, but are not clear on their overall mandate and objectives. For one respondent, it was knowing infoDev’s mandate so that she could better understand what they do in order for her (and others) to better harness infoDev’s strengths and to make use of their products and services. She also went on to say that this was perhaps a result of receiving funds from too many sources, thus diluting infoDev’s central objectives. Another related question some of the respondents had was: who are infoDev’s primary clients? Although most of those we spoke with felt that infoDev was providing the services and products they wanted/needed, some felt that it’s strong ties to the Bank influenced infoDev’s priorities (for example, see finding 2). Understandably, the transition from one type of organization to another, with a new mandate and different objectives, will cause some confusion among infoDev’s stakeholders and partners. However, infoDev does need to address the fact that at least some of its stakeholders/partners are not clear on their mandate and objectives.

“For us, it’s about knowing what their role is. To know what their mandate is would help us to better understand and to better harness their strengths”.


March 2007


102

Finding 19: infoDev has created a niche for itself by concentrating in the education sector but its not clear if it can do the same in other social sectors.

At least one respondent we spoke with felt that the niche that infoDev had created for itself in the education sector may be difficult to reproduce in other sectors. infoDev has a lot of visibility within the education sector, in part because there is not a lot of competition for the kind of work they do: providing global level and strategic knowledge and information to those who need and use it but are unable to generate themselves. Is there the same need in other sectors such as health, governance, rural livelihoods? Or are there other organizations already well-established in those sectors and which are well-placed to provide such knowledge and information, creating competition for infoDev.

4 . 9 C o n c l u s i o n s & I s s u e s

For its Mainstreaming ICT theme, infoDev developed and implemented a strategy that would help them produce an evidence-based program of work. This strategy includes the knowledge mapping exercise, which identified what is known, and what is not known regarding the use of ICTs in education projects in developing countries. This was followed by consultations with stakeholders in order to select issues of priority for further research. As we have learned from this study, this strategy enabled infoDev to develop and produce relevant products for others working in the education sector. By focusing on a particular issue, infoDev is now considered to be an expert in monitoring and evaluation issues in ICT/education projects and has helped to build their credibility and provided staff with opportunities for collaborations with some of the “big players” (e.g., UNESCO, GeSCI and Commonwealth of Learning).

At the same time, however, we have also learned that some of infoDev’s key stakeholders are not clear as to what infoDev’s overall mandate is, or what their objectives are: do they operationalize their research (i.e., toolkits and handbooks) which would help to ensure the relevance of their products to users at the ground level? If infoDev does consider carrying out such activities, what capacities or resources do they need to carry out such a workload? We also know that the issues identified by infoDev in its mapping exercise are long-term issues. Yet infoDev has not yet identified a response strategy to deal with such long-term issues as monitoring and evaluation of ICTs in education. Finally, many people we spoke with talked about infoDev’s relationship with the World Bank. Many of these people indicated that they see both the strategic advantages of this, for example it provides credibility and legitimacy to the work that infoDev does, but they also see disadvantages such as using Bank procedures for grants/contracts for consultants that cause delays and time consuming administrative processes for both consultants and infoDev staff. Further consideration of these issues should be given a great deal of thought.

“Their visibility in the education sector seems to be more than in other sectors. They should concentrate more in education because they have a real added value. There’s not a lot of competition here”.


March 2007


103

List of People Interviewed NAME ORGANIZATION

Michael Trucano infoDev

Robert Hawkins World Bank Institute (South Africa)

Juan Manuel Moreno World Bank

Shobhana Sosale World Bank

Terry Culver GeSCI

Alex Twinomugisha GeSCI

Cedric Wachholz UNESCO

Paul West Commonwealth of Learning

Glen Farrell Commonwealth of Learning

Wadi Haddad Knowledge Enterprise

List of Documents Consulted

Harnessing ICTs to Fight Poverty and Promote Development: An infoDev Research Strategy and Workplan, 2005-2007. Draft for Donor Discussion, Version 1.3, March 7, 2005

Information and Communication Technologies for Development: Linking Knowledge and Action. infoDev’s Business Plan, August 2005. (Draft for Donor Discussion).

Summary of infoDev’s Donors Seminar, Paris, July 9, 2004

infoDev’s Donors’ Committee Meeting, March 16, 2005: Discussion Summary and Record of Key Decisions

Donors’ Committee Meeting, June8, 2006, Washington, DC: Meeting Summary

Knowledge Maps: ICT in Education. What Do We Know About Using Technology Effectively in Education in Developing Countries? (Draft, November 2005).

Monitoring and Evaluation of ICT in Education Projects. A Handbook for Developing Countries. (Draft, November 2005).

Using Technology to Train Teachers. Appropriate Uses of ICTs for Teacher Professional Development. (Draft, December 2005).