QUALITY OF INFORMATION AND COMMUNICATION TECHNOLOGY (ICT ...€¦ · QUALITY OF INFORMATION AND COMMUNICATION TECHNOLOGY (ICT) COMPONENTS IN BANK PROJECTS A QAG ASSESSMENT July 18,

QQUUAALLIITTYY OOFF IINNFFOORRMMAATTIIOONN AANNDD

CCOOMMMMUUNNIICCAATTIIOONN TTEECCHHNNOOLLOOGGYY ((IICCTT))

CCOOMMPPOONNEENNTTSS IINN BBAANNKK PPRROOJJEECCTTSS

AA QQAAGG AASSSSEESSSSMMEENNTT

July 18, 2006

46149

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ACKNOWLEDGEMENTS

A team managed by Amnon Golan prepared this report. Contributions were provided by the ISGEG team led by Deepak Bhatia and the GICT team led by Philippe Dongier. Data processing and analysis was done by Melvin Vaz. Assessment logistics were managed by Amelia Laya and Nidhi Sachdeva. Eduardo Talero drafted the final report and presentation. Luc Pugliatti provided external, independent review. QAG wishes to thank the Bank technology units involved, the moderator, Eduardo Talero, panelists and task teams who provided additional information for their support and valuable contributions.

Prem Garg, Director, Quality Assurance Group, guided the overall effort.

ACRONYMS AND ABBREVIATIONS

AFMIS Afghanistan Financial Management Information System

MIS Management Information System

AFR Africa Region MNA Middle East and North Africa RegionEAP East Asia and Pacific Region OA Overall AssessmentECA Europe and Central Asia Region PAD Project Appraisal DocumentEIB European Investment Bank PIU Project Implementation UnitGICT Global Information and

Communication TechnologiesDepartment

PSD Private Sector Development Department

HR Human Resources PSG Public Sector GovernanceICT Information and Communication

TechnologyQAG Quality Assurance Group

ITC The set of ICT-related activities within all components of a Bank project,

ISG Information Solutions Group

ISGEG Information Solutions Group e-Government Program

QEA Quality at Entry Assessment

QSA Quality of Supervision Assessment ISR Implementation Status and Results Report

SAR South Asia Region IT Information TechnologyTA Technical Assistance LCR Latin America and the Caribbean

RegionTTL Task Team Leader

TABLE OF CONTENTS

PAGE

MAIN REPORT

Background ........................................................................................................................1Methodology........................................................................................................................4Study Limitations.................................................................................................................5Key Findings........................................................................................................................6Strong Aspects .....................................................................................................................9Aspects Needing Improvement..........................................................................................10Conclusions and Recommendations ..................................................................................11

ANNEXES

1. Main Contributors.....................................................................................................142. Approach Paper.........................................................................................................153. List of Projects ..........................................................................................................194. Complex Projects ......................................................................................................205. Panelists’ Comments.................................................................................................226. Assessment Results (% Moderately Satisfactory or Better vs. % Satisfactory or

Better) .......................................................................................................................23

Quality of Information and Communication Technology Components in Bank Projects 1

Background

The current review, which was launched at the request of the ISGEG and GICT units, is designed to identify strengths and weaknesses in the Bank’s approach to Information and Communications Technology (ICT) components in non-dedicated projects.

ICT’s importance can be seen from the following:

• Some 115 projects in the Bank’s portfolio (8% of the total) with an estimated cost of $ 8.0 billion contain ICT components that are critical to project outcomes, and another 40% contain componentsof lesser importance; and

• Total investments in ICT components under active Bank assisted projects are roughly estimated to be $ 4.0 billion.

To lessen the assessment burden, project selection was limited to the 215 operations previously assessed under QEA6-7, about half of which contain some ICT components.


Background

A total of 31 projects in QEA6-7 contain ICT components that meet the following three characteristics:

• Enabling the creation or enhancement of an information system;

• Enabling policy, administrative, or institutional reforms; and

• Being a key success factor for one or more project outcomes.

Due to logistic and cost considerations the review sample was limited to 24 projects with the highest ICT expenditures.

Components in the sample, which range from a scatter of activities across project components to stand-alone components, typically focus on information systems for sectoral functions of government and on the broader ICT agenda such as national ICT infrastructure policy and regulation, ICT industry development, national e-Government strategies and policies, etc.

…cont’d


Background

Total cost of the 24 projects included in this review (Annex 3) is nearly $3.4 billion and the cost of the ICT components is $680 million.

Lax use of coding options for projects containing ICT complicates the identification and tracking of these components and the estimation of their total cost.

For analysis purposes the sample was further divided into a group of 10 projects (Annex 4) where the ICT components are critical to achievement of project outcomes and characterized by a high degree of complexity.

…cont’d


Methodology

Panels consisting of ICT specialists examined PAD’s, PIP’s, QEA findings and other available project documentation.

There were no interviews with task teams.

An external consultant reviewed the panels’ key findings and recommendations.

A customized questionnaire was used to evaluate issues specific to ICT components (Annex 6).


Study Limitations

Documentation gaps were severe and inferences had to be made from available information.

Panelists’ views differed markedly on many aspects being scored, reflecting the absence of standards for this work at the Bank.

Ratings of “Satisfactory” or better and “Moderately Unsatisfactory”or worse reflect consensus among panelists. Borderline ratings where differences of view persisted were often rated “Moderately Satisfactory”.


Key Findings

At 42% “Satisfactory” or better the quality-at-entry ratings for ICT components are significantly below QEA7 (64%).

Aside from highlighting basic shortcomings in the Bank’s approach to ICT activities, the review also found a troubling weakness (40% Satisfactory) among the ten projects containing ICT components that are highly complex and also critical to the overall development outcomes.

There are few instances where specialists staff from Bank’s ICT support units are called upon for assistance.

Although ICT components are frequently critical to overall project outcomes, many Bank staff and managers fail to appreciate their importance and needs:

• They are incorrectly assumed to be primarily technical challenges;• They are frequently glossed over by managers, peer reviewers and evaluators;• Their special risks are frequently overlooked and recognized only when disaster strikes;• Their special procurement requirements are not properly recognized; and• The Bank’s ICT support units are rarely called upon for assistance.


Key Findings

Quality issues arising from the review fall into three areas:

(1) Poor project design and planning – the most significant shortcomings, which reflect the absence of agreed design guidelines, are inadequate implementation planning, poor risk assessment, and inadequate provisions for sustainability.

(2) Inadequate Skills mix – there is a high incidence of isolated, unsupervised ICT operations work outside the established units without input from ICT specialists.

(3) Resource mismatch – primarily between the complexity and resource intensity of ICT operations work and the corresponding time and budget allocations.

…cont’d


Key Findings

OA: Overall

R1: Strategic Relevance and Approach

R2: Technical, Financial and Economic Aspects

R3: Policy and Institutional Aspects

R4: Implementation Arrangements

R5: Risk Assessment

R6: Implementation Oversight

R7: Bank Inputs and Processes

1= Highly Satisfactory

2= Satisfactory

3= Moderately Satisfactory

4= Moderately Unsatisfactory

5= Unsatisfactory

6= Highly Unsatisfactory

Quality Dimensions

Assessment Ratings

QEA7 vs. ICT (All Projects)(% Satisfactory or Better)

* As this dimension was not assessed in QEA7, the entry is based on findings from QSA6.

…cont’d

0

20

40

60

80

100

R1 R2 R3 R4 R5 R6* R7

QEA7 ICT (All Projects)


Strong Aspects

Aspects of strength identified by this assessment are:

• Relevance of the ICT components to the projects’ development objectives (77% Satisfactory);

• Attention to business processes and information flows affected by the ICT (67% Satisfactory);

• Attention to policy requirements for successful installation and long term operation (64% Satisfactory);

• End users’ participation in the design of these components (62% Satisfactory); and

• Justification of the ICT from a business perspective (59% Satisfactory).


Aspects Needing Improvement Project Preparation and Design:

• Provisions to develop and maintain skills for technical management and system operation (14% Satisfactory);

• Implementation planning, i.e., phasing, sequencing and scheduling (18% Satisfactory);

• Risk assessment (22% Satisfactory);

• Attention to sustainability (23% Satisfactory);

• Procurement strategy and the complexity of outsourcing contracts (23% Satisfactory);

• Technical, financial and economic aspects (36% Satisfactory);

• Disconnect with E-Government policy and standards (panelists’ observations); and

• Terms of reference for design and business processes re-engineering studies are often lacking or too general (panelists’ observations).

Bank Inputs and Processes:

• Arrangements for peer review (17%); and

• Absence of a broad operational policy objectives for ICT lending to guide project design (panelists’ observations).


Conclusions and Recommendations

Findings emerging from the review underscore the need to strengthen the preparation and supervision standards of ICT by developing agreed Bank guidelines for managing these components.

There is a need to pay greater attention to implementation planning, skill mix and the budgeting of resources commensurate with the complexity and importance of these components.

Tackling the shortcomings identified by this review requires renewed efforts in addressing the following aspects:

ISGEG & GICT:

(1) Formulate and submit for management consideration a policy statement on objectives, principals and priorities for Bank lending for E-Government information systems.



(2) The above policy statement should also cover operational aspects(similar to OP 10.2 on FM) related to Bank-wide requirements and guidelines for preparation, appraisal and supervision of ICT project components.

(3) Consolidate, nurture and leverage a cadre of senior operational ICT specialists to take responsibility for complex ICT projects in the portfolio.

(4) Redesign the ICT training program to focus TTLs curriculum on new operational and lending policies (above) and ICT specialists curriculum on project preparation and supervision topics.

(5) Prepare a portfolio of case materials and best practices and a simplified toolkit for operational staff from other sectors that are managing ICT components.

…cont’d



Regions:

(6) Combine (whenever reasonable) all ICT activities into stand-alone components within Bank projects.

QAG:

(7) Pay greater attention to ICT aspects in QEA and QSA.

…cont’d


Annex 1

MAIN CONTRIBUTORS

Manager: Amnon GolanModerator and Report Writing: Eduardo Talero

Outside Reviewer: Luciano PugliattiData Analysis: Melvin Vaz

Logistics: Amelia LayaNidhi Sachdeva

Panelists: Henry Forero RamirezFritz KonigshoferMichel MaechlerSamia MelhemAntoun MoussaRobert SchwareRamesh SivaEloy Eduardo Vidal


Annex 2

ASSESSMENT OF THE QUALITY OF INFORMATION AND COMMUNICATION

TECHNOLOGY (ICT)

APPROACH PAPER

BACKGROUND

1. A growing trend of combining under a single operation several activities addressing a number of issues/sectors has given rise to projects containing components falling under the auspices of sectors outside the managing unit1. Several recent studies conducted by IEG and results from different QAG assessments show that these multi-sector projects tend to be complex, the attention to the non-dedicated components is not always adequate, and the standards of appraisal/supervision sometime fail to meet the guidelines established by the relevant Sector Boards. In order to examine the extent and severity of this occurrence, several Sector Boards have asked QAG to investigate this phenomenon. A first such review is currently underway in the Water Supply and Sanitation sector and this will be followed by a review of components relating to the Transport Sector. This Approach Paper addresses the Information and Communications Technology (ICT) components. With projects in practically every sector containing ICT components, the current review will be a joint undertaking between ISGEG and GICT units and QAG. The affected Sector Boards with the largest number of projects in the sample (Education, PSG, Rural, PSD and Financial Sector)will be kept fully informed on the details of this review.

2. The absence of adequate coding options for identifying projects with ICT components complicates the monitoring of these activities. While the current review will help shed some light on quality issues, it will not directly address the coding problems. A detailed review conducted jointly by the ISG, GICT and QAG of the 215 projects evaluated under QEA6-7 shows that about half this sample contains an ICT component. This sample, which provides a good proxy for all projects approved by the Bank during the three year period (FY04-06), exhibits the following characteristics:

There are very few stand alone (dedicated) ICT projects in the portfolio (just 1% of the sample); and

About half the projects have no discernible or a very small (less than $1.0 million) ICT component.

3. Projects containing stand alone ICT components (these frequently involve telecommunication) and/or ICT-enabled information and MIS systems account for the balance (48%). The latter systems are often embedded in a larger sub-component of the

1 Operations of this kind are often referred to as “non-dedicated” or “multi-sector.”


project but as their cost is frequently not shown separately in the PAD, it is difficult to assess the extent of Bank funding for ICT components.

OBJECTIVES AND APPROACH

4. A major objective of this exercise is to examine the quality-at-entry and the quality of supervision of ICT components included in Bank funded projects. Given the limited scope of this review and its experimental nature, it will be conducted in a “learning” rather than “accountability” mode. Accordingly, only the aggregated overall results will be shared with the Regions and the responsible Sector Boards. Furthermore, in order to minimize the burden on task teams, the sample of 24 projects to be reviewed was selected from the 215 operations in QEA6-7. Since all these operations were previously assessed for quality-at-entry, the review will focus exclusively on the quality of the ICT component without any further analysis of other aspects. The panels’ work will be limited to a desk review using information previously supplied by the task team and the findings of the QEA panel. Panelists will also review the ISRs, aide-memoirs and key correspondence with the borrower. In general, interviews with the task team (or the person on the team responsible for the ICT component) will be limited to those cases where the PAD lacked adequate information or where the panel has identified issues that it cannot reconcile without additional clarifications from the task team before finalizing the assessment.

METHODOLOGY

5. A scrutiny of the projects containing stand alone ICT components (see paragraph 2) provided the basis for separating them into a high end and a low end categories based on the importance of the ICT component. Projects falling into the high end category had to include components meeting the following three criteria:

a. Enabling the creation or enhancement of an information system. Aninformation system refers to an ongoing institutional performance capability with its full context of users, technology, process, information, regulations, end users services, etc;

b. Enabling policy, administrative or institutional reforms of central importance to project objectives; and

c. Presenting a major risk to the achievement of project objectives.

6. Projects falling into the low end category were those where the ICT activities are of minimal importance. A total of 31 projects fell into the high end category and the 24 projects with the largest overall cost out of this cohort were included in the sample to be reviewed by the current exercise. With procurement of technology components normally expected in the early years of implementation, the review will also examine the quality of supervision of these projects since Board approval. In order to test the applicability of the approach and the usefulness of the questionnaire, a pilot review of four projects was carried out. Findings from the pilot review are reflected in this approach paper and the final questionnaire.


7. An ICT panelist will rate each operation with the help of a modified questionnaire designed specifically for this review. Because of the specialized nature of these activities there will be no second panelists from the concerned sector though where the panel will find issues with the ICT component, a sector specialist will be consulted. Each panelist will review the operation independently and thereafter meet with the QAG moderator to discuss the findings, agree on the ratings and draft the project’s assessment report. To the extent feasible the review will also involve a discussion with the panel that carried out the earlier quality-at-entry assessment. For each operation the panel will assign a comprehensive rating reflecting the appropriateness and quality of the component, and the adequacy of Bank’s inputs during the design and supervision stages. To permit comparison with the Bank-wide performance, most of the questions correspond to those found in QEA and QSA. Ratings of the questions will be based on the six-point scale used for QEA7 and panelists will be expected to make use of the full range afforded by this scale and to annotate all ratings but especially the outliers. Within each rating category, they will be encouraged to use the comments section for sub-ratings and the summary narrative text to differentiate among stronger and weaker performances. The panels would be expected to address in their overall assessment summary how the ICT component has advanced the key objectives of the project and the extent to which adequate use was made of available technology and experience in designing and implementing the component.

8. With only a handful of dedicated ICT projects in the portfolio (and most of these being telecom and thus not readily comparable with the non-dedicated projects) no attempt will be made to compare these two groups. Instead, results from the present study will be compared with the Bank-wide strengths and weaknesses for similar questions under QEA and QSA. Findings related to quality-at-entry and supervision emerging from the review and a comparison with the Bank-wide results will be summarized in a brief Synthesis Report. The report will also include recommendations for appropriate actions required to address emerging issues. Sector Boards involved in the assessment would be invited to comment on the draft Synthesis Report, before it is finalized by QAG.

TIMETABLE

9. The timetable being proposed for this review is as follows:

a. Completion of the Approach Paper and selection of the sample – April 5

b. Collection of documents by QAG (only in rare cases are TTLs going to be asked to provide additional material to that already on hand) – April 7

c. Commencement of individual projects’ review – April 17

d. Completion of the projects’ review – April 28

e. Draft Synthesis Report – May 5

f. Discussions with the concerned Sector Boards – May 9

g. Final report – May 15


ESTIMATED COST

10. Conducting the pilot review, finalizing the approach paper and preparing an assessment questionnaire required 14 days, while the selection of the projects required an input of 6 days. Assessment of each project in the sample is expected to require an input of 0.7 person/days from the ICT panelist and 0.3 days from the QAG moderator. With 24projects included in the review, this would translate into 24 person days. Drafting of the Synthesis report and QAG’s oversight of the work are expected to amount to 10 days. Adding 5 days for contingencies, the total is estimated to be 60 days. Converting the days into dollars and adding $5,000 for administrative support brings the total to about $50,000. This cost will be shared by QAG and the IT units.


Annex 3

LIST OF PROJECTS

SL# Year Project ID

Title Country Region Sector

Project Total Cost ($ million)

1 QEA7 P066386 RW-Public Sector Cap Bldg TAL (FY05) Rwanda AFR Public Sector Governance 212 QEA6 P071443 Competitiveness & Enterprise Development Burkina Faso AFR Private Sector Development 343 QEA6 P071063 Governance Technical Assistance Credit (GTAC) Kyrgyz Rep ECA Public Sector Governance 104 QEA6 P062916 Central Bank Strengthening Project Bangladesh SAR Financial Sector 46

5 QEA7 P078991 NI - (APL2)HEALTH SECTOR II Nicaragua LCRHealth, Nutrition and Population 82

6 QEA7 P057857 CR EQUITY AND EFFICIENCY OF EDUCATION Costa Rica LCR Education 517 QEA7 P074633 Education for All Project Nepal SAR Education 6648 QEA7 P069852 CN-Wuhan Urban Transport China EAP Transport 598

9 QEA6 P059872 Basic Education 2 (APL #2) Turkey ECA Education 357

10 QEA7 P083543 LA - APL2 Road Maintenance Program Lao People's EAP Transport 70

11 QEA7 P078359 SEISMIC RISK MITIGATION Turkey ECA Rural Sector 40012 QEA7 P069934 BR-PERNAMBUCO INTEG DEVT: EDUC QUAL IMPR Brazil LCR Education 53

13 QEA7 P088060 BAM EARTHQUAKE EMERGENCY RECONSTRUCT Iran Islamic MNA Urban Development 23514 QEA7 P077306 Tax Administration Reform Project Pakistan SAR Public Sector Governance 14915 QEA7 P055991 HN LAND ADMINISTRATION PROGRAM Honduras LCR Rural Sector 3916 QEA7 P049721 AGRIC COMPETITIVENESS Kazakhstan ECA Rural Sector 8317 QEA6 P064919 Judicial Modernization Project El Salvador LCR Public Sector Governance 2418 QEA6 P063546 Pension System Investment Croatia ECA Social Protection 36

19 QEA7 P083890 Economic Management TA Program (EMTAP) Bangladesh SAR Public Sector Governance 25

20 QEA7 P071115 TN-EXPORT DEVELOPMENT II Tunisia MNA Private Sector Development 5521 QEA6 P072960 Custom Devt Russian Fed ECA Public Sector Governance 18722 QEA7 P091344 IQ-EMERGENCY PRIVATE SECTOR DEVELOPMENT Iraq MNA Private Sector Development 5623 QEA6 P072881 BEAC Regional Payment System Africa AFR Financial Sector 2324 QEA7 P075184 PH: Diversified Farm Income & Mkt. Devt Philippines EAP Rural Sector 70


Annex 4

COMPLEX PROJECTS

Selection Criteria

1. A sub-group of ten projects (Table 1) designated as “complex” were analyzed separately as part of the review. These projects were selected in a subjective manner by the study coordinator as being roughly comparable or more complex than the SAP system developed as an operational and MIS system for the World Bank. This choice was made to put in perspective the level of confusion, conflict, expense, time, technology and project management expertise associated with complex ICT components,

2. The “SAP system” is the central ERP system for the Bank and supports all operational, financial, administrative, and managerial business processes of the Bank. It cost approximately $56 million and took close to 18 months to go live and about one year to stabilize. Most Bank staff can be considered highly sophisticated system users needing little system-related training. On the other hand, infrequent users such as managers and some team leaders have found the system to be difficult to use. They are also highly opinionated and, due to many pressures, have a low tolerance threshold. Development of the SAP system was facilitated by prior years of trial and error developing region-specific MIS systems and mature systems in the financial and administrative areas.

6. Table 1 profiles the ten projects that were deemed of similar or higher complexity than the Bank’s SAP system. As can be seen, higher complexity does not always correlate with higher costs. Complexity is mainly associated with “soft” aspects such as identification and streamlining of business processes, development of system specifications, construction and acceptance testing of the software, training and build-up of institutional capacity, system deployment, system maintenance, etc. When ICT components do not include procurement of a large number of software licenses and equipment, the cost of the soft components is rarely over US $ 10 million.

TABLE 1: Complex Projects – Cost Profile

Project description Approx. ICT Cost (US$ Million)

% of Total Project Cost

Africa - BEAC regional payment system 14 60Burkina Faso - trade facilitation 10 34Bangladesh - full Central Bank automation 32 86Russia - customs administration 153 82Kyrgyz Republic - treasury management 6 65PH: agriculture markets information 20 30Turkey: seismic risk mitigation 62 16Salvador - judicial modernization 6 30Pakistan - tax Administration 90 60Tunisia - export development II 11 20TOTAL 404 40


7. The impact of projects in this sub-group is frequently economy-wide and involves creation of operational, administrative and MIS systems for large organizations spread usually throughout a whole country. Some examples of their scope are:

• Creation of an electronic payments system in 6 countries in central Africa for about 16 million economically active people;

• Creation of operational, administrative, MIS and electronic payment systems for 5,800 employees of the central bank and the economically active population of Bangladesh; and

• Creation of operational, administrative, MIS and trade facilitation systems for 62,000+ customs employees in Russia.

TABLE 2: Ratings for Complex Projects

Moderately Satisfactory

or Better

Satisfactory or BetterTotal

# Rated #

Rated% #

Rated%

% Diff

OA OVERALL ASSESSMENT 10 7 70 4 40 30R1 Strategic Relevance and Approach

10 8 80 6 60 20

R2 Technical, Finance and EconomicAspects

10 6 60 3 30 30

R3 Policy and Institutional Aspects 10 10 100 9 90 10R4 Implementation Arrangements 10 7 70 3 30 40R5 Risk Assessment 10 6 60 1 10 50R6 Implementation Oversight 10 10 100 7 70 30R7 Bank Inputs and Processes 10 5 50 4 40 10


Annex 5

PANELISTS’ COMMENTS

1. Key preparation studies are left for the implementation phase with consequent high level of uncertainty.

2. Except for telecommunications, ICT components sorely need operational lending policy guidance. Otherwise they fail to advance strategic priorities like:

Emphasis on providing services, particularly for the poor; Transparency and accountability features in system design; Customer orientation in location and design of user interfaces; and Where applicable, Government technology policy considerations.

3. ICT specialists require knowledge and expertise in:

Public sector business processes, change management & institutional capacity building;

E-Government principles, policies, institutions and implementation strategies; Design, management and evaluation of large public sector ICT projects; and Technology architectures, standards, trends, options and costs.

4. Many ICT components are handled by TTLs, sector specialists, borrower staff or consultants. It is not readily apparent that they possess the required skills.

5. Overly ambitious ICT components are frequently not backed by commensurate feasibility, design and implementation planning studies.

6. Resources required for good preparation are seldom available.


Annex 6

ASSESSMENT RESULTS(% MODERATELY SATISFACTORY OR BETTER VS. % SATISFACTORY OR BETTER)

# Rated

% Moderately Satisfactory or

Better

% Satisfactory

or Better

% Diff

OA OVERALL ASSESSMENT 24 58 42 16

R1 STRATEGIC RELEVANCE AND APPROACH 22 73 45 28

R2 TECHNICAL, FINANCIAL AND ECONOMIC ASPECTS 22 59 36 23

R3 POLICY AND INSTITUTIONAL ASPECTS 22 91 64 27

R4 IMPLEMENTATION ARRANGEMENTS 22 68 23 45

R5 RISK ASSESSMENT 23 57 22 35

R6 IMPLEMENTATION OVERSIGHT 22 77 55 22

R7 BANK INPUTS AND PROCESSES 23 43 35 8

1.1 Relevance of the Information Technology component (ITC) to the development objectives of the project?

22 82 77 5

1.2 Clarity of what the ITC must accomplish? 22 73 41 32

1.3 Consistency of the ITC component with lessons of experience from similar projects?

22 64 41 23

1.4 Level of Borrower ownership, commitment and support? 22 82 55 27

2.1 Appropriateness of the ITC design to the prevailing level of policy, legal or institutional maturity?

22 82 45 37

2.2 Technical soundness of the ITC design? 22 64 50 14

2.3 Adequacy of the provisions for system rollout (TA, user training, and data collection)?

22 45 27 18

2.4 Adequacy of the provisions for ongoing technical support and system maintenance?

22 50 23 27

2.5 Justification of the ITC from an economic or business perspective?

22 77 59 18

3.1 Extent to which the policy requirements for successful installation and long term operation of the ITC are adequately defined and provided for?

22 86 64 22

3.2 Extent to which there has been adequate user participation in ITC design and a sound communication and training program to keep stakeholders engaged?

21 81 62 19

3.3 How adequate are the provisions for analysis, streamlining and documentation of business processes and information flows affected by the ITC?

21 81 67 14

3.4 How adequate are the provisions to develop and maintain the technical management and system operation staff skills needed to guide the implementation of the ITC and later on to keep it running?

22 68 14 54


# Rated

% Moderately Satisfactory or

Better

% Satisfactory

or Better

% Diff

4.1 How adequate is the implementation plan (phasing, sequencing, scheduling) for the ITC?

22 68 18 50

4.2 How adequate and specific are the institutional arrangements (accountability, authority, resource management, and audit) for implementation and operation of the ITC?

22 77 32 45

4.3 Clarity and adequacy of TOR for TA to support Borrower in managing the implementation of the ITC and in formulating related policy, strategy and technical specifications?

22 73 23 50

4.4 Soundness of the procurement strategy for the ITC? 22 59 23 36

4.5 Adequacy of arrangements for monitoring and evaluation of the ITC?

23 70 35 35

5.1 Extent to which the risks of the ITC been properly identified and mitigation provisions specified?

23 57 26 31

6.1 Extent to which the Bank has provided quality technical and procurement advice to the borrower during project implementation?

22 77 55 22

7.1 Appropriateness of the Task Team’s skill mix in relation to the ITC?

23 61 39 22

7.1a At entry? 23 65 43 22

7.1b During supervision? 23 57 39 18

7.2 Adequacy of arrangements for peer review of the ITC?23 52 17 35

7.3 Extent to which the task team sought advice from the Bank's IT units?

23 43 30 13

wb279828M:\Support for Quality Enhancement (SQE)\ICT Assessment\Synthesis Report\ICT Synthesis Report Annexes.doc07/18/2006 11:46:00 AM

Documents

QUALITY OF INFORMATION AND COMMUNICATION TECHNOLOGY (ICT ...€¦ · QUALITY OF INFORMATION AND COMMUNICATION TECHNOLOGY (ICT) COMPONENTS IN BANK PROJECTS A QAG ASSESSMENT July 18,