The Document of The World Bankdocuments.worldbank.org/curated/en/347391468131116977/...The Document of The World Bank Report No: ICR 2976 IMPLEMENTATION COMPLETION AND RESULTS REPORT

The Document of The World Bank

Report No: ICR 2976

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-41160)

ON A

CREDIT

IN THE AMOUNT OF SDR 60.80 MILLION (US$ 93.72 MILLION EQUIVALENT)

TO THE

SOCIALIST REPUBLIC OF VIETNAM

FOR AN

INFORMATION AND COMMUNICATION TECHNOLOGY

DEVELOPMENT PROJECT

June 26, 2014

Transport, Water and ICT Department Vietnam Country Management Unit East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 31, 2013)

Currency Unit = Dong (VND) VND 0.000047 = US$ 1.00 US$ 1.00 = VND 21,100.00

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

CAS Country Assistance Strategy CA Certification Authority CCNA Cisco Certified Network Associate CERT Computer Emergency Response Team CIO Chief Information Officer DCA Development Credit Agreement DIC District Office of Information Communication DOT Department of Transportation EA Enterprise Architecture e-GAF e-Government Assessment Framework G2B Government to Business G2C Government to Consumer/Citizen G2G Government to Government GIS Global Information System GOV Government of Vietnam GSO General Statistics Office HCMC Ho Chi Minh City ICT Information and Communications Technology IDA International Development Association IO Information Officer IOI Intermediate Outcome Indicator LAN Local Area Network MAN Metropolitan Area Network MCSA Microsoft Certified Solutions Associate M&E Monitoring and Evaluation MIC Ministry of Information and Communication MOFA Ministry of Foreign Affairs MPI Ministry of Planning and Investment MPT Ministry of Posts and Telematics ODA Official Development Assistance PAPI Public Administration Performance Index PARI Public Administration Reform Index

PCI Provincial Competitiveness Index PCU Project Coordination Unit PDO Project Development Objective PIU Project Implementation Unit PSO Provincial Statistics Office RF Results Framework SAMCOM State Administrative Management Computerization SHS Statistical Hub System SME Small and Medium-sized Enterprise SSIC System of Statistical Information Collection UNFPA United Nations Population Fund WAN Wide Area Network

Vice President: Axel von Trotsenburg Country Director: Victoria Kwakwa

Sector Manager: Randeep Sudan Project Team Leader: Deepak Bhatia

ICR Team Leader: Deepak T. Bhatia

VIETNAM ICT Development Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 14 4. Assessment of Risk to Development Outcome ......................................................... 22 5. Assessment of Bank and Recipient Performance ..................................................... 23 6. Lessons Learned ....................................................................................................... 27 7. Comments on Issues Raised by Recipient/Implementing Agencies/Partners .......... 29 Annex 1. Project Costs and Financing .......................................................................... 30 Annex 2. Outputs by Component ................................................................................. 31 Annex 3. Economic and Financial Analysis ................................................................. 63 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 68 Annex 5. Beneficiary Survey Results ........................................................................... 70 Annex 6. Stakeholder Workshop Report and Results ................................................... 78 Annex 7. Summary of Recipient’s ICR and/or Comments on Draft ICR .................... 79 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 90 Annex 9. List of Supporting Documents ...................................................................... 91

A. Basic Information

Country: Vietnam Project Name: Vietnam - ICT Development

Project ID: P079344 L/C/TF Number(s): IDA-41160 ICR Date: 06/26/2014 ICR Type: Core ICR

Lending Instrument: SIL Borrower: THE GOVERNMENT OF VIETNAM

Original Total Commitment:

XDR 60.80M Disbursed Amount: XDR 48.01M

Revised Amount: XDR 52.04M Environmental Category: C Implementing Agencies: Ministry of Information and Communications Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 04/11/2003 Effectiveness: 09/29/2006 09/29/2006

Appraisal: 05/23/2005 Restructuring(s): 01/28/2010 06/29/2011 06/20/2013

Approval: 09/15/2005 Mid-term Review: 03/16/2009 03/16/2009 Closing: 06/30/2011 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately

Unsatisfactory

Quality of Supervision: Moderately Unsatisfactory

Implementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:


Overall Borrower Performance:


C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 43 50 General industry and trade sector 22 10 Sub-national government administration 35 40

Theme Code (as % of total Bank financing) Administrative and civil service reform 25 55 Infrastructure services for private sector development 25 25 Micro, Small and Medium Enterprise support 24 10 Other accountability/anti-corruption 13 5 Regulation and competition policy 13 5 E. Bank Staff

Positions At ICR At Approval Vice President: Axel van Trotsenburg Jemal-ud-din Kassum Country Director: Victoria Kwakwa Klaus Rohland Sector Manager: Randeep Sudan Pierre Guislain Project Team Leader: Deepak T. Bhatia Zhenwei Qiang ICR Team Leader: Deepak T. Bhatia ICR Primary Author: Anna F. Roumani F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objectives of the proposed project (as stated in the PAD) were to: (a) accelerate the use of ICT and support institutional development in order to increase the efficiency and

effectiveness of public administration, increase transparency and availability of information, and improve public services delivery to stakeholders; and (b) to foster private sector development in particular by facilitating the increased adoption of ICT by SMEs. Revised Project Development Objectives (as approved by original approving authority) A Level One Board-approved Restructuring in 2010 revised the PDO (and Key Performance Indicators) to focus more narrowly on project activities funded by IDA by assisting the Borrower to: (a) support MIC's institutional development and technical and regulatory capacity for ICT sector oversight and to increase the efficiency and transparency of public service delivery in MIC, GSO, MOFA, Hanoi and Danang; and (b) foster private sector development by increasing access to business-related government services and raising awareness and usage of ICT in Hanoi and Danang. Further adjustments to PDO and Intermediate Outcome indicators were made via a Restructuring in June 2013, approved by the Regional Vice-President. For clarity and consistency with the ICR Guidelines Appendix B requirements, the presentation below is organized as follows: (i) original, PAD-based indicators are listed first in both sections (a) and (b) and are used to measure the initial phase of the Project; (ii) the revised indicators resulting from the 2010 Level One Restructuring are then shown in follow-on sections of (a) and (b) immediately under the PAD originals; (iii) the June 2013 adjusted indicators and results are shown in the "Formally-revised Target Values" column of these two post-Restructuring sections and are used to measure achievement of PDO in the Project's second phase. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Original Key Performance Indicators (from PAD): Increase share of ICT sector in GDP by 2 percent.

Value quantitative or Qualitative)

Zero (PAD did not state existing ICT share of GDP)

2% increased share na

Date achieved 05/23/2005 06/30/2011 01/28/2010

Comments (incl. % achievement)

Not achieved: Indicator was not included in PAD Arrangements for Results Monitoring (ARM). Required sophisticated measurement not feasible within life of project, difficult to attribute to project, and lacked baseline. Dropped by the 2010 Restructuring.

Indicator 2 : Increase in ICT capacity and knowledge in respective government departments. Value quantitative or Qualitative)

Zero Undefined na

Date achieved 05/23/2005 06/30/2011 01/28/2010


Not achieved: Indicator was not included in ARM, scope was too broad, target not quantified and indicator dropped by 2010 Restructuring without measurement, for same reasons as above.

Indicator 3 : 15% use ICT for business operations in 2010 and improve productivity Value quantitative or Qualitative)

3% SMEs use e-commerce (2003 survey data)

15% and improved productivity na

Date achieved 05/23/2005 06/30/2011 01/28/2010 Comments (incl. % achievement)

Not achieved: Indicator was not included in the ARM, scope was too broad and meaning unclear. Dropped by 2010 Restructuring without measurement.

Indicator 4 : Domestic IT service sector grows by an average of 30% every year Value quantitative or Qualitative)

Zero 30% growth annually na


Not achieved: Indicator not mentioned in PAD ARM. Broad scope, project attribution issues and lack of baseline constrained measurement. Indicator dropped by the 2010 Restructuring.

Indicator 5 : Increase in standardization and improvement in transparency of administrative processes


Workflows uncoordinated and fragmented due lack of integrated computer systems

100% of identified workflows and reporting lines automated

na


Not achieved: Indicator too broad and attribution to project problematic. Re-phrased by the 2010 Restructuring and combined into Intermediate Outcome Indicator (IOI). See IOI 40 below.

Indicator 6 : Significantly lowered transaction costs for targeted government services (G2C and G2B)


Lengthy transaction times and high costs (formal and informal) associated with many government services

Undefined

E-services were in process of selection (radio spectrum, publications, standards for integrated communications equipment).

Date achieved 05/23/2005 06/30/2011 01/28/2010


Negligible achievement: Services still being selected at time of Restructuring (2010). Baseline and target not quantified. Indicator was revised by the 2010 Restructuring and became an IOI linked to MIC modernization. See IOI 42 below.

Indicator 7 : Increase in user satisfaction for e-government services and visits to City Portal. Value quantitative or Qualitative)

Low satisfaction among users of many existing government services

Undefined na


Not achieved: Indicator was not mentioned in ARM, too broad, and baseline not quantified to measure incremental gains. Revised by the 2010 Restructuring. See Results Indicator 9 below.

Indicator 8 :

Results Indicators (RI) Post-Restructuring: (01/28/2010 to 12/31/2013) Improved formulation and implementation of ICR policies and common standards


Limited capacity to formulate policies and guide implementation on latest issues in ICT sector. Gap in policy instruments.

2010 Indicator: Priority telecom policies and implementation guidelines formulated; priority policies for information technology and IT industry promotion in place.

Adjusted RI 2013: Formulation, implementation ICT policies, common standards in MIC. Target: 3 standards, Enterprise Architecture (EA); Inter-operability; and, E-Govt Framework and Security.

4 standards formulated: EA, Inter-operability; E-Government; and Security Standard

Date achieved 01/28/2010 06/30/2013 12/31/2013 12/31/2013 Comments (incl. % achievement)

Achieved 100%: The 2010 indicator was clarified in 2013 to improve link to the PDO and measurement. Actual baseline at the time was: EA and Interoperability already formulated; e-Gov Framework and Security under implementation.

Indicator 9 : Reduced transaction costs, increased transparency and users' satisfaction of selected government services through e-applications


Lengthy transaction times and high costs of many government services, and low user satisfaction

Undefined

Adjusted RI: Reduced time for one transaction of selected govt services through e-applications. Target: Transaction time reduced by 1.5 times (Hanoi, Danang)

Danang: 1 G2B service operational (Bus. Registration); devt phase completed, system launched. No time for user survey. Hanoi: 3rd level public services online via Hanoi DIC; transaction times reduced 60%


Achieved 100%: 2010 indicator was clarified in 2013 and scope narrowed. Specific elements (transparency, user satisfaction) became separate IOIs by city. Cost and user satisfaction surveys not completed under this RI.

Indicator 10 : G2B services made available online (Hanoi and Danang)


One G2B service operational (Business Registration Service in Danang)

Undefined

Adjusted RI 2013: No. G2B services available online (Hanoi, Danang). Target: 3 G2B services online; Bus. Reg. Service (Hanoi, Danang); Public Procurement Service (Danang)

Danang: One G2B service (Bus. Registration) launched and available online.


Partially achieved: 33% of revised target. 2010 target clarified and quantified in 2013. Public Proc. Service in Danang cancelled due implementation with Govt funds. Hanoi Business Registration procurement package HAN7 cancelled.

Indicator 11 : More private firms in selected cities use ICT and e-services


SMEs well-informed re existing services provided by Contact Center; e-Learning Portal launched; 400 ICT and soft-skill courses available in e-Learning Portal; 25 software developers trained in Requirement Management

500 managers of SMEs trained

Adjusted RI: More private firms using ICT and e-services in Danang. Target: 500 SME managers trained by MIC to apply IT in enterprises; 1000 SMEs trained to use Danang e-services; 60% SMEs use ICT in business; and 40% use Danang ICT Transaction Center.

E-Learning Portal launched (http://multimedia-lab.org) with 400 courses; 268 SMEs took e-learning courses; 73 software developers trained; 497 CIOs trained; 10% SMEs using ICT for business; basic IT Knowledge program on TV 3 times/week.


Partially achieved: Restructuring 2013 clarified and quantified 2010 indicator to improve link to PDO and measurement. Significant achievements but not linked closely to indicators and targets.

Indicator 12 : New Results Indicator (added by 2013 Restructuring):

Number of e-services and socio-economic information available through eGateway (MIC), portal sites (GSO, Hanoi, and Danang)


Danang one-gate service implemented and operational

1. 3 e-services available via MIC gateway; radio transmitter permit; registr. of plans by publishing houses; registr. imported publication portfolio of publishing co's. 2. 1 GSO Portal 3. 1 Hanoi and 1 Danang one-gate services portal

3 e-services online via MIC; all application processes converted to e-transaction. Danang: one-gate services implemented, operational. GSO: Portal completed, operational. Hanoi: One-gate services portal completed.


Achieved - 100%

Indicator 13 :

New Results Indicator (added by 2013 Restructuring): Upgrading ICT infrastructure to support e-government technical capacity in MIC, GSO, MOFA, Hanoi and Danang.


Almost completed in Danang and under implementation in MIC (HQ and DICs), GSO and Hanoi

Completion of 5 internal ICT networks (LAN): MIC and 12 DICs; GSO; MOFA; Hanoi; Danang.

Hanoi: HAN5 completed, tested, end-user training. MIC: 83 LANS in provinces; at HQ and 4 provs, LAN and IP phone connect all depts. MOFA: Data Center launched; Danang: 95 MAN connections, 32 internet stations, 329 public Wifi access points. GSO: VPN


Achieved - 100%: Also achieved: (i) Danang: public Wifi free of charge until business pricing model in place; (ii) GSO: All PSOs and 3 ITCs connected via VPN using MPLS and public internet connection.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 :

Original Intermediate Outcome Indicators (from PAD): Subproject 1: ICT Leadership and MPT Modernization: Workflows and reporting lines streamlined through hardware upgrades and process automation

Value (quantitative or Qualitative)

Workflows are uncoordinated and fragmented due to lack of computer systems

100% of identified workflows and reporting lines automated

na


Not achieved: Indicator is shown in PAD Results Framework but not ARM. Baseline and targets relating to "workflows and reporting lines" are associated in ARM with IOI 2 - possibly an input error at appraisal. No data available.

Indicator 2 : MPT and DPT architecture in place, including flagship portal allowing interaction and transactions between MPT and government, business and citizens.


Workflows are uncoordinated and fragmented due to lack of integrated computer systems

100% of identified workflows and reporting lines automated resulting in increased standardization, improved transparency of business processes.

na


Not achieved: Baseline and target as presented in PAD ARM and other documents seems more consistent with IOI 1, and may be an error. No results data. Indicator revised by 2010 Restructuring. See IOI 28 below.

Indicator 3 : At least 7 appropriate services online by 2010 through the flagship portal


Lengthy transaction times and high costs associated with many government services

7 e-services launched providing services to government, businesses and citizens

e-Services in process of being selected (radio spectrum, publications, standards for integrated communications equipment).

Date achieved 08/26/2006 06/30/2011 01/28/2010 Comments (incl. %

Negligible achievement: 2010 Restructuring revised this indicator. See IOI 30 below. Progress up to 2010 was incipient: selection of appropriate e-services

achievement) was initiated.

Indicator 4 : Reduced transaction costs (no. days for response turnaround) on average for 7 e-services offered through Flagship Portal


Lengthy transaction times and high costs associated with many government services

Evidence of significantly lower transaction costs for targeted government services (G2C and G2B)

e-Services in process of being selected (radio spectrum, publications, standards for integrated communications equipment)

Date achieved 08/26/2006 06/30/2011 01/28/2010


Negligible achievement: Expected outputs for IOIs 3 and 4 were dependent on selection of e-services to: (i) establish them online; (ii) assess resulting reduction in transaction costs (time). Indicator not in PAD ARM. Broad scope, difficult to measure.

Indicator 5 : IOs and CIOs from various government agencies and organizations trained in ICT issues that are relevant to their position and responsibilities (e.g., technical, legal, knowledge-sharing)


Lack of IO and CIO awareness and knowledge of how ICT can help improve administrative processes and govt services

400 IOs and CIOs trained in relevant ICT areas; engage in CIO Forum to update ICT Roadmap

na


Not achieved: Target was reduced by 2010 Restructuring to 200 CIOs/IOs trained. However, results under revised target not measured/unknown. 2010 Restructuring revised target again for second stage of project. See IOI 38 below.

Indicator 6 : A CIO Forum established to engage CIOs and MPT staff in the next IT Master Plan (2006-2011) and general ICT Roadmap


Zero CIO Forum; ICT Roadmap na


Not achieved: This IOI is included in PAD Results Framework but not ARM, and appears to be split from target for IOI 5 above. No results measured.

Indicator 7 : MPT officials trained in ICT policy and regulatory issues and are knowledgeable of best practices in the sector


MPT staff are ICT literate; mainly DPT staff untrained

50% MPT staff trained; increasing ICT capacity in respective government departments

50 MIC staff trained

Date achieved 08/26/2006 06/30/2011 01/28/2010 Comments Achieved 100%: IOI target was changed pre-Restructuring to reflect institutional

(incl. % achievement)

transition from MPT to MIC and the target was set at 50 MIC staff trained.

Indicator 8 : Selected DPT staff trained in ICT proficiency and usage Value (quantitative or Qualitative)

DPT staff mostly untrained in ICT

159 DPT staff trained na


Not achieved: This IOI is shown combined with IOI 7 in the original and the 2010 Restructured ARM. No results available/measured.

Indicator 9 : Improved security around e-services by establishing CERT, CAs and public key infrastructure that are inter-operable and in line with international standards


Vietnamese telecom policies do not yet meet international standards

Assistance with national CERT, CA and public key infrastructure (PKI) legislation; study of local telecom policies

Draft bidding documents prepared


Negligible achievement: IOI targets were revised in 2010. See IOI 29 below.

Indicator 10 : Improved Monitoring and Evaluation of the improved ICT strategy


No system supporting Govt to monitor and evaluate impact of its national ICT stategy on socio-economic development

Evaluation system and annual data collection to support MPT to assess impact of ICT on socio-economic development

na


Not achieved: Dropped by the 2010 Restructuring

Indicator 11 : E-government framework and IT standards in place to guide nationwide e-government applications and services


No guidelines or standards to develop e-government applications. Lack of IT standards nation-wide

e-Govt framework and IT standards in place to guide PIUs to implement e-Govt applications and services

Guidelines for e-Govt Architecture for project cities and agencies issued in Jan 2009. Common technical standards for inter-operability framework issued in March 2010

Date achieved 08/26/2006 06/30/2011 01/28/2010


Achieved 100%: IOI target was clarified in 2010. See IOI 36 below.

Indicator 12 :

Subproject 2: GSO Modernization GSO IT architecture upgrade in place, database integration and inter-operability completed


Low quality accessability and effectiveness in statistics service delivery to government and public

Upgrade IT architecture enabling GSO to automate admin. processes; 100% of identified workflows and reporting lines automated

na


Not achieved: IOI target was not achieved under the first stage of the project but was clarified in 2010 and achieved under second stage. See IOI 45 below.

Indicator 13 : GSO and 50% of the provincial statistical offices connected through Local Area Network (LAN), streamlining data dissemination


Many PSOs not connected to GSO and data exchange occurs only when there is a specific request for data

GSO and 50% of provincial statistical offices (PSO) connected, improving inter-agency data transfer

40% (25/63) of PSOs connected to GSO through ADSL using government funds (not IDA credit)


Partially achieved: IOI target was revised in 2010. See IOI 46 below.

Indicator 14 : Shorter data collection and dissemination cycles


Data disseminated in hard copy format and limited statistical information provided online

GSO portal and intranet launched enabling streamlined electronic data management and shortened dissemination times

Limited data in 4 policy areas available to public via Level One website (informational); 10 days for response to data requests from govt agencies; 7 days for data from PSOs


Partially achieved: Baseline and target not quantified so results not comparable. GSO Completion Report IOI was revised in 2010 but not included in the 2010 revised Results Framework. See IOI 48 below.

Indicator 15 : Enhanced efficiency of data flow and quality of social, economic and poverty-related data


Similar baseline to IOI 14 above

Target associated with target for IOI 14 above

See results IOI 48 below


Partially achieved

Indicator 16 : Online access to data collection methods, some data analysis and some raw data


Data disseminated in hard copy format; limited statistical information provided online

GSO Portal and Intranet launched as National Statistics hub for citizens, businesses and govt under free/fee-based arrangements

Limited data in 4 policy areas available to public via one website


Partially achieved: This indicator listed individually in Results Framework but combined with/linked to IOIs 14 and 15 in ARM. Revised in 2010. See IOIs 48 and 49 below.

Indicator 17 : Data demands of government agencies, research institutions and other data users met in a shorter period of time.


Data disseminated in hard copy format; limited statistical information provided online

GSO Portal and Intranet launched enabling steamlined electronic data management

Limited data in 4 policy areas available to public via website


Partially achieved: Thsi indicator is listed individually in Resultsisramework but combined with/linked to IOIs 14-16 above in ARM, hence duplication of baseline, targets (partial) and results information.

Indicator 18 :

Subproject 3: E-Applications in Hanoi City-level government IT architecture in place, including front- and back-end solutions enabling online transactions


No IT architecture to connect the various departments

At 5 key departments and all districts, 100% of workflows and reporting lines automated, increasing standardization and improving transparency of

na

admin. processes Date achieved 08/26/2011 06/30/2011 01/28/2010 Comments (incl. % achievement)

Not achieved: Target revised by the 2010 Restructuring. See IOI 51 below.

Indicator 19 : Selected e-applications online for both G2G and G2B and increased user satisfaction


Low satsifaction among users with respect to many government services

Evidence of significantly lower transaction costs for targeted govt services (G2C and G2B) eg, land registry and population management applications

na


Not achieved: Dropped by the 2010 Restructuring

Indicator 20 : Reduced transaction time and cost through modernization of government offices and providing online services


Low satisfaction among users with respect to many existing government services

Min. 3 additional services available online to public and usage encouraged; evidence of increased user satsifaction and visits to City Portal

na


Not achieved: This IOI appears in PAD Results Framework but in ARM was disaggregated into a series of loosely-connected indicators with over-lapping targets. Dropped by the 2010 Restructuring.

Indicator 21 : Increased ICT use by SMEs


Only 20% of SMEs use ICT, and only 5% use it for their business

15% of SMEs use ICT in business; SMEs improve productivity and competitiveness through ICT use

na


Not achieved (in Hanoi): Indicator difficult to measure, target too broad and attribution to project difficult. Dropped by 2010 Restructuring.

Indicator 22 : Enhanced competitiveness of SMEs through training and IT services provided at

the transaction center


Same as IOI 21 above

15% of SMEs use ICT in their businesses; SMEs improve competitiveness and productivity through ICT

na


Not achieved: Baseline and target identical to IOI 21. Indicator too broad and difficult to measure. Dropped by the 2010 Restructuring.

Indicator 23 : Government officials and ICT staff in the city government trained in e-government and IT issues


Lack of awareness and knowledge of how ICT can play a role in improving administrative processes and government services

50% of IT staff and 150 city officials trained. Training increases ICT capacity in respective govt departments

na


Not achieved: Indicator revised and clarified by the 2010 Restructuring. See IOI 53 below.

Indicator 24 :

Subproject 4: Danang City-level government IT architecture in place, including front- and back-end solutions enabling online transactions


Current workflows are uncoordinated and fragmented due to lack of integrated computer systems. No IT architecture to connect the various departments

100% of identified workflows and reporting lines automated, improving transparency and increasing standardization

Danang e-Govt Enterprise Architecture in place. All identified workflows and reporting lines automated.


Achieved 100%: Front- and back-end solutions applied and transactions now online.



Low satisfaction among users of many existing government services

Evidence of increased user satisfaction and visits to City Portal after content update and quality

Draft bidding documents ready for e-Procurement website

check. E-Procurement website launched, linked to City Portal.


Partially achieved: Indicator ambitious and revised by the 2010 Restructuring. See IOI 59 below. Indicator dropped by the 2013 Restructuring.



Low satisfaction among users of many existing govt services

Evidence of significantly lower transaction costs (time) for targeted govt services (online business registration and licensing); and, min. 3 additional services online for public

Draft bidding documents prepared and ready


Negligible achievement: See IOIs 59-61 below

Indicator 27 : Increased ICT use by SMEs


Nationally, only 30% of SMEs use ICT and only 5% for their business operations

15% of SMEs use ICT for their business

SMEs well-informed about existing services at Transaction Center. Localized survey shows 33% of SMEs using ICT and 18% for their business.

Date achieved 08/26/2006 06/30/2011 01/28/2010


Exceeded - 120% of target: The nature and function of the Danang Transaction Center was adjusted after 2010 moving from SME focus to more general Public Service Center for citizens/SMEs/Officials. For SMEs, focus shifted to ICT awareness-building

Indicator 28 : Enhanced competitiveness of SMEs through training and IT services provided at the Transaction Center


Only 20% of SMEs use ICT and only 5% for their business

15% of SMEs use ICT in their business

SMEs well-informed about ICT services at Transaction Center. 33% of SMEs use ICT and 18% for

their business Date achieved 08/26/2006 06/30/2011 01/28/2010 Comments (incl. % achievement)

Negligible achievement: Baseline and target are identical to IOI 27 and do not capture the indicator itself (enhanced competitiveness) which was broad and difficult to measure.

Indicator 29 : Government officials and ICT staff in the city government trained in e-government and IT issues


City officials have limited awareness of ICT use and management at national scale

50% of IT staff and 150 city officials trained; training increases ICT capacity in govt departments

27 leaders and officials received CIO training


Partially achieved: Result poorly-measured and not linked to the target. Revised by the 2010 Restructuring. See IO 58 below.

Indicator 30 :

Subproject 5: Ho Chi Minh City (HCMC) City level IT architecture in place, including front- and back-end solutions enabling online transactions.


Current workflows are uncoordinated and fragmented due to lack of integrated computer systems

100% of identified workflows and reporting lines automated at city level

na


Not achieved: Restructuring in 2010 dropped this indicator due to HCMC having self-financed most activities (zero disbursement of IDA credit) and the HCMC Subproject itself.



No announcements of procurement bids or contracts awarded, for public viewing

e-Procurement website launched and linked to City Portal

na


Not achieved: 2010 Restructuring dropped this indicator (and the HCMC Subproject)



Low satisfaction among users of many government services

Evidence of significantly lower transaction costs for targeted government services (G2C and G2B), eg, driver's

na

license, vehicle registration; and min. 3 additional services available online to public


Not achieved: This indicator was dropped by the 2010 Restructuring

Indicator 33 : Increased ICT use by SMEs Value (quantitative or Qualitative)

Nationally, only 20% of SMEs use ICT and only 5% for their business

Not defined na


Not achieved: This indicator is not included in the PAD ARM and no targets were set. Indicator dropped by the 2010 Restructuring.

Indicator 34 : Enhanced competitiveness of SMEs through training and IT services provided by Transaction Center


Zero Not defined na


Not achieved: Ambitious, difficult to measure indicator, not included in PAD ARM and no targets set. Dropped by 2010 Restructuring.

Indicator 35 : Government officials and ICT staff in city government trained in e-government and IT issues


Zero Not defined na


Not achieved: Indicator dropped by the 2010 Restructuring, as was HCMC Subproject

Indicator 36 :

Post-Restructuring Period (01/28/2010 to 12/31/2013) Subproject 1: ICT Leadership and MIC (formerly MPT) Modernization E-Government framework and common IT standards issued for all government agencies


No guidelines and framework to develop e-government applications, and nationwide lack of IT standards

E-Govt framework in place to guide all provinces and Govt authorities to implement e-govt services; common technical and application

Adjusted IOI: Design and implementation of e-govt framework, common IT standards and ICT policies.

E-govt framework in place and TA delivered for this and IT application standards; integrated email system feasibility study; MIC Laws

standards for e-govt issued

Target: E-govt framework in place; common tech standards for e-govt issued; decrees and circulars issued by MIC to business, citizens and gov agencies

(postal and radio frequency); Information Security Law drafted; MIC decrees on IT and Info Security.


Achieved - 100%

Indicator 37 : ICT policy and implementation capacity through training programs (MIC)


Limited capacity to formulate policies and guide implementation on latest issues in ICT sector. Gap in policy instruments

Priority ICT policies and implementation guidelines formulated; also, ICT sector M and E framework

na


Not achieved: New indicator added by 2010 Restructuring then dropped by the restructuring of 2013. Too broad and its supporting activities not implemented.

Indicator 38 : Improved ICT awrareness and management capacity through training programs (MIC)


Zero Not defined

400 CIOs and IOs trained, of which 260 from Govt agencies, 140 from state enterprises

497 CIOs and IOs trained (277 from Ministries and 63 DICs, 150 from state-owned enterprises, 70 CIOs and IOs from SMEs)


Exceeded - 125%: Revised Results Framework of 2010 added this indicator without providing monitoring details in revised ARM; 2013 Restructuring assigned specific, measurable targets

Indicator 39 : Improved MOFA e-visa services under MIC's oversight


Taking 5 or more days to obtain a visa

Average processing time for visa application reduced by at least one day.

na


Dropped: This indicator was added by the 2010 Restructuring and dropped by 2013 Restructuring. Visa targets were combined with and reported in IOI 42 below.

Indicator 40 : Enhanced MIC and DICs' communications through ICT infrastructure upgrade


Workflows uncoordinated and fragmented due to lack of integrated computer systems. Limited access to ICT infrastructure.

Improved ICT infrastructure for MIC (conferencing facilities, portal infrastructure); and min. 5 DICs equipped with ICT infrastructure and Portal

Adjusted IOI: Completed ICT infrast. upgrade to enhance communications between MIC, DICs and MOFA. Target: Improved ICT infrastr. MIC and MOFA (conferencing facilities, portal infrastr., data center); min. 12 DICs with ICT infrastr., VC and LAN

MIC installed audio and video conferencing, smart building communic. between MIC and 4 agencies (MIC Regional Offices Danang and HCMC, telecom authority in Hanoi and MOFA). E-Gateway (MIC Portal) completed and operational.


Partially achieved

Indicator 41 : Core MIC services for quality and security assurance


Vietnamese telecom policies do not yet meet international standards

Upgrade of Root Certification Authority (CA) to perform advanced PKI services; systems to issue digital certificates to citizens and business for digital signature; Test Center established to evaluate IT products for e-govt.

Adjusted IOI: Quality and security assurance for ICT services in Vietnam through core MIC services. Target: Upgrade CA; systems for digital certificates and signature; Test Center established.

MIC: By end-2013, all systems (Root CA, Test Center) were 100% completed and end-users trained. Systems for digital certificates and signature successfully developed and transferred

Date achieved 01/28/2010 06/30/2013 12/31/2013 12/31/2013 Comments (incl. % Achieved 100%

achievement) Indicator 42 : Three licensing e-services offered to businesses and citizens available online


Lengthy transaction times and high formal and informal costs associated with many govt services

3 licensing services available online

Adjusted IOI: Time savings of new systems (3 licensing e-services and visa online system). Target: Transaction time for visa processing reduced (by min. visits per applicant).

3 licensing services launched in May 2013 including visa online system (95 visa offices worldwide and application process piloted and launched, time reduced to 2.5 days).


Achieved 100%

Indicator 43 : 200 CIOs and IOs trained


Lack of CIO and IO awareness and knowledge of how ICT can play a role in improving administrative processes and govt services

200 CIOs and IOs trained

Target retained by 2013 Restructuring

497 CIOs and IOs trained (277 from Ministries, 63 DICs, 220 from State-owned Enterprises


Exceeded: 249% This indicator is shown in the Results Framework and ARM. Overlaps with IOI 44

Indicator 44 : 200 MIC staff trained on policy formulation, capacity-building and implementation guidelines for IT and telecommunications sectors


Staff in MPT well-versed in ICT literacy, but DPT staff mainly untrained

200 MIC staff trained (as above)


Exactly as IOI 43 above


Exceeded - 249% This indicator is shown in both the Results Framework and ARM. Duplication error with IOI 43

Indicator 45 : Subproject 2: GSO Modernization


Low quality, accessability and effectiveness in statistics service delivery to govt and public

GSO architecture developed and 80% of departments included

Adjusted IOI: Design and implementation of GSO IT architecture. Target: GSO

GSO EA in place; all IT initiatives at GSO must follow standard; GSO EA artifacts captured in System Architect

EA in place; EA artifacts captured in repository tool; centralized and coordinated IT services; effective IT investment and priority-setting.

Software; all GSO investments follow EA; centralized and coordinated IT services via SSIC and SHS; IT priorities based on project activities


Achieved 100%

Indicator 46 : Improved data transfer through LAN connection between three GSO IT Centers and Provincial Statistical Offices (PSOs).


Many PSOs not connected to GSO and data exchange occurs only upon special request

63% of PSOs (40/63) and 3 IT Centers connected to GSO via VPN network solution, improving data transfer between Govt agencies

Adjusted IOI: IT infrastr. upgrade completed (improved data transfer via LAN between GSO, 3 IT Centers, PSOs). Target: Commun. between GSO and PSOs enhanced: 100% PSOs and 3 IT Centers connected to GSO via wide area VPN. See value achieved for other.

All PSOs and 3 ITCs connected with GSO HQ via VPN using MPLS and public internet; new email system for secure document exchanges; data transfer improved GSO and PSOs; network connectivity for GSO Enterprise Applications


Achieved 100%

Indicator 47 : Increased efficiency of data management and report generation through database integration (GSO)


Limited no. of standalone databases some of which contain redundant

Target expanded: Over 60 databases integrated via data

Adjusted IOI: Time saving of new data

3 of 9 sector databases developed (pop,

information. No data standardization or integration methodology. Monthly reports generated manually.

warehouse; data mining and management significantly improved via data extraction applications; and, automated report generation.

management and report generation via database integration. Target: Common SSIC system to support GSO surveys and data collection; 2 GSO depts. pilot SSIC; 6 depts move to SSIC; min. 30-50% reduction turnaround time.

SME and BDS); data standard. and integ. methodology developed; pop reports automatically generated; SSIC implemented at GSO HQ and 3 ITCs. End user training completed.


Partially achieved: No data available on savings in turnaround time.

Indicator 48 : Shortened survey data collection and processing


Previous census data collection and processing took 12 months (for data capture only)

Shorten data collection and processing to 12 months

Adjusted IOI: Time saving of survey data collection and processing. Target: Form Scanning Infrastr. to support large-scale census projects with 50% time improvement; 30-50% improvement in data entry for IT-enabled processing.

Data collection, processing for 2009 Census 8 months (33% reduction); Forms Scanning Infra. used for 2011 Census (50% better o/all processing time, 2 years to 1); SSIC package tested for user acceptance; SHS finalized; CPI survey time reduced 33%.


Achieved 100%

Indicator 49 : Improved transparency through wider access to data and shortened turnaround time for data requests through GSO Portal

Value (quantitative

Statisticians working on standalone PCs;

Shorter dissemination

Adjusted IOI: Number of

GSO: All GSO statistical data

or Qualitative) dissemination through hard copy; limited statistical information online

cycles from increased interaction through GSO Portal and intranet; and GSO Portal as National Statistics Hub for citizens, business and govt (free and fee-based)

survey databases available through GSO Portal. Target: Up to 60% of databases integrated through data warehouse and available on GSO Portal

domains integrated with GSO Portal and basic statistical data is provided to public free of charge. Fee-based system under consideration. 50% of GSO data now integrated in Stat. Data Warehouse.


Substantial achievement: 84% This indicator is not in the 2010 revised Results Framework but is shown in ARM and was formalized by the 2013 Restructuring.

Indicator 50 : Improved capacity of staff in performing streamlined and ICT-enabled GSO data processes


Lack of awareness and knowledge of how ICT can improve administrative processes and govt services.

670 GSO and PSO staff trained and/or accredited

Adjusted target: Number of staff trained to increase knowledge on ICT issues relevant to further modernization of GSO functions

122 CIOs trained and certified; 83 GSO staff and IT experts from 3 centers trained in Network Admin; 395 GSO and 63 PSO staff and statisticians trained in MS Office; 27 IT experts in 3 IT Centers trained in SQL, Share Point, New Windows Server. Total 690


Exceeded - 103%

Indicator 51 : Subproject 3: E-Applications in Hanoi City-level IT architecture in place


No IT architecture to connect city departments

57 Departments, Districts aligned to IT architecture

Revised target: Design EA Master Plan for 3 levels of Govt; and detailed EAs for Departments, Wards and Communes;

EA Master Plan designed for 3 levels of Govt (Dept, Ward and Commune); City-level IT architecture designed; Aligned architecture designed for 57 Departments

develop aligned IT architecture for 57 Departments.


Achieved - 100% Continued by Restructuring of 2013 but with more ambitious targets. Also, the 2013 Restructuring split Hanoi and Danang into 2 components.

Indicator 52 : Better access to government services through the single City Portal


No one-stop-shop available for city government services, which are provided and processes through various departments and locations

One-stop-shop Portal infrastructure in place for access to e-services via many departments and districts

Adjusted IOI: No. of public services available through City Portal. Target: One-stop shop software for govt agencies integrated in Data Center; integration of 57 portals and websites of Departments.

Data Center for City Portal completed and software installed; 18 dept portals and 16 applics. integrated with Hanoi City Data Center; One-stop software for Govt agencies integrated in Data center; 57 dept. portals and websites integrated.


Achieved - 100% Access to Govt information now easier, more reliable. Also,public services integrated via Internet (20 at 2nd level, 8 at 3rd level, 2 at 4th level).

Indicator 53 : Improved ICT awareness and management capacity through training programs (Hanoi)


Lack of information officer and CIO awareness and knowledge of how ICT can play a role in improving administrative processes and govt services

57 leaders receive CIO training; 500 SME managers trained in application of IT to their enterprises

Adjusted IOI: No. of city officials (Hanoi only) take CIO training courses. Target: 340 leaders and key officials receive CIO training

30 leaders trained in Singapore (2011)


Negligible achievement (with IDA funds) of 9%: Training for SMEs was originally contemplated using IDA credit but was implemented with local funds. Over 100 SMEs trained in CIO.

Indicator 54 : Improved awareness and use of ICT among private enterprises including SMEs Value Only 20% of SMEs use Not defined na

(quantitative or Qualitative)

ICT, and only 5% for their business (nationally)


Results unknown, indicator dropped: This indicator was included in the 2010 revised Results Framework both as an Outcome Indicator and - in similar form - an IOI but the ARM provided no targets for latter and it was dropped.

Indicator 55 : Reduction of time required for one transaction of selected government services through City Portal compared to traditional way


Only 20% of SMEs use ICT and only 5% for their business

Transaction time through one-gate system reduced by 50% compared to traditional way (eg, 5 days for business registration)


Transaction time through one-gate system reduced by 50% compared to traditional mode.


Achieved - 100% This IOI was added in 2013 and monitored by ISRs but was not included in the 2013 adjusted Results Framework

Indicator 56 : Subproject 4: Danang City-level Government and IT architecture in place


No IT architecture to connect various departments

92 authority units are aligned with IT architecture

Adjusted IOI: Design and implement. of city-level IT architecture. Target: 91 authority units aligned; e-Govt EA framework and model for Danang approved by PPC; report on strategy, roadmap, arch. standard and prototype of Danang City prep. and approved.

Danang e-Govt EA in place; 27 Depts, 8 Districts, 56 Wards, Communes have LAN and Internet connection; MAN operational providing Fiber Optic Cable connection between 97 state agencies; 30 addit. depts and agencies also equipped. Total 121 units achieved.


Exceeded - 133% Indicators from PAD were sustained by 2010 Restructuring then clarified and adjusted in 2013.

Indicator 57 : Better access to Government services through the City Portal


City government services are provided and processed through various departments and locations

80% of the 18 identified services provided through City Portal are active and used by citizens

Adjusted IOI: Better access to govt services via different channels. Target: Public Wifi covers public areas of city; City Portal with links to other depts, districts etc.; Public Service Info. Portal and Public Contact Center launched. 10 tourist kiosks

Contact Center in place (881 888) providing info on public services and socioeconomic issues; MAN connecting 97 depts., agencies, communes, wards; 32 Internet Stations in communes, wards; 10 inform. kiosks for tourists; 329 free public Wifi access points.


Achieved - 100% Indicator revised by the 2010 Restructuring then adjusted in 2013. Danang has commissioned additional Wifi access points and Wifi is available free of charge until a business pricing model is put in place.

Indicator 58 : Improved ICT awareness andmanagement of city officials through training programs


City officials have limited awareness of ICT use and management at national scale

At least 150 leaders receive CIO training and IT training courses

Adjusted IOI: No. of city officials take training and IT training courses. target: Min. 27 leaders receive CIO training; min. 30 leaders trained in ICT Strategic Management; min. 400 officials trained in IT

27 leaders, officials received CIO training; 8 IT technicians did IT courses; 102 participated in foreign study tours; 141 officials did IT training (CCNA, MCSA, Oracle DB, Database Networking); 400 officials did end-user training.


Achieved - 100% A similar indicator was included in PAD and revised by 2010 Restructuring (see above). Included in the adjusted 2013 Results Framework.

Indicator 59 : Improved transparency and reduced transaction costs (i.e., time) for public procurement

Value Few announcements of e-Procurement na

(quantitative or Qualitative)

procurement bids or contracts awarded available for public view.

system launched and integrated into City Portal; 60% users satisfied with e-Procurement system; 25% of procurement bids processed through this system


Indicator dropped: Indicator was created at appraisal, continued by the 2010 Restructuring but dropped in 2013 because financed by other sources (non-project).

Indicator 60 : Improved transparency and reduced transaction costs (time) for business registration and investment licenses


Business registration and license processing done offline

e-Business Registration system launched and integrated into City Portal; 70% of users satisfied with this system.

Adjusted target: e-Business Registration system launched and integrated into City Portal; 70% of users satisfied; 80% of investors applying for their investment licenses online.

Business Registration System and Investment Project Management System completed development phase and launched in January 2014. No user surveys conducted yet.


Partially achieved: Due to the January 2014 launch, there was inadequate time to conduct user satisfaction survey or to assess degree of online usage for investment licensing.

Indicator 61 : Improved transparency and reduced transaction costs (time) for vehicle registration and driver's license.


Driver's license processing done offline

Vehicle registration system launched and integrated into City Portal; Driver's License system launched and integrated into City Portal; 70% of users satisfied with these new systems

Adjusted IOI: Improved transparency and reduced transaction costs (time) for driver's license. Target: Driver's License system launched and

Driver's license system developed and completed; deployment for commissioning and user acceptance test conducted. System fully operational.

integrated to City Portal; 70% users satisfied with driver's license system.


Partially achieved: Indicator added by the 2010 Restructuring and adjusted by the 2013 Restructuring. Inadequate time since launch to assess user satisfaction with online licensing system.

Indicator 62 : Improved awareness and use of ICT among private enterprises including SMEs


Only 20% of SMEs use ICT and 5% do so for their business.

1000 participants from SMEs trained in e-commerce, e-procurement, and applying e-services; 60% of SMEs use ICT in business; 40% of SMEs use ICT Transaction Center

Adjusted IOI: No. participants from SMEs take ICT training to enhance ICT skills for business in e-commerce, e-procurement and e-services. Target: 1000 SME participants trained in those 3 services; all SMEs supported via Contact Center. See Comments.

SMEs now well-informed of e-services via Contact Center; e-Learning Portal launched and 400 ICT courses available incl. 15 modules for SMEs; 73 software developers trained

Date achieved 01/28/2010 06/30/2013 12/31/2013 12/31/2013


Exceeded: Revised target also included 400 courses in ICT soft-skills on e-Learning Portal and min. 75 SME software developers did software engineering training. Other results: 50 SMEs did e-Govt Platform courses; basic IT knowledge progs on TV weekly.

Indicator 63 : New indicators created by 2013 Restructuring (Danang only): Improved ICT infrastructure of the City Government


No significant ICT infrastructure developed by the City Government

Danang MAN with fiber optic connection to min. 90 Depts., agencies; Data Center; public Wifi; Training and Applied Research

Completed: Metro Area Network; public Wifi with 165 public free access points; additional Wifi points completed; Data Center; e-

Center; Danang e-Learning Portal; e-Govt Framework; all Depts, wards. agencies, communes with end-user IT equipment.

Learning Portal operational.


Achieved - 100% Due to overall strong performance, Danang received over US$10.0 m. in additional IDA funding through reallocation and was able to achieve beyond expectations.

Indicator 64 : Improved transparency and reduced transaction costs (time) for Construction Permits


Zero (not surveyed)

Construction Permit System launched and integrated into City Portal, 30% of applications online; 70% of users satisfied with the system

System completed development and commissioning phases, launched January 2014


Partially achieved: Insufficient time since launch to conduct user survey.

Indicator 65 : Enhance efficiency in Public Transport Management


Zero

Public Transport Management launched; and, 100% of buses are managed centrally and based on GPS and GIS Map.

System was commissioned and 100% of buses are equipped with GIS, managed and monitored centrally by DOT, and based on GIS Map.


Achieved - 100%

Indicator 66 : Enhance efficiency of Water Resource Quality Management


Zero

Water Monitoring System launched; raw water monitored and controlled 24/7 on criteria before treatment and processing stage

Water Monitoring System operational; raw water monitored and controlled 24/7 on criteria (color, smell, taste, muddiness, residual

chlorine, pH, and Ammonia, Fluoride and Chloride content, and coliform) before treatment and processing.


Achieved - 100% Social and health benefits were not studied but expected to be substantial based on global experience.

Indicator 67 : More efficient government administration


Zero

7 districts, 56 Wards use e-District, e-Wards system; 100% use one-stop shop at District, Ward level; Business Process Database for public admin processes; MIS launched; min. 90 Depts and agencies establish a VC Committee.

90 Depts, agencies got video phones; VC service established; Business Process database developed and operational; e-Wards and e-Districts commissioned; 100% one-stop shops connected to MAN and transferring data to Dept Home Affairs.


Achieved - 100%

Indicator 68 : Enhance and develop the ICT Human Resources in Danang City


Zero

Danang ICT HR Devt Master Plan approved; e-Learning portal launched; 400 soft-skills courses in e-Learning Portal; min 75 software developers (pub. and private) get Software Eng. training; min. 27 leaders get CIO training; min. 400 officials trained in IT

ICT HR Devt Master Plan approved by PPC; e-Learning Portal launched, 400 courses; 75 software devel. trained in Software Engineering, Proj. Manag. and Testing; 27 leaders did CIO training; 25 engineers, Integ. Circuit Design; 400 officials trained in IT.

Date achieved 06/30/2013 12/31/2013 12/31/2013


Achieved - 100%

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 10/30/2006 Satisfactory Moderately Satisfactory 0.00 2 07/21/2007 Satisfactory Satisfactory 3.83 3 02/09/2008 Moderately Satisfactory Moderately Satisfactory 3.83 4 09/10/2008 Moderately Satisfactory Moderately Satisfactory 4.50

5 06/11/2009 Moderately Unsatisfactory

Moderately Unsatisfactory 5.40



7 05/02/2011 Moderately Satisfactory Moderately Satisfactory 13.57







11 09/28/2013 Moderately Unsatisfactory Moderately Satisfactory 55.99

12 01/13/2014 Moderately Unsatisfactory Moderately Satisfactory 63.16

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

01/28/2010 Y MU MU 7.35

Level One Restructuring revised the PDO and Outcome Indicators to focus more narrowly on project activities funded by IDA. Hanoi Subproject dropped and components changed.

06/29/2011 MS MS 15.54

Level two Restructuring extended closing date by two years to allow Level One changes to take hold and

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made DO IP

improve progress.

06/20/2013 MU MU 45.18

Level two Restructuring extended closing date 6 months, adjusted Results Framework, reallocated US$10.6 m and cancelled US$10.3 m. Purpose was to allow time to complete all procurement and achieve PDO.

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Unsatisfactory Against Formally Revised PDO/Targets Moderately Unsatisfactory Overall (weighted) rating Moderately Unsatisfactory

I. Disbursement Profile

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1.1.1 Country context and national commitment: Vietnam, like many developing countries, recognizes the value and potential of information and communication technology (ICT) to enhance competitiveness, facilitate economic diversification and increase productivity, and to improve institutional capacity and efficiency especially in government. At the time of appraisal in 2005, the Government of Vietnam (GOV) had repeatedly identified ICT as a top priority, issued high-level decisions and formulated strategies, master plans and decrees to that effect. These included the National ICT Development Strategy 2006-2010, the e-Government Development Master Plan to 2010, the State Administration Modernization and Computerization Program and the Prime Minister’s Decision 169 on administrative simplification. The Ministry of Posts and Telematics (MPT) was to lead implementation of the National ICT Strategy and E-Government Master Plan under the guidance of a National Information Technology Steering Committee. Key targets of the Master Plan included: extensive diffusion of ICT throughout the national economy and society to increase ICT’s share of GDP; establishing a national information and communications network; and, comprehensive ICT skills development. 1.1.2 Important efforts to upgrade ICT infrastructure flowed directly from this enabling framework including: introducing information systems into selected government departments and piloting online services; providing incentives to the ICT industry; and developing ICT skills and improving awareness of ICT in government and to some extent, the private sector. Vietnamese authorities expected that ICT development would be an instrument to facilitate service delivery and a potential source of growth. References to ICT development featured prominently in GOV’s Socio-economic Development Plan (2006-2010). Government had also committed under international trade agreements (including the Vietnam/US bilateral trade agreement) to open the telecommunications sector to outside investment and to increase competition. Substantial progress had been already made in moving towards international norms and best practice. 1.1.3 Constraints and challenges: Government nevertheless faced tough constraints to e-readiness at the time of project appraisal: the high cost of and uneven access to communications infrastructure; a shortage of skilled human resources including in ICT-related disciplines; and limited diffusion of ICT within government, the business sector and citizenry. Achieving the new Master Plan’s targets was challenged further by the need to strengthen the management, coordination, coherence and result-orientation of on-going programs. This included establishing realistic indicators and benchmarks and developing successful demonstration models or applications which could be scaled up over time. 1.1.4 Rationale for Bank involvement: GOV sought Bank support for ICT development for the following reasons: (i) the Bank’s active, multi-country engagement in telecommunications reform, e-government and ICT development throughout the developing world including East Asia and the Pacific region, and comparative advantage in promoting best practice experiences in ICT projects and policy formulation; (ii) established dialogue with GOV on telecommunications and ICT policy issues and knowledge of main stakeholders; (iii) ability to coordinate development partners working in ICT to prevent overlap and duplication; (iv) understanding of the ways ICT could be integrated in other sectors, thus contributing more broadly to growth and poverty reduction; and, (v) capacity to provide long-term, large-scale financing.

1

1.1.5 Contribution to higher-level objectives: The Project supported two key pillars of the Bank’s Country Assistance Strategy (CAS) FY03-FY06: (i) high growth through transition to a market economy, by developing policies for the ICT sector to generate employment, exports and domestic economic efficiency in Vietnam, and through training to build a computer and internet-literate small and medium enterprise (SME) sector more able to deliver growth and engage in international trade and competition; and (ii) adoption of a modern public administration, legal and governance system. Promoting efficiency and transparency of government operations via portals, online applications for business and land registration, e-procurement, and data collection and dissemination would improve government decision-making processes, reduce transaction costs, improve quality and standardization of government documents and limit the scope for corruption. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 1.2.1 The PDO as stated in the PAD and Development Credit Agreement sought: (i) to accelerate the use of ICT and support institutional development in order to increase the efficiency and effectiveness of public administration, increase transparency and availability of information, and improve public services delivery to stakeholders; and (ii) to foster private sector development in particular by facilitating the increased adoption of ICT by small and medium enterprises. 1.2.2 The table shows the alignment between the original PDO and Results Indicators:

PDO Project Outcome Indicators 1. Accelerate the use of ICT and support institutional development to increase transparency and availability of information, and improve public services delivery to stakeholders.

-Increase of 2% in share of ICT sector in GDP -Increase in ICT capacity and knowledge in respective government departments -Increase in user satisfaction for e-government services and visits to City-portal. -Increase in standardization and improvement in transparency of administrative processes -Significantly lowered transaction costs for targeted government services (G2C and G2B)

2. Foster private sector development in particular by facilitating the increased adoption of ICT by SMEs

-15% use ICT for business operations in 2010 and improve productivity -Domestic IT service sector grows by an average 30% per year

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 1.3.1 A Board-approved Level One restructuring in January 2010 revised the PDO and Outcome Indicators to focus more narrowly on project activities funded by IDA, assisting the Recipient: (i) to support the MIC’s institutional development and technical and regulatory capacity for ICT sector oversight and to increase efficiency and transparency of public service delivery in MIC, GSO, MOFA,1 Hanoi and Danang; and (ii) foster private sector development by increasing access to business-related government services and raising awareness and usage of ICT in Hanoi and Danang. 1.3.2 The table below shows alignment between the restructured PDO and Results Indicators as at January 2010. A second (Level Two) Restructuring in June 2013 adjusted the Results Framework including by adding two new Results Indicators. For convenience, these are shown in the table below and discussed in Section 2.3 and Annex 2.

1 Respectively: Ministry of Information and Communication (MIC); General Statistics Office (GSO); Ministry of Financial Affairs (MOFA).

2

Restructured PDO Revised Indicators

1. Support MIC’s institutional development and technical and regulatory capacity for ICT sector oversight and to increase efficiency and transparency of public service delivery in MIC, GSO, MOFA, Hanoi and Danang.

-Improved formulation and implementation of ICT policies and common standards -Reduced transaction costs, increased transparency of and users’ satisfaction with selected government services through e-applications** Added by 2013 Restructuring: -Number of e-services and socio-economic information available through e-Gateway (MIC), portal sites (GSO, Hanoi, Danang) -Upgrading ICT infrastructure to support e-government technical capacity in MIC, GSO, MOFA, Hanoi and Danang

2. Foster private sector development by increasing access to business-related government services and raising awareness and usage of ICT in Hanoi and Danang.

-G2B services made available online. -More private firms in selected cities use ICT

**Also applicable to PDO #2 1.3.3 Rationale for changes: The partially adaptive, partially corrective Level One restructuring was requested by the State Bank of Vietnam for the following reasons:2

• HCMC and Hanoi Subprojects had adequate funds from own sources to conduct activities similar to the Project. In the case of HCMC, the Project Implementation Unit (PIU) had tried hard to absorb the city’s US$3-4 m. budget allocation but could not use the additional IDA resources. HCMC Subproject disbursement was virtually zero.

• Demand for IDA funds in GSO and Danang to finance their ICT development plans was increasing. Both PIUs were strong performers and reallocation of funds to their Subprojects was thought likely to ensure timely completion of project activities.

• Implementation capacity was weak in the Hanoi and MIC PIUs (and PCU), hindering progress in the first half of the Project. See also Section 2.2.

1.4 Main Beneficiaries: 1.4.1 Project beneficiaries were not described explicitly in the PAD but by inference were: government institutions at both the central and decentralized levels, and especially those directly involved in project implementation; small and medium-sized business enterprises; and, the general public/citizens through improved access to e-services. Project restructuring in 2010 did not change the identity of expected beneficiaries but did expand the estimated numbers likely to benefit from increased public access to e-services (see Section 2.2).

1.5 Original Components (as approved) 1.5.1 The Project comprised five Subprojects in three areas: (i) ICT leadership and modernization of the MPT; (ii) modernization of the GSO; and, (iii) e-applications for the municipalities of Hanoi, Danang, and Ho Chi Minh City. The five subprojects covered a total 14 “components” or sets of activities. See Annex 2. Subproject 1: ICT Leadership and MPT (later MIC) Modernization (total estimated cost US$21.7 m. of which IDA US$20.1 m., 93%) financed capacity-building for MPT for the development, implementation and oversight of government’s ICT strategy and policies; formulation of telecommunications and ICT policy; and, improving MPT’s operating efficiency and effectiveness. The Subproject had six components/activity sets: (i) development of e-

2 Bank management was ambivalent about pursuing a first order restructuring based on concerns about the time and effort needed for its approval, especially in Vietnam, potentially adding to existing implementation delays.

3

government capability in MPT; (ii) informational interaction and transactional portal design and activation; (iii) establishing a training program, and the design and implementation of curricula for the Chief Information Officers (CIO); (iv) capacity-building for ICT policy and regulatory development; (v) development of a legal, regulatory and institutional framework for IT and telecommunications; and (vi) development of a national IT architecture and inter-operability framework for e-government. See Annex 2. Subproject 2: GSO Modernization (total estimated cost US$13.77 m of which IDA 100%) supported implementation of the strategic plan for modernization and capacity-building of GSO. The quality, transparency and timeliness of socio-economic and poverty data/information for government agencies, research institutions and other data users were expected to improve, modernizing the foundation for policy, planning and decision-making. Systems would be upgraded and the timeliness, analytical quality and dissemination of data/statistical information would improve. The Subproject had three components/activity sets: (i) ICT infrastructure and process upgrade; (ii) data flow and enhanced statistical analysis through ICT; and (iii) development/expansion of ICT human resources and awareness. See Annex 2. Subprojects 3, 4 and 5: E-Government in Hanoi, Danang and Ho Chi Minh City (total estimated cost for Hanoi US$31.3 m/IDA US$28.06 m (89.6%); for Danang US$17.33 m./IDA US$16.42 m. (95%); and for Ho Chi Minh City US$10.61 m./IDA US$8.93 m. (84%)), sought to improve the ICT environment in these cities by providing the foundation for deployment of e-government in district offices and departments. E-government services, e-applications for business, and ICT skills would improve government accountability, effectiveness, transparency, public and business access to information, citizen and business interaction and public administrative reform efforts. SME competitiveness was expected to improve with spillover effects on trade opportunities, employment and income. The Subproject had six components/activity sets: (i) development of ICT strategy, policies and architecture and upgrade of ICT infrastructure, equipment and systems for these cities; (ii) assessment of business processes and development and implementation of workflow solutions; (iii) development and expansion of electronic gateways and city portals; (iv) implementation of e-applications in selected public services, including geographic information systems; (v) support for the uptake of ICT by SMEs through ICT transaction centers; and (vi) development of ICT human resources and awareness-raising. See Annex 2. Subproject 6: Project Coordination (total cost US$1.5 m. of which IDA 100%) financed a Project Coordination Unit (PCU) within MPT (later MIC) - which was a separate unit from the MIC PIU - for overall coordination of project monitoring and reporting, and knowledge-sharing across subprojects. Implementation would be decentralized; each Subproject would have its own PIU (in MIC’s case, as well as the PCU) and Special Account. See Annex 2.

1.6 Revised Components 1.6.1 The 2010 Restructuring revised several Subprojects/components (see Annex 2):

• Ho Chi Minh City (HCMC): Subproject was cancelled (see 1.3.3). • MIC: Activities added: Test Center for IT product evaluation; consultancies to prepare

plans for e-document interchange and integrated e-mail systems; and installation of an intelligent building system and integrated communications. The planned National Computer Emergency Readiness Team (CERT) was removed (government-funded).

• MOFA under MIC: Data Center and establishment of online visa services added.

4

• GSO: Planned Information Warehouse would add a data warehouse and information dissemination portal. New components added (US$6.7 m increase in credit amount by reallocation): equipment for data collection/processing for the 2009 Population Census; and, GIS application support mapping administration field database.

• Danang: Activities were expanded (US$10m increase in credit amount by reallocation): Data Center added to the Transaction Center, and Danang’s ICT infrastructure upgrade program expanded to include 30 departments, seven districts, and 56 communes and wards.

• Hanoi: IT infrastructure upgrade was removed, the number of e-services was reduced, and a Data Center was added.

1.7 Other significant changes 1.7.1 Restructuring and amendments to the DCA:

• Level Two restructuring of June 29, 2011 approved by senior Bank management extended the closing date by two years to June 30, 2013.

• Level Two restructuring of June 20, 2013, approved by Bank management on an exceptional basis (IP and DO ratings were then MU, and prior extensions totaled 24 months), extended the closing date six months to December 31, 2013; adjusted the Results Framework (see Section 2.3); reallocated US$10.6 m. and cancelled US$10.3 m. of the IDA Credit (and US$4.6 m. of counterpart funds) representing large procurement packages either already financed by government with its own funds or unlikely to be completed by the closing date. The justification was to allow time to: (i) complete major contracts signed by December 31, 2012; (ii) stabilize and consolidate the ICT systems already established; and (iii) support critical turnkey packages signed before December 31, 2012 to ensure that software applications and IT equipment functioned properly.3

• Development Credit Agreement (DCA) amended in 2008 to reflect an agreement during negotiations that IDA would finance the reasonable costs of incremental expenditures by the PCU in coordinating the project.4 US$80,000 was allocated for this function.

• DCA amended in February 2010 to substitute the Ministry of Information and Communication (MIC) as lead executing agency in place of the Ministry of Posts and Telematics (MPT), and Department of Information and Communication (DIC) for Department of Posts and Telematics (DPT).

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry 2.1.1 Preparation and background analysis: The PAD reflects a comprehensive effort to design an ICT in the mainstream of best practice. External specialist sources including academic

3 The rationale included: (i) experience with other ICT projects (difficult to launch and implement); (ii) Vietnam country context (slow project implementation); (iii) importance of ICT to the country’s transparency agenda which the Bank was fostering; (iv) progress on several complex ICT contracting processes and need for time to ensure that newly-installed technologies worked; (v) country fiscal constraints, i.e., unfinished works were unlikely to be finished by the client with own-funds; and (vi) emergence of important project results and impact. Critically, certain pending procurement packages were linked directly to achievement of the PDO. 4 The Bank Task Team believed that the O&M costs of the PIUs in Hanoi, Da Nang and Ho Chi Minh City, and at the GSO, should be borne by counterparts to ensure ownership of project activities and the longer-term financial sustainability of project activities. Incremental operating costs of the Project Coordination Unit however, were strictly project-related, would not have been incurred absent the project, and would not continue beyond closing.

5

were consulted. The internal review process was thorough, and wide-ranging peer input was sought. Concerns were expressed internally about the following: project scope and complexity; institutional arrangements, especially the cost and efficiency of using five separate implementation units; M&E and the quality and relevance of the Results Framework especially impact indicators; overall coherence of the proposed intervention and linkage between the five Sub-projects; inadequate connection between proposed sustainability factors and what was likely to work; and, weakness in the underlying analytical justification for key activities including SME involvement. Evidence suggests that while the project team understood these concerns, project design responded only partially and the internal clearance process acquiesced. 2.1.2 Lessons: Global experience with ICT operations showed that the primary drivers of success were strong country ownership and sustained commitment to implementation. Thus the consistency of project design with government’s National ICT Strategy (2006-10) and other high-level policy documents was equated with full ownership by government. Global lessons in organization, leadership models, personnel and procedures in setting up telecommunications and ICT regulatory bodies shaped design, while experiences balancing progressive change with internal resistance and political issues were used to create a realistic roadmap for policy change. For e-applications development, the Project heeded experiences in e-government projects in the developed and developing world, and World Bank/other case studies. The value of picking high payoff focus areas such as the e-registration of businesses and e-procurement was also studied and incorporated (but the latter was later dropped). Lessons called for balancing supply-driven features by demand promotion: ICT and e-government projects in many countries had been unduly supply-driven. Project design featured awareness-building and skills development to promote demand but in reality ICT infrastructure and e-services supply were the primary focus. Assessment of project design: 2.1.3 The sheer complexity of project design, combined with its equally complex implementation arrangements, is notable. Project activities were largely foundational, whereas the original PDO and results indicators anticipated measurable outcomes more appropriate to a second stage operation: qualitative, behavioral and cost benefits of established systems, strong and measurable demand/usage, and with significant ICT capacity developed and consolidated (see 1.2.2). These are impacts which even in advanced economies can take up to a decade to reap measurable benefits. In practice, under the country conditions including a new client in a challenging political economy and economic slowdown from 2010-2013, and regardless of GOV’s strong commitment to an ICT-based economy, the complex foundation of infrastructure, systems and services as well as M&E capacity, could not be established fast enough to achieve and measure its development impact on the behavior of targeted beneficiaries by project end. The learning curve was steep and the configuration of involved institutions unwieldy. The Project did not get the opportunity to address second stage activities to enhance or measure, the impacts of the infrastructure on behavior. 2.1.4 Several issues affected design. First, bidding documents should have been ready prior to Board approval as part of the primary feasibility studies and would likely have led to better disbursement flow in the initial years. In retrospect, it is also clear that of the 92 procurement packages, some were not essential. Second, this type of project is not primarily about hardware and software – which can readily be purchased off the shelf - but about the eco-system for the human aspects. Project design lacked a substantial focus on the human behavioral change element which proved critical to its progress and to what would be measurable by end-project. Third, a higher portion of the project budget – perhaps as high as 30% - needed to be allocated to change management/initiatives. This would include, as with the procurement packages, getting

6

Enterprise Architecture (EA) itself or EA studies in place/done before Board approval, using Project Preparation Facility (PPF) or Trust Funds to ensure that the foundational elements of the Project could support subsequent procurement. Finally, the issue of ICT - and especially Internet - penetration was not adequately addressed resulting in e-services not evolving readily into end-to-end transactions. This project required something akin to a one-laptop-per-child approach or a subsidized laptop purchase program to accelerate the impact on citizen uptake. 2.1.5 Implementation structure: At preparation, multiple Project Implementation Units (PIU) were justified on the basis of project scope and the responsibilities of relevant government agencies. The project used a decentralized approach; each Subproject had its own PIU, procurement plan and Special (Designated) Account. The Bank’s conversation was quite different with each unit. The PAD argued that similar arrangements had been adopted by other Bank projects with multiple, decentralized subprojects or components. The real constraint was the policy and legal framework for subprojects financed by GOV’s Official Development Assistance (ODA) which does not permit joint PIUs across implementing/supervising agencies.5 The PIUs operated as parallel “silos”; there was no built-in mechanism for fostering inter-PIU collaboration or synergy even though there were common components and similar objectives/outputs across Subprojects. Further, the PIUs were not constituted at appraisal and only three of the five were required to be up/running by effectiveness, which had implications for project ownership, understanding and readiness to launch. 2.1.6 The Subprojects were overloaded, each with multiple components/activities (see Annex 2) and IDA allocations ranging from US$10.0 m. to US$28.0 m., equivalent in practice to five separate projects. This was expected to implement smoothly in an untried, e-unready environment with ICT human resource deficits and varying levels of commitment to (or understanding of) the Project. As the Client was to bear the administrative costs, questions about duplication and inefficiency may have been discounted; from a Bank supervision perspective, similar questions were relevant. Selected cities covered the capital (to demonstrate e-government in the public and private sector), a commercially-developed city (Ho Chi Minh City) and one in a less-developed central region (Danang). The awareness that other donors were operating in the ICT “space” was reflected in plans for collaborative events on ICT/e-government with all key development partners in Vietnam. This was considered administratively manageable, would permit exchanges of information and avoid donor duplication of effort. In practice, it worked well. 2.1.7 Acknowledging the lack of clarity in individual institutional responsibilities – strong coordination being an essential feature of successful ICT operations – and complexity of the institutional and legal framework, the Bank Team sought coordination by linking the Project to the multi-agency National IT Steering Committee chaired by the Deputy Prime Minister, which would review project performance at semi-annual meetings. A Project Working Committee of PIU leadership was also to be established, meeting quarterly on project matters.

5 Each supervising agency must be responsible for its own “project” and form its own PIU for management. Bank staff in Vietnam has tried to rationalize this structure without success - it depends on the commitment of individual agencies. A modified approach could have worked where for example, a joint procurement team might have provided unified services for the three Subprojects based in Hanoi – MIC, GSO and Hanoi.

7

2.1.8 Participatory processes: The Bank project team worked with the involved public agencies and met with associations representing the private sector and with private actors in the ICT industry, incorporating their suggestions and priorities in project design. Private stakeholders suggested and endorsed the emphasis on streamlining government processes burdening their business operations. The PAD recommended that this dialogue – especially with SMEs - continue during project implementation and it did, but intermittently and depending on location. 2.1.9 Risk assessment: The analysis of risk was satisfactory but incomplete. Mitigation stressed training, awareness-raising and technical assistance. Risks also focused on concerns about commitment to and sustainability of ICT policies and systems. Procurement risk was correctly rated as High based on the readiness of bidding documents but the general lack of procurement expertise/human resources was barely mentioned. This was a procurement-dependent operation divided between five centers each with its own procurement plan and no familiarity whatsoever with the Bank’s procurement framework. The most acute risk evident after effectiveness was the limited capacity of implementing agencies (taking procurement processing into account) and lack of skilled IT professionals, i.e., e-readiness, alluded to at appraisal in general terms and rated Moderate. The 2010 Restructuring belatedly acknowledged human resource deficits and e-readiness as “new risks” for which mitigation measures were already activated, including more intensive/frequent Bank supervision. Risks associated with SME/private sector uptake, citizen ICT demand/literacy and design complexity were not mentioned. 2.1.10 Project financing: Financing arrangements were unusual. Six Special accounts were opened for the PCU and five PIUs. The Ministry of Finance (MOF) proposed that MPT, GSO and Danang receive their portion of the IDA Credit for their Subprojects as a grant. Hanoi and HCMC – which MOF argued had sufficient internal revenues to handle repayment - would receive their portions on a part-loan, part-grant basis. The two cities had requested that only those portions of their respective subprojects with cost-recovery or revenue-generating potential be financed through on-lending and the remainder through grants. The loan portion in each case was to be via advances in VN Dong from the state budget on modified IDA terms and with MOF bearing the foreign exchange risk, commitment and service charges.

2.2 Implementation Principal factors affecting project implementation: 2.2.1 Complexity and readiness to implement: Implementation conditions/capacity on the ground, especially for procurement and project/subproject management, was (except for GSO and Danang in key aspects) exceptionally weak, hampered by: (i) lack of experience with Bank projects/procedures and bureaucratic, multi-layered and cumbersome processing and approvals, especially by the People’s Committees; (ii) inadequate approaches at the PIU level magnified by their weak grasp of the Project’s opaque strategy and vision; and, (iii) in the first 2-3 years, frequent organizational and staff changes disrupting continuity and capacity-building, along with under-staffing, low salaries and difficulties in recruiting qualified/specialist personnel. By the 2009 Mid-term Review (MTR), the MIC and Hanoi PIUs still lacked essential qualified staff; (iv) since the MPT/PMU’s operating costs were not covered by the DCA until 2010, the unit lacked sufficient full-time staff to complete the Operational Manual or the website designed to aid inter-PIU communication; (v) implementation of government’s ICT/e-Government Strategy and its Administrative Reform Program was slow and poorly-coordinated, affecting the Project’s operational and policy context. The Project was integrated in the National IT Steering Committee early on but this mechanism for project orientation appears to have lost priority or involvement.

8

2.2.2 Procurement challenges: Procurement encountered specific constraints. First, contracting of international procurement specialists took inordinate time, delaying for up to two years in some cases the acquisition and installation of new infrastructure, systems and services not previously known/utilized in Vietnam. Second, inexperienced PIUs with no prior relationship to the Bank and negligible procurement capacity were required to execute 92 procurement “packages”, many requiring International Competitive Bidding (ICB). The 2010 Restructuring made marginal changes and dropped the HCMC Subproject, but essentially left this situation intact. Third, as noted in the Danang and GSO completion reports, Bank procurement no objections could be slow, and PIU procurement staff in general felt under-trained to use the Bank’s new approach introduced in 2010 for technical specifications in IT bidding documents; almost all bidding documents prepared required revision. Fourth, delays affected technology packages where rapid innovation left some outdated even before their purchase was concluded (see 2.2.3). Finally, GOV cost norms for specific packages could not compete with market conditions, further delaying official approvals. Given the scope of these problems, the Bank country management team authorized a senior procurement expert to provide hands-on advice to PIUs, with positive results, and with further follow-up support, the PIUs slowly mastered procurement processes. In general, PIUs with greater technical competence and strong, collaborative relationships with their People’s Committees (Danang, GSO) devised better technical solutions and more efficient procurement and contract implementation (see 2.4.2-2.4.3, and Section 5).6 2.2.3 Rapid technological change: Related to 2.2.2, rapid technological change and fast evolving systems in the ICT marketplace from preparation through end-project affected progress and had important collateral effects. The marked reduction in ICT product cycles and increasing adoption of the enterprise architecture approach, broadband and Wide Area Networks (WAN), combined with the PIUs’ limited technical competence and institutional capacity, had the following effects: (i) Changing specifications delayed the development of detailed procurement plans with key packages identified in 2008, but bidding documents not ready for up to 18 months; (ii) PIU specialists needed time to update knowledge, digest consultants’ information and translate it into reasonable technical requirements. Some procurement packages were already superseded and could not be processed; (iii) Technical changes also prolonged the bid evaluation process for up to a year. Meanwhile, contracted equipment became unavailable requiring contract amendments and causing further delay. 2.2.4 Mid-term Review (MTR): The MTR was timely, well-staffed, accurately defined the primary obstacles affecting project execution and prepared the strategy and content of a proposed restructuring. Actions agreed as a result of MTR analysis included: (i) Level One restructuring incorporating elements discussed with authorities; (ii) maintenance by the PIUs of the same organizational structure and teams, backed by intensive training; (iii) closer involvement of and speedier approval of procurement packages by PIU leaderships; and, (iv) project-financed technical assistance to strengthen project management and supervise the execution of large procurement packages. Of particular importance, participating agencies (MIC and GSO) and

6 Well into project execution, when procurement issues had become chronic, the Bank team actively considered consolidation of the procurement function across PIUs as well as moving a Bank procurement staff member to work directly on the Client’s side on bidding and other documents. However, the associated difficulties were not easily overcome and the plan was set aside. The ICR suggests based on discussion with the current Bank team, that various solutions could have been devised to deal more decisively with this situation.

9

cities (Hanoi, Danang and HCMC - the latter dropping out after the Restructuring) were to streamline internal procurement appraisal and approval processes with clearer assignment of responsibility and accountability. Results of these decisions/efforts were inconclusive (see 2.2.5). 2.2.5 Restructuring 2010: The 2010 Level One Restructuring scaled the Project back while adding new activities but did not change its fundamental design and approach, or substantively address capacity constraints, perpetuating weak performance. The link between the revised PDO and Results Framework remained unclear, the PIUs still shied away from monitoring and measurement, and Bank supervision did not press the issue. Some activities were cancelled and the HCMC Subproject was dropped. The environmental category and applicable Safeguards policies remained as appraised. Expected benefits of this restructuring were increased numbers of beneficiaries of ICT development overall and better public access to e-services, especially in Danang to which an additional US$10.0 m. were reallocated consistent with its good performance. GSO also received an additional US$7.0 m. to scale up its Subproject. The PMU and PIUs’ limited project implementation capacity remained a risk but participating agencies and cities were to streamline internal procurement processes with clearer assignment of responsibility and accountability. As noted, and with certain exceptions, this restructuring did not improve progress. 2.2.6 High-level joint project review: Consultations in January 2012 between Bank senior management and the Vietnamese Deputy Prime Minister (DPM) galvanized progress in all PIUs and were a decisive factor in ensuring completion of the Project. Notably, some 10 Bank-supported projects in Vietnam were performing poorly at that time and were on the Bank’s watch list; the ICT Development Project was not an outlier. Responding to the Bank’s insistence on action, the DPM issued direct instructions to participating agencies to accelerate execution. Intensified Bank supervision, including a technical procurement mission provided direct support to critical procurement packages linked to achieving the PDO. Detailed action plans were discussed and agreed with the PCU (MIC), PIUs, the Ministry of Planning and Investment (MPI), the State Bank of Vietnam (SBV) and the Ministry of Finance (MOF) on procurement, financial management, and results monitoring. Substantial progress resulted by January 2013 with most outstanding procurement packages signed. Commitments increased to 85% of the total IDA credit. 2.2.7 Proactivity of Bank supervision: Changed Bank task management, combined with the actions described above (2.2.6) reversed the project trajectory and built renewed trust. Further restructuring in mid-2013 extended the closing date and adjusted the Results Framework for clarity/relevance. The project disbursed rapidly in 2013 because a large number of contracts initiated in 2012 were completed. The Bank Task Team provided intensive technical implementation support with additional IT experts, helping to shorten the technical review of bidding documents and bid evaluation reports to the Bank’s service standard of an average of two weeks. M&E was prioritized: PIUs designated individuals responsible for specific segments of the process and monthly updates on results were conducted via video-conferencing. The risk of project cancellation, continuing relevance of the PDO, and importance of the Project to the Bank/GOV relationship and dialogue, pressured the Bank and country counterparts to complete outstanding procurement/training, and ensure that equipment/systems were operational by closing. The Bank portfolio review made senior management in supervising agencies aware of pending issues and take greater control of procurement. But above all, counterpart staff was able to “master” the Bank’s ICT procurement guidelines especially the new regional guidance on the shift from a specification-based to performance-based approach. 2.2.8 Project at Risk Status: The MTR downgraded the project ratings to MU and the Project fell into problem status. Performance remained sluggish until 2012 but, consciously erring on the conservative side given the Project’s generally troubled trajectory, the DO rating was maintained

10

by the Bank team at MU through closing. The IP rating was upgraded to MS due to rapid and conclusive progress on remaining procurement packages in 2013. See Annex 2 matrix of procurement packages completed.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 Design: The PAD called for a Management Information System (MIS), complemented by semi-annual progress reports and impact studies prepared by the PIUs and PCU. This was consistent with the PAD’s assertion (Annex 1) that the challenge facing government was not only how to achieve its ambitious targets but to strengthen the results-orientation of several ongoing programs through the “establishment of realistic indicators and benchmarks for rigorous M&E”. Monitoring and evaluation (M&E) was to be the responsibility of individual PIUs at their subproject level with the MIC PCU consolidating results sent by the PIUs, identifying lessons and disseminating knowledge to stakeholders - including the Bank and government. 2.3.2 Evolution of the PDO and Results Framework: The ICR Data Sheet illustrates the problematic Results Framework. The original PDO was broad, complex and in key aspects unmeasurable within the life of the Project. The causal link between the Project Outcome Indicators and PDO internal elements was unclear, and confused outputs with outcomes. Presentation of the PDO was inconsistent within the PAD, including the Results Framework (RF) and Arrangements for Results Monitoring (ARM), and between the PAD and Development Credit Agreement (DCA), creating difficulties for the ICR. Overall design of the RF was exceptionally weak, with excessive numbers of indicators (especially Intermediate Outcome and the separate slate of outputs tied to the components – see Annex 2) and many were not SMART (specific, measurable, achievable, relevant and time-bound). The ICR adheres to the initial, two-part statement of PDO which also accords with the DCA and as shown earlier in Sections 1.2 and 1.3, aligns the results indicators accordingly. 2.3.3 The 2010 Level One Restructuring narrowed PDO scope to focus more tightly on IDA activities and added several Core Indicators concerning processing time for public services and user perceptions of e-services. Overall measurability improved to some extent, but the number of indicators remained unusually large and their ability to capture the PDO was still problematic. Retention of the private sector objectives was not matched by appropriate tools to deliver the impacts sought. Revisions were overly-focused on input indicators measuring implementation rather than results. No plans were made to conduct an impact evaluation or related study - surprising given that M&E standards in the Bank were much higher by 2009 - virtually no monitoring activity had yet occurred, and this was the logical opportunity to formalize M&E. 2.3.4 The Level Two restructuring in June 2013 further adjusted the RF, while leaving the PDO intact. The new Project Task Team in 2012 mapped key project activities and procurement packages against expected outcomes reflected in the internal elements of the PDO, and found that large numbers of important activities directly affecting PDO achievement were still in the early stages of procurement. The decision to further adjust the RF to adequately measure achievements of the PDO improved measurability of indicators per se and linkage to the PDO, but the overall number of indicators - including many with multiple internal targets - increased (see Data Sheet). 2.3.5 M&E Implementation: Overhaul of both the PDO and indicators in 2010 did not result in a sharper focus on monitoring, and evaluation lacked priority. Continued poor definition/clarity

11

of the RF, limited capacity and focus in the PIUs and continuing inability to see the Project in integrated terms, plus the PCU’s general lack of leadership deterred data collection on project results. The PIUs were monitoring progress on the procurement packages.7 In a major push to have the PIUs focus on PDO achievements, each assigned an M&E focal point for tracking progress and key procurement packages were accelerated, but the overall quality of data reporting was low and collected data was not used for planning or decision-making. Data collection and organization improved in the final 18 months and evaluation/studies made headway, including: an independent evaluation of the Danang Subproject (Leading Associates/KDI Asia, 2013) paving the way for rigorous M&E of future, similar projects; a well-conducted survey contracted by MIC PCU (An Vien Media, 2013) to gauge the effectiveness of the project communications campaign (Annex 5); and, a Project Performance Assessment Report (KDiAsia), intended to provide MIC with internal guidance for future operations.8 MIC/PCU, Danang and GSO prepared Completion Reports (executive summaries in Annex 7). 2.3.6 Utilization: The ICR acknowledges some internal Bank concerns that the adjustments to the Results Framework in mid-2013 seemed too late to be used to measure end-project outcomes, but opted to use that final version anyway, for the following reasons. First, initial analysis prompting the decision to adjust the RF occurred in March 2012 with the expectation that changes would be effected within a reasonable period. However, the multi-layered, multi-agency sign-off process in GOV delayed final Bank approval until June 2013, six months before the revised closing date and 15 months after the initial proposal to restructure. Second, adjustments in 2013 were a serious effort to ensure that the indicators reflected actual project achievements. Indicators were further refined and brought down to project level by specifying units of measurement, linking targets to the institutions where results were expected to occur, eliminating intermediate indicators considered irrelevant, and adding two new Results Indicators to more accurately cover expected PDO outcomes. Such changes were also required by the cancellation of some procurement packages and amendments in the project description. Third, the project archive shows that senior Bank management tacitly assumed that the Level Two adjusted RF would be the version used to measure achievement of the PDO. 9

2.4 Safeguard and Fiduciary Compliance

2.4.1 Safeguards: Due to its nature, the project was an environment Category C, and did not trigger any safeguard policies. The category did not change with restructuring.

2.4.2 Procurement: Procurement execution involved “learning by doing”. Procurement risk was rated High at appraisal because the PIUs had not yet been established nor their procurement

7 Evidence suggests the PIUs – GSO and Danang excepted - always saw the Project as an aggregation of procurement packages since their focus was almost wholly on getting those prepared, approved and executed. Most PIUs lacked any concept of the “Theory of Change” although at end-project they were all able to narrow down the Project’s key impact on behavior – the transition from manual to electronic administration and services provision. 8 See: Impact Assessment Study of Computerized Services Delivery Projects from India and Chile, Bhatnagar, Tominaga, Madon and Bhatia, World Bank Staff Working Paper, #42147 (November 2007).The study demonstrates how to estimate direct and indirect benefits to clients with reasonable accuracy using a framework which includes surveys of clients (citizen and business users) and other stakeholders. While the study shows the potential of ICT to help reduce costs of accessing services, improve convenience and accountability and promote good governance, it also advises looking beyond the “hype” of e-government and that the principles of good project design and management remain important to results. 9 Email from U. Zachau (EAPVP) to V. Kwakwa (EACVF) of January 11, 2013 refers to key remaining issues in project implementation for the 6-month extension period including the need for “….modification of the indicators to measure more accurately the results contributing to the achievement of the project development objectives….”.

12

staff identified. Many procurement packages, including specialized consultant services were unique and challenging, the cycle of no objections and PIU approvals was inordinately protracted in many cases, and some PIUs were reluctant to communicate their need for Bank guidance. Difficulties arose - and persisted over time – in getting agencies’ clearance/approval of project procurement documents due to their insistence on applying the National Procurement Law despite provisions in the DCA - and in the National Procurement Law itself - giving precedence to Bank Guidelines. As discussed, procurement performance was variable between PIUs and greatly affected the Project’s ability to advance (Section 2.2). Most IT projects face similar procurement challenges. The Bank may need to revise its procurement documents for IT projects, moving beyond an input-focused approach to processes more appropriate to a fast-moving sector.

2.4.3 Within this context, the Bank procurement team (based in Hanoi) conducted regular supervision, training, workshops and direct mentoring in procurement planning, cost estimation, terms of reference, technical specifications, RFPs and bidding procedures. Procurement ratings fluctuated between MS and MU over the project period. Complaints associated with procurement packages in MIC and Danang in 2012 resulted in explicit warnings to all PIUs and additional resources to focus attention on bidding processes and transparency. The MU ratings reflected slow procurement progress and weak management, not malfeasance. The PCU/PIUs grasped the importance of (and strove to develop) sound quality control of bidding/other processes and to comply with Bank standards, slowly overcoming deficiencies with time and mastering processes.

2.4.4 Financial Management (FM) and audit: As with procurement, FM required intensive attention/supervision. The FM performance rating fluctuated between MS and MU over the life of the Credit and financial risk was rated mostly Substantial. The main reason was variable capacity between PIUs/PCU. At appraisal and as with procurement, there was no institutional base or identified staff for project FM, and no computerized system (MIS) in place to track project transactions and conduct financial reporting. Institutional arrangements with multiple units and Special Accounts affected FM efficiency. Internal controls, including contract management, eligibility of expenditures, defined FM responsibilities, reporting structures and accounting software – the latter experiencing serious problems of installation, activation, and systemic errors depending on the PIU – were slow to improve. Capacity-building was recommended and training provided. The rating fell to MU in 2012 due to a series of outstanding issues - ineligible expenditures, delayed financial audits and weak contract management - but was restored to MS by the final supervision due to all PIUs showing improved performance in specific aspects.

2.4.5 Audit: Eight audit reports were required and by ICR, seven were received with one pending. Three audits were late and two had issues of accountability and internal controls. Auditors’ opinions were qualified on occasion and audit quality was uneven. Delayed delivery of the 2011 audit and protracted delay in reimbursing the Bank for some ineligible expenses implied the possible suspension of disbursements under BP 10.02, but issues were resolved to the Bank’s satisfaction by closing.

2.4.6 Project costs: Total project cost was US$77.08 m, 72.1% of the US$106.96 m. estimated at appraisal.10 Cancellation of IDA resources totaled US$13.52 m. over the project

10 Total disbursement of US$74.12 m, cancellation of US$13.52 m and undisbursed balance of US$6.23 m, as shown in Bank system, June 26, 2014. Recipient contribution is a best estimate as at same date. See Annex 1.

13

period. The Recipient’s contribution was an estimated US$2.96 m, 22.36% of the appraisal estimate. Costs per Subproject varied significantly from appraisal estimates (see Annex 1). Combined supervision costs for FY12 and FY13 alone were US$384,000, well above regional norms due to the marked intensification of Bank supervision in that period. Due to the tendency for prices to fall, and increasing efficiency in procurement management over time, project costs were reduced by a total US$15.4 m. over the course of the Project. More effective and cheaper solutions became available such as the scanning facility in GSO and online visa application system in MOFA. The Danang PIU Completion Report cites combined savings on several of its packages exceeding US$6.0 million.11 The PCU and all PIUs used savings to purchase additional systems/equipment.

2.5 Post-completion Operation/Next Phase

2.5.1 The Bank and counterpart project teams mapped out a plan for the immediate post-completion operation and maintenance of project infrastructure and systems and their medium-term sustainability (see Section 4). As discussed (2.3), no formal O&M system was established by the Project although GSO by the nature of its data and research functions is well-positioned and motivated to maintain project achievements, as is Danang City which has become a national e-services demonstrator. 2.5.2 An official delegation from Vietnam visited the Bank in December 2013 to present the results of the ICT Development Project and to outline a new ICT operation which might include scaling up the especially successful Danang experience nation-wide, further increasing the scope and efficiency of the GSO, and in the case of Hanoi, replicating and increasing e-services across different levels of government. Given the foundational nature of the Project, continued development of IT professionals and adequate budgets are essential. Annex 5 summarizes recommendations of the Danang study which point the way to post-completion operation and expansion of this/similar projects. Regardless, the ICR suggests, based on reviewers’ comments, that any new ICT intervention be results-based or a Development Policy Credit rather than an Investment Lending (IL) operation. A programmatic approach for such projects, with a second operation contingent upon performance under a first, may be advisable.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation Rated: Moderately Satisfactory 3.1.1 Relevance of PDO: Continuing relevance of the (restructured) PDO is rated Satisfactory, capturing key development themes in the current Country Partnership Strategy (Report #65200-VN of November 7, 2011) and GOV’s current development priorities. The CPS identifies issues affecting the business environment: weak infrastructure, high costs of interacting with government; and, lack of adequate information on policy, all of which resonate directly or indirectly with the PDO to “foster private sector development by facilitating the increased adoption of ICT”. The CPS recommends, inter alia, strengthening Vietnam’s technology and innovation system, and supporting the country’s transition to a successful mid-income country by

11 Danang PIU US$6.0 m, MIC PIU US$3.7 m, MIC PCU US$0.4 m, GSO US$3.7 m and Hanoi US$1.6 m

14

promoting innovation-led and higher-value-added production and services. Strengthening governance is an important cross-cutting theme and links directly to the PDO: “support MIC’s institutional development and technical and regulatory capacity for ICT oversight and to increase efficiency and transparency of public service delivery in MIC, GSO, MOFA, Hanoi and Danang”. 3.1.2 Relevance of design and implementation: The relevance of project design and implementation showed significant shortcomings, and is rated Moderately Unsatisfactory. Prima facie, the technical design was acceptable and responded to defined needs, but the Project’s ambitious vision for the project period was unreasonable on capacity and readiness grounds. The risk assessment was incomplete, under-estimating human resource factors, and overlooking issues such as the demand side, citizens’ IT literacy/uptake and the implications of overall complexity. Mitigation measures were limited/inadequate. The relevance of PDO and Results Framework design was especially weak; the former improved in 2010 but the latter not until the final year. Implementation of M&E was unsatisfactory until the final stages. Few specific activities were tailored to private sector needs or linked to the PDO’s private sector goals. Most of the Sub-projects (components) were implementation inputs and outputs linked causally to improved government services. While the private sector would, like regular citizens, be likely to benefit from many of those services, the link to “private sector development” was not well-justified or measurable. The institutional arrangements fostered inefficiency, duplication and delay throughout the project cycle. They would today be subjected to tougher internal scrutiny and demand joint institutional arrangements and shared functions. The PAD discussed the need to sequence business process change and awareness-building ahead of equipment purchase and application development but in practice, project implementation was supply-driven and the human factor was under-emphasized. The demand side prominent in current Bank policies/methodologies across sectors lacked priority.

3.2 Achievement of Project Development Objectives 3.2.1 Revision of the PDO via a Level One Restructuring entails that the ICR conduct a split assessment of outcome using the pre- and post-restructuring PDO and indicators, weighted based on respective levels of disbursement in the two stages. The following presents project results against the original PDO and indicators up to the date of Board approval of Level One changes (January 29, 2010). Results based on the revised PDO and RF up to closing at end-December 2013 are shown separately using the indicators as restructured in June 2013. Pre-restructuring period (from Effectiveness to January, 2010) 3.2.2 Profound delays in project execution and disbursement under the first stage of the project resulted in few PDO-linked achievements in this period. To the extent that any measurement was conducted, it was of basic progress in the contracting and completion of planned procurement packages which for the PIUs – both at that time and virtually throughout – were synonymous with the Project. Original PDO Part A: Rated Unsatisfactory 3.2.3 The internal elements or themes of the original PDO Part A were four-fold: “to accelerate the use of ICT and support institutional development; to increase the efficiency and effectiveness of public administration; increase the transparency and availability of information; and, improve public services delivery to stakeholders”. With two notable exceptions, there is little to show through end-2009, evidence of the acute problems affecting implementation. All four elements entailed a critical mass of investments and appropriate evaluation including user surveys and case studies to measure behavior change through gains in “efficiency”, “effectiveness”

15

and “transparency”, including several at the level of national indices and probably unmeasurable even under higher levels of achievement at the time. 3.2.4 The following situation prevailed at end-2009:

• MIC issued common technical standards for the Inter-operability Framework in 2008, and guidelines for the development of the E-Government Enterprise Architecture (EA) in 2009. These were important milestones for Vietnam’s overall ICT development.

• Danang City put a city-level EA in place to guide its e-government development. • GSO acquired and developed its Form Scanning system crucial for faster processing of

the 2009 Population Census. • Otherwise, negligible achievement was shown by most PIUs (except Danang and GSO)

in acquiring the infrastructure, equipment, systems and services to move beyond the policy and standards stage.

• By end-2009 citizen usage remained incipient, as did the “increased transparency and availability of information and improved public services delivery to stakeholders”.

Original PDO Part B: Rated Unsatisfactory “Foster private sector development in particular by facilitating the increased adoption of ICT by small and medium enterprises (SME)” 3.2.5 There is in the pre-Restructuring period, some limited evidence of SME adoption of project-financed ICT but no hard evidence that this “fostered private sector development”, a complex linkage difficult to measure in the circumstances. The Bank MTR team reported that SME-related activities had received modest attention in the period. 3.2.6 Addressing individual indicators:

• Anticipated growth of 30% per year in the domestic IT service sector as a result of the project fostering private sector development was unrealistic and lacked a baseline on which to assess incremental gains.

• Facilitation of “increased adoption of ICT” was limited to Danang having established its ICT Transaction Center (TC).

• 15% of SMEs in Hanoi, Danang and HCMC were to be using ICT for business operations by 2010. The baseline is identical for all three cities and is assumed to be national, not city-specific: Danang, measured via canvassing of SME customers using the new TC, registered 18% usage for business purposes by early 2010, but attribution to the Project is unclear, i.e., whether the increase is based purely on impact of the TC on awareness.

Post-Restructuring period (February 2010 to end-December 2013): 3.2.7 Delays continued to plague the post-Restructuring period and for many of the same reasons, but Danang City and GSO continued to lead the way and overall, progress picked up. By closing, all procurement packages not cancelled were fully completed and operational. Evidence is mostly output-based with one exception: Danang PIU financed a study which paves the way for future M&E of ICT development projects and reports findings and lessons directly attributable to the Danang Subproject.12 Some results are included below, with further details in Section 3.6 and Annex 5.

12 See DNG14: Assessing the Status of Danang e-Government Implementation and Building its e-Government Model based on the Assessment. Leading Associates and KDi Asia, 2013.

16

Revised PDO Part A: Rated Moderately Satisfactory “To support MIC’s institutional development and technical and regulatory capacity for ICT sector oversight and to increase the efficiency and transparency of public sector service delivery in MIC, GSO, MOFA, Hanoi and Danang”: 3.2.8 All PIUs established the foundation for improved e-government, formulating frameworks and standards, upgrading key e-government infrastructure, and training human resources at managerial/other levels, both at HQ and in decentralized government units. A fundamental transition occurred from manual to computerized administration and services. Internal procedures and public service systems were improved or developed using ICT and are fully operational including portals and websites. These are likely to have improved the efficiency and transparency of public services delivery, democratic access to information, and government accountability, although the ICR cannot definitively prove this even from the Danang study. Danang and GSO are good cases illustrating how an ICT project can support local e-government and a central agency respectively, to become service- and citizen-oriented facilitators/providers. The following assessment disaggregates the PDO by its main themes. Theme 1: To support MIC’s institutional development and technical/regulatory capacity for ICT sector oversight:

• MIC’s institutional development was supported through the formulation and application of four standards: Enterprise Architecture, the Interoperability Standard, the e-Government Framework and Security Standard.

• MIC’s regulatory capacity was improved through laws, decrees and circulars promulgated/approved: Postal and Radio Frequency Laws; the Information Security Law – of crucial importance in an expanding e-services environment - and MIC decrees supporting the application of IT and Information Security.

• Technical capacity improved through numerous infrastructure upgrades: LAN networks in MIC headquarters and 12 DICs; smart building communications systems connecting MIC to regional offices in Danang and HCMC, MOFA and the Hanoi Telecom Authority; video-conferencing, IP phone and building security systems operational.

• At the district level (DIC), upgrades included: PCs, laptops and printers, video-conferencing facilities, email and LAN/WAN integrated connectivity.

• Training of 500 CIOs/IOs, 220 state-owned enterprises and 70 SMEs supported the operations and sustainability of these upgrades (policy formulation, capacity-building and implementation guidelines for IT and telecom sectors).

Theme 2: Increase the efficiency and transparency of public sector service delivery (MIC, GSO, MOFA, Hanoi and Danang):

• The primary change in behavior was the transition from manual to electronic/digital processing of governments’ key business processes and delivery of public services.

• Reduced transaction times demonstrate efficiency gains: MIC’s Radio Spectrum processing time was halved from 20 to 10 days; registration for imported publications

17

from 30 days to 15; online visa application processing averaged 2.5 days; in Hanoi, average transaction time fell 50% for selected e-government services via the City Portal.

• GSO’s Population Census data collection/processing time was reduced by 42% with an estimated cost reduction of US$500,000 as a direct result of project-supported ICT upgrades; Forms Scanning infrastructure and other upgrades facilitating the 2011 Agriculture and Rural Census reduced processing time by 50% (two years to one); and, data collection/processing for CPI surveys fell by one-third (15 to 10 days).

• Global experience validates the tendency for online services to promote transparency. The Project established, inter alia, online drivers’ licensing, business registration, water quality monitoring, construction permits and public transport management.

• Improved connectivity and information flow G2B, G2G and G2C, as well as local facilities such as transaction and data centers, information kiosks, online booths, e-Learning portals and public WiFi also likely promoted efficiency and transparency.13

Revised PDO Part B: Rated Moderately Unsatisfactory Foster private sector development by increasing access to business-related government services and raising awareness and usage of ICT in Hanoi and Danang.

• Interviews and anecdotal evidence suggest that the “viral” nature of ICT skills acquisition, the breadth of information now available to business and speed of access to it, and efforts by the Project to promote awareness, are positive signs for downstream gains in private sector development beyond the Project.

• Substantial numbers of SMEs took ICT training courses, visited/utilized the new Transaction and Data Centers, and accessed online e-services, e-learning courses, portals, and SME-tailored training modules.

• Localized surveys in Danang showed that 33% of SMEs were using ICT and 18% using it for business purposes. Caveats noted above (3.2.6) apply.

• SME’s awareness of the availability of project-financed, business-appropriate e-services and how to use them had increased.

• Several business-related e-services, e.g., online Business Registration and Licensing, and the Investment Project Management System were launched in Hanoi and Danang, markedly reducing transaction times compared to legacy, paper-based transactions.

• However, despite these gains, there is no hard evidence that private sector development was fostered only that access, awareness and usage by SMEs had improved. This indicator was felt to require more formal evaluation to reveal specific evidence of “development” and that the behavior changes noted did not in themselves, equate to that development.

3.3 Efficiency 3.3.1 Assessing the efficiency of the Project in a rigorous manner would require extensive data, including detailed financial and operating information from government agencies as well as surveys carried out among beneficiaries both before and after the interventions. Given that this data was not available by project closing - and experience shows that a period of maturation is

13 The Danang study (Leading Associates/KDiAsia, 2013) found that – at least at project closing - a majority of potential and actual citizen e-service users interviewed expressed preference for a human face at the “counter” but this is not interpreted as dissatisfaction with online services per se. E-services were still tending to be used for information-seeking not end-to-end transactions. Many factors constrain demand including computer literacy and access to a computer per se, which received insufficient attention from or acknowledgment by, the Project.

18

normally needed for ICT investments before such data is meaningful - certain assumptions can be made about the efficiency of the project based on results as well as what is likely to happen based on experiences in other countries. See Annex 3 including sources. 3.3.2 Project interventions affected efficiency for three groups: government, businesses and citizens. For government, transitioning from manual to computerized processes results in lower costs and shorter processing times. Project-financed training of government officials taught them how to implement IT systems to achieve efficiency gains. Further, sensitizing high level officials is likely to have reduced resistance to restructuring business processes. Project-fostered coordination among government agencies will have reduced duplicate IT systems resulting in improved efficiency. 3.3.3 GSO: Data collection and processing for the 2009 Census was reduced to 14 months from 24 months (42% reduction) as a direct result of project ICT improvements. The estimated cost savings were US$500,000 based on the 450 data entry staff used for the Census, assuming they received the average monthly salary for government workers. Forms scanning hardware used for the 2011 Agriculture and Rural Census generated a 50% improvement in overall processing time, reduced from two years to one. Data collection and processing time for Consumer Price Index surveys was reduced by one third from 15 to 10 days using newly-introduced IT processes. The GSO subproject also reduced duplication by harmonizing and integrating databases within a standard system architecture. The provision of additional/free statistics online would also have improved decision-making by both government and the private sector. 3.3.4 Small and medium enterprises: Interventions aimed at Small and Medium Enterprises (SMEs) included training and online services. These are likely to have positively impacted SMEs in Vietnam. World Bank research suggests that online business registration systems (among the applications developed by the Project) reduce complexity, time and costs leading to a rise in business registrations and employment (as well as generating income for government). Similarly, facilitating construction permits (another project application) through online applications saves businesses time and money. 3.3.5 Danang City e-services: The delivery of online public services benefits citizens by reducing the cost and time of travelling to and waiting at government offices. Examples cited in Annex 3 include Sri Lanka and Europe. Similarly, 45% of respondents to the Danang survey expected project-financed e-services to save time, 35% foresaw greater convenience and 23% expected to save costs. The city-wide free Wi-Fi scheme in Danang saves citizens time and money. Businesses also benefit: customers could find businesses easier using Internet-based location services, and businesses could exploit social media/other applications to create innovative relationships with customers. Estimating the cost savings to citizens from free Wi-Fi means calculating the cost of Internet access payments no longer needed. Assuming the same proportion of Internet users in Danang as the nationwide average and that one third would use the free citywide Wi-Fi service, the aggregate consumer benefit is over US$350,000 per month. 3.3.6 Overall project implementation efficiency was also notable. For example, the total estimated value of the Danang subproject at appraisal was US$27.3 million. After five years of implementation, Danang PIU saved a total of US$6.1 million through competitive procurement practices and good management, and acquired additional equipment with the savings. All other PIUs also registered substantial savings although not on the same scale as Danang. Aggregate cost savings were US$15.4 million. See also 2.4.6 above and footnote 10.

19

3.3.7 Although detailed information necessary for carrying out a comprehensive efficiency analysis is lacking, the anecdotal data suggests that project interventions are already having an impact on reducing costs for government and saving time for SMEs and citizens. These benefits can be expected to grow as e-services evolve in scope and functionality and more Vietnamese use the Internet.

3.4 Justification of Overall Outcome Rating Rating: Moderately Unsatisfactory 3.4.1 Overall PDO rating: Consistent with the ICR Guidelines Appendix B, separate outcome ratings for the original and restructured projects respectively were weighted in proportion to the share of disbursements in the periods pre- and post-Restructuring. PDO Rating - Original and Restructured Project

Item Original PDO Revised PDO Overall (a) Rating Unsatisfactory Moderately

Unsatisfactory Moderately Unsatisfactory

(b) Rating value 2 3 - (c) Weight (% of total available financing disbursed, original and restructured project)14

10% 90%

100%

(d) Weighted value (c x b, rounded)

0.2 2.7 2.9

(e) Final Rating - - Moderately Unsatisfactory 3.4.2 Overall project outcome rating: This rating of Moderately Unsatisfactory takes into account the following:

• Relevance: Assessed against current Bank strategies, PDO relevance remains Satisfactory. However, the ICR asserts that a technically and operationally demanding project design was largely imposed on an inexperienced client. Capacity and readiness to implement were over-estimated at appraisal. Design and implementation relevance are therefore rated Moderately Unsatisfactory. Overall relevance is rated Moderately Satisfactory.

• Efficacy: Achievement under the original PDO is rated Unsatisfactory because with the exceptions noted, progress as measured by level of outputs (completed procurement packages) and disbursements was still negligible by end-2009 and thus most expected outcomes – changes in behavior – lacked the conditions for achievement. Most of the latter came well after the 2010 Restructuring and some of the most critical not until 2013. The post 2010 Restructuring phase is rated Moderately Unsatisfactory, reflecting a conservative approach which acknowledges the Project’s continued difficulties, last minute execution of key procurement packages and lack of time to conduct user surveys, although the vast majority of planned systems, equipment and training were implemented, with substantial cost savings, and with early/preliminary indications of reduced transaction costs and greater efficiency. However, looked at overall, and not just through the lense of the good performers (Danang and GSO), there is a significant risk given likely medium and longer-term challenges, that outcome may not be sustained (see Section 4).

14 The ICR takes total IDA resources disbursed as at July 16, 2014 (US$74.07 m), and for the pre-Restructuring period, total disbursements by January 28, 2010 (US$7.35 m) as a percentage of that total, with the balance attributed to the second, post-Restructuring stage. This results in relative percentages of 10% and 90% respectively.

20

• Efficiency: Based on evidence of efficiency gains in time and cost already reported and likely impacts based on global experiences for public e-services, balanced by the acknowledgment that the project currently has limited hard evidence of citizen/SME gains, and the fact that the efficiency discussion is largely couched in terms of likelihoods, project efficiency is rated conservatively as Moderately Unsatisfactory.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 3.5.1 The project had no overt poverty objectives, and did not address gender either in design or during implementation perhaps due to a perception that they were not immediately relevant goals for the type of project and its stage of development. Neither did the project have explicit social objectives although it can be inferred that more efficient provision of government services and the wide range of e-services already developed in the Project’s lifetime, have solid potential for positive social impacts, e.g., water quality monitoring introduced in Danang; more rapid and transparent business registration; e-learning portals and TV programs delivering courses to enhance citizens’ skills; and, improved management of public transport. Potential social benefits are likely from better planning and policy-making using GSO’s improved population/other data and modernized systems/procedures. (b) Institutional Change/Strengthening 3.5.2 The project has already had significant impact on the administrative and operational efficiency of the participating institutions, most notably (and measurably) the General Statistical Office (GSO). Key government services are more accessible in less time to current and potential users. Most key administrative and operational functions are now electronic/digital rather than manual, representing the most profound of the behavioral changes prompted by the Project. Most staff from the PIUs, now possessing specific skills developed over the course of the Project, have already returned to their parent agencies. Danang representatives refer to their daily activities as being more “citizen-centric” and that coordination and inter-connection among departments and agencies is more streamlined and easier. The Project laid the foundation for changing the science of service delivery and inclusive innovation with ICT. Other new innovation, public procurement and education projects in the Vietnam portfolio will benefit from this achievement. (c) Other Unintended Outcomes and Impacts (positive or negative) 3.5.3 Examples of unanticipated developments include: (i) substantial, unexpected aggregate cost savings of US$15.4 m resulting from competitive and over time, more efficiently-managed procurement; and (ii) collaboration with other donors conducting similar/related projects (as envisaged in the PAD) went further than expected. For example, the close consultation/coordination between GSO and the United Nations Population Fund (UNFPA) within the design and implementation of package GSO 6.1(b) provided more efficient data entry and processing technology for the 2009 National Population Census. UNFPA also helped design GSO’s questionnaires appropriate for mechanized data entry and optical recognition, and trained interviewers and enumerators.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 3.6.1 The Danang Subproject financed the project’s only major evaluation study, but the MIC PCU also financed an important survey of citizens’ awareness of the ICT Project and their opinions concerning the benefits of project-financed e-services. Relevant conclusions from

21

Danang are summarized below, with details on both reports in Annex 5. The Danang study used the e-GAF (e-Government Assessment Framework) which looks at ICT Outcomes, Program Management and Governance, using qualitative, interview-based information in this case from 642 citizens, 75 businesses and 38 public agencies, including transport, health, science and technology, Peoples’ Committees and DICs. The findings of three other methodologies discussed in the Danang study are summarized briefly in Annex 5 but their research ranged more widely and only a small portion of results can be attributed directly to the Project. An important objective of the Danang study was to illustrate methodologies available for future ICT M&E.

• Project-financed Danang ICT Transaction Center, Data Center, Public WiFi and Metropolitan Area Network (MAN) are providing good quality services to the public and government agencies.

• E-services being quite new, their current role is focused more on information than transactions. Citizens’ IT skills/access remain limited and information campaigns about e-service availability/usage were still ongoing at the time of this study.

• With more services migrating online and demand increasing, human resource capacity and budgets must also be scaled up with attention focused on managing information security and measuring ICT performance.

• Danang’s ICT strategy is based on consultative, inclusive processes involving stakeholders and awareness of ICT is high among senior city managers interviewed.

• Due to the Project, Danang City’s e-government was rated top of the Vietnam ICT Index for five years (2009-2013) and its ICT development is entering a maturing phase.

4. Assessment of Risk to Development Outcome Rating: Significant 4.1 Final supervision missions and video-conferences associated with ICR preparation obtained opinions from the PIUs and MIC/PCU on factors likely to foster the maintenance of installed systems. In the country context, waning interest/investment in the ICT sector is not seen as a risk either at the central or provincial levels, while recent legal directives, resolutions and decisions underscore Vietnam’s commitment to ICT in industrialization, modernization and institutional development. Nevertheless, significant risks include: technical, since the project financed a large amount of innovative technology and systems; financial as indicated by the need for adequate budget to fund operation, maintenance and replacement costs as well as human resources development; and social, the latter equated with citizens’ IT literacy and building sustained demand for the services and facilities implemented. The ICR believes it probable that the achievements of strong performers like Danang and GSO will be sustained, but uncertainty is associated with MIC and Hanoi, the assessed project benefits to date are couched in terms of likelihoods based mostly on similar external experiences, and overall sustainability of the Project must be questioned given that even its restructured stage is rated Moderately Unsatisfactory and several large procurement packages were not completed until the final months. 15 Final supervision missions, and video-conferences associated with ICR preparation discussed with the PIUs and MIC/PCU the factors likely to ensure that project development outcomes (or expected outcomes) would be maintained/realized.

15 The project team opted for a conservative approach to rating the post-Restructuring phase, deciding on MU to acknowledge its continuing difficult trajectory, last minute execution of key procurement packages and lack of time to conduct user surveys, although the vast majority of planned systems, equipment and training were implemented and operational, with substantial cost savings, and with early/preliminary indications of reduced transaction costs and greater efficiency.

22

4.2 Technical: GSO intends to increase public access to its statistical databases, standardize additional data sets and install them in its statistical hub for use across departments, open further databases to public access, maximize the data mining capacity of new systems, and pilot automatic on-demand report generation and provision. Hanoi and Danang have specific plans to expand e-services availability while increasing ICT uptake/usage. In most cases where relevant, suppliers delivered O&M training and both national and local authorities are well-informed on the budget needed to maintain connectivity, especially for Internet. The issue of ICT security - flagged in the Danang study - has lagged and needs attention, while rapid technological change, obsolescence (including of the Enterprise Architecture) and development of a national ICT innovation sector are issues of broad national interest and relevant to sustainability. 4.3 Financial and institutional: Project-financed infrastructure, e-services and public facilities will remain operational. Adequate budget is available in most cases to maintain project-financed equipment, infrastructure and e-services, except for MIC/PIU which is still negotiating with related departments in MIC itself, MOFA and MPT for the budget needed for the rest of 2014. MIC’s task is more complex, covering the public sector, citizens and business. It has requested budget for M&E of e-services at the provincial level. Most project-trained staff have returned to their respective agencies with expected institutional benefits, and public e-facilities are open for business and adequately-staffed. PIU teams are aware that extending e-government facilities to other public services entails risks in terms of the availability of specialized/trained human resources, adequate budgets and demand. GSO is recommending that the Statistical Law be updated to create the mechanism for fee-based information services. Dissemination of project results and lessons to stakeholders and promoting demand is an acknowledged priority. 4.4 Social: Recognizing that citizens’ ICT literacy is comparatively low, Danang launched new Citizens Services Centers (one-stop facilities) to build familiarity with ICT, and provide incentives such as inexpensive computers and laptops.16 Danang also plans to roll out the public services developed under several procurement packages, extend free WiFi services until 2016, complete the migration of public applications to its Data Center and pilot common data-sharing. Hanoi plans appropriate public information on e-services available via TV and radio, and seeks to deepen the behavioral change in government’s interaction with citizens/business via the Hanoi Portal, as well as inter-agency relationships. The rapidly-changing socio-economic environment in Vietnam means that GSO will need faster and more accurate data on trends and thus strong O&M of sophisticated systems is essential and well-understood.

5. Assessment of Bank and Recipient Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Unsatisfactory 5.1.1 Quality at entry was Unsatisfactory overall. Among positive attributes, the Project was timely and responded to defined country needs, plans and policies. The Bank team sought to ensure that design reflected best practice, global research findings and lessons. Evidence suggests that the internal discussion/review of the proposed operation was candid and wide-ranging. The basic technical approach was sound as the Project could not achieve its objectives without the ICT infrastructure, systems and training financed. Even so, project complexity exceeded the

16 GOV data shows that by end-2012, some 35% of the population was using internet (31.2 million people), 4.78 million were broadband subscribers, and 18.8% of households had computers.

23

country conditions/context and promised too much. The ICR suggests that the preparation team should have contained more experienced ICT operational staff. The Results Framework lacked coherence and was not understood by the clients, who lacked even basic readiness to implement. The operation was immensely ambitious for a first-round ICT project and IDA funding exceeded absorptive capacity. There was inadequate acknowledgment of known risks and calibration of the project to existing human resource capacity, i.e., a schedule which took the learning curve into account. Project scope was broad, especially private sector goals lacking a convincing rationale for their inclusion in a publicly-led project. The institutional/implementation arrangements lacked synergy and created operational difficulties and inefficiencies although it appears that the multi-PIU approach had few alternatives under ODA rules. But this should have prompted analysis/negotiation of mechanisms for cross-PIU collaboration. (b) Quality of Supervision Rating: Moderately Unsatisfactory 5.1.2 The Bank Team conducted 11 supervision missions over a seven year period (based on Aide Memoires and ISRs), well under the Bank’s norm, with uneven spacing of missions (e.g., gaps exceeding 12 months from 2009 to 2012). While this does not take into account regular Bank Procurement and FM supervision out of Hanoi, the impression is that at least until the final two years, supervision from Bank headquarters was uneven. Management recommended immediately post-effectiveness that the inexperienced PIUs receive intensive Bank supervision but missions from headquarters did not increase even though broader forms of guidance were needed, especially for the most recently-constituted PIUs. The MTR accurately diagnosed the issues affecting progress but the continuing rigidity of project design – technical, operational and institutional – and the Bank team’s inability to simplify the Results Framework, improve its coherence and measurability, and focus on development impact, meant that the 2010 Restructuring – a direct outcome of the MTR - had only modest influence on the subsequent pace and quality of implementation. The ICR notes that certain design elements planned at appraisal disappeared early on from the project/supervision agenda: regular consultation with the private sector/SMEs; introduction of demonstration models of e-government; focus on business process change and mobilization of private investment in the ICT sector; and, establishing the planned IT Innovation Fund. 5.1.3 Supervision intensity and quality improved in the final two years, as did the realism of project ratings. The change in Bank task management turned the supervision focus onto diagnosing the main constraints, assigning ICT procurement specialists to the PIUs, and building awareness of development impact. This intensity was costly in terms of supervision budget, as noted in 2.4.6. The ICR suggests that in challenging countries with complex operations, the Country Management Unit (CMU) may need to provide a higher coefficient for supervision. High-level Bank/GOV consultations in 2012 on the Bank’s Vietnam portfolio, and subsequent ministerial directives to PIUs provided critical “weight” to the task team’s efforts which included a second restructuring to inter alia, promote organized project measurement via further adjustments to the Results Framework.17 Completion of procurement packages linked directly to the PDO became the top priority. Within this finalization scenario, formal impact evaluation had insufficient time and unsuitable institutional conditions for initiation, although Danang and MIC PCU did conduct studies.

17 The 2013 adjustments were the result of an intensive, thoughtful process to link project indicators to the PDO and to reflect what was happening on the ground. However, the excessive number of indicators was not addressed and internal targets for each indicator were increased to ensure the capture of all achievements. See Data Sheet and Annex 2.

24

5.1.4 Fiduciary supervision was satisfactory overall and accompanied the general supervision effort closely because Bank FM and Procurement specialists were located in Hanoi. Procurement supervision was of outstanding quality in the final 18 months, complemented by ICT Procurement Specialists from Bank Headquarters driving the completion of all planned packages. Safeguards supervision was not relevant to this project. 5.1.5 The rating of Moderately Unsatisfactory for Bank supervision quality reflects the overall performance since Effectiveness. The ICR acknowledges the strong- and ultimately successful - effort by the Bank task team working with the PIUs to salvage this project in the final two years. The high cost of supervision (Annex 4) in FY 12-13 testifies to this effort to complete all procurement packages, ensure that installed equipment/systems were operational and oversee the orderly completion of the Project. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 5.1.6 This rating takes into account major weaknesses in the preparation phase affecting quality at entry, the more mixed supervision performance which included well-diagnosed but in practical terms, overly-conservative and unsuccessful efforts to generate progress. While sustained improvement resulted from exceptional measures to salvage the Project in the final years of a seven year operation, this is considered insufficient to raise the rating for overall Bank performance into the Satisfactory range.

5.2 Recipient Performance (a) Government Performance Rating: Moderately Unsatisfactory 5.2.1 Government of Vietnam: Government commitment to the Project was uneven over time and its performance showed significant shortcomings. On the one hand, Government maintained its policy stance on promoting ICT as an engine of modernization and socio-economic development. However: Government dragged its feet in sending the Bank a request for the essential second restructuring despite repeated reminders; its response to high level Bank consultations regarding the Project - within the context of non-performing projects in the Vietnam portfolio – was the result of the Bank’s insistence on action, not its own proactivity, although its directives to participating agencies did have a positive impact on subsequent performance; Government’s last minute request for further extension of the closing date was unjustified; its overall coordination of project activities was deficient; and, implementation of its overall ICT/e-Government Strategy and its Administrative Reform Program was slow with inadequate coordination between the two and effects on the Project including staffing, salaries and skills development. The e-readiness factor was acknowledged in the PAD but project design was not modified, i.e., the Bank shares some responsibility. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory 5.2.2 MIC (PCU and PIU): The Recipient was formally the State Bank of Vietnam with responsibility for project coordination delegated initially to the Ministry of Posts and Telematics (MPT), subsequently changed to the Ministry of Information and Communications (MIC). Pre-MTR, MIC operated with three distinct project teams (PMU, PCU and PIU), a structure not discussed in the PAD. These teams changed twice early on, affecting continuity and the Project’s

25

institutional memory. Ownership of the Project was inconsistent over time. MIC lacked a clear sense of purpose for its Subproject. The Procurement Plan changed repeatedly and new ideas resulted in long and difficult consensus-building. The PCU staff, although large, was inexperienced in project management, did not work proactively with its technical subgroups and had difficulty functioning effectively as the coordinating unit for the entire Project. The technical subcommittees were unable to develop strategic, implementable procurement plans for the Ministry and consultancies were not effectively deployed. Some 18 international study toursMIC delivered a Completion Report of reasonable quality, limited to its own and the PIU/MOFA activities, not the consolidated version agreed before project closing. Competence, leadership, mobilization and coordination issues to varying degrees persisted throughout. Rated: Unsatisfactory. 5.2.3 Hanoi: Several key issues hindered progress in the first stage of the project, including repeated turnover in team membership, lack of qualified staff for procurement, project management and technical aspects, and ineffective use of consultancies. There were protracted delays in mobilizing to effectively implement the Project. Territorial reorganization of the Hanoi City Government further delayed decision-making and obscured project objectives. Approval processes for procurement packages were long and lacked any sense of urgency. Obtaining timely approvals within DIC and from the PPC was a major constraint. Expansion of Hanoi City meant that the development level was uneven city-wide. The PIU sought to focus on rolling out infrastructure services and capacity-building to bridge the digital divide and ensure readiness of users before developing e-applications. Hanoi was the first PIU to complete finalization of bidding documents under the Bank’s new bidding parameters after 2010 and it made reasonable progress in the final years. Rated: Moderately Unsatisfactory. 18 5.2.4 GSO: This agency/PIU performed well, including in the difficult initial years of the Project, with strong disbursement. Positive factors: (i) GSO’s relative autonomy and sense of a shared mission among its 5,000-odd staff (including 64 provincial offices); (ii) direct reporting and access to the Ministry of Planning and Investments; (iii) leadership and commitment of the GSO Director General and management team, especially for using ICT for Change Management; (iv) strength, diligence and technical qualifications of the GSO PIU, although in the final two years the timeliness and quality of some bidding documents was inadequate; and, (v) ability to leverage expertise of international consultants in procurement. GSO planned/executed a good communications and outreach campaign to sensitize GSO staff and the public on the modernization process. It increased the scope and sophistication of additional tools (from the additional reallocation of US$6.2 m in 2010) for data capture/collection, storage, analysis and dissemination. GSO positioned itself as a center of data intelligence and analysis to benefit government, investors, international agencies and citizens, using new technologies for change management, business process re-engineering, and transparent provision of information to clients/stakeholders. It sought to acquire a state of the art statistical information system with business intelligence tools to improve decision-making for policy-makers, investors and development agencies. Rated: Highly Satisfactory

18 The Hanoi PIU, in its comments on the Bank’s draft ICR, strongly disagreed with this rating stating that it felt its performance had greatly improved after the Mid-term Review and merited a Satisfactory. The Bank does not agree with this assessment and retained the rating of MU. See Section 7.

26

5.2.5 Danang City: The Danang PIU performed exceptionally well throughout and the city became a national ICT leader, a remarkable achievement given its modest capacity when formally assessed in 2004. As noted, demand for IDA funds steadily increased to meet the city’s ICT development vision and plans. While disbursement was initially slow, the PIU rapidly laid the foundation for accelerated execution of its Subproject: (i) conducting a city-wide review of government processes; (ii) identifying specific services to be electronically streamlined; (iii) developing a strategy and roadmap for e-government; (iv) effectively training Chief Information Officers (CIOs); (v) making serious efforts to collaborate – especially on training – with other PIUs; and (vi) conducting international study tours. Success resulted from the PIU’s strong, committed leadership, management and technical skills, and clear procurement timetables. It was able to work with specialist consultancies to design/execute sound plans, and had a good grasp of the status and future of ICT infrastructure city-wide. The PIU had the full support of and good relationships with the PPC Leadership (unlike other PIUs) whose high commitment to using ICT for Change Management promoted faster and more efficient approval of procurement packages. Its overall approach was practical, innovation-driven and stakeholder-aware. Danang recognized the technical expertise and management capacity needed to apply the IT EA on a city-wide scale and developed/extended e-government services well beyond what was anticipated at appraisal through skillful use of available (and additional) funds. Rated: Highly Satisfactory. 5.2.6 HCMC: The HCMC Subproject was cancelled by the 2010 Restructuring. The PIU took a mixed loan/grant arrangement but planned outputs (up to the MTR) were implemented with city budget funds, not IDA. HCMC was relatively advanced in its ICT endowment at the time of the 2004 assessment but even so, e-services were receiving an average of only 6,000 requests per month, modest for a city of 8 million. Despite Bank efforts, disbursements of IDA funds were zero at the time of the MTR. The PIU had prepared no procurement or training plans and had no staff assigned to procurement or FM. The main issues affecting HCMC were: (i) DICs’ limited commitment – there was no meeting with the Bank team for two years; (ii) weak leadership of the Subproject and inadequate supervision by the PPC and MIC; (iii) poor implementation capacity, with frequent staff changes resulting in limited understanding of the Subproject approach and Bank procurement and FM procedures; and, (iv) lack of responsiveness to repeated Bank requests about the city’s plans to use IDA funds. The PIU was barely able to absorb its city budget and had no chance of using its IDA allocation. Rated: Unsatisfactory (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory 5.2.9 This rating takes into account the following: (i) Government’s sustained policy stance on ICT-based modernization but moderate shortcomings in terms of the pace and depth of ICT-related policy implementation relevant to the Project, combined with its uneven commitment and responsiveness over time and weak project coordination; (ii) sustained, highly satisfactory performance by GSO and Danang, balanced by uneven performance of Hanoi and unsatisfactory performance of MIC and HCMC. 6. Lessons Learned

The complexity of project design needs to be balanced carefully against the client’s human resource endowment and readiness to execute. Scope, scale and scheduling are critical considerations in an innovative, foundational project, especially when the client lacks experience with the Bank. While many Bank projects entail “learning by doing”, capacity must be assessed realistically, and minimum professional and administrative frameworks should be in place before

27

effectiveness to launch activities and build momentum. Front-loading selected, strategic activities into the pre-Board stage and focusing time and budget on human resource development are essential. Practical examples include: having bidding documents ready prior to Board approval as part of the primary feasibility studies to promote better disbursement flow in the initial years; allocating a significant portion of the project budget to change management/initiatives; using Project Preparation Facility (PPF) or Trust Funds to put Enterprise Architecture (EA) in place or conduct EA studies before Board approval to ensure the foundation for subsequent procurement; addressing the issue of ICT and especially Internet penetration to promote rapid evolution of e-services into end-to-end transactions and accelerate citizen uptake; and, greater focus on the human behavioral change element which can directly affect project progress and measurement.

Multi-unit institutional arrangements, if unavoidable, demand that the Bank negotiate mechanisms for joint management of essential functions and structured opportunities for collaboration. The objective is to avoid duplication of operational and administrative efforts, inefficiencies, and other well-known “silo” effects. Joint frameworks for procurement, financial management and M&E could have simplified project execution and promoted efficiency without sacrificing human capital formation. This could have been tried in Hanoi where the MIC, GSO and Hanoi PIUs were based. Also, led by Danang and GSO, regular exchanges of experiences and knowledge would have been valuable for Hanoi and MIC.

Cancellation would have limited the knowledge gained from implementing this important project but restructuring and extending it in 2010 without addressing design issues was an inefficient use of Bank resources. The ICR concludes that the first restructuring was too conservative and essentially preserved weak design features contributing to implementation delays requiring further restructuring and entailing high supervision costs. The Project needed overall simplification, re-negotiation of the institutional arrangements - at minimum for FM and Procurement – and a marked focus on building leadership.

Bank procurement requirements, processes and documents are not optimal for the fast-evolving IT sector and this needs review. The Bank’s portfolio of investments in IT systems and equipment tends to perform poorly, importantly due to procurement constraints. The no-objection process can be slow, and Bank support to inexperienced counterpart teams inconsistent. GSO and Danang PIUs reported delays from one to two years in some packages due to the need for adjustments and re-submission to the Bank. Meanwhile, innovation caused technical specifications to become outdated, contracting could not proceed and further bid evaluations and no objections were required, compounding delay. The Bank needs to revise its IT procurement documents and procedures which have repeatedly proven unsuitable/unwieldy for Bank projects in the technologically fast-moving IT sector.

Intervention of senior Bank management with national authorities can galvanize project performance in a failing situation. If the Project team continues to see the operation as worth completing but has been unable to promote progress, high level Bank intervention should be considered. Timing is critical and it may need to occur as a standalone event. Also, the renovation of Bank project task management, bringing in the seasoned ICT specialists essential for projects of this kind and a tighter, more vigorous approach to supervision can effectively salvage an operation, even one with chronic difficulties.

ICT projects need to focus on both the supply and demand sides, the latter being an important driver of longer-term sustainability. Plans to further expand online services require attention to user uptake/adoption through repeated communications campaigns tailored to diverse stakeholders and able to solicit feedback to improve e-services quality. The demand side is still weak for many project-supported e-services even in Danang. Citizens prefer traditional “counter-service” accounting for the modest usage of fully transactional online applications even at the end of the Project. That said, many e-services were implemented only in the final year, and demand is also dependent on computer ownership, access and literacy. Finally, the design of a City Portal

28

needs to allow for year-round citizen feedback and engagement functions, with features promoting the smooth transition from manual to digital/e-transactions.

The rationale for including weighty private sector goals in a foundational project focused on e-government should be clearly defined. Adequate tools should be provided and a specific strategy devised to achieve and measure goals which include private sector development and improved productivity. Having the public sector take the lead in fostering private sector development through ICT usage was not matched by appropriate tools, limited in practice to training and awareness-building.

ICT projects can materially impact the behavior and performance of government, business and citizens and require the same high standards of monitoring and evaluation design and execution as projects in other sectors. Proven evaluation frameworks and methodologies for ICT projects/programs are available. This project demonstrated the implications of a weak M&E framework and inadequate focus on establishing an MIS and a time-bound evaluation strategy and products. Bank-wide by late decade, the focus on M&E quality was much stronger, yet as late as 2012 most PIUs lacked understanding of the PDO/indicators and had not attempted their measurement. The ICR suggests inter alia, that the project team could have obtained an independent, third party, objective review to help re-design the RF at the time of the Level One Restructuring.

That said, it is nevertheless unrealistic to expect that the deeper impacts of newly-constructed ICT systems can be measured by end-project. For foundational projects of this type, two indicators – essentially proxies for impact - should be of primary concern: number and type of services online and, numbers of users or uptake of those services. Global experience shows that a minimum of 3-5 years are needed to permit consolidation and second stage developments before impacts such as transparency, user satisfaction, cost reduction and productivity can be measured meaningfully.

7. Comments on Issues Raised by Recipient/Implementing Agencies/Partners (a) Borrower/implementing agencies 7.1 MIC PCU: In response to the Bank’s draft ICR, the MIC PCU sent a letter (see Annex 7) with brief comments, primarily requesting that the Bank reassess its rating for MIC performance on the basis of two reports accompanying the letter (MIC PCU Completion Report and a Project Performance Audit Report (KDiAsia, 2014), designed to provide internal guidance to MIC). The project team reviewed both documents, finding that they accord with the ICR’s representation of the Project’s achievements and experiences. However, the team’s assessment of the overall performance of MIC PCU/PIU was sustained. 7.2 Hanoi PIU: The Hanoi PIU, in its comments on the Bank’s draft ICR, strongly disagreed with the Bank’s performance rating of Moderately Unsatisfactory stating: “Hanoi agrees that there was slow progress in the first stage of the Project. However, since the Mid-term Review in 2010, Hanoi has improved a lot and we cannot deny its achievements in project implementation. Deeply understanding ourselves, Hanoi clearly knows what we can do and what we cannot do within the framework of this Project. That’s why Hanoi decided to drop some packages and focus on the most important and feasible ones. For the remaining packages, we always completed on committed time and the outcomes of these packages are 100% achieved. So we would like to be rated as Satisfactory." The ICR believes the rating of MU is correct. 7.3 Danang PIU: Comments received were editorial and were incorporated. (b) Co-financiers N/A (c) Other partners and stake-holders N/A

29

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components/Activities Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Subproject 1: ICT Leadership and MPT Modernization (MPT) 21.73 23.17 105.7

Subproject 2: GSO Modernization 13.77 19.59 140.8 Subproject 3: Hanoi E-application 31.20 5.35 17.2 Subproject 4: Danang E-application 17.33 27.56 157.3

Subproject 5: HCMC E-application 10.61 0.02 0.1

Subproject 6: Project Coordination 1.50 1.39 84.5

Total Baseline Cost 96.29 77.08 79.0

Physical and Price Contingencies 4.8419

0.00

0.00

Recurrent costs 5.83 0.00

0.00

Total Project Costs 106.96 77.08 71.0 Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00

Total Financing Required 106.96 77.08 71.0

**Aggregate total US$13.48 million cancelled

(b) Financing

Source of Funds Type of Co-financing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Recipient 13.24 2.9620 22.36 International Development Association (IDA) 93.8621 74.1222 78.97

19 Price and physical contingencies are shown in aggregate in project documents 20 This is a best estimate provided by the Recipient as at June 26, 2014. 21 Amount of US$93.86 m is shown in Bank system as “original signed amount” as at June 26, 2014. 22 Original commitment of US$93.86 m; cancellation US$13.52 m; total disbursed US$74.12 m; undisbursed balance US$6.23 m. All data from Bank system as at June 26, 2014.

30

Annex 2. Outputs by Component This Annex comprises the following: (A) glossary of ICT terminology used by the project and in the ICR; (B) matrix of component outputs by Subproject; (C) matrix showing original and revised Project Outcome Indicators and Intermediate Outcome Indicators (Results Frameworks); and (D) completion status of all procurement “packages” financed by the Project. A. Glossary of terminology used by the Project and ICR Term Technical/Common Meaning Interoperability Standards and Framework

Means to enable public administrations to collaborate; framework for ICT-enabled governance; not only a means of improving public services but of enabling multi-level governance; supports information sharing among ICT-enabled administrations including across borders. (Wikipedia)

Enterprise Architecture High-level strategic technique to help senior managers achieve business and organizational change; a “coherent set/whole of principles, methods and models used in the design and realization of an enterprise’s organizational structure, business processes, information systems and infrastructure (Lankhorst et al, 2005).

Portal/City Portal A specially-designed web-page at a website which brings information together from diverse sources in a uniform way. Usually, each information source gets its dedicated area on the page for displaying information; the user can configure which ones to display. (Wikipedia)

e-Service Role of technology in facilitating delivery of services. “…. deeds, efforts or performances whose delivery is mediated by IT including e-tailing, customer support and service delivery”. Three main components: service provider, service receiver, and service delivery (technology). (Wikipedia)

e-Government “Digital interactions between a government and citizens, government and business/commerce, government and employees, and also government and government/agencies”. (Wikipedia)

National IT Architecture Organized set of consensus decisions on policies and principles, services and common solutions, standards and guidelines as well as specific vendor products used by IT providers both inside and outside an IT entity. IT architecture is designed to plan, acquire, build, modify, interface and deploy IT resources throughout departments. – identifying technologies which work together to satisfy user needs. (Wikipedia).

G2B (Government to Business) G2B is the online, non-commercial interaction between local and central government and the

31

commercial business sector rather than private individuals with purpose of providing businesses with information and advice on e-business best practices.

G2C (Government to Consumer/Citizen)

G2C refers to a communications link between a government agency and the general public. Most often refers to what takes place through ICR technologies but can also include direct mail and media companies. G2C can occur at the federal, state or local levels. (Wikipedia)

G2G (Government to Government) G2G is the electronic sharing of data and/or information systems between government agencies, departments or organizations. The goal is to support e-Government initiatives by improving communication, data access and data sharing. (Wikipedia).

Front and back-end solutions Terms used to characterize program interfaces and services relative to the initial user of both. User may be human being or program. Front end – users interact directly. Back end applications or programs serve indirectly to support front end by being closer to the resource or having capacity to communicate with the required resource.

Meta-data Standard Requirements intended to establish a common understanding of the meanings or semantics of the data to ensure correct and proper use and ensure correct and proper use and interpretation of data by its owners or users. It is “data about data”. Structuring information that describes, explains, locates or otherwise makes it easier to retrieve, use, or manage and information source especially in a distribution network environment, e.g., internet or an organization. (Wikipedia)

Transaction Centers City-level facilities for private sector (esp. SMEs) to obtain hands-on training, view exhibits, and get exposure to new tools and technologies, highly-specialized training and business applications.

Tier 1 Applications In project context (Hanoi): Integration/linkage of departmental portals and websites to the Hanoi City Portal

Tier 2 Applications In project context (Hanoi): Integration of public services for citizens via Internet.

Tier 3 Applications In project context (Hanoi): E-government services processed at One Gate Service (business registration; reissuance of birth certificate; temporary construction license; new car registration etc)

Tier 4 Applications In project context (Hanoi): E-government services which require online payment (declaration and payment of income taxes; customs declaration etc).

Semantics Semantic technologies are often proposed as

32

important components of complex, cross-jurisdictional, heterogeneous, dynamic information systems (CSIRO).

Organizational Interoperability Modeling organizational processes, aligning information architecture with organizational goals and helping these processes cooperate via ICT. Has specific relevance to e-government, i.e., not only technology but a broad concept of modernization of public administration including new skills and cultures. (Kubicek, Cimander and Scholl).

Telecom Sector Roadmap Technology/telecom roadmap is a plan that matches short-term and long-term goals with specific technology solutions to help meet those goals. Three major uses: helps reach consensus on a set of needs and technologies required to satisfy them; provides a mechanism to help forecast technology developments; and, provides a framework to help plan and coordinate technological developments.

LAN Local Area Network WAN Wide Area Network MAN Metropolitan Area Network Metronet Fiber-optic network delivering high-speed Internet CERT Computer Emergency Response Team (for Internet)

for computer security incidents Root Certificate Authority In ICT computer security, a root certificate is either

an unsigned public key certificate or a self-signed certificate that identifies the Root Certificate Authority as part of a public key infrastructure scheme. The most common commercial variety is a digital signature from a Certification Authority (CA).

eGovFrame (Korea) E-Government Standard Framework developed in Korea. Is an open-source software development framework “eco-system” which ensures interoperability needed for information-sharing and offers standardization and re-usability. Facilitates development of applications based on open standards, resulting in improved software quality, more efficient development, and greater re-use of software modules.

Application Architecture Framework of strategies and plans for developing e-services. Not solely about ICT infrastructure to build online services but sets out all the structures – business processes as well as technology – for effective e-services.

33

B. Outputs by Component

The Project comprised five “subprojects” with internal “components” in each case, and covering three development themes: (i) ICT leadership and modernization of the MPT; (ii) modernization of the GSO; and (iii) e-applications for the Municipalities of Hanoi, Danang and Ho Chi Minh City (HCMC). Changes were made to some components by the 2010 Restructuring including canceling the HCMC subproject altogether.

Subproject 1: ICT Leadership and MPT (MIC) Modernization

This subproject sought to strengthen the capacity of the Ministry of Posts and Telematics in development, implementation and oversight of Government’s national ICT strategy and policies; strengthen its capacity to formulate telecommunications and ICT policy; and, improve its operating efficiency and effectiveness. The ICT capability of other government ministries would also be improved via the function of the CIO (Chief Information Officer). There were six components: Development of e-Government capability in MPT; Creating an MPT Flagship Portal; Creating a center for training, improving and managing CIOs; Training in ICT Policy and Regulation; IT and Telecommunications Regulation; and, IT Architecture and Interoperability Framework for e-Government. The ICR notes that the original description of the components in the PAD, Annex 4 shows Subproject 1 with 6 components and 32 individual activities, many of which show no clear link as outputs supporting the Subproject-based Intermediate Outcome Indicators in the Results Framework. This list was modified by the 2010 Restructuring and the gap with the also-revised RF seems to have widened.

Component 1.1: Development of e-Government capability in MPT (changed to MIC) Level of

Attainment Planned Actual 1. Review and upgrade MPT’s (MIC’s) current ICT infrastructure and assess future needs to provide e-Government services

MIC’s ICT infrastructure was reviewed and future needs determined. E-government framework, policy and standards set up. Enterprise Architecture and interoperability standard formulated. E-government framework and security completed. Upgrades included: LAN network in MIC and 12 DICs with user acceptance tested and end-user training conducted. IT equipment for smart building communications system installed between MIC and 4 agencies (Regional MIC offices in Danang and HCMC, Telecom Authority in Hanoi, and MOFA). Video-

Achieved

34

conferencing, IP phone and building security system installed and operational.

2. Assess and upgrade the existing DPT (changed to DIC) ICT infrastructure

PCs, laptops and printers, email, video-conferencing and LAN/WAN acquired/installed for 12 DICs (District Offices of the Ministry of Information and Communication). In 12 DICs, 83 LANs built, IP phone system connects all departments and 4 MC departments outside.

Achieved

3. Re-engineer governmental process flows to facilitate and increase interaction within MPT and with related government entities such as DPT (DIC)

Video-conferencing system is facilitating more frequent and efficient communications between MIC and DICs; standardized email system also implemented.

Achieved

4. Create an e-library to incorporate current and new documentation for the MPT. Provide other ministries the opportunity to access the e-library.

e-Library not established. Activity dropped. Not achieved

5. Installation of intelligent system and united communications

New component (2010). Intelligent/smart building system was installed at MIC HQ in Hanoi and MIC Data Center in Hanoi. United communications via email, VC, LAN/WAN installation.

Achieved

6. Design and manage pilot programs for two to three informational e-services

Reduced from 7 to 3 in 2010. MIC implemented 3 licensing e-services/transaction launched in May 2013: radio spectrum, publications, and registration of imported publication portfolio for publishing companies. Paper-based applications no longer accepted.

Achieved

7. Develop complementary informational, interactive and transactional services

Internal email system, Video Conferencing system and MIC Portal developed.

Achieved

8. Combine pilot project experience and develop additional e-services

Also, visa online application system installed in 95 visa offices worldwide – piloted in September 2013 and now fully-deployed.

Achieved

9. Establish in-house or outsource management of central infrastructure, including gateway, portal and content

Achieved through the MIC overall ICT upgrade. Central infrastructure (Gateway, portal and content) established, and managed by combination of in-house and outsourced professionals.

Achieved

10. Designing a Public Key Infrastructure (PKI) scheme to support e-government and develop a national e-authentication framework

Activity dropped Not achieved

35

11. Implementation of the upgrade of the Root Certificate Authority (CA)

Root Certification Authority completed and end-users trained Achieved

12. Technical assistance on e-government framework and IT applications standards

Technical assistance om e-government framework and IT applications standards implemented.

Achieved

13. National Computer Emergency Readiness Team (CERT)

Removed by 2010 Restructuring Dropped

14. Building a test center to evaluate IT products/solutions New (2010). Completed November 2013. Center used for testing software from Yen Bai province and e-office.

Achieved

15. Consultancy on solution and implementation plan for e-document interchange system and on solution and implementation plan for integrated e-mail system

New (2010). Consultant contracted and plans for e-document inter-change and integrated email system developed.

Achieved

16. Upgrade of ICT infrastructure for MIC and selected DICs

See #1 and #2 above Achieved

17. e-Gateway for MIC E-Gateway (phase 2 of MIC’s Portal) completed and operational (www.mic.gov.vn)

Achieved

18. Technical assistance on telecommunications and ICT policy and regulations and on sector monitoring and evaluation system

TA contracted and supplier-delivered for telecom. 50 MIC staff trained in ICT policy and regulations. Modest progress on sector M&E.

Partially achieved

1.2: Creating an MPT (MIC) Flagship Portal 19. Design the enhancement and expansion of the existing MPT (MIC) website

Enhancement and expansion of MIC website designed. Achieved

20. Develop and activate MPT (MIC) informational portal MIC Informational Portal developed and activated Achieved 21. Develop and activate MPT (MIC) interaction portal including complaints management framework

MPT/MIC flagship Portal established. No information re complaints management

Partially achieved

22. Develop and activate MPT (MIC) Transactions Portal MIC Transactions Portal developed and activated Achieved 1.3 Creating a center for training, improving and managing CIOs Selected items to be supported: 23. Design CIOs and IOs job descriptions and responsibilities

CIOs and IOs job descriptions and responsibilities designed. Achieved

24. Design and conduct CIO and IO training for about 400 CIOs

MIC conducted CIO training in Singapore and also invited international trainers to Vietnam to conduct larger scale training. 497 CIOs/IOs trained (277 from Ministries, 150 from state-owned enterprises, 70 from SMEs).

Achieved

36

http://www.mic.gov.vn/

25. Develop the next IT Master Plan (2006-2011) and assist M&E of the National ICT Strategy

IT Master Plan developed. M&E of National ICT Strategy not established.

Partially achieved

26. Establish awareness-raising ICT program and design National Roadmap via public and private associations, NGOs and academia

ICT awareness-raising programs established for stakeholders including citizens and SMEs. National Roadmap not designed.

Partially achieved

1.4 Training in ICT Policy and Regulation 27. Develop and design ICT literacy and technical training for MPT (MIC) and DPT (DIC) staff. Increase ICT literacy of MPT (MIC) staff and DPT’s (DIC) middle and top management.

ICT literacy and technical training designed/delivered. Actual increase in ICT literacy at middle and top management levels is likely to have increased given type of training but was not measured.

Partially achieved

28. Develop and provide telecommunications policy training of MPT (MIC) and selected staff of DPT (DIC) – telecommunications regulatory and ownership issues.

Telecommunications regulatory and ownership issues were part of telecom policy training for MIC and DIC staff.

Achieved

29. Develop and provide training for top management and executives of MPT (10-20% MPT (MIC) staff) on international economic and economic integration, telecom law, international treaties and trade agreements with foreign governments, and the functions and responsibilities of international organizations in ICT.

497 CIOs and IOs trained in diverse aspects of ICT including economic integration, telecom law, trade agreements etc.

Achieved

30. Provide MPT (MIC) top/middle management (30-40% staff) and top management of DPT (DIC, 10-20% staff) with ITC HR training – leadership, time management, strategy and policy-making etc.

497 CIOs and IOs trained in diverse aspects of ICT including leadership, strategy and policy-making

Achieved

31. Organize international study tours and training trips. 18 international study tours conducted to Europe, USA, China, japan, Australia and Korea. Training in ICT services development and delivery, spectrum management, software outsourcing, information security, database development, and knowledge exchange.

Achieved

1.5 IT and Telecommunications Regulation 32. Assess Vietnamese telecommunications regulations and policies – goal was to gradually adapt Vietnamese telecom regulations to international standards.

Partial assessments conducted Partially achieved

33. Address telecom regulatory and policy issues on a per

37

needs basis. Conduct overview training for key MPT staff on selected topics and facilitate knowledge transfer. 34. Review international best practice operations guidelines and responsibilities of a CERT – ownership, organizational structure and security, including of existing systems.

This activity was removed from the project in 2010 but completed by government with own funding.

Achieved outside the project

35. Review international best practice framework, operational guidelines and responsibilities of Certification Authorities. Provide applied standards to set up a national certification authority to issue/verify digital certificates.

International best practices reviewed and CA Authorities assessed. Root Certification and Test Center 100% completed by end-2013 (MIC), and end-users trained. Systems for digital certificates and signature developed and transferred.

Achieved

36. Review international best practices for setting up four public key infrastructure main entities – certification authority, registration authority, certificate directories, and certificate management system.

See #35 above. Achieved

1.6: IT Architecture and Interoperability Framework for e-Government 37. Review international experience and best practices International experience and best practices reviewed Achieved 38. Review existing plans and documentation in Vietnam, including National IT Master Plan and draft e-Govt Plan; city-level information about IT architecture in Hanoi, Danang and HCMC and its IT Master Plan.

Existing plans reviewed. MIC completed National IT Master Plan. Hanoi and Danang completed city-level IT Architecture and IT Master Plan.

Achieved

39. Consult with key stakeholders and user groups involved in national and city level IT and e-Government strategy and implementation

Involved user groups and key stakeholders consulted on national and city-level IT and e-Government strategy and implementation

Achieved

40. Identify high level options for IT strategy, architecture, roadmap and specifications

High levels options partially identified by end-2013 – mainly IT strategy and architecture

Partially achieved

41. Develop a document on the common IT architecture and inter-operability framework for Vietnam

Document developed. See #1 above. Achieved

42. Set up standards for IT systems Standards for IT systems established Achieved 1.7: MOFA under MIC 43. Data Center Added 2010. Data Center completed and operational Achieved 44. Implementation of online visa services Added 2010. Online visa application service established and

functional in 95 countries. Achieved

Subproject 2: GSO Modernization

38

This subproject was to support implementation of a strategic plan for modernizing GSO including improving the quality, transparency and timeliness of socio-economic and poverty-related data and information to fully meet the demands of government agencies, research institutions, and other data users as the basis for policy planning and decision-making; to modernize the system to reach and advanced level in the region; and, to improve data flow, statistical analysis and dissemination of useful data and information. There were three components. Outputs were as follows:

2.1: ICT Infrastructure Upgrade at GSO Level of

Attainment Planned Actual 45. Review, streamline and automate internal workflows within GSO and between GSO and PSOs to facilitate information dissemination, workflows and processes, and create efficiency. Devise and implement an appropriate architecture and strategy.

Achieved through ICT infrastructure upgrade. GSO Enterprise Architecture in place and all IT initiatives at GSO required to follow this standard. Information dissemination, workflows and processes facilitated by: VPN connection between GSO HQ, all PSOs and 3 ITCs using MPLS and public internet connection; new email system used throughout GSO ensuring secure document exchanges. Network connectivity for GSO Enterprise Applications improved/more secure.

Achieved

46. Assess and define future needs for GSO’s networks, strengthen network security and review needed hardware (databases) and software (GIS) options.

Achieved through ICT infrastructure upgrade. Network security improved (see above). Hardware and software options reviewed. Gap bridged between GSO plans and IT for centralized and coordinated IT services completed via SSIC and SHS; effective IT and priority-setting decisions based on project activities.

Achieved

47. Define course standards and design courses to train GSO employees.

Course standards defined and courses designed for GSO employees. Training conducted/completed for GSO staff on new sector databases, database standardization and integration, network administration, and SSIC system. GSO statisticians trained in MS Office; IT experts trained in SQL/Sharepoint etc. Total 690 GSO staff trained including at provincial level.

Achieved

2.2: Data Flow and Statistical Analysis Enhancement through ICT 48. Review methods of data collection and processing, assess and upgrade PSOs’ and selected district statistical offices’ ICT equipment and provide relevant training in

Completed implementation of System of Statistical Information Collection (SSIC) in GSO HQ in Hanoi, and three IT Centers in Hanoi, Danang and HCMC. User training conducted. 3 out of

Achieved

39

data collection and analysis 9 planned databases developed (population, SME and BDS); data standardization and integration methodology developed; population reports automatically generated; SSIC system implemented at GSO HQ and 3 ITCs. End-user training conducted.

49. Re-design GSO’s questionnaires and introduce an optical scanning system to make data entry more efficient. Establish system at HQ and train employees for 2009 Population Census.

Achieved with significant results. Questionnaires re-designed. Optical scanning infrastructure introduced. Efficiency gains: Data collection and processing for 2009 Population Census took 8 months (33% time reduction); for 2011 Agricultural and Rural Census (50% improvement in overall processing time, from 2 years to 1); and for CPI survey (data collection and processing time reduced 33% - from 15 to 10 days).

Achieved

50. Assess and upgrade ICT infrastructure at PSO and DSO levels.

All PSOs and ITCs connected with GSO HQ via VPN using MPLS and public internet. Data transfer between GSO and PSOs improved. Network connectivity for GSO Enterprise Applications.

Achieved

51. Develop and provide necessary ICT equipment training at PSO level; develop and provide statistical analysis training for GSO and PSOs to ensure use of internationally accepted analytical tools.

122 CIOs trained and certified; 83 GSO staff and IT experts from 3 IT Centers trained in Network Administration; 395 GSO and 63 PSO staff/statisticians trained in MS Office; 27 IT experts in 3 IT Centers trained in SQL/Share Point/New Window Server. Total 690 trained.

Achieved

52. GIS application support mapping administration field office.

GSO established field office as specified Achieved

2.3: GSO Portal and Intranet Creation 53. Re-design and development of architecture and functionalities of existing GSO Intranet Website and Portal

GSO Intranet Website and Portal re-designed including architecture and functionality. New portal, Statistics Hub System (SHS) information dissemination portal was implemented and is operational

Achieved

54. Facilitate GSO internal processes, workflows and procedures through the Intranet and Portal.

Achieved through the Statistical Hub. It is set up in-house (GSO Data Center in Hanoi). All GSO statistical data domains have integrated with GSO Portal and basic statistical data is provided to the public free of charge. 50% of GSO data have

Achieved

40

been integrated into Statistical Data Warehouse. 55. Make accessible on the Internet more accurate and recent country statistics, statistical analyses and findings on Vietnam which are (were) delivered in hard copy format. Update information more regularly for better targeted and informed policies.

Achieved through the new Portal. GSO plans to conduct regular update of information.

Achieved

Subprojects 3, 4 and 5: E-Government and E-Development:

Subprojects for Hanoi, Danang and Ho Chi Minh City e-Government had similar objectives and components, supporting activities to enhance the ICT environment in these cities, and providing the basic foundation for deployment of e-Government in their district offices and departments. They were expected to facilitate the development of various types of e-government services, e-applications for business, and development of ICT skills. These were expected to have a positive impact on government accountability, effectiveness, transparency, access to information by citizens and the business community, citizen and business interactions, and public administration reform efforts. The competitiveness of SMEs was expected to improve, with positive spillover effects on their trade opportunities. Despite broad similarities, individual subproject design reflected the different status of ICT development in each city and their particular application needs. Outputs were as follows:

Subproject 3: Hanoi e-Government Level of

Attainment 3.1 Identify Priorities, Key Future Requirements and Develop a Strategy Planned Actual

56. Assess key ICT infrastructure and applications, analyze and evaluate current level and demand

Key ICT infrastructure and applications assessed as well as level and demand.

Achieved

57. Identify priority business processes/services 14 priorities identified: Dept Planning and Investment (issuing investment certificates to firms); Dept. Industry and Trade (issuing certificates to establish branch representative offices to foreign firm); Hanoi Police (registration of car and motorbike); Dept. of Health (grant certificate to practice medicine); Dept. Transportation (grant/change driver’s license); Dept. Information and Communications (grant certificate for importing non-business printed publications); Dept. Construction (construction permits); Districts (grant food safety certificates); Dept. Health (issue certificates for food

Achieved

41

standards); Dept. Justice (issuing judicial history); Districts (grant construction license); District (provide land information from cadastral records); Districts (issue certificate of housing and land use and ownership rights).

58. Develop clear and comprehensive e-Government strategy, roadmap and city-level IT architecture for e-Government in Hanoi. Also covered will be specific issues such as portal layout and taxonomy, operating systems, languages, approach to applications, security, interpretation, standards etc.

Strategy, Roadmap, city-level IT Architecture Framework and IT Standards were developed for Hanoi e-Government.

Achieved

3.2: Local e-Government Enablement 59. Upgrade and expand ICT infrastructure Deleted from the project in 2010 restructuring due to self-

funding by government Cancelled

60. Develop and implement core elements of the shared IT architecture including Data Center

Added in 2010. Core elements of shared IT architecture developed. EA Master Plan designed for 3 levels of government (Department, Ward and Commune). City-level architecture designed. Aligned IT architecture designed for 57 Departments. Data Center completed and operational.

Achieved

61. Develop, expand and implement gateway and city portal, automate information, and provide basic municipal information and services via the portal. This would include some of the over 200 services provided by Hanoi’s municipal government, as well as information about these services and relevant procedures.

Data Center to host City Portal completed and related software installed. 18 Department Portals and 16 applications integrated with Hanoi City Data Center. One-stop software for government agencies integrated in Data Center. Easier, more reliable access to government information. Public services integrated via internet (20 services at 2nd level, 8 services at 3rd level, 2 services at 4th level).

Achieved

62. Manage portal and gateway infrastructure Portal and gateway infrastructure located in the Hanoi Data Center and managed there.

Achieved

63. Implement process improvements and integrate key departments and offices

57 departmental portals and websites integrated. Public services integrated via internet (see # 61).

Achieved

64. Validate and/or develop business model for new ICT centers

Originally included in HAN 9 package but after consideration, was dropped

Cancelled

65. Establish and equip ICT and social/shared facilities in the new ICT Transaction Center

Dropped because Hanoi decided to use local funds for its implementation. (ICT and social/shared facilities in new ICT

Cancelled

42

Transaction Center were established and equipped using own funds).

3.3: Implementation of e-Applications (and transaction center) 66. Implement e-procurement application involving 3-5 different departments: provide e-procurement information via an e-procurement portal to businesses (stage 1).

Dropped because already implemented by a project of the Ministry of Planning and Investment.

Cancelled.

67. Implement online land and population management registration processes

E-services developed after definition as business priority. See # 57

Achieved

68. Implement e-applications, e.g., driving licenses and representative office permits

E-services developed after definition as business priority. See # 57

Achieved

69. Further develop direct (offline) and online services for the public, government and businesses

20 public services integrated via internet – easier, more reliable access to government information

Achieved

70. Develop GIS and GIS-related e-services This is planned to be implemented under a proposed 2nd phase project

Cancelled

71. Develop new/alternative access channels, such as kiosks or online booths (e.g., in post offices) through which the city’s portal and e-government services can be accessed by citizens and business people

This is planned to be implemented under a proposed 2nd phase project

Cancelled

3.4: ICT Human Resource Development and Awareness-Building 72. Develop a Master Plan for ICT human resource development in Hanoi and design a training program for the next five years. Facilitate and conduct consultations to define priority areas, select pilot locations, delivery modalities, standards and other program cornerstones.

This was cancelled due to the availability of government funding. Government completed the study by itself.

Cancelled

73. Upgrade existing training facilities and equip the training centers with ICT equipment and supplementary facilities required to deliver training courses.

See # 72 Cancelled

74. Improve and expand existing training capabilities and competencies: selected foreign and local specialists provide “train the trainer” program in specific ICT-related and e-government areas.

See # 72 Cancelled

75. Develop training material (software, text books, computer-based training, online and multimedia) and

This activity was cancelled by the 2013 Restructuring (HAN 9). It does not include implementation of the Master Plan or

Cancelled

43

databases, coordinate with ICT training facilities. training program itself and is not related directly to PDO achievement.

76. Provide general ICT training and awareness programs for government officials in advanced ICT, good governance, customer orientation/communication, desktop and functional applications and project management (management by objectives, M&E).

30 government leaders trained in Singapore in advanced ICT, governance, project management /other via 5-day course: “E-Government towards Public Service Excellence” organized by Civil Service Collage, Singapore. Target was 340 leaders and key officials trained. Also, 100 SMEs trained in CIO but using local funds.

Partially achieved

77. Provide specialized ICT and ICT-related training on specific topics for all three target groups but esp. government and the business community.

See # 76 Partially achieved

Subproject 4: Danang E-Government 4.1: Identify Priorities, Key Future Requirements and Develop a Strategy 78. Assess key ICT infrastructure and applications, analyze and evaluate current level and demand

Achieved

79. Identify priority business processes/services and departments/districts: identify which core processes, information and services should be e-enabled and prioritize within relevant departments

Priority business processes and services identified. Based on this assessment, Danang PIU implemented 21 e-services (e-government platforms).

Achieved

80. Develop a clear and comprehensive e-Government strategy, roadmap, and city-level IT-architecture for e-government in Danang.

Danang e-government strategy, roadmap and city-level Enterprise Architecture (EA) in place.

Achieved

4.2: Local e-Government Enablement 81. Upgrade and expand ICT infrastructure Target was extended in 2010 Restructuring and exceeded the

revised target. Danang Metro Area Network (MAN) completed; public WiFi completed; 165 public WiFi access points available to citizens free of charge until pricing model introduced; Danang Data Center completed and operational; e-Learning portal completed and operational.

Achieved

82. Develop and implement the core elements of shared IT architecture, including core facilities (data center and administrative systems). Develop and set up security (online and offline) and interoperability standards and

Core elements/facilities developed and implemented. Security systems and interoperability standards/systems and back-up implemented. See Danang study (Leading Associates/KDiAsia 2014) on Danang security.

Achieved

44

systems, and back-up systems. 83. Develop and expand city portal, automate information flow, and provide basic municipal information and services by portal

Contact Center established providing information on public services and social/economic issues; Danang MAN connecting 95 Departments/agencies, Communes and Wards; 32 Internet Stations installed in Communes/Wards where such services limited; 10 information kiosks for tourists; 165 public WiFi access points installed and additional points commissioned.

Achieved

84. Implement workflow solutions and integrate key departments and offices: implement standardized and integrated workflow solutions and applications for all key departments (3-5) and district offices

90 departments/agencies provided with video phones and IP phones; video conference service established; Business Process database developed and commissioned; MIS developed/operational; e-Wards/e-Districts completed/commissioned; 100% of one-stop shop used by Wards and Districts connected to MAN and transferring data to Department of Home Affairs. All identified workflows and reporting lines automated.

Achieved

85. Validate and/or develop business models for ICT transaction center: review plans for ICT transaction center and make required revisions. Incorporate best practice or innovative ideas from other countries.

Business models validated, plans developed and reviewed/revised and best practices incorporated. Based on this, ICT Transaction Center was developed and is now operational.

Achieved

86. Set up ICT and social/shared facilities (administration rooms and offices, conference center, training facilities, serviced offices, data center, meeting and training rooms) in ICT Transaction Center

ICT and social/shared facilities established in Danang ICT Transaction Center

Achieved

87. Manage portal and gateway infrastructure (outsource or joint management)

Data Center was selected (construction of data center was added by 2010 Restructuring), completed and is operational and under management both jointly and via out-sourcing when needed.

Achieved

4.3: Implementation of E-Applications and Transaction Center 88. Implement e-procurement application involving 3-5 different departments: Provide e-procurement information via an e-procurement portal to businesses.

E-Procurement application was cancelled by project but implemented with government’s own funds.

Cancelled

89. Implement e-business registration/licensing e-application: implement an online business registration and

Business Registration System and Investment Project Management System completed development phase and

Achieved

45

licensing process integrated with the gateway and city portal.

launched/operational in January 2014. Integrated with the Gateway and City Portal.

90. Implement 2-3 additional e-services such as drivers’ license registration

Total 21 e-services launched and operational including: e-Drivers’ License system; e-Construction Permits system; Public Transport Management System commissioned with 100% of buses equipped with GPS, managed and monitored centrally by DOT based on GIS map; Water Monitoring System operational with raw water monitored and controlled 24/7 based on best practice criteria before treatment and processing.

Achieved

91. Further develop direct and online services by ICT Transaction Center including equipment and services

See #90 above. Achieved

92. Develop e-applications and e-commerce market/environment for SMEs: promote local software industry by developing/setting up standard applications, e-markets etc. Danang should link to USAID’s software cluster development projects in Hanoi and HCMC

SMEs sell-informed about services provided by the Contact Center; e-Learning Portal launched with 400 courses available in ICT and soft-skills – 15 modules targeted specifically to SMEs. Software developers trained in Project Requirement Management; 5- software developers from SMEs joined e-Government Platform courses. Basic IT Knowledge program televised 3 times/week (68 sessions available). Program contents uploaded to website (www.phobienkienthuc.dnict.vn) and 27000 viewer hits by end-2013 (about 404 viewers average per/lesson. Danang Training center conducted 27 training courses with 1,184 participants of which 773 online.

Achieved

93. Develop alternative/new access channels such as kiosks or online booths (e.g. in post offices) through which the city’s portal and e-government services can be accessed.

Danang PIU developed Danang City Wi-Fi and mobile apps for the City Portal and e-Government services, more convenient and accessible 24/7.

Achieved

4.4: ICT Human Resource Development and Awareness Building 94. Develop a Master Plan for ICT human resource development in Danang and design a training program for the next 3-5 years. Define priority areas, select pilot areas (locations), delivery modalities, KPIs, standards and other program cornerstones

Danang’s ICT HR Development Master Plan was developed and approved by People’s Committee.

Achieved

95. Upgrade and/or expand ICT training programs in educational institutions; implement ICT infrastructure for

Danang PIU developed ICT training modules for SME and private sector. SMEs well-informed about services provided by

Partially achieved

46

http://www.phobienkienthuc.dnict.vn/

ICT training-oriented programs at educational institutions in Danang. Develop training programs and applications for use in classrooms.

Contact Center. 268 SMEs took e-learning courses and over 100 took basic IT training courses. Information on ICT programs in educational institutions/classrooms not available.

96. Expand facilities of training and applied research centers (in ICT Center): set up and equip a training center and applied research center with supplementary facilities (labs, conference, broad band access, online and physical library) within the Transaction Center.

Training and applied research facilities set up in Danang Transaction Center.

Achieved

97. Improve and expand existing training capabilities and competencies: selected foreign and local specialists, “train the trainer” program in specific areas

Training competencies upgraded. Foreign and local specialists contracted under the Project to train selected IT specialists in Danang.

Achieved

98. Develop training programs and materials (software, textbooks, online and multimedia) and databases on various topics, coordinate with ICT training facilities

Danang PIU developed online e-Learning modules. See # 100. Also, training sessions were broadcast by local TV station with recorded sessions uploaded to City Portal.

Achieved

99. Provide general ICT training and awareness programs for government officials in advanced ICT, good governance, customer orientation/communication, desktop and functional applications, and project management

497 CIOs and IOs trained (277 from Ministries and 63 DICs, 220 CIOs/IOs from state enterprises). See # 100

Achieved

100. Provide specialized ICT and ICT-related training in specific topics for all three target groups, especially government and business.

e-Learning Portal operational (http://multimedia-lab.org); 400 courses in ICT and soft skills available in e-Learning portal; 75 software developers trained in Software Engineering, Software Project Management, and Software Testing; 27 government leaders received CIO training; 25 engineers trained in Integrated Circuit Design.

Achieved

Subproject 5: Ho Chi Minh City E-Government 5.1: Identify Priorities, Key future Requirements and Develop a Strategy 101. Identify priority business processes/services and departments/districts: identify which core processes, information and services should be e-enabled and prioritize these within relevant departments

Government-funded. Cancelled

102. Develop clear and comprehensive e-government strategy, roadmap, and city-level IT architecture for e-government in HCMC

Same Cancelled

47

http://multimedia-lab.org/

5.2: Set up ICT and Social/Shared Facilities in New ICT Transaction Center 103. Review plans for ICT Transaction Center (terms, services, facilities, business model etc) and make any required revisions. Incorporate best practice or innovative ideas from other countries

Same Cancelled

104. Equip facilities such as social, shared, and administration rooms and offices, conference center, meeting and training rooms

Same Cancelled

105. Develop and set up security (online and offline) and interoperability standards and systems. Implement middleware, data warehouse, or central databases required for shared services under the IT architecture

Same Cancelled

106. Develop, expand and implement gateway and city portal, automate information, and provide basic municipal information and services via portal

Same Cancelled

107. Implement standardized and integrated workflow solutions and applications for all key departments (up to 5) in district offices.

Same

Cancelled

5.3: E-Application, E-Commerce 108. Provide e-procurement information via an e-procurement portal to business

Same Cancelled

109. Implement 2-3 e-applications in the areas of vehicle registration, driving licenses, representative office licensing. Further develop DPI’s online business registration and licensing process, integrate it with other departments to improve the online process and link with gateway and city portal

Same Cancelled

5.4: ICT Human Resource Development and Awareness Building 110. Develop a Master Plan for ICT human resource development in HCMC and design a training program for the next 5 years. Identify priority areas, select pilot locations, delivery modalities, standards and other program components

Same Cancelled

48

111. Select foreign and local specialists to provide “train the trainer” program in specific areas related to ICT and e-government. Coordinate with MPT on CIO/IO training program (PIU/city-level training, MPT/national training). Provide training for ICT Transaction Center Staff.

Same Cancelled

112. Develop training programs and materials (software, textbooks, online and multimedia) on various required topics, develop databases. Test these in small groups and refine based on feedback

Same Cancelled

49

C. Results Framework: Original (PAD) and Restructured (2010 and 2013) Note: Indicators are listed as an autonomous “set” resulting from each Restructuring. Revisions, deletions and/or additions are not aligned with previous versions. See Data Sheet for details on adjustments over time to individual indicators. Original (PAD) Revised (2010 Restructuring) Adjusted (2013 Restructuring) Project Results Indicators Increase in share of ICT sector in GDP by 2 percent Increase in ICT capacity and knowledge in respective government departments 15 percent use ICT for business operations in 2010 and improve productivity Domestic IT service sector grows by an average of 30 percent every year Increase in standardization and improvement in transparency and administrative processes Significantly lowered transaction costs for targeted government services (G2C and G2B) Increase in user satisfaction for e-government services and visits to City Portal

Improved formulation and implementation of ICT policies and common standards Reduced transaction costs, increased transparency and users’ satisfaction of selected government services through e-application G2B services made available online More private firms in selected cities use ICT

Formulation and implementation of ICT policies and common standards in MIC (R) Upgrading ICT infrastructure to support e-government technical capacity in MIC, GSO, MOFA, Hanoi and Danang (N) Reduction of time required for one transaction of selected government services through e-applications (Hanoi, Danang) (R) Number of e-services and socio-economic information available through e-Gateway (MIC), portal sites (GSO, Hanoi and Danang) (N) Number of G2B services available online (R) Increase the number of private firms using ICT and e-services in Danang (R)

Subproject 1: ICT Leadership and MPT Modernization (MPT)

ICT Leadership and MIC Modernization (MIC)

Workflows and reporting lines streamlined through hardware upgrades and process automation MPT and DPT IT architecture in place, including flagship portal allowing interaction and transactions between MPT and government, business and citizens

E-government framework and common IT standards issued for all government agencies ICT policy and implementation capacity development in at least two major areas Improved ICT awareness and management capacity through training programs Improved MOFA e-visa services under MIC

Design and implementation of e-government framework, common ICT standards and ICT policies Completion of ICT infrastructure upgrade to enhance communications between MIC and DICs, and MOFA (R) Completion of 3 licensing e-services and the visa online service offered to businesses, citizens and foreigners available online (C)

50

At least 7 appropriate services made online by 2010 through the flagship portal Reduced transaction costs (# days for response turnaround) on average for the 7 e-services offered through the flagship portal. IOs and CIOs from various government agencies and organizations trained in ICT issues relevant to their position and responsibilities (technical, legal, knowledge-sharing) A CIO forum established to engage CIOs and MPT staff in the next IT Master Plan (2006-11) and general ICT Roadmap MPT officials trained in ICT policy and regulatory issues and are knowledgeable about best practices in the sector Selected DPT staff trained in ICT proficiency and usage Improved security around e-services through establishing CERT, CAs and public key infrastructure that are inter-operable and in line with international standards Improved Monitoring and Evaluation of the National ICT Strategy E-government framework and IT standards in place to guide nationwide e-government applications and services

oversight Enhanced MIC and DICs communications through ICT infrastructure upgrade Core MIC services for quality and security assurance 3 licensing e-services offered to businesses and citizens available online 200 CIOs/IOs trained 200 MIC staff trained in policy formulation, capacity-building and implementation guidelines for IT and telecommunications sectors

Time savings of new systems (3 licensing e-services and visa online service) (N) Improved ICT awareness and management capacity through training programs (C) Quality and security assurances for ICT services in Vietnam improved through core MIC services (R)

GSO Modernization GSO IT architecture upgrade in place, database GSO IT architecture in place Design and implementation of GSO IT architecture

51

integration and inter-operability completed GSO and 50 percent of the provincial statistical offices connected through LAN streamlining data dissemination Shorter data collection and dissemination cycles Enhanced efficiency of dataflow and quality of social, economic and poverty-related data Online access to data collection methods, some data analysis and some raw data Data demands of government agencies, research institutions and other data users met in a shorter period of time

Improved data transfer through LAN connection between GSO, 3 IT Centers and Provincial Statistical Offices (PSO) Increased efficiency of data management and report generation through database integration Shortened survey data collection and processing Improved transparency through wider access to data and shortened turn-around time for data requests through GSP Portal Improved capacity of staff in performing streamlined and ICT-enabled data processes

(R) Completion of ICT infrastructure upgrade (improved data transfer through LAN connection between GSO, 3 IT Centers, and Provincial Statistical Offices (R) Time saving of new data management and report generation through database integration (R) Time saving of survey data collection and processing through new system (data scanning, capture, and processing) (R) Number of staff trained to increase knowledge on ICT issues relevant to further modernization of GSO functions (R) Number of survey databases available through GSP Portal (R)

Subprojects 3, 4 and 5: E-applications in Hanoi, Danang and Ho Chi Minh City

Hanoi and Danang

City-level government IT architecture in place, including front and back-end solutions enabling online transactions Selected e-applications online for both G2G and G2B and increased user satisfaction Reduced transaction time and cost through modernization of govt offices and providing online services Increased ICT use by SMEs Enhanced competitiveness of SMEs through training and IT services provided at the transaction center

City-level government IT architecture in place (Hanoi, Danang) Better access to government services through the City Portal (Hanoi and Danang) Improved ICT awareness and management capacity of city officials through training programs (Hanoi and Danang) Improved transparency and reduced transaction cost (i.e., time) for public procurement (Danang) Improved transparency and reduced transaction costs (i.e., time) for business

Hanoi: Design and implementation of city-level government IT architecture (C) Danang: Design and implementation of city-level IT architecture Hanoi: Number of public services available through City Portal (R) Danang: Better access to government services through different channels (R) Hanoi: Reduction of time required for one transaction of selected government services through City Portal compared to traditional way (R)

52

Government officials and ICT staff in the city government trained in e-government and IT issues

registration (Danang) Improved transparency and reduced transaction costs (i.e., time) for vehicle registration and driver’s license (Danang) Improved awareness and use of ICT among private enterprises, including SMEs (Hanoi, Danang)

Hanoi: Number of city officials take CIO training courses (R) Danang: Improved transparency and reduced transaction costs (i.e., time) for business registration and investment licenses (R) Danang: Improved transparency and reduced transaction costs (i.e., time) for driving license (R) Danang: Number of participants from SMEs take IT training to enhance IT skills for their business activities in e-commerce, e-procurement and e-services (R) Danang: Improved ICT infrastructure of the city govt (N) Danang: Improved transparency and reduced transaction costs (i.e., time) for Construction Permits (N) Danang: Enhance and develop ICT human resources in Danang City (N) Danang: Enhance efficiency of public transport management (N) Danang: Enhance the efficiency of management of water resources quality (N) Danang: More efficient government administration (N)

53

D. Status of Procurement Packages, all Subprojects (end-2009 and end-2013)

Package Cost USD

Proc. Method

Status at end-2009 Status at end-2013 Planned Actual Planned Actual

Danang City 1. DNG 1: Vehicle 44,976.00 Shopping Contract completed Contract completed 2. DNG 5a: Design, supply, installation, testing and commissioning of Danang Metropolitan Area Network

4,700,536.93 ICB Issuance of BD BD Preparation Completion of Contract Implementation

Completion of Contract Implementation

3. DNG 5b: Supply & Installation of End-User Computing Infrastructure for the Departments and Agencies of Danang City

27,275.25 ICB Issuance of BD BD Preparation Completion of Contract Implementation


4. DNG 6+7: Development and Deployment of ICT Applications for Danang Authority Units

49,021.67 ICB BD Preparation BD Preparation Completion of Contract Implementation


5. DNG 8a: Design, Supply, Installation, Testing and Commissioning of Danang ICT Transaction Center

1,269,457.32 ICB Issuance of BD BD Preparation Completion of Contract Implementation


6. DNG 8b: Design, Supply, Installation, Testing and Commissioning of Danang Data Center

1,434,406.87 ICB BD Preparation BD Preparation Completion of Contract Implementation


7. DNG12: Establishment of Public Wifi System for Danang City

1,189,605.09 ICB Not available Not available Completion of Contract Implementation


8. DNG13: Development of Additional Public Services for Danang City

1,119,610.78 ICB Not available Not available Completion of Contract Implementation


8. DN15:Supply and Installation of Additional IT

3,223,857.43 ICB NA NA Completion of Contract

Completion of Contract

54

equipment and maintenance tools

Implementation Implementation

9. DNG 0: Hire procurement specialist

29,676.58 International Consultant



10. DNG 2: Project Management Consultant

57,417.56 QCBS Completion of Contract Implementation


11. DNG 3: Access key ICT infrastructure & applications and identity priority business processes/s services



12. DNG 4: Develop strategy, roadmap, architecture, standard, and prototype of e-government of Danang City



13. DNG 9: Design ICT HR development master plan and high-level training program


Contract Implementation



14. DNG 11a: Hire local consultant to give instruction and support in the project



15. DNG 11b: Recruit local consultants to work for Danang PIU



16. DNG 14: Assessing the Status of Danang E-Government Implementation and Building its e-Government Model Based on the Assessment

436,565.10 QCBS NA NA Completion of Contract Implementation


17. Purchasing Pick-up Truck to serve for maintenance of Danang MAN and Public Wi-fi

35,385.26 Shopping NA NA Completion of Contract Implementation


18. Purchasing Equipment for IC Design



55

Training and Workshop Plan 19. CIO Training program 12,319.33 Shopping Completion of



20. IC Design Training Course 74,011.88 Shopping NA NA Completion of Contract Implementation


21. Enhance skills on development of open source software



22. Implementation of The Project Training Programs



Seminars and Study Tours 23. Launch Workshop 1,306.48 Shopping Completion of



24. Study Tour 109,526.43 Shopping Contract Implementation




Table 2: MIC PIU 25. MPT 0: Procurement Consultant (Mr. Pham Tuan Phan, Nationality: Vietnamese)

100,000.00 IC Completion of Contract Implementation




26. MPT0.1: Procurement Consultant





27. MPT0.2: Technical Advisor for Procurement





28. MPT 1: Vehicle Mitsubishi Pajero

50,000.00 Shopping Completion of Contract Implementation




29. MPT 2: Project Management Consultant

1,200,000.00 QCBS Start of Contract Implementation




56

30. MPT 2.1: Additional Project Management Consultant


32. MPT 12: Consultancy on Study, Analysis and Draft of e-Government Interoperability Framework

50,000.00 CQS Completion of Contract Implementation




32. MPT 13: Amendment of the Feasibility Study





33. MIC 1-1: Technical advice on designing a PKI scheme to support e-government development and national e-authentication framework

500,000.00 QCBS Start of Procurement

Start of Procurement



34. MIC 1-2: Implement the upgrade of ROOT CA

2,700,000.00 ICB Issuance of Procurement Documents

Start of Procurement Process



35. MIC 1-4: Technical advice on e-Government Framework and IT Applications Standards

1,300,000.00 QCBS Issuance of Procurement Documents



36. MIC 1-5: Technical advice on solution and implementation plan for e-document interchange system

NA QCBS Issuance of Procurement Documents

Cancel Cancel Cancel

37. MIC 1-6: Technical advice on solution and implementation plan for integrated email system

325,000.00 QCBS Issuance of Procurement Documents



38. MPT 3: Consultancy on Review and Assessment of MIC and DIC’s existing ICT infrastructure, future technology needs in public administration and deployment of e-Government public services

500,000.00 QCBS Issuance of Procurement Documents

Cancel Cancel Cancel

39. MIC 2-1: Issuance of 4,250,000.00 ICB NA NA Completion of Completion of

57

Procurement Documents Contract Implementation


40. MIC 2-2: Completion of Contract Implementation

4,000,000.00 ICB NA NA Completion of Contract Implementation


41. MIC 2-3: e-Gateway for MIC (Phase 2 Portal)

220,000.00 ICB NA NA Completion of Contract Implementation


42. MIC 2-4: Implementation of integrated information system & three public services

500,000.00 ICB NA NA Completion of Contract Implementation


43. MIC 2-5: Installation of Intelligent System and Unified Communication System

3,000,000.00 ICB Start Procurement process

Start Procurement process



44. MIC 3-1: Consultancy on IT sector monitoring and assessment

NA QCBS Start Procurement process


Cancel Cancel

45. MIC 3-3: Consultancy on Policy formulation, Capacity Building and Implementation Guidelines on IT

350,000.00 QCBS Start Procurement process




46. MOFA 1 (MIC 4-1): Data center





47. MOFA 2 (MIC 4-2): E-Visa Application





Table 3: MIC PCU 48. PCU 1: Vehicle: Mitsubishi Pajero,Vinh Quang Ltd., Co. Nationality: Vietnamese





49. PCU 2: Equipment for PCU to manage and monitoring the project

120,000.00 NCB Completion of Contract Implementation




50. PCU 2.1: Completion of Contract Implementation

30,000.00 Shopping NA NA Completion of Contract

Completion of Contract

58

Implementation Implementation 51. PCU 2.2: Net meeting software to support the communication between WB and PIUs



52. PCU 7: Upgrade Financial Management Software





53. PCU 7.1: Completion of Contract Implementation



54. PCU 9: Upgrade LAN for PMU-MIC





55. PCU 0: Procurement Consultant (Mr. Alexandru Aurelian Stoea. Nationality: Rumania)





56. PCU 0.3 (PCU 0.1 & PCU 0.2): Technical Advisor for MOFA project

50,000.00 Cancelled

57. PCU 4: Portal for the Project (ICT Service Engineering JSC)





58. PCU 5: Consultancy for monitoring and evaluation of the quality and effectiveness of project implementation

100,000.00 CQS NA NA Completion of Contract Implementation


59. PCU 6: Auditing 42,000.00 CQS Start of Contract Start of Contract Completion of Contract Implementation


60. PCU 6.1: Auditing to the end of Project



61. PCU 6.2: Additional Auditing for Project Extension period



59

62. PCU 8: Financial consultant to support in closing project

20,000.00 Cancelled

63. PCU 10: Media Campaign for VN ICT project

150,000.00 CQS NA NA



Table 4: Hanoi PIU 64. HAN 1: Vehicle (car for project)

50,000.00 Shopping NA Completed 2007

65. HAN5: Development Hanoi Data Center

3,500,000.00 ICB Submit Procurement Plan

Approval Procurement Plan by PPC

Completed Completed March 2013

66. HAN6: Upgrade City Portal and gateway

1,200,000.00 ICB Submit Procurement


Completed Completed Dec. 2013

67. HAN 7: Implementation of e-document workflow

895,000.00 ICB Submit Procurement


Cancel Cancel

68. HAN0: Hire Procurement Consultant to support Hanoi PIU in 1st phase

100,000.00 IC Completed Completed 2008

69. HAN0a: High Level Procurement Consultant: Select a local consultant with rich experience in management and procurement for ODA project, especially WB funded projects.

25,000.00 IC Award contract Completed

70. HAN0b: Technical Consultant

55,000.00 IC Award contract Request NO for TOR

Completed Completed 2010

71. HAN0c: Individual technical Supervision Consultant

50,000.00 IC Submit Procurement

Submit TOR to WB for review

Cancel Cancel

72. HAN3: Access Key ICT Infrastructure Applications (Leading Associates, USA)

490,000.00 QCBS Submit procurement

Implementation of contract


73. HAN4: International consultancy to develop strategy,

500,000.00 QCBS Select Consultant Evaluate Proposals Completed Completed Dec 2013

60

roadmap and IT Architecture and standards for new 10-year Plan for Hanoi City 74. HAN9: Design ICT HR Development Master Plan and High-Level Training Program

410,000.00 QCBS Select Consultant Submit Plan Cancel Cancel

75. HAN10: Contract Administrative Legal Expert to support Hanoi PIU in legal issues of project implementation.

30,000.00 ICB Complete Contract Implementation of Contract


76. HAN 11: Local Technical Consultant

30,000.00 ICB Submit Procurement

Approval of Procurement Plan PPC

Cancel Cancel

GSO PIU 77. GSO 1: Vehicle 47,355.00 Shopping Contract completed Contract completed 78. GSO 5-2: GSO infrastructure upgrade; Integrated Statistical Information System; Intranet Portal

4,851, 916.00 ICB Contract completed Contract completed

79. GSO 5-2a: Information Warehouse and Information Dissemination Portal

80. GSO 6-1b: Equipment and Software Upgrade (HIPT Group)

1,569,961.00 ICB BD Preparation BD Preparation Contract completed Contract completed

81. GSO 6-1b: Data Collection and Processing solution for GSO

4,140,898.00 ICB Start contract execution

Start contract execution

Complete contract Contract completed

82. GSO 8.1: Equipment support for GSO 6-1b and 2009 Census

29,843.00 Shopping Completion of contract

Contract completed

83. GSO 8-2a: Tools support for GSO 6-1b and 2009 Census – Cutting Machines


Contract completed

61

84. GSO 8-2b: Tools support for GSO 6-1b and 2009 Census – Air Conditioners


Contract completed

85. GSO 8-3: GSO Application Support Mapping Administration Field Database in Vietnam: Software Engineering and GIS Center

29,800.00 Shopping Completion of Contract

Contract completed

86. GSO 8-4: Rack cabinets for PSOs; Supporting equipment from SSIC

26,418.00 Shopping na na Contract completion

Contract completed

87. GSO -0: Hire Procurement Specialist

142,137.00 ICB Completion of contract

Contract completed

88. GSO 2-1: Local Project Management Consultant

10,800.00 ICB Completion of contract

Contract completed

89. GSO 2-2: Local Financial Management Consultant

17,305.00 ICB Completion of Contract

Contract completed

90. GSO 2-3: Technical Advisor Consultant

276,395.00 ICB Bid Preparation Completion of contract

Contract completed

91. GSO 3-2: Local Training Consultant – Specialist (Individual)

1,839.00 ICB na na Completion of contract

Contract completed

92. GSO 3-3: Local Project Manager

8,867.00 ICB na na Completion of contract

Contract completed

93. GSO 5-1: ICT Assessment Network, IT Architecture and Security Consultancy for GSO and VPN Pilot

760,707.00 QCBS Contract completed

94. GSO 6-1a: Enterprise Architecture and Meta-Data Study; and IT/EA Governance Support

1,583,118.00 QCBS Bid Preparation Completion of contract

Contract completed

62

Annex 3. Economic and Financial Analysis The following analyzes the impact of the World Bank Information and Communication Technology (ICT) Project in Vietnam. It looks at (1) efficiency gains of specific interventions as well as (2) overall project efficiency. This includes the Danang subproject and General Statistics Office (GSO) subprojects. In many instances the precise impact is difficult if not impossible to measure due to a lack of baseline data and detailed financial and operating information. Instead, the analysis reports what can be reasonably assumed based on documented experiences elsewhere.

Efficiency gains of specific interventions

Project interventions have impacts on three groups: government, businesses and citizens. For government, efficiencies from ICT include higher output with lower expenses due to cost savings including reduction in duplication and service delivery times. Citizens and businesses achieve greater efficiency through time and cost savings when accessing government information and services. There are also benefits that are difficult to measure including the contribution of online public services to the quality of life of businesses and citizens; improved business and citizen decision making due to the availability of easier to access information; and linkages between government and private sector to create new business opportunities (e.g., open data).

Danang Subproject:

The Danang Subproject had impacts on government, Small and Medium Enterprises (SMEs) and citizens.

Government

According to the Vietnam project teams, interventions impacted the government positively due to the transition from manual to computerized methods:

• The competence of city officials grew to a higher level due to knowledge and skills they received through intensive IT training. This opened the way for solid and sustainable development of the city in the coming years.

• A citizen-centric government has been established where interactions between individuals, businesses and government departments will become more streamlined and easier.

• Coordination and interconnection among departments and agencies in delivery of public services improved.

The transition from manual to computerized processes results in lower costs and shorter processing times. Measuring this requires an assessment of the amount of time spent and costs carrying out processes before and after the intervention. In the absence of such information, some inferences can be made:

• Government ICT competency was increased through training programs leading to understanding of how to implement IT systems with consequent efficiency gains. Further awareness of IT benefits is likely to reduce resistance to restructuring business processes.

• Interactions between government, businesses and citizens became more streamlined resulting in quicker processing times.

63

• Coordination among different public agencies resulted in a reduction of duplication leading to greater efficiency and cost savings. For example multiple databases maintained by different city departments were streamlined with common codes and system architecture, reducing duplication and simplifying maintenance.

• It is also noteworthy that Danang has ranked in the top position of the Vietnam ICT Index for five straight years (2009-2013) a period that coincides with the ICT Development Project. Note that prior to the intervention a stakeholder consultation in 2004 rated the city’s status of ICT development as limited/very limited.23

SMEs

Several interventions impacted SMEs including:

• Application systems were built to meet SME requirements including the “Business Registration System", "Investment Project Management System", "Fishing Boat Management System", and "Construction Permit Management System";

• Significant information was provided for SME business activities via the Contact Center (76% public services related to SMEs);

• 400 courses in ICT are available in e-Learning Portal;

• 75 software developers from SMEs have gone through Software Engineering training courses;

• 50 IT experts from the private sector joined the training course about the Danang e-Government Platform.

There is evidence that these types of interventions are likely to have positively impacted SMEs in Vietnam. For example, World Bank research suggests that development of online business registration systems that reduce complexity, time and costs lead to a rise in business registrations and employment as well as generating income for the government. 24 Similarly, facilitating construction permits through online applications saves businesses time and money.25 The contact centers would facilitate business activities by providing a medium for enterprises to easily contact government agencies about matters such as policy, operating hours, general information or to voice a concern. This reduces costs for both the government and business. The training interventions will likely have increased the ability of SMEs to use relevant e-government services as well as incorporate ICTs in their businesses raising productivity.

23 World Bank. 2014. "Vietnam - ICT Development." Report No: ICR 2976

24 World Bank. 2013. Doing Business 2014: Understanding Regulations for Small and Medium-Size Enterprises. Washington, DC: World Bank Group. http://www.doingbusiness.org/reports/global-reports/doing-business-2014

25 Op. cit.

64

Citizens

A number of interventions were aimed at citizens:

• Over 315 public services delivered via the Danang Portal for citizens; • Basic IT training courses (68 sessions) delivered on television; • 400 courses in ICT are available on the e-Learning Portal; • 1,125 citizens received IT training via the Danang Training and Applied Research

Center; and, • Danang Public Wi-Fi with 329 Access Points provides Wi-Fi coverage over the city

center including all main city routes, beaches, offices of departments and agencies, communes and wards and public areas to serve citizens and tourists.

The delivery of online public services benefits citizens by reducing the cost and time of travelling to and waiting at, government offices. Sri Lanka showed significant savings in time from implementing online citizen services: the period to issue birth, marriage and death certificates dropped from hours to just a few minutes.26 A survey of European citizens found that 80% replied that online public services saved them time, 76% appreciated the flexibility of conducting government processes anytime and from home and 62% replied they saved money.27 Similarly a survey carried out for Danang found that 45% of respondents expected the project’s e-services to save time, 35% stated it would be more convenient, and 23% expected to save costs.28 Assuming that the Internet penetration rate is the same in Danang as the nationwide average, then around a third of citizens could take advantage of the online public services implemented through the project. Availability of different ICT training options via television, online portal and physical classes promotes digital literacy of citizens, providing flexibility in how they can access learning opportunities. This will enable them to more successfully use e-government as well as other online services and applications. City-wide Wi-Fi availability saves citizens the time of having to find a physical location when they need use the Internet. Since the intervention was free, it also saves citizens money. Note that there are also benefits to government in terms of making online public services more accessible and allowing government employees to access the Internet when outside the office. Businesses also benefit; for example customers could find businesses easier using location services over the Internet and business could exploit social media and other applications to create innovative relationships with customers.29

26 Information and Communication Technology Agency of Sri Lanka (ICTA). 2011. ICT Usage in Government Organizations Survey.

27 European Commission. 2013. “eGovernment Improving but Citizens Ask for More.” Press Release, May 28. http://europa.eu/rapid/press-release_IP-13-466_en.htm.

28 An Vien Media. 2013. “Report on Performance Assessment of the Communication Campaign for Vietnam ICT Development Project 2013.”

29 According to one source, other impacts of the free city Wi-Fi include "...better IT infrastructure; boost trade promotion, lure more investment, advertise tourism images and raise cultural and educational

65

Estimated costs savings to citizens: An estimate of the cost savings to citizens from free Wi-Fi can be made by calculating the cost of Internet access for which they no longer have to pay. This makes the following assumptions:

• The proportion of Internet users in Danang is the same as that across the country (36% in 2012).30 This results in an estimate of around 345,700 Internet users in Danang.31

• It assumes that one third of such Internet users would use the free citywide Wi-Fi service.32

• The cost of a mobile data package is VND 70,00033 (US$3.3634) per month.

Based on these assumptions the savings to citizens would be US$383,409 a month (114,081 users X US$3.36 (cost of Internet access per month)). The projects also supported quality of life interventions for citizens such as e-service for Water Quality Monitoring (financed for Danang) and improved bus/transport services (GIS equipped/Danang).

GSO Subproject:

For the GSO subproject, the following efficiency gains were observed: Data collection and processing for the 2009 Census was reduced to 14 months from 24

months (42% reduction) as a result of project-supported ICT improvements. The calculated cost savings is US$500,000 based on the 450 data entry staff used for the 2009 Population Census 35 and assuming they receive the average monthly income for

standards for the city." See: “Da Nang Provides Free Wi-Fi Coverage.” 2012. VietNamNet, May 11. http://english.vietnamnet.vn/fms/travel/51876/da-nang-provides-free-wi-fi-coverage.html.

30 VNNIC. 2012. Report on Vietnam Internet Resources. http://vnnic.vn/sites/default/files/tailieu/ReportOnVietNamInternetResources2012.pdf

31 Based on the 2012 Da Nang population of 973,800. See: "Area, population and population density in 2012 by province" on the GSO website at: http://www.gso.gov.vn/default_en.aspx?tabid=467&idmid=3&ItemID=14459

32 Based on a study of Thai Internet users which found that 35.3% used the country's free Wi-Fi service. See: ETDA. 2013. Thailand Internet User Profile.

33 For 600 MB of data. See the Mobifone website at: http://www.mobifone.com.vn/portal/en/services/home/internet/dichvu3g/fast_connect_gc.jsp

34 Converted to US$ using the 2012 annual average exchange rate. See: http://data.worldbank.org/indicator/PA.NUS.FCRF

35 GSO. 2007. Plan for the 2009 Population and Housing Census of Vietnam.

66

government workers.36 The difference in salary costs between data entry staff working 24 months as compared to 14 months is US$500,276.

Forms scanning hardware used for the 2011 Agriculture and Rural Census generated a 50% improvement in overall processing time with time reduced from 2 years to 1.

The time for data collection and processing for Consumer Price Index (CPI) surveys was reduced by one third from 15 to 10 days following the introduction of new IT processes.

In addition, the GSO subproject reduced duplication significantly by harmonizing and integrating databases into a standard system architecture. The provision of additional free statistics online would also have improved decision making by both the government and the private sector.37

Overall project efficiency

Several factors contributed to overall project implementation efficiency for Danang and the GSO. Two key factors were: (i) the skills of the PIU; and (ii) competitive bidding. The Danang PIU project manager had significant expertise in both technology and project management. Further he had good relations with local leaders at all levels. This helped speed up the local authority’s approval for all submissions. Similarly, there was good coordination between the GSO PIU and technical and business staff, enhancing work flows and reducing implementation time. The competitive bidding process for all Danang packages resulted in significant cost savings. The original overall budget of the Danang Subproject was US$ 17,310,000. In 2010 when the project was restructured, additional funding of US$10,000,000 was allocated due to successful implementation. This brought the total value of Danang Subproject to US$27,310,000. Following 5 years of implementation, Danang PIU has saved a total of US$6,146,000 through competitive procurement practices. This allowed the PIU to acquire additional equipment with the savings. Though not as large as Danang, the other PIUs also achieved cost savings on procurement packages. In aggregate, project savings totaled over US$15.0 million and most of this is likely to have been used for other equipment, systems etc. Conclusions Although detailed information necessary for carrying out a comprehensive efficiency analysis is lacking, the anecdotal data suggests that the project interventions are already having an impact on reducing costs for government and saving time for SMEs and citizens. These benefits can be expected to grow as the e-services evolve in scope and functionality and more Vietnamese use the Internet.

36 The average monthly income for government workers was Dong 2.3 million (US$111) in 2011. See "Monthly average income per employee in state sector at current prices by kind of economic activity" on the GSO website at: http://www.gso.gov.vn/default_en.aspx?tabid=467&idmid=3&ItemID=14384

37 For example the Finnish Ministry of Employment and the Economy found that putting statistical data online made it easier for entrepreneurs to search for data and improves analysis. The system gets about 3,000 queries a month. See: Tieto, "Customer case: The Ministry of Employment and the Economy" at: http://www.tieto.com/sites/default/files/atoms/files/case_ministerio_en_1601.pdf

67

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending Christine Zhen-Wei Qiang Task Team Leader CITPO Natasha Beschorner Senior ICT Policy Specialist CITPO Kashmira Daruwalla Senior Procurement Specialist CITPO Naomi J. Halewood ICT Policy Specialist CITPO Andrea Ruiz-Esparza Senior Program Assistant CITPO Peter L. Smith Consultant CITPO

Supervision/ICR

Deepak Bhatia Lead E-government Specialist and TTL TWICT

Kaoru Kimura ICT Policy Specialist TWICT Christine Zhenwei Quiang Task Manager CICTI Dung Thi Thuy Dao Program Assistant EACVF Kashmira Daruwalla Senior Procurement Specialist ECSO2

Quyen Do Duong Finance Analyst CTRDM - HIS

Naomi J. Halewood ICT Policy Specialist TWICT Hung Viet Le Sr. Financial Management Specst EASOS Hanh Thi Huu Nguyen Financial Management Specialist EASFM Samia Melhem Lead ICT Policy Specialist TWICT Hoang Xuan Nguyen Procurement Specialist EASR2 Son Duy Nguyen Senior Operations Officer EACVF Dzung The Nguyen Senior Rural Development Specst EASVS Tam Thi Do Team Assistant EACVF Rohit Gawri Information Analyst SARIM

Hoi-Chan Nguyen Consultant OPCIL-HIS

Cung Van Pham Sr Financial Management Specst EASFM Quynh Xuan Thi Phan Financial Officer GEFOB Rajesh B. Pradhan Consultant FIEID Andrea Ruiz-Esparza Senior Program Assistant TWICT Randeep Sudan Sector Manager TWICT Kien Trung Tran Senior Procurement Specialist EASR2 Anh Tran Phuong Vu Administrative Officer EAPC2 Huong Thi Thu Vu Program Assistant EACVF Anna Roumani Consultant TWICT

68

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD ‘000 (including travel and consultant costs)

Lending FY03 3.33 68.25 FY04 25.46 112.02 FY05 41.05 181.56 FY06 10.41 46.23

Total: 80.25 408.06 Supervision/ICR

FY06 21.61 75.21 FY07 19.03 82.85 FY08 13.91 75.64 FY09 30.49 131.53 FY10 23.68 117.96 FY11 16.54 121.62 FY12 38.40 201.77 FY13 42.58 182.34 FY14 17.18 76.52

Total: 223.42 1,065.44

69

Annex 5. Beneficiary Survey Results 5.1 The following summarizes the results of a major study and an important assessment. The Project financed a study in 2013 under procurement contract DNG14 entitled: “Assessing the Status of Danang e-Government Implementation”, conducted by consultants Leading Associates and KDi Asia. It is important to note that the study does not focus exclusively on the ICT Development Project but it does refer to specific results from the Project where relevant. The study is important for providing a roadmap of how evaluation of ICT projects in Vietnam might be conducted from here on, establishes a form of baseline for doing that, and emphasizes building strong M&E of ICT programs. Its findings – specifically those pertaining directly to the Bank-supported project - are summarized below along with important lessons for such projects. The second report/assessment is entitled “Report on Performance Assessment of Communications Campaign for Vietnam ICT Development Project” conducted by An Vien Telecommunications and Communications Company in December, 2013 under contract to the MIC PCU. Danang Study: 5.2 Methodology: The study proposes and itself uses, the e-Government Assessment Framework (eGAF), which provides a comprehensive review of e-government from three important domains: ICT Outcomes which assesses user satisfaction, service quality, information quality and systems quality; ICT Program Management assesses quantitative data from diverse sources for six dimensions: projects, applications, infrastructure, security and architecture, regulations and human resources; and ICT Governance which looks at five dimensions via questionnaires and an interview-sourced capability maturity assessment in each selected public agency. Conducted under the DNG 14 procurement package, Leading Associates/KDi Asia interviewed 75 SMEs, 642 citizens and 38 public sector departments and agencies - the latter including health, transport and science and technology agencies - and Peoples’ Committees. A summary of the survey framework is attached but does not contain the citizen or business questionnaires (requested from firm but not provided). Results of e-GAF: ICT Outcomes Assessment: This domain used qualitative, survey/questionnaire-sourced data from citizens, businesses and public agencies:

• E-Government in Danang City has brought the city to the top position of the Vietnam ICT

Index for five years continuously (2009-2013). • This occurred within the life of the ICT Development Project and in a city where a

stakeholder consultation in 2004 rated the city’s status of ICT development as limited/very limited under major themes.38

• E-Government in Danang City has entered a maturing phase of its development;

38 Ratings for Danang were as follows in 2004: (i) business process changes, ICT as a strategic tool: Very Limited; (ii) true integration, data sharing, workflow, functional applications: Limited; (iii) training of officials: some progress but low awareness and competence still a key constraint; (iv) connectivity, LANs: more required particularly at district and ward level; (v) WAN: some, but less than other cities; and (vi) basic computerization: medium progress but more needed, particularly districts and wards.

70

• The Danang ICT Transaction Center operates six days/week from 8:00 am to 4:00 pm for general enquiries about public services.

• The Danang Data Center, Public Wi-Fi and Metropolitan Area Network (MAN) are providing good quality infrastructure to the public and to government agencies.

• At the time of the study, online services in Danang were tending to play a greater role in providing information to the public on public service procedures than as a new method for online transactions between Government and the public.

• This is explained by the still-limited ICT skills and competence of citizens and the fact that in a small city like Danang, people can still transact face-to-face with agency representatives, whose services are generally well-rated.

• Citizens also believe that personal transactions foster accountability and transparency, and enable them to participate in government policy-making.

• While most citizens are satisfied with public services of Danang City, the level of satisfaction for online services is expected to further increase once end-to-end services – still in the pipeline at the time of the study - are introduced.

• The study recommends that the City pay attention to the digital society and knowledge economy aspects of ICT and e-Governance development, supporting people’s skills development and access to computers/literacy.

ICT Program Management: This domain used quantitative data from diverse sources which with the exception of Security and Architecture, were collected from existing reports and indices (Danang ICT Steering Committee – on which the project was represented - annual ICT Current Status Report; agencies’ project planning and implementation documents; and Vietnam’s annual ICT Index):

• Successful implementation of all planned procurement packages establishing key systems and deploying a comprehensive ICT infrastructure, financed by the World Bank ICT Development Project, demonstrated Danang’s commitment to strong ICT management.

• Greater emphasis is needed on ICT Information Security practices. • ICT Regulations were established and will need more frequent review and updating to

reflect changing needs. • An important issue is that the current human resource endowment cannot meet further

ICT development/expansion needs without adequate staff and budget. • With more services migrating online and demand increasing, capacity and resources must

also scale up. ICT needs and organizational framework, as well as resources must be mandated to meet sector growth.

ICT Governance Assessment: This domain used questionnaires and interview-sources capability maturity assessment in each selected public agency:

• Danang’s e-Government objectives are clear as enunciated in several forms such as the e-City Program which describes the modernization of the city through technology and ICT.

• The city’s ICT strategy is based on consultative, inclusive processes involving stakeholders and awareness of ICT is quite high among senior city managers.

• Policy instruments still do not define how ICT Governance is to be measured and the performance indicators for measuring progress.

• Danang’s Enterprise Architecture (EA) was formulated in 2009 under the ICT Development Project and attention is now needed to its updating.

71

• Implementation of the Korean e-Government Standard Framework (eGovFrame) represents only the application architecture subset of the full EA scope and the EA should fully-incorporate this Framework.

• Evidence suggests there is no Risk Management Framework and Monitoring and that this will need attention.

• More resources are needed for managing information security oversight. • There is not yet a framework for systematic monitoring, evaluation and assessment of

performance of ICT processes. While some monitoring was in place at end-project, there were no specific performance parameters for assessing the outcomes of e-Government and evaluating the effectiveness of ICT governance.

• When the study was conducted, management of ICT projects was centralized in the PIU and DIC accounting for less awareness among other agencies not typically accountable for managing the ICT Development Project.

• The acquisition and deployment of ICT services under the ICT Development Project have been properly established but there is need for structured, better-defined and activated processes for managing change and releasing new e-services.

• More structured frameworks would improve the planning for and execution of initiatives to deliver the required level of service.

• Identification and monitoring of performance standards for quality of ICT service management and governance are currently centralized, and awareness of performance metrics and targets for existing online services is still developing.

Other Project-related results, events and trends reported by the study: ICT Development Project-financed procurement “packages” have had positive impacts:

• Capacity of the ICT workforce was improved via the development of a Master Plan for ICT Human Resource Development (DNG9).

• New, end-to-end online services based on the e-Government Standard Framework (Korea) were introduced in 2013 (DNG 6-7 and DNG13).

• Danang City in late 2013 officially opened the Danang ICT Infrastructure System which includes Project-supported components: Danang Metropolitan Area Network (MAN); Public Wi-fi, ICT Training, the Applied Research Center and other related components.

• The City has won several international awards from organizations such as FutureGov, the 2013 eAsia Award of AFACT, and Smarter Cities Challenges for successfully adopting ICT for citizens and to serve business organizations, as a direct outcome of the Project.

• Specific examples show that the cost of public administration was decreased through the effective application of ICT systems.

ICT Infrastructure Development Results:

• ICT infrastructure was modernized, becoming an important contact point for the national backbone network covering the city with networks based on international standards, providing good quality, diverse services.

• Danang MAN was re-constructed, connecting all government agencies, departments, districts and branches (enterprises, schools, hospitals) to support daily operations.

• 100% of local government agencies are now connected to LAN and high speed Internet (ADSL)

• 100% of staffs and employees working for all branches, departments and people’s committees of districts now have computers.

72

• Fiber optic network system (Metronet) with 100Mb capacity is connected to 72 government agencies.

• The Danang Data Center, ICT Transaction Center, ICT Infrastructure Development Center, Public Wi-Fi network and ICT Training Center are established and operational.

Human resources development:

• The Human Resource endowment in Danang has improved: the ICT Development Project’s main contribution has been through the development of an ICT Human Resource Development Master Plan (package DNG 9) backed by training;

• Training courses resulting from this plan included: CIO training for 27 city and department leaders; 18 training courses in e-Government and ICT governance skills for government managers and staff; technical training courses for diverse ICT stakeholders; study tours to developed countries to acquire understanding of best practices for e-Government development.

• Such training has improved the quality of the ICT workforce but is hampered by ICT human resources and civil service regulations.

ICT application development results:

• The study asserts that the Project’s installation of One-stop Shop software in 56 communes in 2010 improved the effectiveness of government management. Managers can monitor and control administrative processes and all citizens can access the site for information.

• Almost all departments and agencies have a website/portal accessible by users at all times. • Some 315 online e-services deployed to date were directly attributable to the ICT

Development Project: Driving Licenses Management; Land Use Rights Management; Business Registration Management; e-Procurement; Building License Management; Geographical Information System (GIS) and others were being finalized under procurement packages DNG6-7 and DNG 13 at time of study completion.

• These services are based on e-GovFrame, a Korean open-source e-Government application framework.

Impact of the ICT Development Project in socio-economic development:

• Strong e-governance foundation for Danang. • Positive impact on the operational capacity of the City. • Improved productivity, quality and efficiency of key public services for people and

business as well as public authorities from municipal to commune level. • Significant progress towards establishing a “professional, responsible, active and

effective administration for the people”. • Authorities have significantly reduced the cost of public administration through the

effective application of ICT systems and improved the operating efficiency of the administration while broadening people’s participation, improved the productivity and effectiveness of city government agencies in managing and serving people, increased the transparency of government activities and reduced bureaucracy.

• Equality of access to information through new channels of communication (email, text messages, phone calls/other) has improved, especially information about administrative procedures for people and business and service responsiveness has improved.

73

Other methodologies applied by the study: The e-GAF analysis is supplemented/complemented by three other methodologies seeking to correlate and measure Danang’s achievements in e-Government. The results have been used very selectively, i.e. only where the Project is specifically-mentioned to avoid attribution errors. Public Administration Performance Index (PAPI): This is a policy monitoring instrument providing objective, bottom-up, evidence-based measures of citizen’s experiences and direct interactions in regard to standards of provincial level governance, public administration and public services performance. Since 2010, PAPI has surveyed 32,500 citizens in 63 provinces, of which 13,747 in 2012.

• Achievements of Danang related to the PAPI Index show close correlation to recent e-

Government initiatives such as development of e-District and e-Ward systems, and fully-implemented One Stop Shop system (OSS) and Inter-Agencies One Stop Shop (IAOSS) systems.

• The Danang ICT Transaction Center (including a call center), in operation since 2012, has helped make public service procedures clearer to citizens and businesses.

• New online public services platform developed under packages DNG 6-7 and DNG 13 are expected to continue supporting Danang’s high rating in the PAPI Index.

Provincial Competitiveness Index (PCI): This instrument assesses the ease of doing business, economic governance, and administrative reform efforts by local governments of Vietnamese provinces and cities to promote private sector development and only includes data supplied by private, domestic operations. The 2012 report surveyed the perceptions of 8,053 domestic firms. The PCI comprises nine sub-indices reflecting economic governance areas affecting private sector development. No specific attribution to the Project is intended but it is reasonable to assume that the Project made a contribution.

• Danang improved its rating from 2011 to 2012 in: informal charges; developed and high-quality business support services; reduced time requirements for bureaucratic procedures and inspections.

• These three components are described as having a very close relationship with the development of e-Government services in Danang City.

Public Administration Reform Index (PARI): Looking at three reform indices: Index of Central Authorities; Index of Department Authority; and Index of District Authority and cross-referencing them to the initial e-GAF assessment, the study concludes that:

• Danang’s e-Government initiatives and the Danang ICT Development Project in particular have had many positive impacts on the city’s socio-economic development;

• Comparing the e-GAF assessment on customer satisfaction with online public services and the PAR Index on public services, Danang citizens still prefer counter to online services (explained by the ICR as also stemming from citizens’ lack of access to computers, and to still-modest IT literacy).

• This explains the still modest utilization of fully-transactional online applications which in turn explains the so far modest overall impact of the e-Government program on other indices.

• From 2014 onward, operation of the new ICT infrastructure financed by the ICT Development Project may represent a significant challenge to Danang and therefore a

74

“proactive ICT governance framework should be established and information security provisions should be improved”.

• Importantly, to improve e-services usage rates, Danang needs a proper communications plan to promote ICT usage by stakeholders and to solicit their feedback to improve the quality of online services;

Study recommendations: The study makes an extensive series of recommendations for future ICT development and programs, summarized below: ICT Monitoring and Evaluation:

• Adopt the eGAF framework for monitoring and evaluating e-Government and ICT Development ensuring consistency/linkages to the City’s socio-economic goals.

• Appoint a single unit accountable for coordinating and implementing assessment exercises.

• Conduct the ICT Outcomes assessment to focus on satisfaction levels of citizens and businesses, twice yearly or on a continuous basis through online “pop-ups”.

• Conduct the ICT program Management assessment by aggregating quantitative data on readiness from existing secondary data sources/indices, annually, supplemented by internal reports on adoption and implementation status of projects, ICT applications and infrastructure.

• Conduct the ICT Governance assessment every two years using trained assessors in consultation with responsible agency personnel and provide an objective evaluation of the capability maturity of ICT management in selected public agencies.

ICT Regulatory and Policy Management:

• Establish a central ICT Governance entity responsible for long-term strategy which aligns ICT strategy and includes business sector; integration of city-wide technology initiatives and compliance with ICT policies and standards; and compliance management with ICT Regulations and standards.

• Establish a Risk Management Framework including assessment and awareness-building, update risk registers, monitor risk mitigation initiatives, and establish risk responsibility.

• Minimize overlap, redundancy and cost in ICT-related operations, ensure that proposed new systems are appropriate and a sound business investment, and improve organizational maturity and capacity in effective management of IT with clear and measurable success criteria.

• Develop minimum standards for project management and oversight, define project management performance standards and methodologies, develop systems testing methodologies and promote effective portfolio management with a central pool of resources.

ICT Architecture Management:

• Complement Danang City’s Enterprise Architecture defined in 2009 with detailed guidelines and Reference Architecture Models to reflect 2013 ICT infrastructure, e.g., incorporate Korean eGovFrame into Danang’s reference model and rationalize same

75

within the EA’s Application Architecture. The Danang EA needs updating/revision and participation of stakeholders in this process is encouraged.

• Develop a multi-year ICT Strategic Master Plan to ensure public agencies are on the same page and moving in same direction and update annually.

• Develop a communications platform to disseminate information to the public and private sectors on ICT programs and initiatives.

ICT Resource Management:

• Establish a clear top-down framework for managing ICT services and their governance. • Danang City to establish an industry standard framework for service management with

clearly-defined functions, roles, goals and expectations, based on the UK Government’s IT Service Management (ITSM) framework, and mapped to achieve specific objectives of the ICT Master Plan.

• Establish a formal process to capture new e-service requests or changing operational requirements, i.e., a centralized User Representative function to interact with and interpret user needs.

• With limited budgets and human resources, Danang City’s ICT agencies need to consolidate roles cleanly dividing areas requiring specific knowledge domains or authority related to a business process from generic/user/customer facing ICT processes and consolidating the latter into a singly/centralized ICT Service Unit with a Chief Information Officer.

• Establish a single point of contact – an ICT organization - to manage a structured incident response process that is transparent and measurable and provides feedback and/or resolution of issues. This entity must understand the e-services and processes it supports and keep citizens informed of revisions to e-services to prevent pushback/aversion to using the particular service. This entity must maintain transparent and objective communication with all stakeholders through official channels/forums.

ICT Security Management:

• Establish an effective information security unit in Danang City with authority to manage all security processes across Danang’s e-services.

• Formally define information security roles and functions, and monitor the implementation of security controls selected to mitigate specific risks.

• Enforce security policies and standards through regular compliance reviews and audits, providing feedback to senior officials with validated risk profiles.

B. Assessment of Communications Campaign for the Vietnam ICT Development Project Field interviews were conducted in December 2013 via phone interviews with samples of 300 persons per province (Hanoi and Danang), male and female in the age range 24-45. Results:

• 65% of respondents in Danang read e-news daily vs. 35% in Hanoi • Respondents had mostly learned about the Project via the TV campaign or Internet (27%

and 25% respectively) • About 36% of respondents were aware of and recalled the Project’s communication

themes, a result indicating need for improving campaign methodology

76

• 41.7% of respondents were aware of the construction of e-government for Danang and Hanoi

• 35.3% were aware of motor vehicle registration and drivers’ licensing e-services • 28.3% knew about online licenses • 21.3% knew that the Project was implemented by MIC and the World Bank • In terms of GSO modernization and electronic visa applications/processing, 12% of

respondents were aware. • Benefits of the Project were seen overwhelmingly as “keeping pace with world trends“ • “Convenience” was rated higher in Hanoi (53%) than Danang (35%). Danang is a small

regional city and citizens still preferred personal, face to face transactions (as noted in the Danang Study).

• Relative values by type of benefit were as follows: Type of Benefit/Project Services Danang Hanoi Time savings 45% 48% Convenience 35% 53% Transparency 57% 27% Cost savings 23% 40% Simpler procedures 46% 15% Keep up with trends 46% 15% Better than old way 22% 40%

• The assessment concludes that the Communications Campaign has improved the

awareness level of project content.

• 36% knew about and remembered the communication topic • About 10 million people are estimated to have seen/learned about the Project via TV,

followed by Internet and Press. More time is needed to assess the effectiveness of the new e-services.

77

Annex 6. Stakeholder Workshop Report and Results (if any) N/A

78

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The following are Executive Summaries of the Completion Reports from MIC/PCU, GSO and Danang. A. MIC PCU: Executive Summary of Completion Report Executive Summary Overview of Vietnam ICT Development Project

The Government of Vietnam (GOV) recognizes the potential of Information and Communication Technology (ICT) in its various applications to enhance competitiveness, facilitate economic diversification and increase productivity, as well as to improve the efficiency and capability of institutions, particularly in government. GOV has undertaken several initiatives to upgrade information and communications infrastructure. These include: introducing information systems into selected government departments and piloting online services; providing incentives to the ICT industry; developing ICT skills and enhancing awareness of ICT, in government and to some extent in the private sector.

In 2005, the World Bank approved the project (Vietnam ICT Development Project financed by IDA Credit No 4416) for assisting Vietnam to accelerate utilization of ICTs to foster economic development, modernize public administration and improve delivery of public services to citizens and businesses. Through a combination of business process improvements and deployment of technology, the project will support a substantially enhanced government online presence and content at the national and municipal level, through interactive and dynamic portals at the MIC, and the People’s Committees of Hanoi, Danang and Ho Chi Minh City. MIC, and the above-mentioned municipal administrations would roll out e-government services to businesses in areas such as e-procurement and business/land registration. The project commenced in September 2006 and planned to complete by 30 Jun 2011, but due to slow implementation, the project was extended to 31 December 2013.

Critical Factors Affecting Implementation and Outcomes

Positive Conditions: (1) PMU was received the closely directs and supports from the Leaders of the Ministry of

Information and Communication, specially is Deputy Minister –Dr. Nguyen Minh Hong who directly in charge of the MIC subproject in particular and overall VN-ICT Development Project in general.

(2) The staffs involved in the MIC subproject (of PMU, stakeholders) are qualified, professional qualifications and enthusiasm in the work.

(3) Good Project management consultant firm (MPT2) has helped to promote the progress of projects

(4) Close cooperation between Stakeholders (12 DICs, MOFA and other MIC agencies) during project implementation to ensure sustainable of the project results.

79

(5) Besides that, during project implementation, MIC subproject always gets absolutely strong supports from WB. The WB task team has conducted the evaluation for every 6 months and provided the PMU-MIC the valuable recommendations to support us to solve the difficulty in procurement and financial management.

Negative Conditions

(1) Changing of procurement guideline and personnel from WB: In 2010, there was a change in development of IT procurement guideline together with changing of personnel in WB task team leader. Therefore, most of bidding documents had to re-submit, and some bidding documents took more than 2 years to get the NOL from WB, for example: MIC1.2, MIC1.3, MOFA1, MOFA2 (see table 3 below)

(2) WB’s Comments to the IT technical documents too slow and lack of consistency in the approval of the Bank for the bidding documents (BD).

(3) Changing of director of subproject in PMU-MIC: 02 times: 2008 and 2009.

(4) Lack of project management experience in PMU-MIC because most of the PMU staffs were first involved in the ODA project implementation.

(5) Issue with anonymous complaints: The handling of anonymous complaints was different between Vietnam and the World Bank which also caused delays in the process of implementing the packages.

Achievement of PDO:

Stage 1: before 2010 the achievement against the original PAD-based PDO and data/results against the original Key Performance Indicators. Event that, MIC subproject has had successfully issued the e-Government Inter-operability standard which help to standardize all IT technical standard within project and overall IT investment within government agencies.

Low achievement of PDO of the subproject cause by following reasons:

(1) All PIUs are new to WB IT project which having different procurement process;

(2) MIC subproject has been restructuring to combine PCU and PIU into single organization PMU which to manage entire project activities. From the implementation view, MIC subproject has established 3 technical implementation working group who responsible for 3 components by 2008.

Stage 2: After project restructuring the achievements against the post-Restructuring #1 PDO

At the end of stage 2, most of PDO and related KPI have been archived as follows: - Unified IP communication system has been setup at MIC head quarter which to allow

smoother communication within MIC. In additional 12 DIC provinces also equipped with new hardware infrastructure including Data center and video conferencing system.

- MIC also support MOFA to implement MOFA data center and online visa system. By the end of 2013, online visa system is operating and new data center was utilized to consolidate existing applications of MOFA to help better IT governance within MOFA.

80

- The training and capacity building activities of MIC subproject has helped to improve the capability of MIC policy makers and also improve the IT governance capability of business sector via training to state-own enterprise and SMEs

- With the support from project, several legal and policy documents have been approved including Radio Frequency Law, Postal Law, National Information Security master plan and various IT application decrees

- The National PKI infrastructure also improve via implementing new root CA system which help to ensure higher securer transaction on electronic environment in Vietnam

Reasons for success:

- MIC subproject was meet with demand of improving infrastructure from MIC and DIC

- MOFA component was exactly met the requirement from the operation of MOFA Impacts on Intended Beneficiaries (if any):

- Project is helped MIC to improve the legal environment by supporting MIC to draft and approve several legal and policy documents. Those policy documents will contributing the improve IT application and overall ICT development in Vietnam in the future

- The project is helped to create a comprehensive IT infrastructure at MIC and also DIC which provide a new communication way between central and provincial government

- Online services, especially online VISA system helped to create better environment to the citizen and business in Vietnam

Other unintended Outcomes and Impacts:

- The project is help to improve the image of Vietnam to the international as as providing a

faster and easier method to getting entry visa to Vietnam. It will contribute to promote the trade and tourism in Vietnam.

Risk to the Project’s Development Outcomes With result and outcomes of MIC subproject, we identified following risks may effect to the sustainable of the project as follows.

81

No. Critical Risks Mitigating Activities 1 Sustainability for new

system • MIC should prepare the operation cost and maintenance budget to

ensure smoothly operation of newly implemented system • Provide proper training to technical staff to improve the IT system

operation and management capability. • Maintain a proper IT system operation policy to increase the

availability of the system

2 Handling over of new system to the system operator from PMU

• MIC leaders and relevant stake holders should cooperate closer in post implementation period to ensure the system will be transfer smoothly

3 Lack of IT system management and operation

• Provide frequent training course on the technology which using by new system

• Continuously train the IT staffs of MIC to equip them with update technical standard

4 Resolve the cancellation activities

• MIC should plan to mobilize the government budget to resolve the cancellation packages of project such as MIC2.1, MIC3.1 to setup a new data center to host the e-services and application

Assessment of Bank and Client Performance 1. Project Preparation Period Project preparation at MIC has been very well implemented, but after effectiveness of project, the project plan of MIC has been changed dramatically which created very big challenges for PMU to implement and manage entire project implementation.

82

Changes in project organization structure by 2008, 2009 and 2010 also effected to the project progress at MIC. 2. Project Implementation Period

- Procurement procedure of WB has been change in the midway of project implementation

cause lots of challenging to PMU as the procurement and technical staffs had to learn to familiar with the new process and guideline

- Very complex project organization in MIC subproject with unclear responsibility and accountability required lots of coordination activities from PMU during the project implementation.

- Limited technical capability of PMU and technical working group to create various IT solutions for longer procurement cycle; some complex packages took 3 years to complete.

- Even the WB team is very supportive to the project, but by the end of project, the technical advisors from WB were changed frequently and the technical advisors are less local environment understanding, therefor it created delays in approval of bidding documents.

Lessons Learned

1. IT application project which usually will bring the impact to many different stake-holders and required lots of changes in the operational mind set from the stake holders. Therefore, to ensure the successful of project implementation, a detail and effective project communication and change management plan should be design and implemented throughout the project implementation. This communication and change management activities will help to reduce the resistant from the stake holders and mobilize them to participate more actively to the project.

2. To ensure greater integration of packages, ICT strategy planning should be conducted properly to identify future projects justified through strong business case. ICT as a strategic investments need to be closely aligned to the needs of the organisation’s development and the government’s vision for the service delivery of the future. The result and outcome indicators are an essential part of this planning outputs and they must originate from the formulated business strategies of the organization

3. IT application and e-service development package usually will take very long time to

complete compare to hardware infrastructure packages. Therefor in eGovernment or IT application project, the project owner should consider to plan the application and e-service development package in better alignment maner with purchasing and implementing hardware infrastructure package which to ensure the application and e-services can be operate properly prior to the project closure.

83

4. The introduction of e-services and ICT applications requires a different mind-set in modern organizations i.e. a service management and service delivery improvement outlook. ICT is an integral part of government business operations and can have great socio-economic impact on the service delivery to the citizen and businesses. Making ICT a priority to support and improve services need to be on the mind of all senior government officials. ICT today is positioned to drive improvements and innovations to meet the demands of the constituents

5. The PMU staffs should equipped with proper project management and risk management

prior to the project implementation. Besides that, PMU staffs also should understand the stake holder analysis and business requirement analysis in order to manage and monitor the project and vendors properly.

6. The impact and outcomes of IT project especially for e-service development project only

able to assess properly after live operation of the system for at least 1 to 2 years. Therefor at this moment, its very hard to find the tangible outcomes or impact of this project at this moment.

B. GSO: Executive Summary of Completion Report Overview of Vietnam ICT Development Project

The Government of Vietnam (GOV) recognizes the potential of Information and Communication Technology (ICT) in its various applications to enhance competitiveness, facilitate economic diversification and increase productivity, as well as to improve the efficiency and capability of institutions, particularly in government. GOV has undertaken several initiatives to upgrade information and communications infrastructure. These include: introducing information systems into selected government departments and piloting online services; providing incentives to the ICT industry; developing ICT skills and enhancing awareness of ICT, in government and to some extent in the private sector.

In 2005, the World Bank approved the project (Vietnam ICT Development Project financed by IDA Credit No 4416) for assisting Vietnam to accelerate utilization of ICTs to foster economic development, modernize public administration and improve delivery of public services to citizens and businesses. Through a combination of business process improvements and deployment of technology, the project will support a substantially enhanced government online presence and content at the national and municipal level, through interactive and dynamic portals at the MIC, and the People’s Committees of Hanoi, Danang and Ho Chi Minh City. MIC, and the above-mentioned municipal administrations would roll out e-government services to businesses in areas such as e-procurement and business/land registration. The project commenced in September 2006 and planned to complete by 30 Jun 2011, but due to slow implementation, the project was extended to 31 December 2013.

84

Coherence of the Subprojects

The VNICTDP aimed to promote ICT development in Vietnam at both the national level (MIC and GSO subprojects) and the municipality level (e-Government subprojects) by illustrating:

a. Leadership/systemic effects through: (a) development of ICT policies; (b) development of a common national ICT architecture and interoperability framework; (c) providing improved information/data for timely policy and decision making (GSO); (d) fostering human capacity building at the CIO level.

b. Demonstration effects through: (a) effective changes in business processes; (b) scalability and sustainability of the subprojects; (c) pilot e-applications at the city level; and (d) establishment of a budget to support innovative initiatives in selected ICT areas to involve other stakeholders who are not included in the proposed project. The selection of the cities intended to cover the capital city (where e-government can be best demonstrated), a commercially-developed city (Ho Chi Minh City) and a city in a less-developed central region (Danang).

Major factors affecting/influencing project implementation

1. Strategic top down project planning with mini GSO IT Master Plan at the beginning of the Project using a business driven approach helped GSO to minimize the impact of project changes throughout the Subproject.

2. Commitment of the leader and support from business department during project implementation. Many different types of beneficiaries involved make the Project become more complex and uncontrollable.

3. Government cost norms of Vietnam have a very negative impact on the procurement process as it is very hard for PIU to convince the management agencies for approval of cost estimate for specific packages.

4. The rapid changes of technology and solutions during the procurement process which was often too long (at least one or two years) led to changes required during the project implementation.

5. Changing of procurement processes by the World Bank and inconsistency of IT procurement support from WB cost the GSO Subproject almost 2 years delay from 2009 – 2011.

Lessons learned 1. Good project planning from the beginning will minimize effects of changes to and risks of the Project during implementation. This shows clearly in the GSO Subproject which had only very minor changes in project scope and objectives. In order to achieve this, at the beginning of Project, GSO PIU cooperated with the procurement consultant to plan the Project to ensure close alignment between packages in the Procurement Plan and to ensure that all project activities contributed to at least one specific Project Development Objective.

2. Project must be designed on a business driven basis. All GSO Subproject activities/components have been designed to meet a specific business requirement.

85

3. Have clear plan to manage all stakeholders related to the Project, and ensure that all stakeholder are well informed about the project progress and benefits they will get as well as challenges they will face from the Project.

5. Buying the solution rather than buying equipment. It will minimize the integration issue between different packages.

6. Enterprise architecture is a powerful tool for entire project governance. Since GSO completed GSO6.1a to develop GSO EA, other related IT investment in GSO became clearer and easier to align the investment to GSO development goals. But it was a pity that the delay in selecting GSO6.1a consultant then GSO EA did not adequately support the GSO Subproject.

7. Need a stronger project monitoring and evaluation process, as lots of package implementation has been delayed.

8. GSO leader must be aware to prepare in advance the necessary resource and operational policy for new IT systems which were procured by the Project to maximize the benefits from these new IT systems.

C. Danang City: Executive Summary of Completion Report

Project Objectives The Subproject was designed to enhance the ICT environment in Danang; provide the basic foundation for e-government deployment in the city’s district offices and departments; develop various types of e-government services; promote e-applications for business and develop ICR HR resources. Key Design Features The Subproject consists of four key components which include: (1) Assessment, Strategy, Roadmap, IT Architecture and Standards; (2) Local e-Government Enablement; (3) E-Application, E-Commerce, and (4) ICT HR Development and Awareness-Building. The above four key components (except for e-Commerce) were successfully completed and put in operation in late 2013. Details are as follows: Component Output Assessment Strategy, Roadmap, IT Architecture and Standards

Danang Enterprise Architecture has been developed as a foundation for implementing e-Government for Danang City

Local e-Government Enablement

An ICT Infrastructure which includes Danang MAN, Public Wifi, Data Center, Contact Center etc, has been set up and successfully put into operation in late 2013 to facilitate the delivery of public services to citizens. It consists of more than 300 public services posted on the Danang City Portal to deliver to citizens

E-Application, E-Commerce

315 public services have been developed and provided to citizens and businesses via Danang Portal. E-Commerce was removed since it is part of the national system and

86

financed accordingly. ICT HR Development and Awareness-Building

An ICT Human Resource Master Plan for up to 2020 was developed and issued in 2010. Based on this Master Plan, over 100 leaders joined study tours to countries with advanced e=Government models such as Singapore; Malaysia; Taiwan, China; England; Australia; and Korea. 2,000 officials and 150 IT experts from SMEs went through intensive IT training.

Major Factors Affecting/influencing project implementation: 1. The commitment of the city leaders, the determination, responsibility and dedication of all members of Danang PIU, the valuable support of the Danang DIC, the effort and responsibility of all the contractors; the timely instruction and assistance of the World Bank’s task team members as well as the cooperation of all the beneficiaries are key to the success of the Subproject. 2. Many different types of beneficiaries involved make the project more complex and uncontrollable. 3. Various areas of expertise within IT projects make them difficult to implement since we cannot mobilize enough good experts within the limited budget of the Project. 4. Cost estimates for packages based on Vietnam ICT Norm is rather low and less attractive to qualified bidders. 5. Integration of all the investment systems to ensure their synchronization in operation is extremely difficult for any IT project. 6. The procurement process is often too long (one to two years) given rapid changes in technology and solutions. This leads to delay in contract implementation since several components originally stated in the bidding documents must be adjusted and re-submitted to the Bank and Local Authority for approval. Lessons Learned: Due to the complexity and particularity of IT projects, the Danang Subproject has gone though many difficulties, obstacles, as well as opportunities. Below are some of the key lessons to be drawn from the success of the Danang Subproject. 1. A project can achieve success at the end only if the project manager, beyond professional know-how and technological competence, is not motivated by self-interest but by the interests of the peoples’ success. He must be powerful enough to wield his influence and persuasion upon local leaders of different levels to gain approval for the project as well as to facilitate project implementation. 2. The close cooperation between Danang PIU and all their beneficiaries during project implementation. 3. Transparency, competition and fairness during the process of selecting contractors must be strictly-applied to all packages. This will help award contracts to the winning bidder with real competence. 4. The beneficiaries must be informed of the components which will benefit them and get them involved in the Project from the first steps of the procurement process. 5. New technologies and new solutions must be applied to the Project, taking into account the key factors such as budget, operational costs and systems integration.

87

Letter of June 20, 2014 from MIC PCU with comments on the Bank’s draft ICR:

88

89

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders N/A

90

Annex 9. List of Supporting Documents Project Appraisal Document (Report # 32919-VN, August 12, 2005) Development Credit Agreement Mission Aide Memoires Implementation Supervision Reports (ISRs) Restructuring Paper dated June 26, 2013 Restructuring Paper dated January 12, 2010 Financial Management Supervision documents Procurement Supervision documents Internal Bank Memoranda Ministry of Information and Communication (MIC/PCU): Final Report on Implementation of the ICT Development Project, May 2014 Danang Study: Final Report to Danang Project Implementation Unit of ICT Development Project - DNG 14: Assessing the Status of Danang e-Government Implementation, Leading Associates with KDi Asia, 2014. Impact Assessment Study: Computerized Services Delivery Projects from India and Chile: Bhatnagar, Tominaga, Madon and Bhatia, World Bank IT Staff Working Paper 42147, November 2007 Report on Performance Assessment of the Communications Campaign for the Vietnam ICT Development Project: An Vien Telecommunications and Communications Company and Ministry of Information and Communications PCU, 2013 Project Performance Assessment Report: Consultancy Support on Monitoring and Assessing the Quality and Effectiveness of Implementation of all PIUs, KDi Asia, 2014

91