Framework Development for Evaluating E-government Portals in Myanmar using Analytic

Hierarchy Process

Aung Kyaw Oo 1 , Thein Lwin 4

Department of Computer Technology Defence Services Academy , Pyin Oo Lwin, Mandalay

1 victoraungkyawoo@gmail.com 4 theinlwin@gmail.com

Pyi Aung 3

Depart of Computer Studies, University of Yangon Yangon, Myanmar

3 paung36@gmail.com

Ko Ko Oo 2 Department of Computer Science Defence Services Academy (DSA)

Pyin Oo Lwin, Mandalay Region 2 kko2006@gmail.com

Thant Zin Ohn 5 Department of Computer Science

Defence Services Academy (DSA) Pyin Oo Lwin, Mandalay Region, Myanmar

5 tzinohm@gmail.com

Abstract— The word “e-government” has been initially used by the United States as early as 1993 and the flux of e-government portals has been witnessed worldwide. The formal implementation of e-government project in the Republic of the Union Myanmar was started soon after the promulgation of the Computer Science Development Law (CSDL) in the year 1996. This study extracts the minimally necessary criteria and factors for the evaluation of e-government portals of the various ministries of the Union government as well as the respective governments at all levels in Myanmar. In addition, the study builds an AHP-based Evaluation Model for E-Government Portals (AEMPEG). The proposed AHP model will be applicable for other developing economies whose e-government implementations are in their initial stages. Keywords-- e-government; evaluation framework; portal; developing country; analytic hierarchy process

I. INTRODUCTION Governments at all levels around the world are becoming

more familiar with the word “e-government” in order to get things done more effectively and efficiently regarding government-formulated rules, regulations, norms and information alike. As its original sense is to help citizens, who are the customers of a government’s services of all kinds in a country, e-government systems are being implemented and installed around the world by its own specific type of government to fulfill that unique purpose whether it is termed as government-to-government (G2G) or government-to-citizen (G2C) or government-to-business (G2B) or government-to-employee (G2E) models.

The success and failure of those e-government portals are being evaluated by researchers worldwide. Still, no unanimous

agreement has been made on the type and size of the number of factors for the evaluation. In addition, the researchers should bear in mind that no two similar evaluation criteria should be applied to the cases between the developed and industrialized countries and the developing economies.

This study will broadly consider almost all of the factors and criteria for portal evaluation. Based on the set of all those factors, a set of evaluation factors for e-government portals that should be applicable to the case of any developing countries is proposed. This research paper is composed of eight sections as follows.

II. BACKGROUND LITERATURE E-government portals have been evaluated by researchers

worldwide and a pile of literature on it is to be found for the period of ten years beginning from 2001[4, 5, 11]. Different researchers apply various methods and formulate many different factors and criteria for e-government portals evaluation. Some of them are shown in Table 1.

TABLE I. LIST OF FACTORS FOR WEBSITE EVALUATION [7]

Factors/ criteria

Number of supported

studies

Factors/ criteria

Number of supported

studies Ease of navigation

49 Searching mechanism

26

Content relevancy and usefulness

44 Ease of access 25

Appealing and consistent style

44 Privacy policy 25

Logical structure 39 Quick response to customer

25

Security protection 38 Reliable and innovative

24

978-1-4673-0734-5/12/$31.00 ©2012 IEEE 166

system Interactive communications

37 Accuracy 24

Ease of online transaction

35 Customer service support

23

User-friendly interface

34 Easy to find target information

22

Comprehensive content coverage

33 Online assistance and help

16

Loading and processing speed

32 Data retrieve mechanism

14

Up-to-date content 31 Playfulness 13 Proper multimedia

30 Convenient payment methods

12

Well and quick linkage

29 Know the present location

10

Customized service

28 Overview of selected items

6

Easy to understand and read

27 Easy to cancel or modify order

5

III. E-GOVERNMENT IN MYANMAR After the enactment of CSDL in 1996, e-National

Taskforce has been formed and the respective government ministries especially Ministry of Communications, Post and Telegraphs (MCPT), Ministry of Science and Technology (MOST) and Ministry of Education (MOE) with the collaboration of Myanmar Computer Federation (MCF) try their best to develop and implement the ICT projects formulated and guided by the national policymakers through ICT Master Plans. As early as 2003, some government ministries published their websites so as to convey the information related to role, responsibilities, and services within the ministry concerned. Until 2008, almost 28 government ministries out of 36 came online. However, the focus was still just to be present online and the information was not sufficiently and properly supplied for neither the citizen customers nor the business entities.

A. E-Readiness in Myanmar

During this initial period from 2001 until 2008, Myanmar government strove to implement many e-government projects with the technical assistance from countries like Japan, Republic of Korea, Singapore and Malaysia. Some of these projects include smart school, e-visa, e-passport, government online system, electronic document management system (EDMS) among others.

In 2004 world wide ranking in e-government readiness index, Myanmar ranked 123 out of 191 economies. The ranking worsens in the year 2008 and in 2012 resulting 144 and 141. This shows the country needs to pay specific attention to introduce an online culture among the various communities as well as citizens in the country.

IV. PROBLEM STATEMENT It is not an easy task to specifically define the formal

statement to reflect the intention and objectives of the

researchers at the very start of each business project or academic research. However, after careful consideration of the prevailing situation in practice and intensive review of the literature at hand, the authors are obliged to depict the problem statement of the current research as follow;-

1) What are the important factors for evaluating e-government portals in Myanmar?

2) Are there any other evaluation factors that need to be omitted in the proposed model?

3) Can the factors be applicable for the evaluation of other developing countries’ e-government portals?

In the current study, the reliable and verifiable answer for the three research questions are to be tested and proved technically and scientifically.

V. RESEARCH METHODOLOGY Throughout the current study, research methods such as

online search and library search were applied together with survey assisted and participated by many respondents. The main research methodology of our study is AHP.

Analytic Hierarchy Process (AHP) is developed by Prof. Thomas L. Saaty. It is a mechanism to be applicable to many decision and management problems especially if it is complicated, involved by many decision makers, encountered with tangible and intangible multiple criteria [13]. It is one of the best methodologies in the domain of Multiple Criteria Decision Making (MCDM) problems. The nine-point scale of AHP has already been scientifically tested, verified and validated by researchers worldwide since its introduction around 1970s [13].

TABLE II. SAATY’S NINE-POINT AHP SCALE [13]

Intensity of Importance

Definition Explanation

1 Equal Importance Two activities contribute equally to the objectives.

3 Weak Importance of one over another

Experience and judgment slightly favor one activity over another.

5 Essential or Strong Importance

Experience and judgment strongly favor one activity over another.

7 Demonstrated Importance

An activity is strongly and its dominance is demonstrated in practice.

9

Absolute Importance

Experience and judgment absolutely favor one activity over another.

2,4,6,8 Intermediate values between the two adjacent judgments

When compromise is needed.

The methodology is very helpful and effective when there are many decision makers and it needs a unanimous decision among them. The participating decision makers have to make pair-wise judgments based on the proposed problem hierarchy.

978-1-4673-0734-5/12/$31.00 ©2012 IEEE 167

Another advantage in using AHP is the number of sample size for the participants. In order to successfully carry out the AHP-based research, only a small number of samples are needed, e.g. 15, 20 or 25 participants are acceptable [6, 9].

VI. MODELING EVALUATION FRAMEWORK Among the 30 factors shown in Table 1, some of them are

found inappropriate for the cases of developing economies. According to the authors’ research experience and intuition, the following factors were initially selected among the factors shown in Table I;-

a) Ease of Navigation (EON) b) Content Relevancy and Usefulness (CRU) c) Security Protection (SP) d) Interactive Communications (IC) e) Ease of Online Transaction (EOT) f) User Friendly Interface (UFI) g) Comprehensive Content Coverage (3C) h) Loading and Processing Speed (LPS) i) Up-to-Date Content (UDC) j) Proper Multimedia (PM)

It is evident that only 10 factors out of 30 were chosen as the authors believed and paid credit to the respective researchers who had done a large number of supported studies on the factors concerned. Some of the factors such as “perceived ease of use” and “perceived usefulness” tried in [3, 10, 12] is somewhat similar to the chosen factors (b) and (h). After selecting the ten potential evaluation criteria for our research, the AHP model has been built as shown in the following Fig. 1.

Figure 1. Proposed AHP Model for E-government Portal Evaluation in Myanmar

A. Participants In the AHP model, the goal is “To evaluate e-government

portals in Myanmar” and a single layer includes ten evaluation factors. In carrying out the research, our proposed AHP model needs to get the decision data from the respective respondents regarding the chosen evaluation factors shown above. The research has been planned to carry out in two phases, viz. pilot technical survey and scientific survey. At this initial stage of pilot technical survey, a set of 18-page survey questionnaire has been individually sent to 23 respondents, who are government officials, representing three government ministries in Myanmar. Out of chosen 23 respondents, only 19 respondents responded and sent back the requested survey forms in time. So, the return rate of survey is 82.61%. Among those 19 participants, the responses and feedback of 3 persons were not up to par with the requirement of the current study. Therefore, only 16 participants which represent 69.56% of the sample size were taken into account for the data analysis of the pilot technical survey. Regarding the validity of the number of participating respondents, it was found out according to literature that 16 participants could be acceptable for this technical pilot study [9].

TABLE III. DEMOGRAPHIC PROFILE OF THE RESPONDENTS

Ministry No. of Respondents

Average Age (Year)

Average Education

Commerce 1/1 39.00 Master’s degree Defence 11/17 29.27 Master’s degree Science & Technology

4/5 34.50 Master’s degree

Total

16/23 34.25 Master’s Degree

B. Data Collection and Analysis After the required data had been systematically collected,

they are generated with the aid of decision analysis tool, Expert Choice 2000 Second Version. From that analysis, inconsistency ratios of each of the 16 valid respondents are found to be (0.28, 0.77, 0.37, 0.37, 0.2, 0.22, 0.22, 0.17, 0.14, 0.12, 0.25, 0.15, 0.05, 0.21, 0.33 and 0.31). It is evident that the AHP inconsistency ratio of only one out of 16 eligible participants is directly applicable to the current study. In using AHP method, the inconsistency ratio of any pair-wise comparison matrix has to be less than or equal to 0.10 [13].

TABLE IV. AGGREGATED AHP-BASED RANKING OF EVALUATION FACTORS

Evaluation Factor AHP value Rank Security Protection (SP) 0.186 1 Up-to-Date Content (UDC) 0.150 2 Loading and Processing Speed (LPS) 0.105 3 Interactive Communications (IC) 0.104 4 Ease of Online Transaction (EOT) 0.096 5 Comprehensive Content Coverage (3C) 0.083 6 User Friendly Interface (UFI) 0.080 7 Content Relevancy and Usefulness (CRU) 0.077 7 Proper Multimedia (PM) 0.061 9 Ease of Navigation (EON) 0.058 10 Inconsistency= 0.004 1.000

PM

UDC

LPS

3C UFI

EOT

IC

SP

To Evaluate

E-governm

ent Portals in Myanmar

CRU

EON

978-1-4673-0734-5/12/$31.00 ©2012 IEEE 168

Ranking of E-government Evaluation Factors

0

0.05

0.1

0.15

0.2

SPUDC

LPS IC

EOTCCC UFI

CRU PMEON

Evaluation Factors

Aggr

egat

ed A

HP

Val

ues

Figure 2. Evaluation Factors’ Ranking in Column Graph

From the technical pilot survey, it is found out that the factor SP is the most important and preferred among 10 proposed factors and the factor EON is the least preferred one. The participants favor the factors UDC and LPS as the important ones after the factor SP as well. From the study, it can be assumed that the two factors PM and EON be left out in the proposed model as their significant AHP value is negligible when compared to the AHP values of other factors.

VII. A CASE STUDY: PORTAL EVALUATION IN MYANMAR

There are thirty-two union-level government ministries in Myanmar. In Myanmar, the effort to upload e-government websites has been started with the full support of the then government (The State Peace and Development Council-SPDC). Beginning from the year 2005, some government ministries have uploaded ministry-level websites. Until the end of 2008, there are as much as 28 government web portals for Myanmar people. Initially, most employees and staff in those government ministries did not have much experience, skills and expertise to develop and launch e-government portals and so they have to depend on external ICT specialists and technicians to design and implement those portals. The turn of 2011 saw the new development of e-government environment in Myanmar.

According to myanmar.com website updated last on July 12, 2008, there were (24) government ministries’ websites in Myanmar. But on yatanarpon.com.mm website updated on March 9, 2012, it just shows only (12) government ministries’ websites on the directory. It may be that the other remaining union government ministries have been trying their best to launch their own ministry web portals in line with the changing environment of the socio-economic atmosphere in Myanmar.

A. Website Selection Among many ministerial level websites in Myanmar, the

authors chose only (6) portals within their own selection criteria as follows;-

1) Bilingual format 2) Formal domain name or URL 3) Updated content

4) Contact e-mail We regretfully left the good website of the Ministry of

Industry (1) since the ministry is no more existed itself but it was changed to become the Ministry of Industry by combining two ministries, Industry (1) and Industry (2), and so the authors thought that the functions and activities of both of the ministries will be combined under the roof of the newly set-up ministry, Ministry of Industry. It was supposed that the new web portal of the Ministry of Industry will be existed soon. The same situation also goes to the website of the Ministry of Labor because it could not be loaded for some reasons at the time of analysis. If it had not happened, the Labor portal would have certainly been included in the current analysis.

TABLE V. AHP-BASED RANKING OF EVALUATION FACTORS FOR CASE STUDY

Evaluation Factor AHP value Rank Up-to-Date Content (UDC) 0.328 1 Content Relevancy and Usefulness (CRU) 0.151 2 Comprehensive Content Coverage (3C) 0.109 3 Interactive Communications (IC) 0.104 4 Ease of Online Transaction (EOT) 0.101 5 Ease of Navigation (EON) 0.072 6 User Friendly Interface (UFI) 0.038 7 Loading and Processing Speed (LPS) 0.038 7 Proper Multimedia (PM) 0.037 9 Security Protection (SP) 0.024 10 Inconsistency= 0.10 1.000

The case study results in the factor UDC as the most

preferred one and the factor SP as the least important one among 10 proposed factors (See TABLE V). The factor UDC is over two times more important than the second most preferred factor CRU. From the case study, it can be assumed that the factor SP should be left out from the current AHP model as the AHP value of SP is somewhat negligible compared to the AHP value of other factors.

TABLE VI. AHP-BASED RANKING OF E-GOVERNMENT PORTALS FOR CASE STUDY

Ministry Website AHP Value

Rank

Commerce (MOC)

www.commerce.gov.mm

0.345 1

Agriculture & Irrigation (MOAI)

www.moai.gov.mm 0.249 2

Science & Technology (MOST)

www.most.gov.mm 0.135 3

Communications, Posts and Telegraphs (MCPT)

www.mcpt.gov.mm 0.126 4

Transport (MOT)

www.mot.gov.mm 0.081 5

Sports (MOS) www.mosports.gov. mm 0.064 6 Overall Inconsistency= 0.08

1.000

978-1-4673-0734-5/12/$31.00 ©2012 IEEE 169

Among the chosen e-government portals, the portal of MOC stood first in the ranking followed by that of the MOAI (See TABLE VI). Although it ranked last, the portal of MOS is not much different from that of the portal of MOT compared in the contents and layout of their websites.

Ranking of E-government Portals for Case Study

0 0 0 0 0 0

0.345

0.249

0.135 0.1260.081 0.064

00.050.1

0.150.2

0.250.3

0.350.4

MOCMOAI

MOSTMCPT

MOTMOS

Government Ministries

AH

P V

alue

Figure 3. E-government Portals’ Ranking in Column Graph

VIII. RESULTS AND DISCUSSION From the proposed AHP model and the case study, it is

obvious that three factors, viz. UDC, IC and EOT ranked in the upper half among the 10 factors although the rankings of other factors change inconsistently. It can be firmly said that these three factors could be reliably used not only for the proposed AEMPEG but also for other evaluation models.

However, it is very interesting to see the rank reversal between the ranking of the proposed AEMPEG model and the ranking of the case study. The first-ranked factor SP in AEMPEG stood last in the case study ranking. This is one of the expectations of the authors to explore more about (i) the methodology itself, (ii) the number of evaluation factors and (iii) the number of participants in the later stages of the current study.

IX. CONCLUSIONS Throughout the study, it is thoroughly explained about the

formation of the research, the AHP methodology and the results obtained out of the technical pilot study. Currently, it can be concluded that the three factors UDC, IC and EOT are the important factors for evaluating e-government portals in Myanmar. In addition, the factors PM and EON should be omitted and potentially deleted from the proposed model.

In the next stage of the study, it needs to be tested whether the proposed AEMPEG model and the 10 factors are applicable for the evaluation of other developing countries’ e-government portals.

ACKNOWLEDGMENT The authors are very grateful to the Office of the Chief of

Armed Forces Training and the Defence Services Academy for allowing us to initiate this project and we would like to thank the survey respondents, questionnaire analysts, doctoral candidates, faculty members and the support staff in the Department of Computer Technology in DSA. Thanks are also due to reviewers. The generous R & D spending of the

Ministry of Defence to publish this paper is also highly appreciated and acknowledged.

REFERENCES [1] Al-Nuaim, H. A., An Evaluation Framework for Saudi E-government,

Journal of E-government Studies and Best Practices, DOI- 10.5171/2011. 820912

[2] Ataloglou, M. P. and Economides, A. A., Evaluating European Ministries’ Websites, International Journal of Public Information Systems, Volume 2009:3

[3] Colesca, S. E. and Liliana, D., E-government Adoption in Romania, International Journal of Human and Social Sciences 4:14 2009

[4] Huang, Z., A comprehensive analysis of U.S. counties’ e-Government portals: development status and functionalities, European Journal of Information Systems (2007) 16, 149-164

[5] Lai, C. S. K. and Pires, G., Testing of a Model Evaluating E-government Portal Acceptance and Satisfaction, The Electronic Journal Information Systems Evaluation, Volume 13, Issue 1, 2010, pp 35-46

[6] Oo, Aung Kyaw, A Buyer-centric Study on the Critical Success Factors in Public Procurement: A Case Study on E-government Implementation in Myanmar, Ph.D. Dissertation, Seoul National University, February, 2009

[7] Osman et. al., A New COBRAS Framework to Evaluate E-government Services: A Citizen Centric Perspective, tGov workshp’11, Brunel University, West London, March 17-18, 2011

[8] Oyedele, A. and Koong, K. S., Framework for Evaluating Global E-government Websites; Accessed online at www.swdsi.org/swdsi06/ Proceedings06/Papers/MIS14.pdf

[9] Perera, N. and Sutrisna, M., The Use of Analytic Hierarchy Process in the Analysis of Delay Claims in Construction Projects in the UAE, The Built and Human Environment Review, Vol. 3, Special Issue. 1, 2010

[10] Rokhman, A., E-Government Adoption in Developing Countries; the Case of Indonesia, Journal of Emerging Trends in Computing and Information Sciences, Volume 2 No.5, MAY 2011 ISSN 2079-8407

[11] Thompson, K. M., McClure, C. R and Jaeger, P. T., Evaluating Federal Websites: Improving E-government for the People, George, J. F. (Ed.). Computers in Society:Privacy, Ethics & the Internet. Upper Saddle River, N.J.: Prentice-Hall

[12] Wangpipatwong, S., Chutimaskul, W. and Papasratorn, B.,Understanding Citizen’s Continuance Intention to Use e-Government Website: a Composite View of Technology Acceptance Model and Computer Self-Efficacy, The Electronic Journal of e-Government, Volume 6 Issue 1, pp 55 – 64, 2008

[13] Wind, Y. and Saaty, T. L., Marketing Applications of the Analytic Hierarchy Process, Management Science, Vol. 20, No. 7, July 1980

Aung Kyaw Oo obtained Ph.D. in Engineering from Seoul

National University, Korea. His research interests include decision theory, fuzzy logic and knowledge systems. He is a member of International Association of Engineers (IAENG), International Society for Multiple Criteria Decision making (ISMCDM) and a Regional Ambassador of the International Association of Higher Education Teaching and Learning (HETL) to Myanmar. He has published two books and has written many conference and journal articles. He is on the editorial committee and reviewers’ committee of many international academic journals. He is awarded the Best Review Award (2009) from the Contemporary Management Research (CMR) Journal. He is currently a lecturer at the Defence Services Academy in Pyin Oo Lwin in Myanmar.

Ko Ko Oo got Ph.D. (IT) from University of Computer Studies, Yangon, Myanmar. His research interests include natural language processing and wireless systems. He is

978-1-4673-0734-5/12/$31.00 ©2012 IEEE 170

currently a lecturer at the Defence Services Academy in Pyin Oo Lwin in Myanmar.

Pyi Aung got B.Sc. degree from Defence Services Academy with distinctions in all six subjects. His research interests include network security, database management, e-government platform and web applications. He is currently taking Master’s course in Computer Science at the University of Yangon, Myanmar.

Thein Lwin got Ph.D. in Computer Science from Defence Services Academy, Myanmar. His research interests are image applications computer networking. He is currently a lecturer at the Defence Services Academy in Pyin Oo Lwin in Myanmar.

Thant Zin Ohn earned Master’s degree in Applied Science from the University of Computer Studies, Yangon and got Ph.D. in Computer Science from Defence Services Academy, Myanmar. His research interests include parallel computing and computer networking. He is currently a full professor at the Defence Services Academy in Pyin Oo Lwin in Myanmar.

978-1-4673-0734-5/12/$31.00 ©2012 IEEE 171