2014 UN E-GOVERNMENT SURVEY: E-GOVERNMENT AS AN …unpan1.un.org/intradoc/groups/public/documents/un-dpadm/unpan050717.pdf · 2014 UN E-GOVERNMENT SURVEY: E-GOVERNMENT AS AN ENABLER

2014 UN E-GOVERNMENT SURVEY: E-GOVERNMENT AS AN ENABLER FOR COLLABORATIVE GOVERNANCE Overview paper in support of the Expert Group Meeting (EGM) in New York, USA 4-5 December 2012 Consultant: Kim Andreasson, DAKA advisory1 Version date: November 30, 2012

1 The author can be reached at [email protected]

Overview paper in support of the Expert Group Meeting (EGM) in New York, USA, 4-5 December 2012 By Kim Andreasson

November 30, 2012 Page 2 of 32

Table of Contents

1. INTRODUCTION .................................................................................................................................... 3

1.1 BACKGROUND TO THE UN E-GOVERNMENT SURVEY ........................................................... 3 1.2 THE UN E-GOVERNMENT SURVEY IN CONTEXT: FROM THE MILLENNIUM

DEVELOPMENT GOALS TO THE SUSTAINABLE DEVELOPMENT GOALS ................................. 4 1.3 OBJECTIVE OF THE OVERVIEW PAPER AND THE EGM ........................................................... 4 1.4 FRAMEWORK FOR THE EGM......................................................................................................... 5

2. RECENT E-GOVERNMENT TRENDS AND UN SURVEY DEVELOPMENTS ............................ 6

2.1 RECENT INFORMATION SOCIETY REPORTS.............................................................................. 6 2.1.1 SUMMARY OF TRENDS........................................................................................................... 10

2.2 UN SURVEY DEVELOPMENTS..................................................................................................... 11 2.2.1 UN SURVEY CHALLENGES AND OPPORTUNITIES............................................................. 12

3. REVIEW OF OVERALL METHODOLOGY .................................................................................... 12

3.1 E-GOVERNMENT DEVELOPMENT INDEX................................................................................. 13 3.1.1 Preliminary recommendation(s) ................................................................................................ 13

3.2 TELECOMMUNICATION INFRASTRUCTURE INDEX.............................................................. 13 3.2.1 Preliminary recommendation(s) ................................................................................................ 13

3.3 HUMAN CAPITAL INDEX.............................................................................................................. 14 3.3.1 Preliminary recommendation(s) ................................................................................................ 14

3.4 ONLINE SERVICE INDEX .............................................................................................................. 14 3.4.1 Preliminary recommendation(s) ................................................................................................ 14

4. INDICATOR DEVELOPMENT AND SIX E-GOVERNMENT INITIATIVES FOR 2014........... 15

4.1 INTRODUCTION TO THE OSI METHODOLOGY........................................................................ 15 4.2 AT THE EGM: A SEARCH FOR THEMES AND TRENDS............................................................ 15 4.3 WHOLE-OF-GOVERNMENT ......................................................................................................... 16 4.4 E-PARTICIPATION.......................................................................................................................... 17 4.5 MULTI-CHANNEL SERVICE DELIVERY .................................................................................... 18 4.6 EXPANDING USAGE ...................................................................................................................... 19 4.7 DIGITAL DIVIDE AND VULNERABLE GROUPS........................................................................ 21 4.8 OPEN GOVERNMENT DATA......................................................................................................... 22

5. SURVEY IMPLEMENTATION PROCESS ....................................................................................... 23

5.1 CURRENT APPROACH................................................................................................................... 23 5.1.1 Preparation................................................................................................................................ 23 5.1.2 Execution ................................................................................................................................... 24

5.2 LIMITATIONS AND BOUNDARIES.............................................................................................. 25 5.2.1 Overall approach ....................................................................................................................... 25 5.2.2 Measuring the back-end from the front-end............................................................................... 25 5.2.3 Supply (availability) is trumping demand (usage) ..................................................................... 25 5.2.4 A lack of transparency ............................................................................................................... 25

5.3 ALTERNATIVES AND SUPPLEMENTAL OPTIONS................................................................... 26 5.3.1 Automation and innovation........................................................................................................ 26 5.3.2 From websites to search engines ............................................................................................... 26 5.3.3 Interviews and desk research ..................................................................................................... 26 5.3.4 Link measurement to development objectives ............................................................................ 26 5.3.5 Mobile assessment ..................................................................................................................... 26 5.3.6 Demand that the suppliers supply the data ................................................................................ 27 5.3.7 Increase transparency................................................................................................................ 27



6. TOWARDS THE FUTURE................................................................................................................... 27

6.1 GOALS OF THE EGM ...................................................................................................................... 27

APPENDIX I: UN SOURCES................................................................................................................... 28

APPENDIX II: REFERENCES ................................................................................................................ 29

1. INTRODUCTION This overview paper was prepared in support of the United Nations Expert Group Meeting (EGM) entitled “E-government as an enabler for collaborative governance,” which takes place in New York, USA, on 4-5 December, 2012. In turn, the EGM is designed to support the development of the 2014 United Nations (UN) e-government survey. For those attending the EGM, this overview paper is important background reading in order to fully contribute at the meeting, in particular at the working group sessions, the details for which can be found in subsection 1.4 and the entire section 4. Although there is some replication, particularly in regards to the description of the survey methodology as it has not changed, this paper seeks to avoid overlap to previous efforts and participants are also encouraged to read the public versions of the overview papers in support on the 2008 and 2010 EGMs (Millard 2008 and Andreasson 2010, respectively).2

1.1 BACKGROUND TO THE UN E-GOVERNMENT SURVEY The UN e-government survey began in earnest in 2003 (a 2001 pilot did not cover all UN Member States and can be considered a predecessor of the reports). Appendix I of this document provides a list of the UN e-government survey program to date. The objective remains the same today as when it started: benchmarking e-government development among UN Member States in order to “inform and improve the understanding of policy makers’ choices to shape their e-government programs” (UN 2004). This is done by measuring “the willingness and capacity of countries to use online and mobile technology in the execution of government functions” (UN 2010). Most recently, “to recognize the key role that e-government – and e-governance – can play in support of the establishment of effective institutional linkages necessary for sustainable development (UN 2012).

2 For those interested, the public versions of the 2008 and 2010 overview papers can be found here: http://unpan1.un.org/intradoc/groups/public/documents/un/unpan035535~2.pdf and http://unpan1.un.org/intradoc/groups/public/documents/un/unpan047965.pdf



The UN e-government survey is a flagship publication of the Division for Public Administration and Development Management (DPADM) of the United Nations Department of Economic and Social Affairs (UNDESA) and it consistently ranks as one of the top 10 downloaded publications of UNDESA.

1.2 THE UN E-GOVERNMENT SURVEY IN CONTEXT: FROM THE MILLENNIUM DEVELOPMENT GOALS TO THE SUSTAINABLE DEVELOPMENT GOALS The 2003 World Public Sector report, in which the e-government survey was unveiled, explained that “[t]he conceptual framework adopted by this Survey is the vision of human development provided by the UN Millennium Declaration” (UN 2003b, page 132). It continued: “in keeping with the UN Millennium Declaration, the focus of the Survey is on provision of socio-economic services to the population through the use of e-government as a programmatic tool, as well as on participation” (UN 2003b, page 132). In this context, the survey came to evaluate a government’s national portal(s), and the ministries of education, finance, health, labor, and social welfare, all indicative of development for all and closely associated with the Millennium Development Goals (MDGs). In light of the UN Conference on Sustainable Development 2012, also known as Rio+20, the survey expanded its remit to also capture information and services relevant to the environment.3 As the target year of 2015 for achieving the MDGs approaches, the UN Secretary-General has initiated several projects to develop the post-2015 global development agenda.4 While such efforts are likely to be deeply rooted in socio-economic, participatory and environmental performance, it is also a good opportunity to assess how the UN e-government survey can evolve to meet sustainable development goals of the future. This also means that the focus on development continues to be more important than rankings. While stakeholders around the world often tout their position(s) in the survey, this paper, like the UN survey itself, only refers to leading countries and indicators in order to improve the measurement of development.

1.3 OBJECTIVE OF THE OVERVIEW PAPER AND THE EGM This overview paper seeks to support the EGM in the biennial review of the UN e-government survey, which assesses the 193 UN Member States’ e-government development according to a quantitative composite index consisting of an online service

3 The objectives of the Rio conference can be found on its website: http://www.uncsd2012.org/ 4 See, for example, http://www.un.org/en/development/desa/policy/untaskteam_undf/ and http://sustainabledevelopment.un.org/



index (web measurement), a telecommunications infrastructure index, and a human capital index, each of which carries a weight of 1/3. By reading this paper and adhering to the framework for the EGM (next section), the hope is that we can all provide the UN with concrete suggestions for enhancements to the survey, in particular in regards to six initiatives for 2014 (section 4 of this paper). Based on the EGM, the consultant will prepare a proposed 2014 e-government survey questionnaire on behalf of all participants.

1.4 FRAMEWORK FOR THE EGM In order to ensure a productive meeting, EGM participants should read this paper and bring suggestions for enhancements in regards to:

a) Methodology, or comments on those presented here in section 3; b) Trends and themes to consider for the forthcoming 2014 e-government survey

based on the six initiatives provided in section 4; c) Survey implementation, or comments on those presented here in section 5; and, d) Any other comments or suggestions.

To facilitate a productive meeting, the EGM agenda has been designed in a way that mixes high-level plenary sessions with breakout working groups, which are based on the six initiatives described in section 4: whole of government, multi channel service delivery, the digital divide and vulnerable groups, e-participation, usage, and open government data. These interactive sessions offer the opportunity for participants to discuss in greater detail how the online survey index should evolve and what themes it should cover to better meet e-government development. The working group for each initiative will discuss trends and emerging issues related to their focus area, review the current UN survey and add new suggestions as well as update the current questions. To do so, each group will have a facilitator, a rapporteur and a friend of the rapporteur, all of whom will be provided with specific guidelines to achieve the overall objective of delivering concrete suggestions for enhancement through the following format:

a) Brief presentation of the topic by the facilitator in light of the overview paper; b) A discussion of the topic moderated by the facilitator; c) A review of indicators from the current survey questionnaire; d) Any additional suggestions for themes or emerging trends for the topic; and, e) Concluding remarks by facilitators.

After each session, the rapporteurs are responsible for delivering written suggestions to the entire expert group in light of the discussion. The consultant will take the recommendations into account when finalizing the proposed 2014 survey questionnaire to DPADM.



2. RECENT E-GOVERNMENT TRENDS AND UN SURVEY DEVELOPMENTS This section reviews a selection of international e-government related reports since the 2010 EGM in order to illustrate recent trends, followed by a summary of the latest UN reports. This review builds on similar efforts from the 2008 overview paper, which also provided a summary table of the main comparable measurement frameworks, and the 2010 overview paper, which offered an updated review of the subsequent two years.

2.1 RECENT INFORMATION SOCIETY REPORTS The International Telecommunication Union (ITU) continues to publish its series on Measuring the Information Society. It features two benchmarking tools: the ICT Development Index (IDI) and the ICT Price Basket (IPB), the former which captures progress in ICT development in 155 countries through a composite index of 11 indicators and the latter which tracks the cost and affordability of ICTs in more than 160 countries through a composite index of fixed line, mobile and broadband tariffs over time. The 2011 report focuses on broadband issues such as capacity, speed and quality, as well as the role of education, income, gender, age and location in increasing Internet usage (ITU 2011). In regards to the former, it concludes that there has been a rapid rise in broadband uptake over the past five years but that governments can do more to maximize its impact, such as creating an enabling regulatory framework and improving affordability through open markets. In regards to the latter, it suggests the success of mobile phones can be replicated with mobile Internet access through smart phones, and that public access points can improve uptake, particularly among rural populations. It further suggests the provision of relevant content as a factor to improve usage. The 2012 report analyzes revenue and investment in telecommunications and usage from a global capacity view in terms of measuring communication and capacity in bits and bytes (ITU 2012). The ITU emphasizes the importance of creating strong policy measures to support ICT development as well as its contribution to broader economic goals, for example by encouraging large scale investments in telecommunication infrastructure. The report also calls for more and improved statistics on data traffic and network capacity in order to help policy-makers improve their understanding of the digital divide from a capacity point of view. The United Nations Conference on Trade and Development (UNCTAD) continues to publish its Information Economy Report, most recently in 2011 (UNCTAD 2011). While it focuses on the private sector, important lessons can be learned by looking at the topic from this angle. For example, the overall recommendations provided, all involve government:

Enhance quality of ICT infrastructure, including opportunities associated with mobility, where governments are called upon to meet the needs of enterprises;



Enhance business use of ICT where government can help make services available and affordable; Stimulate the ICT sector by creating competition, offering training, and increasing trust; and, Use ICT to create a provision for effectiveness and reform in order to reduce cost of services and expand their reach.

A separate effort comes from The Partnership on Measuring ICT for Development, which is an international multi-stakeholder initiative of about a dozen international organizations and led by the ITU and UNCTAD (The Partnership on Measuring ICT for Development website). The objective of the partnership is to identify ICT indicators that can be compared on a global basis and, in the process, assist developing countries to collect them and track progress over time, which is currently being done for 68 countries. One of its many relevant publications, entitled Framework for a Set of Egovernment Core Indicators, seeks to identify basic indicators specifically as it relates to e-government, an initiative led by the UN Economic Commission for Africa (ECA), in collaboration with the Economic Commission for Latin America and the Caribbean (ECLAC), the ITU and UNDESA (The Partnership on Measuring ICT for Development 2011). In Europe, the European Commission (EC) adopted the European eGovernment Action Plan 2011-2015: Harnessing ICT to promote smart, sustainable & innovative Government (EC 2010). The plan is designed to implement the goals declared at the 2009 Malmö Ministerial Declaration on eGovernment (EC 2009), which include.

Empower citizens and businesses to use e-government services designed around users, and increase access and transparency; Reinforce mobility in the single market through e-government; Use e-government to improve effectiveness while promoting a sustainable economy; and, Create appropriate legal and technical enablers by setting policy priorities.

These initiatives seek to supplement the efforts of the Digital Agenda, the European strategy for a digital economy by 2020, by contributing to two key objectives: fulfilling the targets of having key cross-border services available online by 2015 and having half of EU citizens use e-government services by the same year. The Digital Agenda, meanwhile, launched in 2010, continues to focus on seven key areas: a vibrant digital single market, interoperability and standards, trust and security, fast and ultra fast Internet access, research and innovation, enhancing digital literacy, skills and inclusion, and ICT-enabled benefits for EU society (EC Digital Agenda website). The premiere benchmarking report in Europe, The 9th Benchmark Measurement, Digitizing Public Services in Europe: Putting ambition into action continues to measure maturity across “20 basic public services” (EC 2010b). Similarly to UNDESA it uses a progressive stage model to determine online sophistication across Member States, namely: (i) information, (ii) one-way interaction, (iii) two-way interaction, (iv) transaction, and (v) targetisation/automation. According to this approach, the average score is now 90%, up from 83% in 2009.



Because several countries are achieving perfect scores (100%) and are increasingly looking to e-government to transform the public sector to achieve greater effectiveness, the benchmarking effort continuously seek to evolve to meet new objectives and trends. The current report draws on three main sources of data: (i) online service analysis across some 10,000 websites, (ii) surveys with nominated representatives from Member States, and (iii) evaluations carried out by experts. New indicators tested for future purposes in the current benchmark include the maturity of “life events” and customer journeys for starting a business and losing and finding a job as well as the availability of key enablers, such as in the back-office, such as eID, interoperability guidelines, open standards and single sign-on features. But, as the report notes, availability is only a first step. Therefore, an increasing focus is placed on take-up and closing the digital divide, as well as better demonstrating outcomes (the value of investment) achieved through e-government. To do this, the report proposes the following efforts to enhance the benchmark approach:

Stabilize the scope of measurement and offer a broader set of benchmarks for countries (and regions) to learn from; Establish Action Learning Groups (ALG) to develop a process for indicator innovation, piloting, and sharing of good practices; and, Increase comparisons to leading international practices.

Additional EC e-government publications released during 2011 and 2012 can be found on the EC e-government website (EC eGovernment website). The Organisation for Economic Co-operation and Development (OECD) published an update to the Government at a Glance publication in 2011 (OECD 2011). The report assesses government performance for 42 countries across 10 policy domains through 58 indicators of good government, including e-government strategies, e-procurement, and uptake of e-government services. A key finding, drawn from its 2010 Survey on e-Government Structural and Economic Data, is that many OECD countries look to e-government as an enabler to public sector reform and a tool to do more with less in the aftermath of the financial crisis. For example, when asked about the main priorities of e-government initiatives, OECD countries most frequently listed reducing administrative burdens (cited by 96% of those who responded), followed by cost cutting (86%), spurring innovation (74%) and improving effectiveness and responsiveness (67%). In regards to usage, the report attributes low levels of constituent consumption to the inability of vulnerable segments of society to use digital channels due to lack of awareness or lack of IT skills. At the same time, it views broadband proliferation and growth in m-government as potential solutions. M-government is also the theme of M-Government: Mobile Technologies for Responsive Governments and Connected Societies, another relevant recent OECD publication done jointly with the ITU and in collaboration with UNDESA (OECD/ITU 2012). It highlights the growth of mobile communication technologies and their impact on economic and



social development. Defining m-government as “the adoption of mobile technologies to support and enhance government performance and foster a more connected society,” the report emphasis the growth in usage: the world went from a global penetration rate of 5% in 1998 to 55% in 2008 and towards an estimated 96% in 2018. According to the ITU World Telecommunication/ICT Indicators database, access to mobile networks (2G, 3G, and 4G) is currently available to 90% of the world population, including 80% of whom live in rural areas. Given the rapid uptake, the report highlights the critical potential of mobile technologies for improved governance and economic and social progress. The World Economic Forum (WEF) measures the extent to which 142 economies around the world use ICT to improve economically and socially through its Networked Readiness Index (NRI) as presented in The Global Information Technology Report 2012, which is produced in collaboration with INSEAD (WEF 2012). However, after announcing in the 2010 report (WEF 2010) that the NRI was being revised, the 2012 edition features exciting changes, including the addition of a sub-index on outcomes. Under the theme of Living in a Hyperconnected World, the report describes the rapid growth of mobile devices, big data, and social media, all whom it consider drivers of hyperconnectivity, defined as an environment where the Internet and associated content is available all the time and at any time. To reflect such transformation, a number of changes to the methodology were made for the first time since the framework was launched in 2002. The primary revisions include (WEF 2012, page 4):

Introduction of an ICT impact sub-index, a fourth sub-index (in addition to the environment, readiness, and usage) to measure the effect of ICT on both the economy and society, a complex but increasingly important task; Focusing the readiness sub-index on infrastructure, affordability, and skills; Inclusion of acts of innovation and entrepreneurship in the ICT environment sub-index, in addition to ICT uptake; Separation of usage from impacts in the usage sub-index along with the creation of an impact sub-index (see above); and, An update to the selection of variables to take into account technology changes; for example, removing the number of fixed telephone lines and adding mobile broadband subscriptions; the composite index now features 53 variables.

Transformation is also the theme of a stand-alone WEF report entitled The Future of Government: Lessons Learned from around the World (WEF 2011). The report presents the FAST (Flatter, Agile, Streamlined, Tech-enabled) framework and recommend governments to move towards such approach in order to be effective in today’s interdependent and rapidly evolving environment. In this endeavor the report also discusses topics such as open government and open data, the civil service in the 21st century, metrics of government transformation, benchmarking public value from the perspective of citizens and cybersecurity for open government, giving examples of how governments around the world are using ICTs, including social media, to transform themselves and engage constituents. The report also notes that new metrics are needed in order to expedite the transformation of government and suggests a holistic framework to measure the various aspects of FAST (readiness) and its “public value” (outcome).



The updated 2012 Waseda University International e-Government Ranking from Waseda University surveys the development of government websites worldwide (Waseda 2012). Now in its eight consecutive year, the survey maintains the main indicators and their weights to maintain backward compatibility but has increased the number of countries it covers to 55. It uses 30 sub-indicators across seven categories: network preparedness, required interface functioning applications, management optimization, national portal, CIO in government, e-government promotion, and e-participation. Beyond the ranking, in 2012, the survey identified the following trends in e-government: cloud computing, social media, big data, business continuity plan / disaster recovery plan, digital inclusion, cyber security, mobile government and ICT applications for ageing society. Some international e-government benchmarks appear to be losing steam. Previously highlighted reports from Brown University / Brookings Institute and the Economist Intelligence Unit (EIU) have not been updated since 2008 and 2010, respectively. Accenture does not rank countries per se but publishes valuable reports, such as the recent Build It and They Will Come? The Accenture Digital Citizen Pulse Survey and the Future of Government Operations (Accenture 2012). Using an online survey of 1,400 respondents from Australia, France, Germany, India, Singapore, the United Kingdom and the United States, conducted in November 2011, its findings are counter intuitive from many other reports. For example, a majority of respondents said they would make greater use of digital services if they were offered by government. As such, it concludes governments are not giving citizens what they want. In addition, the survey also found that 51% believe interacting with governments digitally would make them more engaged while 49% said such interactions would make government more transparent.

2.1.1 SUMMARY OF TRENDS E-government is at a crossroads, to borrow a phrase from the 2003 UN World Public Sector Report (UN 2003). On the one hand, budget cuts and an imperative to do more with less may limit the full potential for investment. On the other hand, e-government is frequently seen as a solution to these problems and viewed as an investment in sustainable development. As usual when it comes to constantly evolving technology development, it is unfair to generalize trends; however, the reports mentioned above offer a few common themes, though are not limited to:

a) Evolving benchmarking: As was also the case in 2010, there is broad agreement that the world of measurement needs to evolve to meet new expectations. Almost all above cited reports are increasing the number of countries they cover while enhancing their indicators and developing new methods to capture trends. As opposed to two years ago, however, when most reports favoured the retention of stability for historical comparison purposes, some have now made greater changes. Of particular note are the efforts of the WEF, which abandoned a relatively stable framework going back to 2002, in search of a refined model that includes an increasing focus on the challenging measurement of outcomes. The



EC benchmarking survey, which also continues to evolve, is placing greater emphasis on piloting and broadening its assessments to the local level.

b) Usage and the digital divide: If people do not have access, they cannot use ICTs.

Renewed efforts to bridge the digital divide are garnering increased attention around the world, illustrated by recent efforts at the ITU, UNCTAD, the EC and the WEF, among others, and usage metrics are now frequent components of various reports and models. But the task of getting people online is only the first step, as questions surrounding ICT skills remain not only across the world but also within countries. Increasingly, what people do with their time online is gaining attention, termed by the EIU as “useful usage”. In this regard, lacklustre data on e-government usage leaves much room for improvement across the world, illustrated in part by the EU’s modest goal of having 50% of the population use e-government although 80% of people there are already online.

c) Sustainable development, outcomes and impact: Many reports, including those

from the ITU, UNCTAD, EC, and the WEF, are increasingly placing a greater emphasis on outcomes and the impact of initiatives, both of which point to a link between e-government and sustainable socio-economic development. While most reports seek to capture this trend implicitly or explicitly, it also presents a great challenge as the task to measure inputs (such as budgets and inter-linkages on the back-end) to outputs (such as return on investment and socio-economic progress) is complex. For governments too, this is an exciting opportunity but also a challenging one in practice as it increasingly turns the focus on micro-targeting people, ranging from reaching vulnerable groups in bridging the digital divide and opening up specific data to capturing the results of such policies.

d) The next phase: New efforts to close the digital divide and improve usage

coupled with open data initiatives and an increased reliance on ICTs – by both governments and users – is likely to enhance efforts in sustainable development; however, it can also put socio-economic progress at risk given that an increase in dependence leads to an increase in vulnerability (Andreasson 2011). Few of the reports surveyed deal with cyber security, despite an increase in cyber crime and cyber threats. Exceptions are found, however, in the EC Digital Agenda and the report from the WEF, which extensively discusses this topic, particularly as it relates to open government (EC Digital Agenda website and WEF 2011). It is likely we will see more over the next couple of years, either quantitatively but also tangentially, as e-government continues to improve.

2.2 UN SURVEY DEVELOPMENTS The two most recent UN reports on e-government capture recent trends well. In Leveraging E-government at a Time of Financial and Economic Crisis, a case was made for improving public trust through greater transparency based on open standards and the sharing of government information (UN 2010). The report also turned the focus on users,



in a variety of ways, including citizen empowerment, inclusiveness and participation. Reflective of the readiness to maturity trend, the report also changed the name of its main model, from “e-readiness” to “development.” Specifically, the report explains, “[t]he term ‘e-government development’ describes how far governments have actually advanced in this field instead of how ready or able they might be to do so, which was how ‘e-government readiness’ described national capacity” (UN 2010, page 3). In E-Government for the People, a case was made for the linkage between e-government policies and sustainable development (UN 2012). In this regard, e-government must not only support improved effectives and inclusiveness to meet higher expectations of what government should do, but also use ICTs to promote greater inclusiveness and enhance participation for all, including growing demand for greater transparency and accountability, in order to support people-centered development. In support of such efforts, the report looks at e-government trends across four areas: increasing usage of e-government, delivery of services through multiple channels, whole-of-government approaches, and bridging the digital divide, especially among vulnerable groups. The report notes, “[t]he overall conclusion that emerges from the 2012 Survey in today’s recessionary world climate is that while it is important to continue with service delivery, governments must increasingly begin to rethink in terms of e-government – and e-governance – placing greater emphasis on institutional linkages between and among the tiered government structures in a bid to create synergy for inclusive sustainable development” (UN 2012, page 9).

2.2.1 UN SURVEY CHALLENGES AND OPPORTUNITIES Compared to other international information society benchmarks, the UN survey is unique in the number of countries it covers and in its people-oriented web measurement approach. This presents challenges as well as opportunities. Given its strengths and the need to maintain comparability, there are inherent limitations, such as staying with the online measurement approach (see also section 5.2). Opportunities abound, however, in adapting to current trends, such as usage, impact, outcomes, and sustainable development from a supplemental approach, or finding proxy indicators by which to measure them.

3. REVIEW OF OVERALL METHODOLOGY This section reviews the overall UN survey methodology. While the basic model has not changed since its inception in 2003, each existing index is discussed in brief whereby suggestions for refinement are provided.5

5 For the complete methodology, please refer to pages 119-125 of the most recent report (UN 2012). Note that any suggestions to add indicator(s) to any composite index warrant data for 193 UN Member States in order to reflect the purpose of the survey.



3.1 E-GOVERNMENT DEVELOPMENT INDEX The UN e-government development index (EGDI), also commonly referred to as the “UN survey,” is a composite index based on the weighted average of three normalized indices where 1/3 is derived from a telecommunications infrastructure index (see section 3.2 below), 1/3 from a human capital index (3.3), and 1/3 from the online service index (3.4). The combined EGDI score gives an indication of e-government development across the 193 UN Member States relative to each other as opposed to being an absolute measure with a finite goal.

3.1.1 Preliminary recommendation(s) It is tempting to follow in the footsteps of the WEF and add a “usage index” as a fourth component to EGDI; however, as the UN must be able to gather data on all 193 Member States and given the complexity of finding reliable statistics even for the most developed countries in this area, such addition appears unfeasible at this point. Otherwise, there are seemingly no strong reasons to change the basic composition of the EGDI and leaving it intact allows for better comparability.

3.2 TELECOMMUNICATION INFRASTRUCTURE INDEX Data for the telecommunication infrastructure index is provided by the International Telecommunication Union (ITU), the UN specialized agency for ICTs. Five normalized indicators form a composite index based on the arithmetic mean of each where the telecommunication infrastructure index =

number of personal computers per 100 persons; number of Internet users per 100 persons; number of telephone lines per 100 persons; number of mobile cellular subscriptions per 100 persons; and, number of fixed broadband subscribers per 100 persons.

3.2.1 Preliminary recommendation(s) As was also suggested in the 2010 overview paper, the frequency of updates needs greater transparency and some indicators should be enhanced to reflect development. Preliminary recommendations include removing the measure of telephone lines, an increasingly outdated indicator, and adding mobile broadband subscriptions (perhaps in addition to regular mobile subscriptions) and potentially adding a measure of broadband affordability. It is highly recommended that the ITU be consulted on the selection and upgrading of indicators within this index as they are providing the data.



3.3 HUMAN CAPITAL INDEX Data for the human capital index is primarily provided by the United Nations Educational, Scientific and Cultural Organization (UNESCO), with gaps being filled through either the United Nations Development Programme (UNDP) Human Development Report, most recently the 2010 version or through proxy indicators from other sources, such as official figures from the United Nations Children’s Fund (UNICEF) or World Bank data. The index is comprised of two normalized indicators: adult literacy rate and the combined primary, secondary, and tertiary gross enrolment ratio in education. Adult literacy is assigned a weight of 2/3 and gross enrolment 1/3.

3.3.1 Preliminary recommendation(s) The index already captures the most common indicators of educational achievement but there are questions around the appropriate weight associated with each and whether additional indicators should be added. One potential addition would be school life expectancy, which is also measured by UNESCO. It is highly recommended that UNESCO, UNDP, UNICEF and the World Bank are consulted on the selection and upgrading of indicators within this index as they are providing the data.

3.4 ONLINE SERVICE INDEX Data for the online service index (OSI) comes from a survey questionnaire, the answers to which are compiled across all 193 UN Member States and collected by a group of researchers under the supervision of UNDESA, meaning that this is original research through a primary collection endeavour (for more details, see section 5). Although the weight of the OSI is equal to the others, there is greater opportunity to improve quickly in this area and affect the overall EGDI ranking as well because improvements in human capital and infrastructure take longer.

3.4.1 Preliminary recommendation(s) Questions surrounding the weight of the OSI relative to the other indices within EGDI abound on occasion, although it appears unclear what benefit it would have to increase the share of OSI to 40% or 50% or more. Interested stakeholders can merely review the sub-scores of OSI if interested in how they fare there – and – in fact, can create their own weightings from the three published sub-indices. As such, it seems the current approach should remain unchanged.



4. INDICATOR DEVELOPMENT AND SIX E-GOVERNMENT INITIATIVES FOR 2014 This section introduces the OSI methodology, presents the framework for discussion at the EGM, and introduces six e-government initiatives for which new indicators are sought as they are likely to be covered in the 2014 report.6

4.1 INTRODUCTION TO THE OSI METHODOLOGY The OSI is determined through a survey questionnaire consisting of about 160 questions, or individual indicators. Researchers use this questionnaire to assess the primary websites of each UN Member State, starting with the national portal, or its equivalent, as well as associated portals and ministries at the national level.7 This means that the survey uses a supply side approach – assessing the supply of information or services on websites – as its primary form of data collection. In order to capture e-government development, the survey questionnaire is updated prior to each biennial survey, thanks largely to the participants at the EGM and efforts by internal staff who review the recommendations from the meeting and add their own suggestions. In 2010, 25 questions were added; 29 modified and 16 removed (UN 2010). It is important to know that all questions in the survey questionnaire are binary as to whether a particular feature is present on a website (yes, or 1) or unavailable (no, or 0).8 Because there are no qualitative questions, quality or design of services are only assessed indirectly. For example, a question may ask, “Can you pay taxes online?” and the answer may be “Yes” and the next question may ask, “Can taxes also be paid via mobile phones?” In this way, the survey remains binary and quantitative but can add questions to probe the depth of services offered, in this example for multi-channel delivery.

4.2 AT THE EGM: A SEARCH FOR THEMES AND TRENDS The next six sections (4.3 through 4.8) introduces the concept and scope for each of the six initiatives in e-government, as pre-determined by the UN, in order to provide a background for the development of new indicators for the 2014 survey.

6 These initiatives are considered specifically for the 2014 survey, although there is overlap with those identified in preparation for the 2012 survey. Interested EGM participants should refer to the 2010 overview paper for a background review on those initiatives. 7 There are minor exceptions but for purposes of the EGM, this is unimportant. 8 Again, there are minor exceptions but for purposes of the EGM, this is unimportant.



While keeping the general OSI methodology (described above) in mind, at the EGM we will seek to identify themes or trends within each initiative as opposed to creating binary questions. It is simply better to cover emerging topics and sub-topics within the initiatives than in carefully crafting the specific question, something which can be done at a later stage. For example, it is more important for purposes of capturing trends in digital divide for the EGM to identify the emerging trend of having screen readers read the content on websites to allow vulnerable populations, such as blind people, be able to pay taxes online. Later, we can take the emerging trend of screen readers and craft a question, such as “Is the tax website compliant with screen reader technology, allowing the text to be read to blind people?” This effort should include online trends as well as offline trends, meaning one trend in e-government could be the government printing the URL of its portal on every piece of hard-copy paper it hands out at local offices (to encourage use of its online services). It is acknowledged that the existing survey questionnaire may contain questions relevant to the six initiatives and that one trend or indicator can be reflective of more than one initiative; however, this would unnecessarily limit the discussion; any remaining overlap can be addressed after the meeting. What follows is a brief summary for each of the six initiatives for 2014, each of which begins with an overview of the topic followed by recent trends and website examples.

4.3 WHOLE-OF-GOVERNMENT Improved back-office operations and the related topic of whole-of-government (WOG) are not new trends, of course, but continue to receive strong attention as governments around the world aim to transform into modern organizational structures that puts users first. This requires flexibility and greater horizontal and vertical connectedness between various government departments with the hope it leads to cost savings and greater effectiveness over time, both for constituents and government itself. One challenge concerns the appropriate approach by which to develop successful WOG as there is no one-size-fits-all solution to date, although some are being proposed based on the numerous examples available (Ojo et al 2011). Singapore, for example, is currently focusing its WOG effort on an enterprise architecture program as the foundation for future initiatives.9 Because of its potential, WOG is also increasingly part of ICT strategies around the world, including one of three main focus areas in e-government leader Singapore’s new strategic plan (IDA 2011). A key point of the back-end work is to create a seamless front-end experience for users. People should not need to know the agency, department or level of government providing the services but rather just be able to find it and use it in an effective manner. 9 http://www.egov.gov.sg/egov-programmes/programmes-by-government/whole-of-government-enterprise-architecture-wog-ea



Because most of the work takes place on the back-end, however, this obviously complicates matters from a supply-side measurement perspective, which therefore require the use of proxy indicators, i.e. website features that are indicative of back-end integration and transformation. At its most basic level this could mean lists and links to local government websites or other agencies, such as is the case in Australia, or result in a combined central portal, such as in the UK that recently integrated its Directgov and Business Link portals into one.10 More advanced versions – and obvious examples of WOG – come with specially designed portals that offer joint services through a single sign-on. For example, the Swedish business registration portal where three government agencies – the Swedish Companies Registration Office, the Swedish Tax Agency and the Swedish Agency for Economic and Regional Growth, have joined forces to enable entrepreneurs to logon and conduct services with all three agencies in one place.11 In Denmark, the citizen portal offers a personalized account of information and services through a single sign-on.12 Meanwhile, some, such as New Zealand, are moving ahead with an “all-of-government” approach that includes cloud computing, sometimes labelled the g-cloud (for government cloud).13 Such clouds can directly benefit, and build upon, WOG initiatives, and may be the latest trend in this area, evident also in Singapore.14

4.4 E-PARTICIPATION The e-participation index has been part of the OSI since its inception in 2003; however, during the first few editions of the survey, the index was mostly qualitative. In recent editions, the index has been purely quantitative, drawn from specific questions in the OSI questionnaire that relate to participatory activities. The e-participation index is composed of three stages, 1) e-information, assessing the extent of information that relates to engagement and the promotion thereof, 2) e-consultation, assessing the extent of interaction between primarily government and constituents, and, 3) e-decision-making, assessing the level of participation in actual policy-making. The e-participation index is unique because there are no other international benchmarks in this area. As a result, the index has garnered strong interest and is, or has been, included in other benchmarks of the information society, such as those from the WEF and the EIU.

10 See http://australia.gov.au/services/service-task/contact/contact-my-local-council and https://www.gov.uk/ respectively. 11 http://www.verksamt.se/portal/en_GB/web/international/home 12 https://www.borger.dk 13 http://ict.govt.nz/ 14 http://www.egov.gov.sg/egov-programmes/programmes-by-government/cloud-computing-for-government



The e-participation trend has been uneven, according to the 2012 Survey, as world leaders are forging ahead with new portals and initiatives while a majority of countries have little to offer still. The Republic of Korea and the Netherlands are leading the way in 2012 with the former showing impressive gains in usage as well. Public awareness of the Korean e-participation portal went from 26.8% in 2006 to 58.9% in 2011; during the same five-year time-span, the number of online petitions went from 400,000 to 1,070,000.15 Development might be uneven but it is encouragingly taking place across the world. Kazakhstan has been cited as a leader in e-participation, in no small part due to its excellent network of blogs by government executives.16 Saudi Arabia, meanwhile, has developed numerous surveys, public consultations and further engagement initiatives using social media, such as Facebook and Twitter.17 Such efforts are all captured by the UN’s e-participation index, which has led to criticism as allowing countries to “window-dress” their websites to score highly and neglecting the link between e-government and “real government,” according to one article (Grönlund 2011). But there seems to be a disconnect here because the index does not attempt to measure “democracy” or “transparency” or some of the other indicators with which the index is compared; it merely seek to measure how Member States are using online tools for interaction rather than prescribing a specific form of participation. Specifically, the index “assesses the quality and usefulness of information and services provided by a country for the purpose of engaging its citizens in public policy making through the use of e-government programs,” according to the 2010 report (UN 2010). Even so, there are lessons to be learned. Whether window-dressing or not, social media engagement and participation has proliferated in recent years. Today, Korea is indicating that an emphasis on increasing e-participation usage may be next on the list.

4.5 MULTI-CHANNEL SERVICE DELIVERY If government services are offered online, most people would use them (Accenture 2012). However, far from everyone prefer digital channels. As such, multi-channel service provision, the delivery of government services through a choice of channels, online and offline, is important in giving constituents what they want and in the process hopefully improve social inclusion. Research in China also shows that digital and traditional channels supplement each other, particularly among vulnerable populations in rural areas (Fu and Xiao 2012). One study also show that the phone remains an effective channel for solving problems whereas websites are more effective for getting information (Reddick and Porter 2012). Therefore,

15 The portal is www.epeople.go.kr; I am indebted to Dr Yeonwoo Lee at NISA, Republic of Korea, for providing the numbers in his presentation at the November, 2012, workshop in Bahrain. 16 http://www.blogs.e.gov.kz/?lang=en 17 http://www.saudi.gov.sa/wps/portal/shareYourOpinion?language=en



governments must provide multiple channels for constituents, allowing them a choice of conducting their business online, over the counter, over the phone, at a kiosk, or via mobile phone. It is, however, in the government’s interest to move from over the counter to online services as it improves efficiency and also offers enormous potential for cost savings. According to a 2009 PwC report, the UK government would save between GBP 3.30 and GBP 12 per transaction by moving transactions online. As with so many other trends (digital divide, usage), mobile services and their consumption are increasingly important in this area, particularly in developing countries. According to the ITU, in 2011, 90% of the world population has access to 2G mobile networks and 45% to 3G networks, making it an attractive option for communication especially because of its general affordability. From simple SMS-based services in poor rural areas to smart-phones and web-based access in urban centers and developed countries, mobility is increasingly a desired and frequently used channels of communication across the world. From a measurement perspective, this can begin by identifying WAP websites, such as the one in Canada.18 Further, offline channels can be measured online by identifying whether websites provide information on how to conduct services via other channels. Highlighted two years ago, the Australian portal remains a best practice in this area as the homepage itself offers the “Government Shopfront Locator” which allows users to identify local government services by postal code, area, or type of service.19 The search feature comes with addresses, open hours as well as an advanced map to find them. As if that was not enough, the “Contact Government” feature on the homepage also offers the following services: telephone hotlines, contact your local member, departments & agencies and government online directories. But just as features offline should be able to be found online, the reverse is true as well. In Sweden, almost all government agencies are now in the habit of printing their website URL on their forms, giving constituents another channel through which to submit them. While mobility is clearly dominating the topic at the moment, trends in multi-channel services delivery include digital-by-default in the UK, meaning that services are primarily being delivered digitally, and the extension of online channels, such as conducting services on websites external to the government, including from the private sector such as Facebook.

4.6 EXPANDING USAGE

18 http://www.wap.gc.ca/Link.action?id=1&l=e 19 http://australia.gov.au/



Despite strong growth in the availability of information and services (supply-side), e-government usage (demand-side) remains tepid, with the possible exceptions of online tax filing and business services, both of which generally achieve higher numbers of users. One reason is the digital divide. In the United States, for example, broadband Internet adoption, as well as computer use, varies across demographic and geographic groups. Lower income families, people with less education, those with disabilities, Blacks, Hispanics, and rural residents generally lagged the national average in both broadband adoption and computer use (NTIA 2011). But while lack of usage overlaps to some extent with a lack of access and other digital divide issues, it doesn’t account for the entire gap. Other recent theories include:

Low usage indicates that current e-services do not match citizen demand (Accenture 2012); There is a lack of awareness or IT skills (EIU 2012; OECD 2011); and, Lack of competition and affordability of ICTs (ITU 2011; ITU 2012).

Despite the challenges in improving e-government usage, few governments appear to promote their services offline or help users online: in 2012, only about a quarter – or 47 countries – provided information on how citizens use services on their websites, a proxy indicator for the governments’ interest in usage. Yet citizens generally embrace online services when they become aware of e-government (Australian Government Information Management Office 2011); however, to measure such efforts of user take-up is difficult using a front-end website analysis. At best, one can look for extended government statistics or annual reports on the topic, such as in Australia.20 Other trends to improve usage include ability to find information and website design. Again, Australia offers a good example where the government portal now offers navigation by a) services, b) people, c) topics, and d) life events, enabling a choice in finding the appropriate task in an intuitive manner.21 Stimulating usage is also a design issue (The Moscow Times 2011). In this area, Sweden’s government page has long excelled in allowing users to change website settings and providing a simplified language version, a video of sign language, and having the website read its text out loud, among other features.22 But most measures of this initiative are likely to be served by proxy indicators, i.e. they will be indicative of government’s interest in usage and training rather than necessarily 20 http://www.finance.gov.au/publications/interacting-with-government-2011/index.html 21 http://australia.gov.au/services http://australia.gov.au/people http://australia.gov.au/topics http://australia.gov.au/life-events 22 Change of setting: http://regeringen.se/sb/d/6910 Sign language video: http://regeringen.se/sb/d/8962 Simplified language version: http://regeringen.se/sb/d/2506



providing a measure of take-up per se. Recent trends in this area include training programs, social media promotional efforts, and offline promotions. In Bahrain, for example, a magazine is published solely to promote e-government.23

4.7 DIGITAL DIVIDE AND VULNERABLE GROUPS There is little disagreement regarding the existence of digital divides around the world, both between countries but also within them. If individuals do not have access to ICTs, they cannot use it; but even after they gain access, they must have the skills to use it appropriately. In 2011, 85% of Europeans had used the Internet but only 75% used it regularly and among disadvantaged groups only 60% had used it, according to Eurostat data, the European statistics office.24 Ironically, it is perhaps because strong gains in access have been mad, that it overshadows the need to make even greater efforts. In 2002, 39% of households in the EU 15 had Internet access; in 2010, the equivalent figure was 73%. Yet, it is the remaining population that is also the hardest to reach. The digital divide is a broad topic that overlaps with e-inclusion and multi-channel service provision, which includes mobile technology. Challenge(s) also vary depending on the geographical remit. In India, for example, lack of access is such an issue that it has prompted access of any kind as a valuable effort to bridge the digital divide, including local kiosks and m-government initiatives (Financial Times 2011). Others, such as Australia, view the digital divide primarily as a broadband access challenge; in Europe, the focus is on enhancing “e-skills” as one of the seven priority areas in the Digital Agenda (Digital Agenda website); in America, some claim it is an issue of speed (New York Times 2011); in Russia – in part – the unveiling of a more user-friendly national portal (The Moscow Times 2011). As technologies evolve, so too does the digital divide challenges and the current divide is also likely to become a next-generation divide. In the U.S., for example, broadband adoption data show that significant gaps remain between groups according to socio-economic factors. But today, two intertwined and unmistakable trends are bringing new imperatives for closing the digital divide: the socio-economic benefits with bringing people online and the promise of mobility, or m-government. Countries around the world are realizing the benefits of bringing more people online. In the UK, for example, studies show the economic benefits of increasing online access, hence a government digital champion was appointed and services there are now “digital by default,” meaning this is the primary channel through which they are delivered (PwC 2009).

23 http://www.emagazine.gov.bh/ 24 http://epp.eurostat.ec.europa.eu/portal/page/portal/information_society/data/main_tables



Through interviews with experts and policy-makers, the EIU identified six smart policies to close the digital divide around the world, including mobility but also more recent trends such as what people do with their access – termed useful usage – as well as local content and consumption (EIU 2012) The digital divide is closely tied to usage, an area difficult to measure from the front-end, although governments increasingly highlight their efforts in this area, items indicative of their interest in closing the divide, one example of which is the UK Cabinet Office blog.25 Similarly, publishing a digital divide strategy would be one way of showing interest. Other ways indicative of closing the divide would be through mobile channels, such as having an m-government portal or a separate m-government strategy, such as is the case in Singapore.26

4.8 OPEN GOVERNMENT DATA A 2007 gathering of experts in Sebastopol, California, defined open government data “as public data which is complete, primary, timely, accessible, machine processable, non-discriminatory, non-proprietary, and license-free, and when compliance is reviewable.”27 From a policy perspective, open government data is believed to bring enhanced transparency and improved accountability while strengthening participation and – along the way – allow users to create, or even co-create, value from public sector information (PSI). In fact, recent data from the EC suggests that open government data policies can increase European business activity by up to EUR 40 billion annually (EC 2011). Because of its socio-economic benefits, open government initiatives and transparency projects are increasingly common across the world and at all levels of government.28 In fact, early examples from the US and the UK were studied for their socio-economic contribution and its applicability to the developing world (Hogge 2010).29 In 2011, Kenya became the first country in sub Saharan Africa to follow in their footsteps as President Mwai Kibaki announced the Kenya Open Government Data Portal.30 From a supply-side measurement perspective, open data initiatives are a particularly welcome trend as the entire purpose of it is to make data publicly available and visible. The greater question surrounds what to measure. For example, Tim Berners-Lee, the so-called inventor of the world wide web, has proposed a five star scheme to encourage governments to publish their data, awarding points for availability, structure, non-proprietary, identification for re-use and linking to other data to provide context.

25 http://digital.cabinetoffice.gov.uk/2011/12/02/tackling-the-digital-divide/ 26 http://www.ida.gov.sg/Collaboration-and-Initiatives/Initiatives/Store/M-Government.aspx 27 http://www.opengovdata.org 28 See, for example, the US: http://www.whitehouse.gov/open and Australia: http://www.finance.gov.au/e-government/strategy-and-governance/gov2/declaration-of-open-government.html 29 US: data.gov and the UK: data.gov.uk 30 https://opendata.go.ke



Beyond open data portals and websites, it is also possible to measure – from the front-end – to what extent governments are interested in such initiatives. For example, are the legal foundations for open information in place and are they listed on government website(s)? Further, with an increasing number of global initiatives, one can also see whether a country has committed to projects such as the Open Government Partnership.31 Although generally acknowledged to be a positive endeavor, open government data also come with limitations (Bannister and Connolly 2011). The WEF, for example, has noted that the balance between open government and potential security ramifications is a key question for public officials, one which is likely to get more scrutiny as these initiatives continue to evolve (WEF 2011).

5. SURVEY IMPLEMENTATION PROCESS This section provides an overview of the most recent approach to survey implementation, both in preparation as well as execution, before discussing current limitations and potential alternatives.32

5.1 CURRENT APPROACH A team of researchers and translators is recruited after UNDESA finalizes the survey questionnaire, based on input from EGM participants and internal staff. During the 2011 assessment (which resulted in the 2012 report), this was done in collaboration with Cornell University, which provided a group of graduate students that worked under UNDESA supervision. Together, they used the survey questionnaire to assess the national portal(s) and associated websites for all Member States. A second review team, recruited primarily from within the UN system, conducted a thorough second review of all websites, followed by additional quality control, such as comparing indicators and trends with results from prior years and conducting additional analysis on outliers.

5.1.1 Preparation All Member States are invited to provide their relevant URLs to UNDESA, which will take them into account; where no response is received, a list of URLs from the previous survey is used to identify websites to be surveyed. In addition, researchers verify all websites and also conduct an extensive additional search to uncover new, or previously unknown, websites.

31 http://www.opengovpartnership.org 32 This section is an interpretation of the web measure methodology as described on pages 120-124 of the most recent report (UN 2012) but also draws on personal experience.



Meanwhile, survey management identifies the linguistic requirements – the aim is to conduct the assessment of every website in the primary language of that website – and map those to the language capabilities of researchers; translators, most of whom are recruited specifically for a particular country, are brought in as necessary to sit with researchers. Researchers are provided with both training sessions and guidelines in the form of specific instructions on how to score each individual question in the survey, as well as actual website examples. To manage the process from start to end, an online system is set up, most recently on a SharePoint platform, which enables researchers to enter their scores into a central database that can be exported into an Excel format for scoring purposes.

5.1.2 Execution Researchers use the survey questionnaire to evaluate the national portal, or other official homepage, as well as associated portals, ministry websites, and other online presence to assign a binary value to each question based on the absence or presence of the specific item measured. The research team follows a citizen-centric approach in their evaluation of the survey questionnaire, essentially trying to mimic average citizen user behavior. This is currently fundamental to the survey because it means that answers are generally based on whether the relevant feature(s) can be found as opposed to whether they exist. As the numbers of national websites grow and their sophistication improves, this becomes increasingly important. While it is possible, although implausible, to spend hours browsing a government website meticulously for all available content and features, this approach misses the point that for information and services to be usable they must be readily accessible by the intended beneficiaries. No citizen would, for example, spend a full day looking for a service that could be accomplished quicker over the phone, or in person. The current approach of the survey methodology, therefore, is in essence a measurement of e-government effectiveness, as opposed to strict supply. In theory, a feature may exist, but unless it is easily accessible, it won’t be scored. It seems researchers tend to spend more time on a given website to look for features than the average citizen would. The actual time spent by someone will vary widely depending on, amongst others, level of development and design (or “user-friendliness”). As a rule of thumb, however, a researcher will typically review one or two countries per day. Since it is currently not feasible to capture real time changes for 193 Member States, a fair way of assessment is to establish firm start and end dates. As such, survey execution is conducted over a specific period of time (the “survey window”), in order to capture development at a particular moment in time.



5.2 LIMITATIONS AND BOUNDARIES The UN survey faces limitations, some self-imposed, and awareness of them may improve our collective efforts to discuss them at the EGM. They include:

5.2.1 Overall approach The UN focus on sustainable development is fundamental. Similarly, the global supply-side analysis, which entails quantitatively assessing websites through content analysis, will also continue. Taking both these factors into account, therefore, the focus of the survey “should remain citizen services and participation” from a web feature analysis perspective with the aim to stay within “’reasonable’ cost and maximum comparability” while, at the same time, “this should not compromise the need for new thinking and better measurement…” (Millard 2008, page 14 and repeated in Andreasson 2010). This is a self-imposed limitation, which also means that any suggestion for a radical new framework or approach is unlikely to be considered.

5.2.2 Measuring the back-end from the front-end There is an increasing focus on the back-end, in particular as it relates to whole-of-government; however, because of the supply-side (i.e. front-end) assessment approach taken here, it inherently limits the amount of back-end measurement that can be accomplished and the survey needs to focus on developing proxy indicators.

5.2.3 Supply (availability) is trumping demand (usage) With the strong growth in availability of services, focus is increasingly shifting to demand, i.e. whether, and how, constituents and businesses use e-government. But because the survey measures the supply of information and services, little is known about demand (or usage) of those same features, unless government websites declare them publicly or share their data, which is unfeasible across 193 Member States.

5.2.4 A lack of transparency The overall UN survey methodology is published as an appendix to the report but neither the researcher guidelines nor the survey questionnaire are publicly available, limiting the potential for external feedback.



5.3 ALTERNATIVES AND SUPPLEMENTAL OPTIONS There is great potential for the UN survey to evolve and capture new trends and meet greater expectations through alternatives and supplemental options. They include:

5.3.1 Automation and innovation Manually conducting the survey is taking longer as websites become increasingly complex. At the same time, improvements have been made to automated tools. Therefore, the survey questionnaire should be evaluated to identify what parts can be conducted automatically (note: The present survey questionnaire contains only one automated test, namely that of the W3C's Web Content Accessibility Guidelines Priority 1).

5.3.2 From websites to search engines Search engines are currently only used to find information and services when they cannot be located on national websites directly. This practice could be reversed: most constituents would first use search and if the feature wasn’t listed on the first, say, 2-3 pages then they would take only a cursory look at a national website before giving up. Mimicking such “true” citizen approach would save time and effort.

5.3.3 Interviews and desk research Conducting interviews with in-country stakeholders and other experts would add a new dimension to the final report as it would get qualitative insight based on the quantitative survey.

5.3.4 Link measurement to development objectives Similarly, desk research could be conducted into online – and offline – efforts by government in areas such as joint e-government promotion by several departments (indicative of whole-of-government). In addition, greater classification details about countries (e-government budget, return on investment, etc) could be added in order to link performance to development objectives to the extent possible.

5.3.5 Mobile assessment It is difficult to understate the importance of mobile devices – in many developed countries they offer an alternative channel of communication, particularly tablets, while in developing countries they offer first time access to government digital services. The



survey captures m-government initiatives from a website supply-side perspective but why not also conduct part(s) of the survey via a mobile device?

5.3.6 Demand that the suppliers supply the data While there are several potential backup options, such as the use of proxy indicators, qualitative assessments, or online indications on usage, there is no substitution for the real deal: the survey should ask that each Member State provides usage statistics and reward those who do by giving them an extra point in the survey or otherwise recognize them; the failure to provide such data is in itself indicative of a lack of interest.

5.3.7 Increase transparency Publishing the actual survey questionnaire may lead countries to implement only the features assessed, a common concern in providing greater transparency. But this may not be an either/or dilemma. For example, if the 2012 survey questionnaire was publicly available, it would allow stakeholders to make suggestions for enhancement while it wouldn’t serve them to implement only those features listed because the 2014 survey questionnaire is under development and – based on experience – may change quite a bit. If the concern is that stakeholders will complain about particular features not being scored appropriately, then individual scores to individual questions can still be withheld; however, it is time to at least publish the questions.

6. TOWARDS THE FUTURE The UN e-government survey program must continuously take emerging trends into consideration in order to stay relevant and meet new demands and ever-higher expectations, one reason for which we are all at the EGM. In part, this is due to evolving technological innovation and in part due to the realization of the benefits of e-government and the commitment of governments everywhere to make it happen. But just as technology enables progress it also challenges benchmarking efforts as measurement becomes increasingly complex, such as the task on estimating outcomes.

6.1 GOALS OF THE EGM Based on the outcome at the EGM, the consultant will prepare a final report to the UN, including proposed revisions to methodology, proposed indicators (additional, revise, or delete) for the six initiatives for the 2014 survey, proposed revisions to survey implementation process, if any, and any other comments or suggestions.



APPENDIX I: UN SOURCES The primary UN sources in regards to the e-government survey and program are as follows: E-Government Survey Questionnaire 2012, as supplied by the UN. United Nations. E-Government Development Knowledge Base (UNKB). http://www.unpan.org/egovkb/ United Nations. 2001. Benchmarking E-government: A Global Perspective. Department of Economic and Social Affairs, New York. http://unpan1.un.org/intradoc/groups/public/documents/un/unpan021547.pdf United Nations. 2003a. Global E-Government Survey: E-government at the Crossroads. Department of Economic and Social Affairs, New York. http://www.unpan.org/egovkb/global_reports/08report.htm United Nations. 2003b. World Public Sector Report 2003: E-Government at the Crossroads. Department of Economic and Social Affairs, New York. http://unpan1.un.org/intradoc/groups/public/documents/un/unpan012733.pdf United Nations. 2004. Global E-Government Readiness Report: Towards Access for Opportunity. Department of Economic and Social Affairs, New York. http://www.unpan.org/egovkb/global_reports/08report.htm United Nations. 2005. Global E-Government Readiness Report: From E-Government to E-Inclusion. Department of Economic and Social Affairs, New York. http://www.unpan.org/egovkb/global_reports/08report.htm United Nations. 2008. Global E-Government Survey: From E-Government to Connected Governance. Department of Economic and Social Affairs, New York. http://www.unpan.org/egovkb/global_reports/08report.htm United Nations. 2010. Global E-Government Survey: Leveraging E-government at a Time of Financial and Economic Crisis. Department of Economic and Social Affairs, New York. http://www.unpan.org/egovkb/global_reports/08report.htm United Nations. 2012. E-Government for the People. Department of Economic and Social Affairs, New York. http://www.unpan.org/egovkb/global_reports/08report.htm



APPENDIX II: REFERENCES Accenture. 2012. Build It and They Will Come? The Accenture Digital Citizen Pulse Survey and the Future of Government Operations. http://www.accenture.com/us-en/Pages/insight-digital-citizen-pulse-survey-summary.aspx Andreasson, Kim. 2010. Overview paper in support of Expert Group Meeting (EGM): E-government survey: Towards citizen-centric service delivery through e-government approaches. http://unpan1.un.org/intradoc/groups/public/documents/un/unpan047965.pdf Andreasson, Kim, ed. 2011. Cybersecurity: Public Sector Threats and Responses. http://www.crcpress.com/product/isbn/9781439846636 Australian Government Information Management Office (AGIMO). 2011. Interacting with Government: Australians’ use and satisfaction with e-government services. http://www.finance.gov.au/publications/interacting-with-government-2011/index.html Bannister, Frank and Connolly, Regina. 2011. The Trouble with Transparency: A Critical Review of Openness in e-Government," Policy & Internet: Vol. 3: Iss. 1, Article 8. http://www.psocommons.org/policyandinternet/vol3/iss1/art8/ EC. 2009. Ministerial Declaration on eGovernment. http://ec.europa.eu/information_society/activities/egovernment/events/past/malmo_2009/press/ministerial-declaration-on-egovernment.pdf EC. 2010. The European eGovernment Action Plan 2011-2015: Harnessing ICT to promote smart, sustainable & innovative Government. http://ec.europa.eu/information_society/activities/egovernment/action_plan_2011_2015/docs/action_plan_en_act_part1_v2.pdf EC. 2010b. The 9th Benchmark Measurement, Digitizing Public Services in Europe: Putting ambition into action. http://ec.europa.eu/information_society/newsroom/cf/item-detail-dae.cfm?item_id=6537 EC. 2011. “Proposal for Amending Directive 2003/98/EC on re-use of public sector information” COM (2011) 877 http://ec.europa.eu/information_society/policy/psi/docs/pdfs/directive_proposal/2012/en.pdf EC. eGovernment website. http://ec.europa.eu/information_society/activities/egovernment/index_en.htm EC. Digital Agenda website. http://ec.europa.eu/information_society/digital-agenda/index_en.htm



Economist Intelligence Unit. 2012. Smart policies to close the digital divide: Best practices from around the world. http://www.managementthinking.eiu.com/digital-divide.html Financial Times. 2011. Digital inclusion is weak in India, worst in sub-Saharan Africa. http://tilt.ft.com/#!posts/2011-03/16811/digital-inclusion-is-weak-india-worst-sub-saharan-africa Fu, Yaqin and Ke Xiao. 2012. Software Engineering and Service Science (ICSESS), 2012 IEEE 3rd International Conference. Promoting sustainable e-government with multichannel service delivery: A case study. http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6269561&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D6269561 Grönlund, Åke. 2011. Connecting eGovernment to Real Government - The Failure of the UN eParticipation Index. http://rd.springer.com/chapter/10.1007/978-3-642-22878-0_3 Hogge, Becky. Open Society Foundation. 2010. Open Data Study. http://www.opensocietyfoundations.org/reports/open-data-study IDA. Singapore eGov2015 Masterplan. 2011. http://www.egov.gov.sg/egov-masterplans/egov-2015/vision-strategic-thrusts International Telecommunications Union. 2011. Measuring the Information Society 2011. http://www.itu.int/ITU-D/ict/publications/idi/index.html International Telecommunications Union. 2012. Measuring the Information Society 2012. http://www.itu.int/ITU-D/ict/publications/idi/index.html Millard, Jeremy. 2008. E-Readiness Survey – Getting to the Next Level: Overview paper in support of Expert Group Meeting (EGM). http://unpan1.un.org/intradoc/groups/public/documents/un/unpan035535~2.pdf National Telecommunications and Information Administration (NTIA). 2011. Exploring the Digital Nation - Computer and Internet Use at Home. http://www.ntia.doc.gov/report/2011/exploring-digital-nation-computer-and-internet-use-home New York Times. 2011. The New Digital Divide. http://www.nytimes.com/2011/12/04/opinion/sunday/internet-access-and-the-new-divide.html?pagewanted=all OECD. 2011. Government at a Glance 2011. http://www.oecd.org/gov/governmentataglance2011.htm



OECD/ITU. 2012. M-Government: Mobile Technologies for Responsive Governments and Connected Societies. Ojo, Adegboyega, Tomasz Janowski, and Elsa Estevez. 2011. Information Polity Volume 16, Number 3 / 2011. Whole-of-government approach to information technology strategy management: Building a sustainable collaborative technology environment in government. http://iospress.metapress.com/content/j428317437g51585/ PwC. 2009. Champion for Digital Inclusion: The Economic Case for Digital Inclusion. http://www.parliamentandinternet.org.uk/uploads/Final_report.pdf Reddick, Christopher G. and Michael Turner. 2012. Government Information Quarterly Volume 29, Issue 1, January 2012, Pages 1–11. Channel choice and public service delivery in Canada: Comparing e-government to traditional service delivery. http://www.sciencedirect.com/science/article/pii/S0740624X11001080 The Australian. 2010. Broadband's digital divide. http://www.theaustralian.com.au/news/features/broadbands-digital-divide/story-e6frg6z6-1225904628003 The Moscow Times. 2011. New E-Government Portal Starts. http://www.themoscowtimes.com/business/article/new-e-government-portal-starts/436563.html The Partnership on Measuring ICT for Development. Website. http://www.itu.int/ITU-D/ict/partnership/ or http://measuring-ict.unctad.org/ The Partnership on Measuring ICT for Development. 2011. Framework for a Set of Egovernment Core Indicators. http://www.itu.int/ITU-D/ict/partnership/material/Framework_for_a_set_of_E-Government_Core_Indicators_Final_rev1.pdf UNCTAD. 2011. Information Economy Report 2011. http://unctad.org/ier2011/ Waseda University Institute of e-Government. 2012. 2012 Waseda University International e-Government Ranking. http://www.waseda.jp/eng/news11/120224_egov.html WEF. 2010. The Global Information Technology Report 2009-2010. http://www.weforum.org/reports/global-information-technology-report-2009-2010 WEF. 2011. The Future of Government: Lessons Learned from around the World. http://www.weforum.org/reports/future-government



WEF. 2012. The Global Information Technology Report 2012. http://www.weforum.org/reports/global-information-technology-report-2012

Documents

2014 UN E-GOVERNMENT SURVEY: E-GOVERNMENT AS AN …unpan1.un.org/intradoc/groups/public/documents/un-dpadm/unpan050717.pdf · 2014 UN E-GOVERNMENT SURVEY: E-GOVERNMENT AS AN ENABLER