Towards e-Government project assessment: European approach i istrazivanja/Arhiva Zbornika radova/mates... · that is called “ITPOSMO”. This panel consists of “Information”

Pavel Mates et al. • Towards e-Government project assessment: European approach Zb. rad. Ekon. fak. Rij. • 2013 • vol. 31 • sv. 1 • 103-125 103

Preliminary communicationUDC 351.712.025:342.51.35:004.77

Towards e-Government project assessment: European approach*1

Pavel Mates2, Tomáš Lechner3, Pavel Rieger4, Jitka Pěkná5

Abstract

The objective of the paper is to analyse current approaches to the assessment of e-Government projects as the specifi c public projects and to suggest how to improve these approaches in order to eliminate their shortcomings. The non-normative theoretical methods are used to analyse empirical results of previous researches; particularly the deduction method is used to prove that current approaches to the evaluation of e-Government projects have some inequalities, the analogy and comparison methods are used to create general typology of e-Government projects and the induction method is used to seek examples of the indicators and metrics. The results are based on analysis of extensive amount of e-Government projects, which have been realized in different European countries in the past twenty years. The basic result of the research is creation of the ten most common types of e-Government projects typology. The fundamental conclusion

* Received: 19-12-2012; accepted: 14-06-20131 This research has been supported by the R&D Project IG508012 “Public Administration and

ICT: Changes in formal and informal relationships” funded by the University of Economics, Prague.

2 Associate professor, University of Finance and Administration Prague, Faculty of Social Studies, Estonská 500, 101 00 Praha 10, Czech Republic. University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Praha 3, Czech Republic. Scientifi c affi liation: administrative law, protection of personal data, public administration. Phone: +420 210 088 831. Fax: +420 224 095 547. E-mail: [email protected].

3 PhD candidate, University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4,130 67 Praha 3, Czech Republic. Scientifi c affi liation: e-Government, applied informatics, public administration. Phone: +420 224 095 620. Fax: +420 224 095 547. E-mail: [email protected].

4 PhD candidate, University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Praha 3, Czech Republic. Scientifi c affi liation: e-Government, applied informatics, public administration. Phone: +420 224 095 620. Fax: +420 224 095 547. E-mail: [email protected] (corresponding author).

5 PhD candidate, University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Praha 3, Czech Republic. Scientifi c affi liation: economic aspects of public administration, ways of communication in public administration, regional development. Phone: +420 224 095 620. Fax: +420 224 095 547. E-mail: [email protected].

Pavel Mates et al. • Towards e-Government project assessment: European approach 104 Zb. rad. Ekon. fak. Rij. • 2013 • vol. 31 • sv. 1 • 103-125

obtained from the results of this research is that this typology can be used as the core of new E-Government General Assessment Framework, which eliminates problems of previous approaches, allows adjusting metrics and indicators to each type of projects, keeps comparability of results and thus making possible the use of benchmarking methods.

Key words: assessment of public projects, management of public projects, e-Government, evaluation of public services

JEL classifi cation: D73, E02, G38, H43

1. Introduction

E-Government as an application of information and communication technologies within public administration has been an integral part of the transformation process since 1990s, although the real concept of e-Government as such began to assert itself just at the beginning of the millennium. E-Government is a very up-to-date issue particularly because of the current public budgets crisis not only in the European countries and the expectation that e-Government could save public expenditures and increase effectiveness of public administration. The process of information and communication technologies implementation within public administration has a lot of aspects: economic, legal, social etc.

According to the statistical data presented by various authors, e.g. (Goldfi nch, 2007), most of the e-Government projects fail. On the basis of the analysis of 40 e-Government projects realized in developing and transitional countries Heeks (2003) states that 35 % of e-Government projects are total failures and 50 % of the projects are partial failures. According to (Heeks and Stanforth, 2007) various authors estimated the proportion of e-Government projects failing (total or partial) in the fi rst decade of the 21st century between 60 and 85 %. Apart from relatively numerous applications of the method of critical success factors (Kamal, 2006), (Altameem et al., 2006), the concept of critical failure factors (Kumar and Best, 2006) was also used in the sphere of e-Government. Among the critical issues of failures according to the mentioned article, there is also counted suffi cient evaluation and monitoring of the e-Government projects. Many authors, such as (Foley and Alfonso, 2009), (Kumar and Best, 2006), (Undheim, 2008), (West, 2005) and others mentioned above, agreed that lack of impact and result evaluation of e-Government projects is reality. If there is any evaluation, it is only ex-ante, however ex-post evaluation is missing totally. Thus the paper focuses on ex-post evaluation, even if both of them are sometimes discussed.

The working hypothesis is as follows: Current inequalities and problems of approaches to the evaluation of e-Government projects, which are realized within the European public administration environment, can be solved by using a new


e-Government general assessment framework consisting of a small amount of partial boards specially designed for each type of e-Government project.

The goal of the paper is to prove this hypothesis, to develop e-Government projects categorization and to show examples of possible metrics or indicators for individual fundamental types of e-Government projects taking into account the developed categorization. We are not creating a completely new evaluation framework for e-Government, but our goal is to show how to proceed to the evaluation of e-Government projects in the way that removes some shortcomings of the previous approaches.

2. Literature review

Evaluation of public administration projects is always a problematic issue stemming from the principles of existence of public administration as such. There are different general approaches to the evaluation of quality and effi ciency of the performance of public administration, such as Searching the Best Practices, PDCA Cycle (Durant and Wilson, 1993), Benchmarking, EFQM Excellence Model (Torres, 2004), CAF, Balanced Scorecard (Bernhard and Hoffschröer, 2003), ISO standards – especially ISO 9001 (Saner, 2002) and Knowledge Management. Most of these methods, primarily coming from the private sector, are implemented in the public sector withvarious intensity (Brignall and Modell, 2000). Differences that have to be taken into consideration are seen especially in the fact that the public organizations operate on the basis of politically defi ned needs, in the public sector it is not possible to use the profi t motive for measuring the performance and fi nally the difference is also in legal environment, the most outstanding factors for public administration management are equality and justice, in contrary to the private sector where there are in the fi rst place or let us say ultimately, profi t and development of the enterprise.

Although mentioned general approaches can be also used for evaluation of concrete e-Government projects, there are other several different sophisticated evaluation frameworks. These approaches can be divided into two categories:

• General comprehensive evaluation of the projects or degree of the development of e-Government, for example in a certain country.

• Concrete evaluation of the selected project drawn up in a specifi c way for a certain project.

The approach of evaluation of e-Government applied by OECD in the form of Proposed Outline for Assessing e-Government Benefi ts (OECD, 2006) belongs to the general comprehensive approaches. The content consists of general methodical


framework that divides types of impacts into direct fi nancial costs and benefi ts (reducing the costs, decreasing administrative burdens), direct non-fi nancial costs and benefi ts (gaining total investment benefi ts, increasing the user´s satisfaction) and indirect costs and benefi ts (strengthening the legitimacy, supporting the growth).

For evaluation of e-Government projects, Heeks (2006) uses the evaluation panel that is called “ITPOSMO”. This panel consists of “Information” that is inserted within the e-Government project, processed and used out of “Technologies” that have to be implemented within the project, out of “Processes”, then out of “Objectives and Values”, out of personal ensuring and necessary qualities in terms of “Staffi ng and Skills”, out of “Management” and out of time, fi nancial and other resources.

Measuring the Expected Benefi ts of e-Government (Offi ce of Government Commerce, 2003) based on the methods of Business case and applied in the Great Britain also belongs to the general comprehensive evaluation panels. In France the method MAREVA (Méthode d´Analyse et de REmontée de la VAleur) is used for the justifi cation of the planned projects, which was created by French ADAE (Agence pour le Développement de l’Administration Electronique) between 2004 and 2007 (Vossey, 2009). There are some other state-level developed approaches: German Wibe, Australian DAM & VAM and Slovenian approach (Jukic et al., 2012). The similar approach to creating national evaluation frameworks is applied also in other European countries.

On the ground of the Department of Information Technology, Government of India (Rama Rao et al., 2004) there was created a quite detailed evaluation model of EAF (E-Governance Assessment Framework) that considers as important metrics all characteristics that affect in some way functioning of public administration and implementation, let us say the very functioning of e-Government. Within the evaluation according to EAF there are allocated different amount of points among the individual criteria and the sum of them can be 100 points at the most (the higher the value of the metrics approaches the target, the more points for evaluation) and at the same time there is set a certain scale of importance for relevant concrete metrics. According to the sum of weighted points it is then possible to evaluate the relevant project.

The advantage of these approaches is their generality, which would enable in their larger extensions for example within the European Union the comparison among the individual projects and countries. However, this situation has never happened until nowadays. And therefore there still prevails disadvantage of these panels, which are not able to cover some important specifi cs of the individual projects due to the total generalization. Completely from the other side there are the specifi c evaluations that are intended either for the individual projects (and due to it they


are defi nitely incommensurable) or they highlight and evaluate only some aspect of e-Government projects, such as ergonomics of e-Government systems (Heeks, 2006) or e-Participation (Macintosh and Whyte, 2008).

Both general comprehensive evaluation of e-Government projects and concrete evaluation drawn up in a specifi c way for a certain project do not offer suffi cient assessment framework for the specifi c issue of e-Government project evaluation.

3. Method

Because the research goal of this paper is of combined functional and objective kind, an appropriate methodology should be chosen. Non-normative theoretical methods as a part of explanatory methods fi t well with research of e-Government assessment framework. As to prove the hypothesis, we have mainly applied analysis of extensive amount of sources of e-Government projects, which have been realized in different European countries in the past twenty years. Thus, we use theoretical scientifi c methods to analyse empirical results of previous researches.

By means of deduction method, we clearly show that both general comprehensive evaluation of e-Government projects and concrete evaluation drawn up in a specifi c way for a certain project do not offer suffi cient assessment framework for the specifi c issue of e-Government project evaluation. To eliminate these problems we suggest division of the evaluated projects in certain categories. Their number must be chosen in such a way that we would not fall to undue details of evaluation that are typical for a monoproject approach, but on the other side in such a way that we would be able to cover signifi cant specifi cs of the individual projects. To achieve this objective we also use other explanatory theoretical scientifi c methods, which are analogy and comparison.

In the fi rst part of the analysis, we use mentioned methods (analogy and comparison) to compare typical attributes of e-Government projects, which are realized within the European public administration environment, such as degree of the discretionary power derived from legal framework, extent of included relations (G2C, G2B, G2G), approach to management, technical solution, scope of functioning (local, regional, national, international), model of fi nancing etc. Projects with analogical properties we put into the same group of projects so that we create general typology of e-Government projects.

In the second part of the analysis, we analyse different kinds of metrics and indicators for evaluation and monitoring. The necessity of evaluation and monitoring arises from the following theses that are often quoted in the sphere of management, e.g. (Deming, 2000):


• If management is able to measure the course of fulfi lment of partial targets, it can also make the corrections of future steps based on the measured values.

• What is not possible to measure, it is not possible to manage either.

For evaluating various spheres of projects management and their support through the means of ICT it is possible to use the metrics (indicators), which are exactly defi ned indicators or evaluation criteria (Parmenter, 2007). The objective of their usage is to compare the measured values from the period before the signifi cant changes in information system with the values measured after implementation of new information system or after implementing some new functions (Xenos, 2006). If the values of the metrics are compared in regular time intervals, it is possible on the basis of the changes in the measured values to draw partial conclusions on functioning of the important processes and on the potential benefi ts of new ICT.

The task is what to measure and how. The theory of management recommends measuring especially such indicators whose value has signifi cant impact on some important activity of the organization and is also easily infl uenced by means of ICT (Earl, 1993). According to the developed categorization we suggest appropriate metrics and indicators, while hard metrics (their value is exactly measurable) are in this analysis preferable to soft metrics or indicators (their value cannot be exactly measured and can be determined only by qualifi ed estimation). Typical examples of the metrics in the private sector are development of profi t, turnover, market share or productivity of the production factors (Lockemann et al., 1983). In the industrial enterprises there are measured also other indicators (Fenton and Pfl eeger, 1997) – consumption of raw materials, level of provisions, share of manufactured scraps or indicators of the level of computer-controlled logistics, such as occupancy of vehicles, mileage distance or total transportation costs (Lejmi and Butterwegge, 2002).

The values of indicators can be infl uenced by a high number of factors out of which only some factors have their origin in implementation of new information system. That is why it is necessary to select such indicators that are related to the evaluated e-Government project and there is also possible to suppose in their case a causal relationship between the implementation of ICT and the gradual changes of the values of the indicator, similarly (Lockemann et al., 1983) or (Čadil and Beránek, 2011). At the proposed metrics we should also determine if the objective is to maximize or minimize the value. If it is not possible to determine it clearly, if it is desirable to maximize or minimize the value, the metrics will serve only for informative purposes.

Through the analysis of e-Government projects we can conclude that these projects are targeted on various partial objectives such as for example increase in openness of public administration, speeding of offi cial procedures or acceleration


of communication. During implementation of the individual e-Government projects some metrics should be assigned ideally to each declared objective, and these metrics would measure the level of reaching these objectives (Basili et al., 1994). The proposed metrics will be divided by us into several groups and we will follow the classifi cation of the metrics by Rieger and Toth (2011).

While seeking examples of indicators and metrics, we use induction method based on the same primary extensive analysis of e-Government projects, which have been realized in different European countries in the past twenty years. Of course we use comparison and analogy as well, but we also have to generalize relevant information regarding individual projects within the whole group as part of e-Government projects typology.

We propose particularly such metrics that are refl ective of functioning of the given e-Government service and simultaneously can be measured or counted up on the basis of data in a concrete database operated by public power authority. Such measured values of the metrics can be operated with zero or very low costs on their measuring (Xenos, 2006). Their other advantage is the possibility of fi nding out the actual values at any time. In such a situation, the public administration management should react faster to the actual development of the evaluated e-Government project and new trends in fulfi lling its objectives. If the evaluated project does not lead to the stated objectives, the public administration management could make a decision that the project should be modifi ed or (in case of major failures) should be completely stopped.

4. Analysis

4.1. Categorization of e-Government projects

First of all, it should be said that the mentioned projects are only random examples. While creating typologies of e-government the list of all implemented projects within European countries is not mentioned, because it is not necessary for this kind of research. The fi rst projects realized within the e-Government dealt with especially two areas: development of communication infrastructure and related support of development of information society, see for example European project IRISI (Dabinett, 2001) or German project MEDIA@Komm (Siegfried, 2007), and creation of static Web pages. Just after that there comes the development of electronic communication that should be, and in some cases it must be, supported by the legal basis at least by the existence of the Act on Electronic Signature. This communication is done either through an e-mail (i.e. common electronic mail usually with use of the institutes of electronic signature), SMS messages or specialized portal solutions.


Within the total development of e-Government it is possible to identify several types of projects. Historically, the fi rst ones are static Web pages; this category will be marked as P1. Information presented through these pages is oriented primarily on private legal subjects. Easier accessibility of information not only on the institutions of public administration, but also on its activities contributes to the development of e-Democracy, although it also depends on the quality of provided information. There exists quite a high degree of variability of the approaches that can be determined on the scale from openness and pluralism to isolation and presentation of opinions of only one side – see the study (Henriksson et al., 2007), (Isacker et al., 2010), (Osimo, 2008) and (Panopoulou et al., 2009).

The centralized portals of public administration at the national levels or at the level of central bodies of state administration can have a specifi c position. There are integrated solutions, which do not contain only static information, but also offer other electronic services. Within our approach to type classifi cation they can be divided into partial subprojects that fi t into our other gradually identifi ed and described categories. Information parts (parts providing passive information) vary from the point of categorization from the previous category P1 only in technical solution and scope of operation, which are typically broader and more complex. By this fact there is refl ected the reality that e-Government copies the structures of public administration (OECD, 2003) and thus the projects of information Web pages copy the scopes of operation and models applied for proper organizations of public administration that realize them.

Within the efforts to some personalization of the approaches it is possible to identify the services of actively distributed and thus requested (or ordered) information through a certain way by individual public administration users. We will denote them as P2. From the technical point of view it can be, for example, information on new published data whose distribution is conducted through set services of RSS channels (Continente, 2010), or there are information systems providing important actual information in time of emergency incidents that are realized through large-scale sending of SMS messages. In France, within this category, there was a successful project carried out by distributing information to parents on possible absence of their child from school (Greffet, 2007), and in Turkey there is an SMS legal information system that provides a notifi cation service for the citizens and lawyers within individual court proceedings (Çam, 2007).When we move further in personalization, we will get into the area of the booking services (P3), which already mean certain mutual interaction between public administration and private legal subjects and serve for the support of the common performance of public administration by allowing the arrangement of the meeting for execution of a certain concrete matter in the offi ce in the concrete time. These e-Government projects affect unambiguously positively the effi ciency of the performance of public administration, because they allow to make better use of the time of the offi cials


and to eliminate the queues through spreading the administrative acts at individual time intervals.

Receiving the electronic fi lings (P4) is the next type of projects. For these projects to be realized there would primarily have to be amended the relevant procedural acts in all countries. It is also true that the discretionary power is hardly ever applicable in this sphere and the regulation is mandatory, that is as it concerns the parties of the proceedings who thus cannot change the set way of communication. When communicating with public administration, it is substantial to identify the relevant subject. From the technical point of view, the fi lings are thus realized either through purely centralized solutions preceded by the registration process, for example in the sphere of the tax administration (Greffet, 2007), (Siegfried, 2007), (West, 2005), or through the technologies administered at the regional or local levels, for example during fi ling to the local government authorities. For identifi cation of the subject, specifi c authentication elements are generally used, especially in the form of qualifi ed certifi cates issued by the accredited providers of the certifi cation services. In such cases, the communication itself is typically under way through e-mails, while at the sophisticated projects, structured data are sent in the attachment, for example, outputs in the web forms. In this sphere, there are also cases of hybrid communication, at which the citizen fi rst makes fi ling through the electronic way without using the electronic signature and then, the citizen confi rms it during a short time period in the paper form, so that this fi ling would be considered as valid. Also this way can be more advantageous than pure paper fi ling, because in the electronic form it is possible to detect easier the possible errors of fi lings.

Since the conventional communication tools (such as e-mail) have all range of disadvantages, for example a non-guaranteed delivery and a necessity to use the tools for the unambiguous identifi cation of the sender, some states build more diffi cult communication tools. In the Czech Republic it is the information system of data boxes (Mates et al., 2010), in Austria it is the system of “e-Zustellung” (Ščerba, 2009), in Germany it is the system of DE-Mail (Reimer, 2012) and at the European level there is prepared a project SPOCS (Project SPOCS, 2012). These systems enable a reliable delivery with similar characteristics comparable to classic postal services, i.e. with the guarantee of delivery by the third independent person and with the preservation of privacy of correspondence, which is part of the documents of the fundamental rights and freedom or direct part of the individual constitutions of the European states. The projects of these types are denoted by us as P5.

Within orientation on the citizens as customers and users of public administration there are efforts to establish a “front offi ce” of public administration, i.e. especially in the form of so-called contact points of public administration (we will denote them as P6). These ones should provide the most comprehensive services to private legal subjects. The technical implementation is possible just by using e-Government


tools. Owing to the fact that these contact points have primarily the form of real (not virtual) offi ces, they are part of assistant positions, and through their realization there does not arise the problem of digital divide. In this way it is also possible to arrange e-Government services to the citizens who do not have access to the Internet or who do not dispose of suffi cient skills or knowledge to use it. E-Government serves in this case especially in the role of subsidiary infrastructure, which enables a relevant position of a “front offi ce” offi cial to offer broader and more complex access to public administration services than it would be without the possibility of e-Government. In the Czech Republic, the project of the contact points of public administration with the assistant position is called “Czech POINT” (Špaček, 2009) and in Hungary it is the connection of network “eCounsellors” with “eHungary points” (Fejer and Posfai, 2008).

The implementation of the basic registers of public administration (P7) belongs then to purely centralized projects. Legally, these projects come primarily from different partial information systems of public administration, nevertheless their signifi cant positions enforce specifi c legal regulation. Although it deals with the projects that function primarily in public administration, these projects improve the quality of the performance of public administration in terms of simplifi cation of procedural acts, because both a natural person and a legal person do not have to prove those data, which are provided from the basic registers. Nowadays, within Europe, these projects are realized in Austria, in the Czech Republic, in Belgium, in the Netherlands, in Sweden, in Finland, in Norway, in Denmark and in Estonia.

Other identifi ed category of e-Government projects is online providing of personalized information from the information systems of the individual bodies and institutions of public administration (P8). Technically, it deals with the projects realized through the portal solutions that are either directly connected to the information systems of the individual institutions of public administration, or use the principle of data warehouses. In France such system is realized within the “retirements accounts” of the individual citizens – project ADELE (Greffet, 2007), in the Czech Republic it is the project providing online overview on the course of the proceedings at cadastral offi ces (Rydval et al., 2005). Next it deals with the Norwegian project “MyPage” (Undheim, 2008) or German project of the solution of the electronic identifi cation of the persons for providing e-Government services (Breitenstrom and Fromm, 2009). The electronic assignment of procurements, at which there are essential the same attributes of identifi cation of the subjects, can be subjoined to this type of projects. Example of this project from France is reported by Cervelló (2009).

The projects of electronic elections (P9) are even by a small degree higher in necessity of precise electronic identifi cation. Here, the necessity of identifi cation of the individual persons is very crucial, but connected with the need of anonymity towards the result of voting. Thus it is to ensure it technically so that only the


authorized persons could vote in the relevant electoral district and potentially in the electoral curia and only once, but on the other side so that it would not be possible to fi nd out how the person had voted. From the technical and legislative points of view, these projects are the most diffi cult ones within e-Government. Examples from Switzerland are listed in the contributions (Cervelló, 2009) and (Chevalier et al., 2007).

The last identifi ed type of e-Government projects is electronic legislation and electronic collection of laws (P10). It is an application of information and communication both within the preparation of legal regulations and within the area of publication of legal regulations. These projects have signifi cant infl uence on the overall development of e-Democracy. Already 17 states of the European Union have introduced binding electronic promulgation of legal regulations and fi ve of them even defi nitely stopped issuing the paper collections. As described in the article (Çam, 2008), these projects have positive effects not only on the elements of e-Democracy, but they also bring considerable economic benefi ts.

Table 1: Typology of e-Government projects

Name of the type Label Main characteristics

Web presentations P1 Static items of information presented online, they have an impact on e-Democracy, their technical solution and scope of functioning copy the structures of public administration

Distribution of requested information

P2 Projects of a large-scale or individual distribution of notifi cations to the signifi cant events of personal or local signifi cance

Online booking systems P3 Projects improving effi ciency of the classic performance of public administration

Electronic registries P4 Projects of e-mail communication or easy web forms

Reliable delivery systems P5 Specialized centralized tools for reliable delivery, which mean electronic equivalent to the postal services with all appurtenances

Contact points of public administration (“front offi ces”)

P6 Orientation on the help of the citizens in using public administration services and solving various life situations that are enabled through the internal use of e-Government without raising the degree of digital divide

Basic registers of public administration

P7 Centralized projects of certifi ed source registers with important data, typically the registers of legal persons, of natural persons, of territorial identifi cation, of real estate etc.

Online providing of personalized information from information systems of the individual bodies of public administration and personalized communication

P8 Technically demanding solutions of integrated projects of online providing of specifi c and most often very strictly personalized information from the information systems of individual bodies and institutions of public administration.


Name of the type Label Main characteristicsElectronic elections P9 Electronic voting, which also very signifi cantly supports

direct democracy Electronic legislation and electronic collection of laws

P10 Improving the legislative process through information and communication technologies and electronic publishing of the legal regulations

Further undifferentiated e-Government projects

P11 Other specifi c e-Government projects, which do not belong to any type mentioned above

Source: Authors

In total, we have now ten types of projects. Certainly, there exists another group of projects that cannot be assigned to these ten main types, because of their considerable particularity, and that is why to the summary in Table 1, we add the eleventh further undifferentiated category (P11). However, we believe that the frequency of projects that we would be forced to classify as the type P11 is very low.

4.2. Indicators and metrics

As already mentioned in Chapter 2, evaluation of public administration projects is a problematic issue arising from the very principles of existence of public administration. This problematic nature leads in many cases to the fact that evaluation of some projects is ignored completely. Let us remember that Kumar and Best (2006) consider critical failure factors in e-Government projects because of their insuffi cient evaluation and monitoring.

Apart from the differences between evaluation of the projects of public and private sectors, it is necessary to take into account within the approach to the evaluation of e-Government projects, the difference between evaluation of the face-to-face services and e-services. The necessity of differentness of the approach to evaluation of such variously provided services is mentioned in Udo et al. (2008).

A frequently declared objective of e-Government is an effort to accelerate the processes in public administration. For this reason, the evaluation indicators must include time metrics, which measure the time of duration of the particular process in a number of days, for example the time required for updating data in a concrete offi cial register (registration of a new owner into the land register, change of an address of a legal person in the commercial register, change of a surname of an individual in the register of the inhabitants, etc.). At the time metrics the objective will consist in minimizing the value.

Volume metrics (for example the number of users of certain service, the number of issued statements from a concrete offi cial register, the number of documents sent electronically) – they serve for information on usage of the service and usually their


objective will not consist in their maximization. Only when the unit costs decrease with the growth of volume metrics, the objective would lie in maximization of their values. It may apply for example to the number of electronically sent documents or to the number of users of certain service (typically for the electronic forms). In these cases the unit costs are lower than in paper documents, but it is necessary to ensure suffi cient capacity of communication to come off to cover high investment costs.

Failure metrics (unavailability of information system after x % of the time fund or x % of the working time, share of unanswered database queries) – the objective of these metrics consists in error states evaluation. The measured values should serve especially to persons responsible for system integration and sometimes to persons from the top management (especially in case of very frequent errors or frequent failure of information system that can exclude temporarily, especially in information system for reliable electronic delivery, all public administration from its functioning, if it happens to fail during the working time). The objective will be to minimize the values.

Quality metrics (for example, the security of information system or conformity of information system with the technical standards) – the objective will be seen usually in the value “yes” (information system meets the required parameters). In many cases such metrics are not directly measurable and can be measured only through certifi cation or other detailed exploration, so it will not deal with the metrics in real sense, but with the indicator.

Andersen and Henriksen (2006) point out that also the number of the offi cial acts provided while waiting (real one stop shopping for citizens) increases with the increasing rate of integration of information systems. Thus also the criterion, whether the relevant service allows to perform some offi cial act while waiting, can belong to the quality metrics.

Cost metrics (e.g. total expenditures on delivering in electronic and paper form, average costs on processing of one form, costs on providing an e-Government service). The objective is to minimize the metrics, but mainly at determining the total costs on use of ICT, the costs do not have to be easily attributable to the concrete e-Government project (Web presentations, electronic delivery systems, contact points of public administration). Purchased hardware, software, costs on the Internet connection, on energy, on offi ce material etc. are not in fact used only by one concrete e-Government project, but by all of them concurrently. Some of the mentioned costs (purchase of PC, Internet connection) would exist anyway regardless of existence or absence of the concrete e-Government service.

Some meaningful metrics are diffi cult to include into the above mentioned categories, thus we introduce the category “other metrics”. Into this category there can be


counted for example demandingness on the measure of computer literacy (the objective is minimization, because the number of persons who are able to use the service decreases with the increase in requirements) or the share of users who use the service voluntarily without being forced by legal regulation (the objective would lie in maximization, the voluntary use means the success of the service or small attractiveness of providing the service through non-electronic way). Regarding the fact that some e-Government projects pursue specifi c goals, the specifi c metrics can be assigned to these goals, and these specifi c metrics will be also included by us into the category of other metrics.

5. Results and discussion

Creation of the evaluation framework for e-Government projects that enables mutual comparison and also affects the specifi cs of the individual categories of e-Government projects is proposed with the division according to the individual types of projects. Most of the projects can be evaluated through metrics from several groups (volume, cost, quality and time metrics). It is necessary to emphasize again that values of several proposed criteria cannot be exactly measured, so in real sense of the word there are not the metrics, but rather the qualitative indicators.

At P1, the cost metrics should be decisive – total costs on creation and maintaining the web presentation, alternatively unit costs per one visitor. Next indicators will already have slightly smaller importance, although for example from the number of users there can be derived the expected demands on hardware and software.

In case of P2 services, apart from the cost metrics, the speed of distributing the information (as soon as possible after the event) should be emphasised especially to measure whether the situation was improved or if the costs per individual got saved after establishing the service. The measurement can also be seen in the fact, how much this information is really usable.

At P3, especially the cost metrics will be decisive. As for the other metrics, there can be particularly interesting the satisfaction of the users (measured for example as the amount of time spent in the offi ce or serviceability of making the booking from home that can be also infl uenced by the technical background of the user himself or herself) and the share of voluntary users of the services (supposing that it is possible to make the booking in some other way than online).

At P4, the number of received and sent messages will belong to the main metrics. If one manages to prove a downward trend of the unit costs, the objective will be to maximize the number of the messages. Other important metrics are the cost ones. As the costs on receiving and sending the messages can vary (for example in delays in receiving an unrequested mail), it would be possible to measure separately the total or unit costs on receiving and sending one data message.


For P5, the volume and cost metrics have again their importance that is also caused by the fact that the unit costs will decrease probably with each new message. At this service, there will be also quite high demands on a long-term availability of the service (especially during the working hours) and on its safety. The attractiveness of the service can be measured here for example as the share of users who use the service, although they are not forced to it through legal regulation. We expect that the number of voluntary users will be also infl uenced by awareness of the users on safety of the service (Horst et al., 2007). Another important specifi c metrics could be the share of messages that the addressee met in the limit of x days after sending.

P6 can be evaluated especially according to the number of various services that are provided by concrete contact points, or let us say according to the number of various services provided while waiting. Even the cost metrics will have relatively large importance, but other groups of the metrics or indicators will generally already have little importance.

P7 should ensure public administration online access to actual data and from that there are derived the most important metrics. The metrics for measuring the errors are of high importance here, because in case of unavailability of the service during the working hours or in case of inaccuracies in data, it will not be possible to make necessary offi cial act easily and quickly. The time metrics will also belong to the important metrics.

Various indicators mentioned above (cost, volume ones, safety parameters, frequency of the errors) will play some role at P8, but we can also follow here one special volume metrics, namely the number of queries without using online access with the aim to minimize the value of this metrics.

At P9, the quality metrics (especially focused on safety) and metrics for measuring the errors will play much bigger role. Unavailability of the service can have quite signifi cant impacts here, because it could prohibit the performance of electoral right. The cost metrics will also play the role here similarly to other types of the projects. If one manages to prove decreasing trend in the unit costs, it would be the aim to maximize the share of voluntary users of the service, similarly to the systems of reliable delivering.

At P10, the time metrics should be the most important, while other groups of the metrics will already have slightly lower importance. This concerns the cost metrics and other metrics that describe primarily the extent of published legal regulations and the possibilities of searching in these legal regulations.


Table 2: Possible metrics or indicators for the various types of e-Government projects

Type of the project Indicator Aim

Web presentation (P1)

C: total costs, unit costs per user MINF: unavailability of the service MINQ: number of different groups of users to whose needs the service is directly adjusted (G2B/G2C/G2G, the blind, foreigners) INF or MAX

V: number of users INF

Distribution of requested information (P2)

C: total costs, unit costs per user MINF: unavailability of the service MINT: how long after the event an information message is sent MINV: number of users, number of services with possibility to send information messages, number of communication channels for sending the messages

INF

X: was the situation improved after implementation of the service YES

Online booking systems (P3)

C: total costs, unit costs per user MINF: unavailability of the service MINV: number of users, number of realized services MAXX: users´ satisfaction MAXX: share of voluntary users of the service INF or MAX

Electronic registries (P4)

C: total costs, unit costs on processing of received data message, unit costs at sent data messages MIN

F: unavailability of the service MINQ: does it meet the safety regulations YESV: number of received data messages, number of sent data messages INF or MAX

Reliable delivery systems (P5)

C: total costs, unit costs on the user, unit costs on the sent message, unit costs on the received message MIN

F: unavailability of the service, unavailability of the service during common working hours MIN

Q: does it meet the safety regulations YEST: when do the legal effects of delivering occur on the average MINV: number of active boxes, number of active users, number of transported messages INF

X: share of the messages read by the addressee in the limit of x days after sending, share of voluntary users of the service (who are not forced to it through the legal regulation)

MAX

Contact points of public administration (P6)

C: total costs, unit costs per user MINF: unavailability of the service MINQ: does it meet the safety regulations YEST: average time necessary for providing the concrete type of the statement MIN

V: number of contact points, number of various types of the statements from the offi cial register provided by concrete contact point (thereof provided while waiting), number of forms of the statements from the offi cial register (electronically, in paper form)

MAX

V: total number of statements issued by the contact points INF


Type of the project Indicator Aim

Basic registers of public administration (P7)

C: total costs, unit costs per user MINF: availability of the service during the common working hours, share of errors during the processing of the database queries, number of the evident errors in data of the register

MIN

Q: does it meet the safety regulations YESQ: number of services of public administration that thanks to faster data updates or easier access to data can be provided while waiting MAX

T: average time of processing of a database query, average number of the days necessary for the concrete change of data in the concrete database

MIN

V: number of database queries of a concrete subject towards the concrete database, number of changes of concrete data, number of users

INF

Online providing of personalized information from the information systems of the individual bodies of public administration and personalized communication (P8)

C: total costs, unit costs per user MIN

F: unavailability of the service MIN

Q: does it meet the safety regulations YES

V: number of users INF

V: number of queries in the former way MIN

Electronic elections (P9)

C: total costs, unit costs per user MIN

F: unavailability of the service MIN

Q: does it meet the safety regulations YES

T: time needed for counting the election results MINX: share of electronically submitted votes (= share of voluntary users of the service) INF or MAX

Electronic legislation and electronic collection of laws (P10)

C: total costs, unit cost per user MIN

F: unavailability of the service MINQ: does the service allow to search the text of the regulation to any given date YES

T: how long after the adoption is the regulation published here MINV: number of users, number of published regulations, number of published actualizations, number of groups of published regulations (supranational, national, regional, local)

INF or MAX

Note: groups of the metrics or indicators used in the table are C – cost, F – failure, Q – quality, T – time, V – volume, X – other.

Source: authors


The examples of the possible metrics or indicators for individual fundamental types of e-Government projects taking into account their attributes are summarized in Table 2. When selecting the metrics, it is necessary to follow the principle that at least one metric should be assigned to each important aim. Indicators that should belong to the important ones for the concrete project are highlighted in italic.

6. Conclusions

The stated working hypothesis of the research has been confi rmed since analysis of previous approaches to the evaluation of e-Government projects shows that these approaches are usually too general to be able to cover the specifi cs of each e-Government project or highly specialized to one project, which in turn does not allow comparison of results of different projects. The basic results of this research highlight the facts that e-Government projects realized within the European public administration environment, can be categorized into ten groups as grouped in the Table 1, and that this typology can serve as the base for creation of new e-Government general assessment framework consisting of a small amount of partial boards especially designed for each type of e-Government project. Due to it, there might be expressed that the main scientifi c contribution of our analysis has been made in the fi eld of administrative science including economics, law and management in the sense of management approaches applied in the public sector and evaluation of e-Government projects as an integral part of the transformation process of public administration with major economic impacts. The fact that the public administration projects are more diffi cult to evaluate than the private ones belongs to the most signifi cant restrictions of our results. Moreover, the mere fact that it is not entirely clear at some metrics whether their aim is minimization or maximization, it is more diffi cult to evaluate accurately even the level of successfulness of achieving the project objectives.

As the appropriate directions for the future research it can be emphasised both giving precision to concrete evaluation panels for each type of e-Government projects and their application by the way that we will measure the values of the proposed metrics and indicators on specifi c realized projects. Next, it would be possible to use the benchmarking of the similar type projects that are implemented in the various regions or states.

According to the current European Union objectives in the area of ICT including its implementation with public administration specifi ed in the form of new digital priorities, the results could be very helpful and signifi cant for achieving these goals. Evaluation of implemented e-Government projects is necessary and the results show the appropriate know-how.


References

Altameem, T., Zairi, M., Alshawi, S. (2006) “Critical Success Factors of E-Government: A Proposed Model for E-Government Implementation”, Innovations in Information Technology Conference (IIT), pp. 1–5. Dubai: IEEE.

Andersen, K. V., Henriksen, H. Z. (2006) “E-government maturity models: Extension of the Layne and Lee model”, Government Information Quarterly, Vol. 23, No. 2, pp. 236–248.

Basili, V., Caldiera, G., Rombach, H. D. (1994) “Goal Question Metric Approach”, In: Marciniak, J. ed. Encyclopedia of Software Engineering. New York: John Wiley & Sons, Inc.

Bernhard, M. G., Hoffschröer, S., eds. (2003) Report Balanced Scorecard – Strategien umsetzen, Prozesse steuern, Kennzahlensysteme entwickeln. 3rd edition. Düsseldorf: Symposion Publishing.

Breitenstrom, C., Fromm, J. (2009) “Case study: Interoperability in the EU services directive implementation”, European Journal of ePractice, Vol. 3, No. 6, pp. 15–24.

Brignall, S., Modell, S. (2000) “An institutional perspective on performance measurement and management in the ‘new public sector’”, Management Accounting Research, Vol. 11, No. 3, pp. 281–306.

Čadil, J., Beránek, M. (2011) “ICT and economic growth – short term causality analysis”, Research Journal of Economics, Business and ICT, Vol. 1, No. 1, pp. 22–26.

Çam, A. R. (2007) “SMS information system: Mobile access to justice”, European Journal of ePractice, Vol. 4, No. 10, pp. 35–41.

Çam, A. R. (2008) “EU principles in modernisation of justice and the Turkish IT project UYAP”, European Journal of ePractice, Vol. 2, No. 3, pp. 47–63.

Cervelló, G. (2009) “The e-Participation project of Neuchâtel”, European Journal of ePractice, Vol. 3, No. 7, pp. 73–78.

Chevalier, M., Warynski, M., Sandoz, A. (2007) “Success factors of Geneva’s e-voting system”, European Journal of ePractice, Vol. 1, No. 1, pp. 47–59.

Continente, M. (2010) Case study: Gencat 2.0 – redefi ning radically the eGovernment portal concept. Available from: <http://www.epractice.eu/cases/gencat20> [Accessed July 1, 2012].

Dabinett, G. (2001) “EU mainstreaming of the information society in regional development policy”, Regional Studies, Vol. 35, No. 2, pp. 168–173.

Deming, W. E. (2000) The New Economics for Industry, Government, Education. 2nd edition. Cambridge (Ma): MIT Press.

Durant, R. F., Wilson, L. A. (1993) “Public Management, TQM, and Quality Improvement: Toward a Contingency Strategy”, The American Review of Public Administration, Vol. 23, No. 3, pp. 215–245.


Earl, M. J. (1993) “Experiences in strategic information systems planning”, MIS Quarterly, Vol. 17, No. 1, pp. 1–24.

Fejer, A., Posfai, M. (2008) “The eHungary program 2.0 – building an army of ecounsellors to fi ght against digital illiteracy”, European Journal of ePractice, Vol. 2, No. 3, pp. 14–21.

Fenton, N., Pfl eeger, S. L. (1997) Software metrics: a rigorous and practical approach. 2nd edition. Boston (Ma): PWS Publishing Co.

Foley, P., Alfonso, X. (2009) “eGovernment and the transformation agenda”, Public Administration, Vol. 87, No. 2, pp. 371–396.

Goldfi nch, S. (2007) “Pessimism, Computer Failure, and Information Systems Development in the Public Sector”, Public Administration Review, Vol. 67, No. 5, pp. 917–929.

Greffet, F. (2007) “The digital republic: renewing the French state via e-Government in Nixon, P. G., Koutrakou”, V. N. Eds. E-Government in Europe: Re-booting the state. Oxon: Routledge.

Heeks, R. (2003) Most e-Government-for-Development Projects Fail. How Can Risks be Reduced? London: IDPM, 2003. Available from: <http://unpan1.un.org/intradoc/groups/public/documents/cafrad/unpan011226.pdf> [Accessed July 4, 2012]

Heeks, R. (2006) Implementing and Managing eGovernment: An International Text. London: SAGE Publications Ltd.

Heeks, R., Stanforth, C. (2007) “Understanding e-Government project trajectories from an actor-network perspective”, European Journal of Information Systems, Vol. 16, No. 2, pp. 165–177.

Henriksson, A., Yi, Y., Frost, B., Middleton, M. (2007) “Evaluation instrument for e-Government websites”, Electronic Government, an International Journal, Vol. 4, No. 2, pp. 204–226.

Horst, M., Kuttschreuter, M., Gutteling, J. M. (2007) “Perceived usefulness, personal experiences, risk perception and trust as determinants of adoption of e-government services in The Netherlands”, Computers in Human Behavior, Vol. 23, No. 4, pp. 1838–1852.

Isacker, K. V., Goranova-Valkova, M., Chalkia, E., Bekiaris, E., Votis, K., Tzovaras, D. (2010) “Accessible web design a burden? Lessons learned from the accessible project”, European Journal of ePractice, Vol. 4, No. 10, pp. 12–20.

Jukic, T., Bencina, J., Vintar, M. (2012) “Multi-Attribute Evaluation of E-Government Projects: Slovenian Approach”, International Journal of Information Communication Technologies and Human Development (IJICTHD), Vol. 4, No. 1, pp. 82–92.

Kamal, M. M. (2006) “IT innovation adoption in the government sector: identifying the critical success factors”, Journal of Enterprise Information Management, Vol. 19, No. 2, pp.192–222.


Kumar, R., Best, M. L. (2006) “Impact and Sustainability of E-Government Services in Developing Countries: Lessons Learned from Tamil Nadu, India”, The Information Society: An International Journal, Vol. 22, No. 1, pp. 1–12.

Lejmi, H., Butterwegge, G. (2002) “Güterlogistische Dispositionssysteme in Verbindung mit B2B-Marktplätzen – Konzept und praktisches Beispiel”, WIRTSCHAFTSINFORMATIK, Vol. 44, No. 6, pp. 535–544.

Lockemann, P. C., Schreiner, A., Trauboth, H., Klopprogge, M. (1983) Systemanalyse. DV-Einsatzplanung. Berlin: Springer.

Macintosh, A., Whyte, A. (2008) “Towards an Evaluation Framework for eParticipation”, Transforming Government: People, Process and Policy, Vol. 2, No. 1, pp. 16–30.

Mates, P., Lechner, T., Bohata, P. (2010) “Elektronische Briefkästen in Tschechien: Eine kleine Revolution in der Kommunikation”, WiRO – Wirtschaft und Recht in Osteuropa, Vol. 19, No. 4, pp. 103–106.

OECD (2003) 2nd OECD Symposium on E-Government. Available from: <http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=GOV/PUMA/EGOV/A(2003)2&docLanguage=En> [Accessed March 23, 2012]

OECD (2006) E-Government Project – Proposed Outline for Assessing E-Government Benefi ts. Available from: <http://webdomino1.oecd.org/COMNET/PUM/egovproweb.nsf/viewHtml/index/$FILE/GOV.PGC.EGOV.2006.1.doc> [Accessed June 10, 2012]

Offi ce of Government Commerce (2003) Measuring the Expected benefi ts of e-government. UK: Offi ce of Government Commerce. Available from: <http://www.esd.org.uk/esdtoolkit/Communities/EffectiveServiceDelivery/Guides/HMT%20Guidance%20and%20Tools.pdf> [Accessed July 30, 2012]

Osimo, D. (2008) “Benchmarking eGovernment in the WEB 2.0 era: What to measure, and how”, European Journal of ePractice, Vol. 2, No. 4, pp. 33–43.

Panopoulou, E., Tambouris, E., Zotou, M., Tarabanis, K. (2009) “Evaluating eParticipation Sophistication of Regional Authorities Websites: The Case of Greece and Spain”. In Macintosh, A., Tambouris, E. (eds). Electronic participation: fi rst international conference, ePart 2009. Linz: Springer.

Parmenter, D. (2007) Key Performance Indicators: Developing, Implementing, and Using Winning KPIs. Hoboken (NJ): John Wiley & Sons, Inc.

Project SPOCS (2012) Project SPOCS. Available from: <http://www.eu-spocs.eu> [Accessed March 2, 2012]

Rama Rao, T. P., Venkata Rao, V., Bhatnagar, S. C., Satyanarayana, J. (2004) E-Governance Assessment Frameworks (EAF Version 2.0). Available from: <http://www.egovindia.org/eaf.pdf> [Accessed June 24, 2012]

Reimer, H. (2012) “De-Mail-Diensteanbieter akkreditiert“, Datenschutz und Datensicherheit – DuD, Vol. 36, No. 4, pp. 285–286.


Rieger, P., Toth, P. (2011) “Assessing the Successfulness of e-Government on an Example of the Czech Republic”, Research Journal of Economics, Business and ICT, Vol. 1, No. 1, pp. 35–41.

Rydval, J., Slaboch, V., Tomandl, L. (2005) “Strengths and Weaknesses of the Czech Cadastral System: An Analysis of Present System”, From Pharaohs to Geoinformatics. FIG Working Week 2005 and GSDI-8. Cairo, Egypt, April 16th–21th, 2005. Available from: <http://www.fi g.net/pub/cairo/papers/ts_28/ts28_06_rydval_etal.pdf> [Accessed March 15, 2012]

Saner, R. (2002) “Quality Assurance for Public Administration: A Consensus Building Vehicle”, Public Organization Review, Vol. 2, No. 4, pp. 407–414.

Ščerba, T. (2009) “Some questions on the Act No. 300/2008 – e-Government Act”, Masaryk University Journal of Law and Technology, Vol. 3, No. 3, pp. 401–414.

Siegfried, T. (2007) “E-government in Germany”. In Nixon, P. G., Koutrakou, V. N. Eds. E-Government in Europe: Re-booting the state. Oxon: Routledge.

Špaček, D. (2009) Czech POINTS – The Czech Showcase? State and Administration in a Changing World. May, Budva, Monte Negro. Bratislava: NISPAcee. Available from: <http://www.nispa.org/fi les/conferences/2009/papers/200906081039120.Spacek_CzechPOINT.pdf> [Accessed June 5, 2012]

Torres, L. (2004) “Trajectories in public administration reforms in European Continental countries”,Australian Journal of Public Administration, Vol. 63, No. 3, pp. 99–112.

Udo, J. G., Bagchi, K. K., Kirs, P. J. (2008) “Assessing web services quality dimensions: the E-SERVPERF approach”, Issues in Information Systems, Vol. 9, No. 2, pp. 313–322.

Undheim, T. A. (2008) “Best practices in eGovernment – on a knife-edge between success and failure”, European Journal of ePractice, Vol. 2, No. 2, pp. 23–46.

Vossey, B. (2009) Measuring Returns from e-Government: The case of French Local Government. Paris: MSIT. Available from: <http://www.slideshare.net/benjaminvossey/measuring-returns-from-egovernment-the-case-of-french-local-government> [Accessed March 24, 2012]

West, D. M. (2005) Digital government. New Jersey: Princeton University Press.Xenos, M. (2006) Software Metrics and Measurements. In Khosrow-Pour, M. (ed).

Encyclopedia of E-Commerce, E-Government, and Mobile Commerce. Hershey (Pa): Information Resources Management Association.


Vrednovanje e-Government projekata: europski pristup1

Pavel Mates2, Tomáš Lechner3, Pavel Rieger4, Jitka Pěkná5

Sažetak

Cilj je članka analiza današnjih pristupa vrednovanju e-Government projekata kao specifi čnih projekata javne uprave, s tim da se predlažu izmjene tih pristupa radi uklanjanja uočenih slabosti. Za analizu empiričkih rezultata ranijih studija korištene su ne-normativne teoretske metode. Naime, za provjeru nedostataka današnjih pristupa korištena je metoda dedukcije, za formiranje opće tipologije e-Government projekata korištene su metode analogije i uspoređivanja, a metoda indukcije korištena je za određivanje pogodnih metrika i indikatora. Rezultati se temelje na analizi znatnog broja e-Government projekata koji su realizirani u različitim europskim državama tijekom zadnjih dvadeset godina. Temeljni rezultat analize je formiranje tipologije deset najčešćih tipova e-Government projekata. Iz rezultata istraživanja proizlazi zaključak da se ova tipologija može koristiti kao jezgra novog okvira za vrednovanje e-Government projekata koji uklanja probleme ranijih pristupa, omogućuje podešavanje metrike i pokazatelja za svaki tip projekta, pri čemu zadržava međusobnu usporedivost rezultata i time omogućuje korištenje benchmarking metode.

Ključne riječi: vrednovanje javnih projekata, menadžment javnih projekata, e-Government, evaluacija javnih usluga

JEL klasifi kacija: D73, E02, G38, H43

1 Ova je studija dobila potporu za podršku istraživanja i razvoja IG508012 “Javna uprava i ICT: Promjene u formalnim i neformalnim odnosima”, dotaciju je pružila Visoka ekonomska škola u Pragu.

2 Izvanredni profesor, University of Finance and Administration Prague, Faculty of Social Studies, Estonská 500, 101 00 Prag 10, Češka. University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Prag 3, Češka. Znanstveni interes: upravno pravo, zaštita osobnih podataka, javna uprava. Tel.: +420 210 088 831. Fax: +420 224 095 547. E-mail: [email protected].

3 Doktorand, University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Prag 3, Češka. Znanstveni interes: e-Government, primijenjena informatika, javna uprava. Češka. Tel.: +420 224 095 620. Fax: +420 224 095 547. E-mail: [email protected].

4 Doktorand, University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Prag 3, Češka. Znanstveni interes: e-Government, primijenjena informatika, javna uprava. Tel.: +420 224 095 620. Fax: +420 224 095 547. E-mail: [email protected] (kontakt osoba).

5 Doktorandica, University of Economics, Prague, Faculty of Economics, náměstí Winstona Churchilla 4, 130 67 Prag 3, Češka. Znanstveni interes: ekonomski aspekti javne uprave, metode komunikacije u javnoj upravi, regionalni razvitak. Tel.: +420 224 095 620. Fax: +420 224 095 547. E-mail: [email protected].

Documents

Towards e-Government project assessment: European approach i istrazivanja/Arhiva Zbornika radova/mates... · that is called “ITPOSMO”. This panel consists of “Information”