It Strategic Audit Plan387

The Auditor General of Québec

Strategic Plan For IT Performance Audit

A three-year plan of proposed audits with a framework to guide the audit approach.Related strategies for audits to be performed in the fields of information technology and electronic communications in the ministries and agencies of the Québec government.

February 2001

Prepared by:Yves Denis Clarence Kimpton Denys Martin Guy Perron

This document has been translated from French using a translating software.

For information, clarification and queries on this report, contact: Mr. Denys Martin, Phone: 418-691-5935 ext. 4085Email: [email protected]

THE AUDITOR GENERAL OF QUÉBEC

CONTENTS Foreword ..............................................................................…….......... 1

I Introduction .....................................................................…........ 3

II IT Environment Definition.................................................….......... 3

A. Governmental Directions and Recent Developments.....…....... 3

B Legal and Regulatory Framework..................................…..... 4

C Roles and Responsibilities............................................…..... 6

D Resources Invested.....................................................….... 7..III Strategy of intervention ........................................................…....

8

A. Steps Carried Out.......................................................…..... 8

B Benchmarks And Audit Limitations ......….............…….…..... 10

C Principles at the Basis of the Strategy..........................…...... 10

D. Audit Universe.........................................................…….... 11

E. Problems Associated with IT& C .............................…….... 12

F Reference Works.......................................................…...... 12

G Audit Selected. .....................................................……...... 12

H. Follow Up on Previous Audits..............................…......…..... 13

I Audits Report....................................................…….…....... 14

J Audit Resources Requirements......................................…..... 14

K Annual Audit Cycle and Timeline........................……....…...... 15

L Training ………...........................................……........……..... 16

M. Audit Process and Project Management ....................……….... 16


IV. Conclusion ...................................................……...............……..…... 19

Appendices

1 Québec Policy for the Internet To act differently: Initiatives associated with the priority " Bringing together the State the citizen and the companies "

2 Plan of action " the governmental information highway – For better serving the citizens and the companies " ( Québec)

3 Description of key issues covered by legislative auditors in Canada and

overseas

4 Comparison of the principal models of analysis for IT& C

5 List of Audit Subjects

6 Project Organisation Manual Table of Contents



Foreword

The growing strategic importance of the fields of Electronic Communications and Information Technology (IT& C) led the Auditor General of Québec (AGQ) to dedicate to it of the specific work of audit in February 1997. Within the AGQ office, a directorate, Information Systems Audit Management (DVSI) - section optimisation, was given this task and was provided April 1998 of a first strategy with intervention identifying the particular problems which are attached to the fields of IT& C.

Thus, the DVSI constituted and carried out various projects of optimisation of the resources specific to the fields of IT& C and collaborated in other audit projects on IT aspects. Work to date led to the principal following reports:

Overall Funds (aspect information system): Report 1997-1998, volume 1; Adaptation to year 2000 of the IT systems to the government of Québec: Report

1997-1998, volume 1; Observations of the Auditor General (aspect adaptation to year 2000 of the IT

systems): Report 1997-1998, volume 2; Management of the social services for young people (aspect information systems):

Report 1997-1998, volume 2; Adaptation to year 2000 of the IT systems to the government of Québec: Report

1998-1999, volume 1; Acquisition of services (partnerships public/private phase): Report 1999-2000

divided into volumes; Process of development and system maintenance of information for the Ministry of

revenue: Report 1999-2000, volume 1: Management of electronic communication and information technology for the

Ministry for Transport: Report 1999-2000, volume 2; ERP Project ("GIRES") Report 1999-2000, volumes 1 and 2.

Regarding audit priorities for the three next years, those were established by selecting projects comprising of high level of importance and risk starting from a summary list of principal projects from specialised documents, other legislative auditors, recognised stakeholders or team-members.

This strategy is a tool for periodic analysis on the evolution of IT& C and related audit interventions carried out, approaches used and the results obtained, in order to ensure the relevance of our work and continuously to improve our practice. It seeks to be also a tool to identify the types of audit interventions to be most useful and in consideration of the priorities of the AGQ and the government of Québec vision for IT& C.

THE AUDITOR GENERAL OF QUÉBECStrategic IT Audit Plan

Prepared by Denys Martin Email: [email protected]

document.doc 23/03/2001 11:07:00 PM Page 2

mailto:[email protected]



I Introduction

Fruit of convergence between IT one and telecommunications, IT& C represents a strategic sector for the public administration, because they appear among the principal factors of modernisation and improvement of the productivity. In Québec, information technology constitutes one of the priority axes of the governmental strategy of renewal of the public service.

The Treasury Board of Québec defines information technology as being any software, electronic material or combination of these elements used to collect, store, process, communicate, reproduce, protect or delete from information. The electronic communications constitute a specialised subset of information technology supporting the processes of transmission of remote information using electronics, radioelectrical, and optical or electromagnetic means. Also, by extension, taking into account their intimate links with information technology, the processes of management of information belong to the sphere of activity of IT& C. The whole of these elements is gathered under the name of IT Resources.

II. IT Environment Definition

A. Governmental Orientations and Recent Developments

In its advising in the management of IT resources, the Treasury Board, contributes to set up a renewed framework of governmental management and ensures the coherent national deployment in order to improve quality and the effectiveness of the public service and to support the work of modernisation of the public administration and the socio-economic and cultural development of Québec.

For the financial year 2000-2001, the Treasury Board pursues, via its secretariat, the following goals: To ensure the co-ordination of the IT resources so as to preserve governmental

coherence; To implement an ERP system ("GIRES" (ERP for human, material et financial

Resources)); To define required initiatives to ensure the protection of the personal and

confidential information as well as the security of information and electronic exchanges;

To support the implementation of the Québec Policy for internet information; To support the interdepartmental projects of partnership aiming at simplifying,

accelerating and at facilitating services to citizens and companies.

The principal stake is to allow the harmonious deployment of the governmental information highway, by stressing overall co-ordination and by supporting ministerial initiatives. This information highway constitutes an economic and regional development tool and should make it possible to profit from productivity in the governmental administration, to ensure a good exposure for Quebec abroad, to offer better information to citizens as well as services well adapted to their living and working conditions, to improve education and the social development and finally, to promote the French language and the culture québécoise. In addition, initiatives





associated with the priority of “Bringing Together the State, Citizens and Companies” and " the Québec Policy for the Internet – Acting differently” are provided in appendix 1. Moreover, the contribution of the Public administration to the implementation of the information highway is framed more particularly by the plan of action titled The governmental information highway – For better serving the citizens and the companies The strategy of implementation and work engaged appear in appendix 2.

Several governmental projects have started a few months ago. Among the most significant, "GIRES" ( Oracle ERP ), an integrated system of human, financial and material resources, SERTIR, the server for all ministries and agencies to facilitate electronic transactions with citizens and companies, and the telecommunications network for social services which, by being secure, provides fast and confidential exchanges between the stakeholders and the managers of the network, will contribute to the evolution of initiatives such as access to a single patient file and the issuing of a smart card for the Government Medical Insurance Board of Québec ( RAMQ ). In fact, several projects are moving in all the governmental spheres of activity:

Programs, services and administrative forms (Ministry for Relations with citizens and Immigration);

Laws and payments (Publications of Québec); Electronic Trade (Commission of the health and the occupational safety ); Register of personal and rights (Ministry for Justice); Municipal Information highway (Ministry for municipal affairs and the greater

Montréal); Hello Québec.com (Québec Tourism); Computerised Program being used for processing of files and information retrieval

(Public records); Multi-media Catalogue Iris (national Library of Québec); Multi-media Québec (Ministry for Culture and Communications); special Products (Ministry for natural Resources); Products and services cadastral (Ministry for natural Resources); Electronic Exchanges and forms electronic (Ministry for Revenue); Linguistic Resources (Office of French language); Certificates of birth, marriage or death by Internet (Director of Civil Status).

B Legal And Regulatory Framework

Several laws, regulations, directives and decisions govern the management of information technology to the government of Québec. However this framework varies according to the influence of various laws and Treasury Board instructions.

Thus all the relevant ministries, agencies and entities whose human resources comes under the Law on Public Administration. Only some elements of the regulatory framework and directives and decisions touch the agencies distinctly from one organisation to another. For their part, the companies of the government are not really affected, because they manage their resources in an autonomous way.

The principal elements which frame the field of information technology.





Initially, it is Law on Public Administration in its articles 64 to 66, which specifies the principles and responsibilities for management relating to IT resources. These resources must be managed in order to: to use in an optimal way possibilities of information technology and

communications like means of management of human, budgetary and material resources;

to contribute to the attainment of the objectives of accessibility and simplification of the services to citizens;

to support dialogue enters the ministries and agencies and the division of their expertise and their resources.

As regards acquisitions of property and services, Law on Government Purchasing Services includes provisions for buying or rental services, for ministries and agencies concerned and the required deliverables for their activities. Moreover, the Treasury Board sets up a framework on the procurement agreements. Similar activities in information technology must comply with the regulatory framework. Law on governmental services in ministries and agencies regulate the specific functions and capacities of the ministry, defines procedures for services and institutes specific funds (printing services, air service, IT services, etc.).

The National Assembly currently considers a bill (161) concerning the legal framework of information technology. It aims at ensuring legal security of communications carried out by people, associations, companies or the State by means of documents, the functional equivalence of electronic documents and their legal authenticity. On the regulatory plan, Management Framework for IT Resources in the Government of Québec (CT # 187036 of April 4, 1995) stipulates that leaders of ministries and agencies must fully assume the management of their IT resources, in observance of regulation and governmental orientations, in order to achieve strategic goals selected. They must, for this purpose, ensure projects and investments relevance, efficiency and effectiveness.

Ministries and agencies must also comply with the recent Directive on security of digital information and electronic exchanges in governmental Administration, which took effect on February 4, 2000. This one states the guiding principles of security of digital information and electronic exchanges in governmental administration, identifies the stakeholders concerned with related security management, determines responsibilities for ministries and agencies and plans the introduction of mechanisms of suitable co-ordination and collaboration in order to ensure availability, integrity, confidentiality of digital information, authentication of users and irrevocability of documents which they compile or of actions that they pose. Another directive adopted in October 1999 relates more specifically to the processing and destruction of any information, register, data, software, operating system or another good protected by a royalty, stored on microprocessing equipment.Ministries and agencies concerned must also comply with specific decisions and technological guides of standardisation from the Treasury Board. Various other obligations apply to the whole of the information held by ministries and agencies. These obligations are fixed inter alia by the Law on access to documents of public agencies and on protection of personal information and by the Law on personal files.





Finally, since 1992, the government of Québec has a policy of usage of French in information technology in order to promote the use of French as a language of design, use, diffusion and training.





C Roles and Responsibilities

The responsibilities for the fields of IT& C in the government administration are entrusted to ministries and agencies, the Treasury Board and ministries responsible for particular mandates. Graph 1 presents a global view of these stakeholders.

Graph 1

Electronic Communication And Information Technology With The Government Of Québec

Environment Model

Ministry for Relations with citizens and Immigration

Ministry for Industry and

Trade

Regulator / co-ordinator /

catalyst

Suppliers of services

Under-secretariat information highways and IT resources

DGSIG DGT Office of the

French language

Co-ordinating committee of governmental information highway

Ministries and Agencies

The Québec government management framework for delegates responsibility to ministries and agencies for management of its IT resources. However, the Treasury Board must approve investment related to information technology whose original costs are higher than the threshold established with the regulation. Among the other obligations, ministries and agencies must submit to the Treasury Board of Québec, their forecasts of appropriations for IT resources and a triennial plan detailing their projects. In addition, four funds of information technology were made up for financing IT projects.

The Treasury Board develop rules to ensure security of IT resources, including protection of personal information and other confidential information. It also plans initiatives to promote common infrastructures and services. It also manages an incentive fund for interdepartmental partnership.

In term of governmental services, the Treasury Board offers a government wide server. Its directorate (DGSIG) has the role of providing various IT services to





ministries and agencies of IT services on various platforms. These include computer processing, access and connectivity, advice on standards and computerisation, data warehousing. Its general directorate of telecommunications (DGT), for its part, offer the following: Telephony Services ; Network Services; Wireless Communications (WAP); Information Highway Services.

Co-ordinating committee of governmental information highway

This committee is charged to ensure coherence of action and relative co-ordinate work of key stakeholders implies in the implementation of the plan of action adopted by the government with the constitution of an information highway for public administration.

Ministry for Relations with Citizens and Immigration

This Ministry has the mission of supporting the recourse to new technology to give access to direct transactions between citizens and the administration.

Ministry for Industry and Trade

This ministry proposes, with the government or Standing committee of purchases, orientations, priorities related to the uses of the governmental purchasing power likely to support economic and technological development.

Office of French language

The Office ensures follow up on the plan of technology focussing on promoting the French language for each Ministry and agencies as well as the diffusion of information on the availability of IT French products.

D. Resources Invested

The government of Québec spent CAN $737 million in the field of information technology in 1995-1996, most recent year for available statistics and represents more than two thirds of the global expenditure of the government.





Réseau de la santé18%

Réseau de l'éducation

14%

Ministères et organismes budgétaires

46%

Organismes extra

budgétaires22%

Figure 1 Source : Bilan et perspectives 1995-1996, Secretariat du Conseil du trésor.

55 p. cent of 432 millions dollars spent in 1998-1999 impacted on these five ministries:

Social Solidarity 86,6 M$ Revenue 78,2 M$ Natural Resources 30,5 M$ Transport 23,5 M$ Education 20,8 M$

In addition, expenditure in information technology agencies comes mainly from the Commission Of Work Health And Safety, Québec Automobile Insurance Board, the Control of revenues of Québec and the Control of health insurance of Québec. In 1997-1998, budgets of expenditure in information technology of these entities were respectively 57, 50, 32 and 20 million dollars for a total of 159 million dollars.

A more recent source, the consolidated state of the fixed assets of the government of Québec at March 31, 1999, mentions that the net amount of the fixed assets relating to the IT development is 581 million dollars.

III. Strategy of intervention

Several audit interventions are possible in IT& C. The development of this strategy of audit intervention required a significant effort to target sectors where the Auditor General can best support parliamentary control while carrying out the promotion of sound practices of management. What is required above all, it is the relevance of our interventions.

The present section describes initially the step for our strategy, benchmarks and audits limitations, specifies the principles at the basis of the strategy such as our audit universe. It also surveys audits carried out here and elsewhere in the fields of IT& C and document their work. Lastly, it provides details on audits follow up, resources required and training, timelines and project management.





A. Steps Carried Out

Activities carried out to define this strategy:

Survey and analyses of benchmarks and audit limitations Strategic orientations and standards of the Auditor General of Québec Limits of interventions in the field of IT& C

Definition of guiding principles for the strategy of intervention Analysis of strategic orientations and standards of the AGQ Meeting with AGQ senior staff Analysis from experiences of previous audits

Survey the Québec government current initiatives and concerns Interviews with the Treasury Board of Québec Survey and analyses of Treasury Board of Québec decisions as regards IT& C Held a one day roundtable with significant stakeholders on the management of

IT& C with the government of Québec Analysis of documentation

Analyse audit approaches in the management of IT& C (legislative auditors of Canada and provinces, internal auditors of various agencies and companies) Contacts (AG – Canada, Hong Kong, Auditor General of Western Australia) Analysis of audit reports from legislative auditors Analysis of strategic plans of audit in the field of IT& C

(ex: Hydro-Québec, AG – Saskatchewan, State of Florida)

Analyse leading IT & C Business Models ( COBIT, CMM, etc.)

Selection of criteria to support our proposals for audit projects Analysis of criteria to support the selection of audit projects Analysis of Risks Models

Selection and documentation of audit projects

The selection of projects was done in three steps:

1. Survey of audit subjects based on risks. Each team-member had to identify in an individual way various audit subjects considered significant and/or risky.

Criteria of importance which were to be considered by team-members are those appearing with the handbook of audit is: resources used by the entity; intrinsic importance; economic, social and environmental incidence of the subject; degree of sensitivity and visibility of the subject; topicality of the subject.





As for the criteria of risks, several models of analysis of risks were analysed and we used that knowledge to develop our own, focussing on management processes. Four types of risks were considered:

Inherent risks of the potential audit subject of (concerns expressed by IT audit experts, documentation and audits carried out here and elsewhere);

Risks related to strategic management Risks related to operational management (planning and organisation,

development and implementation, operations and support, monitoring); Risks related to the management of customers service

As a result, we then agreed to sixteen (16) potential audit subjects

2. Evaluation and classification of the most relevant audit projects. Each team member initially rank the relevance and the risks associated with each subject. By consensus, we then arrived at a list.

3. Proposal for a sequence of projects to be completed in the three next years by taking account initially of the scope and the risks of the projects, and then of various considerations and impact on the strategy.

B. Benchmarks And Audit Limitations

The strategy of intervention is supported by several key elements. Initially the strategic orientations of the Auditor General require that we maximise the impact of our audit work in optimisation of resources and our recommendations. Our work focuses on resources optimisation for activities with significant expected deficiencies. These orientations recognise the increasing importance of IT& C and related audit work in optimisation of resources.

These audits should be performed on an annual basis and the focus is on the field of education, health and social services. The orientations further stipulate that our work must encourage the government to improve its management practices, the measurement of its performance and its Accountability. In addition, a handbook of audit dedicated to the audit of the optimisation of the resources specifies the methodology that must be applied for this type of audit.

C. Principles At The Basis Of The Strategy Our audit vision of IT& C is based on an analysis of the context and acquired experience:

Focus on management and use of technology in governmental administration;

We focus on management and use of technology in governmental administration. We therefore do not consider audit of administrative programs such as subsidies to business and citizens for connecting to the Internet.

Identification of audits where an IT management expertise is required;

Various work of audit is completed in the fields of IT&C and this, as well by the regular teams of audit as the DVSI. The present strategy makes state only





interventions where a particular expertise in audit management of IT&C is specifically required.

Priority with government wide audits;

This type of audit supports observations reflecting an overall situation, more useful for the members of Parliament. Moreover, these audits emphasise the deficiencies on a comparative basis.

Audit focus on risks;

A structured approach of evaluation of the risks was tested successfully at the time of our audits relating to the development of the information systems. As recommended in the new strategic orientations of the Auditor General, an evaluation of the risks is carried out for the selection of the projects.





Implementation of audits for ongoing projects;

The government is setting up various IT structures (information highway, ERP, etc…) whose design and implementation should take a few years, an approach of audit a priori (during the development stages) is required to ensure the publishing of timely recommendations. This does not exclude that the approach a “after the fact.

Closer links between the constitution of audit projects and their completion;

Authorisations from senior AG executives earlier in the annual cycle.

Priority with the audits with restricted SCOPE and of short duration;

The fields of IT& C evolve quickly. Consequently the formulated observations are likely more quickly to become obsolete. These fields cover also a broad range of activities and the whole of the functions of management. It is thus desirable that our audits are sufficiently targeted and that the fruit of these audits is available quickly.

D. Audit Universe

Entities likely to be audited

The Law on the Auditor General stipulates that the audit of the books and accounts of the Funds consolidated of the revenue, of a public organisation and an agency of the government include financial audits, compliance audits, and performance audits.

This law also mentions that the Auditor General can proceed to audit registers, files, documents and accounts of agencies, associations or companies who use any subsidy granted by a government agency.

In the light of this information, the Auditor General of Québec is entitled to audit:

Government Agencies Treasury Board; Ministries; National Assembly; Lieutenant-governor; Citizen Ombudsman; Electoral Commission;

Government enterprises (with agreement with the Administration Board) Network of services

Health and social services Education

Demarcation with standard audits of optimisation of the resources

The regular teams of audit perform work on IT& C activities. They relate mainly to information required to manage the programs (sufficiency, accuracy, up to date, comparability) while focussing on some specific aspects of the general management





of IT security and information, system development technology. In terms of authorised work, a relatively tiny portion of the budget of time is dedicated to this field of activity.

Although they make it possible to comment on the most obvious gaps, this work does not present a vision of the whole of the management of IT& C deployed by the entities. It is thus difficult, based only on this work, to inform the members of Parliament about the management of IT& C in the governmental administration.

The work of the DVSI makes it possible to obtain this vision, in particular by the present strategy of intervention, and to cover subjects with the most impact. All the aspects of the management of IT& C can thus be considered, which includes also the management of information.

E. IT& C Related Issues

Many legislative auditors carried out audits in the fields of IT& C and highlighted several problems such as project management, security and benefits management. The most active is certainly the US General Accounting Office (GAO). Appendix 3 gives an outline and results of their audits.

F. Reference works

The literature proposes also various models of analysis likely to be used in the framework of our audits. The principal ones, described summarily in Appendix 4, are as follows:

ISACA (Information Systems Auditing Control Association) – CobiT Guidelines Management

CICA – The management of IT controls GAO – Information Technology Investment Management (ITIM): With framework

for Assessing and Improving Maturity Process CICA and AICPA –Systrust ms/md – Principles and criteria of reliability of the systems SEI – Model evolution of the software capacities (CMM) ITRB (Information Technology Resources Board) – Managing Systems Information:

In Practical Assessment Tool PIM (Project Institute Management) – A Guide to the Management Body of

Knowledge NSA (National Security Agency) – Systems Security Engineering (SSE) – Capability

Maturity Model GAO –Information Security Management: Learning from Leading Agencies

Several of these models have a similar analytical structure. Most impressive are CMM and CobiT. These models help evaluating activities in the fields of IT& C especially in relation to the levels of performance and quality of the IT resources. However, the model recommended by " Information Systems Auditing Control Association " (CobiT) is the most complete. As a result, interventions of the AGQ in the fields of IT& C should initially rest on CobiT, yet exploit the best of the other models.

G. Audit Selected





Sixteen audit subjects were considered in the development of this strategy. Those were identified using the knowledge obtained of the use of IT& C in the government of Québec and the audits which were carried out, of the examination of various decisions of the Treasury Board of Québec, the audits carried out by other legislative auditors, exchanges with stakeholders concerned and models of analysis available.

Starting from a descriptive card of each subject (see Appendix 5), those were initially evaluated according to their criticality (combination of the scope of an subject and risks which is associated there) by using the criteria mentioned with the section relating to the step used. The results are presented at the following graph. According to the grid used, a project whose risk and scope are estimated at 10 comprises greatest criticality. It is thus to say that all the evaluated projects obtained a note equal or higher than 5 for the two aspects considered.

Criticité des objets de vérification

0

5

10

0 5 10 Ampleur

Risque

1

2,34,5

6,7,891014

11,1215

13

16

1. Central and ministerial management of the electronic service delivery (ESD); 2. Audit Subject3. Audit Subject4. Audit Subject 5. Audit Subject 6. Audit Subject 7. Audit Subject 8. Audit Subject 9. Audit Subject 10. Audit Subject 11. Audit Subject 12. Audit Subject 13. Audit Subject 14. Audit Subject 15. Audit Subject 16. Audit Subject

By drawing up the passing note to 6 out of 10 compared to each of two examined dimensions, four subjects were eliminated for the moment, that is to say projects 13, 14, 15 and 16. Of the twelve remainders, we thereafter structured projects of which we can audit over a three years period.

H. Follow Up On Previous Audits





The new standards for follow up of the recommendations of the audits of the optimisation of the resources aims at informing the members of Parliament of the degree of application of recommendations from the Auditor General. All audits must have a follow up within a maximum of three years and with a level of high insurance. The follow up must be planned at the end of the audit of the optimisation of the resources and this planning must be revised, if it is required, after a parliamentary committee.

The analysis of the audits carried out during the last years by the DVSI and strategies of follow up:

Year Of Publicatio

n

Audits Follow Up Strategy

1997-1998 (volumes 1 and 2) 1998-1999 (volume 1)

Adaptation to year 2000 The initial audit on December 31, 1997 was the subject of two follow-ups: one in October 1998 regarding the activities of the Treasury Board of Québec and at December 31, 1998 for the whole of the audit.

1999-2000 (volume 1)

Development of the information systems – Ministry for the Revenue of Québec

A follow up in two years, in April 2002.

1999-2000 (volume 2)

Management of information technology the– Ministry for transport of Québec

Taking into account the scope of the reports and recommendations, not very useful to carry out a follow up before summer 2003, which will leave time to the Ministry to bring the corrective measures required.

I. Audit Report

As several proposed audits are of governmental scale, it is advisable consequently to define a strategy. For this type of audit, we plan to survey about fifteen entities in addition to the central agencies, which could required the drafting of several sectoral reports in addition to the report to the National Assembly.

Taking into account previous work, we plan to produce only one report whose observations and recommendations will be customised to obtain relevant responses from each audited entity.

J. Audit Resources Requirements

Various elements must be considered to determine the resources required to the audits:





Overall workload In this respect, the present strategy presents a first selection of 12 projects at added value, which could be reviewed at the time of subsequent years. One also should not neglect work of follow up and up to date setting of the strategy of intervention.

The frequency of publication to the annual Report In corollary with the guiding principle consisting in carrying out audits of short duration derives frequent publications. Two reports will be published annually, which means that the human resource must be adjusted consequently.

Critical Mass The implementation of audits in as specialised and wide fields as those of IT& C requires the constitution and the maintenance of a team having various expertise and experiences to ensure required synergy.

The stability of the existing team The effort of work was evaluated according to the team in place which has an unquestionable experience as well in audit as into IT. Its basic elements to date carried out successfully various audits. Workload must be compatible with the resources available.

Taking into account these elements, five resources are required to support the workload with an adequate framework (planning, support with the drafting of the reports, quality assurance, and accountability).

K. Annual Audit Cycle and Timeline

Various activities must be carried out annually to correctly assume our responsibilities with regard to the audit for the fields for IT& C in the government for Québec.

Audit of critical subjects identified with the strategy of intervention

Deployment of approximately 3 000 hours over seven months are planned for each audit, which represents an approximate cost of 210 000 dollars. This estimate is based on projects where questionnaires will be used each time that it will be possible to do it and use of previous experience in the domain. Obviously, this estimate will be revised after the completion of each audit.

Audit Time will be distributed as follows:

Audit Phase Workload Distribution

Preliminary analysis 25 %Preliminary Report of analysis including its validation near the audited entities

5 %

Detailed examination including the validation of the observations near the audited entities

45 %

Audit report 20 %Parliamentary committee and preparation of the follow up 5 %

Audit Follow Up





As mentioned previously, a follow up of the audits must be carried out within three year. During three next years, two follow-ups adding up 2 800 hours will be carried out (Revenue Ministry and Transport Ministry ).

Updating The Strategy Of Intervention

The present strategy will be updated annually. 450 hours is planned each year for this exercise. The following table presents the timeline of implementation with five resources. Five audits could be carried out in the long term from now to December 31 2003 in addition to the follow-ups and annual up to date of the strategy of audit.

L. Training

This strategic plan has a critical impact on training audit resources. Knowing the next audit mandates, it will be thus much easier to synchronise the work of training with the needs for the audits. These specific needs will be listed inside the regular process of development of the human resources.

M. Audit Process and Project Management

The principles at the base of this strategy require certain adjustments with the audit process and the mechanisms of project management. Without compromising the quality neither of the audits nor to derogate from the code of practice, it is recommended to reduce certain stages of work such as the process of authorisation of the projects.

The implementation of mandates of governmental scale within rather short times requires a very tight management of project. In this context, the following changes are proposed:

Develop, as well with the stage of the preliminary analysis as of the detailed examination, a Project Organisation Manual detailing the required work, the timelines and deliverables such as the mechanisms of management of the contents, management of the changes and validation of the deliverables (see standard table of content to appendix 6). This Project Organisation Manual is to be approved by the principal director.

Systematic Use of a group of review of the contents of the deliverables composed members of the DVSI or others to discuss and optimise the strategies, work and observations.

Introduction of a mechanism of quality assurance for formal approvals on specific deliverables.

Formalisation of follow up using monthly reports of project (work, timeline, implementations of the period in progress and those to come) and, if need be, of reports of progress (variations compared to the strategies planned and the anticipated observations, solutions suggested).

The following table presents the interventions suggested of quality assurance and peer review and of validation with the audited entity, which aim at ensuring the quality of work and the results. It is suggested these mechanisms of quality form an integral part of the framework of project management for the DVSI.









TIMELINE OF IMPLEMENTATION OF THE PROJECTS OF AUDIT (5 resources) 2000-2001 2001-2002 2002-2003 2003-2004

H 108 120 132 126 138 126 72 108 120 138 132 96 108 120 126 132 138 120 78 102 126 138 126 102 108 120 126 132 132 126 78 102 132 138 120 108 M 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12 01 02 03 04 05 06 07 08 09 10 11 12

A NUMBER OF RESOURCES 1 3,7 3,5 4,0 4,5 4,0 0,3 0,2 2 1,0 4,0 4,0 5,0 4,0 2,0 3,0 2,7 0,2 0,3 3 0,5 1,0 0,5 1,0 1,0 0,5 1,0 0,5 1,0 1,0 0,5 1,0 0,5 1,0 1,0 4 2,0 2,0 2,0 2,0 2,0 2,0 5 0,3 6 3,0 5,0 3,0 7 1,0 4,8 4,5 4,0 4,5 2,0 0,2 0,3 8 4,5 4,0 4,5 1,0 9 2,0 5,0 3,6 1,4 3,0 5,0 5,0 0,2 0,3 10 4,0 5,0 5,0 5,0 3,8 1,7 3,0

HOURS 1 444 462 504 621 504 32 34 2 126 288 432 600 552 264 288 400 28 38 3 66 126 69 138 132 63 132 69 138 126 63 132 66 138 120 4 264 192 252 204 240 216 5 36 6 360 390 306 7 108 576 567 528 621 240 28 38 8 567 528 594 126 9 204 630 497 176 306 540 600 28 36 10 504 390 510 660 524 204 324 Early

480 528 630 690 630 288 432 600 690 660 480 540 610 630 660 690 600 390 510 630 691 630 510 540 600 630 660 660 630 390 510 660 690 600 540

# Project

1

Activity 2000-2001 2001-2002 2002-2003 2003-2004 TOTAL

1 1 Central and ministerial management of the electronic service delivery (ESD) 2535 66 - - 2601 2 846 2104 66 - 3016 7 261 534 525 258 1578 - - 456 456 456 1368 - 36 - - - 36 - - 1056 - - 1056 4 - 2640 66 - 2706 - - - 1815 - 1815 3 - 204 2749 64 3017 5 - - 1404 1712 3116

TOTAL 3678 7060 7081 2490 20309

THE AUDITOR GENERAL OF QUÉBEC 23/03/2001 11:07:00 PM Page 20




NOTE: The black squares indicate the date of publication to the annual Report. 1: See the list of the projects on page 19.





Deliverable AGQ Peer Revie

w

Validation with the audited entity

Project Organisation Manual (POM) – preliminary analysis phase

X

POM for detailed examination phase X Outline of the audited field X X Roles and responsibilities X X Regulatory framework X X Audit Project of (objectives and criteria) X X X Expected deficiencies and significant observations X X Strategy of the detailed examination X Timeline of the detailed examination X Resources required for the detailed examination X Strategy of Report X Deliverables for the detailed examination X Final Report X X X Evaluation of the project of audit X

In addition, it is required also to constitute a permanent Consultative Committee in order to always ensure the relevance of our interventions. This one will have to validate the present strategy of intervention and its annual up to date settings just as certain deliverables of the projects of audit the such Report of the preliminary analysis, the strategy of audit and the various produced reports.

It is suggested the following flow chart, presenting the functional links recommended at the time of the implementation of a project of audit, is retained. This structure supports the efficiency of the activities and the accountability of the stakeholders concerned.


Principal director And director

of audit

Consultative Committee

Audit Project Leader

Assistant Auditor General

Team-member

Team-member

Project Review Group




IV. Conclusion

The present analysis allowed the development of a strategy of intervention, which it would be advisable to retain with regard to the fields of IT& C for the government of Québec. This analysis shows that the resources devoted to this sector are considerable and that they take an increasingly significant place in the control of the governmental activities. The Auditor General has thus all the reasons to be interested in the audit of IT activities in government. On this subject, the strategy of intervention suggested and the topics suggested will be able to guide our audit work.

The present strategy will be updated annually in the light of the results of next work of the DVSI and the development of the general expertise on the AGQ in the fields of IT& C .







Appendix 1

POLICY QUÉBEC OF THE INTERNET OF INFORMATION - TO ACT DIFFERENTLY INITIATIVES ASSOCIATED WITH THE PRIORITY " TO BRING THE STATE THE CITIZEN

AND THE COMPANIES "

Initiatives Persons in charge

To define architecture and the means in order to make sure that the ministries and agencies make available on the information highway all the general information considered as being from public interest that they produce and hold.

The Ministry for Relations with the citizens and Immigration, to set up the governmental Repertory québécois in order to allow the citizens and the companies to have access to the description of the services offered to the population, the references concerning the governmental documents like to the co-ordinates of the employees of the State; the electronic repertory will be accessible in Internet network.

To make the provisions required so that the citizens and the companies can communicate directly, by electronic way, with the employees of the State charged to provide them information and services.

To develop, from here June 1998, required applications to allow the ministries and the government agencies to comply with governmental engagement to make available in Internet the forms administrative most frequently used by the citizens and the companies.

To be appropriate with the ministries and the agencies of initiatives of adaptation of the human resources of the public office to the changes rising from the setting in place of the governmental information highway, in particular by making this adaptation one of the priorities as regards development of the human resources for the next years.

To ensure a coherence of action, to co-ordinate the implementation of the information highway in the public sector. This function results in particular in the responsibility: to assume the presidency and the secretariat of the Co-

ordinating committee of the governmental information highway, formed of the administrators concerned;

to conceive and make evolve/move a vision of the governmental information highway and to propose the means to make it divide by the whole of the Administration;

to exert a regular and rigorous follow up governmental plan of action and to ensure the annual up to date setting of it;

to take care to ensure technological coherence required to the division of the infrastructures and the services ;

to report to the government annually.

To carry out an evaluation of the mechanisms of the current process of selection of the partners and to draw up an assessment of their application; to specify expectations of the government in the application of the principles of equity and transparency like in the implementation of the concept of division of risks and benefit.

To establish strategies aiming at early tracking and the follow up of the public and parapublic markets strategic. Moreover, to take care to promote the introduction of technological innovations into the public and parapublic markets and their use by way of technological window.

To constitute funds, for one two years period, dedicated to

Ministry for the Relations with the citizens and Immigration

Treasury Board of Québec

Ministries and government agencies

Ministry for the Relations with the citizens and Immigration




Treasury Board of Québec and Ministry for Industry, the Trade, Science and technology



Appendix 1

the setting in place of services common related to the deployment of the information highway in the public sector like to the incentive on the organisational partnerships in the service of public services.

To continue the steps having the aim of equipping, within two years, the sector sociosanitaire of an Intranet allowing to benefit from the many possibilities of the information highway.

To take care that are made available on the information highway from the services of information intended to increase the autonomy of the citizens in the prevention of the diseases and the protection of their health and their wellbeing.

To develop a policy and to define the methods of deployment of services of telemedecine, telediagnosis and remote monitoring.

To gradually establish a chart health with microprocessor to replace the current chart of health insurance.

To continue the steps in order to establish applications of support for the practice in the various mediums of intervention such as the maintenance for residence, the urgencies, the lodging of the old people and the protection of youth.

To define, in collaboration with the Control of the health insurance of Québec, the orientations with regard to a network of information sociosanitaire protected with an aim of supporting these needs for information.

Ministry for Health and the Social services



Medical Health Insurance Board of Québec




Appendix 2

PLAN OF ACTION THE GOVERNMENTAL INFORMATION HIGHWAY – FOR BETTER SERVING THE CITIZENS

AND THE COMPANIES

Strategy of implementation

To ensure an effective co-ordination and leadership; Focus on partnerships; To quickly set up the basic infrastructures and common services; To simplify the processes and to increase coherence in the service of the services to the citizens and the companies; Focus on human resources for better managing the change; To exploit all the potential of the existing modes of financing.

Plan of action

Field " human Resources "

Analyses of impact on work, the development of competencies, management of the changes, information, the involvement and the training.

Field " Process "

It is a question of establishing new shared horizontal applications and reengineering of systems of management to integrate the management of the material, financial and human resources (project "GIRES" (ERP), to develop a catalogue of public purchases interns electronic goods and services (CAPE), forms, and systems of transactions and payments electronic.

Field " administrative Framework "

Frameworks of management of the IT resources, the human resources, security and architectures, legal framework and strategic planning, of the tools and the services of day before allowing to analyse the most significant innovations developed in the world have regard with the deployment of information technology in the public administration and the documentary engineering.

Field " Information "

Access to information and services of the government by the means of Web site, of the governmental Intranet, banks and catalogues of data like the governmental repertory, the diffusion of information, the development of Web sites; it will be a question of bringing up to date on the one hand, the concept of State network by developing single windows for services intended for targeted customers by branch of industry and on the other hand, the concept of " government on line " using forms on line and of direct transactions on line.

Field " common Infrastructures and services "

They are the grid systems of the data, images, sounds, voice, such as the RICIB (integrated Network of IT and office automation communications), the network sociosanitaire (the RTSS of the network of health and the social services), the infrastructure with public keys, counters multiservices, message handling facility, CRIMP (search engine, services of electronic trade, etc), the Intranets and extranets, the connections with Internet and the electronic mail, the access points, the local infrastructures and the development of information highways, the systems of videoconference.


Appendix 2


Appendix 3

DESCRIPTION OF PRINCIPAL SUBJECTS COVERED BY LEGISLATIVE AUDITORS

Years reviewed: 1998 –1999 - 2000

A search of subjects of interest covered by legislative auditors during last years was completed in autumn 2000. It aimed at giving a progress report on the principal tendencies of audit in the fields of IT& C. The exhaustive inventory of these subjects of interest is available on request.

Except for the GAO (United States General Accounting Office), research was carried out starting from the two following steps: initially, an examination of the contents of the reports deposited by the principal legislative auditors referred in the " good addresses " of the AGQ; then, provided it was possible a search inside these reports using the search tools provided by the site itself, being " Netscape " or " Acrobat "

For the GAO, a research, initially by the title and then by the contents, was made for all the reports submitted by the group " Accounting and Information Management Division " (AIMD). We have the title of the relevant audits and the results obtained as presented in marginal notes in the reports because of the abundance of the covered subjects and our desire to remain concise in our descriptions.

Several of the listed subjects belong to work that exceeds the strict framework of the fields of electronic communication and information technology. Thus our search identified 135 Audits report related to the fields of IT& C, distributed according to following breakdown:

57 related to audits highly relevant to our needs 36 related to audits moderately relevant to our needs 42 related to audits related specifically to security, that had been carried out

by the GAO

Overall, twenty-five (25) highly relevant audits relate to implementation activities by an entity for the development and maintenance of information systems in way similar to the work undertaken by the AGQ at the Ministry for Revenue and with that of Transport; several of these audits were carried out by the GAO. In addition, seventeen (17) highly relevant audits relate to specific information systems, including seven (7) for ERP. Lastly, six (6) other highly relevant audits focused on strategic aspects of management of IT& C, four (4) with various aspects of the electronic service delivery, three (3) with architectural considerations or overall infrastructures and two (2) with activities associated with the field with telecommunications.

As for moderately relevant audits, those relate to aspects of the fields of the rather marginal and very specific IT& C issues.

Lastly, the audits connected to security, carried out by the GAO, give a report on deficiencies as regards to IT security in all American agencies. These deficiencies generally focus on 1) security management of the programs, 2) access control, 3) change management for the process of development, 4) segregation of duties, 5) internal controls in the information systems, 6) controls related to the continuity of the services. Reports were also produced during the proliferation of IT virus and following malicious acts carried against Web sites. Some reports focus on protection of the critical infrastructures.


Appendix 3

All these audits report indicate the majority of legislative auditors carried out sectoral rather than government wide audits, due to their traditional approach to auditing. There is a slight move towards performing audits focussing on the strategic management in the field of IT& C rather than on their operational management or on activities associated with only one information system with an entity. We also noted some highly sophisticated audit models to guide the audit work carried out by the GAO.


Appendix 4

COMPARISON OF THE PRINCIPAL MODELS OF ANALYSIS FOR IT& C

This document presents and comments on nine (9) models of analysis which can be used with regard to the activities undertaken in the fields of electronic communication and information technology (IT& C).

It appears timely to stress that all the models presented here comply with the spirit of subparagraph 32 of section 4220 of the handbook of audit of the Auditor General of Québec which stipulates that " the standards enumerate three types of criteria which are regarded as generally recognised and which, if they are relevant for the mandate, must be privileged: 1) criteria established in the laws and the payments; 2) criteria of the CICA; the 3) criteria establish by other agencies of recognised experts who follow a procedure of approval calling upon consultations and public discussions ".

ISACA- CobiT – Guidelines Management

ISACA has a framework based best practices and audit control of the information systems. It particularly aims at helping the leaders to understand and manage the risks relating to IT and the links between the management processes, the technical questions, the needs for control and the risks.

The framework of reference is known under the acronym CobiT (Control Objectives for Information and related Technology). It is structured around four main fields of management implying 34 processes of management associated with information technology:

Planning and organisation To define a strategic IT plan To define the architecture of information To determine the technological orientation To define the organisation and the working relationships of the IT function To manage the investment into IT To communicate the objectives and the orientations of management To manage the human resources to ensure conformity with the external requirements To evaluate the risks To manage the projects To manage quality

Acquisition and setting in place To identify the solutions To acquire and maintain the application software To acquire and maintain architecture technical To develop and maintain the procedures IT To install and validate the systems To manage the modifications

Distribution and support To define the levels of service To manage the services ensured by thirds To manage the performance and the capacity To ensure a continuous service


Appendix 4

To ensure the security of the systems To identify and charge the costs To raise awareness and train the users To assist and advise the customers of the IT services To manage the configuration To manage the problems and the incidents To manage the data To manage the installations To manage the operations

Monitoring To control the processes To evaluate the adequacy of the internal control To acquire an independent insurance To perform an independent audit

Each process of management has goals and objectives, the critical factors of successes related to the adequate implementation of this process of management, resources and IT characteristics, indicators making it possible to measure the performance as well as a narrative description of the five potential levels of maturity (derivatives of the model of evolution of the software capacities (CMM).

CICA – IT Management Controls This model is based on the concept of the roles and comes to establish the responsibilities security control which results from this. In this context, the roles are broken down according to seven axes with knowing, 1) the general direction, 2) the head of the service of information, the 3) owners, the 4) agents and 5) users of the information systems just as the suppliers of services, than it is in the chapter of the 6) development that of 7) the IT operations of the support of the systems. These roles are also broken down according to the activities that result from this. The table below summarises the links between the roles and the activities retained by the business model Roles Activities Directorate-General Approval of the strategies, the policies and the standards;

distribution of the responsibilities; Develop and approval of the plans of businesses

Head of the service of information

Develop of the strategies, the policies and the standards; service of the services of technical support; directorate of centralised services

Owners Definition and written requirements; responsibility for the control and security; confirmation of the controls; evaluation of the risks; classification; delegation; agreements

Agents Comply with of the policies and the standards; logical and physical access authorisation and control of the changes

Users Comply with the requirements of the owners; responsibility for IT resources

Suppliers of services – development

Development and acquisition of systems of application; comply with the policies and the standards; management of the changes; documentation

Suppliers of services – Agreements on the levels of service; planning; operations;


Appendix 4

IT operations and support of the systems

management of the problems; safeguards; disaster recovery plans; management of the changes; support of the systems; physical access

The model makes also a distinction between the concepts of authority, responsibility and accountability.

On these bases, the model then comes to specify the responsibilities as regards management for the risks and control before giving a report on the control, broken down in objectives, standards and techniques:

planning IT acquisition, the development and the maintenance of the IT systems IT operations and the support of the systems IT security plans of continuity and the resumption of IT services controls on the applications

GAO – Information Technology Investment Management: In Framework for Assessing and Improving Process Maturity 1

This model results from the work undertaken by United States General Accounting Office. It identifies the critical processes ensuring success of the investments in the fields of IT& C and organises them around five levels of maturity (in a way similar to the CMM). It is also based on guides developed by the GAO (Assessing Risks and Returns: With Guide for Federal Evaluating Agencies' IT Investment Decision-Making (GAO/AIMD-10.1.13, February 1997)) and the OMB (Evaluating Information Technology Investments, A Practical Guides, Executive Office of the President, Office of Management and Budget, November 1995).

This model focus on investments in the fields of IT& C; according to the following phases:

Selection of the projects: determination of the projects which best support the needs related to the mission for the organisation by taking account of the risks and the returns on the investment.

Control of the projects: assurance that the projects continue to meet the needs and the required levels.

Evaluation of the projects: comparison of the results anticipated and reached.

To satisfy the preceding goals, the model is broken down in five levels of maturity and fifteen critical processes. The model also presents, for each process criticises, the goal of this process, the required prerequisites, essential engagements of the top management, the activities which must be ensured to satisfy the critical process in question just as the objective elements which prove as this critical process is formalised in a suitable way in the evaluated entity. The unit also describes the key practices (tasks) which it is essential to carry out to satisfy the critical process in question.

1 United States General Accounting Office Information Technology Investment Management: With Framework for Assessing and Improving Maturity Process, GAO/AIMD-10.1.23, may 2000


Appendix 4

The table below shows the five levels of maturity and the fifteen critical processes that are associated for them.

Level of maturity Critical processes Training course 1 Creating Investment

Awareness

IT expenditure without a structured investment processes

Training course 2 Building the Investment

Foundation

IT Investment Board Operations IT Project Oversight IT Asset Alignment Business Needs Identification for IT Projects Proposal Selection

Training course 3 Developing a Complete

Investment Portfolio

IT Investment Board Portfolio Selection Criteria Definition Investment Analysis Development Portfolio Portfolio Oversight Performance

Training course 4 Improving the Investment

Process

Post-Implementation Reviews and Feedback Portfolio Performance Evaluation and Improvement Systems and Technology Succession Management

Training course 5 Leveraging IT for Strategic

Outcomes

Investment Benchmarking Process IT-Driven Strategic Business Changes

Lastly, an appendix describing the process of evaluation that the teams of audit should adopt when they undertake work resting the recommended model accompanies the document deposited by the GAO.

CICA and AICPA – SysTrust MS/MD – Principles and criteria of reliability of the systems 2

American Institute of Public Certified Accountants (AICPA) and the Canadian Institute of the Chartered Accountants (CICA) offer a professional service of certification on the reliability of the information systems called " SysTrust ".

In the framework of this service, the auditor evaluates and audits up to what point an information system is reliable compared to four essential principles as regards reliability of the systems: 1) the availability of the system according to agreements' taken; 2) security; 3) integrity, 4) scalability

This model seeks to determine if an information system is reliable, i.e. if a system is able to function without significant error, breakdown and failure during a given period.

2 Talk-survey AICPA/CICA SysTrust – Principles and criteria of reliability of the systems (version 1.0), July 15, 1999


Appendix 4

Criteria established for each of the four principles evoked previously: the definition and documentation relating to the objectives of performance, the

policies and the standards compared to the expected performances and the engagements of the entity like their communication with the human resources concerned;

procedures implementation with an aim of achieving the goals of performance, in accordance with the policies and the standards;

activities of monitoring of the system and the environment allowing to identify any potential degradation and to take suitable initiatives.

SEI – Models evolution of the capacities software (CMM 3

This model results from work of the Software Institute Engineering of the university Carnegie Mellon de Pittsburgh. It is known under the term " Capability Maturity Model " (CMM) in English.

This model makes it possible to evaluate the capacities (power to make) of an organisation development maintenance of information systems. It comprises eighteen key sectors gathered around five levels of maturity. The table below presents the correspondence between the levels of maturity and the key sectors.

Level of maturity Key sectors 1 – Initial The Nile 2 – Can be replicated Management of the requirements

software Project planning Follow up and supervision of software project Management of subcontracting software Quality assurance software Management of configuration software

3 – Defined organisational focusing on the processes Definition of the process of the organisation Training scheme Management integrated software software Engineering of products Co-ordination joint committee Peers Review

4 – Controlled quantitative Management of process Management of software quality

5 – Optimised Prevention of the deficiencies Management of the technological changes Management of the changes of the process

Thus an entity can have a given level of maturity if all the key sectors of this level of maturity and preceding levels of maturity (if required) are satisfied. A key sector known as is satisfied if the very large majority (more than 80%) of the key practices of this sector are adequately controlled by the entity.

It is possible to observe that business model is at the base of the principles of evaluation recommended by CobiT as well as some of the processes of management 3 Software Institute Engineering Model of evolution of the capacities software, version 1.1, CMU/SEI-93-TR-24, ESC-TR-93-177, February 1993


Appendix 4

of this last. It was however observed that CobiT disregarded sometimes certain elements of business model

This model is also at the base of two tools for analysis of the risks in the fields of IT& C used with the government of Québec, S:PRIME and S:P 2 RAM

ITRB (Information technology resources Board) – Managing Systems Information: In Practical Assessment Tool 4

This reference work is the fruit of experiences accumulated by Information Technology Resources Board (ITRB), group of leaders in the fields of IT& C of many American federal agencies. It is in fact a tool for evaluation which aims at making it possible the American entities governmental federal to better understand how the strategic implementation of the fields of IT& C can support their mission and improve their products and services.

The grid of analysis comprises nearly three hundred questions (yes/no) gathered around three prospects: the strategy, which makes it possible to know where the organisation moves: determination of a mission and a vision the need comprehension of the customers the presence of a plan of business the leadership, which makes it possible to mobilise the people actions deployed by the general directorate the decision-making process and of strategic planning the process of management of project the process of management of the performance the technology, which makes it possible to set up information systems the process of acquisition of the goods and the services the presence of architectures (work, data, systems, technological, flow of

information)

SME (Project Institute management) – A Guide to the Project Management Body of Knowledge5

This guide focuses on best practices for project management of project. The aspects approached revolve around the various elements of management required to a powerful management of the projects (of any nature).

More precisely, the document identifies and describes the practices of management of project generally accepted which should implementation in the agencies. It gathers them around nine sectors of expertise (presented below).

The guide also positions the various phases of a project (initiation; planning; implementation; control) as well as the practices of management of project related to each one of these phases. The practices of management of project generally accepted can thus be considered according to two axes: according to the processes to be

4 Information Technology Board Resources Managing Systems Information: In Practical Assessment Tool, February 1999 5 Project Management Institute Standards Committee A Guides to the Project Management Body of Knowledge, 1996


Appendix 4

satisfied (e.g. management of the cost of the projects) or according to phases' of the projects (e.g. planning).

The list below presents the nine sectors of expertise to be satisfied as well as the aims:

Management of the integration of the projects: to co-ordinate the various components of a project such as planning and execution of the project, and control of the change, cost and quality of the project

Management of the scope of the projects Time Management of the projects: Management of the cost of the projects: Management of the quality of the projects: Management of the human resources assigned to the projects Management of communications in projects: Management of the risks of the projects: to detect and control the risks associated

with the project Management of acquisitions : to take care to obtain services or products of quality

NSA (National Security Agency) – Systems Security Engineering – Capability Maturity Model (CMM) 6

This model initiated by the National Security Agency describes the essential characteristics of an organisational architecture of security in the fields of IT& C according to the practices generally observed in the agencies. It covers the following aspects:

the whole of the cycle of life of an information system: the whole of the organisation, including the activities of management, organisation

and software engineering interactions with the various fields of IT& C interactions with the other sectors such those of acquisitions, the management of

the systems or of certification

The model breaks down initially security in the fields of IT& C according to three axes: 1) the evaluation of the risk: determination and scope of the threats; 2) the setting in place of measurement: design and implementation of the required solutions; 3) assurance: corroboration and needs for security.

As for the CMM, the model proposes to determine a level of maturity (from 1 to 5) of the security of the evaluated organisation. It examines the status of the twenty-two practical issues of security and management and to compare them with a grid of maturity defined in term of results. Level 1 implies that all the activities related to security are carried out at least in a basic way whereas the higher levels require than the activities are planned and followed (level 2), well defined (level 3), controlled quantitatively (level 4) and continuously optimised (level 5).

6 Carnegie Mellon University Systems Security Engineering - Capability Maturity Model: Model Description Document, Version 2.0, April 1, 1999


Appendix 4

GAO – Information Security Management: Learning From Leading Agencies 7

This model results from work undertaken by the GAO in eight private agencies recognised as leaders for IT security. It identifies critical issues required to ensure an adequate management of IT security.

Co-ordination of the activities To set up a group dedicated to IT security person in charge for the group can reports to top management To provide the group with human and financial resources required To ensure a continuous training and required professional certifications

Evaluation of the risks and determination of the needs Acknowledge IT resources are critical to the organisation To evaluate the risks and the security Ensure users are accountable To manage the risks on a continuous basis

To set up suitable policies and controls To lay down policies and controls in relation to risks Ensure the central group can support the policies

To implement the required programs of involvement Ensure training of users with regard to the risks and adopted policies

To measure and evaluate the effectiveness of the policies and the controls To evaluate the factors which affect the risks and which undermine security To take account of the results of evaluation to determine the later needs and to

report to the authorities.

7 United States General Accounting Office Security Information: Serious Deficiencies Places Critical Federal Operations and Assets At Risk, GAO/AIMD-98-92, September 1998


Appendix 5

SUBJECT OF AUDIT IN THE FIELDS OF IT& C

Name of the subject Central and ministerial management of the electronic service delivery (ESD)

Summary description The government of Québec use the fields of IT& C to increase its performance and to improve to a significant degree services to the citizens and the companies. With this intention, it counts, like other public administration, to carry out significant investments in structuring projects aiming at the setting in place of new fashions of organisation of work in a context of electronic service delivery (ESD).

In a very simplified way, the ESD of the government to the citizens and the companies implies four large functional components: 1) services gateway; 2) batch services or specific; 3) integration with the information systems of the ministries and agencies (M/O); 4) the specific and shared infrastructure.

However, the design and the standardisation of several of the strategic elements composing this ESD remain to carry out whereas significant work are already made by the M/O in the framework of the modernisation of the public administration. Moreover, the interdepartmental committees of required work were not set up yet. In this context, significant challenges of management can be identified, related to the diversity of the existing resources which must be connected in the context of an integrated solution. Moreover, of new common or divided components must be made available, operated and managed from the point of view of the overall needs.

The following challenges are currently listed: 1) the overall management of the ESD to the citizens and the companies; 2) the management of the change; 3) the management of the security and the confidentiality of information; 4) the standardisation of information and the mechanisms of exchanges; 5) the management of the development of the information systems associated with the ESD; 6) the management and operations of the common infrastructures.

Brief description of the importance of the subject and its risks Importance of the subject 8/10

This subject implies significant investments (Difficult to quantify but several hundreds of million dollars) in many governmental entities. Many initiatives are already in hand. The ESD must make it possible to radically modernise the way in which the public administration relates with the citizens and the companies and offer electronic services.

Risks of the subject 9/10 This subject involves very significant risks based on the scope of work required in the long term and the gaps corroborated by another AGQ audit team. Four risks can be raised: 1) of erroneous alignments of the ESD offered by the entities taking into account the governmental and ministerial strategic needs; 2) uncoordinated work; 3) management (planning, organisation, co-ordination, evaluation) overall incoherent in the absence of a framework of management, an overall architecture and well defined scenarios of implementation; 4) a management " all alone " rather than in " network " of the changes (organisation of work, structures organisational, culture of the agencies).


Appendix 5

I - Objectives of audit and Expected Deficiencies

Name of the project Central and ministerial management of the electronic service delivery (ESD)

Question of importance: Do mechanisms put in place by the government for the ESD ensure it is with economy and efficiency and that it contribute in a substantial way to a greater efficiency of the State and to better quality of the overall services offered.

Preliminary audit objectives Expected preliminary deficiencies To ensure that the development and the deployment of the projects associated with the implementation with the ESD fall under a governmental plan and ministerial plans including: a business model (governmental or ministerial),

objectives of businesses and a strategy a plan of migration including the priorities, the costs,

the benefit, the financing, the risks and the timeline

ESD is carried out in an anarchistic way, without examination of respective priority, of problems of financing, contribution to the modernisation of the public administration in a multiannual vision.

No overall management for the setting in place of ESD the management of the changes, when it exists, is only

sectoral

To ensure government and the ministries and agencies evaluate the results compared to the governmental and ministerial objectives strategic relating to the modernisation of the public administration

No effective management by results processes in place.

To ensure that standards are established to frame the interactions with the customers (citizens and companies) in a context of ESD

standardisation is incomplete and many standardised elements are not respected

poor corporate image To ensure the setting in place and the optimal use of the common or shared infrastructures

Sectoral infrastructures is set up whereas it would be possible to be based on common infrastructures

Common infrastructures (e.g. TO CRIMP) were created but they are little used

To ensure mechanisms were installed to constitute and share at lower cost the expertise specific to the ESD

Mechanisms implemented but they are insufficient (participating entities; subjects considered)

Strategy of audit: Budget of time adding up 2 600 hours over a 6 months period by a team made up of four people. The budget of time includes the indirect work associated with the quality assurance, with the review by the peers, the drafting of the Audit report and a possible parliamentary committee on the subject. The majority of work will be held between February 2001 and June 2001, being understood that the final Report would be published in volume II of the Report to the national Assembly for the year 2000-2001.

To the level of the central entities, our work will bring us to the Treasury Board of Québec and the Ministry for the Relations with the citizens and Immigration. On the level of the sectoral entities, our work will imply the publishing of a questionnaire to some fifteen entities (ministries and the most significant agencies of the government of Québec) and work on the spot near five of them.

It is expected that the preliminary analysis report of this project of audit will be submitted to the principal director concerned in April 2001.


Appendix 5

I – Evaluation of the project of audit proposed

Audit Project Name Central and ministerial management of the electronic service delivery (ESD)

Timely period of

implementation

YES X NOT Justification: the activities considered are upstream many activities of implementation to come and they

condition at least partly success of them Of the stakeholders pleads that the central management of the ESD is at present deficient the issue of the modernisation of the public office is a contemporary subject of interest for the

member of Parliament

Criteria of evaluation

Summary of the principal characteristics Evaluation

COPLAN

Importance of the deficiencies

Gaps on overall management

Risks identified: alignments of the ESD; resumption of work; incoherent overall management; management " all alone " of the changes

The consequences of the gaps can be large: corporate image of the government; effective implementation of modernisation; efficient implementation and deployment of work / 25 PTAs

Innovative aspects

The governmental services are delivered traditionally by mail, telephone or in person; it is only since very recently in a way integrated by electronic means

The central and ministerial management of the ESD has never been the subject of a performance audit. / 20 PTAs

Importance The changes brought by the ESD are of a great visibility for the citizens and the companies

The modernisation of the public administration includes the implementation of the ESD

Hundreds of million dollars will be allocated to the ESD during the next years / 20 PTAs

Public exposure The level of the debate is very high and covers the two facets of overall management (central and sectoral aspects)

/ 15 PTAs

Efficiency of the project

For total work of some 2 600 hours, the performance audit would make it possible to the national Assembly to have independent information on the overall management of this aspect related to modernisation of the public administration and better services to the citizens and the companies / 20 PTAs

TOTAL / 100 PTAs


Appendix 5


Appendix 6

PROJECT ORGANISATION MANUAL TABLE OF CONTENTS

1. INTRODUCTION - Description of the audit project - Scope - Purpose

2. ORGANISATION - Flow chart of functional links - Description of the roles and the responsibilities

3. PROJECT DEFINITION

Summary of the project: - Definition - Objectives - Question of importance- Principal observations

Detailed examination: - Audit Project – audit objectives and evaluation criteria - Expected deficiencies and significant issues

4. WORK PLAN

- Preliminary analysis strategy - Timeline of the preliminary analysis - Resources required for the preliminary analysis - Deliverables from the preliminary analysis:

- Outline of the audited field and scope of the project - Roles and responsibilities - Regulatory Framework - Audit Project – audit objectives and evaluation criteria - Expected deficiencies and significant issues - Strategy of the detailed analysis - Timeline of the detailed analysis - Resources required for the detailed analysis - Report Strategy

For the detailed examination: - Detailed analysis strategy of the (identification of the deliverables) - Timeline of the analysis detailed by deliverables - Resources required for the detailed analysis - Report strategy - Audit Programs - detailed Plans are in appendix - deliverables at the end of the detailed analysis - Report (final)

5. RULES OF MANAGEMENT


Appendix 6

- General Principles - Follow up mechanisms

- timelines- monthly project report to the director of audit - Contents Management- Quality assurance (Q&A), Peers Review (PR) and validation of deliverables

For the detailed examination, to add: - Follow up on deficiencies observed / observations, at review progress meetings

6. RISKS ASSOCIATED WITH THE AUDIT PROJECT


Documents

It Strategic Audit Plan387