Audit of IT SystemsSARQA / DKG Scandinavian Conference, October 2002, Copenhagen

Sue Gregory

Sue Gregory, Genmab A/S, October 2002 2

Purpose of IT System Audit

• To assure that established standards are met for all phases of the validation, operation and maintenance of computerised systems.

• To monitor the GxP compliance of computerised systems.

Sue Gregory, Genmab A/S, October 2002 3

Types of IT System Audit

• "Spot Check" – not an audit in its own right, but conducted as part of a facilities-type audit

• Vertical – (specific) looks at defined elements in great depth

• Horizontal – (general) looks at the entire system but in less depth

Or maybe combination – review of the entire system in general and then specific elements in depth

Sue Gregory, Genmab A/S, October 2002 4

IT System Audit - Auditor Requirements

• Auditing skills

• Knowledge of applicable regulations and regulatory expectations

• Knowledge of computer system validation process

• Knowledge of software development life cycle (SDLC)

• Technical IT skills / knowledge

Sue Gregory, Genmab A/S, October 2002 5

Some applicable regulations and references

• GLP Consensus document, The application of the principles of GLP to computerised systems, environment monograph 116, OECD 1995

• Rules governing medicinal products in the European Community, Volume 4 Annex 11, computerised systems, Eudralex.

• 21 CFR part 11 Electronic Records; Electronic Signatures, Final Rule, FDA 1997

• Guidance for Industry, Computerized Systems used in Clinical Trials, FDA 1999.

Sue Gregory, Genmab A/S, October 2002 6

Some applicable regulations and references

• PDA Journal of Pharmaceutical Science and Technology, Technical Report No 31 – Validation and Qualification of Computerized Laboratory Data Acquisition Systems, 1999 supplement, Volume 53, Number 4

• GAMP guide for validation of automated systems in Pharmaceutical Manufacture, version 4, GAMP forum, 2001

• International Standard, ISO/IEC 12207 – Information Technology – Software life cycle processes, 1995 and amendment 1, 2002

• Guidance for industry, General principles of software validation; final guidance for Industry and FDA staff, FDA, 2002

Sue Gregory, Genmab A/S, October 2002 7

Some applicable regulations and references

• And of course:

– Any relevant internal policies, guidelines and procedures

Bear in mind that the area is evolving and new interpretations arefrequent. Monitor the literature and relevant websites for currentdevelopments, e.g.:

– FDA warning letters, GMP trends etc– www.crsc.nist.gov/publications/nistpubs/index.html– www.pda.org/techdocs/index.html – www.groups.yahoo.com/group/21cfrpart11/messages

Sue Gregory, Genmab A/S, October 2002 8

IT System AuditRequired skillA

udit Type

Auditing Validation SDLC Technical

Spot check Vertical ? ? ?

Horizontal

Sue Gregory, Genmab A/S, October 2002 9

Skills vs System compliance levelR

equir

ed a

uditor

skill

s

0% 100%

0%

100%Auditing skills

Validation knowledge

SDLC knowledge

Technical IT skills

IT System validation compliance level

Sue Gregory, Genmab A/S, October 2002 10

Technical Skills vs System Compliance Level

I ncreasing compliance level,increasing technical skills

required

Lowcompliance

level, minimaltechnical skills

required

Audi

tor

- re

quire

d te

chnic

al s

kills

leve

l

IT System - validation compliance level

Highcompliancelevel, hightechnicalskills levelrequired

Sue Gregory, Genmab A/S, October 2002 11

Software Development considerations

• Same standards apply to purchased software and software developed in-house

• Documented SDLC; followed

• Documented specification of requirements for the system; fully traceable

• Documented specifications of functionality and design; fully traceable

• Documented standards for coding; followed

• Documented testing by supplier; unit, integration and system level

Sue Gregory, Genmab A/S, October 2002 12

Approach to IT system "Spot Check"

• Determine implementation date

• Ascertain whether there is a validation report, check date, authorisation and conclusion

• Ascertain whether there is a log of changes since the implementation date

• Obtain a list of SOPs related to the system, ascertain that these are authorised and cover use, maintenance, ……… etc.

Sue Gregory, Genmab A/S, October 2002 13

Horizontal IT audit - basics

• User / System Requirements Specification“It is not possible to validate software without predetermined and documented software requirements” FDA, principles of software validation, 2002

– Authorised (internally) and chronologically correct– Precise requirements covering all functions the

system will perform– Uniquely identified– Verifiable

Sue Gregory, Genmab A/S, October 2002 14

Horizontal IT audit - basics

• Traceability– Check that each requirement is traceable through

the subsequent specifications and tests

– Is there evidence that each requirement has been addressed?

Sue Gregory, Genmab A/S, October 2002 15

Horizontal IT audit - basics

• Validation Plan

“The validation must be conducted in accordance with a documented protocol”FDA, principles of software validation, 2002

– Authorised and chronologically correct

– Describes who does what and when

– Describes or references how

Sue Gregory, Genmab A/S, October 2002 16

Horizontal IT audit - basics

• User Testing– Test Plan

– Test acceptance criteria

– Test records

– Final test report

• Ensure the system can properly perform its intended functions

• Ensure the users can understand and use the system

Sue Gregory, Genmab A/S, October 2002 17

Horizontal IT audit - basics

• Validation Report– Authorised and chronologically correct

– Summarises the validation exercise

– Describes deviations and errors encountered

– Includes clear statement of success or otherwise of validation

Sue Gregory, Genmab A/S, October 2002 18

Horizontal IT audit - basics

• Authorised operating procedures covering:– Maintenance and repair– Disaster recovery – Security– Back-up and restore– Administration– Periodic review– Data collection and handling– Change and configuration management

• Evidence of their implementation

Sue Gregory, Genmab A/S, October 2002 19

Horizontal IT audit - basics

• Training– Staff involved in the validation

– Staff involved in routine use of the system

– Staff involved in development and maintenance of the system

Sue Gregory, Genmab A/S, October 2002 20

Additional considerations

• Vendor Audit

• Installation

• Development Processes

• Internal IT department

Sue Gregory, Genmab A/S, October 2002 21

Additional considerations

• Vendor Audit (software development)– ISO Quality Systems

– SDLC

Sue Gregory, Genmab A/S, October 2002 22

Additional considerations

• Development Processes– Coding – written standards, followed– Code review – pre-planned, documented– Unit tests – owned by developers, documented– Configuration management – Testing:

• Test Strategy

• Test Plan, scripts, cases

– Error reporting– Release procedure– User documentation (help files, user manual etc)

Sue Gregory, Genmab A/S, October 2002 23

Additional considerations

• Installation– IT department SOP

– Protocol, pre-approved and followed

– Records

– Report

Sue Gregory, Genmab A/S, October 2002 24

Additional considerations

• Internal IT Department processes– Installation

– Change Control

– Security

– Training

– Document control

etc.

Sue Gregory, Genmab A/S, October 2002 25

Practice makes perfect…..

• Start small

• Define audit’s scope

• Allow plenty of time

• Start with the general requirements

• Focus on the words audit and system

Sue Gregory, Genmab A/S, October 2002 26

….start practising!