Is your documentation accurate, comprehensive and accessible?

© Montrium Inc. 2016

1

WEBINAR

How to Validate Computerized GxP Systems: Is your documentation accurate, comprehensive and

accessible?

Presented by Chrysa Plagiannos Senior Validation & Compliance AnalystMontrium


2


Your Presenter

Chrysa PlagiannosSenior Validation & Compliance Analyst – Montrium

Educational Background:• Degree in Chemical Engineering from McGill University

Career Overview:• 13 years experience in pharmaceutical industry• Extensive validation experience, including validation of manufacturing processes,

equipment, facilities, and computerized systems• Experience in QA functions (approval of controlled documents, complaint

management)• Experience in Engineering functions (equipment selection and purchasing,

investigation reporting)


3


About Montrium

• Montrium is a technology company focused on the delivery of enterprise content management solutions and professional services for the Life Sciences industry.

• Montrium’s ECM platform, Montrium Connect, consists of solutions for Electronic Trial Master Files, Regulatory Document Management & Submission Planning and Quality Management.

• We offer also services in validation, quality assurance, auditing and strategysurrounding technologies.

• Montrium was founded in 2005 and is headquartered in Montreal.• Montrium works with pharmaceutical companies in North America, Europe and Asia.


4


Today’s Focus

1. Introduction• What is a GxP System?• What is an Electronic Record?• Why is Testing Important?• Validation Terminology

2. Preparation for Testing• Planning • Defining the scope of testing• Tips for writing test scripts• Assembling a Team

3. Executing Test Scripts • Setting up Prerequisites • Good Documentation Practices• Creating Screen Captures• Documenting Non-Conformances

4. Traceability & Reporting5. Conclusion & Recommendations


5

Introduction• What is a GxP System?• What is an Electronic Record?• Why is Testing Important?• Validation Terminology

1


6


What is a GxP System?

• GxP: Set of compliance regulations including but not limited to, Good Laboratory Practice (GLP), Good Manufacturing Practice (GMP), and Good Clinical Practice (GCP).

Examples of computerized systems used for GxP activities :

- Systems used to generate electronic records (ex. training records, complaint files, change requests, incident reports).

- Systems used to store/ manage data that is subject to regulations, such as adverse event reports and raw data obtained during laboratory and production activities


7


What is an Electronic Record?

Definition as per 21 CFR Part 11.3:“Electronic record means any combination of text, graphics, data, audio, pictorial, or other information representation in digital form that is created, modified, maintained, archived, retrieved, or distributed by a computer system.”


8


Why is Testing Important?

Reason 1: Validation provides documented evidence that a system is fit for its intended use.• System requirements provide an objective standard to which the system is tested

during validation. • System functionality is tested by executing test scripts designed to demonstrate

that the requirements were met. • Requirements are based on:

– Regulatory requirements– End-user/ business needs– Risk assessment/ mitigation


9



Reason 2: Validation documents are auditable records. • Inspectors are looking for:

– Documented evidence that validation activities were performed in accordance with approved procedures

– Readily accessible records (can be produced in a timely manner)– Proof of sufficient testing, based on intended use and potential risk

• Scope of validation should be tied to the risk the system poses to:– Patient safety– Product quality– Data integrity


10



Reason 3: Electronic records managed by GxP systems can be auditable records. • These records will also be presented to Inspectors/ Auditors• Examples of records

– Complaint files– Certificates of Analysis – Standard Operating Procedures (SOP)

• Failure to perform adequate testing could lead to:– Inability to produce documents upon request – Loss of critical data – Compliance violations ex. missed regulatory filing deadline


11


Why is Testing Important?• Real-life Example: Warning Letter


12



• Summary of Issues Identified in the Warning Letter: – System not fully validated upon release– Critical issues found with interim report used to release system:

• Inadequate training• Incomplete documentation (SOPs and work instructions)• Inaccurate data migration of legacy database

– Technical issues with system also identified: • Inaccurate dates captured in system• Follow-up information recorded as new event

– System deficiencies led to inaccurate assessments of adverse events and untimely submission of alerts


13


Validation Terminology

• Qualification: Process of demonstrating whether an entity is capable of fulfilling specified requirements.

• Validation: Process of establishing documented evidence which provides a high degree of assurance that a specific process will consistently produce a product meeting its predetermined specifications and quality attributes

• Note: Qualification is part of validation.

• Visit Montrium blog for more information

Presenter

Presentation Notes

Say verbally.


14


Validation Terminology

Presenter

Presentation Notes

Tool is validated (used within a business process). Prior to validating the application, the underlying infrastructure must be qualified.


15


Types of Testing

• Verification that the appropriate system was delivered and that it is installed correctly (i.e. in a manner consistent with requirements).• Ex. IQ

• Verification that the system is capable of consistently operating in accordance with established functional specifications.• Ex. OQ

• Verification that the system consistently operates in accordance with established requirements in its typical operating environment and is fit for its intended use.• Ex. UAT, PQ

• Note: Terminology can vary from organization to organization. What’s important is to ensure that the system is thoroughly tested regardless of the term used to describe the testing.


16

Preparation for Testing• Planning • Defining the scope of testing• Tips for writing test scripts• Assembling a Team

2


17


Validation Planning

• Validation Plans should be formally documented• Types of Planning Documents:

– Validation Master Plan– Qualification/ Validation Plan(s)

Governed by Validation Master

Plan

GxP System

Stage 2: Validation Plan(s) for

ApplicationApplication (s) Verified by

Stage 1: Infrastructure

Qualification Plan

Infrastructure• Physical Hardware• Virtual Machines

Verified by


18


Validation Planning

• Information to be included in the Planning Document(s):– Scope of qualification/ validation activities– Project roles and responsibilities– Required deliverables– Required procedural controls

• For simple systems with few requirements, validation deliverables can be combined (ex. combined IQOQ Protocol).

• Important: Conduct validation activities in accordance with the Plan. If the sequence of activities outlined in the plan is not followed, document any outstanding issues along with the justification for proceeding.


19


Scope Definition

• Ensure that the scope is clearly defined.

• For software, it is important to understand the system’s core functionality to be able to define the scope of testing


20


Scope Definition (GAMP®5)

• GAMP®5 methodology can be used to define the scope• What is GAMP®5?

– Guidance documentation which provides a risk-based approach to computer system validation

– A system is assigned to a category based on its intended use and complexity.• Determine the system Software Category as per GAMP®5 • This category is based on the extent to which the system is customized since the

risk of failure and defects generally increases as the complexity of the system increases

• Rule of Thumb: Increased customization leads to more extensive testing


21


Where to Test

• Testing is often performed in multiple environments• It is recommended to perform testing in three different environments:

– Development: Serves as a non-controlled, pre-QA environment for informal testing– Test (QA): Serves as a controlled, pre-Production environment for testing and training– Production: Serves as a controlled operational environment for day-to-day activities

• It is not necessary to perform the same tests in all three environments. Where testing is done is determined using a risk-based approach


22


Advantages of Testing in Multiple Environments

• Prevent the transfer of issues to Production environment• Avoid creating dummy data within the Production environment• Testing that may be destructive to the Production environment can be performed

elsewhere (ex. infrastructure stress testing)• Reduced system downtime• Reduced maintenance and support costs


23


Test Scripts: Basic Characteristics

• Test Script: Used to define and document testing activity• Traceable to requirement(s). • If the test passes, the associated requirements have been met. • Main Test Script Components:

– Test Objective– Acceptance Criteria – Prerequisites– Test Instructions (Step-by-Step)– Expected Results– Designated spaces for recording what was observed


24


Test Scripts: Recording Results

• Data recorded by testers:– Actual Results– Identification of testing sequence (ex. run number)– Overall Result (i.e. Pass, Fail, Conditional Pass, Not Performed)– Cross-references to attachments– References to any non-conformances – Tester identification information (ex. name/ initials) – Date actions were performed


25Example: Test Script

Reference supporting

docs

Reference to Prerequisites

Actual Result

Overall Result

Tester Name

and Date

Run Number

Step by Step

Instructions


26

Test Scripts: Recording Results

Poll: How does your organization record actual results during test script execution?

Option 1: Actual results are written in fullor

Option 2: Results are documented as ‘as expected’ or ‘not as expected’

It is recommended that Actual Results be written in full i.e. document exactly what was observed, instead of simply writing “As Expected”

• Why? : Testers are more likely to detect/ report issues when additional details are provided.


27


Characteristics of Well-Written Test Scripts

• Clearly defined prerequisites

• Concise, reproducible test instructions

• Fields available for recording data

• Precise expected results

• Results independently review


28


How to Record Results? Electronic, Paper or Hybrid

• Execution can be performed in the following ways:– Electronically: Results are recorded electronically, use of e-signatures – On Paper: All results documented by hand– Hybrid Approach: Results documented electronically, printed and signed off

with wet-ink signature• Executing electronically means that the executed test script is

considered an Electronic Record• Electronic execution process must be validated to ensure that the

solution is fit for use and compliant with applicable regulations


30


Advantages to Executing Test Scripts Electronically

• Managing documents electronically – Trace tests electronically to requirements

• Accessibility of Records – Remote access to documentation– Can lead to reduced approval times

• Secure Access– Can assign permissions to electronic documents

• Save Time– Easier to fill out (type results, use of drop-down menus)

• Save Money– No costs related to printing, scanning and storage of physical documents


31


Review of Test Results

• Test results are independently reviewed (by a member of the validation team other than the tester).

• Provides assurance that execution was done properly and facilitates the reporting of any issues.

• This review involves:– Ensuring that executed test script is complete– Verifying that all appropriate documentation has been generated– Checking cross-references to supporting documents– Confirming the overall result


32


Time to Assemble Your Testing Team

• Choose testers: Review the testing documentation and determine how many people are required for testing

• A good tester should:– Understand validation and quality principles – Be knowledgeable of good documentation

practices– Be familiar with the system being tested– Be aware of the purpose of testing

• Select testers based on the type of testing being performed


33


Who Should You Recruit?

Type of Testing Who should be involved? AdvantagesTechnical Testing (ex. IQ)

Involve technical personnel:• System Administrators• IT Professionals

• Leverage system knowledge

• Subject Matter Experts (SME) to evaluate issues

Business Process Testing (ex. UAT)

Involve end-users:• Training Managers• Investigators • QA

• Ability to assess system suitability

• Training opportunity


34


Train Your Testing Team

• Ensure testers have been adequately trained prior to participating in validation activities

• Especially important when involving non-validation personnel (i.e. IT, end-users)• Conduct training in accordance with approved procedures• Successful completion of training must be documented • There are 2 levels of training:

– System Training• Ex. Training on how to use the system

– Validation Training • Ex. Training on validation policy/ procedures, documenting non-conformance, good documentation

practices etc.


36

Executing Test Scripts• Setting up Prerequisites • Good Documentation Practices• Creating Screen Captures• Documenting Non-Conformances

3


37


Preparing Prerequisites

• Prerequisites: Conditions that must be satisfied prior to executing a test script• Recorded within the executed test script document• Examples of Prerequisites:

– Definition of technical components, such as server names, IP addresses– Presence of data that allows for a particular scenario to be tested

• Tips:– Gather supporting documentation, such as user guides– Create dummy data before starting to test


38

Example of Prerequisites


39


Good Documentation Practices

• Governed by internal procedures.• Ensure that testers have been trained prior to execution.• Principles of Good Documentation Practices:

– Accurate– Legible– Contemporaneous – Original – Attributable


40


Annotations: Correcting Text

• Any correction to preprinted text may be made only if it does not alter the original intent of the test.

• Corrections to preprinted or handwritten text may be made by crossing out the incorrect information with a single line stroke and without obscuring the original information.

• The correct information may be entered below, above, or next to the error. • If additional space is required, use a numbered footnote and record the correct

information in the page margin. • All corrections must be initialed and dated


41


Annotations: What Not to Do

Correction Error in AnnotationThis result is bad. good Correction not initialed/ dated.

This result is bad. goodCP 15-JUN-2016

Original text is obscured.

This result is bad.CP 15-JUN-2016

No correction was made.

This result is bad. GoodCP 15-JUN-2016

Illegible correction.


42


Annotations: Best Practices

CorrectionThis result is bad. good 1

This result is bad. good CP 15-JUN-2016

1 Entry Error. CP 15-JUN-2016


43


When is an Annotation Allowed?

• When there is no impact on the original intent of the test

Test Execution Instructions & Results

Step Instructions Expected Result Actual Result Step Status

Initial/ Date

12. Verify that the Training Report displays training requirements.a) Navigate to the Training

Report. b) Verify that training

requirements are displayed for the Trainee identified in prerequisite PRQ-1. PRQ-2

CP 15-JUN-2016

The Training Report displays training requirements.

Result:

______________Attachment:

______________Validation Non-Conformance:


44


When Are Annotations Not Allowed?

• When a correction would alter the original intent of the test • Should be addressed via a non-conformance report

Test Execution Instructions & Results

Step Instructions Expected Result Actual Result Step Status

Initial/ Date

13. Verify that a task is assigned to the Trainee. a) From the Training

Management application, navigate to the Task List.

b) Verify that a new task was assigned and that it is associated to the Training Course identified in prerequisite PRQ-5.

c) Generate a screen capture.

A task is assigned to the Training Manager.

Result:

______________Attachment:

_______________Validation Non-Conformance:


45


Why Are Screen Captures Important?

• Screen captures provide evidence of what was observed in the system during testing (typically included in attachments)

• Used to corroborate the testers’ findings i.e. show whether the system’s state matches the expected result

• While not mandatory, they facilitate review by third party (QA, Auditor/ Inspector)


46


When are Screen Captures Necessary?• Proving Step: Demonstrates that the test objective was met• Non-Proving Step: Intermediary step that allows the tester to perform a proving step (no screen capture required)


47


Tips for Generating Screen Captures

• Make sure screen captures depict the elements being verified• Take screen captures only when necessary (proving vs non-proving steps)• Ensure that screen captures are legible especially when printed• When possible, screen captures should include test conditions which prove that

the test was properly executed (ex. URL, login names)

Presenter

Presentation Notes

Can’t contest results Not mandatory.


48


Screen Captures: What Not To Do

• Verification: Email attachment of Non-Conformance document was received by Tester3 QA

• Issues with this Screen Capture:– Does not show email recipient name, i.e. Tester3 QA– Attachment information is not visible


49


Screen Captures: What Not To Do

• Verification: Email attachment of Non-Conformance document was received by Tester3 QA

• Issues with this Screen Capture:– Relevant information is not visible– Image distorted by resizing (illegible)– Image of entire screen is not necessary in this case


50


Screen Captures: Best Practices

• Verification: Email attachment of non-conformance document was received by Tester3 QA

• Why is this an appropriate screen capture?– Relevant information is visible– Use of highlight to draw attention to elements verified (if your procedures permit the use of

highlight)– Image is not distorted


51


What are Non-Conformances?

• Non-conformances: Event that prevents test acceptance criteria from being met.• Terminology can vary. Other commonly used terms are:

– Deviation – Exception – Test Incident

• Examples of non-conformances:– Discrepancy between expected and actual result– Errors that occurred during execution– Changes to test methodology resulting in test’s intent being altered


52

Poll Question:

What does your organization call non-conformances?

• Non-Conformances• Deviations• Exceptions• Test Incidents• Other


53


Documenting Non-Conformances

• A non-conformance report should:– Be clear and concise– Assess impact (blocking vs non-blocking issue)– Pinpoint root cause(s)– Incorporate SME input – Propose appropriate corrective action(s) – Provide a rationale for actions taken– Avoid overly technical language, define all abbreviations– Include supporting documentation when necessary


54


Resolving Non-Conformances (Step-by-Step Approach)

• Step 1: Provide a detailed description of the non-conformance • Step 2: Determine the root cause of the issue reported

– Examples of possible root causes:• System configuration error• System deficiency• Execution error • Protocol generation error

• Step 3: Identify corrective actions (approved prior to implementation)• Step 4: Implement corrective actions and provide documented evidence that the

issue is resolved


55

Example: Non-Conformance Description


56


Example: Non-Conformance Investigation


57


Example: Non-Conformance Corrective Action/ QA Approval

Presenter

Presentation Notes

QA close out not mandatory. QA in loop to ensure no regulatory impact/ CAPA not nutty


58

Traceability and Reporting4


59


Traceability

• Provide a link between system requirements and the test scripts/procedural controls that verify them.

• Reminder: System functionality is tested by executing test scripts designed to demonstrate that the requirements were met.

• Establishing traceability provides proof that the system functions as intended. • May be presented in a Traceability Matrix or for simple systems (few requirements) can be

documented within another validation deliverable. • A requirement may not be met via testing alone. Procedural controls are often necessary. • If a requirement is not explicitly tested, a rationale should be provided (risk-based

approach).


60Example: Traceability Matrix

Presenter

Presentation Notes

Add rationale for not testing in example low risk out of box.


61


Summary Report

• Issued at the end of the validation. • Main components:

– Identification of all deliverables (Document ID, approval date)– Test Result Summary– Discussion of any non-conformances encountered during testing– Statement of system acceptance (specify any system limitations)

• Report is approved by stakeholders.• Possible stakeholders (depends on type of validation done):

– Process Owner – System Owner – QA


62


Conclusions and Recommendations

• Proper planning is key• Team Work

– Involve stakeholders– Pick the right people for each phase of the project

• Well-written test scripts lead to smoother test executions• Document any testing issues via non-conformance reports• Be sure to establish traceability• Prepare a Summary Report (often reviewed by Inspectors)• If it’s not documented, it didn’t happen!


63


Thank you for your attention!

Questions

[email protected]


64


Have a question? Get in touch!

[email protected]

Montrium Inc.507 Place d’Armes, Suite 1050

Montreal (QC) H2Y 2W8Canada

+1 (514) 223-9153

Montrium S.A.2A, rue Alophe Diederich

L-5820 FentangeLuxembourg

+352.20.88.01.30

Schedule a call with one of our validation experts

http://www.montrium.com