Integrate Risk Management into Validation - · PDF filemanufacturing and computerized systems...

Isabel Rodriguez, PE, CSQEPresident IRP Consulting, LLC

Integrate Risk Management into Validation

Key Concepts

What is Risk-Based Validation?

How to Implement Risk-based Validation

Additional Techniques for Implementing Risk-based Validation

Session Outline

Part 1: Key Concepts

Because we have to

To meet Regulations and Guidelines

Because it is for Good Business

To save lives

Why do we Validate?

Key Sources of Regulations and Guidelines

FDA Guidance & Reference

Documents

PIC/S Guidance Documents

GAMP Guides (ISPE)

IEEE Guides

Regulations (Laws)

FDA 21 CFR…Eudralex Volume…

ICH Guidelines

Company Policies and Procedures

WHO Guidelines

21 CFR Part 11:

Requirement 11.10(a) - Validation

–Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.

Requirement 11.10 (i) System Checks

–Use of Operational System Checks to enforce permitted sequencing of steps and events, as appropriate

What does the FDA say about Validation?

How do we actually Validate?

Budget

Time

Team Morale

Focus on Documentation

The Real Cost of Validation

The objective of Validation is to achieve Compliantmanufacturing and computerized systems that are fit for intended use.

This can be achieved implementing Scalable and Risk Based Validation approach

Why we should Validate

How we should Validate

Principles of Quality Risk Management

Source:ICHQ9

The evaluation of The risk to quality should be based on scientific knowledge and ultimately link to the protection of the patient.

The level of effort, formality and documentation of the quality risk management process should be commensurate with the level of risk.

A systematic approach for the assessment, control, communication, and review of risks to patient safety, product quality, and data integrity

Enables effort to be focused on critical aspects in a controlled and justified manner

Quality Risk Management

Source:GAMP5

Identification and Management of Risks

Scaling of Life Cycle activities according to system impact and risks

Justification for Use of Supplier documentation

Therefore, reducing costs and effort based on scientific and documented system impact and risks

Benefits of Risk-Based Quality

ICH Q9: Quality Risk Management

ASTM E2500: Standard Guide for Specification, Design, and Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment

GAMP5: Science Based Risk Management

ISO 14971: Application of Risk Management to Medical Devices

Standards and Guidelines for Risk Management

ASTM E2500: Standard Guide for Specification, Design, and Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment

ICH Q9: Quality Risk Management

Risk Review

Ris

kC

om

mu

nic

ati

on

Risk Assessment

Risk Evaluation

unacceptable

Risk Control

Risk Analysis

Risk Reduction

Risk Identification

Review Events

Risk Acceptance

Initiate

Quality Risk Management Process

Output / Result of the

Quality Risk Management Process

Risk

Ma

na

gem

entto

ols

GAMP5: A Risk-Based Approach to Compliant GxPComputerized Systems

Planning Reporting

Purchase, Coding or

Configuration

VerificationSpecification

1. Initial Risk Assessment2. Risk-based decisions during Planning

3. Detailed / Functional Risk Assessment

4. Risk-based decisions during Test Planning/ Incident Management

5. Risk-based decisions during Operational Activities

GAMP5 was created by International Society of Pharmaceutical Engineers (ISPE) and has integrated industry standards such as ICH Q9 and ASTM e2500e.

We will mostly focus: on GAMP 5 Guideline

GAMP 5

Part 2: Evaluating Risk

Hazard =A potential source of harm

Risk= The combination of the probability of occurrence of a hazard

and the severity of the harm

Risk Assessment = A comprehensive evaluation of risks and

associated impacts

Risk Mitigation = Actions taken to reduce the impacts of risks

Risk Terminology

Quality Risk Management Process

Step 1: Perform Initial

Risk Assessment

and Determine System Impact

Step 2: Identify Functions with

Impact on Patient Safety,

Product Quality and

Data Integrity

Step 3: Perform

Functional Risk Assessments and Identify

Controls

Step 4: Implement and

Verify Appropriate

Controls

Step 5: Review Risks and Monitor Controls

The Risk Management Process may be scaled according to risk, complexity and novelty of individual systems.

Scalability

What are the hazards?

What is the harm?

What is the impact?

What is the probability of a failure?

What is the detectability of a failure?

What is the detectability of failure?

How will the risk be managed?

Applying Risk Management System–Understanding the Business Process

Quality Risk Management may include:

– Training

– Change Management

– Documentation

– Periodic Review

– Audit Management

– Etc.

We will focus on aspects directly impacting Validation Activities

Quality Risk Management/Risk-based Validation

1. Initial Risk Assessment

2. Risk-based decisions during Planning

3. Functional Risk Assessment

4. Risk-based decisions during Test Planning

5. Risk-based decisions during Operational Activities

6. Functional Risk Assessments in Change Control

7. Risk-based decisions when planning system retirement

Risk Management Throughout the System Life Cycle

Risk-Based Decision Making throughout Life Cycle

Planning Reporting

Purchase, Coding or

Configuration

VerificationSpecification

1. Initial Risk Assessment2. Risk-based decisions during Planning

3. Detailed / Functional Risk Assessment

4. Risk-based decisions during Test Planning/ Incident Management

5. Risk-based decisions during Operational Activities

Risk Assessment Roles and Responsibilities

Business/Process OwnerIdentify, Evaluate and Classify Risks

Technology Owner/ Subject Matter Expert

Provide information on how the software works and evaluate failures and impact

Quality Assurance

Evaluate Risksassociated with Regulatory Compliance and Company Policies

Performed at the beginning of the Project Phase.

Focuses on GxP risks and to business process

Can be performed simultaneously with URS

Provides early identification of key areas that require focus in subsequent stages, such as Critical Quality Attributes(CQA) and Critical Process Parameters.

Initial Risk Assessment

Critical Quality Attributes(CQA)

– Chemical, physical, biological and microbiological attributes that can be defined, measured, and continually monitored to ensure final product outputs remain within acceptable quality limits.

Critical Process Parameters(CPP)

– Operating parameters that are considered essential to maintaining product output within specified quality target guidelines.

Initial Risk Assessment-Definitions

Initial Risk Assessment should include a determination whether the system is GxP regulated.

If so, a list of applicable regulations must be provided.

Initial Risk Assessment – GxP Determination

Initial Risk Assessment must determine the overall impact that the system may have on patient safety, product quality and data integrity.

Need to Understand and take into account the complexity, novelty, and use of system.

Initial Risk Assessment – System Impact

Risk Assessment Steps

Identification Evaluation Classification Mitigation

Risk Assessment Activities

Determine and document the hazards associated with use of the system

Determine the severity of the identified hazards

Categorized the risks according to severity

Perform activities that reduce the severity of the risk or the likelihood of the risk

Risk Assessment Steps: Identification

Areas of Focus

– Which are the features or functions that would negatively impact?

Patient Safety

Product Quality

The integrity of associated data

What could go wrong with the system?

Risk Assessment Steps: Evaluation

What are the consequences of the hazard?

What is the likelihood the hazard will occur?

Risk= The combination of the probability of occurrence of a hazard and the severity of the harm

Probability is measured as Complexity

Severity is measured as Criticality

Risk Assessment Steps: Probability Evaluation

Complexity Level

Definition Examples

LowStandard, Non-configuredfunctions with off-the-shelf systems

• Test result report within an off-the-shelf laboratory system

MediumConfigured functions within off-the-shelf systems

• Calculation configured in a laboratory system

HighCustom developed functions within purchased or custom systems

• Custom report developed in Crystal Reports

Risk Assessment Steps: Severity Evaluation

Level Definition Examples

High Criticality

Direct Control of:• Manufacturing• Labeling• Distribution• Product Testing• Product ReleaseDirect Impact on:• Product Quality• Product Efficacy• Patient Safety• Personnel Safety

• ManufacturingControls

Medium Criticality

Indirect impact on patient safety, product quality, or the integrity of associated data.

• Calibration Tracking

Low Criticality

No impact on patient safety, product quality, or the integrity of associated data.

• Manufacturing Cost Reports

Part 3: How to Implement Risk-based Validation

Risk Based Approach to Validation TestingCriticality High High High Medium Medium Medium Low Low Low

Complexity High Medium Low High Medium Low High Medium Low

Path TestingAll Paths, Multiple Scenarios

All Paths, Single Scenario

All Paths, Single Scenario

All Paths, Single Scenario

All Paths, Single Scenario

Sampling* Sampling* Sampling* Sampling*

Boundary Testing ✓ ✓ ✓ ✓ Optional Optional Optional Optional Optional

Test Case degree of detail

Specific, Detailed

Specific, Detailed

Specific, Detailed

Medium Detailed

Medium Detailed Medium DetailedGeneral* General* General*

Test Data Similar to Production Data

✓ ✓ ✓ ✓ Optional Optional Optional Optional Optional

Testing Evidence

Screen Prints of Inputs and Outputs, Signed Protocol

Screen Prints of Inputs and Outputs, Signed Protocol

Screen Prints of Inputs and Outputs, Signed Protocol

Screen Prints of Outputs, Signed Protocol

Screen Prints of Outputs, Signed Protocol

Screen Prints of Outputs, Signed Protocol

Signed Protocol* Signed Protocol* Signed Protocol*

User AcceptanceTesting

✓ ✓ ✓ ✓ Optional Optional Optional Optional Optional✓Required * When documented testing is performed

Risk Based Approach to Validation Documentation

Criticality High High High Medium Medium Medium Low Low Low

Complexity High Medium Low High Medium Low High Medium Low

Change Request ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Risk Assessment ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

Project Plan ✓ ✓ ✓ ✓ ✓ Optional Optional Optional Optional

Validation Plan ✓ ✓ ✓ ✓ ✓ Optional Optional Optional Optional

User Requirements ✓ ✓ ✓ ✓ ✓ Optional Optional Optional Optional

FunctionalSpecification

✓

HighlyDetailed

✓

Highly Detailed✓

Highly Detailed✓

Medium Detail✓

Medium Detail✓

Medium DetailOptional

Low DetailOptional

Low DetailOptional

Low Detail

System Design Documents

✓ ✓ N/A ✓ ✓ N/A Optional Optional N/A

Test Plan ✓ ✓ ✓ Optional Optional Optional Optional Optional Optional✓Required * When documented testing is performed

Risk Based Approach to Validation Documentation

Criticality High High High Medium Medium Medium Low Low Low

Complexity High Medium Low High Medium Low High Medium Low

Validation Protocols (IQ, OQ, PQ)

✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional

Validation Incident Reports

✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional

Testing Summary Optional Optional Optional Optional Optional Optional Optional Optional Optional

Trace Matrix – URS to FS ✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional

Trace Matrix – FS to Design

✓ ✓ N/A ✓ Optional N/A Optional Optional N/A

Trace Matrix – FS to Val Test

✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional

Trace to ✓ ✓ N/A ✓ ✓ N/A Optional Optional N/A

Validation Summary ✓ ✓ ✓ Optional Optional Optional Optional Optional Optional

Deployment Plan ✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional

Data Conversion Plan ✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional✓Required * When documented testing is performed

Risk Based Approach to Validation Activities

Criticality High High High Medium Medium Medium Low Low Low

Complexity High Medium Low High Medium Low High Medium Low

Vendor Assessment N/A ✓ ✓ N/A ✓ ✓ N/A Optional Optional

Unit Testing ✓ N/A N/A ✓ N/A N/A Optional N/A N/A

Code Review ✓ N/A N/A ✓ N/A N/A Optional N/A N/A

Design Review ✓ ✓ Optional ✓ Optional Optional Optional Optional Optional

Part 11 Compliance Assessment

✓ ✓ ✓ ✓ ✓ ✓ N/A N/A N/A

Verification of SystemUse and Support SOPs**

✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional

✓Required * * System Use and Support SOPs include Back-up, Recovery, Security and Access, Training Requirements, Incident Handling, Change Management, Technical Operation and Routine Maintenance, User Operation, etc.

Risk Assessment Steps: Risk Mitigation through Procedures

SOP Risk Mitigation based of Risk Classification

Vendor Assessment Vendor Assessment Method

Audit Trail Audit Trail Requirement

Security and Access Password Change Frequency

Training Requirements Training Format

Back-up Backup Location and Backup Media Audit Frequency

Alternate RecordsAlternate Procedure Required or Second Person Verification Required

Incident Handling Resolution Priority

Risk Analysis Methods: GAMP

Risk Analysis Methods: FMECAFailure Mode Effect Criticality Analysis

C = P x S X N

– C: Criticality

– P:Probability of Failure Mode

– S:Severity

– N:Probability of Not Detecting Failure Mode or Effect

Greater Criticality = Higher Risk

FMECA makes you think

through the consequences of

failures

Part 4: Additional Techniques for Implementing Risk-based Validation

Avoid Duplication of testing

– Family Approach

– Using Vendor Documentation

Evaluating the need for screenshots and evidence

Eliminating unnecessary approvals

Avoid transcription errors

Risk Assessment Questionnaires for Initial Risk Assessment

Think Lean/Eliminate Waste

Appropriate when related but different processes are represented by one of the group which demonstrates the common properties or is worst case

Family Approach

Interactive Exercise:Family Approach

A

CB

E

D

Process 1

A

C

E

D

Process 2

A

CB

F

D

Process 3

What should we test?

Requirements:

For Lower/Medium Risk testing

Documented in SOPs, Planning documentation and/or Test Cases

Vendor documentation must follow SOPs specially Good Documentation Practices

Vendor Assessment

Must be communicated and coordinated with vendor

Using Vendor Documentation

Screenshots and evidence have are one of the most impactful factors in testing timeline.

Should take into consideration risk and criticality.

Evaluate the need for screenshots in intermediate steps.

Evaluate alternatives to screenshots such as audit trail or system exports/reports.

Evaluating the need for screenshots and evidence

Requirements:

May be used for configuration, installation or general operational verification activities

May be approved as SOPs or Test Cases

Need to have approved specifications and drawings to test against that are either attached or electronic version is traceable and available.

Benefits:

Eliminates pre-approvals

Eliminates transcription errors

Evaluate the need to provide detailed evidence and/or screenshots

Using Procedures and Pre-Approved Forms Low/Medium Risk Activities

Using Risk Assessment Questionnaires

High Risk

High Level changes can cause significant impact and disruption to the existing systems. High impact changes could affect patient safety, product quality or data integrity. These changes affect system functionality and the overall process of how the system is used.

High Level changes are scalable from a pre-approved specification and test scripts/protocols to a full validation life cycle implementation depending on the complexity and risk of the change.

A full implementation might require a separate risk assessment to determine the required validation deliverable(s).

In the case that a High Level change will not undergo a full validation life cycle implementation, a justification must be provided.

Using Risk Assessment Questionnaires

Medium Risk

Medium Level changes can cause moderate impact and disruption to the existing systems. Medium impact changes do not pose imminent risks to patient safety and product quality. These changes can affect system functionality and can affect the overall process of how the system is used.

Recommendation: Pre-approved test documentation (scripts or protocols). May be able to make decisions based on risk level for streamlining evidence.

Using Risk Assessment Questionnaires

Low Risk

Low Level changes can cause minor impact and disruption to the existing systems.

These changes can affect system functionality; however the functionality does not directly affect regulated functions.

Low level changes do not pose imminent risks to patient safety, product quality or data integrity.

Recommendation: Screen shots that show the change was made successfully and documentation must indicate that the change produced the specified result.

Infrastructure Classification:

Does the change impact infrastructure that supports regulated systems?– Yes. Complete Questionaire.

–No. N/A

Example: Performing a Risk Assessment Questionnaire

Change Classification: The change has no impact to existing applications or systems: 0

Class I: Non-regulated system that does not support any major process : 0

Class II: Non-regulated system that does support a major business process (business critical systems): 10

Class III: A system that affects non-GxP regulations: 10

Class IV: a GxP (GLP, GMP, GCP or regulatory affairs) system that is not directly involved in: 15

– Regulatory submissions

– Controlling regulated data in pre-clinical, clinical and manufacturing

– Control data that influences product release, recall or complaints

– Product and patient safety (Pharmacovigilance)

Class V: a GxP (GLP, GMP, GCP and regulatory affairs) system that is directly involved in: 30

– Regulatory submissions

– Controlling data in pre-clinical, clinical and manufacturing

– Control data that influences product release, recall or complaints

– Product and patient safety (Pharmacovigilance)

Example: Performing a Risk Assessment Questionnaire

GAMP5 Category Impact Assessment:

Category 1: Infrastructure Software:0

Category 3: Nonconfigured products: 5

Category 4: Configured products: 10

Category 5: Custom applications: 20

Example: Performing a Risk Assessment Questionnaire

Regulatory Change Impact:

The change does not directly impact any regulatory function, process or decision making process:0

The change directly impacts non-GxPRegulation function, process or decision making process: 15

The change directly impacts GxP function or a GxP decision making process: 30

Example: Performing a Risk Assessment Questionnaire

Security Impact Assessment:

The change does not affect system security (i.e. login, access control for the system): 0

The change affects system security:15

Example: Performing a Risk Assessment Questionnaire

Data Integrity Impact AssessmentThe change does not affect data

integrity (where and how the data is stored, manipulated or migrated from one location to another): 0

The change affects non GxP data integrity: 10

The change affects GxP data integrity: 30

Example: Performing a Risk Assessment Questionnaire

Functionality Impact Assessment:

The change does not add new function or impact existing functionalities (code change): 0

The change does introduce new functionalities or impact existing functionalities:10

Example: Performing a Risk Assessment Questionnaire

Multi-System Assessment

The change affects a single system:5

The change affects multiple systems:10

Example: Performing a Risk Assessment Questionnaire

Testing Environment Assessment

The change uses the Development Environment for testing: 0

The change uses a Validation / QA Environment for testing: -5

No testing is done prior to production implementation: 15

Example: Performing a Risk Assessment Questionnaire

Change Downtime Assessment

The change requires no downtime for implementation: 0

The change requires downtime for implementation: 5

Example: Performing a Risk Assessment Questionnaire

Change Method Familiarity AssessmentThe method and procedure to

implement the change is not new (approved procedure to make the change exists): -5

The method and procedure to implement the change is new (approved procedure to make the change does not exist): 5

Example: Performing a Risk Assessment Questionnaire

Change Downtime Assessment

The change requires no downtime for implementation: 0

The change requires downtime for implementation: 5

Example: Performing a Risk Assessment Questionnaire

Complexity Assessment

The change affects a single functionality: 0

The change affects multiple functionalities: 10

Example: Performing a Risk Assessment Questionnaire

Report Assessment

The change does not affect how data is presented on report(s): 0

The change affects how data is presented on report(s) which are not used for GxPdecision making processes: 5

The change affects how data is presented on report(s) which are used for GxPdecision making processes: 20

Example: Performing a Risk Assessment Questionnaire

Complexity Assessment

The change affects a single functionality: 0

The change affects multiple functionalities: 10

Example: Performing a Risk Assessment Questionnaire

Total Value= ???????

Risk Level = ???????

Low (value range: 0-45)

Medium (value range: 46 - 85)

High (value range: > 86)

Example: Performing a Risk Assessment Questionnaire

High Risk

High Level changes can cause significant impact and disruption to the existing systems. High impact changes could affect patient safety, product quality or data integrity. These changes affect system functionality and the overall process of how the system is used.

High Level changes are scalable from a pre-approved specification and test scripts/protocols to a full validation life cycle implementation depending on the complexity and risk of the change.

A full implementation might require a separate risk assessment to determine the required validation deliverable(s).

In the case that a High Level change will not undergo a full validation life cycle implementation, a justification must be provided.

Example: Performing a Risk Assessment Questionnaire

Questions????