Embed Size (px)
Citation preview
Isabel Rodriguez, PE, CSQEPresident IRP Consulting, LLC
Integrate Risk Management into Validation
Key Concepts
What is Risk-Based Validation?
How to Implement Risk-based Validation
Additional Techniques for Implementing Risk-based Validation
Session Outline
Part 1: Key Concepts
Because we have to
To meet Regulations and Guidelines
Because it is for Good Business
To save lives
Why do we Validate?
Key Sources of Regulations and Guidelines
FDA Guidance & Reference
Documents
PIC/S Guidance Documents
GAMP Guides (ISPE)
IEEE Guides
Regulations (Laws)
FDA 21 CFR…Eudralex Volume…
ICH Guidelines
Company Policies and Procedures
WHO Guidelines
21 CFR Part 11:
Requirement 11.10(a) - Validation
–Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.
Requirement 11.10 (i) System Checks
–Use of Operational System Checks to enforce permitted sequencing of steps and events, as appropriate
What does the FDA say about Validation?
How do we actually Validate?
Budget
Time
Team Morale
Focus on Documentation
The Real Cost of Validation
The objective of Validation is to achieve Compliantmanufacturing and computerized systems that are fit for intended use.
This can be achieved implementing Scalable and Risk Based Validation approach
Why we should Validate
How we should Validate
Principles of Quality Risk Management
Source:ICHQ9
The evaluation of The risk to quality should be based on scientific knowledge and ultimately link to the protection of the patient.
The level of effort, formality and documentation of the quality risk management process should be commensurate with the level of risk.
A systematic approach for the assessment, control, communication, and review of risks to patient safety, product quality, and data integrity
Enables effort to be focused on critical aspects in a controlled and justified manner
Quality Risk Management
Source:GAMP5
Identification and Management of Risks
Scaling of Life Cycle activities according to system impact and risks
Justification for Use of Supplier documentation
Therefore, reducing costs and effort based on scientific and documented system impact and risks
Benefits of Risk-Based Quality
ICH Q9: Quality Risk Management
ASTM E2500: Standard Guide for Specification, Design, and Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment
GAMP5: Science Based Risk Management
ISO 14971: Application of Risk Management to Medical Devices
Standards and Guidelines for Risk Management
ASTM E2500: Standard Guide for Specification, Design, and Verification of Pharmaceutical and Biopharmaceutical Manufacturing Systems and Equipment
ICH Q9: Quality Risk Management
Risk Review
Ris
kC
om
mu
nic
ati
on
Risk Assessment
Risk Evaluation
unacceptable
Risk Control
Risk Analysis
Risk Reduction
Risk Identification
Review Events
Risk Acceptance
Initiate
Quality Risk Management Process
Output / Result of the
Quality Risk Management Process
Risk
Ma
na
gem
entto
ols
GAMP5: A Risk-Based Approach to Compliant GxPComputerized Systems
Planning Reporting
Purchase, Coding or
Configuration
VerificationSpecification
1. Initial Risk Assessment2. Risk-based decisions during Planning
3. Detailed / Functional Risk Assessment
4. Risk-based decisions during Test Planning/ Incident Management
5. Risk-based decisions during Operational Activities
GAMP5 was created by International Society of Pharmaceutical Engineers (ISPE) and has integrated industry standards such as ICH Q9 and ASTM e2500e.
We will mostly focus: on GAMP 5 Guideline
GAMP 5
Part 2: Evaluating Risk
Hazard =A potential source of harm
Risk= The combination of the probability of occurrence of a hazard
and the severity of the harm
Risk Assessment = A comprehensive evaluation of risks and
associated impacts
Risk Mitigation = Actions taken to reduce the impacts of risks
Risk Terminology
Quality Risk Management Process
Step 1: Perform Initial
Risk Assessment
and Determine System Impact
Step 2: Identify Functions with
Impact on Patient Safety,
Product Quality and
Data Integrity
Step 3: Perform
Functional Risk Assessments and Identify
Controls
Step 4: Implement and
Verify Appropriate
Controls
Step 5: Review Risks and Monitor Controls
The Risk Management Process may be scaled according to risk, complexity and novelty of individual systems.
Scalability
What are the hazards?
What is the harm?
What is the impact?
What is the probability of a failure?
What is the detectability of a failure?
What is the detectability of failure?
How will the risk be managed?
Applying Risk Management System–Understanding the Business Process
Quality Risk Management may include:
– Training
– Change Management
– Documentation
– Periodic Review
– Audit Management
– Etc.
We will focus on aspects directly impacting Validation Activities
Quality Risk Management/Risk-based Validation
1. Initial Risk Assessment
2. Risk-based decisions during Planning
3. Functional Risk Assessment
4. Risk-based decisions during Test Planning
5. Risk-based decisions during Operational Activities
6. Functional Risk Assessments in Change Control
7. Risk-based decisions when planning system retirement
Risk Management Throughout the System Life Cycle
Risk-Based Decision Making throughout Life Cycle
Planning Reporting
Purchase, Coding or
Configuration
VerificationSpecification
1. Initial Risk Assessment2. Risk-based decisions during Planning
3. Detailed / Functional Risk Assessment
4. Risk-based decisions during Test Planning/ Incident Management
5. Risk-based decisions during Operational Activities
Risk Assessment Roles and Responsibilities
Business/Process OwnerIdentify, Evaluate and Classify Risks
Technology Owner/ Subject Matter Expert
Provide information on how the software works and evaluate failures and impact
Quality Assurance
Evaluate Risksassociated with Regulatory Compliance and Company Policies
Performed at the beginning of the Project Phase.
Focuses on GxP risks and to business process
Can be performed simultaneously with URS
Provides early identification of key areas that require focus in subsequent stages, such as Critical Quality Attributes(CQA) and Critical Process Parameters.
Initial Risk Assessment
Critical Quality Attributes(CQA)
– Chemical, physical, biological and microbiological attributes that can be defined, measured, and continually monitored to ensure final product outputs remain within acceptable quality limits.
Critical Process Parameters(CPP)
– Operating parameters that are considered essential to maintaining product output within specified quality target guidelines.
Initial Risk Assessment-Definitions
Initial Risk Assessment should include a determination whether the system is GxP regulated.
If so, a list of applicable regulations must be provided.
Initial Risk Assessment – GxP Determination
Initial Risk Assessment must determine the overall impact that the system may have on patient safety, product quality and data integrity.
Need to Understand and take into account the complexity, novelty, and use of system.
Initial Risk Assessment – System Impact
Risk Assessment Steps
Identification Evaluation Classification Mitigation
Risk Assessment Activities
Determine and document the hazards associated with use of the system
Determine the severity of the identified hazards
Categorized the risks according to severity
Perform activities that reduce the severity of the risk or the likelihood of the risk
Risk Assessment Steps: Identification
Areas of Focus
– Which are the features or functions that would negatively impact?
Patient Safety
Product Quality
The integrity of associated data
What could go wrong with the system?
Risk Assessment Steps: Evaluation
What are the consequences of the hazard?
What is the likelihood the hazard will occur?
Risk= The combination of the probability of occurrence of a hazard and the severity of the harm
Probability is measured as Complexity
Severity is measured as Criticality
Risk Assessment Steps: Probability Evaluation
Complexity Level
Definition Examples
LowStandard, Non-configuredfunctions with off-the-shelf systems
• Test result report within an off-the-shelf laboratory system
MediumConfigured functions within off-the-shelf systems
• Calculation configured in a laboratory system
HighCustom developed functions within purchased or custom systems
• Custom report developed in Crystal Reports
Risk Assessment Steps: Severity Evaluation
Level Definition Examples
High Criticality
Direct Control of:• Manufacturing• Labeling• Distribution• Product Testing• Product ReleaseDirect Impact on:• Product Quality• Product Efficacy• Patient Safety• Personnel Safety
• ManufacturingControls
Medium Criticality
Indirect impact on patient safety, product quality, or the integrity of associated data.
• Calibration Tracking
Low Criticality
No impact on patient safety, product quality, or the integrity of associated data.
• Manufacturing Cost Reports
Part 3: How to Implement Risk-based Validation
Risk Based Approach to Validation TestingCriticality High High High Medium Medium Medium Low Low Low
Complexity High Medium Low High Medium Low High Medium Low
Path TestingAll Paths, Multiple Scenarios
All Paths, Single Scenario
All Paths, Single Scenario
All Paths, Single Scenario
All Paths, Single Scenario
Sampling* Sampling* Sampling* Sampling*
Boundary Testing ✓ ✓ ✓ ✓ Optional Optional Optional Optional Optional
Test Case degree of detail
Specific, Detailed
Specific, Detailed
Specific, Detailed
Medium Detailed
Medium Detailed Medium DetailedGeneral* General* General*
Test Data Similar to Production Data
✓ ✓ ✓ ✓ Optional Optional Optional Optional Optional
Testing Evidence
Screen Prints of Inputs and Outputs, Signed Protocol
Screen Prints of Inputs and Outputs, Signed Protocol
Screen Prints of Inputs and Outputs, Signed Protocol
Screen Prints of Outputs, Signed Protocol
Screen Prints of Outputs, Signed Protocol
Screen Prints of Outputs, Signed Protocol
Signed Protocol* Signed Protocol* Signed Protocol*
User AcceptanceTesting
✓ ✓ ✓ ✓ Optional Optional Optional Optional Optional✓Required * When documented testing is performed
Risk Based Approach to Validation Documentation
Criticality High High High Medium Medium Medium Low Low Low
Complexity High Medium Low High Medium Low High Medium Low
Change Request ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Risk Assessment ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Project Plan ✓ ✓ ✓ ✓ ✓ Optional Optional Optional Optional
Validation Plan ✓ ✓ ✓ ✓ ✓ Optional Optional Optional Optional
User Requirements ✓ ✓ ✓ ✓ ✓ Optional Optional Optional Optional
FunctionalSpecification
✓
HighlyDetailed
✓
Highly Detailed✓
Highly Detailed✓
Medium Detail✓
Medium Detail✓
Medium DetailOptional
Low DetailOptional
Low DetailOptional
Low Detail
System Design Documents
✓ ✓ N/A ✓ ✓ N/A Optional Optional N/A
Test Plan ✓ ✓ ✓ Optional Optional Optional Optional Optional Optional✓Required * When documented testing is performed
Risk Based Approach to Validation Documentation
Criticality High High High Medium Medium Medium Low Low Low
Complexity High Medium Low High Medium Low High Medium Low
Validation Protocols (IQ, OQ, PQ)
✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional
Validation Incident Reports
✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional
Testing Summary Optional Optional Optional Optional Optional Optional Optional Optional Optional
Trace Matrix – URS to FS ✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional
Trace Matrix – FS to Design
✓ ✓ N/A ✓ Optional N/A Optional Optional N/A
Trace Matrix – FS to Val Test
✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional
Trace to ✓ ✓ N/A ✓ ✓ N/A Optional Optional N/A
Validation Summary ✓ ✓ ✓ Optional Optional Optional Optional Optional Optional
Deployment Plan ✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional
Data Conversion Plan ✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional✓Required * When documented testing is performed
Risk Based Approach to Validation Activities
Criticality High High High Medium Medium Medium Low Low Low
Complexity High Medium Low High Medium Low High Medium Low
Vendor Assessment N/A ✓ ✓ N/A ✓ ✓ N/A Optional Optional
Unit Testing ✓ N/A N/A ✓ N/A N/A Optional N/A N/A
Code Review ✓ N/A N/A ✓ N/A N/A Optional N/A N/A
Design Review ✓ ✓ Optional ✓ Optional Optional Optional Optional Optional
Part 11 Compliance Assessment
✓ ✓ ✓ ✓ ✓ ✓ N/A N/A N/A
Verification of SystemUse and Support SOPs**
✓ ✓ ✓ ✓ ✓ ✓ Optional Optional Optional
✓Required * * System Use and Support SOPs include Back-up, Recovery, Security and Access, Training Requirements, Incident Handling, Change Management, Technical Operation and Routine Maintenance, User Operation, etc.
Risk Assessment Steps: Risk Mitigation through Procedures
SOP Risk Mitigation based of Risk Classification
Vendor Assessment Vendor Assessment Method
Audit Trail Audit Trail Requirement
Security and Access Password Change Frequency
Training Requirements Training Format
Back-up Backup Location and Backup Media Audit Frequency
Alternate RecordsAlternate Procedure Required or Second Person Verification Required
Incident Handling Resolution Priority
Risk Analysis Methods: GAMP
Risk Analysis Methods: FMECAFailure Mode Effect Criticality Analysis
C = P x S X N
– C: Criticality
– P:Probability of Failure Mode
– S:Severity
– N:Probability of Not Detecting Failure Mode or Effect
Greater Criticality = Higher Risk
FMECA makes you think
through the consequences of
failures
Part 4: Additional Techniques for Implementing Risk-based Validation
Avoid Duplication of testing
– Family Approach
– Using Vendor Documentation
Evaluating the need for screenshots and evidence
Eliminating unnecessary approvals
Avoid transcription errors
Risk Assessment Questionnaires for Initial Risk Assessment
Think Lean/Eliminate Waste
Appropriate when related but different processes are represented by one of the group which demonstrates the common properties or is worst case
Family Approach
Interactive Exercise:Family Approach
A
CB
E
D
Process 1
A
C
E
D
Process 2
A
CB
F
D
Process 3
What should we test?
Requirements:
For Lower/Medium Risk testing
Documented in SOPs, Planning documentation and/or Test Cases
Vendor documentation must follow SOPs specially Good Documentation Practices
Vendor Assessment
Must be communicated and coordinated with vendor
Using Vendor Documentation
Screenshots and evidence have are one of the most impactful factors in testing timeline.
Should take into consideration risk and criticality.
Evaluate the need for screenshots in intermediate steps.
Evaluate alternatives to screenshots such as audit trail or system exports/reports.
Evaluating the need for screenshots and evidence
Requirements:
May be used for configuration, installation or general operational verification activities
May be approved as SOPs or Test Cases
Need to have approved specifications and drawings to test against that are either attached or electronic version is traceable and available.
Benefits:
Eliminates pre-approvals
Eliminates transcription errors
Evaluate the need to provide detailed evidence and/or screenshots
Using Procedures and Pre-Approved Forms Low/Medium Risk Activities
Using Risk Assessment Questionnaires
High Risk
High Level changes can cause significant impact and disruption to the existing systems. High impact changes could affect patient safety, product quality or data integrity. These changes affect system functionality and the overall process of how the system is used.
High Level changes are scalable from a pre-approved specification and test scripts/protocols to a full validation life cycle implementation depending on the complexity and risk of the change.
A full implementation might require a separate risk assessment to determine the required validation deliverable(s).
In the case that a High Level change will not undergo a full validation life cycle implementation, a justification must be provided.
Using Risk Assessment Questionnaires
Medium Risk
Medium Level changes can cause moderate impact and disruption to the existing systems. Medium impact changes do not pose imminent risks to patient safety and product quality. These changes can affect system functionality and can affect the overall process of how the system is used.
Recommendation: Pre-approved test documentation (scripts or protocols). May be able to make decisions based on risk level for streamlining evidence.
Using Risk Assessment Questionnaires
Low Risk
Low Level changes can cause minor impact and disruption to the existing systems.
These changes can affect system functionality; however the functionality does not directly affect regulated functions.
Low level changes do not pose imminent risks to patient safety, product quality or data integrity.
Recommendation: Screen shots that show the change was made successfully and documentation must indicate that the change produced the specified result.
Infrastructure Classification:
Does the change impact infrastructure that supports regulated systems?– Yes. Complete Questionaire.
–No. N/A
Example: Performing a Risk Assessment Questionnaire
Change Classification: The change has no impact to existing applications or systems: 0
Class I: Non-regulated system that does not support any major process : 0
Class II: Non-regulated system that does support a major business process (business critical systems): 10
Class III: A system that affects non-GxP regulations: 10
Class IV: a GxP (GLP, GMP, GCP or regulatory affairs) system that is not directly involved in: 15
– Regulatory submissions
– Controlling regulated data in pre-clinical, clinical and manufacturing
– Control data that influences product release, recall or complaints
– Product and patient safety (Pharmacovigilance)
Class V: a GxP (GLP, GMP, GCP and regulatory affairs) system that is directly involved in: 30
– Regulatory submissions
– Controlling data in pre-clinical, clinical and manufacturing
– Control data that influences product release, recall or complaints
– Product and patient safety (Pharmacovigilance)
Example: Performing a Risk Assessment Questionnaire
GAMP5 Category Impact Assessment:
Category 1: Infrastructure Software:0
Category 3: Nonconfigured products: 5
Category 4: Configured products: 10
Category 5: Custom applications: 20
Example: Performing a Risk Assessment Questionnaire
Regulatory Change Impact:
The change does not directly impact any regulatory function, process or decision making process:0
The change directly impacts non-GxPRegulation function, process or decision making process: 15
The change directly impacts GxP function or a GxP decision making process: 30
Example: Performing a Risk Assessment Questionnaire
Security Impact Assessment:
The change does not affect system security (i.e. login, access control for the system): 0
The change affects system security:15
Example: Performing a Risk Assessment Questionnaire
Data Integrity Impact AssessmentThe change does not affect data
integrity (where and how the data is stored, manipulated or migrated from one location to another): 0
The change affects non GxP data integrity: 10
The change affects GxP data integrity: 30
Example: Performing a Risk Assessment Questionnaire
Functionality Impact Assessment:
The change does not add new function or impact existing functionalities (code change): 0
The change does introduce new functionalities or impact existing functionalities:10
Example: Performing a Risk Assessment Questionnaire
Multi-System Assessment
The change affects a single system:5
The change affects multiple systems:10
Example: Performing a Risk Assessment Questionnaire
Testing Environment Assessment
The change uses the Development Environment for testing: 0
The change uses a Validation / QA Environment for testing: -5
No testing is done prior to production implementation: 15
Example: Performing a Risk Assessment Questionnaire
Change Downtime Assessment
The change requires no downtime for implementation: 0
The change requires downtime for implementation: 5
Example: Performing a Risk Assessment Questionnaire
Change Method Familiarity AssessmentThe method and procedure to
implement the change is not new (approved procedure to make the change exists): -5
The method and procedure to implement the change is new (approved procedure to make the change does not exist): 5
Example: Performing a Risk Assessment Questionnaire
Change Downtime Assessment
The change requires no downtime for implementation: 0
The change requires downtime for implementation: 5
Example: Performing a Risk Assessment Questionnaire
Complexity Assessment
The change affects a single functionality: 0
The change affects multiple functionalities: 10
Example: Performing a Risk Assessment Questionnaire
Report Assessment
The change does not affect how data is presented on report(s): 0
The change affects how data is presented on report(s) which are not used for GxPdecision making processes: 5
The change affects how data is presented on report(s) which are used for GxPdecision making processes: 20
Example: Performing a Risk Assessment Questionnaire
Complexity Assessment
The change affects a single functionality: 0
The change affects multiple functionalities: 10
Example: Performing a Risk Assessment Questionnaire
Total Value= ???????
Risk Level = ???????
Low (value range: 0-45)
Medium (value range: 46 - 85)
High (value range: > 86)
Example: Performing a Risk Assessment Questionnaire
High Risk
High Level changes can cause significant impact and disruption to the existing systems. High impact changes could affect patient safety, product quality or data integrity. These changes affect system functionality and the overall process of how the system is used.
High Level changes are scalable from a pre-approved specification and test scripts/protocols to a full validation life cycle implementation depending on the complexity and risk of the change.
A full implementation might require a separate risk assessment to determine the required validation deliverable(s).
In the case that a High Level change will not undergo a full validation life cycle implementation, a justification must be provided.
Example: Performing a Risk Assessment Questionnaire
Questions????
