Improve Data Integrity
Practices through Real-Life
Examples
IVT Data Integrity Validation
Conference
August 18, 2016
www.QACVConsulting.com 1
Objectives
www.QACVConsulting.com 2
• How are data integrity issues identified?
• What mechanisms can be established to
prevent data integrity issues?
• How to identify and implement mitigation
measures to reduce or eliminate risks?
Prevent & Detect Data Integrity
Issues
www.QACVConsulting.com 3
FDA expects that data be reliable and
accurate. CGMP regulations and guidance
allow for flexible and risk-based strategies to
prevent and detect data integrity issues.
Firms should implement meaningful and
effective strategies to manage their data
integrity risks based upon their process
understanding and knowledge management
of technologies and business models.
Current Regulatory Requirements
and Guidance
www.QACVConsulting.com 4
March 2015
• MHRA -GMP Data Integrity
Definitions and
Guidance for Industry
September 2015
• WHO -Guidance on Good Data and Record
Management Practices
April 2016
• FDA – Data Integrity
Guidance and
Compliance with CGMP
Current Regulatory Requirements
and Guidance
www.QACVConsulting.com 5
July 2016
• MHRA - GxPData Integrity
Definitions and
Guidance for Industry
August 2016
• PIC/S - Good Practices for
Data Management and Integrity in Regulated GMP/GDP
Environments
August 2016
• EMA – Data Integrity
Guidance Q&A
Data Integrity
www.QACVConsulting.com 6
What is data integrity and what
does it mean to you?
ALCOA
www.QACVConsulting.com 7
Data Integrity Completeness, consistency, and accuracy of
data.
Attributable
Legible
Contemporaneous
Original
Accurate
Enduring
Complete
Consistent
Retrievable
(or true copy)
www.QACVConsulting.com 8
Data Integrity
Data Availability
Data Retention
Data – Key Considerations
What is Data?
www.QACVConsulting.com 9
•Define data for each system, including each file type.
Chromatography Systems
• Raw Data File
• Integration Parameters
• Quantitation
• Sequence File
Other Laboratory Instruments
• Raw Data File
• Separate Audit Trail Log
MHRA Guidance - Data
www.QACVConsulting.com 10
Raw Data (Source Data) Original records, retained in the format in
which they were originally generated, or as a ‘true copy’. Raw data must be contemporaneously and accurately recorded by permanent means.
The definition of ‘original records’ currently varies across regulatory documents. By its nature, paper copies of raw data generated electronically cannot be considered as ‘raw data’.
MHRA Guidance - Data
www.QACVConsulting.com 11
Original record Data as the file or format in which it was originally
generated, preserving the integrity (accuracy, completeness, content and meaning) of the record, e.g. original paper record of manual observation, or electronic raw data file from a computerised system
True Copy: A copy of original information that been verified as an
exact (accurate and complete) copy having all of the same attributes and information as the original. A true copy may be retained in a different electronic file format to the original record, if required, but must retain the equivalent static/dynamic nature of the original record.
FDA Guidance
www.QACVConsulting.com 12
•It is not acceptable to record data on pieces of paper that will be discarded after the data are transcribed to a permanent laboratory notebook.
Similarly, it is not acceptable to store data electronically in temporary memory, in a manner that allows for manipulation, before creating a permanent record.
Electronic data that are automatically saved into temporary memory do not meet CGMP documentation or retention requirements.
8/23/2016 www.QACVConsulting.com 13
Data Lifecycle
Generate ModifyReview / Approve
UseRetain / Retrieve
Destroy
Data Integrity Considerations
www.QACVConsulting.com 14
8/23/2016 www.QACVConsulting.com 15
FDA Guidance
Key Points
• Define “data” and how it is used
• Static vs dynamic
• Metadata and audit trails
• Review of electronic records
• Validation
• Operation and use of systems
• Reporting and Retention
• Handling of Data Integrity Issues
FDA GuidanceStatic & Dynamic Records
www.QACVConsulting.com 16
Static
• Fixed-data document such as a paper record or an electronic image.
Dynamic
• Format that allows interaction between the user and the record content.
• Chromatographic Record
• Allows user to change the baseline.
• Reprocess chromatographic data.
• Resulting peaks may appear smaller or larger.
• Spreadsheet
• User modification of formulas or entries used to compute test results.
FDA GuidanceStatic & Dynamic Records
www.QACVConsulting.com 17
Acceptability of retaining paper printouts or static records instead of original electronic records from stand-alone computerized laboratory instruments?
Control strategies must ensure that original laboratory records, including paper and electronic records, are subject to second-person review to make certain that all test results are appropriately reported.
Generate ModifyReview / Approve
UseRetain / Retrieve
Destroy
Static / Dynamic Data
www.QACVConsulting.com 18
FDA Guidance
www.QACVConsulting.com 19
System Suitability: %RSD < 2.0%
%RSD 3.9% X
FDA GuidanceAudit Trails
www.QACVConsulting.com 20
Audit trails that capture changes to critical data be reviewed with each record and before final approval of the record.
FDA recommends routine scheduled audit trail review based on the complexity of the system and its intended use.
Generate ModifyReview / Approve
UseRetain / Retrieve
Destroy
FDA GuidanceAudit Trails - Finding
www.QACVConsulting.com 21
•Upon review of the audit trail configuration within LIMS, audit trails were turned off for many of the tables within the XXX Module.
Documentation was not available to justify why the audit trails for these tables was turned off. For example,
• The audit trail was not turned on for the XXX table, which is used when jobs, such as stability time pulls, are cancelled or suspended.
• This was observed by reviewing the audit trail configuration within the system.
• Audit trail configuration is documented within Design specification, XXX, YYY, version 2.0, approval date 11-Mar-2016.
8/23/2016 www.QACVConsulting.com 22
Data Lifecycle
Generate ModifyReview / Approve
UseRetain / Retrieve
Destroy
Specify
Design
Configure
Verify
Validation
www.QACVConsulting.com 23
“Standard Build”
• Date/time stamp controls
• Network backup
• Access controls
How can network controls impact
GxP Processes?
Non-Standard
Build
X
• Audit logs not backed up
• User access not controlled
Data Integrity
www.QACVConsulting.com 24
ELISA Data Process Flow
Data
Flow
LIMSELISA SOftware Company Network
Protocol(.xyz file)
Sample Analysis
Setup Run
Data File(.db file)
Export .txt Data File
Secure Network Location
Secure Network Location
Save .db Data File
.db File backed up
.txt File backed up
LIMS Database
Import .txt file to LIMS
.db File archived
Secure Network Location
Backup Location
www.QACVConsulting.com 25
Data Lifecycle
Generate ModifyReview / Approve
UseRetain / Retrieve
Destroy
Specify
Design
Configure
Verify
Validation
26
Validation
• Comply with EU GMP Annex 11.
• Requires an understanding of the computerisedsystem's function within a process.
• The acceptance of vendor-supplied validation data in isolation of system configuration and intended use is not acceptable.
• In isolation from the intended process, vendor testing is likely to be limited to functional verification only, and may not fulfil the requirements for performance qualification.
MHRA GuidanceTerms and Definitions
www.QACVConsulting.com 26
FDA GuidanceWorkflows
www.QACVConsulting.com 27
Validation of “workflows”
• A workflow, such as creation of an electronic master production and control record, is an intended use of a computer system to be checked through validation.
• If you validate the computer system, but you do not validate it for its intended use, you cannot know if your workflow runs correctly.
28
Data Integrity Vendor Supplied Validation Documentation
www.QACVConsulting.com 28
Approval of Records
• Requested configuration specification – none available
• Internal assessment of Stability LIMS
• Vendor Supplied Documentation Provided
• User Requirements Specification
• User Acceptance Test
• URS – Included statement “the system has a ‘configurable option’
for ….. electronic signatures”.
• Reviewed configuration within system – esigs turned off
• Reviewed UAT documentation – esigs functionality passed
• Record integrity issue – lack of approved stability protocols
• First step of test – turn on esig functionality
• Last step of test – turn off esig functionality
• Requested system demonstration – approval by pressing approve button
www.QACVConsulting.com 29
FDA GuidanceControls to manage risks – understand your
data flows
Material
Weight
FDA GuidanceElectronic Signature
www.QACVConsulting.com 30
ALCOAAttributable
EDC System
• How long of a delay? 2-3 hours, sometimes next day
• Issue – system response is slow at times
• Cause – batch jobs being run cause slow system response
• Type of batch jobs? Principle Investigator approval of eCRFS
• What date/time is applied for electronic signature?
• Answer: When batch is run.
• Data integrity issue – date and time stamp is not the same as
when PI entered electronic signature user ID and password.
31
MHRA GuidanceTerms and Definitions
31 www.QACVConsulting.com 31
32
MHRA GuidanceTerms and Definitions
32 www.QACVConsulting.com 32
Data Lifecycle
www.QACVConsulting.com 33
The eClinical Forum and PhRMA EDC Task Group
Electronic Signature Issue
www.QACVConsulting.com 34
EDC Issue
• Principal Investigator eCRF Signature Page signed by PI
• System demonstration requested
• Review administrative functions
• No Principal Investigator Role in System
• No electronic signature functionality within system
• Principal Investigator logged into system one time in 2009
• Study was initiated in 2014
• Requested audit trail for eCRF Page
• Principal Investigator is responsible for accuracy of data
• Data Entry personnel entered PIs signature onto page
• Requested audit trail for Principal Investigator
• GCP compliance and data integrity issue
Data Integrity Issue
www.QACVConsulting.com 35
EDC Issue
• After Source Document Verification and PI sign off locking
data additional medications or AE follow-ups were able to be
added to system as child records
• EDC system included “parent-child” records
• Used for concomitant medications and adverse events
• No data had been submitted to FDA
• System design and functionality issue
• Required unlocking of database and additional SDV and PI signoff
• Impacted accuracy and completeness of clinical data
• GCP compliance and data integrity issue
Data Migration - Data Integrity
Issue
www.QACVConsulting.com 36
Safety Systems
• Notified customers of upgrade for drug safety system to
newer version of system.
• Required migration of data from old version to new version
• Outsourced Pharmacovigilance Vendor
• Vendor halted migration process
• Data was lost during migration process
• Data integrity issue for safety data
Data Changes - Data Integrity
Issue
www.QACVConsulting.com 37
REMS Issue
• REMS system was not mature to include data change
capabilities within application.
• REMS vendor received many requests for data changes from
pharmacies, physicians, etc.
• Help Desk tickets were not approved.
• Data changes had to be made through database backend.
• Pharmacies and physicians were instructed to submit help desk tickets to
REMS vendor to request data changes.
• No justification or rationale was provided.
• Over 5000 requests for data changes in 7 years
• Data Integrity Issue
FDA Guidance
www.QACVConsulting.com 38
•Backup and
Restore
Disaster Recovery
Finding
• The site DRP plan has not been tested.
• Disaster recovery plans do not guarantee recovery time objectives (RTO) that allow compliance with FDA requirements, such as safety reporting or recall.
• The DRP allows for a 10 day RTO, which does not meet FDA requirements for safety reporting and would not allow for timely recall processes.
Generate ModifyReview / Approve
UseRetain / Retrieve
Destroy
www.QACVConsulting.com 39
Backup and Restore
• Backup and Restore SOP
• Failed backup required a help desk ticket
• 3 failed backups require notification to Director of
Infrastructure Technology
• 5 failed backups require notification to CTO
• Bioanalytical Laboratory
• No help desk tickets generated for month of June
• No successful backups for June or first two weeks of July
• Director and CTO were not aware of failuers
• Potential Data Integrity Issue
• Requested backup logs for three months
Data Integrity Issue
• Director and CTO were not notified
FDA MedSun Reports
The time on the meter read ~3 hours ahead of
time. When the physician read the results, the
results indicated the glucose levels had gone
down (due to the incorrect reading on the
meter). The meter clock stops when the meter
is turned off. When the meter is turned on
again the clock starts, giving an incorrect time
stamp.
www.QACVConsulting.com 40
FDA MedSun ReportsThe system locks up during patient care documentation at
the "save" function. The system locks up only at the save
function, allowing RNs to continue to document critical
patient data only to discover at the end of the case, that
no data will save. In order to get out of the patient chart,
the computer must be re-booted. All patient
documentation is lost. Nurses are forced to do their best
guess as to surgical times, implants, medications, etc
and re-chart the entire document.
This ongoing software program leaves us in a difficult
position - due to the risk of data loss.
www.QACVConsulting.com 41
Summary
www.QACVConsulting.com 42
• Understand data flows and data lifecycle
processes.
• Identify and mitigate potential data
integrity risks.
• Ensure adequate and robust validation,
security, and other data integrity controls
are established to minimize data integrity
risk.
Chris Wubbolt
QACV Consulting, LLC
www.QACVConsulting.com
Telephone: 610-442-2250
E-mail: [email protected]
43
Contact Information
www.QACVConsulting.com