REALIZING THE OPERATIONAL BENEFITS OF UTILIZING A …...Adequacy of CRA resourcing. 3%: 11%. 86%:...

REALIZING THE OPERATIONAL BENEFITSOF UTILIZING A THIRD-PARTY QUALITY

OVERSIGHT PROGRAMCASE STUDY

JOHN R. WILSON, JR., PH.D.SENIOR VICE PRESIDENT

OCTOBER 16, 2014EXL PHARMA CLINICAL QUALITY OVERSIGHT FORUM

WHY DID THE CASE EXIST?

• Global pharmaceutical firm needed the following• Assurance that their trial was being conducted

appropriately• Actionable data and information with an eye toward

early process improvement• Documentation of appropriate oversight of the CRO

• Why did they use a third party?• Sponsor didn’t have appropriate global resources• Sponsor didn’t have a Quality Oversight Plan• Sponsor knew what they didn’t know

2

DESCRIPTION OF CASE

• Global pharmaceutical company• Top 5 CRO• Phase III trial• 400 sites, 2500 subjects• 35 countries• Project start (planning and training): March 2013• Site visits and assessments: April 2013-January 2014• Inspection readiness: February 2014-July 2014

3

A WORD ABOUT FLOW…

• Let’s start with what was of optimal importance to the sponsor

• Then let’s discuss the process used to get there

4

ACTIONABLE DATA AND INFORMATION(THINK HOLISTICALLY)

Data and Information

High Level Overview of

Issues

Issues by Country Issues by Site Issues by CRA

5

HIGH LEVEL OVERVIEW OF ISSUES

Key Risks Identified by Co-Monitors: Cumulative Dashboard Red Yellow Green

Adequacy of CRA resourcing 3% 11% 86%

CRA knowledge and understanding of study product, therapeutic area, project goals, objectives, timelines 4% 9% 87%

CRA organization and efficiency - 7% 93%CRA adherence to the Clinical Management Plan 7% 25% 68%

CRA communication style appropriate and has established positive working relationships with site personnel 4% - 96%

CRA ability to identify problems and recommend solutions - 10% 90%CRA management of investigational product issues 5% 6% 89%CRA oversight and management of maintenance of site personnel blinding status 5% 2% 93%Adequacy of PI and site staff qualifications and training 9% 23% 68%Adequacy of site facilities 6% 18% 76%Site compliance with protocol 15% 39% 46%Site compliance with GCP 24% 33% 43%Site compliance with sponsor/IRB/EC safety reporting procedures 5% 13% 82%Site compliance with data entry and query resolution timelines 7% 16% 77%

Site compliance with investigational product management procedures 9% 18% 73%

Adequacy of PI oversight 4% 18% 78%

6

SCORES BY COUNTRY

The percentages in the pie charts below reflect yellow and red ratings as a percentage of total ratings by country, broken out by region. For example, 59% of Georgia’s ratings were yellow and red.

Australia, 25%

India, 54%

South Korea, 26%

Thailand, 25%

by Asia Pacific Country

Georgia, 59%

Romania, 30%

Spain, 29%

Germany, 29%

Estonia, 28%

Russia, 27%

Latvia, 27%

Poland, 27%

Hungary, 22%

Ukraine, 17%

Bulgaria, 15%

Moldova, 15%

Israel, 13%Croatia, 7%

Serbia, 6%

Lithuania, 0%Slovenia, 0%

by European Country

In this example, the majority of the issues occurred in Georgia, India, and Romania.

7

SCORES BY SITE

15

8

30

1315

8

41

0

5

10

15

20

25

30

35

100% 90% 80% 70% 60% 50% 40% 30%

Percentage Green Key Risk Factor Scores

Tota

l Num

ber o

f Site

s

Percentage of Green Key Risk Factor Scores by Site

As shown in the example above, 66 sites of the 94 sites that were assessed had at least 70% green scores on the key risk factors, with 15 of those sites at 100% green.

8

SCORES BY CRA

CRA 1 and CRA 2 had the greatest percentage of red and yellow scores. In this example, the issues that contributed to CRA 1 and CRA 2’s ratings were discrepancies in ISF, inadequate documentation, limited site staff experience, and numerous protocol deviations.

9

EARLY PROCESS IMPROVEMENT

• Assessors noted several issues that were evaluated for early process improvement (EPI) opportunities to mitigate risk and save time and/or cost for both current and future programs

• Specific examples included:• 100% source data verification in a RBM trial• Unacceptable freezer storage• Lack of adequate PI involvement

• High value to sponsor

10

RISK BASED MONITORING

CRAs conducting 100% Source Data Verification (SDV)

Discrepancy between CRO and sponsor monitoring manuals regarding SDV

Immediate CAPA

Develop training guides for future studies that provide step by step instructions on SDV

Implement into future studies to save time and cost

Issue Noted EPI Opportunity

11

INAPPROPRIATE EQUIPMENT

Unacceptable storage freezer was being used to store bacterial isolates

Temperature monitoring was not being conducted for the freezers

Immediately contacted other sites in the region to ensure the correct type of storage freezer was used to store bacterial isolates

Prevented loss of isolates due to inadequate storage

Issue Noted EPI Opportunity

12

PI INVOLVEMENT

A common trend of inadequate Pi involvement was detected early in the program.

Lack of PI involvement often led to other issues such as inexperienced study staff

For both current and future programs, retraining of the PI and study staff, or audits focused on PI involvement

Mitigated risk for the program

Issue Noted EPI Opportunity

13

HOW WERE THE DATA AND INFORMATIONDERIVED AND REPORTED?

Developed a comprehensive Program Plan• Risk Assessment• Assessment Plan

Organized a team of highly trained quality oversight assessors• Recruited from SCRA ranks, plus mentoring/management experience• Extensive project-specific training

Conducted Quality Oversight Visits• Data Gathering tools• Quality Oversight report templates

Provided data analytics and real-time reporting• Dashboards and reporting

14

COMPREHENSIVE PROGRAM PLAN

Analogous to a CMP for Quality Oversight

Cornerstone of project logistics

Flexible in terms of focus

Collaborative effort with sponsor

15

HIGHLY TRAINED ASSESSORS

• Global network assessors were approved by the sponsor, each with the following minimum education and professional qualifications:• Minimum eight years of monitoring experience, with

experience as a trainer, mentor, leader of CRAs, or GCP compliance responsibilities

• Qualified rating from a subject matter expert assessment, including GCP, monitoring practice, and quality oversight knowledge

• Based primarily in same country as sites visited• Same region as contingency

16

ASSESSMENT VISIT REPORT

• Assessment reports provided narrative descriptions regarding KPI/KQI observations – What is the meaning and

significance?– Isolated human error?

• Assessment reports also included CAPA recommendations when appropriate

QUALITY ANALYSIS AND REPORTING: SCORECARD ASSESSMENT TOOL

• Scorecard focused on key quality indicators, e.g.,– Timely reporting of SAEs– IP accountability– Protocol adherence

• Every question was scored Red, Yellow, or Green either on-site or centrally

QUALITY ANALYSIS AND REPORTING: QUALITY OVERSIGHT DATABASE

• Scorecards were entered into proprietary database

• Customized according to sponsor output requirements

• Tools and scoring provided powerful focus on key study performance parameters

HIGH LEVEL OVERVIEW OF ISSUES

Key Risks Identified by Co-Monitors: Cumulative Dashboard Red Yellow Green

Adequacy of CRA resourcing 3% 11% 86%

CRA knowledge and understanding of study product, therapeutic area, project goals, objectives, timelines 4% 9% 87%

CRA organization and efficiency - 7% 93%CRA adherence to the Clinical Management Plan 7% 25% 68%

CRA communication style appropriate and has established positive working relationships with site personnel 4% - 96%

CRA ability to identify problems and recommend solutions - 10% 90%CRA management of investigational product issues 5% 6% 89%CRA oversight and management of maintenance of site personnel blinding status 5% 2% 93%Adequacy of PI and site staff qualifications and training 9% 23% 68%Adequacy of site facilities 6% 18% 76%Site compliance with protocol 15% 39% 46%Site compliance with GCP 24% 33% 43%Site compliance with sponsor/IRB/EC safety reporting procedures 5% 13% 82%Site compliance with data entry and query resolution timelines 7% 16% 77%

Site compliance with investigational product management procedures 9% 18% 73%

Adequacy of PI oversight 4% 18% 78%

20

CONCLUSIONS

• Global pharmaceutical firm needed an objective third party for quality oversight• Large Phase III trial in 35 countries

• Primary deliverable was actionable data and information• Work performed via

• Comprehensive Program Plan• Highly trained assessors• Quality Oversight Visits• Data analytics and real-time reporting

• Significant value delivered in terms of • Early process improvement• Meeting regulatory agency expectations for oversight of vendors

JOHN R. WILSON, JR., PHD, MPH

SENIOR VICE PRESIDENT

JWILSON@BEAUFORTCRO.COM757-383-6004