Planning and experiences with

Risk Based Monitoring

M ONDAY 25 JAN 2016 16 .10 -16 .50

THOM AS COQUELLE , M SC PHARM

About me

MSc Pharm 2000

15 years working in Clinical Research

6-7 years working for Lundbeck in Global Study

Management latest as Principal Clinical Study

Manager

Recently started as International Project Leader in

TFS – Trial Form Support.

WHAT IS RISK BASED MONITORING?

No visibility

No

assurance

about the

quality

before the

harvest

Using a lot

of resources

Conventional

WHAT IS RISK BASED MONITORING

Look for

trends and

signals

Target our

actions and

use our

resources

wisely

Detailled

knowledge

about quality

before data

harvest

Risk Based Monitoring

Background

Mitchel et al. Aug 01, 2013 Applied Clinical Trials

(http://www.appliedclinicaltrialsonline.com/risk-based-approaches)

Defensive

practices (SDV,

many site visits)

for monitoring of

clinical trials have

been adopted.

fear based on

perception and interpretation of regulatory

requirements

Current processes play a

large role in high costs without

delivering reasonable

value

Propagation of practices based on paper CRFs without considering optimization of the processes

to take advantage of the technology provided by EDC

1996

institutionalization /prisoner of high cost, unproven

value, manpower-intensive practices that do little to

serve the interests of subject safety or quality data

TIME TO RETHINK

Mitchel et al. Aug 01, 2013 Applied Clinical Trials (http://www.appliedclinicaltrialsonline.com/risk-based-approaches)

Authorities reacted

the MHRA (Oct 2011), FDA (Aug 2013), EMA (Aug

2011) and NICE published guidance/reflection paper.

Trancelerate

TransCelerate BioPharma Inc. was launched in 2012 as a non-profit organization to improve the health of people around the world by simplifying and enhancing the research and development of innovative new therapies.

Members: Abbvie, Allergan, Amgen, Astellas, AstraZeneca, Boehringer-Ingelheim, BMS, EMD-Serono, GSK, J&J, Lilly, medgenics, Merck, Novo-Nordisk, Pfizer, Roche, Sanofi, Shionogi, UCB

Partners with Societies/Associations representing CROs, Clinical sites/Clinical Trial Standards and collaborating with FDA, EMA and PDMA.

Five strategic priorities:

Improve Site Investigator Experiences

Facilitating sharing of information

Enable harmonization of Clinical Trial Processes

Enhance sponsor efficiencies

Improve patient experience

http://www.transceleratebiopharmainc.com/

Trancelerate and Risk

Based Monitoring

The Risk Based Monitoring Initiative was

established in 2012 as one of the five initial goals

created by TransCelerate to drive efficient and

effective solutions into the R&D industry.

May 2013 Position paper: Risk-Based Monitoring

Methodology.

http://www.transceleratebiopharmainc.com/

Definitions

Critical data and processes: Data or processes should be prospectively be identified, that if performed inaccurately, not performed, or performed incorrectly, would threaten the protection of human subjects or the integrity of the study results.

Critical data and processes may include:

Data that support primary and key secondary objectives

Data critical to subject safety (e.g. serious adverse events, other events leading to discontinuation of treatment)

Processes that underpin subject safety and ethical treatment (e.g. seeking appropriate medical consultation or scheduling extra visits/procedures in the event of significant clinical or laboratory findings)

Processes that underpin data quality (e.g. blinding, referring events for adjudication, controlling inter-rater variability)

Food and Drug Administration Aug 2013: Oversight of Clinical Investigations - A Risk-Based Approach to Monitoring

Trancelerate Position Paper: Risk-Based Monitoring Methodology 4, FINAL 30May2013

Definitions

Source Data Verification (SDV) is the process by which

data within the CRF or other data collection systems

are compared to the original source of information

(and vice versa) to confirm that the data were

transcribed ccurately (i.e. data from source matches

data in the CRF or other system and vice versa)

Position Paper: Risk-Based Monitoring Methodology 4, FINAL 30May2013,

Definitions

Source Data Review (SDR) involves review of source

documentation to check quality of source, review

protocol compliance, ensure the Critical Processes and

source documentation (e.g. accurate, legible,

complete, timely, dated) are adequate, to ascertain

Investigator involvement and appropriate delegation,

and assess compliance to other areas (e.g. SOPs, ICH

GCPs). SDR is not a comparison of source data against

CRF data. SDR is necessary to evaluate areas that do

not have an associated data field in the CRF or system

available for more timely remote review.

Position Paper: Risk-Based Monitoring Methodology 4, FINAL 30May2013,

Definitions

Mandatory data (Informed consent, SAEs, safety scales)

ACTIVITIES RELATED TO RBM

Risk assessments and categorization

Study Risk Categorization

Request for proposal and work order

Critical Data and Processes

Targeted SDV module in electronic CRF

Risk Based Monitoring plan

Monitoring Plan

Risk Based Monitoring review meetings

CRO

Selection

FPFV

Risk Assessment and level

source data monitoring (example)

Study/program category

based on decision tree

SDV

mandatory

data

SDV

Critical

data

Source

Data

Review

A

Randomized Controlled

Trials (RCTs) with regulatory

purpose and/or studies with

high risk/impact

100% 30% 30%

B RCTs other than category A 100% 20% 20%

C Pragmatic or open-label

studies 100% 10% 10%

D

pharmacoepidemiological

NIS but with non-routine

diagnostic or monitoring

procedures

100% 0% 0%

(%of patients

pr. site)

Example category C study

in primary care

Alcohol Dependence

5 major countries in Europe (DE, UK, FR, IT, and ES)

Approx 70 primary care sites (GPs)

Target 475 patients (There should be 95 in each

country)

Medidata Rave as eCRF

SQM and TSDV modules

CATEGORY C

ON-SITE MONITORING EXPECTATIONS

Study/program category

based on decision tree

SDV

mandatory

data

SDV

Critical

data

Source

Data

Review

C Pragmatic or open-label

studies 100% 10% 10%

(%of patients

pr. site)

100% of patients/site:

Mandatory Data

(Informed consent, SAEs)

10% of patients/site: Critical data

(Primary endpoint, Key secondary

endpoints, Selection Criteria,

AE, IMP Compliance)

10% of patients/site reviewing compliance

to the protocol

Study team decided no overlap with SDV

patients Sudy team decided to have SDR for the first

patient

SDV algoritm programmed into

targeted SDV-module in eCRF

CRITICAL DATA AND PROCESSES

Type Data Process TSDV in eCRF

Primary /

secondary

Alcohol

consumption data TLFB interview TLFB calendar

Secondary Clinical Global

Impression score

Clinical Global

Impression rating

Clinical Global

Impression form

IMP compliance IMP data TLFB interview / IMP

accountability

TLFB calendar

Safety endpoint adverse events safety reporting AE forms

Selection criteria

Vital signs, medical

history, recent

medication, illegal

drugs

Vital signs and

collection of

demographics

Relevant forms

Secondary Liver function tests Blood samples to

central lab

Not applicable –

followed by

Medical monitor

Note: Medical monitor is reviewing all critical data (Centralized monitoring)

Monitoring activities

Monitoring activities planned pr. WORK ORDER

3 onsite visits pr. site

2 remote visits pr. site

Assumes 20% the sites would need for-cause visit

Monitoring activities planned in SMT

Medical monitoring / Safety Review

SQM (System tool)

RBM review

Monitoring visit schedule

in monitoring plan

Montoring visit type When Monitoring includes

Pre-check: remote Data Review

1st patient enrolled (V2 done)

Remote Data Review

On site MV#1

Within 2 weeks of 2nd

patient enrolled (V2 done)*

SDR 1st patient + part of tSDV 2nd patient. Focus to

review processes (Inclusion/Exclusion criteria, ICF,

AEs/SAEs, staff resources, IMP inventory and

accountability, lab kits, ISF, findings from data review, TLFB, CGI, PSS, PROs)

Remote MV#1

3rd patient completed

(Week 16 – visit 7 safety follow up)

Process review (see also 21.2/ 3.b from remote monitoring section)

On site MV#2 6th patient enrolled (V2

done)

remaining SDR first patient + remaining tSDV of

patient#2 in Critical Data + SDV on all mandatory

data for all patients. Review processes with site.

Reassess recruitment plan + confirm new IMP shipment)

Remote MV#2 9th patient enrolled (V2

done) Process review (see also 21.2/ 3.b from remote

monitoring section)

On site MV#3

After last patient

completed (including

follow-up period)

SDR on 11th patient + tSDV on critical data for new

randomly selected patient (if applicable) + process

review

*If the 2nd patient is not screened before the 1st patient is completed, then move on with the 1st MV after the 1st subject has been completed.

This at least will enable a full SDR of the first patient

Risk Based Monitoring plan

Definition of study specific risks (Risk indicators)

Critical Data Variables and Processes

Communication at on-site visits and remote

monitoring visit (CRO)

Remote Monitoring (Centralized Sponsor Review)

Regular RBM Review Meetings

Risk indicators

Risk indicators can be derived by operational or clinical data from EDC.

Risk indicators can also be obtained by selected reviewers (Medical monitor, data manager, CQA, safety reviewes, onsite monitors)

Risk Indicators should always be measured in red, yellow green either by thresholds calculations by the system or the the responsible reviewers make a professional evaluation of coloring red/green.

Primary endpoint

Data entry time

Health Economic data

Health Econo-

mic data

Risk indicators

# Risk Indicator Category Risk Level Risk Indicator (How to

measure): Threshold (Green,

Yellow, Red) Responsible

1 Visit Data to CRF Completion Date

High SQM tool: Visit to Data Entry Time

Automated System tool

Data Manager

2 Data Quality-Excessive Queries, low queries outliers

High

SQM tool Query Rate Automated System tool

Data Manager

3 Excessive Query Response Time

High SQM Tool Query Response Time

Automated System tool

Data Manager

4 Screen failure rate higher or lower than anticipated

High SQM Screen Failure Rate metric

Automated System tool

Data Manager

5 Outliers and trends in subject discontinuation

High

SQM: Early Termination Rate

Automated System tool

Data Manager

Qlikview MMSR/eCRF: Reason for Withdrawal

feedback from Medical Expert

Medical Expert

6 Outlier or trend in number of Adverse Events per subject per site

High Safety Advisor will provide reports from MMSR of AEs and SAEs.

feedback from Safety Advisor

Safety Advisor

7 Protocol Deviations High Monitor IPD frequency, Other PD trends (IPD, PD Log)

Feedback from CRO CRO

Additional risk indicators

added after 1st RBM

review # Risk Indicator

Category Risk Level

Risk Indicator

(How to

measure):

Threshold (Green,

Yellow, Red) Responsible

8 Data Manager High

Feedback on

data

entry/queries

Subjective

evaluation Data Manager

9 Medical Monitoring

feedback

High

Feedback from

Medical

Monitoring

Subjective

evaluation Medical Expert

10 MMSR feedback High Feedback from

Safety Review

Subjective

evaluation Safety Advisor

11 RQM feedback High

Feedback from

spotchecks/site

visits

Subjective

evaluation Quality Manager

12 CRO feedback High

General

feedback from

remote and

onsite monitoring

Subjective

evaluation CRO (Lead CRAs)

RBM plan:

Communication with

investigators

SDV/SDR Process reviews

on-site or remote

RBM plan:

Communication with

investigators

Process reviews

on-site or remote

Process review: Ask if the study supplies are adequate (IMP and other supplies such as lab kits) Ask for follow up on eTMF/iTMF

documents (for issues identified via site visits and spot check assessments) Discuss any findings from data review to assess issues/risks and evaluate action needed Obtain feedback from PI/Sis/raters,

whether there are questions/doubts about endpoint assessments other key assessments

Confirm Informed Consent Process

review: All subjects have signed and dated current IRB/IEC approved version of informed consent This has been obtained by all subjects prior to the start of data collection

Consent discussion and signature has been performed by delegated staff member listed on the site signature/responsibility log Subject received a full copy of the signed ICF

Discuss Inclusion/Exclusion criteria have been met by all subjects in the study Discuss procedire for AEs/SAEs reporting Discuss about IMP procedures for

IXRS, dispensing and accountability are being followed. Discuss the Source Data requirements are being followed. Confirm that site staff has adequate time and resources for the study. If new staff hired, discuss training

needs.

After SDV/SDR/Remote monitoring of systems and processes, the CRA should

follow up with the investigator(s)/ site staff to

review any questions and issues. Process reviews should

also be performed by the CRA in a proactive fashion on

topics/processes where there are no immediate issues

identified during the monitoring

to confirm that the investigator are following the correct

procedures.

RBM tracker – visual map

Data

Manager

Quality

Manager Safety

Advisor Medical

Expert CRO

Risk indicators

Int.Study

Manager

Action / Outcome

RBM tracker – example green

site #1

Data Manager Medical Expert Safety Advisor CRO

RBM tracker – example green site #2

Medical Expert Safety Advisor CRO CRO

Quality Manager

Trial Manager

RBM tracker – example red/yellow site #1

Data Manager Medical Expert Safety Advisor

CRO Medical Expert

RBM tracker – example red/yellow site #2

Safety

Advisor CRO CRO Quality

Manager Trial Manager

RBM tracker – risk evaluation site XX002

16-Oct: Site enrolled 5 patients in one day

RBM review

date

Risk indicators/feedback Recommended actions Outcome

12-nov-2014 9 IPDs identified, No alcohol data entered. Lack of

understanding procedure for conducting patient

interview (no telephone allowed), eligibility criteria

and screening procedures not followed, IMP

assignment before inclusion visit.

Retraining needed and PI need

to prioritize study.

For-cause visits and

accompanied visit with LM.

Email from ISM – IVRS turned off

i.e new necruitment not

allowed

Site difficult to reach

OSV planned 24/nov

Additional OSV 9/dec

performed

16-dec-2014 System triggers now red, Site visits 24/nov and 9/dec

No source documentation in Scales binder, Medical

History , Physical Exam. New Important PDs identified.

reconsent of patients not followed up. Lack of Source

documentation continues and there is huge data

entry backlog.

Request for Quality Manager

visit.

14-jan-2015 System triggers for data entry time red. Open queries

up to 11w old, still a lot data missing in eCRF – difficult

for medical monitor to monitor compliance. No

progress is seen,

CRO is trying to setup weekly

calls – but is difficult.

Two calls with PI in

Janury but there was

not enough time to

resolve outstanding

issues

11-feb-2015 System triggers data entry time and query response

time are still red. Missing data entry from many follow

up visits. Treatment compliance is good. Quality

manager feedback from Site Visit 22/jan is that there

is poor source data to non-existing - many data

entered directly into the eCRF, much later than the

visits toke place. PI instructed the patients to take one

table every day, which is not according to protocol.

IMP return was checked by CRA and not PI. PI expect

the CRA to be on site and resolve all issues. Difficulities

to use eCRF. No progress notes for CGI.

After this review SMT decided to

inform the site that no further

recruitment will be allowed.

ISM wrote email to the site.

Still a lot of outstanding issues

with this site.

Site quality was not

increased, however

the contibution of

poor data were

mitigated rapidly

WHAT ABOUT THE MONEY?

Cost savings RBM

Total direct fees Total direct fees

Total pass-throughs Total pass-throughs

Risk Based Monitoring Full Monitoring

= 291kEUR

= 62 kEUR

Total costs Total costs

= 722 kEUR

= 187 kEUR

= 353kEUR = 909 kEUR

Total cost saving*: 500 to 600 kEUR * Potential effect of changed monitoring strategy on site management costs and project management costs is not included

in calculation

Other CRO costs

Risk Based Monitoring costs

Additional cost (Full monitoring)

Current CRO costs (w/RBM): 1.499 kEUR

Additional monitoring: 500-600 kEUR

Total CRO costs (w. full m): 1.999-2099 kEUR

Estimated savings from

RBM:

25-29 %

Risk Based Monitoring – is it worth it

Thank you - questions