Linda Battiato, MSN, RN,OCNAssociate Administrator, Clinical Research Office

Indiana University Simon Cancer Center

Workload Assessment & Allocation:“How many coordinators does it take?”

Clinical Research Office Administrator’s Magic Wand

Oncology Clinical Trial Workload Metrics

Use• Budgetary planning, Cost recovery• Workload allocation, Staffing decisionsPotential Benefits• Efficient utilization of resources• Job satisfaction/Turnover• Identification of educational

needs/performance issues

Workload assessment and allocation methods

• “Survival Method” *• Specialization (i.e. disease, modality)• Accrual data• Number of trials• Encounters per month• Acuity, Complexity scoring• Points systems* Fowler et al, Research Practitioner, 4: 2, 2003

CCIRC Algorithm• Developed in 1992 by an NCI committee for use by

cooperative groups to determine budget allotments for member institutions

• Inappropriately adopted by some sites to assess productivity and make day to day staffing decisions

Points System Algorithm - example• 1.0 – Phase II or III subject enrollment; 0.5 –lab subject

enrollment; 0.25 for follow-up; 0.15 – subjects enrolled or f/u by affiliate

• A sum of 40 credits in 12 months = 1 FTE• Emphasis on enrollment

Factors Affecting Cancer Clinical Trial Workload: NCI of Canada Study

Objectives:• Calculate time unit measurements for trial

coordination and data management tasks• Identify and assess the effect of factors that

may impact task times• Use this information to develop a workload

measurement tool for sites to use in trial cost and feasibility estimations

Roche et al JCO 20:2, 2002

Factors Affecting Clinical Trial Workload: NCI of Canada Study

• 83 CRAs (24 centers) collected information on time spent on various trial related tasks

• Task categories assigned to each of 4 stages – Protocol management, Eligibility & Entry; Treatment; Follow-up/Final stage

• 41 tasks, 156 subtasks• Tasks recorded for 30 consecutive days• Data collection form – lab coat pocket sized –

task times noted and relevant subtasks ticked

Roche et al JCO, 20: 2, 2002

Factors Affecting Clinical Trial Workload: NCI of Canada Study - Results

• Phase and sponsor significant independent factors

• Industry trials – higher overall mean times• Local (IIT trials) time intensive during protocol

management stage (start up), but less time intensive at eligibility and entry stage

• Early phase studies more time intensive

Roche et al JCO 20:2, 2002

NCI Trial Complexity Elements & Scoring Model

• CTWG (cooperative group & NCI staff) “Operation Efficiency Initiative”

• Recommended aligning reimbursement for NCI –sponsored trials with trial complexity

• Goal – simple standardized model representative of trial elements deemed to involve increased effort =

Number of Study arms Feasibility and personnel impact

Informed consent process Data collection complexity

Registration/randomization steps

Follow-up requirement

Complexity of study tx. Ancillary studies

Length of study tx. Participant feasibility & Enrollment

Version 1.2 Last updated 4/16/2009

NCI Trial Complexity Elements & Scoring ModelStudy Element Standard (0 points) Moderate (1 point) High (2 points)

Study Arms 1 or 2 3 or 4 > 4

Registration or Randomization Steps

One Step Separate registration/randomizations; Simple Central Path review

Multiple steps; intraoperative randomizations; Complex central path review

Complexity of Investigational Treatment

Outpatient single modality

Combined modality treatments; Simple inpatient treatment

Tx. with potential for ↑toxicity – gene transfer, BMT

Etc. – 10 elements

Scoring:•Give each element 0, 1 or 2•Sum scores of all elements for a total complexity score

Version 1.2 Last updated 4/16/2009

Evaluating Trial Complexity: Ontario Protocol Assessment Level

Working Group; 17 Cancer Centers

Objectives: Develop a standard rating scale to evaluate clinical trial complexity; Apply scale to research sites

Guiding Principles: • Simple• Specific• Useful for determining capacity• Include all trial phases

Smuck et al. JOP 7:2; 2011

OPAL Pyramid Scale

Optional Elements0.5 – Inpatient0.5 – 100% source doc submission0.5 – multiple (> 3) surveys/questionnaires0.5 – Sponsor/CRO factor

Smuck et al. JOP 7:2; 2011

Pilot Project: Application of OPALProtocol WorkloadOPAL Score= Type of intervention + # of

incremental proceduresProtocol workload for administrative componentCase Workload# of active patients X OPAL score(half for patients in F/U)Total WorkloadCase workload + OPAL score

Smuck et al. JOP 7:2; 2011

Using OPAL ScoresTrial OPAL Active Cases F/U Cases Workload

A 4 5 1 26

B 5 12 0 65

C 7 3 1 31.5

D 5 4 0 25

Total --- 24 2 147.5

Smuck et al. JOP 7:2; 2011

Trial OPAL Active Cases F/U Cases Workload

E 6 3 1 27

F 7 2 0 21

G 2 14 0 30

H 6 5 1 39

Total --- 24 2 117

Measuring Workload in a CCOP• Trials assigned score of 1-4 according to estimated

workload• Research nurse workload scores calculated:# patients per trial X Trial Acuity Score• Scores for all protocols added together for individual

nurse acuity score• Research nurses received individual scores and group

average • Acuity scores calculated and reported monthly over

11 year period

Good et al JOP 9 (4) 2013

Trial Status

FY 1999 FY2009 Score(Low)

Year Score(High)

Year 11 year average

Treatment 22.8 37.6 (65% ↑)

19.3 2000 45.6 2008 30.6

Cancer Control

13.8 38.8 8.97 2001 69.8 2006 37.8

Off-Study 14.2 39.3 10.4 2001 39.3 2009 15.9

Good et al JCO 9(4) 2013

Results: Annual Average Acuity Scores by On and Off-Study Trial Status

•Acuity Scores increased 65% for treatment trials•Results, monitoring → staffing adjustments to decrease research nurse FTE acuity scores and number of patients per FTE•Scores used to guide hiring decisions

Effort Tracking Metrics for Budgeting and Workload Management: U of Michigan

• Home grown web based system – Research Effort Tracking Application (RETA) – 4 years of data

• 4 Major determinants of staff effort:1. Study Volume (actual accrual)2. Study Accrual rate3. Study enrollment status4. Study sponsor status

25% to 30% of a data manager’s effort-pre activation and/or closed to enrollment

James et al JNCCN 9:12, 2011

Effort Tracking Metrics for Budgeting and Workload Management: U of Michigan

• 1st year learning curve for data entry• 70-75% time allocated to study-related tasks• Data Management efforts on industry trials required 2.45 X more effort than IIT’s• Industry sponsored require ~30-50% more regulatory

effort• Inverse relationship for data management effort related to

accrual rate – Effort per accrual for studies enrolling 10+ pts/yr less than ½ the effort for studies that enroll 1-5/yr

James, et.al. JNCCN, 9:12, 2011

Nursing Effort in Hours on Active Industry Trials – Indiana University

5065

30

13

5463

70

49

31

74

105

67

5037

120

0

20

40

60

80

100

120

140

Orders Pt phonecalls

Pt Education Pt Consent Scheduling

August

September

January

Total clinical nursing hrs

Workload Assessment and Allocation

• Trial Complexity, Acuity important• Most studies focused on coordinators, nurses,

data managers• Lack of standardization• Lack of benchmarking data• Variability in staffing and business models• Workload management tools integrated with

CTMS ideal