March 13, 2007: I. SimInformatics for Clinical Trials Epi 206 – Medical Informatics Ida Sim, MD, PhD March 13, 2007 Division of General Internal Medicine,

March 13, 2007: I. Sim Informatics for Clinical TrialsEpi 206 – Medical Informatics

Ida Sim, MD, PhD

March 13, 2007

Division of General Internal Medicine, and Center for Clinical and Translational Informatics

UCSF

Informatics for Clinical Trials

Copyright Ida Sim, 2007. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.


Outline

• Clinical Trial Informatics– clinical trial management systems (CTMS)

– NCI/NIH vision

• Velos at UCSF Cancer Center• Running a Trial• Reporting Trials• Summary


Biomedical Informatics Spheres

Administrative Clinical Care Research

ClinicalBilling

Physical Networking

Standard Communications Protocols (e.g., HL-7)

Standard Vocabulary

PracticeManagement

Systems

Medical BusinessData Model

ElectronicMedicalRecord

Clinical CareData Model

??

??


CTMS

Administrative Clinical Care Research

ClinicalBilling

Physical Networking

Standard Communications Protocols (e.g., HL-7)

Standard Vocabulary

PracticeManagement

Systems

ElectronicMedicalRecord

Clinical Trial Management

Systems

Medical BusinessData Model

Clinical CareData Model

Clinical StudyData Models


The Clinical Trial Cycle

New Ideas

Protocol & Funding

Findings

Approval &Preparation

Design Trial

Activate TrialConductTrial

UtilizeResults •trial simulators

•trial costing•protocol authoring

•IRB approval•CRF design

•data management•remote data entry•GCP compliance

•data analysis•reporting


Anything Missing?

for patientcare & policy

for clinical researchNew Ideas

Protocol & Funding

Findings


DesignStudy

ActivateStudy

ConductStudy

UtilizeResults for basic research

ClinicalCare


Nothing Talks to Nothing...

New Ideas

Protocol & Funding

Findings


Design Trial

Activate TrialConductTrial

UtilizeResults •trial simulators

•trial costing•protocol authoring

•IRB approval•CRF design

•data management•remote data entry•GCP compliance

•data analysis•reporting


Interoperability Review

• in·ter·op·er·a·bil·i·ty– ability of a system...to use the parts or equipment

of another systemSource: Merriam-Webster web site

• interoperability– ability of two or more systems or components to

exchange information and to use the information that has been exchanged.

Source: IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries, IEEE, 1990]

• Syntactic -- grammar or composition of what is said• Semantic -- meaning of what is said


2 Routes to Interoperability

• “Microsoft” approach– one end-to-end system from trial inception to end

– used by all PIs, industry, CROs, worldwide

• Modular, component-based, interfaces approach– define common terms, models, interchange

protocols

– provide software components for assembly

– provide way to “certify” compatibility of systems


NCI CTMS in caBIG

Clinical Trial Management Infrastructure

Adverse Event Reporting

Study Calendar

Routine Data Exchange Financial Billing

Structured Protocol

Representation

Laboratory Interfaces

Clinical Trials DatabaseSystems

Interoperation

CRF Standardization

InvestigatorAnd

Site Credentialing


NCI/NIH “xx”BIG Vision

caBIGTM Architecture WorkspacecaBIGTM Architecture Workspace

caBIGTM Vocabularies and Common Data Elements WorkspacecaBIGTM Vocabularies and Common Data Elements Workspace

Strategic Planning

Workspace

Strategic Planning

Workspace

Training Workspace

Training Workspace

Integrative Cancer

Research

Workspace

Integrative Cancer

Research

Workspace

In Vivo Imaging

Workspace

In Vivo Imaging

Workspace

Tissue Banks &

Pathology Tools

Workspace

Tissue Banks &

Pathology Tools

Workspace

Data Sharing & Intellectual

Capital Workspace

Data Sharing & Intellectual

Capital Workspace

Clinical Trials

Management Systems

Workspace

Clinical Trials

Management Systems

Workspace


Critical Components

• Terminologies– base terms used to describe biomedical concepts

• e.g., SNOMED, NCI Thesaurus

• Common Data Elements– clinicallly-agreed upon data items for research

• e.g., “menopause” defined a certain way

• Common data model of study protocol– study information: e.g., eligibility criteria, treatment, outcomes

• CTOM, SDTM, BRIDG, etc. etc.

• Common interchange standards– so design, monitoring, reporting systems, etc. can talk


Outline


– NCI/NIH vision



Commercial CTMSs• Functions

– document management (protocol, CRFs (Case Report Forms))

– finances, IRB– study calendar (what to do to whom when)– data management and analysis– reporting

• Examples– proprietary

• OracleClinical, C3D, Velos, etc. etc.

– open source• OpenClinica


Cancer Center Needs

• Reports need to continue as an NCI-designated Cancer Center– e.g., trials initiated, pt demographics

• UCSF CC needed good data for reports– bought Velos eResearch in 2003 for reporting

– required all CC investigators to use Velos• licenses issued to PIs in CC and some in GCRCs


What Gets into Velos?

• Study summary data, as free text– administrative (PI, contact info, etc)– protocol (treatments, outcomes, etc)– IRB-related

• submitted protocol, approval status, current protocol• tied into Cancer Center PRS review, but not CHR

• Patient data– PI can define CRFs, data entered would be stored

in a Velos database• basic report functions• export to SAS or Excel for analysis


How is it Working?• Adoption

– rocky initially, better since integrated with PRS • For NCI reporting, successful• For data hosting and analysis

– just started this in 2007(!)– “easy to get data in, hard to get data out”

• biostatisticians are not too happy

• underlying data schema and access are opaque, by design

• less reponsive/customizable to reporting and analysis needs

• Not used for finances, study calendar, standardization of terms, etc.


Outline


– NCI/NIH vision



Clinical Trial Tasks

• Investigator credentialing• Study design

– find previous trials

– design protocol, CRFs

• IRB approval• Trial registration• Participant recruitment• Randomization• Run study

– distribute CRFs, collect data per study calendar

• Analyze


Investigator Credentialing

• For FDA-related trials, Form 1572 to be filed every year, multiple other layers of credentialing

• Today– mostly paper-based

– no coordination

• Tomorrow– one-stop, one-shop

– NCI’s caBIG priority task

– CRIX’s Firebird, collaboration of FDA, NCI, pharma








• Analyze


Find Previous Trials• Trial registration

– to be considered for publication in NEJM, JAMA, etc., all “hypothesis-testing” trials must be registered at inception

• 20-item Trial Registration Data Set http://www.who.int/ictrp

• in US, use ClinicalTrials.gov

– WHO Search Portal to all national/intl registers in beta• Unclear what proportion of trials are being registered

– no penalties for disobeying FDAMA law “requiring” registration of all “life-threatening” condition trials

– pending Enzi-Kennedy act will require registration and reporting of all trials

• Registration entires not routinely linked to results


Other Sources• Published

– Medline

– Cochrane Controlled Trials Register• ~328,000 records of controlled trials

• manual logging of CCTs by hand searching journals

• accessible from UCSF machines (IP address) only

• Commercial: mostly for patient recruitment– www.centerwatch.com

– www.ClinicalTrialFinder.com

– www.controlled-trials.com

– www.clinicaltrials.com

– etc., etc., etc.

http://www.updateusa.com/clib/CLIBINET.EXE?A=1&U=&P=

http://ww.centerwatch.com/

http://www.clinicaltrialfinder.com/

http://www.controlled-trials.com/

http://www.clinicaltrials.com/trials/trials.asp








• Analyze


Protocol Design Assistance• Simluations

– protocol simulation• ePCRN: apply eligibility criteria to research network to estimate

potential recruitment pool

• some pharma companies use Markov models to simulate trial outcomes, couple with value-of-information theory to make go no-go decisions

– pharmacokinetics simulation available• e.g., PharSight

• Critiquing of study design– automated help is topic of informatics research (e.g.,

Design-a-Trial)– at UCSF, go to CTSI’s BREAD for advice


Data Elements• To ensure sharable data, research data variables

should be standardized– at conceptual level, terms should be from a controlled

vocabulary (e.g., SNOMED)• e.g., SNOMED term menopause, available from NCI’s

Enterprise Vocabulary System (EVS)

– at study/scientific level, common variables should be used (e.g., NCI’s CDEs)

• e.g., menopause with full clinical definition, to be used in all breast cancer trials

• at caDSR (Data Standards Repository), you can– use existing CDEs that others have defined– you can define new CDEs using terms from EVS


Case Report Forms

• Why reinvent the wheel for common forms?• caDSR has a Forms Builder

– pull together CDEs into CRFs• can download as HTML, XML, Excel, Word, PDF

• Homework #5 due Friday, March 23– http://cdebrowser.nci.nih.gov/CDEBrowser/

• NCI now building library of CRFs – e.g., Demographics CRF built from CDEs

• PDF, Word, etc. or directly to CTMS for direct data entry

• Velos– can design CRFs in Velos, with direct access to NCI’s CDEs– if caBIG compatible, could in future access NCI’s CRF

library directly

http://cdebrowser.nci.nih.gov/CDEBrowser/

http://cdebrowser.nci.nih.gov/CDEBrowser/








• Analyze


IRB Approval

• IRBs very fragmented, overwhelmed– institutional (e.g., university, community hosp)– for-profit (e.g., Western IRB certifies majority of

commerical trials)• Large overlap of jurisdiction

– multi-site trials, international trials, etc.• Some exploration of new IRB models

– e.g., one NIH IRB, regional IRBs, etc• For now, no coordination, good luck...








• Analyze


Trial Registration

• By “responsible registrant”– PI or sponsor (e.g., pharma) or NIH Project Officer

• Today– diffuse responsibility, risk

of duplicate registration, double-entry required

– http://prsinfo.clinicaltrials.gov/ for US

• Tomorrow– registration “falls out” of CTMS

trial design and/or IRB approval modules

– trial gets a UTRN (Universal Trial Reference Number)

– indexed in WHO Register Network

http://prsinfo.clinicaltrials.gov/

http://prsinfo.clinicaltrials.gov/








• Analyze


Eligibility Determination

• Study enrollment is big bottleneck• Eligible patients: patients whose characteristics

match with eligibility criteria• For computerized matching, need to have

computer-interpretable descriptions of– patient characteristics

– the eligibility criteria

Match Eligible Patients

EHR

Eligibility Rule


Eligibility Example• Eligibility criterion: women who are 2 or fewer years

post-menopause, as defined in NCI’s Common Data Elements set

• Allowed values:Above categories not applicable AND Age < 50

Above categories not applicable AND Age >=50

Post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy and not currently receiving therapy with LH-RH analogs [eg. Zolades])

Post (Prior bilateral ovariectomy, OR >12 mo since LMP with no prior hysterectomy)

Pre (<6 mo since LMP AND no prior bilateral ovariectomy, AND not on estrogen replacement)


EHR Data Needed

• Gender• Age• Time since LMP, whether

– 6 or fewer months, or 12 or more months

• Past surgical history– bilateral ovariectomy and/or hysterectomy

• Therapy– LH-RH analogs, or

– estrogen replacement


Computable Eligibility Rules

• Logical rules– (Prior bilateral ovariectomy) OR

(>12 mo since LMP AND no prior hysterectomy)• Temporal constraints

– greater than 12 months since LMP...– representing time requires second-order logic

• Can do simple cases with database rules and triggers• More robust solution not yet available

– CDISC, HL7, NCI, Trial Bank Project all working on related standards

• syntactic: how to express logical and temporal meaning• clinical: standard eligibility rules for particular domains (e.g., breast

cancer)


Need to Query EHR

itsa

coordcenter with DSS

ucsf.edu

LAN

Site 1/GE EHR Site 2/Epic EHR

1. Is C. Jones eligible for this trial?

2. …Exclusion Rule #3: Does C. Jones have a history of congestive heart failure?

3. Return Yes if “congestive heart failure” is in Past Medical History

HL-7 communications protocol

4. If Yes to history of CHF, C. Jones is not eligible


Promising, but...

• Coding of eligibility rules is difficult• Richly detailed, coded EHR not widely

available– need a common EHR model (e.g., HL7 CDA v 2)

• At present, can only expect computer to suggest potential subjects. EHR can– prompt MD in real-time to refer patient to study, or– periodically batch notify MD of eligible patients, or– send letter of solicitation to patients

• Similar problems bedevil automated identification of guideline eligibility








• Analyze


• Requirements– a web-based data collection form to collect patient

information– programs to verify eligibility and randomize patient– program to generate a response to the enroller

– security, privacy, and backup provisions• Commercial systems available for this

Web-based Randomization

Project Central Enroller

patient info

randomization results








• Analyze


Study Calendar

• Protocol-driven care: what to do to whom when– alert and reminder functions

– should be integrated to protocol and EHR

• Available in some CTMS products– e.g., Velos, OracleClinical

• Same informatics issues for supporting guideline-based care


Distributing CRFs• e.g., from a project website

– latest protocol, IRB approvals, CRFs, listservs, etc.• Requirements

– a web server• internal to UCSF

– free web hosting by UCSF OAAIS– free hosting by School of Medicine if using SOM templates

» if your own design, $22-89/mo

• external web hosting service (see http://www.cnet.com/)– website design

• School of Medicine: 80 hours free design if use their templates– if using your own design, $62/hr for design services

• Note recent design guidelines from UCSF Pub Affairs


CRF Distribution

itsa

smoking

ucsf.edu

KaiserOakland

KaiserSanta Clara

Internet

at homedial-in to itsa.ucsf.edu via modem

pacbell.net

aol.com

LAN

KaiserSan Diego

• “smoking.ucsf.edu” “serves” forms such ashttp://smoking.ucsf.edu/case-form.pdf for printing out


Electronic Data Capture (1)

• Capturing data from manual entry into CRFs– “active” CRFs must be served over the web

• e.g., basic HTML to fancy ActiveX, Ajax, etc. input forms

– need security mechanisms• e.g., user login, timeouts, audit trails

– need data validation checks built into forms entry• e.g., age between 0 and 120

– input forms must send data directly to a database• needs ODBC (Open DataBase Connectivity) drivers, etc.

• e.g., using Access Visual Basic, Access and FrontPage, any other back end database (Oracle, mySQL, etc.)


Capturing Smoking CRF Data

itsa

smoking

ucsf.edu

KaiserOakland

KaiserSanta Clara

Internet

at homedial-in to itsa.ucsf.edu via modem

pacbell.net

aol.com

LAN

KaiserSan Diego

• “smoking.ucsf.edu” “serves” online entry formssuch as http://smoking.ucsf.edu/case-form.asp.Users enter data, which get checked at the clientside, and data is sent back to “smoking.ucsf.edu.”


Electronic Data Capture (2)

• From EHR– CTMS needs to know how to access particular data

items in an EHR• e.g., get Mrs. Jones’ serum K+ on March 3, 2006

– which table, which data element, name of data element?

– need a standard EHR data model, so CTMSs can get serum K+ from any EHR without having to know each EHR’s relational database schema

– need standard vocabulary for data element names...

• Bottom line...not happening








• Analyze


Analyzing Data

• Almost all systems can export a “flat file” to SAS, STATA, etc.– Velos exports to SAS, Excel, etc.

• Can share views of Access database on web

Access DB

Internet Information Server 4.0

KaiserOakland

KaiserSanta Clara

Internet

KaiserSan Diego

Active Server Pages (ASP)

smoking.ucsf.edu


Clinical Trial Tasks Summary


– find previous trials– design protocol, CRFs

• IRB approval• Trial registration• Participant recruitment• Randomization• Run study• Analyze

Clinical Trial Tasks SummaryTask Today Tomorrow FutureCredentialing ✓

Study design

find trials ✓ ✓ ✓

protocol design ✓

CRF design ✓ ✓ ✓

IRB approval ✓?

Trial registration ✓ ✓ ✓

Recruitment ✓ ✓ ✓

Randomization ✓ ✓

Run study

project website ✓ ✓

EDC from EHR ✓

Analyze ✓


Outline


– NCI/NIH vision



Data Aggregation Across Trials

• How many Phase III RCTs does NIH fund?• What percentage of depression RCTs are

placebo-controlled?• What percentage of patients in CHF trials

have renal failure?• Which acute MI trials report 5 year mortality?• Is industry funding correlated with positive

outcomes?• What percentage of trials report outcome

results selectively?


Trial Results Reporting• Publication bias now recognized

– ICMJE requires trial registration & protocols to be submitted with manuscripts, to ensure full and complete reporting

• Results reporting policies– NIH: all grants over $500,000 direct required to

“report all data publicly”

– CIHR, UK, and EU moving towards mandatory reporting

– WHO defining required “Minimum Trial Report”


Results Databases

• Should be computable and sharable– Trial Bank Project rctbank.ucsf.edu

• computable reporting: RCT data model, UMLS terms...

• Results database initiatives– ClinicalTrials.gov under pressure from Congress

– caBIG Clinical Trials Database of results

– European Union “Clinical Trials Data Repository”

– Global Trial Bank


Globally Integrated Knowledge

• Results databases need to interoperate to– WHO Registers Network

– bioinformatics databases• DNA sequences: GenBank

• metabolic pathways: BioCyc

• pharmacogenetics, pharmacogenomics: PharmGKB

• protein sequence: UniProt

• protein structure: PDB

• SNPs: SNPdB, etc.

• and to EHRs, for clinical decision support


Outline


– NCI/NIH vision



Summary

• Clinical trials still mostly paper– fragmented systems, lack of wide EHR adoption, lack of

standard clinical and research coding

• Commercial CTMSs – mostly document centered (e.g., PDF) rather than

computable data

– some are more caBIG compatible (e.g., Velos)

– UCSF plans for CTMS support are not yet defined

• Great changes happening...towards eScience?– in NIH investment in clinical trial informatics infrastructure

– in trial registration and reporting for knowledge management

Documents

March 13, 2007: I. SimInformatics for Clinical Trials Epi 206 – Medical Informatics Ida Sim, MD, PhD March 13, 2007 Division of General Internal Medicine,