Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
2
Establishing a High-Quality Real-World
Data Ecosystem
healthpolicy.duke.eduSubscribe to our monthly newsletter at
Broadcast to Begin Shortly
3
Establishing a High-Quality Real-World
Data Ecosystem
Virtual Public Workshop — July 13 & July 14, 2020
Day 1 – Monday, July 13th
4
Welcome and OverviewMarta Wosińska
Deputy Director, Duke-Margolis Center for Health Policy
5
Jacqueline Corrigan-CurayDirector, Office of Medical Policy (CDER)
U.S. Food & Drug Administration
What is a High-Quality Data Ecosystem and Why is it Needed?
6
Lesley CurtisProfessor and Chair, Department of Population Health Sciences
Duke University School of Medicine
Cultural and Institutional Considerations to Align Stakeholders Around a High-
Quality RWD Ecosystem
7
Session 1: Identifying Data Capture
Challenges and Understanding Stakeholder
Needs1:10 pm – 1:30 pm
Modernizing and designing evaluation frameworks for connected sensor technologies in medicine
Establishing a High-Quality Real-World Data Ecosystem
Duke-Margolis
July 13, 2020
2020 | ELEKTRALABS.COM
Andy CoravosCEO @ Elektra Labs
Research Collaborator @ Harvard-MIT Center for
Regulatory Science
Advisor @ Biohacking Village at DEFCON
If we don’t get to your questions today, feel free to message me on Twitter: @andreacoravos
Formerly
Why collect digital measurements in real-time at home?
Remote sensing offers a more holistic view of a person’s lived experience
Data collected in traditionalclinical trials.
Data collected from decentralizedtrials, at home.
VISIBLE DATA POINTSEpisodic, in the clinic.
Source: “Visible vs. Invisible Data” chart designed by Evidation Health
INVISIBLE DATA POINTSContinuous, in daily life
Chronic conditions and their
interventionsimpact a person’s
daily life, from vitals to activity to
sleep.
The future is now: across the industry, sponsors are remotely collecting patient information to support primary and secondary endpoints
Source: The Digital Medicine Society (DiMe) Crowdsourced Digital Endpoint Library https://www.statnews.com/2019/11/06/digital-endpoints-library-clinical-trials-drug-developmen
Digital primary, secondary, or label claim
Exploratory Only
2.5%
25%
30%
17.5%
25%
>50% of examples
N = 39 Digital Endpoints
Development starts early Endpoint positioning
•••••••• 8 Primary Endpoints
••••••••••••••• 15 Secondary
Endpoints
• 1 Label claim
•••••••••••••••• 16 Exploratory
39 Unique Endpoints
Elektra Labs advances remote monitoring by enabling the safe and effective use of connected biosensors at home.
We support and build organizations collecting digital measurements and endpoints in clinical trials.
Source: https://www.fda.gov/drugs/development-approval-process-drugs/public-workshop-patient-focused-drug-development-guidance-4-incorporating-clinical-outcome
Select your measures BEFORE selecting your technology
Use a process derived from best-practice Patient-Focused Drug Development (PFDD)
Step 1: Meaningful Aspect of Health
Step 3: Measure
Step 4: Technology Selection
Step 5: Finalize Endpoint Measurement
Step 2: Concept of Interest
Determine target measures BEFORE technology selection
Example concept
● A change in activity in XX population
Develop hypotheses for target measurement
● Duration, examples:○ Minutes / day walking○ Cumulative daily min per bout of
sedentariness● Frequency, examples:
○ # of daily sedentary bouts○ # of vigorous intensity bouts
● Intensity, examples:○ Average accelerometer counts/day○ Total step counts/week○ METs per minute○ ...
Step 1: Meaningful Aspect of Health
Step 3: Measure
Step 4: Technology Selection
Step 5: Finalize Endpoint Measurement
Step 2: Concept of Interest
Compare the risks/benefits of certain technologies
Step 1: Meaningful Aspect of Health
Step 3: Measure
Step 4: Technology Selection
Step 5: Finalize Endpoint Measurement
Step 2: Concept of Interest
A deep dive on these four components is available in the Appendix.
Think of an evaluation process like a drug or
nutrition label, balancing five components
based on needs in a clinical trial.
A moment to define “real world data”
The 21st Century Cures Act, passed in 2016, places additional focus on the use of these types of data to support regulatory decision making, including approval of new indications for approved drugs.
Congress defined RWE as data regarding the usage, or the potential benefits or risks, of a drug derived from sources other than traditional clinical trials.
Real-world data are the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. RWD can come from a number of sources, for example:
● Electronic health records (EHRs)● Claims and billing activities● Product and disease registries● Patient-generated data including in home-
use settings● Data gathered from other sources that can
inform on health status, such as mobile devices
SOURCE: https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence
20
Session 1: Identifying Data Capture
Challenges and Understanding Stakeholder
Needs1:10 pm – 1:30 pm
Ernesto Ramirez, PhDDesign Lead: Research, Analytics, and Learning, Evidation Health@eramirez
Identifying Data Capture Challenges and Understanding Stakeholder Needs
ESTABLISHING A HIGH-QUALITY REAL-WORLD DATA ECOSYSTEM
JULY 13, 2020
Evidation believes there is a
massive opportunity to change
how we understand and
measure health:
– all via the now shortest
route, the individual managing
health and disease in their
everyday lives
Individually-generated
and Individually-permissioned
data sources
Passive
Continuous
Remote
Objective
Sensitive
The Evidation Way for the Science of Patient Input and Participant Engagement
1
2
3
4
5
Patient Journey MapOnset of symptoms, diagnosis (who/when/how), treatment (decision making/access), disease management and monitoring
Voice of Patient Studies for Medical Product DevelopmentMost burdensome symptoms, daily impacts, perspectives on available treatments, benefits/risks, and unmet need
Patient Input into Measure DevelopmentMeaningful aspects of health, concepts of interest, content validity, feasibility
Participant Input on Study DesignsFeedback from target population on inclusion/exclusion criteria, endpoints/outcomes, benefits/risks, ICF, participant burden, ideal participant experience
Participant Input on Value and IncentivesParticipant motivations, participant feedback on proposed incentives (informational, financial, social) and perceived value to be tested and scaled
Patient involvement is always moderated through an informative and clear consent process and attention to return of value to participants.
Study 1 Study 3Study 2 Study 4
Consent Consent Consent Consent
Data Data Data Data
Evidation, along with collaborators at Eli Lilly and Apple, recently completed a study using PGHD in participants with cognitive decline
Sent and received text
messages
Accessed the web before bedtime
Interrupted sleep
Woke up before 8am
Opened weather, phone, message,
and map apps
Took a nap
Walked and talked on the
phone
Chen, Richard, et al. "Developing measures of cognitive impairment in the real world from consumer-grade multimodal sensor streams." Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019. DOI.ORG/10.1145/3292500.3330690
Ensuring data quality with PGHD is a multistep and continuous process
Issue Mitigation Strategy
Missing data! Develop deep understanding of data flows. Real world data = real world problems.
Missing data! Keep it, it could be informative!
This data doesn’t make sense! Develop methods robust to outliers. Examine outliers, they may be important!
Do I need this much data? What is the minimum amount needed to support inferences and outcomes?
How do I keep people engaged?
Use the features of PGHD to inform real-time compliance monitoring.
evidation.com
Ernesto Ramirez, PhD
Research, Analysis, and Learning
Evidation Health, [email protected]
28
Session 1: Identifying Data Capture
Challenges and Understanding Stakeholder
Needs1:10 pm – 1:30 pm
29
Session 1 — Audience Q&AIdentifying Data Capture Challenges and Understanding Stakeholder Needs
31
Session 2: Emerging Insights and Lessons
Learned from Initiatives to Optimize Data
Capture at the Point of Care2:20 pm – 3:30 pm
32
Monica BertagnolliRichard E. Wilson Professor of Surgery in the Field of Surgical Oncology
Harvard Medical School and Brigham and Women’s Hospital
Optimizing Data Capture at the Point of Care
Monica M. Bertagnolli, MDDuke-Margolis Center for Health PolicyJuly 13, 2020
Integrating
Clinical trials
And
Real world
Endpoints
data
Study-specific case report forms (CRFs)Completed by site clinical research staff
NCTN clinical sitesMultiple EHR vendors
Proprietary Electronic Research
Data Management System
Study-specific CRF Data, Proprietary Format
Alliance Statistics and Data Center
Alliance Data Innovation Lab
DATA LAB
CLINICAL TRIAL PARTICIPANTS
OBSERVATIONAL COHORT
NON-PARTICIPANTS
• treatment planned vs received
• treatment outcomes survivaltreatment responsePROS
• resources utilized
DATA COLLECTED
ALL PATIENTS TREATED AT A CLINICAL SITE WITH A SPECIFIC DIAGNOSIS
• Meet trial eligibility criteria
• Consent to trial participation
• May or may not meet trial eligibility criteria
• Consent to collection of EHR data
• Do not wish to participate in research
Integrating Clinical Trials and Real World Endpoints Data
Advantages of using the EHR to conduct clinical trials
Single data input system maximizes efficiency, reduces cost, and eliminates errors associated with double entry
Can place study results in a context of contemporary real world clinical care by collecting data on both protocol-eligible and non-protocol patients
Provides a mechanism for longitudinal follow up of study patients
1
2
3
Improves the quality and utility of real world data by providing an environment where new EHR data standards can be developed
4
Example: EHR-based collection of overall survival can continue long after clinical trial completion
Challenges
Heterogeneous data and a lack of interoperability
Lack of meaningful outcome data
Is the treatment working?
How is the patient feeling?
Significant existing clinician burden
1
2
3
• Purpose: To develop and maintain standard computable data formats, known as Minimal Common Oncology Data Elements (mCODE), to achieve data interoperability and enable progress in clinical care quality initiatives, clinical research, and healthcare policy development
39
CRF – mCODE Mapping
mCODE Element Implementation
Priority
STEAM Team Epic
ImplementationEpic Playbook
Clinical site Epic mCODE
Implementation
usedby
resultsin
usedby
CRF-mCODEGaps
insights into
possibleavenue
to address
STEAM = Scalable Technology for EHR Adoption of mCODE
▪ Builds community and platform to accelerate interoperable data modeling and implementation around mCODE
▪ Run pilots at scale
Source: http://build.fhir.org/ig/HL7/fhir-mCODE-ig/branches/master/index.html#Modeling
How well does mCODE v1.0 cover the data elements required to complete case report forms for Alliance treatment trials?
Characterization VariablesDiagnosis of concernDemographicsCo-morbiditiesFunctional statusConcomitant medications
Outcome VariablesSurvivalResponse to treatmentRequired change in treatmentAdverse eventsTreatment administeredQuality of lifeResource utilization
mCODE v1.0under development
CRFCRF
CRFCRF
mCODE data elementsMapped
to
A071701, A021703, PALLAS,
PATINA, etcElements rolled
up to profiles
mCODE profiles
• mCODE 1.0 released, includes disease status data element– Mechanism for tracking disease natural
history and treatment response
Under development:Linkage of disease status data element to other relevant data:
imagingpathologylaboratory assessment
Data Input
▪ Low burden data collection
▪ Epic implemented an enterprise solution for capturing
ICAREdata in EHR via SmartData Elements
▪ Available via Special Update (released in March)
or May 2020 Epic release
▪ https://userweb.epic.com/Topic/549
ICAREdata Implementation Priority: Streamlining Clinical Workflows
▪ Software for data sharing available
▪ MITRE-developed ICAREdata client available for
sharing ICAREdata in mCODE standard
▪ Epic is implementing an enterprise solution for
extracting ICAREdata from EHR and mapping it to
mCODE standard
FHIR Message
Automated Interface to
Extract and Share
IT Staff
Extract mCODE
Share mCODE
EHR Database
Alliance Statistics and Data Center
NCTN clinical sitesMultiple EHR vendors
FHIR transfer of eCRF data:mCODE core data elements
CT scans
EHR-Enabled Research Protocol
Alliance Foundation Data Innovation LabDATA LAB
• EHR-ready study-specific order sets• eCRF formats for EHR data capture
Integrating
Clinical trials
And
Real world
Endpoints
data
ICAREdata®
44
Thanks to:
45
Session 2: Emerging Insights and Lessons
Learned from Initiatives to Optimize Data
Capture at the Point of Care2:20 pm – 3:30 pm
46
Laura EssermanDirector, Breast Care Center
The University of California, San Francisco
47QLHC
Transforming data capture to Integrate clinical
care and research
7.12.2020
Collaborators: Adam Asare, Cal Collins, Ronak Ahir, Clarence So, Mitra Rocca, Frank Weichold,Sue
Dubman, Mark Wheeldon, Heidi Collins
UCSF Breast Care Center and Staff, WISDOM study team, I SPY 2 team, I SPY COVID team, UCSF IT team
Laura Esserman, MD, MBA (UCSF, QLHC)
Alfred A. de Lorimier Chair in General Surgery
Professor of Surgery and Radiology
Director, UCSF Carol Franc Buck Breast Care Center
48QLHC
Why do we need better systems? T H E O N E S O U R C E S O L U T I O N
“There are so many people working so hard and achieving so little”
– Andy Grove
CommentaryJuly 27, 2005Efficiency in the Health Care Industries A View From the OutsideAndrew S. Grove, PhDAuthor Affiliations Article InformationJAMA. 2005;294(4):490-492. doi:10.1001/jama.294.4.490The health science/health care industry and the microchip industry are similar in some important ways: both are populated by extremely dedicated and well-trained individuals, both are based on science, and both are striving to put to use the result of this science. But there is a major difference between them, with a wide disparity in the efficiency with which results are developed and then turned into widely available products and services
49QLHC
Old Processes yield Old Results
ONESOURCE SOLUTION
50QLHC
Before Electronic Health Records
THE ONESOURCE SOLUTION
51QLHC
THE ONESOURCE SOLUTION
It’s our clinical systems that are the problem
AFTER Electronic Health Records
52QLHC
THE ONESOURCE SOLUTION
Enable improvements in technology with changes to clinical workflows
VISION: Business Process Change
Principles of One Source
Systems orientation
Quality at the source
Enter once, use many
Mind sets, skill sets, tool sets
CALM “One Source puts patients at the center of new data collection
Puts new info into user configurable apps, drives meaningful analytics
MARKET ENVIRONMENT FDA / CERSI supported
• We should be practicing to improve▪ Not just practicing
• That means we have to know what we are doing and what the consequences are▪ Assessing Outcomes Should not be the heroic effort of chart review ▪ We should not be asking for permission to evaluate outcomes▪ We should only seek permission if we choose NOT to evaluate outcomes (and cost)
• Clinical research is just a special case of clinical care▪ Trials summaries should visible to clinicians▪ Trials often create more discipline about process, which should then be applied to all patierts
• Refocus the clinic on gathering the mission critical data to support decision making▪ And create a framework to assess outcomes → Integrate clinical care and research
What is the Purpose of Clinical Medicine?
54UCSF
• Problem▪ Clinical data collected at the point of care is unstructured
▪ Manual data extraction and re-entry into disparate data platforms required to facilitate research and quality improvement
▪ Creates a workflow that separates clinical care and research, thereby increasing costs, decreasing efficiency and compromising quality, and slowing knowledge turns
• Solution▪ Implement technological and process changes that drive high quality, structured data
capture at the point of care
Background
EHRProvider notes
Clinical results
Patient demographics
Administrative data
Trial CRFs
Research Registries
Data Repositories
Current solution examples
Source Data Capture from EHRs to Improve Quality and Efficiency
55AMIA 2019 Annual Symposium | amia.org
CRFs ask for data that is not in or easy to find in the notes
56QLHC
Structured data as “source”Enable improvements in technology with changes to clinical workflows
Point of Care Data Collection
EHR system
57UCSF
OneSource Technology Stack
Providers
• Electronic Health Record
Using a FHIR standard allows an EHR
agnostic approach
58UCSF
OneSource Technology Stack
• Checklist data entry & write back to EHR
• Clinical Care, Research Registry and Trial Case Report Forms (CRFs)
Read discrete data from and write back to EHRs (via FHIR standards & APIs)
Providers
Form Entry
Single Sign On: Log into EHR and access OpenClinica CRF
• Electronic Health Record
Bidirectional flow of information
59UCSF
OneSource Technology Stack
• Terminology mappings• FDA submission standards
• Checklist data entry & write back to EHR
• Clinical Care, Research Registry and Trial Case Report Forms (CRFs)
Read discrete data from and write back to EHRs (via FHIR standards & APIs)
Providers
Form Entry
Single Sign On: Log into EHR and access OpenClinica CRF
• Electronic Health Record
Data standards mapping
Standards in the background
60UCSF
OneSource Technology Stack
• Terminology mappings• FDA submission standards
• Checklist data entry & write back to EHR
• Clinical Care, Research Registry and Trial Case Report Forms (CRFs)
Read discrete data from and write back to EHRs (via FHIR standards & APIs)
Providers
Form Entry
Single Sign On: Log into EHR and access OpenClinica CRF
• Electronic Health Record
• Research data repository
Direct access of discrete data
Researchers
Data standards mapping
Data seamlessly flows to
trials with consent
61UCSF
OneSource Technology Stack
• Terminology mappings• FDA submission standards
• Checklist data entry & write back to EHR
• Clinical Care, Research Registry and Trial Case Report Forms (CRFs)
Read discrete data from and write back to EHRs (via FHIR standards & APIs)
Providers
Patients
Form Entry
Single Sign On: Log into EHR and access OpenClinica CRF
• Electronic Health Record
• Patient surveys (i.e., I-SPY electronic patient reported outcomes)
• Research data repository
Direct access of discrete data
Researchers
Data standards mapping
Patient reported information integrated whether for clinical care or trial
62UCSF
OneSource Technology Stack
• Decision support displays • “Northstar” metrics tracking
• Terminology mappings• FDA submission standards
Read access
• Checklist data entry & write back to EHR
• Clinical Care, Research Registry and Trial Case Report Forms (CRFs)
Read discrete data from and write back to EHRs (via FHIR standards & APIs)
Providers
Patients
Form Entry
Single Sign On: Log into EHR and access OpenClinica CRF
• Electronic Health Record
• Patient surveys (i.e., I-SPY electronic patient reported outcomes)
• Research data repository
Direct access of discrete data
Researchers
Data standards mapping
QI, metrics seamlessly integrated !!
63QLHC
“Enter Once, Use Many”
Domain / Data ElementsNew PATIENT
(AT ANY PHASE) SCREENING DIAGNOSTICTREATMENT PLANNING SURGERY
SYSTEMATIC TREATMENT
RADIATION TREATMENT FOLLOW-UP RESEARCH
Patient-Reported Outcomes
Patient Health History
Imaging
Biopsy Pathology
Clinical Exam and Stage
Clinical Trial Matching
Treatment
Final Pathology
New data Confirm or Update View Only Confirm/Additional Data Added
64QLHC
Process is the keyMap: AS IS and TO BE
7/15/2020Presentation Title and/or Sub Brand Name Here64
65UCSF
Use CasesBreast surgery checklists: What clinicians need for care
I-SPY 2 case report forms
Electronic patient reported outcomes survey platform
Trial adverse events collection
I-SPY COVID TRIAL
66UCSF
• “Enter once, use many”
• Diagnostic Checklist is the firs phase of a patient’s journey
• Allows surgeons and radiologists and nursing staff to share a common tool to track breast lesions
Breast Surgery Checklists
• “Enter once, use many”
• Surgical Checklist is just one phase of a patient’s journey
• Clinician-vetted discrete data elements: 105 including branched logic fields
• Links patient’s procedure, pathology, operative key elements, surgical complications, and action taken
Breast Surgery Checklists
Surgery Final Pathology Pathologic Stage Complications
Use CasesBreast surgery checklists
I-SPY 2 case report forms: What clinicians need for care are in the case report form
Electronic patient reported outcomes survey platform
Trial adverse events collection
I-SPY COVID TRIAL
I-SPY | The right drug. The right patient. The right time. Now.™
The Original Goal was to Capture Data from Care to Trials
• I-SPY 1: used caBIG tools and platforms (caINTEGRATOR)
• ISPY 2: Tolven→ Salesforce→OpenClinica/Formedix/Salesforce/Tableau
I SPY 2: Neoadjuvant therapy for high risk breast cancer
July 1: Complete CRF redesign • 33 forms to 15• 75% of data elements removed• Focus on critical fields for care and
research• Integrated workflow• Primary endpoint entered as
SOURCE
Aug 1: Complete form design and testing
August 31: Launch new system
Use CasesBreast surgery checklists
I-SPY 2 case report forms
Electronic patient reported outcomes survey platform: Incorporating patient reported data
Trial adverse events collection
I-SPY COVID TRIAL
71UCSF
• Quality-of-life (QoL) prospective sub-study
▪ Explores how patient and tumor subtypes, exposure to investigational
therapies, and residual cancer impact QoL for I-SPY 2 trial participants
▪ Allows assessment of Clinical Benefit Index (efficacy AND toxicity)
▪ Surveys administered from screening, drug treatment, and surgery through
two years postoperatively
• Patient-Reported Outcomes Measurement Information System (PROMIS)
• Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse
events (PRO-CTCAE)
Tracking Patient Reported Outcomes in I-SPY 2
72UCSF
• Current state: Paper-based surveys in the trial
▪ At least 1 survey collected from 1150+ patients
▪ Low completion rate: ~38%
▪ Inefficient process for patients and clinical research coordinators
▪ Manual data entry prone to transcription error and delayed reporting
• Electronic data in the clinic: >75% of patients fill it out
Tracking Patient Reported Outcomes in I-SPY 2
73UCSF
Mapping the I-SPY QoL Survey Workflow (PROMIS 1/2)
Screening surveyDate open: Screening consent signed
Date close: Scheduled date/time of
C1D1 infusion
CRC action required• Before visit
• After patient is registered and
subject ID is generated, enter
patient’s email in OpenClinica
(OC)
• Note: Subject record will be
automatically created in OC
• During visit
• On C1D1, check if patient has
completed survey. If not,
provide tablet to patient to
complete before starting
infusion
CRC action required• Before visit
• Enter C1D1 date/time in OC
and be responsible for keeping
it updated
• Note: Randomization date will
be automatically pushed from
EDC
• During visit
• On C1D1, check if patient has
completed both baseline and
C1D1 surveys. If not, provide
tablet to patient to complete.
Can complete during infusion
CRC action required• Before visit
• Enter AC day 1 date in OC and
be responsible for keeping it
updated
• During visit
• On AC day 1, check if patient
has completed survey. If not,
provide tablet to patient to
complete
C1D1 surveyDate open: 3 days before C1D1 OR
Randomization date (whichever comes later)
Date close: C1D1 at 11:59pm
Inter-regimen surveyDate open: 7 days before AC day 1
Date close: AC day 1 at 11:59pm
CRC action required• Before visit
• Enter surgery date/time in OC
and be responsible for keeping
it updated
• **During visit
• Prior to surgery date, check if
patient has completed survey. If
not, provide tablet to patient to
complete
End of drug regimen 2 / pre-op surveyDate open: 14 days before surgery date
Date close: Scheduled date/time of surgery
74UCSF
Mapping the I-SPY QoL Survey Workflow (PROMIS 2/2)
Follow up 30-dayDate open: 21 days after surgery
Date close: 60 days after surgery at
11:59pm
CRC action required• Before visit
• No action required IF follow-up surveys are sent based on surgery date
• Note: important for CRCs to have the correct surgery date in OC
Follow up 6-monthDate open: 5 months after surgery
Date close: 8 months after surgery at
11:59pm
Follow up 1-yearDate open: 11 months after surgery
Date close: 15 months after surgery at
11:59pm
Follow up 2-yearDate open: 23 months after surgery
Date close: 30 months after surgery at
11:59pm
75UCSF
Translating Workflow to Business Requirements
Requirements for automated survey scheduling system adapted for I-SPY 2 TRIAL
76UCSF
ePRO Survey Pathway
Coordinator triggers survey
notification to patient
Patient completes
eSurvey prior to or
during appointment on
own device or clinic
tabletResearchers have direct & immediate
access to composite survey data
Sets off cascade of automated
scheduled surveys
77UCSF
Use CasesBreast surgery checklists
I-SPY 2 case report forms
Electronic patient reported outcomes survey platform
Trial adverse events collection: Tracking what matters to clinicians, regulators, and pharma
I-SPY COVID TRIAL
78UCSF
• Current I-SPY TRIAL process at UCSF
▪ Source of truth is the oncologist’s note of patient’s treatment course
▪ Coordinators assess note to document AE and Grade
▪ Oncologist reviews / verifies information and assigns Attribution (whether AE is related to study treatment)
• Terminology mapping
▪ Common Terminology Criteria for Adverse Events (CTCAE)
Adverse Events: Current Process
79UCSF
• Streamline AE coding
▪ Simplify the set of allowable CTCAE terminology that can be chosen
▪ Pre-populate official Grade definitions to support Grade designation
• Start with the key questions:
▪ Is there an adverse event to track?
▪ Will the patient continue on therapy?
▪ If no, why not
▪ If yes, dose modification?
Adverse Events Redesign: Form Examples (1/2)
80UCSF
Use CasesBreast surgery checklists
I-SPY 2 case report forms
Electronic patient reported outcomes survey platform
Trial adverse events collection
I-SPY COVID TRIAL: Adverse events→Visual; Write back standard
81UCSF
7/15/2020 81
• Innovative ways to display AE data
• Differs from traditional reports
• Mind numbing tables
• Rapidly find Signal above noise
• (std of care arm)
82UCSF
Write Back
Write Back is Crucial
• What should go back?
• Trial summary
information
• Trial arm assignments
• Ability to populate notes,
problem list from data
captured in a structured
way
Scalable Std for Write Back
• FHIR writeback API
• Standard in the EPIC
EHR
• Scalable across sites
• Hopefully across all EHRs
© 2020 Epic Systems Corporation
83UCSF
• Reconciliation of data elements in I-SPY and clinic redesign CRFs
• Completing integration of single sign on (SSO) in OpenClinica from Epic
• Development of dashboards to enable decision support with clinical data we already collect
OneSource Clinic Redesign: Looking Ahead
Acknowledgements : It takes a village
• UCSF
• Adam Asare• Heidi Collins (EPIC IT)
• Andrew Robinson (EPIC IT)
• Amrita Basu
• Ronak Ahir
• Sarah Schrupp
• Jasmine Wong
• Hope Rugo
• Michelle Melisko
• Merisa Piper
• Garry Peterdson
• QuantumLeap Healthcare Collaborative• Adam Asare
• Lisa Weiss
• Steven Cosari
• Amy Wilson
• Sue Dubman
• James Palazzolo
• UCSF Breast Care Center
• I SPY TRIAL TEAM
• I SPY COVID TRIAL TEAM
• FDA• Mitra Rocca• Frank Weichold• Amy Abernethy• Jacqueline Corrigan- Curay• Many others
• Salesforce/Tableau• Clarence So• John Kim• Andrew Conn• Vidya Balikrishna• CJ Callendar
• Open Clinica• Cal Collins
• Formedix• Mark Wheeldon
85
Session 2: Emerging Insights and Lessons
Learned from Initiatives to Optimize Data
Capture at the Point of Care2:20 pm – 3:30 pm
86
Session 2 — Audience Q&AEmerging Insights and Lessons Learned from Initiatives to Optimize Data Capture at
the Point of Care
88
Amy AbernethyPrincipal Deputy Commissioner of Food and Drugs
U.S. Food & Drug Administration
Real-world Evidence Accelerator –Lessons Learned from COVID-19
Amy P. Abernethy, MD PhD
Principal Deputy Commissioner
Acting Chief Information Officer
U.S. Food and Drug Administration
“reliable information is the best weapon we have against Covid-19.”
Charlie Warzel, NYT Opinon April 3, 2020
NOT FOR DISTRIBUTION
91
WHY RWD? • Urgent need to rapidly understand the natural history of COVID-19• Many critical clinical evidence needs but limited clinical trial resources
(patients, time, competing tasks)– RWD evaluation of treatment patterns and impact provides understanding– RWD can help prioritize research questions to be answered with clinical trials– RWD can improve study design and support participant enrollment– Pragmatic and platform/adaptive study designs can improve efficiency and
generalizability
• Near real-time performance of diagnostics authorized under EUA• Near real-time vaccine performance of future potential vaccines
Real-World Data for COVID-19
Sits within a larger RWD Community
Evidence
Accelerator
RUF/FOCR managed workstream
Health Data &
Technology Partners
A community of
data and analytic
partners ready to
urgently address
questions about
COVID-19
Meetings and forum for rapid cycle feedback
and learning
The
RUF/FOCR*
Evidence
Accelerator
Common protocol for repeated analysis of priority
research questions across multiple data partners (the
“parallel analysis”)
Prioritized research questions
Common data elements and translation tables
between common data models
Our Tools
*Reagan-Udall Foundation (RUF) for
the FDA /Friends of Cancer
Research (FOCR)
Individual Accelerator communities focused on
specific topics (e.g., therapeutics, diagnostics)
The
RUF/FOCR*
Evidence
Accelerator
*Reagan-Udall Foundation (RUF) for the FDA
/Friends of Cancer Research (FOCR)
https://evidenceaccelerator.org/
The
RUF/FOCR*
Evidence
Accelerator
*Reagan-Udall Foundation (RUF) for
the FDA /Friends of Cancer
Research (FOCR)
Prioritized Research Questions
The
RUF/FOCR*
Evidence
Accelerator
*Reagan-Udall Foundation (RUF) for
the FDA /Friends of Cancer
Research (FOCR)
Parallel Analysis Projects
Collaborative
group effort to
align and work
on a common
research
question
Summary Analysis
Data Source #1
Results
Data Source #1Analysis
ParallelAnalysis
Data Source #2
Results
Data Source #2Analysis
ParallelAnalysis
Data Source #3
Results
Data Source #3Analysis
ParallelAnalysis
Results
Data Source #4, 5,..
Data Source #4, 5,…Analysis
ParallelAnalysis
The
RUF/FOCR*
Evidence
Accelerator
*Reagan-Udall Foundation (RUF) for
the FDA /Friends of Cancer
Research (FOCR)
Lab Meeting
99
UC Health DailyDashboards
https://twitter.com/UofCAHealth
100
UC Health DailyDashboards
https://twitter.com/UofCAHealth
101
Veterans Affairs study
Rentsch CT, Kidwai-Khan F, Tate JP, et al. Covid-19 by Race and Ethnicity: A National Cohort Study of 6 Million United States Veterans. Preprint. medRxiv. 2020;2020.05.12.20099135. Published 2020 May 18. doi:10.1101/2020.05.12.20099135
102
Veterans Affairs study
Rentsch CT, Kidwai-Khan F, Tate JP, et al. Covid-19 by Race and Ethnicity: A National Cohort Study of 6 Million United States Veterans. Preprint. medRxiv. 2020;2020.05.12.20099135. Published 2020 May 18. doi:10.1101/2020.05.12.20099135
103
Veterans Affairs study
Rentsch CT, Kidwai-Khan F, Tate JP, et al. Covid-19 by Race and Ethnicity: A National Cohort Study of 6 Million United States Veterans. Preprint. medRxiv. 2020;2020.05.12.20099135. Published 2020 May 18. doi:10.1101/2020.05.12.20099135
The
RUF/FOCR*
Evidence
Accelerator
Tailored to Specific Topics
Identifying
unique
populations,
connecting the
dots
*Reagan-Udall Foundation (RUF) for the FDA /Friends of Cancer Research (FOCR)
Data or analytic EA partners participate in one or more groups
TBD
[VACCINES EA]
o Research questions generated within each accelerator
o Work groups cross boundaries of the different EAs
RUF / FOCR* Evidence Accelerator Work Streams
*Reagan-Udall Foundation (RUF) for the FDA /Friends of Cancer Research (FOCR)
DIAGNOSTICS EA
THERAPEUTICS
EA
Weekly Lab Meeting
Weekly Parallel Analysis
Weekly Lab Meeting
Weekly Parallel Analysis
Weekly Lab Meeting
Weekly Parallel Analysis
On
colo
gy w
ork
gro
up
Oth
er w
ork
gro
up
s
COVID-19 Evidence
Accelerator (EA)
Launch
April 16, 2020
First “Lab” Meeting
May 28, 2020
HCQ +/- azithromycin in
hospitalized patients
Parallel Analysis
Project 1
April 22, 2020
Oncology
Workgroup
June 2020
Diagnostics EA
Launch
Weekly
Lab EA
Weekly
Lab EA
Weekly
Lab EA
Weekly
Lab EA
Weekly
Lab EA
Weekly
Lab EA
Weekly
Therapeutics
Lab EA
June 4th
Real-World Data for COVID-19
NIH, VA, PCORI, etcReal-World Data
Approaches
Other
GovernmentFDA
Sits within a larger RWD Community
Evidence
Accelerator
RUF/FOCR managed workstream
Health Data &
Technology Partners
Our
Responsibility
*Reagan-Udall Foundation (RUF) for
the FDA /Friends of Cancer
Research (FOCR)
The EA with
RUF/FOCR provides
a “safe space” for
key players across
the ecosystem to
lead, scrutinize and
“get this right”
• Data selection
• Protocol design
• Transparency
• Data provenance
• Data quality
• Analytical integrity
• Peer review
• Press interactions
RWD Priorities
DRAFT PRINCIPLES for the EARespect for patient privacy01
Act fast, Traceability and provenance – understand data generation, processing, curation, and analytics02
Transparency, ruthless transparency03
Traceability and provenance – understand data generation, processing, curation, and analytics04
Sharing – show process, explore limitations, pitfalls, and celebration successes – bring work and
learnings to the community05
Build trust – show processes. Show curation approaches. Show comparisons. Curation is
expensive and takes time, many “eyes” along the way, yields trust, understanding, and
confidence in the results
06
Embrace convergence and discordance to facilitate understanding 07
Learning is additive, and continuously integrated to improve knowledge and
understanding 08
Dissemination – responsible evidence generation ( show what good looks like)09
112
Closing RemarksMarta Wosińska
Deputy Director, Duke-Margolis Center for Health Policy
113
Join us for Day 2 Tomorrow, Tues July 14th
Register Online… …and Follow
DukeMargolis
@DukeMargolis
@DukeMargolis
Duke Margolis
healthpolicy.duke.edu
Subscribe to our monthly newsletter at
DC office: 202-621-2800
Durham office: 919-419-2504
1201 Pennsylvania Avenue, NW, Suite 500
Washington, DC 20004
114
Establishing a High-Quality Real-World
Data Ecosystem
healthpolicy.duke.eduSubscribe to our monthly newsletter at
Broadcast to Begin Shortly
115
Establishing a High-Quality Real-World
Data Ecosystem
Virtual Public Workshop — July 13 & July 14, 2020
Day 2 – Tuesday, July 14th
116
Welcome and Opening CommentsMark McClellan
Director, Duke-Margolis Center for Health Policy
117
Session 3: Translating Early Successes of
Core Data Element Programs to
Therapeutic Areas beyond Oncology1:10 pm – 2:20 pm
The Corrona Independent Registry Model for
Autoimmune Conditions
11
9
The Corrona independent registry model is the gold standard for
deep, real-world evidence across autoimmune diseases
120Confidential and proprietary information. Not for distribution.
Rheumatoid Arthritis
>54,000 pts
PsA/SpA
> 4,000 pts
CERTAINRheumatoid
Arthritis bio samples
2,800 pts
Psoriasis
>10,000 pts
Japan RA
≈ 1,900 pts
2001 2010 2013 2015 2016
Portfolio of six registries in five diseases, collecting data from >500 sites in the USA, Canada, and Japan
IBD (Crohn’s & UC)
≈ 2,000 pts
Multiple Sclerosis
≈ 1,000 pts
US US US US/Can
US
US
2017
2017
Key Benefits of Corrona’s Approach
1. It is “fit for purpose” and flexible, with prospectively designed custom data collection tools, avoiding the data gaps found in retrospective approaches
2. It is comprehensive, providing substantial insight into clinical decision making by collecting information directly from physicians and PROs from patients
3. It is credible with the scientific community – over 160 full length manuscripts and over 500 abstracts using Corrona data in top tier journals
4. It is unique with systematic collection of RCT-level outcomes in autoimmune patients cared for in a real-world setting
How does Corrona collect high
quality regulatory grade Real World
Data ?
12
1
Corrona generates Fit for Use RWD that is reliable and relevant for research & global regulatory agencies
122
• In Corrona, virtually all (99%) data are prospectively collected using structureddata fields in registry questionnaires completed by physicians and patients, and “fit for purpose” as defined by 21st Century Cures – not extracted from free text EMR notes
• Systematic collection of variables including validated outcomes typically used in RCTs and patient reported outcomes at every visit (approximately every 6 months)
• Comprehensive, granular safety data are collected, including information about serious adverse events (SAEs) that may not be captured or asked about in a routine clinical encounter with a specialist (e.g., UTI, bronchitis, pneumonia treated by PCP)
• Detailed source documents are collected to support reporting of SAEs and AEs of special interest (Targeted AEs), with case validation and processing by Corrona’spharmacovigilance team of drug safety specialists
12
3
Corrona collects data using a structured approach to study both real-world drug effectiveness and drug safety
Provider FormCollected from physician at
enrollment and Q6 month follow up
01
Patient FormCollected from patient at baseline and Q6 month follow up
02
Targeted Safety EventsCollected from physicians starting
after enrollment during follow up
03
Data Collection Process
✓ Data entry into a part 11 compliant EDC system
✓ q6 month data collection
✓ Questionnaires are revised based on changes in the relevant field
✓ Extensive QA/QC process to ensure data completeness and accuracy of the analytic database
✓ Source documents requested for SAEs and TAEs for case validation
Over the past 20+ years, Corrona has developed & evolved the
approach to collect high quality data across 500+ sites
124Confidential and proprietary information. Not for distribution.
Questionnaires
Electronic Data Capture
Analytic database
▪ Questionnaire design
▪ Site selection and
training
▪ Site monitoring
▪ Site feedback
• EDC design and
automatic edit
checks
• Routine site
performance reports
• Routine analytic
data checks
based on logic
Sites
Data Capture
Data
Reports
An illustration of the longitudinal nature & data depth
in the registry
Rheumatoid Arthritis
12
5
Corrona RA Registry USA – more than 54,000 patients enrolled
126Confidential and proprietary information. Not for distribution.
✓ 193 private and academic sites
✓ 827 participating rheumatologists
✓ Sites in 42 US states
✓ Over 400,000 registry visits
✓ Total follow-up time of 199,111 person years
✓ Median follow-up is 4.5 years
Summary of key variables collected in the Corrona RA Registry
Demographics
• Age
• Gender
• Race/ethnicity
• Height/weight
• BMI
• Education
• Work status
• Insurance type
• Smoking Status
• Alcohol use
Treatment History
• Conventional synthetic DMARD, Biologic DMARD and Small
Molecules, Biosimilar DMARD, Targeted synthetic DMARD,
NSAID use, Other concomitant medications
Disease Characteristics and Disease
Activity
• RA year of onset and diagnosis
• Comorbidity and medical history
• ACR functional class
• Subcutaneous nodules
• Secondary Sjögren’s syndrome
• Physician global assessment of arthritis
• 28 tender and swollen joint count
Patient Reported Outcomes
• Health Assessment Questionnaire and usual abilities
• Pain VAS (0-100), Patient global assessment VAS (0-100)
• Fatigue VAS (0-100), Stiffness VAS (0-100)
• Days unable to work due to RA
Lab Measures*
• CBC, Liver function, Inflammatory markers, autoantibodies
(CCP and Rheumatoid factor), Lipid panel, 25-OH Vitamin
D, Vectra DA, Joint Radiographs: abnormalities present/not
present in erosions, joint space narrowing, deformities; Chest
Radiographs; Joint MRI: synovitis, erosions, osteitis, bone
marrow edema; Ultrasound (Synovitis, Erosions, Doppler
signal); Bone densitometry (machine type, lumbar spine,
femoral neck, 1/3 radius)
Targeted Adverse Events
• Serious Infections, Malignancy, Cardiovascular, Neurologic,
Hepatic, General Serious, Anaphylaxis, Autoimmune,
Gastrointestinal, Pregnancy Event
127Confidential and proprietary information. Not for distribution.
Patient
Reported
Patient
Reported
Physician
Reported
Physician
Reported
Corrona has built a strong publication track record in both comparative effectiveness and safety studies, with 500+ abstracts and 160+ manuscripts
128
Select Comparative Effectiveness Publications
Harrold LR, et al. Comparative effectiveness of abatacept versus tocilizumab in rheumatoid
arthritis patients with prior TNFi exposure in the US Corrona registry. Arthritis Res Ther. 2016 Dec
1;18(1):280; 1-8.
Harrold LR, et al. The comparative effectiveness of abatacept versus anti-tumor necrosis
factor switching for rheumatoid arthritis patients previously treated with an anti-tumor
necrosis factor. Ann Rheum Dis 2015 Feb; 74(2):430-6.
Harrold LR, et al. Comparative effectiveness and safety of rituximab versus subsequent anti-
tumor necrosis factor therapy in patients with rheumatoid arthritis with prior exposure to anti-tumor necrosis factor therapies in the United States Corrona registry. Arthritis Res Ther
2015;17:256.
Select Drug Safety Publications
Harrold LR et al. One-year risk of serious infection in patients treated with certolizumab
pegol as compared with other TNF inhibitors in a real-world setting: data from a national
U.S. rheumatoid arthritis registry. Arthritis Res Ther. 2018 Jan 2;20(1):2.
Pappas DA, et al. Herpes zoster reactivation in patients with rheumatoid arthritis:
Analysis of disease characteristics and disease modifying anti-rheumatic drugs. Arthritis Care Res (Hoboken). 2015 Dec 67(12):1671-8.
Harrold LR et al. Risk of infection associated with subsequent biologic agent use after
rituximab: results from a national rheumatoid arthritis patient registry. Arthritis Care Res
(Hoboken). 2016 Dec;68(12):1888-1893.
The registry data capture model enables evaluation of disease factors associated with outcomes
129
Model CModel BModel A
Ha
za
rd R
atio
1.0
0.8
0.6
0.4
0.2
HighModerateLowRemission
Disease activity
Solomon DH et al. Arthritis Rheumatol. 2015
Model A is adjusted for age and gender only, Model B is adjusted
for age, gender, and CV risk factors; Model C is adjusted for all
variables in Model B PLUS RA THERAPIES
IRRs of SIEs, malignancies and CVEs comparison of a later TNFi to earlier
TNFis before (a) and after (b) PS matching for baseline characteristics
associated with line of therapy
The impact of line of therapy when
comparing IRRs within a drug class
Harrold LR et al. Arthritis Res Ther. 2018
A. Before PS Matching
B. After PS Matching
Impact of cumulative disease activity
on CVD (MACE) events
Select use cases of Corrona’s data
13
0
Corrona RWE data has been leveraged to support regulatory approvals, label expansion and market
access
• Post-authorization Safety Study (PASS) to meet FDA and EMA requirements as part of new drug approval post marketing requirements (PMR)
• Effectiveness and safety data for new formulation approval for European regulators based on US real-world Corrona registry data
• Secondary indication approval by leveraging safety data in primary indication from Corronaregistry as part of regulatory submission
• US regulatory approval to extend label for approval as monotherapy, extending the current approval as combination therapy
• US market access negotiations with payers to expand market access for use as earlier line of
therapy
• European regulator negotiations to approve use of biologic in earlier lines of therapy
Confidential and proprietary information. Not for distribution. 131
Results of 5-Year Post-Authorization Safety Study for Pfizer on Tofacitinib vs Biologic DMARDs as part of FDA Commitment at Approval (PASS analytic protocol nested in the Corrona RA Registry)
132Confidential and proprietary information. Not for
distribution.
Hazard Ratios for MACE, SIEs and HZ
tofacitinib vs biologic DMARDs
Note: HZ includes non-serious and serious events; bbDMARD initiators were the reference population for calculation of HRs
Kremer J et al. EULAR, June 2019, OP0028
0.1 1.0 10.0
MACE
SIEs
PS-trimmed
PS-matched
HZ
MACE
SIEs
HZ
Lower rate with tofacitinib Lower rate with bDMARDs
1.15 (0.85, 1.55)
0.99 (0.72, 1.36)
2.28 (1.41, 3.68)
2.12 (1.22, 3.66)
0.66 (0.34, 1.29)
0.60 (0.30, 1.18)
0.356
0.947
0.001*
0.007*
0.225
0.137
1.01 (0.73, 1.38)
1.01 (0.73, 1.39)
2.27 (1.33, 3.88)
2.26 (1.28, 3.99)
0.58 (0.29, 1.15)
0.58 (0.30, 1.15)
0.967
0.970
0.003*
0.005*
0.121
0.120
Unadjusted HR
Adjusted HR
Generating RWE from Corrona RA Registry to support acceptance by European regulators for earlier use of abatacept for RA patients
133
Corrona demonstrated favorable Benefit/Risk profile for Rituximab vs Anti-TNF switching for RA patients with prior anti-TNF use
134
Trimmed populations
TNF inhibitor Rituximab
Events PY Event/100 PY Events PY Event/100 PY
Malignancy 13 760.9 1.7 (1.0,2.9) 5 270.5 1.8 (0.8,4.4)
Serious Infection 19 760.9 3.2 (2.0,5.0) 3 270.5 1.5 (0.5,4.5)
CVD Event 12 760.9 1.6 (0.9,2.8) 5 270.5 1.8 (0.8,4.4)
Adjusted Odds
Ratios for Major RA
Outcomes
Harrold et al. Arthritis Research & Therapy (2015) 17:256
Corrona RWE clinical effectiveness and safety data support EMA approval of new formulation
135
Cohen S, et al. Clinical Effectiveness of Tofacitinib 11mg Once Daily (QD) Versus Tofacitinib 5mg Twice Daily (BID) in the Corrona US RA
Registry [abstract]. Arthritis Rheumatol. 2018; 70 (suppl 10).
Primary and secondary outcomes at 6-month visit for patients who initiated
tofacitinib 11mg QD vs 5mg BID
Thank you
13
6
137
Session 3: Translating Early Successes of
Core Data Element Programs to
Therapeutic Areas beyond Oncology1:10 pm – 2:20 pm
138
Chhaya ShadraVice President, Product Management - Data and Informatics
Verana Health, Inc.
CONFIDENTIAL AND PROPRIETARY
Chhaya ShadraJuly 2020
Generating Regulatory-Grade Real World Evidence Beyond Oncology
CONFIDENTIAL AND PROPRIETARY140
Oncology Longitudinal careBinary
(mortality rate)
Complete - patient experience
centralized in oncology care
Other specialtie
sEpisodic care
Variable, measured with various tools,
processes
Incomplete -snapshot of patient experience, focused on symptomatic care
Generating real world evidence in specialty areas outside oncology presents unique challenges
Continuity of care
Primary outcome measure
Completeness of health record
CONFIDENTIAL AND PROPRIETARY141
Verana Health combines deep clinical expertise with a highly automated and scalable next generation data infrastructure to generate RWE across specialty areas
Expert team of clinicians, biostatisticians and epidemiologists
Close partnerships with leading specialty societies
Specialty agnostic data architecture
EHR data curation through semantic harmonization & NLP
Data linking for enrichment
Deep ClinicalExpertise
Scalable DataInfrastructure
CONFIDENTIAL AND PROPRIETARY142
On the Forefront of Clinical Care
Trusted Relationships with Trial Sites and Patients
Leadership in Regulatory Engagement
Partnerships with specialty societies enable medical expertise and high trust relationships with physicians
• Influence clinical guidelines nationally• Deep experience in study design and challenges
• Relationships with large and prominent trial sites• Product feedback loop enabled by physician experts
• High credibility with regulatory agencies (FDA, CMS)• Leaders to advance adoption of real-world data for trials
CONFIDENTIAL AND PROPRIETARY
The Verana team is rooted in deep clinical experience
14
3
Data Science and Epidemiology Expertise
Medical Expertise
● Academic Institutions – Stanford University, Duke University, Dana-Farber Cancer Institute, Broad Institute of MIT and Harvard
● Leading Companies – Flatiron, Syapse, ORNL, IQVIA, Aetion, Prometheus
● Dr. Matthew Roe, Chief Medical Officer ○ Duke University ○ Leadership of PCORnet- and FDA-
funded projects leveraging EHR data for prospective randomized studies
○ 15+ years of leadership experience in the National Cardiovascular Data Registry with ACC
● Team of specialty-focused medical directors, medical advisors and clinical consultants that span clinical practice, academic research and bioinformatics
● Through our specialty association partnerships, access and thought partnerships with key opinion leaders
8MPH
12 MD
12PhD
30% of the Verana team has an MPH, MD, and/or PhD.Over 60% of the Verana team has a Master’s degree or higher.
CONFIDENTIAL AND PROPRIETARY144
The scale of Verana’s partner registries presents unique challenges
The Axon Registry®
• 5 years of longitudinal EHR data
• 12.4 million patient visits
• 2.4 million unique patients
• 1,126 contributing physicians
• 37 EHRs integrated
The IRIS® Registry
• 6 years of longitudinal EHR data
• 343 million patient visits
• 72 million unique patients
• 15,713 contributing physicians
• 55 EHRs integrated
As of April 1, 2020 As of May 1, 2020
The AQUA Registry
• 6 years of longitudinal EHR data
• 37.8 million patient visits
• 6.3 million unique patients
• 1,800 contributing physicians
• 36 EHRs integrated
As of December 31, 2019
CONFIDENTIAL AND PROPRIETARY145
Data challenges are amplified by the number of unique EHRs integrated and nature of specialty care delivery
Data variability, inconsistency, & incompletenessProvider behaviors & EHR layouts vary
Timeliness and lag dependent on practice delivery scheduleData continuity depends on how frequently practices push/allow pull of data
Setting represented in registries skewed towards specialty community careMissing health system encounters from PCPs, inpatient, other specialists
CONFIDENTIAL AND PROPRIETARY146
EHR data is our starting point to create regulatory-grade data sets with specialty depth
Access data from large patient populations by collaborating with medical societies and
existing registries
Specific details about patient populations becomes crucial as our understanding of disease improves and drugs become
more targeted
Assess safety and efficacy in the real world using outcomes data
(e.g. laterality, IOP, EDSS)
FootprintBreadth
ClinicalDepth
Outcomes
CONFIDENTIAL AND PROPRIETARY147
● EHR data collected from specialists
● Diagnosis, procedures, notes, specialist meds
● Structured/unstructured data streams
Linking additional data enables specific targeted analyses and insights
Commercial / Man. Medicare Rx
Images
Lab Data & Genomics
CMS Claims Data
EHR Alone
Linked Data Type Potential Analyses
● Outcomes associated with Tx regimen, adherence, mono vs. combo use, line of therapy● Health-resource utilization● Holistic patient demographics
● Comorbidity burden in elderly● Medicare health-resource utilization
● Imaging confirmation of Dx● Disease staging and segmentation by imaging● Validation of imaging CT endpts
● Patient journey
● Real world comparative effectiveness
● Potential safety issues in new therapies
● Potential strategies for trial enrichment
Insights
● Population segmentation ● PASS / safety associated with treatment● AI analytics for predictors of response
CONFIDENTIAL AND PROPRIETARY148
Verana’s data architecture is highly automated and specialty agnostic
CONFIDENTIAL AND PROPRIETARY149
Fit for purpose information models: Data ingestion
Shape: Database Model
Patient Medication
Encounter
Shape: ReSTful (JSON)
{
}
{ … }
[ … ]
{ … }
[ … ]
Shape: Delimited
Clinical Document
Shape: CCDA
Shape: VH Common Model
● Design Considerations○ C-CDA○ HL7 Feed formats○ FHIR Schema○ Custom Domain Fields○ Unstructured Text Fields○ Binary Image Fields○ Genomic Data Support
CONFIDENTIAL AND PROPRIETARY150
Fit for purpose information models: Observational research
Concepts, Ontologies and Vocabularies
patient identifier patient identifier patient identifierpatient identifier
Design Considerations
● PCORnet● OMOP● i2b2● Sentinel CDM
Patient data
● Demographics○ Birth year○ Gender○ Race○ Ethnicity○ Region
● Medical history
● Comorbidities
● Insurance type
Coded encounter data
● Visit dates and practices
● Diagnoses harmonized to ICD-10-CM, ICD-9-CM, SNOMED-CT
● Procedures & visit types harmonized to CPT, HCPCS
● Medications harmonized to RxNorm
Algorithmic enhancements derived from unstructured
data
● Real-world endpoints○ VA○ IOP○ T25-FW○ EDSS○ Gleason
● Disease severity measures
Additional data sources
● Linkages possible○ Medical claims○ Pharmacy
claims○ Lab test
results ○ Genetic test
results○ Images○ Non-specialist
EMRs○ Prescription
records○ PROs
Provider data
● NPI identifier
● Region
● Practice type○ Academic
Center○ Multi-Specialty
Group○ Urban/Rural
CONFIDENTIAL AND PROPRIETARY151
Data curation: Semantic harmonization
DOMAIN TERMINOLOGY OR ONTOLOGY
Conditions ICD-9-CM or ICD-10-CM or SNOMED-CT
Medications RxNorm
Procedures CPT or HCPCS or ICD-9-PCS or ICD-10-PCS
Observations (Relevant Subset) LOINC
Marital Status HL7 Administrative Gender
Race CDC Race Category
Ethnicity HL7 Ethnicity
CONFIDENTIAL AND PROPRIETARY
Verana has developed a custom disease ontology that is relevant for clinical use cases in our specialty areas
152
• Disease ontology that is relevant for clinical use cases
• Based on how providers categorize and think about diseases
• We do not use an out of the box administrative classification to fit all purposes
• Clinical input is a key component of our Data Module specification and QA process
Verana disease ontology for Age-Related Macular Degeneration (AMD)
CONFIDENTIAL AND PROPRIETARY153
An end-to-end data quality framework is applied across the entire data pipeline
Cornerstones of Data Quality
Doesthe Data...
Technical Clinical Scientific
Completeness encompass the entire clinical picture?
Field completeness is assessed among fields where data is expected. (e.g. each diagnosis must have a documented date)
Data completeness in clinical context exists (e.g., intraocular pressure is expected to be documented for patients with a diagnosis of Glaucoma)
Factors (e.g., confounders) have been considered in study design & analyses
Accuracyaccurately reflect patient chart/reality?
Data conforms to expected data types & constraints
EHR effectively captures patient journey & provider patterns
Results are within range of scientific acceptability
Traceability contain provenance back to source? Data elements & transformations are clear and auditable during ingestion and curation
Study specifies a clear, auditable patient cohort
Study design, methods, and analysis are clear and transparent
Consistencymaintain integrity across structures, time, releases?
Data are represented in a consistent data model, under congruent architecture & format
Cohort-specific trends & rates are tracked across time
Data is validated against published studies & external sources
Generalizability represent a minimally-biased sample?Data elements are harmonized to industry standards
Biases have been assessed & accounted for in clinical interpretation
External comparisons are used to identify and adjust for biases
Timeliness reflect recent practice patterns?Data is refreshed at appropriate frequency
Current practice patterns, treatments are incorporated
Data timeframe is relevant to current study
1. Miksad, RA and Abernethy, AP. Harnessing the Power of Real-World Evidence (RWE): A Checklist to Ensure Regulatory-Grade Data Quality. Clin Pharm & Ther 2017;103:202-205.
2. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological
Health (CDRH), Use of Electronic Health Record Data in Clinical Investigations Guidance for Industry, July 2018.
3. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological
Health (CDRH), Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies Using Electronic Healthcare Data. May 2013.
CONFIDENTIAL AND PROPRIETARY154
Scaled approach to data quality: Overall scores
Appropriateness - Disease Burden . Patient Journey . CT Pre-Screening . Market Tracking
HEOR . Comparative Effectiveness Research . CT Site Selection and Recruitment . Scientific
Publications . Clinical Outcomes Research . Post-Authorization Safety Studies
CMS submissions for VBC/MIPS Quality Reporting . FDA Submission Support to Life-Sciences
Business Grade
ResearchGrade
Regulatory Grade
AGGREGATED DATA QUALITY SCORES
Ophthalmology
EHRs: 56 | Practices: 3128 | Providers: 18,234 | Patients: 60,231,521
Completeness Consistency Traceability Timeliness Accuracy Overall
0.82 0.79 0.65 0.95 0.83 0.81
Data Quality Scores Compared to Previous Data Refresh
(+10%) (+7%) (-5%) (+12%) (+5%) (+10%)
CONFIDENTIAL AND PROPRIETARY155
Key Takeaways
• There are meaningful opportunities to expand upon the successes of RWE in oncology for impact in other specialty areas, but also unique challenges
• Specialty society partnerships facilitate access to some of the largest specialty data sets in medicine and enables direct collaboration with physicians for data enrichment
• A scalable specialty-agnostic data platform can be built with generalizable data curation processes
• Data quality can be measured at scale yet must be adjusted to be fit for use case, though applying stringent data quality thresholds may lead to reduced cohort sizes
• Better data standardization and documentation in original systems of record like EHRs will reduce information loss across the data life-cycle
156
Session 3: Translating Early Successes of
Core Data Element Programs to
Therapeutic Areas beyond Oncology1:10 pm – 2:20 pm
157
Session 3 — Audience Q&ATranslating Early Successes of Core Data Element Programs to Therapeutic Areas
beyond Oncology
158
Session 4: Emerging Tools & Technologies
to Support High-Quality Data Capture2:30 pm – 3:40 pm
159Copyright ©2018 VeradigmTM | All rights reserved
Emerging Tools & Technologies to Support High-Quality Data Capture
Stephanie Reisinger, VP and GM, Veradigm Life Sciences
160Copyright ©2018 VeradigmTM | All rights reserved
Automated Delivery• Drones• Self driving cars
Augmented Reality• Connected eyewear• Operating Room of
the future
Genetic Technologies• Low-cost sequencing• Gene editing
Internet of Everything• Smart appliances• At-home diagnostics• Connected clothing
Cheaper Computing• Private data cloud• Big data analytics
Robotics & Automation• Robotic surgery• Robotic caregivers• Exoskeleton
Blockchain• Patient medical records• Drug supply chain integrity• Clinical trials
3D Printing• Bioprinting – organs,
bones, teeth• Surgical instruments• Devices (e.g. pacemakers
Device Miniaturization• Smart tattoos and bandages• Medical grade wearables• Digital pills
Artificial Intelligence• Drug discovery/
development• Diagnosis• Patient Monitoring
DIGITAL HEALTH DEL IVERY IS CREATING AN
INFORMATION EXPLOSION…
… (real-world) data can be generated and disseminated nearly simultaneously
Macro Healthcare Trends
161Copyright ©2018 VeradigmTM | All rights reserved
Research In the Context of Digital Health Delivery
<2% >95%
Of people in the U.S. participate in Clinical
Research
Of people in the U.S. receive Digital Healthcare
Services
• Specialized treatment protocols with narrow patient selection criteria
• Safety and efficacy focus
• Difficult to generalize across populations
• Broad heterogenous patient populations with many comorbidities
• Diverse treatment protocols
• A much better model to understand treatment generalizability
Would be willing to participate in Research
Ideal World Real World
~90%
>95%
McHugh, Kelly & Swamy, Geeta & Hernandez, Adrian. (2018). Engaging patients throughout the health system: A landscape analysis of cold-call policies and recommendations for future policy change. Journal of Clinical and Translational Science. 2(6). 384-392. 10.1017/cts.2019.1.
162Copyright ©2018 VeradigmTM | All rights reserved
Today’s Reality:
Research is disconnected from digital healthcare delivery
Operating model developed before healthcare digitization
Data captured de-novo for each study
Forms-based data capture
Study data is siloed, difficult to integrate and reuse
Disconnected from point of care workflows - main focus on data curation post-hoc
Current research model is unable take advantage of digital and consumer-driven healthcare delivery
163Copyright ©2018 VeradigmTM | All rights reserved
The Integrated Research Strategy
Research capabilities embedded into existing point of care workflows & technology
“Research-enabled” Point of Care
ENGAGED PROVIDERS
Research data a natural output of care process
ENGAGED PATIENTS
Alerts
EMR-embedded research
Messaging, Apps
Surveys, PROs
InsightsInsights
InsightsStudy Protocols
CONNECTED RESEARCH SPONSORS
164Copyright ©2018 VeradigmTM | All rights reserved
Opportunities for Integrated Research (partial list)
Study Feasibility: data about patient characteristics & availability, site performance and investigator availability centralized & real-time
Digitized Protocols: standardize key protocol elements in quarriable format
Protocol, Physician, Patient Match-making: real-time identification of patients matching study protocols “at scale”
Provider Alerting: at the point of care
Patient Engagement: utilizing patient portals, and EHR patient engagement technologies
360° Insights: analytics customized to needs of different healthcare stakeholders from one “source of truth”
Digital Healthcare Journey: interoperability standards enable disparate digital healthcare data to be linked and connected
Synthetic Control Arms: modeled from real-world data previously collected
Virtual Trials and Remote Patient Monitoring
165Copyright ©2018 VeradigmTM | All rights reserved
Telehealth in the time of COVID
~0 minutes per week in Jan. to 75,000-100,000 minutes per day in April-June
Telehealth utilization in PracticeFusion EHR Platform (~30,000 independent and small practice doctors)
166
Session 4: Emerging Tools & Technologies
to Support High-Quality Data Capture2:30 pm – 3:40 pm
167
Session 4 — Audience Q&AEmerging Tools & Technologies to Support High-Quality Data Capture
168
Establishing a High-Quality Real-World
Data Ecosystem
healthpolicy.duke.eduSubscribe to our monthly newsletter at
Broadcast to Resume at 3:40 pm ET
169
Session 5: Next Steps Toward Establishing
a High-Quality RWD Ecosystem3:40 pm – 4:30 pm
170
Session 5 — Audience Q&ANext Steps Toward Establishing a High-Quality RWD Ecosystem
171
Closing RemarksMark McClellan
Director, Duke-Margolis Center for Health Policy
172
Thank You!
Contact Us Follow Us
DukeMargolis
@DukeMargolis
@DukeMargolis
Duke Margolis
healthpolicy.duke.edu
Subscribe to our monthly newsletter at
DC office: 202-621-2800
Durham office: 919-419-2504
1201 Pennsylvania Avenue, NW, Suite 500
Washington, DC 20004