Patient-Powered Research Networks for Relative ......Original research was conducted in only two articles: a survey of PPRN PIs and a comparative effectiveness study First author Author

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement no [115546], resources of which are composed of financial contribution from the European Union’s Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. www.imi.europa.eu

Patient-Powered Research Networks for Relative Effectiveness Research

Final report, 20 September 2016

This report was prepared by Chloe Brice1, Clara Isaac1, Pall Jonsson2, Rob Thwaites3,

Antonio Ciaglia4, Adeeb Ahsan2, Eleanor King1, Rory O’Brien1, Brian Buysse1 and Joseph

Kim1 on behalf of the GetReal Consortium

Affiliations: 1IMS Health, Real-World Evidence Solutions, London, UK; 2NICE, Manchester, UK; 3Takeda, London, UK; 4IAPO, London,

UK

This work was funded by an unrestricted grant from Boehringer Ingelheim.

Results of a Pragmatic Literature Review and Desktop Research


Table of Contents

Abbreviations Results (continued)

Background Desktop Research

Objectives Flowchart of Search Results

Methods List of 33 Eligible PPRNs

Pragmatic Literature Review Characteristics of PPRNs

Desktop Research Launch Dates

Results Data Types

Pragmatic Literature Review Disease Areas

Flowchart of Literature Search Results Available Parameters

Characteristics of Identified Publications Classification- Usefulness of PPRNs for CER

Results of Thematic Analysis Caveats and Limitations

Successful Example of the Use of a PPRN for CER Pragmatic Literature Review

Desktop Research

Conclusions

Pragmatic Literature Review

Desktop Research

List of Stand-alone Documents

References of Publications Included in the Pragmatic Literature Review

Appendix


Abbreviation Description

CER Comparative Effectiveness Research

EMR Electronic Medical Record

HTA Health Technology Assessment

ICD International Classification of Diseases

IMI Innovative Medicines Initiative

PCOR Patient-Centered Outcomes Research

PCORI Patient-Centered Outcomes Research Institute

PCORnet Patient-Centered Outcomes Research Network

PI Principal Investigator

PPRN Patient-Powered Research Network

PRO Patient-Reported Outcome

QoL Quality of Life

Abbreviations


Background


• Patient-Powered Research Networks (PPRNs) are online platforms where patients

share their health data with other people or organisations of their choice, with

health data coming from either the patients directly (e.g., patient-reported

outcomes [PRO]) or their electronic medical records (EMR), or both.

• The usefulness and acceptability of PPRN data in comparative effectiveness research

(CER) is unclear.

• The aim of the current pragmatic literature review is to provide an overview of the

recent literature/publications on PPRNs, while the aim of the desktop research is to

identify existing PPRNs and assess them for their suitability for CER.

Background


Objectives


1. Understand available PPRNs and propose classification based on their

characteristics with regard to CER

2. Characterise the potential usefulness of PPRNs (especially Patient Centred

Outcomes Research Network [PCORnet] PPRNs) to regulators, Health Technology

Assessment (HTA) bodies and payers for evaluating comparative effectiveness of

products by:

a) Showcasing successful examples of generation of comparative effectiveness

data (if any) on licensed products

b) Identifying any disease areas for which PPRNs are particularly useful

c) Understanding the limitations and practical feasibility of using PPRNs for CER

Objectives


Methods – Pragmatic Literature Review


Methods: Pragmatic Literature Review (1/2)

• Titles and abstracts of identified records were screened for relevance

• Full-texts of relevant publications were reviewed for eligibility

• Any additional publications identified by the desktop research were added

• Relevant data were extracted to and managed in spreadsheets

• Depending on the type of publication, data synthesis was quantitative (e.g., results of CER) or qualitative (e.g., opinions held by authors)

• Medline, EMBASE and Google Scholar were searched for relevant literature published from 1st January 2011

• The search was conducted on 03 March 2016 and restricted to publications in English language

• More details were provided in a Search Strategy Document

• CER results were reported for individual quantitative studies and opinions and views of authors were reported for opinion papers


Quotes were grouped according to the study objectives and underwent a thematic

analysis, according to the following steps:

1. Themes were generated from the content of each quote and tagged to any

individual quotes representing that theme

2. Themes that arose from subsequent quotes, and could not be classified into

existing themes, became new themes

3. All quotes were checked again to ensure that they did not contain newer themes

Methods: Pragmatic Literature Review (2/2)


Methods – Desktop Research


• For each of the searches, the first 20 results (URL and name of URL) were extracted to a spreadsheet

• Each URL was visited and those identifying PPRNs were recorded

• Any additional PPRNs identified by the pragmatic literature review were added

• Relevant information on each PPRN meeting the working definition was extracted to a spreadsheet

• Parameters of interest included those relating to: CER, source of data, medication, outcomes and potential covariates

• The desktop research aimed to find PPRNs and was performed in Google (www.google.co.uk) with Google Chrome as browser

• A total of 23 individual searches were carried out, each consisting of a different set of search terms (see Search Strategy Document for details)

• Characteristics of PPRNs were reported

• An scoring system was developed to classify PPRNs with regard to usefulness for CER

Overview of Methods


To be suitable for CER, a PPRN had to meet all of the following criteria:

1. Collected data on the type of healthcare intervention/treatment

2. Collected data on outcomes, either PROs or from EMR

3. A total CER suitability score of at least 8 points

Construction of a Scoring System for Suitability for CER


• PPRNs with sufficient available data were classified by grouping identified key elements into three components: Therapy, Outcomes and Covariates.

• A scoring system was created to allocate points to data relating to the three components. Points were allocated using the appropriate fields in the desktop research data extraction excel file (please see slide 46 for embedded file).

* Required to be counted as a PPRN suitable for CER

** Six variables were assessed (age, gender, ethnicity, country, height and weight), with 1/6 point allocated per

available variable.

Component Item Points Total (weight)

1. Therapy:

a) Therapy type* 1

4 (36%)

b) Dosage and Frequency 1

c) Compliance 1

d) Concurrent medication 1

2. Outcomes:

a) Effectiveness data from PROs* 1

4 (36%)

b) Effectiveness data from EMR* 1

c) Clinical functional scores 1

d) Adverse event 1

3. Covariates:

a) Demographic variables** 1

3 (28%)

b) Co-morbidities 1

c) Family history 1/2

d) Lifestyle factors 1/2

TOTAL 11 (100%)

The CER Suitability Scoring System


Results – Pragmatic Literature Review


386 records identified by searching MEDLINE/EMBASE

446 unique records screened for titles and abstracts

31 full-text publications assessed for eligibility

18 publications eligible for inclusion in review *

2 additional publications identified by searching Google

100 records identified by searching Google Scholar

415 records excluded

Records identified by the pragmatic literature review

Records identified by the desktop research

15 full-text publications excluded

* See reference list.

Search Results and Publications Eligible for Inclusion


Characteristics of Identified Publications (1/2) Original research was conducted in only two articles: a survey of PPRN PIs and a comparative effectiveness study

First author Author Affiliation Funding source Article type Journal, Year of publication

Ahmed McGill University, School of Physical and Occupational Therapy

Fonds de la Recherche en Santé du Quebec

Review Med Care, 2012

Ali Johns Hopkins Berman Institute of Bioethics

PCORI Original article: survey of PPRN PIs

Account Res, 2015

Califf Duke University School of Medicine; PCORnet Coordinating Center

National Institutes of Health

Invited commentary N C Med J, 2014

Daugherty PCORI PCORI Brief communication J Am Med Inform Assoc, 2014

Fleurence PCORI PCORI Brief communication J Am Med Inform Assoc, 2014

Garber Harvard University Amgen, Genentech, Takeda & McKinsey

Analysis & commentary Health Aff, 2011

Kupfer-Schmidt

Center for Medical Technology Policy None declared Conference abstract Value Health. 2012

Levesque University of Pittsburgh School of Medicine

UCB and Genentech Current opinion BioDrugs, 2012

Reeve International Society for Quality of Life Research (ISOQOL); University of North Carolina

PCORI Review

Qual Life Res, 2013


Characteristics of Identified Publications (2/2) Original research was conducted in only two articles: a survey of PPRN PIs and a comparative effectiveness study

First author

Author Affiliation Funding source Article type Journal, Year of publication

Rich Center on Health Care Effectiveness, Mathematica Policy Research

None declared News & views J Comp Eff Res, 2012

Sorenson London School of Economics and political science

None declared Review J Health Polit Policy Law, 2014

Sox Geisel School of Medicine at Dartmouth

California HealthCare Foundation

Analysis & commentary Health Aff, 2012

Stefanou Department of Clinical Neuroscience, Institute of Psychiatry, King's College London

None declared Report Amyotroph Lateral Scler Fronto-temporal Degener, 2015

Snyder Johns Hopkins School of Medicine Academy Health Review Med Care, 2013

Westrich Social &scientific systems Inc. None declared Conference abstract Value Health, 2014

Wicks PatientsLikeMe None declared Conference abstract Mult Scler, 2014

Wicks PatientsLikeMe None declared Original article: CER Nat Biotech, 2011

Wu Center for Health Services and Outcomes Research, Johns Hopkins University

National Cancer Institute; Agency for Healthcare Research & Quality

Review & recommendation J Clin Epi 2013


Thematic Analysis (1/3) Themes were extracted from the 36 quotes classified as relating to future use

Thematic analysis results Examples

15 themes were identified as shown in the graph below

0 2 4 6 8 10 12 14 16 18

PPRN impartiality and independence

EMR

Real-world

Large network

Standardise capture

Surveillance

Cost-effectiveness/payer

Rich data/wide scope

Sustainability

PRO

Improve evidence qulity

Stakeholder/patient involvement

Improve health care/clinical practice

CER

Decision making

Frequency of theme detection

“…comparative effectiveness assessments that can be readily disseminated to, and used by, a

wide range of health care stakeholders, especially providers and patients.”

“PROs have the potential to improve the quality and patient-centeredness of medical

care in a variety of ways””

“Comparative effectiveness research is intended to provide better evidence to help patients, clinicians, purchasers, and policy

makers make informed health care decisions. Its wide scope makes it likely that it will be

used in diverse ways.”

“…Coupled with increased stakeholder involvement in the design and conduct of

studies, this may help ensure greater understanding of CER, use of the resulting

evidence in policy and practice, and commitment to its sustainability.”


Thematic Analysis (2/3) Themes were extracted from 33 quotes classified as relating to potential challenges

Thematic analysis results Examples

18 themes were identified as shown in the graph below

0 5 10 15

Confounding in observational studies

Perceived lack of impact of CER on cost…

Acceptability if evidence disrupts current…

May dictate access to treatment and…

Disease specific vs. disease-neutral

Data sharing

Informed consent

Perceived impartiality

Privacy/confidentiality

Sustainability

PROs-selection

Patient engagement

Priortise research questions

Scaling size of networks

Regulatory/governance

Perceived purpose for rationing care

Demonstrate impact

Standardisation across different EMRs


““Rapidly building a new national resource to facilitate a large-scale, patient-centered

CER is associated with a number of technical, regulatory, and organizational challenges””

“…past experiences suggest that while the concept of CER may be widely accepted in

principle, support may wane if resulting evidence challenges current policy and

practice.”

“Despite the promise of EHRs, considerable work remains before interoperable data will

be available across PCORnet- although efforts have been made to tabulate the data based on common elements, This approach

constitutes a substantial challenge, given the documented variability in EHRs both within

and among health systems”


Thematic Analysis (3/3) Ten quotes were classified according to the study’s objective-derived categories but did not

undergo thematic analysis within these categories due to low numbers

Objective-based categories Examples

0 2 4 6

Classification

Unmet need

Successful example


”clinical data research networks (CDRNs) based on the electronic health records and other electronic sources of

very large populations receiving healthcare within integrated or networked delivery systems and patient-

powered research networks (PPRNs) built by communities of motivated patients, forming partnerships with

researchers.”

“A popular paper on the effects of lithium in amyotrophic lateral sclerosis included a report on a clinical trial in

humans. Subsequently, a follow-up study using patient-reported data did not confirm the original result, opening

a scientific debate on how patient-reported data could contribute to neurological research”

“Existing estimates suggest that less than half of medical care in the United States is based on or supported by

evidence about its effectiveness, often resulting in care that is inappropriate and unnecessary”


Successful Example of the Use of a PPRN for CER (1/4) • Amyotrophic lateral sclerosis (ALS) is a progressive disease without effective therapy with a

median survival time from the onset of symptoms of 2-5 years.

• In 2008, a small non-randomised trial in 44 patients reported that lithium carbonate slowed

down the progression of ALS.

• This apparent beneficial effect prompted ALS patients to self-medicate with lithium.

• PatientsLikeMe subsequently built a tool to collect data in all 348 registered ALS patients who

began taking lithium off-label.

Wicks P, Vaughan TE, Massagli

MP, Heywood J. Accelerated clinical discovery using self-

reported patient data collected online and a patient-matching

algorithm. Nat Biotchnol. 2011;29(5):411-4.


Observational study with historical control patients, using a matching algorithm to select a control

group matched on pre-treatment Functional Rating Scale (FRS) progression

Successful Example of the Use of a PPRN for CER (2/4)

Study design

4,318 ALS patients on PatientsLikeMe (28 February 2010)

3,674 (85%) provided at least basic demographic and diagnosis data

348 (9%) reported taking lithium

149 ‘intent to treat’ (lithium ≥2 months)

78 ‘full course’ (lithium for all 12 months)

447 intention-to-treat-matched controls

390 full course-matched controls

3,326 (91%) did not report taking lithium


There was no difference in ALS progression at 12

months between patients who used

lithium and those who did not.

The apparent beneficial effect of lithium carbonate reported in the original trial was not

corroborated in the CER study using PPRN data


ALS disease progression was assessed with the Revised ALS Functional Rating Scale (ALSFRS-R12), which measures patient-reported functional impairment in domains such as speech,

swallowing, walking, arm function and respiratory function. ITT, intention to treat. (Diagrams above were taken from Wicks et al., Nat Biotchnol. 2011.)

ITT analysis Full course analysis


• Results of this study have been repeated in subsequent studies.

• This study prompted a scientific debate on how patient-reported data could

contribute to neurological research.

• Of note, lithium is not licensed as a treatment for ALS.

• The study provides evidence to strengthen the argument that PPRN data could

become a valuable and potentially reliable source of information.


Impact


Results – Desktop Research


460 hits identified by searching Google

377 unique hits after removing duplicates

79 hits identified PPRNs

33 unique PPRNs met the definition and were included in the study*

2 additional PPRNs identified by searching MEDLINE, EMBASE and

Google Scholar

PPRNs identified by the pragmatic literature review

PPRNs identified by the desktop research

83 duplicate hits

298 hits did not identify PPRNs

48 duplicate PPRNs removed

Thirty-three Unique PPRNs Were Identified and Included in the Analysis

* Characteristics are shown on the next 2 slides


No. PPRN Name Launch Year

Country Disease/condition PCORI Sponsored?

1 PatientsLikeMe 2005 US All conditions No

2 Interactive Autism Network 2007 US Autism Yes

3 CureTogether 2008 US All conditions No

4 Patients Know Best 2008 UK All conditions No

5 Crohnology 2011 US Crohn’s disease and ulcerative colitis No

6 Esperity 2012 Belgium Cancer No

7 ABOUT Network 2013 US Breast and ovarian cancer Yes

8 AR-PoWER 2013 US Arthritis and related conditions Yes

9 CCFA Partners 2013 US Crohn’s disease and ulcerative colitis Yes

10 REN 2013 US Epilepsy Yes

11 CENA 2013 US All conditions Yes

12 COPD PPRN 2013 US COPD Yes

13 DuchenneConnect 2013 US Duchenne and Becker muscular dystrophy Yes

14 Health eHeart Alliance 2013 US Cardiovascular health Yes

15 ImproveCareNow 2013 US Paediatric Crohn’s disease and colitis No

16 NephCure Kidney International 2013 US Primary Nephrotic Syndrome Yes

List of 33 Eligible PPRNs (1/2)


No. PPRN Name Launch Year

Country Disease/condition PCORI Sponsored?

17 PARTNERS 2013 US Rheumatic disease Yes

18 Phelan-McDermid Syndrome Data Network 2013 US Phelan-McDermid Syndrome Yes

19 PI-CONNECT 2013 US Primary immunodeficiencies Yes

20 VPPRN 2013 US Vasculitis Yes

21 Citizen Pscientist 2013 US Psoriatic disease Yes

22 MyApnea 2013 US Sleep apnea Yes

23 PEER 2014 US Any disease/condition No

24 ALD Connect 2014 US X-linked adrenoleukodystrophy Yes

25 CPPRN 2015 US Behavioural health Yes

26 MOOD 2015 US Depression and bipolar disorder Yes

27 iConquerMS 2015 US Multiple sclerosis Yes

28 AD PCPRN 2015 US Alzheimer's disease and dementia Yes

29 PRIDEnet 2015 US Any disease/condition Yes

30 MyLymeData 2015 US Lyme disease Yes

31 iCureCeliac 2016 US Celiac and gluten intolerance Yes

32 Cloudy with a chance of pain 2016 UK Chronic pain and arthritis No

33 100 For Parkinson’s 2016 UK Parkinson’s disease No

List of 33 Eligible PPRNs (1/2)


• The search identified 29

US and 4 European

PPRNs

• Of the 29 US PPRNs, 24

were PCORI funded

• Of the 4 European

PPRNs, 3 were based in

the UK

0

2

4

6

8

10

12

14

16

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

Nu

mb

er o

f P

PR

Ns

Launch Year

US Other

US PCORI-funded

Europe

The Majority of PPRNs Were from the US and Funded by PCORI


• Most PPRNs are able to collect patient-

reported data

• 58% of the PPRNs collected patient-

reported data without EMR

• 36% of the PPRNs collected patient-

reported data and EMR

• No PPRN collected only EMR data

• Some PPRNs offer the option to upload

additional data (e.g. test results) or

connect to electronic devices such as Fitbit

and mobile apps (‘Additional’ in graph)

16

3

8

4

2

0

2

4

6

8

10

12

14

16

18

Standard Additional Standard Additional

Patient-reported dataonly

Patient-reported dataand EMR data

Unknown

Nu

mb

er o

f P

PR

Ns

Type of Data Collected

The PPRNs Contained a Rich Mixture of Patient-reported and EMR Data


• There were 6 PPRNs covering

any or multiple disease areas

and 6 PPRNs for diseases of

the nervous system

• An example of a PPRN

covering multiple diseases is

PatientsLikeMe, which any

patient can join, regardless of

their disease

• The disease areas which

PPRNs are specific to are all

chronic conditions Diseases are grouped according to ICD-10 chapter. Percentages shown

are relative to the total number of PPRNs under study (n=33).

1 (3%)

1 (3%)

1 (3%)

1 (3%)

1 (3%)

1 (3%)

1 (3%)

1 (3%)

2 (6%)

3 (9%)

4 (12%)

4 (12%)

6 (18%)

6 (18%)

0 1 2 3 4 5 6 7

Endocrine, nutritional and metabolic

Infectious/Parasitic

Blood related

Circulatory system

Respiratory system

Skin and subcutaneous tissue

Genitourinary system

Chromosomal abnormalities

Neoplasms

Muscoloskeletal and connective tissue

Mental, behavioural & neurodevelopmental

Digestive system

Nervous system

Multiple diseases

Number of PPRNs

PPRNs Most Commonly Collected Data for Multiple Diseases and Diseases of the Central Nervous System


Percentages shown are relative to the 30 accessible PPRNs.

Demographics include age, gender, ethnicity, country, weight and height. Among the

PPRNs that collect data on demographics

(87%), the range in the number of collected

demographic variables was 3 to 6. Patient-

reported QoL includes non-standard symptom

reporting.

13 (43%)

30 (100%)

26 (87%)

12 (40%)

1 (3%)

12 (40%)

4 (13%)

27 (90%)

8 (27%)

26 (87%)

15 (50%)

14 (47%)

0 5 10 15 20 25 30 35

Patient

EMR

Specific current/past medications

Dosage and/or frequency

Dispensing (e.g., potential pharmacy link)

Concurrent medication

Adherence

Patient-reported quality of life (QoL)

Drug-related adverse events/side effects

Demographics

Co-morbidities

Family history

Sou

rce

of

dat

aM

ed

icat

ion

Ou

tco

mes

Co

vari

ates

Number of PPRNs

Medication, Outcomes & Covariates Were Captured to Varying Degrees


* PPRNs are ranked in descending order of CER score. Only PPRNs that had an accessible website are included (three PPRNS were not accessible: CPPRN, ImproveCareNow and PRIDEnet). CER points may not have been given due to the following reasons:

o Information could not be obtained due to limited access to the website; o PPRN collects data on various conditions using tailored questionnaires sent out at different times; o PPRN uses EMR data and therefore does not require patients to self-report. For example, prescribed medication name, dosage and frequency; o Some conditions are not generally treated using medication and therefore do not gain points for therapy.

0

1

2

3

4

5

6

7

8

9

Use

fuln

ess

fo

r C

ER s

core

Covariates

Outcomes

Therapy

Classification of 30 Accessible PPRNs According to Their CER Score*


Classification of the 22 Accessible PPRNs Funded by

PCORI Based on Their CER Score

0

1

2

3

4

5

6

7

8

9

Use

fuln

ess

fo

r C

ER s

core

Covariates

Outcomes

Therapy

• Points may not have been given due to the following reasons:

o PPRNs which scored zero due to lack of identified/accessible website. These were removed (CPPRN and PRIDEnet); o Information could not be obtained due to limited access to the website; o PPRN collects data on various conditions using tailored questionnaires sent out at different times; o PPRN uses EMR data and therefore does not require patients to self-report. For example, prescribed medication name, dosage and frequency; o Some conditions are not generally treated using medication and therefore do not gain points for therapy.


Scoring to Signify Whether Distinct PPRNs are Useful for CER Based on the Criteria Detailed on Slide 13

0

1

2

3

4

5

6

7

8

9

Use

fuln

ess

fo

r C

ER s

core

s Therapy typeEffectiveness data from PRO or EMRRemaining points

Based on our scoring criteria, only two PPRNs collect therapy type along with effectiveness data from EMRs or PROs as well as scoring a total of 8 of more points. These are Patientslikeme and Patients Know Best. •Points may not have been given due to the following reasons:

o PPRNs which scored zero due to lack of identified/accessible website. These were removed (CPPRN, ImproveCareNow and PRIDEnet); o Information could not be obtained due to limited access to the website; o PPRN collects data on various conditions using tailored questionnaires sent out at different times; o PPRN uses EMR data and therefore does not require patients to self-report. For example, prescribed medication name, dosage and frequency; o Some conditions are not generally treated using medication and therefore do not gain points for therapy.


*Please see slide 46 for embedded file (Desktop research data extraction excel file) to see the appropriate fields that were used to allocate points.

Breakdown of CER scores by PPRN


* Three inaccessible PPRNs were excluded (CPPRN, ImproveCareNow and PRIDEnet)

Summary of CER scores Median (25th, 75th percentiles) CER Score*

Funding Type (no. of PPRNs)

Therapy Outcome Covariates Overall

PCORI (22) 1.0 (1.0, 1.9) 2.0 (1.0, 2.0) 1.6 (0.7, 2.5) 4.5 (3.6, 5.6)

Non-PCORI (8) 3.0 (2.0, 4.0) 2.0 (1.0, 2.0) 1.8 (1.3, 2.4) 6.6 (5.5, 7.5)


Caveats and Limitations


Caveats and Limitations: Pragmatic Literature Review

• The pragmatic literature review identified only a limited number of publications, with

only one CER study using PPRN data. As most PPRNs have only recently been launched,

this number is expected to increase in the near future.

• Because of the lack of a universally consistent term for PPRNs and because the search

was conducted in English only, not all existing European PPRNs may have been

identified in the desktop research.

• Most of the authors of the papers were proponents of PPRNs. This may mean that the

thematic analysis reflects more positive views about PPRNs and negative themes may

be underrepresented. This may be addressed by phase 2 of the project (a survey

among key-opinion leaders and healthcare decision makers), where a more even

spread of respondent types is expected.


Caveats and Limitations: Desktop Research

• Even though the Desktop Research used a wide range of terms to identify possible

PPRNs, the search was conducted in English only and any PPRNs in different languages

or with names not covered by the search strategy may have been missed by the search.

• Some PPRNs could not be fully characterised because of missing information on

different data fields, which limits the ability to accurately classify PPRNs for their

usefulness for CER.


Conclusions


• The vast majority of publications on PPRNs were reviews or reports expressing views

and opinions.

• Only one study reported on CER using PPRN data (Wicks et al., 2011). However, the

comparator treatment(s) were not recorded for that study, so it is unclear whether

control patients received any treatment for ALS.

• There is an ongoing debate in the literature about standardising and harmonising PRO

data as well as EMR data for CER.

• Though the outlook to use PPRN data for CER is generally positive, it is still approached

with caution and critical attitude.

Conclusions from the Pragmatic Literature Review


Conclusions from the Desktop Research • The majority of the 33 identified PPRNs were from the US, with most of these funded

by PCORI. Only four European PPRNs were identified.

• The identified PPRNs cover a wide range of disease areas (accounting for 13 out of all

23 ICD-10 disease chapters). Most PPRNs cover any or multiple diseases and diseases of

the nervous system.

• All identified PPRNs collect patient-reported data, with only some of them also

collecting EMR data. This is consequential because PPRNs may be most useful for CER

when they contain both data sources.

• The identified PPRNs collect at least some information considered essential for CER,

e.g., current treatments, outcomes and demographics.


• However, other important variables are inconsistently collected, including dosage and

frequency of pharmacological treatment, medication adherence and co-morbidities.

• This is reflected by the 2 out of 30 accessible PPRNs (6.7%) which are sufficient for CER

• The lack of EMR data in most PPRNs may hamper CER when the outcomes are clinical

measures.

• The usefulness of PPRN data for CER would also increase if more aspects of drug

exposure and covariates were collected in the PPRN.

• Non-PCORI PPRNs had higher scores than PCORI PPRNs, which is remarkable given that

PCORI funds PPRNs for their ability to participate in research.

Further conclusions from the Desktop Research




Title Embedded file

List of full-text publications included in the Pragmatic Literature Review

List of publications excluded from the Pragmatic Literature Review

List of PPRNs and URLs included in the Desktop Research

Desktop Research data extraction Excel file



References of Publications Included in the Pragmatic Literature Review


1. Ahmed, S., Berzon, R., & Revicki, D. The use of patient-reported outcomes (PRO) within

comparative effectiveness research: implications for clinical practice and health care policy. Med

Care. 2012; 50(12): 1060-1070.

2. Ali, J., Califf, R., & Sugarman, J. Anticipated Ethics and Regulatory Challenges in PCORnet: The

National Patient-Centered Clinical Research Network. Account Res. 2016; 23(2): 79–96.

3. Califf, R. The Patient-Centered Outcomes Research Network A National Infrastructure for

Comparative Effectiveness Research. N C Med J. 2014; 23(2): 79-96.

4. Daugherty, S; Wahba, S; Fleurence, R. Patient-powered research networks: building capacity for

conducting patient-centered clinical outcomes research. J Am Med Inform Assoc. 2014; 21(4):

583-586.

5. Fleurence, R, Curtis L, Califf, R, Platt, R, Selby, Brown, J. Launching PCORnet, a national patient-

centered clinical research network. J Am Med Inform Assoc. 2014; 21(4): 578-582.

6. Garber, A. How the patient-centered outcomes research institute can best influence real-world

health care decision making. (2011). Health Aff. 2011: 30(12): 2243-2251.

References


7. Kupferschmidt, B; Lavallee, D; Tambor, E; Desai, P; Tunis, S. (2012). Establishing principles and

indicators for collaborative research (EPIC): Results of pilot work. Value Health. 2012;

8. Levesque, M. Biologic rheumatoid arthritis therapies: Do we need more comparative

effectiveness data? (2012). BioDrugs. 2012; 26(2): 65-70.

9. Reeve, B; Wyrwich, K; Wu, A; Velikova, G; Terwee, C; Snyder, C; Schwarz, C; Revicki, A; Moinpour,

C; Mcleod, L; Lyons, J; Lenderking, W; Hinds, P; Hays, Ron; Greenhalgh, J; Gershon, R; Feeny, D;

Fayers, P; Cella, D; Brundage, M; Ahmed, S; Aaronson, N; Butt, Z. ISOQOL recommends minimum

standards for patient-reported outcome measures used in patient-centered outcomes and

comparative effectiveness research. Qual Life Res. 2013; 22(8): 1889-1905.

10. Rich, E. From Methods to Policy: Past as prologue: How comparative effectiveness research

became patient-centered outcomes research. J Comp Eff Res. 2012; 1(6): 475-477.

11. Sorenson, C; Gusmano, M; Oliver, A. The politics of comparative effectiveness research: lessons

from recent history. J Health Polit Policy Law. 2014; 39(1): 139-170.


12. Sox, H.The patient-centered outcomes research institute should focus on high-impact problems

that can be solved quickly. Health Aff. 2012; 31(10): 2176-2182.

13. Stefanou, M-I; Amygdalos, I. Patient-led research in amyotrophic lateral sclerosis: Quo vadis?

Amyotroph Lateral Scler Frontotemporal Degener. 2015: 16(5-6): 418-422.

14. Snyder, C., Jensen, R., Segal, J., & Wu, A. Patient-reported outcomes (PROs): putting the patient

perspective in patient-centered outcomes research. MedCare. 2013; 51(18 Suppl 3): S73-9.

15. Westrich, K.D, Schur C., Adams, A. The state of the comparative effectiveness research (CER)

environment: Surveys of stakeholders and influentials. Value Health. 2014; 17(3): A23.

16. Wicks, P. Patients as partners in clinical research - Does the Internet harm or help? Mult Scler.

2014; 20(S1): 65-66.

17. Wicks, P., Vaughan, T. E., Massagli, M. P., & Heywood, J. Accelerated clinical discovery using self-

reported patient data collected online and a patient-matching algorithm. Nat Biotech. 2011;

29(5): 411-414.


18. Wu, A; Kharrazi, H; Boulware, L; Snyder, C. Measure once, cut twice - Adding patient-reported

outcome measures to the electronic health record for comparative effectiveness research. J Clin

Epidemiol. 2013; 66(8 0): S12-S20.


Appendix


Appendix: Quotes About Future Use of PPRNs (1/4) Author Title Quotes

Ahmed

The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy

“PROs provide a standardized method of capturing patient perspectives and experiences”

“...value of PROs within CER, types of measures that are likely to be useful in the CER context and criteria for selecting appropriate PRO measures, and key challenges associated with using PROs in CER. We recommend how to select and apply PROs in research, clinical Practice, and population surveillance”

“Standardized and defined strategies for integrating PROs across clinical settings and research studies will facilitate CER in “real-world” situations. In turn, this will likely increase the uptake of findings in patient, clinical, payer, and health policy decision making”

Califf

The Patient-Centered Outcomes Research Network A National Infrastructure for Comparative Effectiveness Research

“The success of PCORnet will provide an unprecedented platform for both observational studies and clinical trials. The massive size and distributed design of PCORnet should make it possible to improve the quality of published observational studies. Methods can be considered and refined by a national “brain trust” to control as best as possible for the many issues that often cause observational studies to have irreproducible results. When an interesting result is observed, it should be possible to rapidly attempt to replicate that finding in a different network.

Garber How the patient-centered outcomes research institute can best influence real-world health care decision making

“Assessments will typically contain comprehensive, formally structured reviews of the available evidence, culminating in clear statements about what we know about the effectiveness of the interventions under consideration... in an impartial and complete manner.”

“These[loosely coordinated/duplicative research, lack of scale, expense of large-scale clinical trials, lack of uniformity in research methods, rating standards, assessment approaches, or reporting.] are problems that the ... Institute will address, but it will not entirely replace the work of... other government and private groups ... likely to build on the institute’s work to meet their own needs, for instance by adding analyses of cost-effectiveness.”

“investments in comparative effectiveness research can lead to high-value health care through the following cycle: Comparative effectiveness assessments inform clinical practice guidelines and outcome metrics, which in turn inform more effective payer coverage and payment policies, which in turn encourage providers and patients to make better care decisions. Comparative effectiveness research holds great promise for improving health and health care in the United States, but a large effort will be necessary to ensure that its potential is realized. I believe that a central challenge is to produce comparative effectiveness assessments that can be readily disseminated to, and used by, a wide range of health care stakeholders, especially providers and patients. The Patient-Centered Outcomes Research Institute can lead government and private organizations in meeting that challenge.”

“Comparative effectiveness research is intended to provide better evidence to help patients, clinicians, purchasers, and policy makers make informed health care decisions. Its wide scope makes it likely that it will be used in diverse ways”

“Constructing a comparative effectiveness assessment “I propose that a comprehensive assessment include three main components: a clinical comparative effectiveness assessment; an analysis of economic considerations; and a discussion of broader clinical and nonclinical considerations”

“A comprehensive assessment should also include a set of concise but not oversimplified ratings of the quality of evidence—for example, superior, modestly superior, comparable, modestly inferior, inferior—and a discussion of the level of uncertainty or confidence corresponding to each component”



Daugherty Patient-powered research networks: building capacity for conducting patient-centered clinical outcomes research

[Potential impact on patients] ”PPRNs are providing an unprecedented opportunity for patients, clinicians, and researchers to work together in partnership in all stages of building network capacity. Despite daunting challenges, the PPRNs are supported by a robust peer network from which to exchange knowledge, build a set of promising practices, and adopt or enhance novel methods. As aresult of PPRN efforts, PCORnet will be well-positioned to support meaningful and relevant patient-centered outcomes research that will further inform healthcare decision-making and improve healthcare outcomes.

Fleurence Launching PCORnet, a national patient-centered clinical research network

[USEFULNESS & FUTURE ROLE] “The Patient-Centered Outcomes Research Institute (PCORI) has launched PCORnet, a major initiative to support an effective, sustainable national research infrastructure that will advance the use of electronic health data in comparative effectiveness research (CER) and other types of research.”

Levesque Biologic rheumatoid arthritis therapies: Do we need more comparative effectiveness data?

“An added feature of comparative effectiveness studies in usual care settings is the likelihood that novel trial designs comparing the effectiveness of biologic DMARDs can be performed at significantly lower expense.”

“Therefore, while there is currently relative lack of comparative effectiveness research to inform clinical decisions about biologic DMARDs for RA patients, it appears likely that there will be wider availability of such data in the near future to improve clinical care.”

Reeve

ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research

“This initiative’s goal was to identify minimum standards for the design and selection of a PRO measure for use in PCOR and CER”

“To draw valid research conclusions regarding patient-centered outcomes, PROs must be measured in a standardized way using scales that demonstrate sufficiently robust measurement properties”

“Although not the primary objective of this study, our approach allowed us to also identify criteria that were not deemed as a necessary minimum standard, but would rather be considered ‘‘best practice’’ standards for PRO measures”

“The development of these minimum measurement standards is intended to promote the appropriate use of PRO measures to inform PCOR and CER, which in turn can improve the effectiveness and efficiency of healthcare delivery.”

“The identification and selection of PRO measures meeting and exceeding these current ISOQOL recommended minimum standards will increase the likelihood that the evidence generated in PCOR and CER reliably and validly represents the patients’ perspective on health-related outcomes.”

“This PRO evidence, based on instruments with sound measurement properties, can then be used to inform clinical and health policy decision making about the benefits and risks associated with different health interventions or to monitor population health.”

Rich

From Methods to Policy: Past as prologue: How comparative effectiveness research became patient-centered outcomes research

[FUTURE ROLE] “Various policy makers had noted that far too few studies provided clear answers regarding the proper role of innovations in typical clinical settings, resulting in systematic reviews of available evidence that were often not able to provide clear guidance on important clinical questions. Prominent analyses observed that many authoritative clinical practice guidelines relied on weak evidence, and even widely applied quality metrics incorporated controversial assumptions regarding benefits of interventions in under-studied populations.”

“Clearly, health¬care researchers must do a better job of find¬ing clear answers to the tough questions that patients and their clinicians grapple with every day. Fortunately, substantial new resources and infrastructure are now available in the USA to help investigators with this challenging work.”

Snyder Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research

“PROs have the potential to improve the quality and patient-centeredness of medical care in a variety of ways”

“Integration of PROs in PCOR requires linking PRO data with clinical information from a range of data sources. Electronic data methods play a pivotal role in executing this linkage”



Sorenson The politics of comparative effectiveness research: lessons from recent history

“linking PCORI’s financial support to a separate fund derived from both public and private revenue streams will help protect it from the whims and unpredictability of the annual budget process... the fact that the institute is independent from government may go a long way in providing an offense and defense against potential claims that its work is simply a guise for government control over health care decisions. The fact that PCORI’s leadership and committees contain a diverse set of representatives from multiple federal agencies and other organizations may also help improve coordination across different CER-related activities...although PCORI does not have an ‘‘authoritative voice’’ over direct applications of CER in policy and practice, it does have significant influence over determining the research agenda and methodological standards, and in funding both public- and private-sector comparative studies... Given these challenges and the experiences of past initiatives, it will be important to continuously promote the value of CER over time. It is natural to advocate the virtues of policy initiatives in the beginning to gain support and momentum, but such efforts will need to be maintained to reemphasize the benefits of the research and proactively address any false impressions held by the public and other stakeholders, especially as existing evidence”

“to design CER studies so that the research addresses questions most important to patients, providers, and policy makers, and to tailor communication of the findings to the needs of different end-users. Coupled with increased stakeholder involvement in the design and conduct of studies, this may help ensure greater understanding of CER, use of the resulting evidence in policy and practice, and commitment to its sustainability”

“Comparative effectiveness research offers significant promise to provide better information on available health care interventions and methods of care delivery and thus ultimately to improve health care decision making and practice. However, as evidenced by previous related experiences that sought to meet similar aims, it is not without challenges. While much has already been done to support an integral and sustained role for CER in the American health care system, considerable efforts will be required in the days ahead to ensure the success and sustainability of PCORI and the overall CER effort.”

“In such cases, the research (and general overall approach) is seen as a potential threat to the interests of vested stakeholders, namely, organized medicine, industry, and patient advocacy organizations—groups who typically possess significant resources to influence policy makers”

“To guard against future losses of political support and funding, it may be advisable to position and integrate CER as part of a wider range of reform initiatives to improve the quality and value of US health care”

Sox

The patient-centered outcomes research institute should focus on high-impact problems that can be solved quickly

“Patient-Centered Outcomes Research Institute (PCORI) to help patients, clinicians, and policy makers make well-informed decisions about health care. Because its funding expires in 2019, the institute has little time in which to produce timely, practice-changing results that will build public support for comparative effectiveness research. PCORI should plan its research agenda strategically, so that it addresses research questions that comparative effectiveness research could answer quickly and decisively. To date, the institute has not chosen this path. In January 2012 PCORI’s first research agenda described broad research priorities rather than specific clinical questions. The institute must drive the burgeoning discipline of comparative effectiveness research forward, starting with a research project agenda that conveys a sense of urgency and strategic direction”

“Patients, researchers, payers, and policy makers expect much from comparative effectiveness research because it is trying to address unmet needs of patients who want to understand the choices that they face in an increasingly complicated health care system”



Stefanou Patient-led research in amyotrophic lateral sclerosis: Quo vadis?

“Indicative of these inherent inconsistencies of patient-led trials is the 2014 follow-up paper by PatientsLikeMe, which correctly predicted that dexpramipexole had no clinically significant benefit, but also incorrectly predicted a beneficial effect from NP001. In the latter case and according to the phase II clinical trial by Neuraltus, NP001 had no statistically or clinically significant beneficial effect in ALS patients. Nonetheless, patient-led trials could become a valuable and potentially reliable source of information (demographic data, adverse effects, symptoms, quality of life or disease scales ratings), convenient, cheap and direct in highlighting issues that require further clinical assessment”

“patient-led trials can operate as a surveillance mechanism facilitating both accelerated data collection and preliminary testing of the treatments’ efficacy”

Westrich The state of the comparative effectiveness research (CER) environment: Surveys of stakeholders and influentials

“Health care stakeholders recognize the importance of CER but continue to believe, as in previous surveys, that significant impact of CER on treatment decisions is still in the future”

Wicks Patients as partners in clinical research - Does the Internet harm or help?

“Fortunately, the advent of “Patient Powered Research Networks” (PPRNs) has advanced the field considerably in a short space of time. What was once a series of narrative anecdotes swapped between anonymous message boards and blogs is now a structured quantitative dataset supported by medical ontology, with capture of relevant demographics, treatments, symptoms, and validated outcomes that can be combined together to create datasets high enough in quality to be published in peer-reviewed scientific journals”

Wicks Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm

“There are several potential advantages of collecting patient-reported outcome data online” Ø The first is speed Ø The second is patient access. There is a potential to rapidly recruit widely dispersed patients with rare conditions and to overcome selection bias favouring patients living near specialist centres Ø The third is availability of control participants Ø The fourth is cost Ø The fifth is patient engagement


Appendix: Quotes About Potential Challenges (1/3) Author Title Quotes

Ahmed

The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy

“we discuss the value of PROs within CER, types of measures that are likely to be useful in the CER context, PRO instrument selection, and key challenges associated with using PROs in CER”

Ali

Anticipated Ethics and Regulatory Challenges in PCORnet: The National Patient-Centered Clinical Research Network

“The most pressing ethics issues identified related to informed consent, patient engagement, privacy and confidentiality, and data sharing. High priority regulatory issues included IRB coordination, privacy and confidentiality, informed consent, and data sharing”

Califf


“Major challenges: ...extraction and harmonization of data extracted from multiple different EHRs, creation of a common data model, ethical and regulatory issues posed by cluster-randomized designs and comparative effectiveness studies, and methods for engaging patients and consumers. The task forces are working hard to address these challenges, but due to the short timeline for development of the network, some choices about how to organize a clinical research network must be made in the absence of empirical proof of best practices. For example, should a central institutional review board (IRB) be used, or should IRB reciprocity be required or encouraged? In most cases, a range of options will be available, although in some key areas a PCORnet-wide standard approach will be needed to construct a functional national network”.

Califf


“Despite the promise of EHRs, considerable work remains before interoperable data will be available across PCORnet”- although efforts have been made to tabulate the data based on common elements, “This approach constitutes a substantial challenge, given the documented variability in EHRs both within and among health systems”

“Even after data are collected- “Unfortunately, research regarding many important questions will inevitably be disruptive to health systems, and a number of research procedures—including consent—are seen as intrusive, expensive, and burdensome to the clinical care system“”

Daugherty Patient-powered research networks: building capacity for conducting patient-centered clinical outcomes research

“enrolling a diverse and representative membership, engaging participants in the governance of their Research networks, designing the data infrastructure and data Collection approaches (including enabling participants to collect and use their own data and standardising the collection of patient health data and mapping to the PCORnet common data model), engaging patients in sharing their data securely while protecting privacy, engaging patients to prioritise research questions and disseminate research results, scaling small networks into a larger network and identifying pathways to sustainability.”


“Rapidly building a new national resource to facilitate a large-scale, patient-centered CER is associated with a number of technical, regulatory, and organizational challenges”




“Rapidly building a new national resource to facilitate a large-scale, patient-centered CER is associated with a number of technical, regulatory, and organizational challenges”

Garber How the patient-centered outcomes research institute can best influence real-world health care decision making

[FEASIBILITY] “Successfully merging ... [to perform] comparative effectiveness assessments and applying them to clinical practice will require close cooperation among several entities”

[LIMITATION] “to inform payers’ decisions about coverage, reimbursement, and utilization. And even though such research is often helpful to providers and patients, they may view even well-designed, well-reported research with mistrust when it is sponsored or conducted by payers and other organizations perceived to be acting in self interest.”

“Most important, the institute is responsible for ensuring that studies are applicable to real world clinical decisions”

Reeve

ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research

[LIMITATION/CHALLENGES] “The challenge for PCOR and CER is how to best capture patient-reported data in a way that can inform decision making in healthcare delivery, research, and policy settings”


“While CER has generally captured bipartisan support, detractors have raised concerns that it will be used to ration services and heighten government control over health care.”

“Many unresolved questions remain, including how CER will be implemented and operationalized in practice and, ultimately, what impact it will have on its intended aims to improve the quality of health services, eliminate inappropriate and wasteful care, and ‘‘bend the cost curve.’’”

“Some of the key policy questions fundamental to these discussionswere issues related to the governance, structure, and funding of such an entity. Of particular importance was whether it would be connected with or independent of government and what role the research would have in Medicare coverage decisions and other areas of policy, such as the development of clinical practice guidelines.”

“Such opposition centered on fears that CER would restrict patients’ access to care and physicians’ autonomy and threaten biomedical innovation”

“the bill language prohibited the use of quality-adjusted life years (QALYs), a metric used in cost-effectiveness to measure net health gain, as well as the use of research findings to dictate coverage, reimbursement or other policy recommendations”

“past experiences suggest that while the conce”pt of CER may be widely accepted in principle, support may wane if resulting evidence challenges current policy and practice.

“many people who supported the creation of previous agencies to carry out CER-related activities often heralded unrealistic aims about what the research could achieve and within what time frames”

“previous organizations often suffered from a lack of a single, authoritative ‘‘voice’’ in defining research agendas and offering guidance and recommendations”

“explicit consideration of costs has often placed agencies at threat by opponents, particularly those against the use of cost-effectiveness analysis in policy making”

[LIMITAIONS] “in the United States, where volume of care rather than value is rewarded, providers have had limited incentive to consider comparative effectiveness, quality, or costs of care in their practice decisions.”



Sox The patient-centered outcomes research institute should focus on high-impact problems that can be solved quickly

“Most were supportive of PCORI’s direction, but one theme recommended that PCORI focus on specific conditions or disease areas instead of proposing what PCORI called its “condition-neutral” research agenda”

Snyder Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research

“Challenges include the proprietary nature of PRO measures, governance related to setting standards, selection of tools for clinical use, confidentiality issues, and coordination among organizations in all of the above”

“There are a number of unanswered questions related to using PROs in PCOR, including how to select the PRO measure and interpret the results. Thousands of PROs have been developed and published. Which of these are appropriate for clinical use?”

“It must be determined how missing data are to be handled in analysis. Organizations should consider this question prospectively and establish standard/default procedures in consultation with experts in PROs and missing data”

Westrich

The state of the comparative effectiveness research (CER) environment: Surveys of stakeholders and influentials

“Only five percent of respondents believe the evidence base is sufficient to inform treatment decisions, and just 10 percent indicate that real-world evidence is being used to support decision-making”

“The environment for CER is changing, and PCORI is recognized as a key organization in the spectrum of activities related to CER. The evidence base is not sufficiently complete to inform treatment decisions, and the influence of real-world evidence and variability in patient response are not yet apparent and warrant monitoring”

Wu

Measure once, cut twice - Adding patient-reported outcome measures to the electronic health record for comparative effectiveness research

“We pose the question, ‘‘what needs to be standardized around the collection of patient-reported outcomes in electronic health records for comparative effectiveness research?’’”

“To make such standardization a reality, harmonizing efforts are needed on multiple levels. First, much greater coordination is needed among the components of individual institutions”

“At each level, various degrees of standardization may be required to ensure that patient-reported outcome measures are collected, shared, and retrieved in a meaningful way. The standardization efforts should ensure that similar patient-reported outcome measures are available across organizations, and the results are interpreted similarly”

“It must be acknowledged that there is also a downside to standardization; there are advantages to tailored use of specifically selected questionnaires to meet institutional needs. Therefore, standardization should be conducted with care”

Wicks Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm

“Unlike randomized trials, which match the comparison groups on all possible confounding factors, subject only to chance variation, observational studies cannot control for unmeasured covariates”


Appendix: Other Quotes Author Title

Category based on study’s research objectives

Quote


CLASSIFICATION ”clinical data research networks (CDRNs) based on the electronic health records and other electronic sources of very large populations receiving healthcare within integrated or networked delivery systems9 and patient-powered research networks (PPRNs) built by communities of motivated patients, forming partnerships with researchers.”


SUCCESSFUL “previous initiatives in the United States point to the importance of ‘‘political champion(s)’’ or ‘‘political entrepreneur(s)’’ to gain support for CER, especially in terms of its use in policy and practice.

Stefanou Patient-led research in amyotrophic lateral sclerosis: Quo vadis?

SUCCESSFUL “A popular paper on the effects of lithium in amyotrophic lateral sclerosis included a report on a clinical trial in humans. Subsequently, a follow-up study using patient-reported data did not confirm the original result, opening a scientific debate on how patient-reported data could contribute to neurological research”

SUCCESSFUL

“Motivated by these promising findings, they designed a parallel-group randomized study on a total of 44 patients, with the treated group receiving lithium plus riluzole and the control group receiving riluzole alone” “Their findings, as published in the 12 February 2008 edition of PNAS, suggested that: within a follow-up period of 15 months all patients treated with lithium were alive and the disease progression (assessed with the aforementioned scales) was significantly slower in the treated group compared to the control group”

SUCCESSFUL

“PatientsLikeMe designed a lithium-specific data collection tool and incorporated it in their site. This invited patients to fill in their FRS scores, their symptoms and the body domains affected by ALS, the treatments they used (time of initiation/withdrawal) and also some demographic data. An adequate data set was collected by February 2010” “The outcomes at the end of the 12-month study showed no statistically significant difference in FRS scores or survival between patients taking lithium and controls”

SUCCESSFUL “the 2011 study by PatientsLikeMe managed to readily recognize lack of lithium effectiveness and became a paradigm for patient-led research in ALS”


UNMET NEED ”CER strains the current clinical research paradigm because of its emphasis on assessing effectiveness in typical care delivery settings, its requirement for very large study populations to study effectiveness heterogeneity, and, often, its need for treatments to be allocated by randomization.”

Levesque

Biologic rheumatoid arthritis therapies: Do we need more comparative effectiveness data?

UNMET NEED “while there is currently a relative lack of comparative effectiveness research to inform clinical decisions about biologic DMARDs for RA patients, it appears likely that there will be wider availability of such data in the near future”


UNMET NEED “With growing concerns over health care costs and the quality of care, determining which therapies, care strategies, delivery models, and public health programs are most effective makes good sense.”


UNMET NEED “Existing estimates suggest that less than half of medical care in the United States is based on or supported by evidence about its effectiveness, often resulting in care that is inappropriate and unnecessary”