Decisionmaking for rare diseases: developing evidence-based clinical practice guidelines for growth hormone therapy in Prader-Willi Syndrome using the EVIDEM framework

Michèle Tony MSc,1,2,3 Mireille M. Goetghebeur PhD,2,3,4 Hanane Khoury PhD,3 Monika Wagner PhD,3 Renaldo Battista MD, ScD,1,4 Cheri L. Deal PhD, MD4,5

1Health Administration, Medical Faculty, University of Montreal, Montréal, Québec, Canada; 2EVIDEM Collaboration, Dorval, Québec, Canada; 3BioMedCom Consultants inc., Dorval, Québec, Canada; 4Research Center, CHU-Sainte-Justine and 5Department of Pediatrics, Medical Faculty, University of Montreal, Montréal, Québec, Canada

Background & Objective• Clinical Practice Guidelines (CPGs) are meant to “assist practitioner and

patient decisions about appropriate healthcare for specific clinical circumstances”.1 To be more consistent and efficient, the process to generate them needs to be systematic in its assessment of evidence and in the development of questions and recommendations.2-4

• The Appraisal of Guidelines Research & Evaluation (AGREE) Collaboration provides an instrument to “appraise the quality of the process and reporting of CPGs development”.5,6

• The EVIDEM framework is a systematic approach to synthesize and consider evidence by decision criteria.7,8 It could provide a consistent structure to organize evidence and facilitate CPGs development, in agreement with the recommendations of the AGREE collaboration.

• Prader-Willi syndrome (PWS) is a rare and complex multisystem disorder, with serious long-term consequences. Use and coverage of growth hormone (GH) in patients with PWS vary widely across Canada and there is a need to clarify its benefits.

• Objective: To test a pragmatic framework (EVIDEM) to facilitate systematic development of evidence-based CPGs for the use of GH in patients with PWS.

Methodology• An extensive literature review was performed to identify available evidence

on GH for PWS for each criterion of the EVIDEM framework covering disease impact, therapeutic context, treatment outcomes (efficacy/effectiveness, safety, patient-reported outcomes), type of benefits and economic impact (MCDA Core Model) as well as ethical and contextual considerations (Contextual Tool).

• Given the complexity of the efficacy outcomes measures reported, the efficacy criterion was subdivided into sub-criteria for each type of outcome measured including growth, body composition, exercise tolerance, metabolic effects, bone health, cardiovascular health, psychomotor development and behavioral outcomes.

• Data was organized in evidence tables and synthesized; level and strength of evidence were assessed using respectively the Centre for Evidence-Based Medicine (CEBM) levels of evidence9 and EVIDEM data quality assessment instruments.

• To support the development of CPGs, in which, clinical aspects are the primary focus, the framework was adapted into a CPGs module and used to elicit questions for CPGs development; an interactive web information system was developed to facilitate the CPGs process.

Key Steps for Development of CPGs Using EVIDEM CPGs Module

ResultsCPGs Module • Based on the analysis of the literature and feedback from experts, CPGs

questions reflecting current issues in management of patients PWS were developed and organized using the adapted EVIDEM framework (Intervention & Overview, MCDA Core Model and Contextual Tool).

• This framework structure and questions identified will be used to structure the program of the workshop and to assign questions to CPGs working groups.

Clinical AspectsIntervention Overview

MCDA Core Model

Indication: 1. Do patients with PWS need GH testing: In infancy? In childhood? In adulthood? 2. What baseline evaluations need to be performed before GH treatment? Intervention duration: 3. For how long should GH therapy be pursued? Administration/Description: 4. What clinical lab tests or imaging studies need to be done to monitor treatment? 5. What doses should be used for GH therapy: In infants? In children and adolescents? In adults? 6. Is there an optimal level of circulating IGF-I to obtain with GH treatment? 7. Should GH dose be titrated to IGF-I, and if so, at what frequency? 8. What is the frequency of follow-up visits necessary to adequately monitor GH therapy? Comparator(s): 9. What other therapies/interventions have been tried in PWS?

Decision Criteria

Proposed questions

Disease impact D1 - Disease severity

10. What is the frequency of the various genetic subtypes among various populations?

11. How has evolution of our genetic testing methodology changed genetic subtype frequency?

12. Are all patients with PWS equally GH deficient? 13. Are there genotype-phenotype correlations relevant to specific to clinical

outcome measures targeted with GH therapy? Other correlations? 14. What are the important co-morbidities that need to be considered when

considering GH therapy? 15. What is the life expectancy of PWS subjects? 16. What are the major causes of death in PWS subjects?

D2 - Size of population

17. What is the birth incidence/prevalence of PWS?

Therapeutic context of intervention C1 - Clinical guidelines

18. Why are physicians divided in their belief about the benefits of GH therapy?

C2 - Comparative interventions limitations (unmet needs)

19. For each of the other therapies/interventions tried in PWS, what were: The specific outcomes? The efficacy per outcome? The safety/tolerability of the therapy/intervention?

20. What specific therapies/interventions have been tried concomitant to GH therapy?

21. What are the nutritional recommendations for: Infants with PWS? Children with PWS? Adolescents with PWS? Adults with PWS?

Intervention outcomes I1 - Improvement of efficacy/ effectiveness

22. What are the most important clinical outcome priorities when initiating GH therapy in subjects with PWS: In infancy? In childhood? In adolescence? In Adulthood?

23. What is the best way to measure GH effectiveness on: a. Growth g. Metabolic benefits b. Body composition h. Resting energy expenditure c. Motor development i. Cardiovascular status (infants and children) j. Bone health d. Neurological status k. QoL (specifically in intellectually- e. Physical activity disabled individuals) f. Muscle strength

24. What is the impact of other hormonal deficiencies on GH treatment? 25. Does response to GH vary by:

a. age at start of treatment d. degree of dietary control b. dose e. level of physical activity c. body composition at start

I2 - Improvement of safety & tolerability

26. What are the major serious adverse events of GH treatment of PWS subjects?

27. What is the evidence that GH treatment in PWS increases the risk of: a. Sleep apnea h. Oedema b. Sudden death i. Breast tenderness/enlargement c. Scoliosis j. Risk of infection d. Diabetes k. Joint pain e. Intracranial hypertension l. Neoplasia f. Epilepsy m. Arterial Hypertension g. Slipped capital femoral epiphyses n. Strole/intra-cranial bleeding

28. What is the tolerability of GH: In published clinical trials (dropouts rate)? In patient-reported data? In phase 4 trials? In smaller observational studies?

29. What are the main reasons given for patient withdrawal from clinical trials of GH in PWS?

Decision Criteria

Proposed Questions

I3 – Improvement of patient reported outcomes

30. What are the most significant benefits reported by patients and/or parents after GH treatment?

Type of benefit T1 - Public health interest (e.g., prevention, risk reduction)

31. Is there any risk reduction associated with GH treatment in patients with PWS?

T2 - Type of medical service (e.g., cure, symptom relief)

32. What are the least important, but significant, clinical outcomes to GH therapy?

33. What known GH effects have not been adequately studied in PWS?

Quality/uncertainty of evidence Q2 - Completeness and consistency of reporting evidence

34. What are the confounding variables that are difficult to control in PWS GH clinical trials?

35. What is the best way to report efficacy data according to current recommendations, and why?

36. Is there a place for a therapeutic trial of GH, and if so, how long before assessing GH effectiveness?

37. When sources of potential study bias are considered (adequate randomisation and blinding of patients and health professionals, adequate description of withdrawals and dropouts, provision of intention-to-treat analysis), what proportion of the clinical trials have a high risk of bias, ie, one or more of the previous criteria not met)?

38. What questions with regards to GH use in PWS require further study? 39. What are the major research areas with regards to PWS that need to be

addressed beyond issues of GH use?

Q3 - Relevance and validity of evidence

Economic burden of illness: 40. What are the major sources of healthcare costs related to the care of patients with PWS? 41. What are the major costs of treating morbid obesity? 42. What are the major costs of treating diabetes?

Decision Criteria

Proposed Questions

Economics of intervention E1 - Budget impact on health plan (cost of intervention)

43. What is the cost of GH treatment in patients with PWS? 44. What is the budget impact at the country level?

E2 - Cost-effectiveness of intervention

45. What is the cost-effectiveness of GH treatment in patients with PWS?

E3 - Impact on other spending (e.g., hospitalization, disability)

46. What are the economic consequences (beyond drug cost) of GH treatment in patients with PWS?

Decision Criteria

Proposed Questions

Ethical criteria

Et1 - Utility - Goals of healthcare

47. Is the use of GH in patients with PWS aligned with the mission and scope of healthcare systems?

Et2 - Efficiency - Opportunity costs & affordability

48. How do we prioritize resources for PWS care, and how does GH fit into this?

Et3 - Fairness - Population priority & access

49. Is access to GH therapy available to all PWS patients, and if not, why? 50. Are there issues of fairness in withholding GH treatment, or in targeting

specific sub-populations of PWS subjects for GH therapy?

Overall context

O1 - System capacity & appropriate use of intervention

51. How do we organize the comprehensive care of the PWS patient, to optimize GH treatment and particularly to decrease/prevent potential side effects?

52. What are the evidence-based steps that are needed to harmonize care of patients with PWS?

O2 - Stakeholder pressures/ barriers

53. Are there any pressures/barriers for the use of GH in patients with PWS?

O3 - Political/ historical context

54. Are there any specific political/historical context impacting the use of GH in patients with PWS?

Collaborative Web Site • A collaborative web site was built to facilitate the CPGs process including:

• Access to evidence and CPGs questions structured in a consistent systematic manner

• Interactive online validation by international experts (about 50 clinicians and other experts) of: synthesized data; quality assessments of evidence; and CPGs questions

• Reflection by CPGs working groups on assigned questions prior to the workshop

• Involvement of all CPGs workshop participants prior to the consensus workshop

• To facilitate knowledge transfer, validated synthesized data on GH for PWS and by-criteria CPGs process (CPGs module) will be made publicly available, hosted by the EVIDEM collaborative registry under a Creative Common license.10

Discussion and Conclusion• CPGs questions elicitation and organization was facilitated by the EVIDEM

framework, providing a pragmatic means to ensure systematic consideration of evidence for a wide range of criteria and associated issues.

• By clearly revealing current knowledge and gaps, the framework will facilitate developing evidence-based CPGs and identifying research needed for continued improvement of management of patient with PWS.

• Using the EVIDEM framework, a by-criteria HTA model, to develop CPGs is expected to facilitate alignment of clinical and policy decisionmaking and ultimately optimize resource allocation and healthcare system sustainability.

AcknowledgmentsWe would like to acknowledge the contributions of the members of the CPGs workshop organizing committee who participated in this study. This workshop is sponsored by the Growth Hormone Research Society and a complementary grant from the Foundation for Prader-Willi Research. Internal sources of support were provided by BioMedCom Consultants. Inc.

References1. Committee to Advise the Public Health Service on Clinical Practice Guidelines Institute of Medicine, Institute of Medicine. Field MJ, Lohr KN, editors. Clinical practice guidelines: directions for a new program. Washington, D.C.: National Academy Press; 1990. 2. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527-30. 3. Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Developing clinical guidelines. West J Med. 1999;170(6):348-51. 4. Oxman AD, Fretheim A, Schunemann HJ. Improving the use of research evidence in guideline development: introduction. Health Res Policy Syst. 2006;4:12. 5. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421-4. 6. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1):18-23. 7. Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and Value: Impact on DEcisionMaking - the EVIDEM framework and potential applications. BMC Health Serv Res. 2008;8(1):270. 8. Goetghebeur MM, Wagner M, Khoury H, Rindress D, Gregoire JP, Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decisionmaking framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8(1):4. 9. Centre for Evidence Based Medicine. CEBM levels of evidence version #2. On Centre for Evidence Based Medicine website. Available from: http://www.cebm.net/index.aspx?o=5653.10. EVIDEM Collaboration. Open access prototypes of the Collaborative registry. On EVIDEM Collaboration website. Available from: http://www.evidem.org/evidem-collaborative.php.

MCDA Core Model (continued)

Resource Allocation and Ethical AspectsOverview

MCDA Core Model (continued)

Contextual Tool