Protocol

Mobile Applications for the Management of Diabetes Mellitus: A

Systematic Review

Plain English Summary

Diabetes is a lifelong condition in which the blood glucose level is high as a result of a problem in insulin

production. There are three main types of the condition; Type 1 diabetes, Type 2 diabetes, and Gestational

diabetes. Treatment for diabetes aims to keep blood glucose level at normal or near-normal levels, and to reduce

the risk factors for developing complications. Management of diabetes requires frequent monitoring of blood

glucose levels, use of oral medications or insulin, and adjustment of lifestyle. Current advances in technology

provide new opportunities for supporting diabetes management. In particular, mobile technology has potential to

integrate healthcare with everyday life. Increasingly, mobile devices have been used by patients to manage their

own health. Clinicians are also increasing their use of mobile devices at work. This systematic review aims to

evaluate evidence around the use of mobile applications to support people with diabetes and their care providers

to achieve treatment goals, improve outcomes, and improve the quality of care.

1. Background

Diabetes is one of the most common chronic diseases in the world. The prevalence of diabetes is increasing

globally. The World Health Organisation (WHO) estimated that 347 million people worldwide have diabetes in

2013.1 According to WHO report 2010, diabetes accounts for about 4 in every 100 deaths worldwide.2 WHO

further projected that diabetes will be the 7th leading cause of death in 2030.2 Many of the deaths are preventable,

being due to unhealthy lifestyle behaviours.3

International Diabetes Federation (IDF) was initiated in an effort to raise the population awareness of the

increased prevalence of diabetes.4 Diabetes therapy is directed at control rather than cure of diabetes. IDF

recommends a management program to provide effective diabetes care by encouraging patient involvement in

their treatment. Appropriate management of diabetes requires attention to educating patients, families and

communities about diabetes self-management and cooperative care skills, and partnership between healthcare

providers and patients.

1.1. Description of the intervention

The term mHealth refers to the use of mobile devices for health services and information.5 The main goal of

mobile health technology is to support the delivery and management of healthcare over distance.5 The

widespread use of smartphones and tablets by physicians and patients has led to mHealth solutions being broadly

used in monitoring, treatment, and education.

There is no standard definition of app found in the literature, but in general it refers to any form of application on

a computing device.6 Therefore, mobile app can be defined as software application designed to run on mobile

devices’ operating systems. They are typically available to the public through web-based application distribution

platforms named as app stores, for example, Apple App Store, Google Play (Android), BlackBerry App World,

Windows App Store, and Nokia Ovi Store. In this review, apps comprise all forms of mobile-based programs which

can be installed from app stores. Those apps may either have one or multiple components that could support the

management of diabetes. Mobile devices include smartphones as well as handheld and tablet computers e.g., iPad,

and personal digital assistants (PDAs).

1.2. How the intervention might work

Mobile devices hold a great promise to improve diabetes management and self-care. The widespread use of

mobile phones will enable patients to receive healthcare from anywhere. It especially benefits patients where

access to healthcare is affected by some barriers such as distance or disability.5 Mobile apps may offer a way of

delivering education and tailored care for patients with diabetes. Electronic diary and associated reminders may

support daily blood glucose monitoring and facilitate adherence. In addition, apps may support remote

monitoring of patients allowing them to collect and instantly transmit clinical data i.e. blood glucose readings to

care providers. On the other hand, mobile apps can deliver tools that may support clinical communications and

workflow such as medical calculators and drug databases as well as providing tools that assist both patients and

healthcare providers in problem solving and decision-making. Therefore, the use of mobile apps may offer a

highly accessible and cost-effective means for supporting diabetes care.

1.3. Adverse effects of the intervention

Although studies of technological interventions have addressed some difficulties with the uptake of these

interventions,7 there is limited evidence documenting adverse effects of mobile interventions. Remote-care

interventions may introduce new risks for patients and/or providers. Possible adverse effects of this technology;

as adapted to some extent from Finkelstein and Friedman;8 include:

Where a manual data input is used, there is a risk of inaccurate data entry and the patient taking action

according to them;

Patient receiving wrong advice, or misinterpreting the information and guidance;

The information source is unclear in many apps which might question the credibility of those apps; as

some may provide patients with conflicting advice or incorrect management practice;

The transfer of patient’s data electronically resulting in hacking or data breaches which may threaten

patient’s privacy and security;

Risks related to technical issues of the technology; e.g. limitations of coverage area, delayed or lost

messages;

Patient inability to use technology for some reasons such as difficulty in reading due to vision or literacy

problems;

Breaking down the doctor-patient relationship as a result of reduced contact between the patient and the

physician;

Data generated by remote-monitoring may bring together an additional workload for healthcare

providers and increase the proportion of time clinicians spend on daily care.

However, some of these risks may be overcome; for instance, manual data entry can be replaced with automatic or

wireless transmission, establishing appropriate regulations for mobile medical apps by a number of regulatory

bodies and ensuring the compliance of app developers with those regulations, or implementing some techniques

such as encryption of data to ensure patient safety and confidentiality. The demographics of the population

necessitate the use of tools such as mobile devices to help professionals cope with increased workloads. Yet,

mobile solutions are not intended to replace healthcare professionals; it is aimed to support and complement

healthcare beyond the traditional settings such as hospitals and physician offices.

1.4. Scoping search

In December 2013, preliminary searches were undertaken concerning the use of mobile applications in diabetes

management. The aim of the scoping search was to gain insight into potential studies, and the types of

populations, interventions, comparisons, and outcomes that were being assessed. Two main databases were

searched; MEDLINE and EMBASE; through OVID interface. In addition, the National Institutes of Health

ClinicalTrials.gov Registry was searched to assess ongoing trials. The databases search used various combinations

of the following terms and medical subject headings (MeSH): diabetes mellitus (MeSH), cellular phone (MeSH),

handheld computers (MeSH), diabetes, cell phone, mobile phone, smartphone, personal digital assistant, PDA,

apps, mHealth. The search was not restricted by year of publication.

Our initial literature search revealed a broad range of studies on the use of mobile interventions in the

management of diabetes. Early studies of mobile interventions (published in or prior to 2007) used cell phones as

a moderator to enable data flow between patients and care providers. Studies in this category used cellular phone

functions (other than apps) as the primary means to manage diabetes, and this was mainly dominated by Short

Message Service (SMS).

The scoping search suggested that most studies are inconsistent in study design and purpose of intervention. Few

numbers of RCTs on mobile apps have been completed, whilst pilot studies with a small number of participants

were predominated. There was a lack of studies evaluating the cost-effectiveness of mobile apps. Six systematic

reviews and two meta-analyses were identified on the use of mobile phones to support people with diabetes;

however, conclusions appear to be different among those reviews. In general, studies of mobile apps in diabetes

management are limited and on a small scale.

This review, therefore, will be restricted to studies conducted from 2008 onwards; since the first mobile app store

was the Apple App Store, and was launched in July 2008.6 For the purposes of this review, only studies involving

apps as the primary technological intervention will be included because with multiple interventions it would not

be possible to separate the effects of mobile apps from others. In several studies, app has been combined with

other secondary intervention strategies such as Short Message Service (SMS) or health consultations. As a result,

the multiple intervention components projected to complicate the interpretation of combined findings. In some

studies, applications have been designed and developed by a research team and only available as part of their

research study, whereas others are used commercially available apps. This review will include both; studies used

study-specific application and studies used commercial application. Additionally, it will bring together studies of

app intervention which are designed to support people with diabetes or healthcare professionals in charge of

diabetes care.

1.5. Previous reviews

A plethora of reviews have been found that considered the use of information technology (IT); including mobile

technology; to support diabetes self-care. Past reviews on the effectiveness of mobile interventions in diabetes

management have produced mixed results. Previous systematic reviews and meta-analysis have been

summarised and presented in Box 1.

A. Krishna 2008:9 An early systematic review examined the impact of cell phone interventions in diabetes care

up to 2007 and found significant improvements in HbA1c in 9 out of 10 studies. However, the majority of

included studies utilised text messages as the main intervention with exception of few studies where they

used one or more of other cell phone applications such as voice mail, games, Internet or email.

B. de Jongh 2012:10 A Cochrane review up to 2012 assessed the effects of mobile phone messaging; SMS and

MMS; on supporting self-management skills for people with long-term conditions. The authors found limited

quality evidence that mobile messages may benefit in supporting self-management of long-term illnesses and

this is due to the small number of included trials and the low total number of participants in those trials.

C. Herbert 2013:11 A recent systematic review of text messages for young people with type 1 diabetes concluded

that text messages are feasible, engaging, and youth enjoying them, but their long-term clinical effect is not

clear yet.

D. El-Gayar 2013,12 Tao 2013:13 A recent systematic review and a meta-analysis on the use of IT in diabetes self-

management; looked at a range of technologies including mobile phones; have confirmed the early findings

that IT can have positive impacts on diabetes self-care. The meta-analysis provided evidence that IT was

associated with a significant reduction in HbA1c compared to usual care, but significant heterogeneity existed

among the included trials.

E. Tatara 2009:14 A systematic review of mobile based interventions for diabetes self-management covering

publications up to 2008 suggested that studies of mobile interventions are difficult to compare because of the

variation between studies, and concluded that only seven out of 15 studies have shown a statistically

significant decrease in HbA1c.

F. Liang 2011,15 Liu 2012:16 Two meta-analyses investigated the effect of mobile phone interventions on

glycosylated haemoglobin (HbA1c) levels and provided a strong evidence of significant improvements in

glycaemic control, specifically for patients with type 2 diabetes.15 However, the result of those reviews is

based on heterogeneous set of studies, and they did not distinguish between the different types of mobile

interventions; for instance, SMS, apps, mobile phone calls, games, and emails; in their calculation of the overall

effect size, in addition to the small sample size in most included trials.

G. Holtz 2012:17 A systematic review of the potential role of mobile phones in supporting patients with diabetes

at a clinical level showed positive effects on glycated haemoglobin (HbA1c) levels, self-efficacy, and self-

management behaviours. The review concluded that only 19% of the studies examined the use of mobile

phones from the perspective of healthcare providers. Another reported issue is that most trials did not report

the power calculations; which in turn will limit the generalizability of their results to other populations.

H. Baron 2012:18 A systematic review evaluated the clinical effect of mobile monitoring on glycosylated

haemoglobin and concluded that comparison across studies was not possible due to study variability and

notable methodological flaws in most studies. They reported the evidence on mobile solutions for blood

glucose monitoring is inconclusive in adult population with type1 or type 2 diabetes. However, this review

was restricted to remote monitoring aspect where patient transmit blood glucose readings to care provider

through mobile device and receive feedback.

I. Frazetta 2012:19 The most recent review was centred exclusively on the use of smartphone applications for

self-management of patients with type 2 diabetes. It primarily investigated the effect on glycaemic control and

found seven studies; with four had a positive effect on glycaemic control. The review has identified some

technical difficulties and usability issues, and suggested that further research is needed. However, the

evidence of this review was debatable due to some methodological weakness as they did not attempt to assess

the quality of included trials which limited the reliability of their conclusion.

Box 1. Previous systematic reviews and meta-analysis on the use of mobile phones in diabetes management.

An on-going systematic review20 of mobile applications for diabetes self-management will consider the clinical,

informational, behavioural, and economical facets. Yet, it is restricted to the aspect of self-care, limited to patient’s

use, and excluded diabetes during pregnancy.

The use of mobile applications to support healthcare professionals involved in diabetes care has not yet been

explored in a systematic review. One exception is a systematic review aimed to assess the effect of mHealth

interventions on healthcare delivery processes and concluded that trials investigating mobile interventions for

HCPs support have reported some promising results for clinical management, appropriate testing, referral,

screening, diagnosis, treatment, and triage.21 However, this review was general across all specialities and not

specific in diabetes care. Classification of smartphone-based healthcare applications according to their

functionalities has been explored in another systematic review that addressed a broad range of medical

applications rather than being restricted to diabetes related-applications, and was limited to MEDLINE

literature.22

In conclusion, the majority of reviews found inconsistent evidence of clinical effectiveness, variable and

inconclusive results for other outcomes and none have focussed particularly on cost-effectiveness of mobile

interventions. With respect to the target population, none of the past reviews considered participants with

gestational diabetes; most reviews examined patients with type 1 and type 2 diabetes. No review has examined

mobile apps specifically; apart from one review19 which attempted to evaluate the use of smartphone apps; but it

only covered the self-management domain of diabetes care. Furthermore, this review alongside most previous

reviews did not make a distinction between the different modalities of mobile phones technology i.e. SMS, apps,

mobile phone calls, games, and internet, despite their differences in the intervention components in both diabetes

management and use of technology.

1.6. Why it is important to do this review

Diabetes is prevalent and costly, and recent study estimated that the total increase in numbers with diabetes from

2011 to 2030 is 50.7%, at an annual growth of 2.7%.23 The global trends suggest that increases will continue

because of population growth, ageing population and rapidly rising numbers of overweight and obese people.

Diabetes care may account for up to 18% of the total national healthcare expenditure24 in view of the fact that

people with diabetes require at least double the healthcare resources compared to their peers who do not have

diabetes.25

Global smartphone ownership almost approaches 30% according to recent study by mobile network vendor

Ericsson,26 and the rate of smartphone adoption is increasing. Smartphone offer much more functionality than

cellular phone; for instance; Internet access, and installation of a variety of apps including health related apps. A

large number of apps are free of charge or at a low cost. Therefore, apps offer an innovative intervention which is

relatively inexpensive. Mobile applications seem to be suitable specifically for diabetes self-care as it involves

continuous monitoring which can be facilitated by the ubiquitous nature of mobile devices. Comprehensive

applications that enable patients to monitor all the facets of diabetes care and avoid complications are available.

Currently, a wide range of mobile apps is being utilised in many areas of healthcare: education, health

management, and health promotion. Yet, there is a lack of systematic consideration around the effectiveness of

mobile applications.

Previous research has explored extensively the use of mobile phone short message service (SMS), with any limited

consideration of apps according to our definition and criteria. As it is an emerging field, the majority of app trials

were published between 2012 and 2013, while previous reviews covered publications up to 2011. As a result, a

high quality, rigorous evaluation is needed to understand the potential for mobile apps to complement the care for

the growing number of people with diabetes. Thus, this protocol details a review which aims to evaluate the

benefits and the drawbacks of including mobile apps within the care pathways for people with diabetes. The

result of this review evaluation will aim to provide a robust evidence base for clinicians and policy makers, and

may potentially influence future policies to support important investment decisions in mobile health technologies.

2. Objectives

The overall objective of this review is to assess the clinical effectiveness and quality impacts of using mobile

applications in diabetes management. Other objectives are as follows:

A. Quantitative objectives:

1. To evaluate the effect of using mobile apps in measures of blood glucose control and other clinical

outcomes in diabetes;

For this objective, the study design will be restricted to RCT, however, if the number of identified RCTs is very

limited, other study designs will be considered following the hierarchy of evidence for intervention studies.27

2. To evaluate additional outcomes for patients with diabetes such as quality of life, knowledge, adherence,

skills, and psychological outcomes;

3. To examine if the effect of mobile apps varies across groups defined by diabetes type (Type 1 diabetes,

Type 2 diabetes, or Gestational diabetes), purpose of intervention (telemonitoring, self-care), age group

(<18 years, 18-50 years, >50 years);

4. To evaluate the effect of using mobile apps on clinical workflow and care team coordination.

B. Qualitative objectives:

5. To identify the desired features of mobile apps among diabetes patients and healthcare professionals, and

the main factors that may affect the adoption of apps;

6. To identify the risks involved in using mobile apps to patients;

7. To find out whether mobile apps are accessible and acceptable to both patients and healthcare

professionals;

8. To assess patients and healthcare professionals perceptions and views of the use of mobile apps;

9. To explore healthcare professionals’ tasks that would be best served by mobile apps;

10. To identify the potential obstacles to integrating mobile apps into diabetes care.

3. Method

3.1. Criteria for considering studies for this review

The criteria for including studies are described in detail below:

3.1.1. Types of studies

Eligible study designs include:

Randomised controlled trials (including randomised crossover studies, and cluster randomised trials);

Quasi-experimental studies (including interrupted time series studies);

Controlled before and after (CBA) studies;

Observational studies (cohort, case-control, cross-sectional studies);

Qualitative studies (e.g. interviews, focus groups);

Partially published work (e.g. conference abstracts).

The rationale for including non-randomised studies is that during the scoping search limited number of RCTs has

evaluated the use of mobile apps in diabetes management.

Pilot studies will be included; as the initial search has identified a number of published pilot studies; to enable us

to understand the current state of applications in this area. Reports of ongoing or unpublished work will be

included too.

3.1.2. Types of participants

Eligible populations include:

1. All individuals with diabetes as diagnosed by a clinician of either:

Type 1 diabetes mellitus;

Type 2 diabetes mellitus;

Gestational diabetes.

Individuals without a diagnosis of diabetes will be considered only if they are a parent of, or caregiver for, a

person with diabetes.

2. Healthcare professionals involved in diabetes care e.g. GPs, consultants, diabetes specialist nurses,

diabetes dieticians.

No participant will be excluded on the basis of any socio-demographic characteristic (e.g. age, gender; ethnicity;

marital status; geographic location; employment status; education; income or health status) in any care setting.

3.1.3. Types of interventions

All mobile apps that support one of the following domains:

1. Telemonitoring: mobile apps that aim to facilitate remote monitoring of patient with the help of healthcare

providers from a distance, which involve information exchange i.e. two-way communication between patient

and provider. Care provider feedback may include treatment recommendations, medication adjustment,

reminders, advice, encouragements, or corrections to lifestyle.

2. Self-care: mobile apps that aim to develop or support at least one of the seven diabetes self-care behaviours as

defined by the American Association of Diabetes Educators (AADE7)28:

a. Healthy eating;

b. Being physically active;

c. Self-monitoring of blood glucose and other biometrics;

d. Medication adherence;

e. Problem solving skills;

f. Risk reduction behaviours and preventive care;

g. Healthy coping skills and maintaining motivation.

Apps in this domain may include one or more functions designed to initiate or reinforce those behaviours with or

without the help of healthcare professionals or peers from a distance.

3. Clinical practice: medical apps that target healthcare professionals engaged in diabetes care in a clinical

settings. This encompasses apps providing on-demand, instant clinical support in treatment, diagnosis, and

decision making at the point of care, or facilitating care coordination between care team.

3.1.4. Comparisons

Studies that incorporate a comparison may compare mobile apps with:

usual care or any other control intervention;

or

other intervention variant.

Studies with no comparison will still be eligible for inclusion.

3.1.5. Types of outcome measures

All reported outcomes related to patients or healthcare professionals will be considered.

Examples of outcomes related to patients include:

HbA1c Levels;

Other measurements of diabetes control (fasting blood glucose, serum fructosamine);

Other relevant biomarkers (blood pressure, cholesterol, BMI);

Health-related quality of life (QoL) measured using any validated instrument;

Diabetic complications including hypoglycemic episodes;

Insulin dose;

Hospital admissions;

Emergency visits;

Healthy lifestyle;

Knowledge;

Self-efficacy;

Adherence; both short-term and long-term;

Retention and sustainability;

Empowerment;

Motivation;

Attitudes and coping skills.

Examples of outcomes related to healthcare professionals include:

Adherence to guidelines;

Prescription errors;

Workflow;

Time spent with patient;

Providers’ perceptions of the barriers and enablers to integrating apps into diabetes care.

Examples of common outcomes related to both patients and healthcare professionals include:

Feasibility of mobile apps;

Usability;

Usefulness of mobile apps;

Utilisation of mobile apps;

Safety impacts associated with mobile apps;

Satisfaction with mobile apps components;

Acceptability of mobile apps;

Accessibility;

Convenience;

Preference, perceptions and views of the use of mobile apps;

Experience with mobile apps.

Papers that report outcomes which are not in this list will not be excluded.

Outcomes measured at completion of an intervention and any subsequent time points (follow-up) will be

included. Short-term follow-up defined as outcomes monitoring that completes within 30 days of the completion

of the intervention, long-term follow-up defined as monitoring that continues 6 months after the completion of

the intervention and medium-term follow-up as intermediate of the two.

3.2. Exclusion criteria

Studies examined other primary interventions besides mobile apps;

Studies lie outside the self-management, remote monitoring, or clinical practice domains;

Studies not aimed at individuals with diabetes or their caregivers, or healthcare professionals involved in

diabetes care;

Studies based on desktop computers, laptops, or netbooks;

Studies target medical training or continuing medical education i.e. medical and nursing students;

Studies fall within complementary or alternative medicine;

Studies fall within preventive care.

4. Search methods for identification of studies

4.1. Electronic searches

The following databases and journals will be searched for relevant studies from 2008 to the present:

The Cochrane Central Register of Controlled Trials (CENTRAL, The Cochrane Library);

MEDLINE;

EMBASE;

PsycINFO;

IEEEXplore;

Journal of Medical Internet Research (JMIR);

Journal of Mobile Technology in Medicine (JMTM);

Journal of Diabetes Science and Technology (JDST).

A detailed search strategy has been developed for MEDLINE (Appendix 1), and it will be adapted as appropriate

for other databases.

An appendix will be attached for the search results for all databases.

No study will be excluded on the basis of Language.

OVID AutoAlerts and PubMed MyNCBI will be set up to send monthly updates for new literature; monthly

searches also will be performed in Cochrane Library and other journals.

4.2. Searching other resources

Grey literature will be searched through the following sources:

Google Scholar;

ProQuest Dissertations.

Also, databases of completed and ongoing trials will be searched in the following resources (2008-present):

1. NIH ClinicalTrials.gov (http://www.clinicaltrials.gov/)

2. Current Controlled Trials (http://www.controlled-trials.com/)

3. WHO International Clinical Trials Registry Platform (ICTRP) (http://www.who.int/ictrp/en/)

Additional studies will be sought by searching the reference lists of retrieved primary trials, systematic reviews,

meta-analyses, and health-technology assessment reports.

5. Data collection and analysis

5.1. Selection of studies

Reference management software EndNote29 will be used to combine search results from electronic databases, and

remove duplicated records. To determine the eligibility of studies, one author (HA) will scan titles and abstracts of

every record retrieved. A second author (PS) will independently assess a random sample comprising 20% of all

identified studies due to the vast number of potentially relevant literature. Agreement on inclusion will be

calculated using the kappa statistic to ensure consistency. Both reviewers will investigate the full text of all papers

identified at the abstract sift, and the reasons for exclusion will be recorded. A third reviewer will be sought for

any disagreements about inclusion. If resolving disagreement is not possible, author will be contacted for

clarification. The study flow and reasons for exclusion of full text papers will be documented in an adapted

PRISMA (preferred reporting items for systematic reviews and meta-analyses) study flowchart30 (Appendix 2).

5.2. Data extraction and management

One reviewer (HA) will carry out the data extraction from studies that fulfil inclusion criteria, and will be checked

for accuracy by the second reviewer (PS). Relevant population and intervention characteristics will be extracted

using standard data extraction templates (Appendixes 3,4) with any disagreements to be resolved by discussion,

or if required by a third party. An email request will be sent to contact persons of published studies to enquire

whether authors are willing to answer questions regarding their trials. Thereafter, relevant missing information

on the trial will be sought from the original author/s of the article, if required. All extracted information of

included and excluded studies along with reasons of exclusion will be summarised in the tables 'Characteristics of

included studies' and 'Characteristics of excluded studies' respectively.

5.3. Dealing with duplicate publications and companion papers

In the case of duplicate publications and companion papers of a primary study; to maximise yield of information,

all available data will be evaluated simultaneously.

5.4. Quality Assessment of included studies

To evaluate the methodological quality of included RCT studies, the Cochrane Collaboration’s Risk of Bias Tool31

will be used (Appendix 5). This method uses the following bias criteria:

Random sequence generation (selection bias);

Allocation concealment (selection bias);

Blinding of participants and personnel (performance bias and detection bias);

Blinding of outcome assessment (performance bias and detection bias);

Incomplete outcome data (attrition bias);

Selective outcome reporting (reporting bias);

Other bias.

A modified version of the risk of bias tool will be used to assess the risk of bias of other study designs (Appendix

6) following the guidelines in Chapter 13 of the Cochrane Handbook.31

Quality assessment will be undertaken by one reviewer (HA) and checked by a second reviewer (PS). Any

disagreements regarding the quality of a trial will be resolved by consensus, or with consultation of a third party.

Where data are missing, reviewers will attempt to contact the author/s at their last known address.

Risk of bias criteria will be judged as 'low risk', 'high risk' or 'unclear risk', and then will be summarised as 'yes'

(indicating a low risk of bias in all domains), 'no' (indicating a high risk of bias in at least one of the domains) or

'unclear' (indicating an uncertain risk of bias in at least one of the domains).

Individual bias items will be evaluated as described in the Cochrane Handbook for Systematic Reviews of

Interventions.31 A 'Risk of bias graph' figure and ‘Risk of bias summary’ table will be attached.

For the quality assessment of qualitative studies, the quality criteria defined by Critical Appraisal Skills

Programme (CASP)32 will be used (Appendix 7). This tool contains ten items relating to rigour, credibility and

relevance. Three categories will be established for each item: clearly met, clearly not met, and unclear.

5.5. Measures of treatment effect

Dichotomous data will be expressed as odds ratio (OR) or risk ratio (RR) with 95% confidence intervals (CIs).

continuous data will be expressed as differences in means (MD) or standardised mean difference (SMD) with

95% CI depending on whether the outcomes are reported on similar measurement scales or not. To account for

other study design, the adjusted estimates effects will be calculated using a generic inverse-variance method.

5.6. Unit of analysis issues

The method of randomisation will be taken into account, such as cross-over trials, cluster-randomised trials and

multiple observations for the same outcome as discussed in Chapter 16 of the Cochrane Handbook.31

5.7. Dealing with missing data

Relevant missing data will be obtained from authors whenever possible. A modified intention-to-treat (ITT)

analysis will be used to investigate attrition rates, for example drop-outs, losses to follow up and withdrawals;

using.

5.8. Assessment of heterogeneity

The degree of statistical heterogeneity across studies will be identified through inspection of the forest plots, and

employing the I2 statistic33,34 where an I2 statistic greater than 50% indicates a considerable level of

inconsistency.31

If heterogeneity identified, the potential reasons for it will be sought by examining individual study and subgroup

characteristics. The following characteristics are expected to introduce heterogeneity:

Reported outcome measures e.g. (HbA1c vs. fasting blood glucose, BMI vs. weight);

Intervention features and components;

Population characteristics;

Diabetes type.

5.9. Assessment of reporting biases

If sufficient data points are available, publication bias will be assessed through visual inspection of funnel plots

with respect to plot asymmetry and use of linear regression tests, and a regression slope of zero will be

interpreted as no publication bias. There are a number of explanations for the asymmetry of a funnel plot35 and

results will be carefully interpreted.36

The original study protocol will be sought for all included studies in order to compare the reported methods and

outcome measures against their original plan. If this is not possible, and the missing data suspected to introduce

serious bias; the sensitivity analysis will be used to investigate the impact of excluding such studies in the overall

results.

6. Data synthesis

6.1. Quantitative evidence synthesis

Where appropriate, meta-analysis will be performed according to the statistical guidelines referenced in the

newest version of the Cochrane Handbook for Systematic Reviews of Interventions.31 However, if meta-analysis is

considered unsuitable due to the heterogeneity and/or small number of studies, a narrative overview37 of the

findings of included studies will be presented with tabular summaries of extracted data.

Primarily, the narrative synthesis will be structured according to the intended purpose of the mobile app (self-

care, telemonitoring, or clinical practice), and then, studies will be organised by the objectives addressed above.

Typically, study, intervention, population, and outcome characteristics will be summarised in text, evidence, and

summary tables. Results for each outcome of interest will be presented separately and stratified according to the

type of diabetes and type of intervention, and compared qualitatively and, where possible, quantitatively in text

and summary tables. A ‘Summary of finding’ table will be produced for each outcome.

6.2. Overall quality of evidence (GRADE system)

The overall quality of evidence for an outcome will be assessed using the GRADE approach38,39 which uses the

summary of the risk of bias of the outcome across studies to assess the robustness of the evidence (Appendix 8).

The GRADE approach is based on assessments across five domains: study limitations, consistency of results,

directness of the evidence, precision of the results, and publication/reporting bias. This approach categorise the

levels of quality as high, moderate, low, and very low. The assessment of overall quality of evidence will follow the

guidelines in Chapter 12 of the Cochrane Handbook,31 and the process will be provided in an appendix.

6.3. Qualitative thematic synthesis

A qualitative thematic synthesis40 will be performed for all identified relevant qualitative studies which utilise

qualitative methods of data collection and analysis; whether a stand-alone study or a discrete part of a mixed-

method study. Studies assessing the same outcome will be grouped and coded, and the free text of included

studies will be extracted and coded using NVivo41. Information will be stratified according to the purpose and type

of intervention and type of diabetes, and a summary of qualitative outcomes will be presented in a separate table.

6.4. Subgroup analysis and investigation of heterogeneity

Subgroup analyses of the quantitative outcomes will be carried out if the studies were homogeneous. The

following subgroup analyses are planned:

Purpose of the intervention (telemonitoring, self-care);

Diabetes types (Type 1 diabetes, Type 2 diabetes, Gestational diabetes);

Number of interventions (whether app used alone or combined with other secondary intervention);

Age groups (<18 years, 18-50 years, >50 years).

6.5. Sensitivity analysis

Sensitivity analysis will be performed in order to explore the influence of the following factors on effect sizes:

restricting the analysis to RCTs;

restricting the analysis to published studies;

restricting the analysis taking into account risk of bias, as specified above;

excluding studies with high risk of bias;

restricting the analysis to very long or large studies to establish how much they dominate the results;

excluding studies with results differ from other studies;

restricting the analysis to studies using the following filters: language of publication, source of funding

(industry versus other), and country.

7. Meta-analysis

7.1. Criteria for performing a meta-analysis

Depending on the level of statistical heterogeneity and publication bias, a meta-analysis/subgroup will be

conducted for any of the quantitative outcome measures if at least two studies are identified. For any study, if the

mean and/or standard deviation for the continuous outcome of interest cannot be determined, then this

particular study will be excluded. The number of meta-analysis will be determined by discussion amongst review

authors.

7.2. Meta-analysis procedure

The guidelines for meta-analysis described in Chapter 9 of the Cochrane Handbook31 will be followed using the

RevMan42 for analysis, and both fixed-effects and random-effects analysis will be performed.

8. Contributions of authors

Hala Alhodaib (HA): Development of the protocol, scoping search, searches and screening, data extraction, quality

assessment, statistical analysis, report writing.

Dr. Paul Sutcliffe (PS): Contributed to the development of the protocol, selection of studies, data extraction, quality

assessment, statistical analysis, report writing.

Dr. Krishnarajah Nirantharakumar: Reviewed the drafts of the protocol, statistical analysis, report writing.

Dr. Sailesh Sankar: Clinical input, report writing.

9. Declarations of interest

None known.

References:

1. WHO. Fact sheet No. 312 2. WHO. Global status report on noncommunicable diseases 2010 (Geneva, 2011). 3. Kapustin, J. Chronic Disease Prevention Across the Lifespan. The Journal for Nurse Practitioners 6, 16-24

(2010). 4. IDF. http://www.idf.org/. 5. Akter, S. & Ray, P. mHealth - an Ultimate Platform to Serve the Unserved. IMIA Yearbook 2010: Biomedical

Informatics: Building Capacity Worldwide 5, 94-100 (2010). 6. Gröger, C., Silcher, S., Westkämper, E. & Mitschang, B. Leveraging Apps in Manufacturing. A Framework for App

Technology in the Enterprise. Procedia CIRP 7, 664-669 (2013). 7. Glasgow, R., et al. Recruitment for an Internet-Based Diabetes Self-Management Program: Scientific and Ethical

Implications. Annals of Behavioral Medicine 40, 40-48 (2010). 8. Finkelstein, J. & Friedman, R.H. Potential Role of Telecommunication Technologies in the Management of

Chronic Health Conditions. Disease Management & Health Outcomes 8, 57-63 (2000). 9. Krishna, S. & Boren, S.A. Diabetes self-management care via cell phone: a systematic review. Journal of Diabetes

Science & Technology 2, 509-517 (2008). 10. de Jongh, T., Gurol-Urganci, I., Vodopivec-Jamsek, V., Car, J. & Atun, R. Mobile phone messaging for facilitating

self-management of long-term illnesses. The Cochrane database of systematic reviews 12, Cd007459 (2012). 11. Herbert, L., Owen, V., Pascarella, L. & Streisand, R. Text message interventions for children and adolescents

with type 1 diabetes: a systematic review. Diabetes technology & therapeutics 15, 362-370 (2013). 12. El-Gayar, O., Timsina, P., Nawar, N. & Eid, W. A systematic review of IT for diabetes self-management: are we

there yet? International journal of medical informatics 82, 637-652 (2013). 13. Tao, D. & Or, C.K. Effects of self-management health information technology on glycaemic control for patients

with diabetes: a meta-analysis of randomized controlled trials. Journal of telemedicine and telecare (2013). 14. Tatara, N., Arsand, E., Nilsen, H. & Hartvigsen, G. A Review of Mobile Terminal-Based Applications for Self-

Management of Patients with Diabetes. in eHealth, Telemedicine, and Social Medicine, 2009. eTELEMED '09. International Conference on 166-175 (2009).

15. Liang, X., et al. Effect of mobile phone intervention for diabetes on glycaemic control: a meta-analysis. Diabetic medicine : a journal of the British Diabetic Association 28, 455-463 (2011).

16. Liu, L. & Ogwu, S.-M. A Meta-Analysis of Mobile Health and Risk Reduction in Patients with Diabetes Mellitus: Challenge and Opportunity. Journal of Mobile Technology in Medicine (JMTM) 1, 17-24 (2012).

17. Holtz, B. & Lauckner, C. Diabetes management via mobile phones: a systematic review. Telemedicine journal and e-health : the official journal of the American Telemedicine Association 18, 175-184 (2012).

18. Baron, J., McBain, H. & Newman, S. The impact of mobile monitoring technologies on glycosylated hemoglobin in diabetes: a systematic review. Journal of diabetes science and technology 6, 1185-1196 (2012).

19. Frazetta, D., Willet, K. & Fairchild, R. A systematic review of smartphone application use for type 2 diabetic patients. Online Journal of Nursing Informatics (OJNI) 16(2012).

20. Huckvale, K., Velthoven, M.v., Cash-Gibson, L., Felix, L. & Car, J. Self-care using apps for smartphones and other mobile device software: a systematic review of the literature using the example of diabetes mellitus. (PROSPERO, 2011).

21. Free, C., et al. The Effectiveness of Mobile-Health Technologies to Improve Health Care Service Delivery Processes: A Systematic Review and Meta-Analysis. PLoS Med 10, e1001363 (2013).

22. Mosa, A.S., Yoo, I. & Sheets, L. A systematic review of healthcare applications for smartphones. BMC medical informatics and decision making 12, 67 (2012).

23. Whiting, D.R., Guariguata, L., Weil, C. & Shaw, J. IDF Diabetes Atlas: Global estimates of the prevalence of diabetes for 2011 and 2030. Diabetes Research and Clinical Practice 94, 311-321 (2011).

24. IDF. IDF Diabetes Atlas. (2013). 25. ADA. Economic Consequences of Diabetes Mellitus in the U.S. in 1997. Diabetes Care 21, 296-309 (1998). 26. Ericsson Mobility Report. (http://www.ericsson.com/ericsson-mobility-report, 2013). 27. Tacconelli, E. Systematic reviews: CRD's guidance for undertaking reviews in health care. The Lancet Infectious

Diseases 10, 226 (2010). 28. Tomky, D., Cypress, M., Dang, D., Maryniuk, M. & Peyrot, M. AADE Position Statement - AADE7™ Self-Care

Behaviors. The Diabetes Educator 34, 445-449 (2008). 29. EndNote X7. Vol. X7.0.2 (Thomson Reuters Corporation, New York, USA, 2013). 30. Liberati, A., et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that

evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology 62, e1-e34 (2009).

31. Higgins, J.P.T. & Green, S. (eds.). Cochrane Handbook for Systematic Reviews of Interventions, (The Cochrane Collaboration, 2011).

32. CASP. Making sense of evidence. 10 questions to help you make sense of qualitative research. (Public Health Resource Unit, England, 2006).

33. Higgins, J.P.T. & Thompson, S.G. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine. 21, 1539-1558 (2002).

34. Higgins, J.P.T., Thompson, S.G., Deeks, J.J. & Altman, D.G. Measuring inconsistency in meta-analyses. BMJ 327, 557-560 (2003).

35. Sterne, J.A.C., Egger, M. & Smith, G.D. Investigating and dealing with publication and other biases in meta-analysis. BMJ 323, 101-105 (2001).

36. Lau, J., Ioannidis, J.P.A., Terrin, N., Schmid, C.H. & Olkin, I. The case of the misleading funnel plot. BMJ 333, 597-600 (2006).

37. Rodgers, M., et al. Testing Methodological Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: Effectiveness of Interventions to Promote Smoke Alarm Ownership and Function. Evaluation 15, 49-73 (2009).

38. Guyatt, G., et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of clinical epidemiology 64, 383-394 (2011).

39. Brozek, J., Oxman, A. & Schünemann, H. GRADEpro. (GRADE Working Group, 2008). 40. Thomas, J. & Harden, A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC

Medical Research Methodology 8, 45 (2008). 41. NVivo. (QSR International Pty Ltd, Doncaster, Australia, 2012). 42. Review Manager (RevMan). (The Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen, Denmark,

2014).

Appendices

Appendix 1

Search strategy details for MEDLINE

Ovid MEDLINE(R) 2008 to March Week 2, searched on 20 March 2014.

No. Search

1 diabet*.ab,ti.

2 IDDM.ab,ti.

3 NIDDM.ab,ti.

4 MODY.ab,ti.

5 exp Diabetes Mellitus/

6 hyperinsulin.ab,ti.

7 insulin sensitive*.ab,ti.

8 insulin resist*.ab,ti.

9 insulin depend*.ab,ti.

10 insulin?depend*.ab,ti.

11 non insulin depend*.ab,ti.

12 noninsulin depend*.ab,ti.

13 non insulin?depend*.ab,ti.

14 noninsulin?depend*.ab,ti.

15 exp Insulin Resistance/

16 impaired glucose tolerance*.ab,ti.

17 glucose intoleran*.ab,ti.

18 exp Glucose Intolerance/

19 typ* 1 diabet*.ab,ti.

20 typ* 2 diabet*.ab,ti.

21 typ* I diabet*.ab,ti.

22 typ* II diabet*.ab,ti.

23 gestational diabet*.ab,ti.

24 Or/1 - 23

25 telephon*.ab,ti.

26 phon*.ab,ti.

27 25 or 26

28 mobil*.ab,ti.

29 cell*.ab,ti.

30 28 or 29

31 27 and 30

32 exp Cellular Phone/

33 (smartphone? Or smart?phone?).ab,ti.

34 (window? adj3 phone?).ab,ti.

35 (window? adj3 mobile?).ab,ti.

36 nokia.ab,ti.

37 palm OS.ab,ti.

38 (palm adj3 computer?).ab,ti.

39 symbian.ab,ti.

40 iphone?.ab,ti.

41 ipad?.ab,ti.

42 ipod? touch.ab,ti.

43 personal digital assistant?.ab,ti.

44 PDA.ab,ti.

45 android?.ab,ti.

46 blackberry*.ab,ti.

47 exp Computers, Handheld/

48 apps.ab,ti.

49 exp Mobile Applications/

50 applet?.ab,ti.

51 (mobile adj3 app*).ab,ti.

52 (software adj3 app*).ab,ti.

53 *Medical Informatics Applications/

54 exp Decision Making, Computer-Assisted/

55 (ehealth or “e health” or e-health or e?health).ab,ti.

56 (mhealth or “m health” or m-health or m?health).ab,ti.

57 (telehealth or tele?health).ab,ti.

58 (telemedicine or tele?medicine).ab,ti.

59 exp Telemedicine/

60 (telemonitor* or tele?monitor*).ab,ti.

61 (remote adj2 (consult* or diagnos* or monitor* or treat* or care)).ab,ti.

62 “mobile health”.ab,ti.

63 Or/ 31 - 62

64 24 and 63

65 limit 64 to yr="2008 -Current"

Appendix 2

PRISMA study flow diagram

Records identified through database

searching

(n = )

Additional records identified through

other sources

(n = )

Records after duplicates removed

(n = )

Records screened

(n = )

Records excluded at title

and abstract level

(n = )

Full-text articles assessed

for eligibility

(n = )

Full-text articles excluded,

with reasons

(n = )

Studies included in

qualitative synthesis

(n = )

Studies included in

quantitative synthesis

(meta-analysis)

(n = )

Appendix 3

Data extraction sheet for included quantitative study reports

Name of the reviewer: Date of review:

Study Details

Study ID (Ref man):

Study Title:

First author surname:

Year of publication:

Publication type: (peer review journal / journal supplement / abstract / other)

Language:

Country of origin:

Study design:

Study setting:

Number of centres:

Method of recruitment:

Recruitment context: (i.e. where people were recruited from)

Dates of recruitment:

Duration of study:

Follow up period:

Funding: (commercial / non-commercial / other funding)

Aim of the study:

Inclusion/Exclusion criteria

Inclusion criteria: (e.g. ability to use phone, phone ownership, access to Internet)

Exclusion criteria: (e.g. comorbidities, pregnancy, mental illness, not willing to monitor diabetes, lived outside of

coverage area)

Ethical standards

Was ethical committee approval obtained?

Informed consent obtained?

Participants

Total number of participants:

Sample attrition/drop out:

Geographic location:

Main receiver of intervention: (patient, caregiver, parent, healthcare worker)

Characteristics of participants:

Age group:

Gender:

Ethnicity (ethnic groups):

Education:

Type of diabetes:

Severity of diabetes:

Time since diagnosis (duration of diabetes (mean years)):

Treatment:

Comorbidities:

Power calculation:

Incentives for participation:

Providers (Details of the healthcare worker(s) or systems responsible for providing or supporting the app)

Who delivers the intervention? Number of providers: Training offered in delivery of intervention? Yes/No Paid for mobile phone service? Yes/No Intervention

Name:

Hardware (device):

Software (platform):

Purpose: (telemonitoring, self-care, clinical practice)

Functions: (reminder, diary, entertainment, information)

Content:

Application type: (study-specific/ commercial)

Any comparison (control):

Any supporting intervention? Yes/No

Phone provided (phone ownership): Yes/No

App installation Process:

Transmission of BG readings? Yes/No

Method of data transmission: (Bluetooth, Wi-Fi, wired, infrared)

Feedback provided: Yes/No

Type and nature of feedback:

Frequency of feedback:

Mode of data entry: (manual, automatic e.g. from connected monitoring device)

Data inputted:

Frequency of data input:

Training offered:

Mode of interaction:

Frequency of interaction:

Duration of interaction:

Management activities: (eating, exercise, BG)

AADE7 skill targeted:

Process and timing for data download:

Require Internet access? Yes/No

Security arrangements:

Number of participants Intervention Comparator

Screened

Randomised/Included

Excluded

Missing participants

Withdrawals

Patient’s baseline characteristics Intervention Comparator

Number of patients (N)

Age (years)

Mean (SD)

Gender - female n/N (%)

Weight (kg) / BMI (kg/m2) Mean (SD)

Income (£) Mean (SD)

Ethnicity

Ratio of diabetes type

Ratio of diabetes treatment

Comorbidity

Outcomes

Primary outcomes:

Secondary outcomes:

Method of assessing outcomes:

Timing of assessment (time points):

Study end point:

Length of follow up:

Confounders:

Technical issues:

HbA1c level: Yes/No

Other measurements of diabetes control: Yes/No

Relevant biomarkers (e.g. blood pressure, cholesterol, BMI): Yes/No

Health related quality of life: Yes/No

Complications: Yes/No

Insulin dose: Yes/No

Hospital admissions: Yes/No

Emergency visits: Yes/No

Healthy lifestyle: Yes/No

Self-efficacy: Yes/No

Knowledge: Yes/No

Adherence: Yes/No

Usability: Yes/No

Satisfaction: Yes/No

Retention and sustainability: Yes/No

Adverse event: Yes/No

Results Time Points:

HbA1c level Intervention Comparator

Other measurements of diabetes control Intervention Comparator

Relevant biomarkers Intervention Comparator

Quality of life Intervention Comparator

Complications Intervention Comparator

Insulin dose Intervention Comparator

Hospital admissions Intervention Comparator

Emergency visits Intervention Comparator

Healthy lifestyle Intervention Comparator

Self-efficacy Intervention Comparator

Knowledge Intervention Comparator

Adherence Intervention Comparator

Usability Intervention Comparator

Satisfaction Intervention Comparator

Other outcomes Intervention Comparator

Adverse events Intervention Comparator

Authors conclusion

Reviewer’s conclusion

Appendix 4

Data extraction sheet for included qualitative studies

Name of the reviewer: Date of review:

Study Details

Study ID (Ref man):

Study Title:

First author surname:

Year of publication:

Publication type: (peer review journal / journal supplement / abstract / other)

Language:

Country of origin:

Study setting:

Number of centres:

Method of recruitment:

Recruitment context: (i.e. where people were recruited from)

Dates of recruitment:

Duration of study:

Funding: (commercial / non-commercial / other funding)

Aim of the study:

Inclusion/Exclusion criteria

Inclusion criteria: (e.g. ability to use phone, phone ownership, access to Internet)

Exclusion criteria: (e.g. comorbidities, pregnancy, mental illness, not willing to monitor diabetes, lived outside of

coverage area)

Ethical standards

Was ethical committee approval obtained?

Informed consent obtained?

Participants

Total number of participants:

Sample attrition/drop out:

Geographic location:

Main receiver of intervention: (patient, caregiver, parent, healthcare worker)

Characteristics of participants:

Age group:

Gender:

Ethnicity (ethnic groups):

Education:

Type of diabetes:

Severity of diabetes:

Time since diagnosis (duration of diabetes (mean years)):

Treatment:

Comorbidities:

Power calculation:

Incentives for participation:

Providers (Details of the healthcare worker(s) or systems responsible for providing or supporting the app)

Who delivers the intervention? Number of providers: Training offered in delivery of intervention? Yes/No Paid for mobile phone service? Yes/No Intervention

Name:

Hardware (device):

Software (platform):

Purpose: (telemonitoring, self-care, clinical practice)

Functions: (reminder, diary, entertainment, information)

Content:

Application type: (study-specific/ commercial)

Phone provided (phone ownership): Yes/No

App installation Process:

Transmission of BG readings? Yes/No

Method of data transmission: (Bluetooth, Wi-Fi, wired, infrared)

Feedback provided: Yes/No

Type and nature of feedback:

Frequency of feedback:

Mode of data entry: (manual, automatic e.g. from connected monitoring device)

Data inputted:

Frequency of data input:

Training offered:

Mode of interaction:

Frequency of interaction:

Duration of interaction:

Management activities: (eating, exercise, BG)

AADE7 skill targeted:

Process and timing for data download:

Require Internet access? Yes/No

Security arrangements:

Data collection and analysis

Method of data collection:

Duration of data collection:

Method of data analysis:

Outcomes

Empowerment: Yes/No

Motivation: Yes/No

Attitudes & coping skills: Yes/No

Feasibility: Yes/No

Usefulness: Yes/No

Utilisation: Yes/No

Safety impacts: Yes/No

Acceptability: Yes/No

Accessibility: Yes/No

Convenience: Yes/No

Preference, perceptions and views: Yes/No

Experience: Yes/No

Results

Empowerment

Motivation

Attitudes & coping skills

Feasibility

Usefulness

Utilisation

Safety impacts

Acceptability

Accessibility

Convenience

Preference, perceptions and views

Experience

Other outcomes

Authors conclusion

Reviewer’s conclusion

Appendix 5

The risk of bias assessment of included randomised controlled trials (adapted from Higgins et al.,

2011)31

Name of the reviewer:

First author surname year of publication:

Bias domain Source of bias Support for judgment* Authors’

judgment**

Selection bias Random sequence generation

Allocation concealment

Performance

bias Blinding of participants and personnel

Detection bias Blinding of outcome assessors to intervention

allocation

Attrition bias Incomplete outcome data

Reporting bias Selective reporting of the outcome, subgroups, or

analysis

Other bias

Bias due to problems not covered elsewhere; such as

funding source, adequacy of statistical methods used,

type of analysis [ITT/PP], baseline imbalance in

important characteristics

ITT=intention to treat; PP=per protocol

* Statement, description or quote supporting the judgment

** Low risk of bias, high risk of bias, or unclear risk of bias

Appendix 6

The risk of bias assessment of included non-randomised controlled trials (adapted from Higgins et al.,

2011)31

Name of the reviewer:

First author surname year of publication:

Bias domain Source of bias Support for judgment* Authors’

judgment**

Selection bias

Comparable baseline characteristics between-group

(e.g., age, sex, duration and severity of diabetes,

complications, treatment, pre-study compliance, co-

intervention)

Performance

bias Blinding of participants and personnel

Detection bias Blinding of outcome assessors to intervention

allocation

Attrition bias Incomplete outcome data

Reporting bias Selective reporting of outcomes, subgroups, or

analysis

Other bias

Bias due to problems not covered elsewhere; such as

funding source, adequacy of statistical methods used,

type of analysis [ITT/PP]

ITT=intention to treat; PP=per protocol

* Statement, description or quote supporting the judgment

** Low risk of bias, high risk of bias, or unclear risk of bias

Appendix 7

Methodological quality assessment of qualitative studies (CASP Checklist)32

Name of the reviewer:

First author surname year of publication:

Screening Questions YES Can’t tell No IS THIS PAPER WORTH READING? 1. Was there a clear statement of the aims of the research? HINT: Consider

What was the goal of the research? Why it was thought important? Its relevance

2. Is a qualitative methodology appropriate? HINT: Consider

If the research seeks to interpret or illuminate the actions and/or subjective experiences of research participants

Is qualitative research the right methodology for addressing the research goal?

Is it worth continuing? ARE THE RESULTS CREDIBLE? 3. Was the research design appropriate to address the aims of the

research? HINT: Consider

If the researcher has justified the research design (e.g. have they discussed how they decided which method to use)?

4. Was the recruitment strategy appropriate to the aims of the research? HINT: Consider

If the researcher has explained how the participants were selected If they explained why the participants they selected were the most

appropriate to provide access to the type of knowledge sought by the study

If there are any discussions around recruitment (e.g. why some people chose not to take part)

5. Was the data collected in a way that addressed the research issue? HINT: Consider

If the setting for data collection was justified If it is clear how data were collected (e.g. focus group, semi-structured

interview etc.) If the researcher has justified the methods chosen If the researcher has made the methods explicit (e.g. for interview

method, is there an indication of how interviews were conducted, or did they use a topic guide)?

If methods were modified during the study. If so, has the researcher explained how and why?

If the form of data is clear (e.g. tape recordings, video material, notes etc) If the researcher has discussed saturation of data

6. Has the relationship between researcher and participants been adequately considered?

HINT: Consider If the researcher critically examined their own role, potential bias and

influence during a. Formulation of the research questions b. Data collection, including sample recruitment and choice of

location How the researcher responded to events during the study and whether

they considered the implications of any changes in the research design

WHAT ARE THE RESULTS? 7. Have ethical issues been taken into consideration? HINT: Consider

If there are sufficient details of how the research was explained to participants for the reader to assess whether ethical standards were maintained

If the researcher has discussed issues raised by the study (e.g. issues around informed consent or confidentiality or how they have handled the effects of the study on the participants during and after the study)

If approval has been sought from the ethics committee

8. Was the data analysis sufficiently rigorous? HINT: Consider

If there is an in-depth description of the analysis process If thematic analysis is used. If so, is it clear how the categories/themes

were derived from the data? Whether the researcher explains how the data presented were selected

from the original sample to demonstrate the analysis process If sufficient data are presented to support the findings To what extent contradictory data are taken into account Whether the researcher critically examined their own role, potential bias

and influence during analysis and selection of data for presentation

9. Is there a clear statement of findings? HINT: Consider

If the findings are explicit If there is adequate discussion of the evidence both for and against the

researchers arguments If the researcher has discussed the credibility of their findings (e.g.

triangulation, respondent validation, more than one analyst) If the findings are discussed in relation to the original research question

ARE THE FINDINGS OF THIS STUDY TRANSFERABLE TO A WIDER POPULATION? 10. How valuable is the research? HINT: Consider

If the researcher discusses the contribution the study makes to existing knowledge or understanding e.g. do they consider the findings in relation to current practice or policy? Or relevant research-based literature?

If they identify new areas where research is necessary If the researchers have discussed whether or how the findings can be

transferred to other populations or considered other ways the research may be used

Appendix 8

GRADE evidence profile for gradable outcomes (adapted from Guyatt et al., 2011)38

Quality assessment

Summary of findings

No of patients Effect Quality of

the evidence (GRADE)*

Outcome No of

studies Study design

SROB across studies

Limitations

Inconsistency

Indirectness

Imprecision

Publication bias

Intervention Comparator

Pooled effect

estimate (95% CI)

Intervention 1 vs. Intervention 2 (n studies) Outcome 1 Outcome 2 Outcome 3 Outcome 4 Intervention 1 vs. Intervention 3 (n studies) Outcome 1 Outcome 2 Outcome 3 Outcome 4 GRADE= Grading of Recommendations, Assessment, Development, and Evaluation; CI=confidence interval; SROB=summary risk of bias

*GRADE categories: high, moderate, low, very low