Enhancing the Delphi method in Health decision- making: … · among them the Delphi Method. In this method, worldwide participants can advocate for their points of view and discuss

Enhancing the Delphi method in Health decision-

making: Designing a Methodology to get Insights from

Participants’ Text Answers

Cátia Sofia Araújo Franco

Thesis to obtain the Master of Science Degree in

Biomedical Engineering

Supervisor: Professor Doutora Ana Catarina Lopes Vieira Godinho de Matos

Professor Doutora Mónica Duarte Correia de Oliveira

Examination Committee

Chairperson: Professor Mário Jorge Costa Gaspar da Silva

Supervisor: Professor Doutora Ana Catarina Lopes Vieira Godinho de Matos

Members of Committee: Professor João Carlos Da Cruz Lourenço

October 2019

ii

Declaration

I declare that this document is an original work of my own authorship and that it fulfills all the

requirements of the Code of Conduct and Good Practices of the Universidade de Lisboa.

iii

Preface

The work presented in this thesis was performed at the Management Study Center of Instituto

Superior Técnico (Lisbon, Portugal) under the supervision of Professor Doutora Ana Catarina

Lopes Vieira Godinho de Matos and Professor Doutora Mónica Duarte Correia de Oliveira.

iv

Acknowledgments

I would like to thank to Professor Ana Vieira and Professor Mónica Oliveira for all the aid, support

and guidance through this dissertation.

For my family I’m thankful for the eternal support and words are not enough to describe the

appreciation that I have for you. To all my friends thank you, without you this work will be less fun.

Abstract

There are several ways of engaging health stakeholders in the process of decision making,

among them the Delphi Method. In this method, worldwide participants can advocate for their

points of view and discuss them. The traditional Delphi encloses a first open round and the

subsequent analysis of the participants’ answers. This process is very time consuming and prone

to numerous biases, as it is usually performed manually by analysts without any automation tools,

which leaves scope for improvement. This thesis proposes a methodology to automatically

process answers from a Delphi which will enhance participation and collaboration in Health

decision-making. The EURO-HEALTHY project was used as a case study. This project aimed for

the development a Population Health Index (PHI) and the construction of scenarios to inform the

evaluation of policies. The methodology of this dissertation has four phases and followed the

concept of Content Analysis to extract the drivers using the software NVivo. Following the

methodology described in this thesis, 9 themes emerged with 218 associated drivers, compared

with 6 themes and 178 associated drivers derived from the EURO-HEALTHY package. Three of

these themes were the same in this dissertation and in the case study namely Economic, social

and environmental. This work allowed to automatically gather unbiased results when compared

to the ones obtained by analysts, while also decreasing the time of the analysis

Keywords: Methods of Data Collection, Delphi, Content Analysis, NVIVO, EURO-HEALHTY;

vi

Resumo

Existem muitos modos de envolver os stakeholders de saúde no processo de decisão, um deles

é através do método de Delphi. Neste método, participante do mundo inteiro podem defender e

discutir os seus pontos de vista. O Delphi tradicional comtempla uma primeira ronda aberta e

uma análise subsequente dos resultados. Este é um processo demorado e que pode levar a

inúmeros enviesamentos já que é normalmente feito manualmente pelos analistas sem ajuda de

automatismos o que deixa aqui uma oportunidade de melhoria. Esta tese propõe uma nova

metodologia para obter informação sobre as respostas de um Delphi aumentando assim a

participação e a colaboração na decisão em Saúde. O projeto EURO-HEALTHY foi usado como

caso de estudo. Este projeto pretendia desenvolver um índice de Saúde da População (INES) e

a construção de cenários para informar nas avaliações de políticas. A metodologia desta

dissertação tem 4 fases e segue o conceito de análise de conteúdo para recolher drivers usando

o software NVivo. Deste processo um total de 9 temas e 218 drivers emergiram diferente do

EURO-HEALHTY onde surgiram 6 temas e 178 drivers. Cada tema tem drivers associados.

Observou-se que há temas em comum nesta dissertação e no EURO-HEALTHY nomeadamente

os nós ambiental, económico e social; os drivers associados a cada nó também são semelhantes.

Nesta dissertação, foi possível diminuir o tempo gasto na análise das respostas.

Key-Words: Métodos de Recolha de Informação, Delphi, Content Analysis, NVIVO, EURO-

HEALHTY;

vii

Content

Acknowledgments ..................................................................................................................... ii

Abstract ..................................................................................................................................... v

Resumo ......................................................................................................................................vi

Content ..................................................................................................................................... vii

List of Abbreviations .................................................................................................................. x

List of Tables and figures ........................................................................................................... xi

1. Introduction....................................................................................................................... - 1 -

2. Literature Review of Concepts .......................................................................................... - 3 -

2.1 Participation .................................................................................................................... - 3 -

2.1.1 Health Participation ..................................................................................................... - 3 -

2.1.2 Participation in other fields .......................................................................................... - 4 -

2.2 Methods of data collection and Analysis ........................................................................ - 4 -

2.2.1 Questionnaire ............................................................................................................... - 5 -

2.2.2 Interviews ..................................................................................................................... - 5 -

2.2.3 Group Participation ...................................................................................................... - 5 -

2.3- The Delphi Method ........................................................................................................ - 7 -

2.3.1 Design of the Delphi ..................................................................................................... - 8 -

2.3.2 – Design and Implementation of a Delphi Process .................................................... - 10 -

2.3.2.1- Phases of a Delphi .................................................................................................. - 10 -

2.3.2.2- Selection of Participants ........................................................................................ - 11 -

2.3.2.3- Round 1 design ....................................................................................................... - 11 -

2.3.2.4- Other rounds design .............................................................................................. - 12 -

2.3.2.5- Stopping Criteria .................................................................................................... - 13 -

2.3.2.6- Level of Agreement ................................................................................................ - 13 -

2.3.2.7- Feedback and Anonymity ....................................................................................... - 13 -

3. Qualitative Data analysis and techniques of Text Processing ......................................... - 14 -

3.1- Machine Learning ......................................................................................................... - 15 -

3.2- Natural Language Processing ....................................................................................... - 15 -

3.3- Jumping Curves: A new approach ................................................................................ - 16 -

viii

3.4- Classical Content Analysis ............................................................................................ - 17 -

3.5. Phases of the process ................................................................................................... - 17 -

3.5.1 Preparation phase ...................................................................................................... - 17 -

3.5.2 Organization Phase .................................................................................................... - 17 -

3.5.3 Reporting Phase ......................................................................................................... - 18 -

3.5.4 Types of approaches: Inductive and Deductive ......................................................... - 18 -

3.6 – The software chosen: NVivo ....................................................................................... - 19 -

4. Case-study ....................................................................................................................... - 21 -

4.1- Overview of the Case-study Methodology: The EURO-HEALTHY Study ...................... - 21 -

4.2- Methodology for the manually identification of drivers .............................................. - 22 -

4.2.1- Gathering of information in a Web-Delphi study ..................................................... - 23 -

4.2.2- Define the criteria to be a driver............................................................................... - 24 -

4.2.3- Identify the reasons through the search of coordinating conjunction ..................... - 24 -

4.2.4 -Search for redundancy between the reasons previously identified ......................... - 24 -

4.2.5- Cluster drivers into PESTLE categories ...................................................................... - 24 -

5. Methodology ................................................................................................................... - 26 -

5.1- Proposed General Methodology: A Content Analysis Approach ................................. - 26 -

5.2- Adapting the general methodology to this dissertation .............................................. - 28 -

5.3- Methodology applied to the Case-study ...................................................................... - 28 -

5.3.1- Gathering of information from the Web-Delphi already performed and Define the

criteria to be a driver ........................................................................................................... - 29 -

5.3.2- Perform automated insights using NVIVO to obtain potential drivers ..................... - 29 -

5.3.3- Filtering the data in NVIVO to avoid redundancy, see if it is a driver and Refine the

Clusters using specific queries ............................................................................................ - 30 -

5.3.4- Comparasion between this thesis and the EURO-HEALTHY methodology ............... - 31 -

6.Results .............................................................................................................................. - 33 -

6.1- Results from Gathering of information from the Web-Delphi already performed ..... - 33 -

6.2 - Results from the Content Analysis using NVivo: Perform automated insights using

NVIVO to obtain potential drivers ....................................................................................... - 33 -

6.2.1- Hierarchical Structure produced with the Software NVivo derived from step 2 of the

methodology ....................................................................................................................... - 34 -

ix

6.3- Results from the themes obtained in this dissertation after the Filtering of the data in

NVIVO to avoid redundancy and see if it is a driver and Refine the Clusters using specific

queries ................................................................................................................................. - 35 -

6.3.1- Ageing ....................................................................................................................... - 35 -

6.3.2- Economic ................................................................................................................... - 35 -

6.3.3- Education .................................................................................................................. - 35 -

6.3.4- Employment .............................................................................................................. - 36 -

6.3.5- Environment .............................................................................................................. - 36 -

6.3.6-Health ......................................................................................................................... - 36 -

6.3.7- Policies ...................................................................................................................... - 37 -

6.3.8-Road Safety ................................................................................................................ - 37 -

6.3.9-Social .......................................................................................................................... - 37 -

6.4- Analyse of the results : Comparation between the results from the EURO-HEALTHY

Study and the Results from this Dissertation ...................................................................... - 37 -

6.4.1- Comparation between the node Economic .............................................................. - 39 -

6.4.2- Comparation between the node Social .................................................................... - 40 -

6.4.3- Comparation between the node Environmental ...................................................... - 40 -

7 -Discussion ........................................................................................................................ - 41 -

7.1- Comparasion between the manual methodology and NVivo ..................................... - 41 -

7.2- Nvivo Pros and Cons ..................................................................................................... - 41 -

7.3- Comparison between the EURO-HEALTHY Clustering and the Clustering from this

dissertation.......................................................................................................................... - 41 -

7.4 - Time used by the investigators ................................................................................... - 42 -

7.5-Technology and Data Representation .......................................................................... - 42 -

7.6 -Level of Detail of each investigation ............................................................................ - 43 -

8. Conclusions and Future Work ......................................................................................... - 44 -

References ........................................................................................................................... - 46 -

x

List of Abbreviations

AI Artificial Intelligence

CA Content Analysis

EURO-HEALTHY Shaping EUROpean policies to promote HEALTH equity INES Índice de Saúde da População

ML Machine Learning

NLP Natural Language Processing

PAME Participatory Assessment, Monitoring and Evaluation Techniques

PESTLE Political, Economical, Social, Technological, Legal, Environmental

PH Population Health

PHI Population Health Index

xi

List of Tables and figures

Tables

Table 1-Methods of data collection: Advantages and Disadvantages adapted from (Slocum, 2003)

................................................................................................................................................... - 5 -

Table 2- Types of Delphi adapted from (Hasson & Keeney, 2011) .......................................... - 9 -

Table 3 - Types of Qualitative Analyses Methods; Adapted from (Onwuegbuzie, Leech, & Collins,

2012) ....................................................................................................................................... - 14 -

Table 4 – Performing Content Analysis: NVivo Functions used in this methodology- When and

why to use it ............................................................................................................................. - 31 -

Table 5- Machine Learning Methods ....................................................................................... - 49 -

Table 6 – Table of syntactic, semantic and pragmatic curve. Adapted from Bush, Bryce, & Direito,

2016 ............................................................................................................................................. 51

Figures

Figure 1. Delphi Method Flowchart (Slocum, 2003) .................................................................. - 8 -

Figure 2- Key aspects of Content analysis - Adapted from (Shannon,2005) .......................... - 18 -

Figure 3- NVivo interface (“O poderoso NVivo: Uma introdução a partir da análise de conteúdo,”

2016) ....................................................................................................................................... - 19 -

Figure 4 – Methodology used in NVivo from (Dollah et al., 2017) .......................................... - 20 -

Figure 5 - overview of the EURO-HEALTHY methodology using information from (Alvarenga et

al., 2019) .................................................................................................................................. - 22 -

Figure 6 - Overview of the procedure to manually build EURO-HEALTHY scenarios ............ - 22 -

Figure 7 - Print Screen of the Web-Delphi platform concerning the Economic conditions, social

protection and security adapted from EURO-HEALTHY WP7 working materials. ................. - 23 -

Figure 8 - Proposed Methodology based in CA (Shannon,2005) ........................................... - 27 -

Figure 9 – a) and b) Comparison between the methodology for the manually identification of

drivers (left side a)) and the proposed methodology to automatically identify the drivers (right side

b)); ........................................................................................................................................... - 29 -

Figure 10 -Results from the Content Analysis – Nodes and number of drivers ...................... - 34 -

Figure 11 - Themes obtained in this dissertation VS PESTLE ............................................... - 39 -

- 1 -

1. Introduction

1.1- Context

Nowadays participation is a topic widely recognized. There are a lot of different participation methods:

individual and group participation methods. Questionnaires and interviews are examples of individual

methods of participation. The problems of these methods are that the input comes from only one person

and with that the diversity of opinions decrease (Jorm, 2015). In group participation methods, there are

different opinions that comes from different heads. Examples of that type of methods are the focus

group, charrette and world café. These methods have other problems such as these type of methods

are presential and to join a group of people in the same room at the same time is quite difficult (Jorm,

2015).

Health is a worldwide important topic that affects the whole population. Health participation is a topic

that has been gaining traction and consists on individuals being entitled to participate in the health

decisions that directly affect them, including the design, implementation, and monitoring of health

interventions (Cook et al., 2018). Deciding for a treatment, for a hospital or even for a doctor are

problems that can be reduced if patients have the necessary knowledge on the field. If this happened,

patients are empowered to manage their own health and health care as well as the health of their

families. Patient participation, redistribution of power and acknowledging the patients competency

regarding this topic are the main topics of patients empowerment (Kvæl, Debesay, Langaas, Bye, &

Bergland, 2018, p.3).

1.2- Objective

This master thesis intends to enhance participation and collaboration in Health decision-making through

the development of a methodology to get insights from participant’s answers obtained within the context

of implementing the Delphi method; and the methodology is developed for the context of a study about

Health inequalities across Europe. For that, the information generated in a previous research study

developed in the context of the EURO-HEALTHY research project is used (Alvarenga et al., 2019).

To develop the methodology, it is necessary to study the methods of participation that are frequently

used in health contexts, with a special emphasis on the Delphi Method. The Delphi method is a

structured communication technique originally developed as participation method that brings opinions

from experts from different fields worldwide through a questionnaire (Slocum, 2003). This method is

taken as the information setting context to this dissertation. In some Delphi formats, after collecting

opinion data in a text format from the experts/patients/population/physicians/stakeholders, an analysis

needs to be performed using techniques of text processing that enables the automatic process of the

patients answers questionnaire (Slocum, 2003). These types of methods help in the treatment of the

data that we can acquire from different sources and reduce the time consumed in the treatment and

analysis.

1.3- Chapters Description

- 2 -

The following chapter – review of concepts - contains a description of the context where the theme of

this dissertation emerged, the concept of participation and more specifically health participation. Other

relevant concepts to better understand the content of this thesis, such as methods for data collection

and analysis are also explained. Finally, a comparison between distinct data collection method is

presented.

The third chapter refers to the literature review and describes the type of data collection that exists as

well as the approaches that were used in this master thesis. It starts with a description of the Delphi

method and related activities, including round design and the stop criteria. An explanation of text

processing techniques that are relevant and can be incorporated into the work of this thesis is also

provided. These techniques include Machine Learning (ML), Natural Language Processing (NLP) and

Classical Content Analysis (CA).

The following chapter describes the research worked carried out within the EURO-HEALTHY project

regarding scenario building and that sets the grounds to test the new methodology developed on this

dissertation. The study on scenario building was carried out within “WP6 Multicriteria modelling of the

population health index and evaluation, foresight and selection of policies” that was designed,

essentially, to inform the evaluation of policies and what can affect health and health inequalities across

European regions in the future (Alvarenga et al., 2019).

The fifth chapter refers to the developed methodology and, subsequently, to the tools that are used.

Initially an explanation of how to use the methodology is given, followed by a description of the different

stages of the methodology. It also describes in detail the tools and software that are used.

The sixth chapter refers to the results obtained in this dissertation, as well as an analysis and a

comparison between the results obtained and the results from the EURO-HEALTHY project (Alvarenga

et al., 2019). Lastly, chapter seven is the conclusions that arise may from this dissertation and possible

future work.

- 3 -

2. Literature Review of Concepts

2.1 Participation

Participation is a topic that we deal with in our daily lives and therefore can be defined in different ways.

It is used in different fields such as health, economics and politics. In the study of Jewell, citing

Brownlea, participation is defined as “getting involved or being allowed to become involved in a decision-

making process or the delivery of a service or the evaluation of a service, or even simply to become one

of a number of people consulted on an issue or matter” (Jewell, 1994, p.434).

Depending on the field of study, participation should involve as many persons as needed. Some studies

involve the participation of experts, others the public. It is suggested by Jorm (Jorm, 2015) that a group

opinion can produce better results than the best individual experts (Jorm, 2015) which is an indicative

that participation will enhance the decision making.

More specifically, in the field of health which is the focus of this master thesis, to improve the health

system and health decision making it is important to take into consideration different stakeholders

according to the aim of the study. It could vary between patients, doctors or even the public.

In a study of Basco-Carrera (Basco-Carrera, Warren, van Beek, Jonoski, & Giardino, 2017), it is defined

a ladder of participation that differentiates among seven levels of stakeholders’ engagement. The level

of engagement increases from the bottom to the top of the ladder. In the first level- Ignorance- the

stakeholders do not know what is happening. In the second level- Awareness- the stakeholders start to

know that something is happening. After this level, stakeholders start to have information. In the third

level- Information- the stakeholders receive information and in the fourth- Consultation- the stakeholders

are consulted. In the last three levels, the interaction between the stakeholders increase. In the fifth

level-Discussion- is a two-way interactive relationship between stakeholders. Then there is the co-

design level in which the stakeholders feel sense of ownership and the last level – the co-decision

making – where stakeholders have mandate to act (Basco-Carrera, Warren, van Beek, Jonoski, &

Giardino, 2017).

2.1.1 Health Participation

Health participation has increase since the World Health Organization’s Alma Ata Declaration asserted

people’s “right and duty to participate individually and collectively in the planning and implementation of

their health care” (Almeida, 2016,pg.1). An idea that participation “makes for better citizens, better

decisions and better government”, (Almeida, 2016,pg.1) along with evidence that participation can

actually help in health fostered investment in public and patient participation (Almeida, 2016).

Patient participation can be defined as “a patient’s rights and opportunities to influence and engage in

the decision making about his care through a dialogue attuned to his preferences, potential and a

combination of his experiential and the professional’s expert knowledge” (Castro, Van Regenmortel,

Vanhaecht, Sermeus, & Van Hecke, 2016, p.1929). In this specific case, the decision making becomes

more patient centered (Kvæl et al., 2018), exists a preoccupation in a relationship between the staff and

the patient based in shared knowledge, sensitivity and trust (Kvæl et al., 2018).

- 4 -

There are different levels of patient’s participation: micro – individual care, meso- service development

and macro- policy (Kvæl et al., 2018). Most of the times it is important to involve the patient’s family in

the health decision making to improve the quality of care. A study about patient participation in

intermediate care revealed that communication is essential between the patient, medical staff and the

family in order to increase trust (Kvæl et al., 2018). The medical staff should explain the medical

conditions with a familiar terminology for a better understanding to the patients and families. Nowadays,

the patient participates in his heath decision making so it is important to him to be health literate (Kvæl

et al., 2018).

To better understand the concept of participation and to compare between papers, it was made a

research in other fields such as energy, politics and finances.

2.1.2 Participation in other fields

In a study about the Society and Nuclear Energy (McCarthy, 2002), the author defended the need for

an augmentation of public participation on decision making. McCarthy says that participation is one of

the five principles of good governance that this relies on policies proposed with a basis of reasonable

decisions that were communicated and discussed with the public (McCarthy, 2002). Another Study about

the Radioactive Waste defends that there is an increasing demand for stakeholder involvement to

improve the quality and sustainability of policy decisions. Contrary to the past, nowadays the time

spending dialoguing and bringing new and improved ideas to the table is a time well spent (Australia,

1971).

A study of Hsiao & Tsai shows the relationship between the literacy and participation in the financial field

and the authors conclude that a high literacy of the individuals contributes for a participation on the

derivative markets. In this case financial literacy is a determinant on financial decision making (Hsiao

& Tsai, 2018). In McCarthy’s study about brownfields redevelopment, he defends that the involvement

of the community can prevent protests and also provide ideas about economic activities that fit the needs

of the people (McCarthy, 2002).

2.2 Methods of data collection and Analysis

There are a few types of methods for data collection both at an individual and group level but the most

used in qualitative health research are interviews and questionnaires (Gill, Stewart, Treasure, &

Chadwick, 2008). These are individual methods for data collection. Alternatively, Rozados (Rozados,

2015) affirms that several heads think better than one since the number of facts considered on the case

of a group is bigger than in individual opinion. Each specialist of the group could bring more knowledge,

providing more information to the discussion (Rozados, 2015) which will increase acceptance and

legitimacy (Rowe & Wright, 1991). However, group methods have some inconveniences such as the

social pressure that the individuals suffer to choose a certain option over other forms. Also, group

methods are time consuming, have ambiguous responsibility and sometimes problems with minority

domination and unequal participation (Langton and Robbins, 2007).

- 5 -

In the next section it will be defined some concepts about individual and group participation and a

distinction between both concepts will also be made.

2.2.1 Questionnaire

Questionnaires are the most common way of collecting information in which a set of questions are

developed to collect qualitative and quantitative information (Boynton & Greenhalgh, 2004). It is

important to know when to use a questionnaire and if this type of questionnaire was already made and

validated. For example, health research very often uses standard questionnaires. In this method, most

of the differences in results come from differences across the participants (Boynton & Greenhalgh,

2004).

Since paper survey response rates have been declining over the past decade, another way of doing

questionnaires has been considered: the web-based questionnaire. Web based questionnaires have

less costs and could be an alternative to the classical questionnaire (Hohwü et al., 2013). Another way

of improving response rates is adding monetary incentives, Hohwü et al verify an increase of 2

percentage points by the use of US $5 versus a US $2 incentive questionnaire (Hohwü et al., 2013).

2.2.2 Interviews

Interviews are a method to obtain information through a series of questions performed by an expert. The

interviews can be performed in person or by telephone to collect and discuss the information needed

(Gill et al., 2008). Questions should be well prepared from simple to complex to, progressively, gain the

confidence of the interviewee (Gill et al., 2008). This method is divided into three types: structured, semi-

structured and unstructured interviews (Gill et al., 2008). Structured interviews are a type of pre-defined

questionnaire that is fast to administer. Unstructured interviews are performed with no previous

preparation of a questionnaire and are time consuming because of the nature of the open questions and

responses. The most used in health, semi-structured interviews, consist of defining a set of questions

within an area to be explored and then pursue an idea (Gill et al., 2008).

2.2.3 Group Participation

The level of participation by the stakeholders will be distinct according to the objectives of the studies

and it should be defined previously (Australia, 1971).

In the table 1, a selection of group participatory methods is presented, together with their advantages

and disadvantages:

Table 1-Methods of data collection: Advantages and Disadvantages adapted from (Slocum, 2003)

Method Aim of the Study Participants Advantages Disadvantages

Charrette Create consensus among the panel and form an action plan.

Stakeholders and average citizens.

Widely used in problems less complex; require a short period of time.

Needs a recorder

- 6 -

The charrette is used when it is necessary to reach consensus in a short period of time (Slocum, 2003).

The objective is to divide a main topic into parts and assign people to each sub-group. Feedback from

all the parts will be needed to share the conclusions among all the group members and advance to the

next round. This sequence is repeated until consensus is reached (Slocum, 2003).

The Citizens Jury method is a way of involving the common public into policy decisions. One difference

from the other methods is that the jury is composed of randomly selected citizens, who are informed by

several perspectives. Sub-groups are formed to help in the deliberation process and to focus on different

aspects. A final report is produced by the jurors (Slocum, 2003).

Citizens Jury Turning input into policy decisions.

Randomly selected citizens.

Require a short period of time; Democratic decision making.

Given the high complexity of the problems it’s difficult to have a good sample of the population.

Consensus Conference

Create consensus among the panel and decide a controversial topic.

Randomly selected citizens.

Real time process.

Require a long period of time.

Delphi Achieve consensus on a complex issue.

Experts.

Most of the times is an online process.

Sometimes, people simply follow the other opinions.

Expert Panel Aggregate ideas and produce recommendations; highly technical and require complex decisions.

Experts. Different visions of the problem can produce better decisions.

Difficult to reunite all the panel together.

Focus Group Use groups opinions on an issue and why these holds.

Stakeholders and average citizens.

Short period of time.

Not all participants give feedback.

PAME (Participatory Assessment, Monitoring and Evaluation Techniques)

Evaluating and learning.

Stakeholders. Highly certain on the events;

Require a long period of time.

Planning Cells

Citizens choose between different options.

Average citizens, experts and stakeholders.

Short period of time.

Not always executable.

Scenarios Vision-building. Anyone. Gives a notion of evolution in the process.

Sometimes people think that the scenarios are the possible futures.

World Café Share of ideas on a café environment.

Anyone. Informal environment.

No decision at the end.

- 7 -

Consensus conference is a means of involving public citizens in a socially controversial topic. Usually,

it is organized at a national level (Australia, 1971). The goal is to broaden the debate on a given issue

and include the viewpoints of non-experts to inform policy-making person (Slocum, 2003).

The Expert panel is used in more technical subjects and require complex decisions, so the participants

need to be experts in that subject (Slocum, 2003).

Focus group is a discussion among a small group of stakeholders facilitated by a moderator. It is a

combination of a focused interview and a discussion group (Slocum, 2003).

The PAME involves the stakeholders of a given project to discuss the future decisions of an enterprise

(Slocum, 2003).

The planning cell select randomly a small group of people forming cells to present solutions for a problem

(Slocum, 2003).

Scenarios involves multiple options of a given hypothesis to the future. It studies the relationship

between events and decision points (Slocum, 2003).

The world café is a different concept in decision making because it happens in a café environment to

people get more comfortable sharing their thoughts. The participants discuss the subject in a café table

and then change to another table. There is always one “host” that remains in the table to summarize the

previous conversation to the next people in the table (Slocum, 2003).

In a Delphi process, participants are experts in a certain area and will participate in a series of

discussions until consensus is reached. It is an anonym process so all the participants have equal

consideration in their views (Australia, 1971). More about this process will be discuss in the next chapter

since it is the focus of this thesis.

2.3- The Delphi Method

The Delphi method is mostly distinguished from the other methods because besides allowing the

gathering of data, it allows the anonymity of the panel and an asynchronous participation from experts

that can be located across the world (Sekayi & Kennedy, 2017).

The Delphi method was created by Helmer and Dalkey in 1953 at the RAND corporation with the aim to

solve military issues. Notwithstanding, this technique became more commonly utilized 10 years later in

the technological forecasting and corporate planning (Lang, 2000). It was studied the effect of using

group information instead of statistical treatment of individual opinions and introduced the concept of

iteration with controlled feedback (Dalkey, 1969).

The Delphi method consists in gathering specific and complex information from different experts that

work in different fields (Renzi & Freitas, 2015). It is a method characterized by combining diverse

opinions using a questionnaire to produce a group opinion and eliminate problems that could arise from

a presential meeting. Usually, it is used in cases with no means of prediction confirmation and to identify

ruptures or innovation in a specific field of knowledge (Renzi & Freitas, 2015). It is important to say that

- 8 -

the final aim of this process is not to produce consensus but to obtain a reliable group opinion using a

group of experts (Guzys, Dickson-Swift, Kenny, & Threlkeld, 2015).

In the Delphi method, each participant will fill a questionnaire that was previously made by the

researchers of the study and then return it to the facilitator. After this, the experts will receive feedback

with the whole set of responses. After this feedback, and in another round, they fill the questionnaire

again and have the opportunity to revise their initial beliefs or provide the other members of the Delphi

panel with better explanations on their own point of view. This process will be repeated as many times

as needed; however, the stopping criteria is usually defined at the start of the process to know when to

finish the study. The goal is that the entire group can benefit from the different fields of expertise of the

panel. Thus, in most Delphi processes the amount of consensus increases from round to round (Slocum,

2003).

A flowchart of a typical Delphi process is shown in figure 1, using information from (Slocum, 2003).

Figure 1. Delphi Method Flowchart (Slocum, 2003)

2.3.1 Design of the Delphi

There are many different Delphi formats and the type of Delphi used depends on the focus problem.

The Delphi method is based on structuring the information flow, give feedback to the participants, allow

anonymity and statistical aggregation of group response (Slocum, 2003) (Rowe & Wright, 1991). Some

subjects of the Delphi are more technical, while others are simpler, therefore implying different choices

of panels.

In the table 2 the type of Delphi is described according to the aim of the study, the information sources,

the panel composition, the administration of the Delphi, the stopping criteria and the type of design on

- 9 -

the first round. There is a certain variance within each type, for example the “modified” Delphi is used

when the level of complexity in the design is high, moreover the modified Delphi can employ a focus

group, interview, or results of a systematic review to develop the first round (Hasson & Keeney, 2011).

Table 2- Types of Delphi adapted from (Hasson & Keeney, 2011)

Type of Delphi

Objective Panel Compositio

n

Administration

Stop Criterio

n

Source of Informatio

n

Round 1 Design

Classical Discuss ideas and reach consensus.

Participants are selected based on the objectives.

Postal. Number of rounds: 3 or more.

Qualitative data for an open answer.

Open Answer: Qualitative.

Modified According to the aim of the project, from predicting future events to achieving consensus.


Postal or online.

Number of rounds: more than 3.

Qualitative data for a close answer.

Panellists provided with pre-selected items, drawn from various sources, within which they are asked to consider their responses.

Decision Structure decision-making and create the future.

Decision makers select based according to hierarchical position and level of expertise.

Postal, online, Decision conference.

Varies. Qualitative data.


Policy Create different views on policy decisions .

Policy markers.

Postal, online, Decision conference.



Real Time Discuss ideas and reach consensus.




E-Delphi Objective can vary depending on the nature of the research.


Via e-mail. Varies. Qualitative data.


Technological According to the aim of the

Participants are selected

Use of hand-held keypads allowing

Varies . Qualitative data.

Open Answer:

- 10 -

project, from predicting future events to achieving consensus.

based on the objectives.

responses to be recorded and instant feedback provided.

Qualitative.

Online According to the aim of the project, from predicting future events to achieving consensus.


Online. Varies. Qualitative data.


Argument Develop arguments and different opinions on a specific subject.

Participants representing the issue from different perspectives .

Varies. Varies. Qualitative data.

Similar to modified Delphi.

Disaggregative Policy

Constructs future scenarios in which experts are asked about the future.


Varies.

Varies. Adoption of modified format using cluster analysis.

The Classical, Real time and Modified are types of Delphi processes in which the principal aim is to

build consensus. The E-Delphi, Technological and online Delphi depends on the nature of the research.

The policy Delphi it to generate opposing views. The Decision Delphi is to structure decision making

and the Argument Delphi is to develop arguments and expose reasons. Lastly, the Disaggregative policy

is to construct future scenarios.

There are other characteristics that could vary between the Delphi such as, the level of anonymity and

feedback given, the inclusion criteria and the method of analysis. Since personal experience influences

the judgments of the experts, personal bias could occur at the Delphi (Hasson & Keeney, 2011).

2.3.2 – Design and Implementation of a Delphi Process

2.3.2.1- Phases of a Delphi

The Delphi method could be divided in four main phases. The preparation, the design, the

implementation and the evaluation (Hasson & Keeney, 2011).

- 11 -

The first phase – preparation- consist in identifying and select the experts, prepare the evidence (if

needed) and the data and prepare supporting letters (Hasson & Keeney, 2011).

The second phase - the Design - is important to define the type of first round used, the total number of

rounds, the number of questions and response categories, the feedback and the stopping criteria

(Hasson & Keeney, 2011).

The third phase – the implementation- is where it is defined aspects like timing, documentation and

results. The first aspect is important because the wider the time gap the higher the probability of a

change in individual’s opinions. The documentation of the results is also important because it is essential

to have a record of every point of view shared in the process (Hasson & Keeney, 2011).

The last phase consists in performing an evaluation of the process conducted. These phases are

optional according to aim of the Delphi. This phase should consider the reliability and validity of the

Delphi, the trustworthiness and the Post-group consensus (Hasson & Keeney, 2011).The reliability

refers to the similarity of results under constant conditions on different cases and the validity measures

the generalizability of the findings (internal) or the confidence and the effect relationship (external)

(Hasson & Keeney, 2011). The trustworthiness should consider aspects such as credibility,

dependability, confirmability and transferability (Hasson & Keeney, 2011). Lastly, the Post-group

consensus refers to the agreement of each individual in relation with the final group opinion (Hasson &

Keeney, 2011).

2.3.2.2- Selection of Participants

The participants should be chosen according to the field in which the study is being conducted. That is,

there should be participants from different areas within the same study and sometimes from different

geographic areas in order to promote difference of opinion and exchange of information (Hasson &

Keeney, 2011). Trevelyan & Robinson (Trevelyan & Robinson, 2015) advocate that the term “expert” is

contentious and that the experts are defined as “informed individuals” or “specialists”, someone with

experience and knowledge in the field (Trevelyan & Robinson, 2015) (Riggs, 1983). Another

consideration within the decision of the Delphi process is the heterogeneity or homogeneity of the panel.

On the one hand, choosing a diverse panel leads to a better performance since it allows a wide range

of alternatives and perspectives, on the other hand some studies could require a homogeneous group

depending on the objectives (Trevelyan & Robinson, 2015). The size of the panel can vary between 4

and 3000 participants (Riggs, 1983).

2.3.2.3- Round 1 design

The design of the first round depends on the type and objective of Delphi that will be used. A classical

Delphi approach, usually, uses open-ended questions to generate qualitative data – it is an exploratory

approach (Trevelyan & Robinson, 2015). It can also be used a Likert scale in which the respondent can

answer the question according to their level of agreement or disagreement (Trevelyan & Robinson,

2015).

- 12 -

Alternatively, the researcher could identify the issues and generate the data needed to this round

through literature review or consultation with stakeholders (Trevelyan & Robinson, 2015). In some

cases, the researchers collect data from questionnaires, interviews, and focus groups using an inductive

form of analysis (Trevelyan & Robinson, 2015).

Following the classical approach of a Delphi, the point of the first round of this method is to gather

individual opinions on a topic using electronic means that will be the basis for the next round

questionnaire that will be presented in a form of a series of statements (Sekayi & Kennedy, 2017).

Sekayi & Kenned proposed a qualitative version of this method since they find that “the path between

the raw data resulting from the brainstorming and the list of statements is not well defined” (Sekayi &

Kennedy, 2017, p.2756).

In a study about the consensus research priorities for paediatric status epilepticus, define the first round

as an open round in which clinical participants were asked to answer the main question of this study by

answering one question “Thinking about experience with paediatric convulsive status epilepticus, what

are the most important research questions that need addressing” (Furyk et al., 2018). This question was

posted electronically via e-mail and the participants should answer in a form of free text in PICO format

(Population, Intervention, Comparison, Outcome). After that, the questions from the round one is

aggregated into themes and transformed into mutually exclusive research question using the Nvivo 11

for Mac. The answers should be analysed using grounded theory, content analysis and open coding to

categorize items into themes and finally, the answers are revised and included in the next round of the

Delphi (Furyk et al., 2018). This study will be the basis to the methodology of this work that will be

presented in chapter 5.

2.3.2.4- Other rounds design

Typically, the second round takes form of a structured questionnaire including data from the previous

round which, in this master thesis case, is ranked on a Likert scale (Trevelyan & Robinson, 2015).

Notwithstanding, the optimal number of response categories is variable. Trevelyan & Robison (Trevelyan

& Robinson, 2015) suggest that studies with three or four-point scales have poor reliability and

discriminating power, but studies with more than ten categories could lead to inconsistency in category

interpretation and misleading results. Therefore, it is suggested an optimal number between seven and

nine (Trevelyan & Robinson, 2015). There is also a need to decide if the Likert scale will have a mid-

point or not (Trevelyan & Robinson, 2015). A Likert Scale is a scale used in survey research to classify

an answer. It can be divided in strongly disagree, disagree, neutral, agree and strongly agree.

In the third and next rounds, the panel is given feedback about the previous round. The feedback

includes any comments that the participants think are important and individual ranking for each

questionnaire item. The aim of this round and other subsequent rounds is to achieve a level of

agreement (Murry & Hammons, 1995).

- 13 -

2.3.2.5- Stopping Criteria

The Stopping criteria are the set of rules that define when to stop the Delphi. The team should decide if

the stopping criterion will be the number of rounds or if is until the panel reach the consensus (Hasson

& Keeney, 2011). A normal number of rounds it is between 3 and 4. Less than 3 rounds does not allow

to measure the stability of the answers (Trevelyan & Robinson, 2015).

2.3.2.6- Level of Agreement

The level of agreement is a measure of the extent to which participants agree with each other. In the

Murry & Hammons study (Murry & Hammons, 1995), a few methods were identified to achieve this level

of agreement depending on the type of data used being the most frequently used median scores above

a predefined threshold and a high level of agreement between panel members (Murry & Hammons,

1995).

2.3.2.7- Feedback and Anonymity

The controlled feedback is the key for success in the Delphi method and it will depend on the type of

Delphi used. The anonymity is achieved by allowing the participants to answer the questionnaire

privately without any social pressure (Rowe & Wright, 1991). It is essential to guarantee the anonymity

of the participants since one may be influenced by another member of the panel if they know who they

are (Hasson & Keeney, 2011). Bolder & Wright (Bolger & Wright, 2011) discuss two common situations

of how people in groups behave. First, if one member of the group is considered as an expert in that

field or if one member has more authoritarian in virtue of his position, the rest of the group will follow this

specific individual opinion. Second, usually the minority feel pressured to change their opinion and agree

with the majority since the minority feels inexpert (Bolger & Wright, 2011).

- 14 -

3. Qualitative Data analysis and techniques of Text Processing

Since the goal of this dissertation is to extract information from participant’s answers, it is important to

know the types of qualitative data analysis that exist as well as techniques of text processing.

Qualitative data refers to non-numeric information such as text documents, interviews, notes or even

audio and video recording and can be divided into some categories such as content analysis, text

mining, theme analysis, taxonomic analysis and discourse analysis. These categories are explained

below in the table 3.

Table 3 - Types of Qualitative Analyses Methods; Adapted from (Onwuegbuzie, Leech, & Collins, 2012)

Type of Analysis Description Applicability

Classical Content Analysis Counting the number codes

after systematically reducing

sources deductively or

inductively

Yes, provides insight

information on different levels

Text Mining Discover of semantic

information through the

naturally occurring text within

multiple sources

No, provides little insight and

since the source of this

dissertation is just one the

EURO-HEALTHY package it

is not applicable

Theme analysis Search for relationships

among domains as well as

how these relationships are

linked to the overall cultural

context

Partially, but is usually used to

derive cultural meaning

Taxonomic Analysis Categorizes the domains in a

pictorial representation

through a creation of

classification system

Partially

Discourse Analysis Selecting unique segments of

the language use and

examining in detail for

rhetorical organisation,

variability, accountability and

positioning

No, usually used in review

sections of empirical articles

Semiotics Using talk and text as

systems of signs under the

assumption that no meaning

No, not related to the aim of

this thesis

- 15 -

can be attached to a single

term

In the table 3 there was a description of some techniques used in Qualitative analysis as well as if these

tecnhiques are applicable to this work. In the study of Onwuegbuzie et al. (Onwuegbuzie et al., 2012),

they relate the source of the information with the type of qualitative tecnhique that can be used. In this

specific case, the source of information are the answers from the questionnaire that are in the form of a

document. And there is mentioned at least twelve techniques (Onwuegbuzie et al., 2012).

The classical content analysis is the method more appropriate due to provide insight information on

different levels (Onwuegbuzie et al., 2012) for this it is the one that will be used in this dissertation.

Beside these techniques referred above there are other techniques of text processing emerging that are

more technological advanced and uses artificial intelligence such as Natural Language Processing

(NLP) and Machine Learning (ML) that could be used in this dissertation. For that, a brief explanation of

these techniques will be given in the next section

3.1- Machine Learning

The concept of Machine Learning is one of the most important on the field of artificial intelligence and

nowadays it is applied broadly across the world in daily activities. It was first described in 1950 by Alan

Turing which propose the “Turing Test” in which a computer should be able to impersonate a human

(Fernandez Montenegro & Argyriou, 2017). Essentially, this method makes uses of computer algorithms

to autonomously learn from data and information thought a simulation of the human learning (Portugal,

Alencar, & Cowan, 2018).

This technique can be divided into: Supervised, unsupervised, semi-supervised and Reinforcement

Learning. In Supervised Learning the point is to map a function from the input to the output providing

training data and correct answers to ML algorithm. The ML algorithm task is to learn based on this

information. On the other hand, in unsupervised Learning, there is no training set. The aim in this type

is to learn from the data on their own trying to find hidden patterns in the data. The semi-supervised

Learning is a mixture of both supervised and unsupervised learning, it uses a training set but with

missing information. Lastly, there is the Reinforcement learning which is based on external feedback. If

the algorithm perform well the task it will have a positive feedback and if not will have a negative

feedback (Portugal et al., 2018).

To perform Supervised and Semi-supervised Learning it is necessary to have a training set which is not

the case of this work, so these two methods are not applicable. The unsupervised learning doesn’t need

a training set but has another limitation: the size of the data is too small. A table detailed with examples

of types of ML are presented in the section Annexes.

3.2- Natural Language Processing

- 16 -

Natural Language Processing is used for example in Google, in translation problems and to perform

information retrieval. NLP was first mentioned in 1954 in the IBM-Georgetown Demonstration where it

was showed an automatic translation from Russian to English. A few years later it start to perform

information retrieval, which became an important tool for the day-to-day life (Mohamed, 2018).

In Liddy’s work Natural Language Processing was defined as a “theoretically motivated range of

computational techniques for analysing and representing naturally occurring texts at one or more levels

of linguistic analysis for the purpose of achieving human-like language processing for a range of tasks

or applications “ (Liddy, 2001, pg.2).

The type of approach to natural language processing can be divided into three major categories:

symbolic, statistical and connectionist. The symbolic approach is related to human-developed rules and

lexicons which is the vocabulary of a person. Usually, in this type of method a set of rules are recorded

by linguistic experts for computer systems to follow. In the statistical approach a set of rules are

developed through an analysing and identification of trends in a large set of a text. The connectionist

approach is a combination of the previous two. It starts with accepted rules of language and transform

them into specific applications from input derived from statistical inference (Liddy, 2001).

One limitation of applying NLP in this master thesis is the volumetry of the data, that is too small. Also,

sometimes is difficult to interpret the human language for example some words can be used in different

contexts and it is difficult to the machine to know what the true meaning in that phrase is.

3.3- Jumping Curves: A new approach

More recently in a review article called “Jumping NLP curves” it was proposed a new approach to the

NLP using the intersection of three overlapping curves: Syntactic, Semantics and pragmatics curve

which is said to lead NLP research to natural language understanding (NLU). A new approach was

proposed defending three paradigms: the bag-of words, the bag-of-concepts and bag-of-narrative

models (Bush, Bryce, & Direito, 2016).

It was defended that with the increase of the user-generated content (UGC) in the web there is a need

to jump the curve from the syntactic level to the semantic, meaning that the NLP systems will switch

from the word-based techniques into the semantics and sentics. This last level includes capabilities like

common-knowledge and common-sense which are more like human behaviour. The next jump is from

the semantic level to the pragmatic level which tries to compare in parallel and in a dynamic way the

semantic with the sentics. This will allow NLP to be more adaptive, context-aware and intent-driven. At

this last level each part of the text will be described as an interconnected episodes which leads to a

better understanding and sensible computation (Bush et al., 2016).

Considering the applicability described in table 3 and since the dataset used in this dissertation is not

big enough to use techniques of ML and NLP. In this dissertation, it will be used Classical Content

analysis using the software NVivo. Classical Content Analysis is a symbiosis between NLP and ML. This

concept will be explained in the next section as well as the phases of the process and the types of

approaches.

- 17 -

3.4- Classical Content Analysis

Classical Content Analysis (CA) was first mentioned in the Scandinavia in the 18th century but it was

only first used in the 20th century in the United States of America. Nowadays, this method is widely used

in health. CA was defined as a “research method for the subjective interpretation of the content of text

data through the systematic classification process of coding and identifying themes or patterns”

(Shannon,2005, p.1278.). Through this technique is possible to make replicable and valid inferences

from the data, provide understanding of the subject and as an outcome categories describing the

phenomenon (Elo & Kyngäs, 2008).

There are three types of content analysis - the conventional, the directed and the summative. The

choose of the type of content analysis depends on the aim of the study. The conventional content

analysis is mostly used in open-ended questions and interviews. In this case, the coding categories are

directly obtained from the text data. A more structured approach- direct content analysis- starts the

coding through previous research. The summative content analysis start by counting the frequency of

the words in a given text to understand the global substance followed by the interpretation

(Shannon,2005).

3.5. Phases of the process

Classical Content Analysis is a process that can be divided in three parts. The first stage of this method

is called the preparation phase and is focused to select the unit of analysis. The second phase is the

organization phase and consists in categorization and abstraction, interpretation and

representativeness. The last phase is called reporting phase and is essentially to report the results of

the process analysis (Elo & Kyngäs, 2008).

3.5.1 Preparation phase

The first phase -preparation phase- starts with the selection of the unit of analysis that could be a word,

a topic, a paragraph, a portion of pages or even a whole document. This selection should take into

consideration the aim of the research study as well as the research question. When selecting the unit

of analysis, it is important to decide what to analyze and with what level of detail. In the case that the

unit of analysis is too big is necessary to use probabilities or judgment sampling (Elo & Kyngäs, 2008).

3.5.2 Organization Phase

In this phase it is important to group the unit of analysis into categories and then define an optimal

number of categories. The grouping is performed by collapsing those that are similar or dissimilar into a

category. Data will be classified as belonging to a certain category or to another. If the number of

categories is too high, it could mean that the coder was unable to group the concepts and also that the

probability of having overlapping categories will increase. On the other hand, if the number is too low it

means that the researcher is incapable of finding differences and classify them as different (Elo &

Kyngäs, 2008).

- 18 -

3.5.3 Reporting Phase

In the last phase of this analysis it is produced a systematic report that will have the concepts that were

identified and some connections or relationships. It is important to evaluate the categories and see if a

certain unit of analysis is the right category (Elo & Kyngäs, 2008).

3.5.4 Types of approaches: Inductive and Deductive

Elo & Kyngäs (Elo & Kyngäs, 2008) defend that CA could be used in either qualitative or quantitative

data; moreover, it is made also a distinction between an inductive and a deductive way. The inductive

approach will be preferred when there is not enough information about the subject. It is favorable to

open coding, creating categories and abstraction. The categories are created according to similarities,

on other words, a category will be a group of content of the same issue. On the other hand, if it has

previous information available a deductive form will be chosen. It will start with a categorization matrix

and the coding will be made according to those categories (Elo & Kyngäs, 2008).

In the figure 2 is represented the seven major steps of content analysis, regardless the type of approach

used.

Figure 2- Key aspects of Content analysis - Adapted from (Shannon,2005)

The steps described in the figure 2 can be related with the three phases of the process of content

analysis. In the preparation phase it happens the step one and two respectively formulate the research

problem and selecting the sample to be analysed. At the first one the problem should be described an

at the second the unit of analysis need to be select (word, phrase) (Shannon,2005). The steps three,

four and five are related to the organization phase: to define the categories, to outline the coding process

and the coder training and to implement the coding process. These steps are related to the process of

1.Formulate the research problem

2.Selecting the sample to be analyzed

3.Defining the categories to be applied

4.Outlining the coding process and the coder

trainning

5.Implementing the coding process

6.Analyzing the results

https://www.linguee.pt/ingles-portugues/traducao/respectively.html

- 19 -

coding the unit of analysis defined previously (Shannon,2005). The last step analysing the results is

related to the reporting phase and is essentially the conclusions of the process (Shannon,2005).

In this master thesis, to perform content analysis it is essential to choose a software to use. This will be

explained in the next section.

3.6 – The software chosen: NVivo

The software chosen to use in this dissertation to perform content analysis was NVivo. Some general

characteristics of this software will be explained in this section.

According to QSR International (2015), NVivo is used not only at the universities but also outside for

worldwide users. It is estimated the over a million people uses NVivo across 150 countries (Dollah,

Abduh, & Rosmaladewi, 2017). NVivo is a software of qualitative analysis that aims to extract important

insights from the qualitative data available (Dollah et al., 2017).

An example of the software interface is presented below:

Figure 3- NVivo interface (“O poderoso NVivo: Uma introdução a partir da análise de conteúdo,” 2016)

As it can be noticed, this software is user-friendly following a Microsoft interface. Here, the data is

organized following a format of nodes and attributes. The nodes in this model corresponds to the themes

founded in the data (Dollah et al., 2017). It allows to gather related information in one place and the

attributes are characteristic of these nodes. At the same time, another term used in NVIVO are the cases

that are a “unit of observation” that can represent places, people and organizations. Another important

thing is that it is essential to define if the questions posed are in an open or closed format because the

closed format corresponds to attributes in the model and the open ones the nodes. An open question is

- 20 -

a question where the respondent can freely write, a close question corresponds to a list of options where

the respondent can choose (Dollah et al., 2017).

This software allows to import data from different sources like pages of internet and in the case of this

dissertation an excel file. The subsequent analysis of this data in independent of the type of data. After

that, an analysis could be made using the tool automated insights that will provide informations like the

themes of the data. It also allows to perform sentiment analysis, a process the gathers sentiment from

the text classifying as positive, negative or neutral, that consist in extracts sentiments of data usually

from social media (Dollah et al., 2017). Automated insights use word frequency to detect words that are

frequent in a text. Another analysis that can be performed with this software is cluster analysis where it

can be notice patterns and similarities between the data (Dollah et al., 2017).

The method that should be used when using NVivo is described in figure 4.

The first step is to import the data, then to explore the data that was acquired, to code, query and finally

the visual representation of the results and a memo. Some steps are optional, for example to record

your sights (Dollah et al., 2017).

The major advantage of this software, as said before, is the fact that it is user friendly, intuitive, manage

data easily, save time for data classification, it is easy to find themes and faster. Notwithstanding, it is a

paid software (Furyk et al., 2018).

In the next section there is a case study named EURO-HEALTHY that served as a reference for this

dissertation as well as an explanation of the methodology used in the case of study.

Figure 4 – Methodology used in NVivo from (Dollah et al., 2017)

- 21 -

4. Case-study

This chapter describes the case-study used in this dissertation, as well as the methodology used in the

case-study to gather the data and its results.

4.1- Overview of the Case-study Methodology: The EURO-HEALTHY Study

The EURO-HEALTHY project stands for ‘Shaping EUROpean policies to promote HEALTH equitY’ and

proposed “a multicriteria Population Health Index (PHI) as a tool to help reflecting upon the future of PH

inequalities and to assist policy evaluation“ (Alvarenga et al., 2019, p.3).

To meet this objective the EURO-HEALTHY project had the aim of developing a Population Health Index

(PHI) and after building the PHI a construction of scenarios to inform the evaluation of policies

(Alvarenga et al., 2019). These scenarios depicting key factors that may affect the evolution of PH

inequalities across European regions (Alvarenga et al., 2019).

The methodology of the EURO-HEALTHY to build scenarios for population health inequalities can be

divided into three main steps as showed in figure 5:

• The identification of the drivers- from experts’ and stakeholders’ views of drivers;

• The generation of scenarios structures- from drivers to scenario structures;

• The validation- from scenario structures to scenario narratives.

Each of these parts was divided in a social and technical part. Usually the social corresponds to the

Web-Delphi or the Workshop and the technical part to the techniques and methods in use. In the first

part the technical method was the group elicitation method, in the second the scenario building

Methodology and in the last part the scenario validation and scenario narrative building.

During the first step of this methodology it was performed a two round Web-Delphi process. In the first

round it was presented a questioning protocol, which will be analysed producing as output a list of

reasons that could result in a list of potential drivers. This list was the input to the second round of the

Web-Delphi. At the second round, the question protocol was presented to select the drivers and identify

the full list of drivers. At this stage the participants’ expressed their level of agreement through a Likert

scale which is a 5 point scale that aims to measure the level of agreement of a respondent to a statement

(Hartley, 2014). These drivers were clustered into PESTLE categories (Political, economic, social,

technological, legal and environmental) and analyzed by the researchers. After these two steps, it was

performed the generation of scenarios through a realization of a workshop. The workshop promotes the

discussion and had as output a validation of the Delphi results and the exploration of possible scenarios’

structures. Next in this process was the drivers’ analysis in which the output will sustain scenarios’

- 22 -

structures. To finish this methodology a second workshop was made with the aim to validate the

scenarios.

Since the focus of this master thesis is the link between the first round and the second round of the

Web-Delphi process, the next section there is an explanation about this specific part, as well as the

results of the applied methodology within the EURO-HEALTHY project.

4.2- Methodology for the manually identification of drivers

Since defining the drivers is really important to shape the future, this is one of the first steps when

scenario building and therefore one of the main steps in this methodology (Raalte, 2008). In this work,

the drivers are collected directly from the reasons presented in participants’ answers to use in the

scenario building.

Figure 6 is a schematic representation of the methodology used by the three analysts that processed

manually this part of the study that aimed to identify the drivers and to cluster into PESTLE categories.

Gathering of information in a

Web-Delphi study

Define the criteria to be a driver

Identify the reasons through

the search of coordinating conjunction

Search for redundancy between the

reasons previously identified

Cluster drivers into PESTLE

categories

Figure 5 - Overview of the EURO-HEALTHY methodology using information from (Alvarenga et al., 2019)

Figure 6 - Overview of the procedure to manually build EURO-HEALTHY scenarios

- 23 -

4.2.1- Gathering of information in a Web-Delphi study

The gather of the information happens during the first of two rounds of a Web-Delphi process concerning

the EURO-HEALTHY study. The first round had a question protocol followed by an answer analysis and

has as an output a list of reasons that need to be afterwards analyzed to see if it is a list of potential

drivers. The second round had, also, a question protocol that had as input the resulting potential drivers

from the analysis of the results of the first round and as output the level of agreement participants had

around the drivers.

In the first round of the Web-Delphi study questions were asked separated by area of concern, that is

an area of interest in this working package, in which the participants were able to choose between a set

of options. After selecting the option that the participant thought it was better suited, the participant

should justify his answer. This justification was used to collect the drivers identified later in the scenario

study. The aspect of this questionnaire is presented in figure 6.

The questions were separated by 9 areas of concern: Economic conditions, social protection and

security, education, demographic change, lifestyle and health behaviours, physical environment, built

environment, road safety, healthcare resources and expenditure and healthcare performance; For each

one of this field there was 4 options to choose; The first three options have this form: Until 2030, there

will be [an increase, a decrease, no change] in [area of concern] inequalities across European regions

for the following main reasons [indicate reasons] and the last is Don’t know, Don’t want to answer. Next

to the options to the answers, it was presented indicators concerning the subject in study to help the

participants chosing the options together with the gap of the indicator.

Figure 7 - Print Screen of the Web-Delphi platform concerning the Economic conditions, social protection and security adapted from EURO-HEALTHY WP7 working materials.

- 24 -

These data were later exported from the platform in the form of an excel file for posterior analysis

separated by code of the user, area of concern, answer and text. The list of reasons and later the drivers,

was withdraw from the last field of the excel, the text field.

4.2.2- Define the criteria to be a driver

To extract drivers from the information extracted from participants, researchers defined four criteria for

a driver (Santana, 2017): address a specific issue- meaning that there will be no drivers based on

generic statements-, to be non-redundant – statements with similar construction and focus should be

grouped and reduced and two concepts with similar content needed to be analyzed to evaluate the

possibility of merging-, simple- if a concept is too complex it need to be split into simpler concepts- and

understandable- the language that the concept is written need to be clear to avoid misunderstandings.

Furthermore, it should be looked for variations of the driver for potential inclusion to apply the Extreme-

World method for scenario building and, to look for explicit relations among the drivers’ directions and

the respected increase, decrease or no change regarding the focal issue (Santana, 2017). (Alvarenga

et al., 2019)

4.2.3- Identify the reasons through the search of coordinating conjunction

After defining the criteria to be a driver, researchers moved to analyzing the data and try to identify

possible drivers. Three analysts during the EURO-HEALTHY project did this process manually, reading

all the text and organizing in an excel file. First, this process was performed by each one of the

investigators followed, by an aggregation of the three different results and consequently a discussion of

the same subject. This made this process bias since it depends on the opinion of the three investigators.

The researchers looked for the reasons that the participants gave during the answers of the first round

of the Delphi. Usually these reasons appear in the text after a coordinating conjunction that is a join

between two or more sentences and refers to an explanation. It was identified, during this step, 364

answers that were scanned to 412 causes.

4.2.4 -Search for redundancy between the reasons previously identified

It is important to look if there was any redundancy between the set of reasons obtained. That is, if the

list of reasons has the same reasons but presented in different forms then it should be clustered or

discarded. At this stage, it will be checked, for example, if there are any answers that are a simple

repetition of the question and if there are repetitions of the same concept among the different answers.

Also, it was checked if the list of drivers followed the criteria to be a driver- to address a specific issue,

to be non-redundant, simple and understandable. It was identified a list of 178 potential drivers.

4.2.5- Cluster drivers into PESTLE categories

From the previous step in this methodology it was identified a list of 178 potential drivers. Since this list

has different concept there is a need to agglomerate this data into themes or subjects. Because of that,

in the last phase of this methodology the investigators manually assigned categories according to the

PESTLE taxonomy that is constituted by Political, Economic, Social, Technological, Environmental and

- 25 -

Legal factors. From the 178 drivers, there was 24 Political, 37 Economic, 76 Social, 12 Technological,

11 Legal and 18 Environmental and that was the input for Round 2.

The political factors are issues related to the government like government leadership, government

stability, tax regulations, stability of neighbors, employment and operational laws, trade restrictions or

reform, bureaucracy levels and corruption levels. Economic factors usually refer to issues like finance

and credit, cost of living, GDP and GNP, working practices, inflation, taxes and duties, exchange rates

and globalization. Social factors are concerning attitude and beliefs, demographics, cross-cultural

communications, historical issues, ethics and religion, social mobility, education and lifestyle.

Technological factors are issues regarding the production efficiency, patents and licenses, intellectual

property, quality and pricing, knowledge management systems, eliminate bottlenecks, network

coverage, research and development, use of outsourcing, government activity and legislation and rate

of change. The legal factors are related to import and export, regulatory bodies, compliance, health e

safety, advertising, consumer, taxation and employment. Lastly, the environmental factors talks about

energy availability and cost, ecological consequences, legislation, contamination, disposal of materials,

social implications, infrastructure e cyclical weather (Newton & Bristoll, 2013). When performing this

task, the categories were assigned according to these conditions.

It was important to try to automatize the processing of participants’ answers in survey and Delphi

contexts since in this case study this step took a lot of effort from the analysts to perform the analysis.

Besides that, since the process of answer analysis was performed by the analyst it could be bias. For

that, one of the purposes of this thesis was to decrease the time consumed at this phase and to decrease

the interference of the investigator in the analysis process. In the next section there is the proposed

methodology used in this work. It is to develop a methodology that provides insights from participants’

answers in survey and Delphi contexts.

- 26 -

5. Methodology

The objective of this dissertation is to design a methodology to automatically get insights from

participant’s text answers in survey and Delphi contexts. To achieve this, the methodology was tested

in a Delphi study already performed in the context of the EURO-HEALTHY project, that will be used as

a case-study in this dissertation.

To meet the objectives previously referred, a description of the methodology used is needed, as well as

the type of approach used, some advantages and some challenges. It will be described two versions of

a methodology: the first is a general methodology that can be replicated in other works and the second

a specific version of the general methodology that is the methodology applied to the EURO-HEALTHY.

The protocol used is also described and the steps of the methodology that can be automatized are

identified along with the procedure to automate them. The software used in this work was also identified

as well as the pros and cons of using it.

The data used in this dissertation was obtained through a Web-Delphi that had 51 participants. The

questions are separated in 9 areas of interest. The results from the Web-Delphi should inform the

evaluation of policies and what can affect health and health inequalities across Europe. The developed

methodology uses concepts of the Delphi method and Classical Content analysis to extract insights from

participants answers.

5.1- Proposed General Methodology: A Content Analysis Approach

The methodology followed in this work contains a different set of techniques to develop a general

methodology that can be used in this case -the EURO-HEALTHY Study- but also in different contexts.

For that the concepts described in chapter 3 about Classical Content analysis were taken into

consideration as well as the methodological approach of performing CA.

- 27 -

In the figure below, it is a scheme of a general proposition of the methodology that uses the principles

of content analysis refereed in the previous chapter (Shannon,2005).

There are different steps that need to be made to extract insights from the participants answers. Firstly,

there is a need to collect the data that is going to be analyzed using this methodology, independently of

the data source. Data must be in text format; otherwise classical content analysis can be performed in

the next step. In this step there is also a need to know what is going to be analyzed and with what level

of detail in order to be able to analyze this data, to code and to identify patterns. For that the unit of

analysis is defined, that could be word, sentence or paragraph level which should be chosen according

to the aim of the study.

In step two a CA is made with the aid of a software for qualitative analysis for example the R Studio,

Python or NVivo. The coding categories are directly obtained from the data. According to the software

selected to use there are steps that could be different. For example, the query of the data usually is to

avoid redundancy in the data and to find patterns faster but if it was used Python or R there also another

step that could be performed that is the cleaning of the data using function like tokenization, stemming

and stop word removing which allows to decrease the volume of data.

In step three, it is important to query and clustering the data according to the aim of the study and to

verify the drivers. The data is clustered according to some similarities between the drivers.

Last step is to analyze the results according to the aim of the study. This last analysis is performed by

the analyst to reduce some errors or redundancy that could come from the process of CA.

The methodology presented above is a general proposition, a more specific will be is presented below

according to the case study used in this thesis: The EURO-HEALTHY study.

1. Collect the data to be analysed and define the unit of analysis

2. Perform content analysis with the aid of a software

3. Query and Clustering the data according to the objectives of the study

4. Analyze the results

Figure 8 - Proposed Methodology based in CA (Shannon,2005)

- 28 -

5.2- Adapting the general methodology to this dissertation

To fit the goals of this thesis the steps shown in figure 8 need the following adaptations concerning the

EURO-HEALTHY:

The first step - “Collect the data to be analysed and to define the unit of analysis” corresponds to the

Gathering of information from the Web-Delphi to the NVivo software and to define the unit of analysis.

The Second step – Perform content analysis with the aid of a software- was made with the aid of the

software NVIVO using the automated insights tool.

The third step – Query the data according to the objectives of the study and clustering the data-

corresponds to the filtering of the data in NVIVO using specific queries.

The last step- consist in the final analysis of the data.

These steps will be explained in detail in the next section in comparison with the EURO-HEALTHY

methodology.

5.3- Methodology applied to the Case-study

Since one of this dissertation goals is to automate the answers processing of the first round of a Delphi

in the EURO-HEALTHY, the methodology used in this dissertation was not only based in the process of

CA but also taking into account the methodology of the EURO-HEALTHY package explained in the case

study chapter. That is, the proposed methodology concatenate concepts of CA methodology with the

EURO-HEALTHY methodology.

In the figure 9 is an overview of the proposed methodology in comparison with the methodology for the

manually identification of drivers defined in the EURO-HEALTHY study that was defined in chapter 4 –

Case Study. In figure 9 a) is the manual methodology used in the EURO-HEALTHY study and in figure

9 b) is the automatic methodology applied to the EURO-HEALTHY used on this dissertation. Each step

of the figure 9) corresponds to the general methodology defined in figure 8.

- 29 -

5.3.1- Gathering of information from the Web-Delphi already performed and Define the criteria to

be a driver

The first step of this procedure consists in importing the data that is going to be analysed from the Web-

Delphi platform to NVivo. At this stage, it is essential to define if the questions posed are in an open or

closed format because the closed format corresponds to attributes in the model and the open ones the

nodes.

As said before, the participant’s answers were withdrawn from the Web-Delphi and were codified like

an open answer. This field will be a node in the model. The cases in NVIVO were created according to

the area of concern referred in the questionnaire. One example of a case is a person; a attribute of this

case will be the color of the eyes or the height. Therefore, it was created 9 cases by the software

representing the different areas of concern presented in the questionnaire.

5.3.2- Perform automated insights using NVIVO to obtain potential drivers

Subsequently, the next step is to perform the automated insights, that in NVivo denotes the auto coding

feature that code themes from the data. For that, there is a need to choose what is the scope of the

6.Cluster drivers into PESTLE categories

5.Does this reason contain a driver?

4.Search for redundancy between the reasons previously identified

3.Identify the reasons through the search of coordinating conjunction

2.Define the criteria to be a driver

1.Gathering of information in a Web-Delphi study

4.Analyse the results

3.Filtering the data in NVIVO to avoid redundancy and see if it is a driver and Refine the Clusters using specific

queries

2.Perform automated insights using NVIVO to obtain potential drivers

1.Gathering of information from the Web-Delphi already performed and Define the criteria to be a driver

Manual methodology used in the EURO-HEALTHY

study

Automatic methodology proposed on this

dissertation

Figure 9 – a) and b) Comparison between the methodology for the manually identification of drivers (left side a)) and the proposed methodology to automatically identify the drivers (right side b));

- 30 -

analysis- a sentence, a paragraph, a page of a document or an entire document. In this case, it is the

word level because of the level of detail needed.

This type of approach has a special trade-off. An advantage of this approach is that the analyst can get

a quick result but on the other side the accuracy of this tool is not high. When performing the analysis,

it is important after assigning the categories to analyse all the references assigned to each category

(Yearworth & White, 2013).

During this step, some issues arose concerning the analysis of the data that was acquired. Ideally, and

according to the methodology for the manually identification of the drivers, a driver is usually in a text

after the use of a coordinating conjunction, considering that a driver is an explanation of a point of view.

However, when analyzing the data collected from the EURO-HEALTHY, it was concluded that in some

cases that was not true.

The participants that answered the questions from the EURO-HEALTHY study are in fact specialists

from varied locations in Europe, which means that they are not all native English speakers. This means

that these responses need more attention when studied, considering that since the answer may not

have the coordinating conjunction to actual explain their point of view, or in other cases answering the

question with part of the question without further explanation. In this specific case, it can be affirmed

that there’s no driver, since a driver needs to be to address a specific issue, non-redundant, simple and

understandable. Another issue is that since the participants are specialists from different areas, the

vocabular used did not follow any pattern.

The results obtained from the automated insight feature are represented in a hierarchical structure

correspondent to the nodes of the data that are the themes that emerged from the content analysis. This

hierarchical structure will be presented in the next chapter.

After all this process and analysis, the references from the results will be the drivers found in this study.

Because of that, there is a need to check if all the references are drivers and confirm if there is no

redundancy. This analysis is made adding filter to the results and performing queries.

5.3.3- Filtering the data in NVIVO to avoid redundancy, see if it is a driver and Refine the Clusters

using specific queries

This step was performed using queries and filters from the NVivo to study for example, if it has the same

references more than one time and if there are references that are the same but written in different

ways. A query is a way of requesting information to a database through a specific language. In this step,

it was also necessary to analyze the themes that emerged from the previous step.

The process of content analysis was made using the software NVIVO. In table 4 is an explanation of

why each step of the process of classical content analysis -was made in the methodology of this

dissertation, which function of the software NVivo uses and if it was completely automatic, manual or a

mix of automatic with the aid of an analyst. In this case automatic means that the process was made

totally by the software, a mix means that it had an automatic part followed by a part where the analyst

- 31 -

should analyse the data and help in the decision and manual means that it was made by the analyst

without the help of the software.

Table 4 – Performing Content Analysis: NVivo Functions used in this methodology- When and why to use it

Since it was performed CA, it was necessary to understand what words were more common in all the

questionnaire independent of the area of concern. For that, the tool automated insight from NVivo was

used to determine the word frequency to discover distributions, patterns and to code the themes from

the data. This is related to step 3 of the methodology described in figure 9b). Then, there was a need to

see if this phrase that result from the references is in fact a driver. And for that, the analyst needs to

consider the criteria to be a driver which brings to the next step that is to analyse the redundancy.

Redundancy was analyzed through queries with the software and manually by the analysts. This step

corresponds to step three of the methodology. The last phase is the final clustering which is a mix

between an automatic function and the aid of the analyst. This last step is related to the step four of the

methodology.

5.3.4- Comparison with the EURO-HEALTHY methodology

There are some similarities and some differences when perming the EURO-HEALTHY methodology and

this dissertation methodology. In this dissertation case, the methodology was reduced from six steps to

four. When analysing figure 9 a) and b) it is noticed that the first step is almost the same: gathering the

information from the Web-Delphi. In the case of the EURO-HEALTHY package the analysts retrieve the

Analysis for what? Correspondence to step of the

methodology of the Figure 8 b)

NVIVO Function Automatic or Manual

Understand what words were more

common and perform a first clustering of the

data

Step 2 Automated Insight using Word Frequency,

distributions and patterns

Automatic

Redundancy: Is this a driver?

Step 3 Using the filters in the NVivo to clean the

text

Automatic + Manually performed

by the Analyst

Final clustering Step 4 Cluster sources

Manually performed by the Analyst

- 32 -

data from the Web-Delphi to an excel file separated user, area of concern and answer. This excel already

constructed was the source of data in this dissertation. In this step in the Automatic methodology it was

also defined the criteria to be a driver since it was the same criteria as the EURO-HEALTHY package.

Step two of the automatic methodology was the main change between the two methodologies. This step

is the performance of content analysis with the aid of the software NVIVO instead of manually looking

for the reasons, the software can identify the main themes of the data acquired, at the same time while

identify themes the software is clustering the data. The search for the redundancy was made in the

automatic version by filtering the data in the software instead of manually looking for differences between

the drivers. Last step is to analyse the data.

The results output in Euro-Healthy case and here were the same: a list of phrases with the potential

drivers. These results will be showed in next section: Results.

- 33 -

6.Results

The present section discusses the results obtained in this master thesis. The protocol developed in this

work is also discussed as well as the choices that were made and the pros and cos of using it. The

results obtained manually in the EURO-HEALTHY project are compared with the results obtained in this

dissertation and an analysis is made. The results are present according to the proposed methodology.

6.1- Results from Gathering of information from the Web-Delphi already performed

Following the automatic methodology described in chapter 4, the data was collected from the EURO-

HEALTHY platform according to the name of the expert, area of concern, answer and justification. The

data was exported from the Web-Delphi to an excel file and then imported to the software NVivo.

6.2 - Results from the Content Analysis using NVivo: Perform automated insights using NVIVO

to obtain potential drivers

The data was imported to the NVivo, the unit of analysis was selected, and the feature named automate

insights- a tool that uses word frequency to assign themes- was used to perform content analysis. The

scope of analysis chosen was the word level once it allows to provide a narrower scope due to the

resulting identification of relationships that trespassed both sentence and paragraph boundaries.

After performing automate insight, a hierarchical structure was produced according to the relationship

between the themes assigned and the references of the themes. The references to the themes that are

the drivers of this work are presented in Appendix. The hierarchical structure is node- that is a theme,

sub-node – that is a sub-theme- and references- that are the drivers in this dissertation. This structure

is presented in the next section.

- 34 -

6.2.1- Hierarchical Structure produced with the Software NVivo derived from step 2 of the

methodology

The data was clustered into 9 nodes using the tool automated insights from NVivo that were: ageing,

economic, education, employment, environment, health, policies, road safety and social. Each theme

has drivers associated, ageing 25, economic 16, education 18, employment 14, environment 30, health

45, policies 26, road safety 14 and social 30, a total resultant of 218 drivers. The search for the

redundancy was also performed in this step reducing the drivers from 250 to 218. The references in this

case were the drivers of this work. The obtained drivers will be compared with the ones obtained from

the EURO-HEALTHY Study.

In the EURO-HEALTHY three analysts looked for the drivers. First, this process was performed by each

one of the investigators followed, by an aggregation of the three different results and consequently a

discussion of the same subject. The researchers looked for the reasons that the participants gave during

the answers of the first round of the Delphi. Usually these reasons appear in the text after a coordinating

conjunction.

Differently from the manually identification, the drivers that were obtained in this work were identified

though word frequency associated to each node. If it was one of the most frequent words a node will be

Content Analysis

Ageing 25 drivers

Economic 16 drivers

Education 18 drivers

Employment 14 drivers

Environment 30 drivers

Health 45 drivers

Policies 26 drivers

Road Safety 14 drivers

Social 30 drivers

Figure 10 -Results from the Content Analysis – Nodes and number of drivers

- 35 -

formed, and the references associated with this node will be the drivers. Since the themes were

assigned according to the word frequency the drivers weren’t forced to be reasons but the criteria to be

a driver are the same: to be simple, non-redundant and understandable.

6.3- Results from the themes obtained in this dissertation after the Filtering of the data in NVIVO

to avoid redundancy and see if it is a driver and Refine the Clusters using specific queries

In this section it is a description of some drivers obtained in this investigation according to the theme

assigned. A more detailed version of the drivers is presented in the annexes.

6.3.1- Ageing

This node refers to the ageing of the population and it is a topic that affects the worldwide population.

The increase of life expectancy implies an increase of the elderly and consequently a reduction of young

population that comes along with a low birth rate. There are more people in retirement and less people

working which can lead to an increase of the poverty rate of the elderly and, consequently, the risk of

poverty too, because elders are a most vulnerable group, including at an economic level. Since

retirement, usually, implies income reduction - directly and indirectly, due the latter to a decreased

earning ability there is also a reduction of the general GDP.

6.3.2- Economic

Economic is related to the previous topic: ageing. With the increase of the life expectancy and

consequently an increase of the elderly population a reduction of GDP will be noticed. The worsening

of material conditions in some populations groups in some areas will contribute to increase of

inequalities.

Although unemployment rate could be decreasing, in accordance to the economic cycle, income

distribution will be worsening and its gap widening, unless the economic rate of less developed countries

in Europe grows higher than the most developed countries. The demographic change and an increasing

economic pressure like the cuts in the pension system will lead to an increase in inequalities and a

higher risk of poverty for the elderly. The gap on the Demographic Changes will tend to increase due to

the economic instability in Europe mainly affecting the South, increased emigration in countries already

affected by population ageing and decrease in natality.

The access of healthcare also depends of the economic conditions especially in more economically

vulnerable countries.

6.3.3- Education

The increase of the elderly and the decrease of young people will have impact in education. On the one

hand, there is an increase in secondary and tertiary education with the hope of getting better jobs and

on the other hand there are less cost of education systems with young people.

The monitoring of education results will make decision-makers to pay more attention to education

polices. The improvement in general education of the European populations and increase in health

- 36 -

literacy of European citizens will contribute to the adoption of healthy lifestyles. The smoking rate and

obesity will decrease. In general, the educational level of people will increase due to the academization

of our society.

6.3.4- Employment

According to the expert’s opinions the unemployment rates will probably increase in Europe, considering

the instable political developments in almost every country within the EU. In this way, unemployment is

becoming an increasing problem mostly because of the economic crisis. Due to this problem and low

birth rate, Europe will face early retirement. To face this problem, there will be a trend for general higher

education and an academicization of job profiles that were based on vocational training before. With

reduced employment opportunities for unskilled work and more efforts to reduce the number of drop-

outs the number of early leavers will decline.

The trend of employment with only critically low income/limited social benefits, as well as the trend to

multiple employments and/or temporary employments will continue and might even further increase

what leads to an increase of the poverty rate. A stronger integration of EU labor markets could reduce

unemployment rates in some regions but could be linked with somewhat higher unemployment rates in

other regions, or higher inequalities between groups of employees in other regions.

6.3.5- Environment

The increase expected in the general education of European Citizens will enforce the politicians to

deliver better build healthier environment programs. European environmental regulations, alongside

with international and worldwide agreements and population-based advocacy will decrease inequalities

at an environmental and European level. Central policies for taxation, imposition of recycling quotas,

policies to improve air pollution and to decrease traffic noise will be implemented.

There is also an increased concern with natural resources efficiency, namely energy efficiency Built

environment it is expected to increase. The current instability/unpredictability of the development of the

"green" agenda vs oil-based agenda to support economic growth may impact the development of

European economies, with strong environmental impact.

6.3.6-Health

Available information on better lifestyles, higher levels of education and literacy lead to more healthy

individual behaviors all over the world. There have been a lot of efforts towards decreasing obesity and

encourage people to healthy diet, reducing smokers and alcohol consumption, however poverty can be

an important obstacle to this behavior but in general, there will be a greater commitment to following EU

directives and WHO guidelines.

It is expected a further increase in total health expenditures due to more costly treatment options and a

trend to a privatisation of services. Innovation costs and population ageing are a threat for all national

health systems since improving health national expenses depends on improving GDPs.

- 37 -

6.3.7- Policies

This theme is related to other themes and can be associated to economic, educational, environmental

and health policies.

Economic policies - austerity policies that will reduce investments in social help due to the economic

crisis in Europe will be created.

Educational policies - local or national public policies to reduce early leavers from education and training

and national policies to improve the upper secondary and tertiary education levels;

Environmental policies - policies with the aim of reducing the carbon emission and the persecution of

restrictive policies on vehicles and industry, favors the reduction of these indicators.

Health policies - restriction policies related to the tobacco that promotes a decrease in the consumption.

6.3.8-Road Safety

In relation to road safety there are different opinions. In the one hand it was said that Road traffic is a

part of modern society and no improvement would be expected in area/regions with heavy traffic, on the

other hand there are experts that said that better road conditions and better road safety national

regulations will occur in Europe. The general trend is to improve road safety due the increase of

regulation in this area.

6.3.9-Social

This theme involves all the others, the ageing, economic, employment, environment, education, health,

policies and road safety.

The increasing ageing of population will lead to some difficulties in poor countries since some social

security national systems are sub funded. The recent increase in unemployment in late ages will

increase the poverty rate in some areas. The tendency is to people get more than one job to get more

money.

Although the EU and National governments will put policies in place to combat the increasing challenges

to access to education the tradeoff is just not good enough and will lead to no change in inequality

across Europe’s access to education. A reduction of the gap in lifestyle and health behaviour can occur

due to a higher education in general of the European society, and due to the demographic change by

looking at the increasing figures of life expectancy at the same time.

Although smoking is increasing among women in Europe there is a hope that in general the smoke rate

will decrease due to the academization of the population. Problems like obesity and mothers under age

of 20 will also decrease.

6.4- Analyse of the results : Comparison between the results from the EURO-HEALTHY Study

and the Results from this Dissertation

Since this dissertation used data collected from the EURO-HEALTHY, a comparation needs to be made

with the results that were obtained in the study.

- 38 -

In the manual process of the identification of the drivers, they were separated by the PESTLE taxonomy.

The PESTLE analysis uses six external factors- Political, Economic, Social, Technological, legal and

environmental and from 364 answers, founded 412 causes and 178 potential drivers. From this process

it were extracted 24 Political factors, 37 economic, 76 social, 12 Technological, 11 Legal and 18

environmental from a list of 178 potential drivers the investigators narrow down in the second round of

the Delphi to 49 drivers.

In the EURO-HEALTHY study the data was clustered according to the PESTLE taxonomy obtained 6

nodes. In the case of this master thesis the data was clustered according to word frequency which lead

to a total of 9 nodes. The nodes obtained in this master thesis, in some cases, correspond to the area

of concern in the EURO-HEALTHY questionnaire.

The EURO-HEALTHY questionnaire had 9 areas of concern namely, Economic conditions, Social

protection and Security, Education, Demographic Change, Lifestyle and Health Behaviours, Physical

Environment, Built Environment, Road safety, Healthcare resources and expenditure and Healthcare

performance. This corresponds to 5 of the 9 nodes obtained which can mean that the area of concern

defined in the beginning is related to the themes emerged in the end of the process and it can lead to

the conclusion that in a next Delphi the themes defined in the beginning of the process can define the

clusters made in the end. So, it is crucial to be careful in the construction of the questionnaires. The time

consumed performing the analysis of the Delphi in this dissertation is lower than in the manual way. The

automatic part and some preliminary analysis was made in three weeks and it was performed by one

analyst instead of three.

Figure 11 represents the different clusters obtained in the EURO-HEALTHY case and in this dissertation.

- 39 -

It is noticed that there are three themes that are the same in this dissertation and in the EURO-HEALHTY

that are Economic, Social and Environmental. The results from these three nodes can be directly

compared. In the EURO-HEALHTY it was founded 37 economic potential drivers, 76 social and 18

environmental. In the case of this dissertation it was founded 16 economic potential drivers, 30 social

and 30 environmental. The number of drivers on these nodes seems to decrease from the EURO-

HEALTHY to this dissertation, what makes sense since in this work the number of the themes is bigger

and so the level of specification in each node is expected to be also higher. On the other hand, the

PESTLE analysis is more general then the results obtained here since the data is more agglomerate.

When analyzing the drivers of each node it is important to notice that some differences comes from the

process of clustering that in this thesis was automatic and generate nine themes and in the EURO-

HEALTHY was chosen and had 5 themes. It is also important to remember the criteria to be a driver to

address a specific issue, to be non-redundant, to be simple-and understandable.

6.4.1- Comparison between the node Economic

The node economic in the EURO-HEALTHY has 37 potential drivers while in this work has 16 potential

drivers this number gives 43% of the drivers which is half of the drivers that was founded in the case

study. From the 16 drivers found in this work and directly comparing with the EURO-HEALTHY it is

noticed a correspondence of 68% between the drivers. These divergences could come from the number

of nodes existents in each study and consequently the different themes; In the EURO-HEALTHY study

there were some drivers concerning employment in the Economic node but in this work, there is a node

that is Employment, so the correspondent drivers are mostly at this node.

The principal considerations having here in both the EURO-HEALTHY and here are almost the same.

• There is a significant preoccupation with the healthcare since economic conditions affect the

healthcare efficiency especially in economically vulnerable countries;

• Financial crisis and the worsening of economic conditions; income distribution will be worsening

and its gap widening;

Ageing

Economic

Education

Employment

Environmental

Health

Policies

Road Safety

Social

Political

Economic

Social

Tecnhological

Legal

Envirnomental

Figure 11 - Themes obtained in this dissertation VS PESTLE

- 40 -

• Increase of life expectancy that increases the unemployment in late ages;

• A general increase in unemployment rates in Europe;

6.4.2- Comparison between the node Social

The node Social in the EURO-HEALTHY has 76 potential drivers while in this work has 30 potential

drivers this correspond to 40% of the drivers which mean that it was found less than half comparing to

the case study. From the drivers found in this work and comparing directly with the EURO-HEALHTY it

was noticed that 60% were the same.

The principal considerations having in both the EURO-HEALTHY and here were a little different since

the social node in the EURO-HEALHTY package has more drivers and was a lot more general while in

this dissertation there were other nodes the had these drivers such as employment, education, policies

and road safety.

Nevertheless, there were some drivers that were the same:

• Increase in smoking among women in Europe;

• Health problems that comes from obesity like diabetes and hypertension;

• Inequalities in the access do education;

• Problems concerning health access;

• Higher concentration of people at risk of poverty and social exclusion

6.4.3- Comparison between the node Environmental

The node Environmental in the EURO-HEALTHY has 18 potential drivers while in this work has 30

potential drivers which means that in this case there were more drivers found in this dissertation then in

the case study. Some considerations having in both the EURO-HEALTHY and here were a little different

since in this work some drivers from the politics in the EURO-HEALTHY appeared in this node. However,

there were similar points that were:

• The green agenda and the green economy;

• Increased concern with the natural resources’ efficiency;

• Decrease in the quality of the built environment

- 41 -

7 -Discussion

7.1- Comparison between the manual methodology and NVivo

In the EURO-HEALHTY package to avoid bias when performing the methodology to extract insights

from participants answers it was necessary three analysts. And here was a scope for improving that was

to produce a methodology that automatize answers analysis performing by only one analyst; For that,

and to promote transparency in data analysis it was performed content analysis to the answers extracted

from the Web-Delphi using NVivo. Using a software to perform content analysis it was possible to extract

insights from the participants answer without the bias that comes from analyst itself and that are inherits

in being a human being.

Another difficulty in this work was that the participants were not English native speakers and did not use

formal language; besides that, they were from different fields of study which can lead to even more

different opinions ant type of language. The analysis of this type of answer is harder since it didn’t follow

any pattern. This type of language originates a problem in this thesis that is redundancy which was

partially resolved in this work. On the hand, with the aid of queries it was possible to reduce the

redundancy but in the other hand a final analysis was necessary by the analyst.

NVivo allows to collect and archive almost any data type, to connect to your transcribed data, to search

large data sets and to organize them according to your needs. It also allows, to create codes to identify

patterns and to cluster the information into themes. In this case, software allowed to perform content

analysis and to obtain drivers from the data. It also allowed the decrease of the analysis time in relation

to the time that was needed in the case study, the time invested for the investigators will be discussed

in topic 7.4.

7.2- Nvivo Pros and Cons

NVivo has also some limitations. One limitation is related to the commands that are possible to perform

within the software, most of them must be accomplished individually without the possibility to give a

general command and apply it to multiple dimensions. Another disadvantage is the cost; It is a paid

software with discounts for students and a free trial. The time consumed to learn the software is high

what can be also a disadvantage (Dollah et al., 2017)

On the other hand, NVivo is time efficient and transparent; is able to capture quantitative and qualitative

data (Dollah et al., 2017). This software is usually used in social sciences to aid in interviews,

questionnaires and meeting transcripts. One example of the use of this software is the work of Furyk in

their study about the consensus research priorities for paediatric status epilepticus where the answers

should be analysed using grounded theory, content analysis and open coding to categorize items into

themes and finally, the answers are revised and included in the next round of the Delphi (Furyk et al.,

2018).

7.3- Comparison between the EURO-HEALTHY Clustering and the Clustering from this

dissertation

- 42 -

During the clustering it can be noticed differences between this methodology and the methodology used

in the EURO-HEALTHY study. In this methodology the clustering is made in the beginning of the process

while in the EURO-HEALTHY the clustering was performed in the end. That occurs because of the tool

used to do the clustering that was the Automate Insights from NVivo that requires the clustering to be

done in the beginning. The fact that the clustering was made in the beginning reveals to be appropriate

since the results converge to a total of nine themes that were the same number that the number of areas

of concern defined in the questionnaire. This reveals that the area defined in the beginning of the study

will influence the themes that emerged in the clustering. This finding can lead to the conclusions that

the area of concern of the study should be well defined in the beginning. On the other hand, PESTLE

has an advantage in comparison with the clustering used here: It is a taxonomy used worldwide what

makes this taxonomy widely recognized and easier to compare to other studies.

7.4 - Time used by the investigators

For the EURO-HEALTHY study it was necessary three investigators dedicating their time to the project

for one month only to the first part of their work that was to identify the drivers from the first round to the

second round of the Delphi. The process was performed by each one of the investigators followed, by

an aggregation of the three different results and consequently a discussion of the same subject. The

researchers looked for the reasons that the participants give during the answers of the first round of the

Delphi. During this step, 364 answers that were scanned and narrow down to 412 causes.

In this dissertation the work was done by one investigator using the software of qualitative analysis

NVivo. It was necessary not only to learn NVivo but to learn the process of Classical Content Analysis

and how to aggregate in a methodology the Web-Delphi, Classical Content Analysis and scenarios

structures what was a complex process. The time consumed in the process of Classical Content analysis

itself decreased from one month to three weeks and with only one analyst.

In this dissertation, the data was collected in the EURO-HEALTHY project and then processed with the

aid of the software, a total of 218 drivers were obtained. After that a careful analysis was made to check

the data, see if the drivers followed the criteria to be a driver and test redundancy. It was a more

automatic process that can be used in other contexts out of this work when aiming to automatize the

analysis of answers from questionnaires.

7.5-Technology and Data Representation

The data was imported to NVivo and was treated according to the word frequency that is the counting

of the words that happen in a given text. NVivo allows to represent the data in various ways including

word clouds, causal maps and tree maps. There are options to filter the data according to the aim of the

study. The filtering of the data was useful when dealing with redundancy. Redundancy was one of

challenges found in this work. Deal with redundancy was difficult both in this work and in the EURO-

HEALTHY Study and it was solved using an automatic and manual part. The first one was performed

using filters associated with the software when looking for the drivers. The second one was in the end

check all the drivers and see if it still has some repetitions.

- 43 -

When importing the data cases were defined, the cases are the area of concern of the study. Then the

clusters of the drivers were formed according to the word frequency. In the end, the data was exported

and presented in an excel file.

7.6 -Level of Detail of each investigation

The case study participants were not English native speakers and, due to their multidisciplinary

background, did not have a common scientific language; because of that answer analysis is harder since

the answers did not follow any pattern. This fact can lead to redundancy that was one of the biggest

issues in the EURO-HEALTHY package.

In the manually investigation to find the drivers from the answers, the investigators look for a coordinate

conjunction in a phrase. A coordinate conjunction implies causality and since there were looking for

causality the phrase that appears after the conjunction was considered a driver. In this case the search

for the drivers was made through word frequency so it didn’t necessarily implied causality. To compare

the drivers can be difficult because of this difference.

Since the drivers were found according to word frequency, it was noticed that it is possible to have the

same driver associated with different themes what leads to some redundancy. For a machine to take

into account redundancy is always difficult but that is an aspect that could be improved in the future with

the aid of other methods like NLP using sentence simplification together with clustering could help with

his problem by clustering smaller units (Thadani & Mckeown, 2008).

- 44 -

8. Conclusions and Future Work

The theme of this work arises from the challenges founded in the EURO-HEALTHY project. When

developing the EURO-HEALTHY project, the investigators found that the work was really time

consuming and thought in a way of automatizing this process to decrease this problem. Another issue

found by the investigators was the language; Since dealing with qualitative data implies dealing with

linguistic and the EURO-HEALTHY participants were not English native speakers, the type of vocabular

used did not follow any pattern which lead to redundancy and to a exhilarating analysis.

During a month, in the EURO-HEALTHY project, three investigators worked in a way of manually finding

the drivers. The aim of this dissertation was to find the drivers with only one investigator, performing in

less time and to automatically process the answers from the Web-Delphi. For that, this master thesis

was proposed, and a methodology was developed under this context. In fact, the methodology

developed in this dissertation allowed to find the drivers that were needed to go to next round of tje

Delphi and to define the scenarios in the EURO-HEALTHY project and with only one investigator

performing the automation phase for less time with the aid of the software of qualitative analysis named

NVivo.

The methodology defined in this work was based on the developed in the EURO-HEALTHY but using

the NVivo and had five steps: Collect the data to be analysed, define the unit of analysis, perform content

analysis with the aid of a software, query the data according to the objectives of the study and clustering

of the data. The automation occurs from the third step until the end using concepts of content analysis

to achieve the main goals that was to find the drivers. In this case the drivers were founded based on

the word frequency associated to a theme and not by the causes that happen in a given answer that

was what happened in the Euro-Heathy case. The most frequent words had references that were

associated in a given theme forming the drivers.

It was necessary a deep knowledge of the case study as well as the methodology used and the

techniques associated. The types of participation methods were studied in particular the Delphi Method

and Scenarios since it was the one used in the EURO-HEALTHY package. It was necessary to study

how can a work using a Web-Delphi be automatize and for that a study of methods of text processing

was needed. At this stage, it was chosen the Classical Content Analysis as the method to perform

qualitative analysis. After that, it was necessary to study how to merge these methods: Web-Delphi,

Classical Content Analysis and Scenarios. A study of the methodologies associated to this type of

approach were also studied so a final methodology using all of these approach was created. That was

one of the difficulties founded in this work; to study all of these methods and to create a methodology

using all of the concepts according to rules associated with each one.

The results that were found in this work were similar to the ones found in the manual work. The themes

that arise from the automatic analysis were different from the ones from the EURO-HEALTHY and were

similar to the areas of concern defined in the beginning of the questionnaires which can lead to the

thought that the areas of concern defined in the beginning should be careful defined to help in the

analysis afterwards and to help in the definition of the drivers.

- 45 -

This methodology can contribute to other works because it is replicable and easy to follow. It can be

used not only by applying the Delphi method but in other methods of data collection like questionnaires.

An example where this method can be used is in Census studies promoting a faster analysis of the

answers.

In the future, another analysis should be made regarding the themes that were found in this work and

the themes from the PESTLE taxonomy. It was noticed that although a direct correspondence did not

exist it was possible that different themes were correlated; for example, the Education node corresponds

to the Political node in the PESTLE taxonomy.

Using NLP can help when dealing with aspects like redundancy since it is a more advanced way to

approach the problem using programming. It can be also another way of finding the drivers instead of

using word frequency it can be used NLP to search for the drivers.

In general, this thesis achieves the goals of finding the drivers reducing the number of investigators and

the time consumed in the analysis.

- 46 -

References

Almeida, J. P. (2016). Porto Biomedical Journal. Porto Biomedical, 1(1), 12–24.

https://doi.org/10.1016/j.pbj.2017.05.001

Alvarenga, A., Bana e Costa, C. A., Borrell, C., Ferreira, P. L., Freitas, Â., Freitas, L., … Vieira, A. C.

L. (2019). Scenarios for population health inequalities in 2030 in Europe: the EURO-HEALTHY

project experience. International Journal for Equity in Health, 18(1), 100.

https://doi.org/10.1186/s12939-019-1000-8

Australia, ). (1971). Radioactive Waste Management Stakeholder Involvement Techniques Short

Guide and Annotated Bibliography ORGANISATION FOR ECONOMIC CO-OPERATION AND

DEVELOPMENT. In Hungary Korea (Vol. 22).

Basco-Carrera, L., Warren, A., van Beek, E., Jonoski, A., & Giardino, A. (2017). Collaborative

modelling or participatory modelling? A framework for water resources management.

Environmental Modelling and Software, 91, 95–110.

https://doi.org/10.1016/j.envsoft.2017.01.014

Bolger, F., & Wright, G. (2011). Improving the Delphi process: Lessons from social psychological

research. Technological Forecasting and Social Change, 78(9), 1500–1513.

https://doi.org/10.1016/j.techfore.2011.07.007

Bush, T., Bryce, C., & Direito, S. (2016). Review Article. 16(1), 48–57.

https://doi.org/10.1089/ast.2015.1295

Castro, E. M., Van Regenmortel, T., Vanhaecht, K., Sermeus, W., & Van Hecke, A. (2016). Patient

empowerment, patient participation and patient-centeredness in hospital care: A concept

analysis based on a literature review. Patient Education and Counseling, 99(12), 1923–1939.

https://doi.org/10.1016/j.pec.2016.07.026

Cook, T., Springett, J., Salsberg, J., Wright, M., Wallerstein, N., Harris, J., … Oetzel, J. (2018).

Searching for the Impact of Participation in Health and Health Research: Challenges and

Methods. BioMed Research International, 2018, 1–12. https://doi.org/10.1155/2018/9427452

Dalkey, N. (1969). The Delphi Method: An Experimental Study of Group Opinion. The RAND

Corporation, pp. 408–426. https://doi.org/10.1016/S0016-3287(69)80025-X

Dollah, S., Abduh, A., & Rosmaladewi, M. (2017). Benefits and Drawbacks of NVivo QSR Application.

(November). https://doi.org/10.2991/icest-17.2017.21

Elo, S., & Kyngäs, H. (2008). The qualitative content analysis process. Journal of Advanced Nursing,

62(1), 107–115. https://doi.org/10.1111/j.1365-2648.2007.04569.x

Fernandez Montenegro, J. M., & Argyriou, V. (2017). Cognitive evaluation for the diagnosis of

Alzheimer’s disease based on Turing Test and Virtual Environments. Physiology and Behavior,

- 47 -

173, 42–51. https://doi.org/10.1016/j.physbeh.2017.01.034

Guzys, D., Dickson-Swift, V., Kenny, A., & Threlkeld, G. (2015). Gadamerian philosophical

hermeneutics as a useful methodological framework for the Delphi technique. International

Journal of Qualitative Studies on Health and Well-Being, 10.

https://doi.org/10.3402/qhw.v10.26291

Hartley, J. (2014). Some thoughts on Likert-type scales. International Journal of Clinical and Health

Psychology, 14(1), 83–86. https://doi.org/10.1016/S1697-2600(14)70040-7

Hasson, F., & Keeney, S. (2011). Enhancing rigour in the Delphi technique research. Technological

Forecasting and Social Change, 78(9), 1695–1704.

https://doi.org/10.1016/j.techfore.2011.04.005

Hohwü, L., Lyshol, H., Gissler, M., Jonsson, S. H., Petzold, M., & Obel, C. (2013). Web-based versus

traditional paper questionnaires: a mixed-mode survey with a Nordic perspective. Journal of

Medical Internet Research, 15(8), e173. https://doi.org/10.2196/jmir.2595

Hsiao, Y. J., & Tsai, W. C. (2018). Financial literacy and participation in the derivatives markets.

Journal of Banking and Finance, 88, 15–29. https://doi.org/10.1016/j.jbankfin.2017.11.006

Jewell, S. E. (1994). Patient participation: what does it mean to nurses? Journal of advanced nursing

[Online] . Available at: http://www.ncbi.nlm.nih.gov/pubmed/8014302. 19(Brownlea 1987), 433–

438.

Kvæl, L. A. H., Debesay, J., Langaas, A., Bye, A., & Bergland, A. (2018). A Concept Analysis of

Patient Participation in Intermediate Care. Patient Education and Counseling.

https://doi.org/10.1016/j.pec.2018.03.005

Lang, T. (2000). An Overview of Four Futures Methodologies (Delphi, environmental scanning, issues

management and emerging issue analysis). Manoa Journal of Fried and Half-Fried Ideas (about

the Future), 7(Woudenberg 1991), 1–28. Retrieved from

http://www.futures.hawaii.edu/publications/half-fried-ideas/J7/LANG.pdf

Liddy, E. D. (2001). Natural Language Processing. Natural Language Processing. In Encyclopedia of

Library and Information Science, 2nd Ed. NY. Marcel Decker, Inc.

McCarthy, L. (2002). The brownfield dual land-use policy challenge: Reducing barriers to private

redevelopment while connecting reuse to broader community goals. Land Use Policy, 19(4),

287–296. https://doi.org/10.1016/S0264-8377(02)00023-6

Mohamed, Z. (2018). Series Editor Patrick Paroubek Natural Language Processing and

Computational Linguistics 2.

Newton, P., & Bristoll, H. (2013). PESTLE Analysis: strategy skills. Free-Managment-Ebooks, 998(5),

1–31. https://doi.org/http://www.free-management-ebooks.com/dldebk-pdf/fme-pestle-

analysis.pdf

- 48 -

O poderoso NVivo: Uma introdução a partir da análise de conteúdo. (2016). Revista Política Hoje -

ISSN: 0104-7094, 24(2), 119–134.

Onwuegbuzie, A., Leech, N., & Collins, K. (2012). Qualitative analysis techniques for the review of the

literature. The Qualitative Report, 17(56), 1–28. Retrieved from

http://www.nova.edu/ssss/QR/QR17/onwuegbuzie.pdf

Portugal, I., Alencar, P., & Cowan, D. (2018). The use of machine learning algorithms in recommender

systems: A systematic review. Expert Systems with Applications, 97, 205–227.

https://doi.org/10.1016/j.eswa.2017.12.020

Raalte, D. van. (2008). Scenario planning. (July). Retrieved from

http://dspace.ou.nl/bitstream/1820/1803/1/MWDvan Raalte150908.pdf

Renzi, A. B., & Freitas, S. (2015). The Delphi Method for Future Scenarios Construction. Procedia

Manufacturing, 3(Ahfe), 5785–5791. https://doi.org/10.1016/j.promfg.2015.07.826

Riggs, W. E. (1983). The Delphi technique. Technological Forecasting and Social Change, 23(1), 89–

94. https://doi.org/10.1016/0040-1625(83)90073-2

Rowe, G., & Wright, G. (1991). The Delphi Technique: a Re-Evaluation of Research and Theory.

Technological Forecasting and Social Change, 39(January 2015), 235–251.

Santana, P. (Coordination). (2017). Promoting population health and equity in Europe: from evidence

to policy. 103. https://doi.org/10.14195/978-989-26-1500-4

Sekayi, D., & Kennedy, A. (2017). Qualitative Delphi Method: A Four Round Process with a Worked

Example. The Qualitative Report, 22(10), 2755–2763. Retrieved from

http://nsuworks.nova.edu/tqr/vol22/iss10/15

Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. 15(9), 1277–1288.

https://doi.org/10.1177/1049732305276687

Slocum, N. (2003). Participatory Methods Toolkit: A practitioner’s manual. In United Nations University

– Comparative Regional Integration Studies (UNU/CRIS). https://doi.org/90-5130-506-0

Thadani, K., & Mckeown, K. (2008). A Framework for Identifying Textual Redundancy. (August), 873–

880.

Using the Delphi expert consensus method in mental health research. (2015). Australian & New

Zealand Journal of Psychiatry, 49(4910), 887–897. https://doi.org/10.1177/0004867415600891

- 49 -

Appendix

Table 5- Machine Learning Methods

Method Objective Advantages Disadvantages

SVM Is a type of supervised

Learning technique

suitable for binary

classification. Given a

training set, a SVM build a

model that assigns the new

data to one category or

another.

Is effective in high

dimensional spaces;

The data is learnedly

separable; Can be

robust even when

training sample has

some bias; Memory

efficient; Has a unique

solution.

Lack of transparency in the

results;

Sensitive to outliers,

Difficult to incorporate

background knowledge

Naïve

Bayes

It is based on the Baye’s

theorem with

independence

assumptions between

predictors. It assumes that

the probability

P(BɅCɅD|A) can be

substituted by a “naïve”

approximation that

assumes the value of the

attributes to be

independent.

Robust to isolated

noise points and to

irrelevant attributes;

Missing values are

ignored; Easy to

implement; Requires

a small amount of

training data to

estimate the

parameters.

Independence

assumptions may not hold

for some attributes which

may cause loss of

accuracy; Used only for

categorical variables.

k-Nearest

Neighbor

It is used for both

classification and

regression problems; In

this algorithm, the function

is only approximated

locally, and all computation

is deferred until

classification.

Training set is very

fast; Possible to learn

complex target

functions; Robust to

noisy training data.

Slow at query time-

Memory limitation. It is

sensitive to the local

structure of the data;

Decision

Tree

Is a predictive model that

uses a decision tree to go

from observations about an

object to conclusions about

the final representation.

Nonlinear

relationships between

parameters do not

affect tree

performance;

Highly complex; possibility

of duplication with the

same sub-tree on different

paths; Training set is

expensive; Does not

- 50 -

Easy to interpret,

explain and generate

rules;

handle continuous variable

well; may occur over-fitting;

Neural

Network

It is composed of artificial

neural networks; the

connections of the network

are modelled with weights.

A positive weight is

excitatory and a negative is

inhibitory.

Can adapt to unknow

situations; handle

errors well; prediction

accuracy is high; can

solve any machine

learning problem.

Large complexity of the

network structure; can’t

understand how or why the

learned networks works;

Time consuming process;

Random

Forest

It constructs a multitude of

decisions trees at training

time and outputting the

class that is the mode of

the classes or mean

prediction;

It is efficient in large

data sets; can handle

lots of variables

without variable

detection; generated

forests can be saved

for future use on other

data.

Sometimes overfitting

happens; a large number of

trees make the algorithm

slow; for data including

categorical variables with

different number of levels,

random forests are biased

in favour of those attributes

with more levels;

K-means The aim is to do n partitions

into k clusters and to see in

which cluster each

observation belongs.

It is efficient in small

data sets;

Difficult to choose the

centroid (K); Must

determine the number of

clusters beforehand; Most

often clusters are non-

spherical;

Hidden

Markov

Model

The system modelled is

assumed to be a Markov

process with unobserved

states. Each state will have

a probability distribution

over the possible output

tokens.

Can handle input of

variable length; can

take place directly

from raw sequence

data;

Large number of

unstructured parameters;

cannot express

dependencies between

hidden states.

51

Table 6 – Table of syntactic, semantic and pragmatic curve. Adapted from Bush, Bryce, & Direito, 2016

Syntactic Curve

Keyword Spotting- it is based in a classification of a text into categories based on the presence of certain word.

Semantic Curve

Endogenous NLP - it is built

a structure that approximate concepts from a set of documents using machine learning techniques. Some examples are: latent semantic analysis, latent Dirichlet allocation, MapReduce and genetic algorithms Hidden Markov Models, association rule learning feature ensembles and probabilistic generative models

Pragmatic Curve

The last jump is from

the semantic curve to

the pragmatic curve.

In this curve, the aim is

to decode how

narratives are

generated and

processed in the

human brain to better

understand human

intelligence. Some

examples of work in

this field are

argument-support

hierarchies, plan

graphs and common-

sense reasoning

Lexical Affinity- assigns to arbitrary words a probabilistic affinity of a given category.

Taxonomic NLP- NLP the

objective is to construct a universal taxonomies or Web ontologies to understand the subsumptive or hierarchical semantics associated with natural language expressions. The subsumptive knowledge is based in IsA relationships which come from syntactic patterns for automatic hypernym discovery. Examples of this approach are WikiTaxonomy, YAGO and NELL.

Statistical Methods- comprises machine learning algorithms such as maximum-likelihood and expectation maximization which aims to feed this algorithm with a training corpus to learn the valence of the words as well as the valence of

arbitrary keywords.

Noetic NLP- usually

generate context dependent results or tries to find new semantic patterns that are not encoded in the knowledge base. Examples of this method are connectionist NLP, deep learning, sentic computing and energy-based knowledge representation.

52

Table of Results from the Automatic Methodology

Ageing

• With the current trend of cutting down social benefits and the critical employment situation

the poverty rate of elderly is likely to increase; the aging index will depend on whether the

current opposition to immigration will prevail or be reversed;

• Population ageing increase; Refugees and migrants increase; Reduction of GDP

increase pace all over European regions;

• Reduction of young population (less costs of education systems);

• Poor countries will have difficulties facing increasing ageing of population; Some social

security national systems are sub funded;

• Innovation costs and population ageing are a threat for all national health systems;

Improving health national expenses depends on improving GDPs;

• The recent increase in unemployment in late ages will increase the poverty rate in some

areas;

• At a European level, demographic ageing in Southern countries was latter then in

Northern countries, but much more intense and still on going;

• Ageing index ratio will increase and, consequently, the risk of poverty too, because elders

are a most vulnerable group, including at an economic level (retirement implies income

reduction - directly and indirectly, due the latter to a decreased earning ability (disease,

disability);

• Southern (and Eastern) countries, the most impacted by the financial and economic

crisis, will be "obliged" to spend more money due to their higher ageing index and,

perceptually, due to the post-crisis economic growth, to so the gap will be narrowing in

case this scenario is valid;

• A balanced age structures (value <1) will increase (higher Health expectancy and higher

the number of young persons (from 0 to 14)) in rich regions;

• Aging index will go along to the demographic aging scenario which, except for a few

countries in the north, indicates a progressive increase;

• Expected increase of the elderly population due to increase in life expectancy;

• The ageing index is relatively easy to calculate and the evolution till 2030 can be

predicted;

• Furthermore, the demographical change and the over-aging of the society will increase

expenditures on care for elderly;

• The birth rate could be affected stronger by the economic situation in countries with a

currently advantageous age pyramid like Spain, Greece, and Romania etc. comparing to

German;

• The demographic changes within EU will be the reason for an increase of the Ageing

index;

53

• Due to a higher education in general of the European society, and due to the demographic

change by looking at the increasing figures of life expectancy at the same time, a

reduction of the gap in lifestyle and health behaviors is probable;

• Performance gap of Ageing index is proportionally smaller than performance gap of at

risk of poverty rate of older people - aged 65 years or over;

• Although ageing index (ratio) will be similar in 2030, the performance gap of indicator at

risk of poverty rate of older people - aged 65 years or over (%) seems to increase

inequalities;

• No or minor decrease in Demographic Change inequalities will appear from the following

reasons:- aging of the populations in countries as Germany, Italy, Spain gradually slows,

while in Central and Eastern Europe is increasing in recent time - it effects on reducing

inequalities between countries- due to migration crisis in many European countries might

slightly change age structure of the population with effect to the reducing the average age

and slowing aging of the population;

• Ageing of the populations in countries as Germany, Italy, Spain gradually slows, while in

Central and Eastern Europe is increasing in recent time - it effects on reducing

inequalities between countries;

• As regards obesity, daily smokers, live births by mothers under age of 20, current trends

shows that developed countries will be stable in these areas and less developed countries

(such as Eastern European countries) will improve, what indicates there will be a

decrease in Lifestyle and Health Behaviours inequalities across European regions;

• A gradual deceleration in overall population growth in relation to the "housing capacity"

will affect to improvement of housing conditions;

• There is a risk of increasing inequalities in poverty among elderly because of differences

in working beyond retirement;

• The number of births is decreasing, and this will probably lead to an increase in the

ageing ratio;

Economic

• Reduction of GDP increase pace all over European regions;

• The worsening of material conditions in some populations groups in some areas will

contribute to increase the inequalities;

• As evidence of economic development suggest big gap it will be visible also in healthcare

resources and expenditure inequalities across European regions;

• Although I believe unemployment rate will be decreasing, in accordance to the economic

cycle, income distribution (fairness) will be worsening and its gap widening, unless the

economic rate of less developed countries in Europe grows higher than the most

developed countries;

54

• Overall economic growth, especially within UE countries, will decrease those gaps in a

scenario of economic growth fairness (economic conditions associated to health

behaviors);

• Concerning amenable deaths to healthcare it also depends, primordially, on the

attributable proportion of GDP to the healthcare sector, especially in more economically

vulnerable countries and, primary (i.e., directly), to access to healthcare;

• The gap on the Demographic Changes will tend to increase due to the economic

instability in Europe (mainly affecting the South), increased emigration in countries

already affected by population ageing and decrease in natality;

• There may be an increase with politics as usual and when EU regulations continue to be

imposed on economically unequal nations (especially inside the eurozone), but there may

as well be a political correction on the current economic and social developments;

• The demographic change and an increasing economic pressure (like the cuts in the

pension system) will lead to an increase in inequalities and a higher risk of poverty for the

elderly;

• The demographical change will affect all healthcare resources and related expenditures

within the next few decades;

• Furthermore, the expenditure on care for elderly will increase due to the demographic

change in Germany and Europe;



• Current status quo will lead to in Healthcare resources and expenditure inequalities

across European regions;

• Some of the countries in EU region (mostly from former Eastern block) have still

possibilities for economic growth as they benefit from their membership in EU and from

the economic and political reforms they had to undergo before accepted as a EU

members;

• Those countries which are suffering the most from the 2008 economic crisis (such as

Greece, Spain, Italy) will most likely continue in this trend, as they mostly do not follow

the recommendations of EU necessary for the stabilization of their economic situations

• Expenditure on care for elderly will increase due to the demographic change;

Education

• Education policies all over Europe; Reduction of young population (less costs of

education systems);

• The monitoring of education results (although biased towards productivity indicators) will

make decision-makers to pay more attention to education polices;

55

• In the New EU countries situation will stabilized as they suffer from demographic decline

and they are after peak of tertiary education enthusiasm after which there was no revival

in the chances of getting better jobs;

• There is a negative correlation between education attainment and early leavers, at an

ecologic (country) level;

• National policies to improve the upper secondary and tertiary education levels;

• The improvement in general education of the European populations and increase in

health literacy of European citizens will contribute to the adoption of healthy lifestyles;

• It could be a growth of far-right extremism in the EU and in the Member States, therefore

policies allocating few resources for public education and decreasing its quality;

• Although the EU and National governments will put policies in place to combat the

increasing challenges to access to education the tradeoff is just not good enough and will

lead to no change in inequality across Europe’s access to education;

• In general the educational level of people will increase due to the academization of our

society;

• In most industrial countries the smoking rate decreases with increasing education level;

• The performance gap of indicator Population aged 25-64 with upper secondary or tertiary

education attainment (%) is almost three times higher than the value of indicator;

• Educational system in most EU countries is stabilized and we do not expect any changes

within particular educational systems as it seems that other problems are currently being

solved (e.g., migration crisis, etc.);

• The majority of population is increasing their level of education, therefore a higher % of

the population will have higher level of education, decreasing education inequalities;

• There might be less tertiary education attainment due to higher university fees in some

European countries, but this could be compensated by higher secondary education

attainment;

• The smoking rate and obesity decreases with increasing education level.

• Increasing the share of immigrants with low education level;

• Widening and increasing educational attainment is a common target of all the EU

countries; it may not reduce social inequalities in health within country, but it should

reduce health inequalities between countries;

• I believe that in the next years more people in Europe will have access to secondary and

tertiary education;

• In the next years in Europe, more people will be mitigating resulting to an increase in

demographic change inequalities;

56

Employment

• While the overall economic conditions measured in GDP, productivity will increase though

growing only slowly the social security for the majority of the population will decline or

stay unchanged at the best; the trend of employment with only critically low income/limited

social benefits, as well as the trend to multiple employments and/or temporary

employments will continue and might even further increase; thus the disposable income

and risk of poverty for the disadvantaged part of the society will develop not in a way that

would lead to closing the current gaps; expenditures on care for elderly will increase due

not the least to the large voting bank of the elderly population; crime statistics will be

adjusted based on political considerations and might not reflect the economic or social

development;

• The trend is for general higher education and an "academicization" of job profiles that

were based on vocational training before; with reduced employment opportunities for

unskilled work and more efforts to reduce the number of drop-outs the number of early

leavers will decline;



current opposition to immigration will prevail or be reversed (the later seems not very

likely at the moment);




developed countries;

• EU has been gathering a rising unemployment rate in population under 30 years in a

large nº of countries while, at the same time, facing a huge structural unemployment

related with low qualifications.

• Long-term structural unemployment scenarios contribute in the medium term to more

fragile social protection for the elderly (65+);

• A stronger integration of EU labor markets could reduce unemployment rates in some

regions, but could be linked with somewhat higher unemployment rates in other regions,

or higher inequalities between groups of employees in other regions;

• The current reactions to address unemployment issues and to "invest" in economic

growth is to "invest" in (formal) education;

• In regions of Spain or Greece for instance, which are currently affected very strong by

e.g. unemployment and long-term unemployment, it is not expected that the situation

there will change dramatically and become worst within the next decades, because it is

already very bad;

• It is not expected that countries like Austria or Germany will always keep their advances

in economic power and employment situation;

57

• Unemployment rates in general will probably increase in Europe, if we consider the

instable political developments in almost every countries within the EU.

• New developments in manufactural industries will affect the way of employment rates

dramatically;

• Early retirement, due to increased unemployment and low birth rate, as a result of

economic crisis in some countries of the Southern Europe;

• Inequalities related to smoking between countries may increase as in some western

countries smoking prevalence is decreasing more rapidly than in eastern and southern

European countries;

• In Europe, the unemployment is becoming an increasing problem due to the economic

crisis

Environment

• Tightened leading slowly to a reduction of daily pollutant concentrations; GHG emissions

will be reduced following the Paris agreements and the political aim to keep a leading role

in sustainable industrial development; exposure to noise pollution will also but slowly

decline due to restrictions for inner cities and as a co-benefit of energy efficiency

(double/triple glassing) in the part of Europe that have usually cold winters; while there is

a lot of effort to build new retention areas for rivers the climate change and increase in

mean temperature will lead to further and more intense extreme events thus the

population affected by flooding might not decrease, actually to the contrary there is some

likelihood that this number would increase;

• As long as EU won't collapse altogether, physical environment standards should

converge due to harmonized and quite well-endorsed (straightforward, funded and thus

implemented) related European legislature;

• European (UE) environmental regulations, alongside with international and worldwide

agreements and population-based advocacy will decrease inequalities at an

environmental and European level;

• Population awareness concerning environmental issues and the impact of environment

on health is growing steadily today, especially among youngsters and young European

adults;

• The consequences of the climate change and the unsuitable urban growth;

• Central policies for taxation and imposition of recycling quotas;

• Considering public investment in Europe; increasing concerns related to natural

resources efficiency, namely energy efficiency Built environment it is expected to

increase, according to the exposed criteria;

• The current instability/unpredictability of the development of the "green" agenda vs oil-

based agenda to support economic growth (mainly in USA, but possibly extending to EU

58

countries) may in fact impact the development of European economies, with strong

environmental impact;

• The increase expected in the general education of European Citizens will enforce the

politicians to deliver better build healthier environment programs;

• There is an increase of policies with the aim of reducing the carbon emission (climate

change policies);

• Existence of European policies that try to improve air pollution and decrease traffic noise;

• Due to heavy traffic no improvements or more likely the increase of PMx concentration

and traffic noise in most affected areas would be expected while on the other hand the

not affected areas would not change;

• Climate change will produce of increase of inequality between regions, i.e. between

regions affected by flooding or not;

• Road traffic is a part of modern society, no improvement would be expected in

area/regions with heavy traffic;

• The main reason is the growing awareness of the problem and the increasing pressure

to take measures to improve air and noise conditions;

• Due to climate change and increased extreme weather events, an increase in flooding

will be expected;

• Due to further infrastructure improvements and decrease in built environment inequalities

will be reduced across Europe;

• Where the air and environmental pollution in general will increase worldwide, the air

pollution indicators will probably decrease slightly within the next 14 years within the EU;

• The reasons are political programs and initiatives to reduce the CO2-emission in the

countries of Europe, and increasing numbers of environmental friendly vehicles in

general;

• Possible decrease in differences in population affected by flooding as more regions

currently not high in the levels of the indicator may increase as result of climate change;

• No change in air pollution levels as shown by trends in particulate pollution levels in

European countries;

• Developed countries will most likely continue in the sustainable development trends,

though in less developed countries the situation will be different due to acceleration in

industry development and generally lower "environmental awareness";

• Actions against climate change should work and decrease inequalities;

• Due to policies and measures in various countries to improve environmental conditions;

• Physical Environment inequalities across European regions are likely to decrease as EU

environmental protection and climate change legislation is being implemented;

• Recycling rates are likely to increase in most European regions, particularly in regions

where there is a lot of scope for improvement;

• Physical environment is nowadays a major social and political preoccupation;

59

• I expect a reduction in Physical Environment inequality due to the approved legislative to

reduce pollution and improve quality of physical environment;

• Given that compliance to environmental regulations may be strongly related to economic

development, whose inequalities will hardly change by 2030, even inequalities in Physical

environment indicators will remain stable;

• The pollution has been decreasing the most of the developed countries and thus I believe

there will be a decrease in physical environment inequalities across Europe;

• I would say a decrease for the four first indicators, however, I tend to think that it could

increase for flooding due to climate change.

Health

• While sedentary lifestyles and the trend to unhealthy diets and eating habits will continue

and therefore not result in much change in regard to the percentage of obese adults,

smoking will continue to decline as will alcohol consumption though the latter to a much

lower degree; regarding the young motherhood I would not be able to identify a trend;

• While I do not think that the ratio of medics/health workers per population is necessarily

an accurate indicator for access to care and rather a reflection of the specific organisation

of the respective health system I expect a further increase in total health expenditures

due to more costly treatment options and the continuation of a trend to privatisation of

services; at the same time there will be a continuation of the trend to increase co-

payments by patients (out of pocket payments) thus increasing inequality among regions

and within regions;

• As the underfinancing of the health systems will most likely prevail and the social

determinants will not much improve there will be a negative trend for both indicators;

• More healthy individual behaviors are spreading all over the world, facilitated by higher

levels of education and literacy; Available information on better lifestyles is reaching more

people everyday;

• More healthy individual behaviors are spreading all over the world, facilitated by higher

levels of education and literacy; Available information on better lifestyles is reaching more

people; Although, poverty can be an important obstacle to a healthier consumption of

food or to the reduction of tobacco and alcohol consumption

• Innovation costs and population ageing are a threat for all national health systems;

Improving health national expenses depends on improving GDPs;

• There is a lack of political commitment in this area together with insufficient ideological

movements to support fair health care policies;

• Medical progress gives a chance for rapid development of Healthcare resources if there

is enough money for that goal;

• As evidence of economic development suggest big gap it will be visible also in healthcare

resources and expenditure inequalities across European regions;

60

• More discharges due to diabetes and other chronic conditions does not mean "good

news": it depends on the effectiveness of prevention of new cases of disease, secondary

to public health policies and cleaner (outdoor) air/social and physical environment;

• Concerning amenable deaths to healthcare it also depends, primordially, on the

attributable proportion of GDP to the healthcare sector, especially in more economically

vulnerable countries and, primary (i.e., directly), to access to healthcare;

• There will be a greater commitment to following EU directives and WHO guidelines

(considering the health outcomes related with built environment);

• For this group of indicators: Total health expenditure (THE), Private households ‘out-of-

pocket on health as percentage of total health expenditure, Public expenditure on health,

PPP$ per capita;

• Improvement of the primary health care services: quality and accessibility;

• More aggressive educational and public health campaigns and increased taxation may

contribute to a slowdown in the trend and increase healthy behaviors;

• The improvement in general education of the European populations and increase in

health literacy of European citizens will contribute to the adoption of healthy lifestyles;

• Economic disparities across countries will account for disparities in private and public

health care services, with strong impact on management of chronic diseases;

• Decreasing proportion of daily smokers among most of the regions;

• Daily smokers decrease due to better awareness and restriction policies;

• Alcohol consumption: although awareness policies will further tackle alcohol consumption

due to stress increase and unhealthy lifestyle habits no overall change will be visible;

• The correlation between (formal) education and health inequities is not hammered in

stone; higher education does not automatically mean less inequalities if hierarchies in

socio-economic positions will not change or even growth.

• Medical staff: situation might become worth, especially for rural areas, due to

demographic developments and migration between (and within) regions;

• A reduction of disparities in medical standards and health behavior could be expected,

which would lead to an increase of aging within regions;



• Hence, a general increase of employees and staff with in the health sector is probable

and can be expected;

• It is also possible that expenditure for health resources will increase disproportionally in

the regions, which are currently not particularly high;





61



• Health costs and expenditures per capita will increase;

• Maybe due to the demographical change, the gap of healthcare performance (Hospital

discharges and amenable deaths to health care) will probably increase.;

• Health personnel (nurses and midwives, dentists, pharmacists and physiotherapists), per

100 000 inhabitants, Total health expenditure (THE), PPP$ per capita, WHO estimates

Public expenditure on health, PPP$ per capita, WHO estimates have performance gaps

more than 5 times higher than the value if indicators;

• Reduction of inequalities in this area request a huge restructuralisation of healthcare

systems of some regions, which is costly and mostly hard to perform;

• as regards obesity, daily smokers, live births by mothers under age of 20, current trends

shows that developed countries will be stable in these areas and less developed countries



• This will depend on the economic situation and investment in public health and health

care systems across Europe;

• While the overall health situation in Europe is improving, inequalities have been

increasing since the 1980s;

• The concentration of equipment and clusters of medical activities observed throughout

Europe is accompanied by a widening gap in the provision of care;

• Possible reduction in health expenditure at the top and medical doctors at the top may

result in reduction of inequalities but this would probably be minimal;

• The higher education in general should initiate a the gap in lifestyle and health behaviors

is probable;




• Health care resources and expenditure is directly related to GNP, given that inequalities

in GNP are expected to remain unchanged even inequalities in health care expenditure

will remain the same;

• Health care systems are strongly committed to appropriateness all over Europe, we

expect this trend to benefit more the countries with worst performance;

• Economic inequalities end health care access inequalities;

• There have been a lot of efforts towards decreasing obesity and encourage people to

healthy diet, reducing smokers and alcohol consumption;

• In some European countries, there have been considerable changes in the healthcare

systems towards reducing public expenditure;

• As before, in some European countries, there have been considerable reductions of the

public expenditure in the healthcare system;

62

Policies

• In most of Europe, there are long-term trends of growing economic within-country

inequalities with ever greater proportions of populations falling into the trap of precarious

living conditions in most countries, mainstream political parties fail(ed) to prevent de-

powering of the state on behalf of big business and are now being replaced by nationalist

radicals elected by growing discontent masses who aim and gradually become

successful at further fragmentation and dismantlement of the previously hard-built

traditional political structures which stabilize civil societies such as universal access to

decent living conditions;

• As long as EU won't collapse altogether, physical environment standards should

converge due to harmonized and quite well-endorsed (straightforward, funded and thus

implemented) related European legislature;

• Inequalities will increase for the lack of strong policies to reduce them;

• Depending on the heterogeneity of economic evolution (traffic density), the effect could

cancel present policies;

• There is a lack of political commitment in this area together with insufficient ideological

movements to support fair health care policies;

• The impact attributable to those regulations will be highest within former soviet satellite

countries, thus decreasing inequalities between European regions;

• Individual countries have distinct health systems and economic and demographic

patterns;

• More discharges due to diabetes and other chronic conditions does not mean "good

news": it depends on the effectiveness of prevention of new cases of disease, secondary

to public health policies and cleaner (outdoor) air/social and physical environment;

• Local or national public policies to reduce early leavers from education and training;

• National policies to improve the upper secondary and tertiary education levels;

• EU regulations and countries policies that can promote healthy environments;

• The persecution of restrictive policies on vehicles (inside and outside city environment)

and industry, favors the reduction of these indicators, despite the degree of uncertainty

that may be associated with the alteration of the American commitment (Paris, 2016);

• It could be a growth of far-right extremism in the EU and in the Member States, therefore

policies allocating few resources for public education and decreasing its quality;

• There is an increase of policies with the aim of reducing the carbon emission (climate

change policies);

• It is possible that these countries increase its motorization and improve its policies for

reducing traffic accidents, therefore eastern countries indicators could come more similar

to the rest of the countries;

• Existence of European policies that try to improve air pollution and decrease traffic noise;

63

• The risk of poverty in the other hand depends on policy measures and the increase or

decrease of the range depends as well from the best as from the worst performers;

• Cutting public spending, privatization of public services and market deregulation.

• With further privatization of services in the neoliberal governmental approach, the access

to education through increasing privatization will be more difficult;

• Daily smokers decrease due to better awareness and restriction policies;

• Alcohol consumption: although awareness policies will further tackle alcohol consumption

due to stress increase and unhealthy lifestyle habits no overall change will be visible;

• Some of the countries in EU region (mostly from former Eastern block) have still

possibilities for economic growth as they benefit from their membership in EU and from

the economical and political reforms they had to undergo before accepted as a EU

members;

• Decreasing inequalities related to households with central heating, flushing toilets and

connection to public water supply due to improving housing conditions and urban

infrastructure in European regions;

• Liberal economic policies / nationalism / Austerity policies;

• Austerity policies that will reduce investments in social help;

Road Safety

• Despite some unexpected negative changes recently (increase of fatal accidents in

Germany) the general trend is improved road safety (this includes safety in new cars,

stricter controls, and improved EMS services) thus a decline of severely injured and killed

victims is very likely;

• Better road safety national regulations; Better road conditions;

• Higher quality of roads in southern and eastern Europe;

• Progressively more penalizing traffic policies, greater vehicle safety, although with

significant regional asymmetries;

• The decrease in inequalities will be mainly accounted by the increase in road and cars

safety;

• It is possible that these countries increase its motorization and improve its policies for

reducing traffic accidents, therefore eastern countries indicators could come more similar

to the rest of the countries;

• Road traffic is a part of modern society, no improvement would be expected in

area/regions with heavy traffic;

• Due to reduction, of quality EMS service in remote and or rural areas, an increase of

fatality rate due to road traffic accidents can be expected.

• Due to improved EMS service and better road infrastructure overall an decrease in road

safety inequality across Europe can be expected.

64

• The performance gap of indicator Victims in road accidents - injured and killed, per 100

000 inhabitants is relatively small to value of indicator;

• Based on this one cannot expect introduction of new road safety policies which will

decrease inequality within regions;

• I am not aware of measures that could reduce road accidents;

• Road safety is likely to improve in most European regions, particularly where there is a

lot of scope for improvement;

• Improvements are being made in road safety;

• there have been a lot of measures for improving road safety in Europe, so i think that

there will be a decrease in road safety inequalities;

Social

• While the overall economic conditions measured in GDP, productivity will increase though

growing only slowly the social security for the majority of the population will decline or

stay unchanged at the best;

• The trend of employment with only critically low income/limited social benefits, as well as

the trend to multiple employments and/or temporary employments will continue and might

even further increase; thus the disposable income and risk of poverty for the

disadvantaged part of the society will develop not in a way that would lead to closing the

current gaps; expenditures on care for elderly will increase due not the least to the large

voting bank of the elderly population; crime statistics will be adjusted based on political

considerations and might not reflect the economic or social development;



current opposition to immigration will prevail or be reversed (the later seems not very

likely at the moment);

• Poor countries will have difficulties facing increasing ageing of population; Some social

security national systems are sub funded;

• The recent increase in unemployment in late ages will increase the poverty rate in some

areas;




developed countries.

• More aggressive educational and public health campaigns and increased taxation may

contribute to a slowdown in the trend and increase healthy behaviors;

• Given that in some cases countries are already at the best possible level, the

improvement in other countries can only lead to reductions in inequalities;

65

• The gap on the Demographic Changes will tend to increase due to the economic

instability in Europe (mainly affecting the South), increased emigration in countries

already affected by population ageing and decrease in natality;

• Concentration of highly educated population in the most prosperous areas and on the

other hand high proportion of population with low level of education in deprived areas;

• Reduction of inequality due to increasing proportion of population connected to public

water supply, population connected to wastewater treatment plants, increasing proportion

of households with indoor flushing toilet;

• Most probably there will be a decrease in inequality between regions because worst

performers can easier realise some progress and, except in case of exceptional and

enduring economic and social crises, improvement in performance tends to be the

general trend.

• Although the EU and National governments will put policies in place to combat the

increasing challenges to access to education the tradeoff is just not good enough and will

lead to no change in inequality across Europes access to education;

• Adults who are obese will further rise due to unhealthy eating habits and economic stress

increase;

• Demographic developments and social reforms seem to guide us into this direction;

• A general increase of employees and staff with in the health sector is probable and can

be expected;





• The performance gap of indicator Pure alcohol consumption - aged 15 as well as

performance gap of indicator Adults who are obese (%) predict increase in Lifestyle and

Health Behaviours inequalities;

• New forms of housing make living possible in a new places which can make the

performance gap of indicator Population density (inhabitants/km2) smaller and reduce

inequalities between regions;

• as regards obesity, daily smokers, live births by mothers under age of 20, current trends

show that developed countries will be stable in these areas and less developed countries



• Smoking is still increasing among women in some European countries and mainly among

manual workers, increasing inequalities;

• Austerity occurred with the current financial crisis has worsened the health services and

mainly this affects poor populations who can not afford to pay private services;

• If "social protection" increases, one would expect that "security inequalities" would

decrease;

66

• Increasing urbanization is likely to increase inequalities related to population density and

average number of rooms per person

• Decreasing inequalities related to households with central heating, flushing toilets and

connection to public water supply due to improving housing conditions and urban

infrastructure in European regions.

• The smoking rate and obesity decreases with increasing education level;

• I think that the main reason will be aging of population and increasing proportion of

economically dependent population;

• Most of EU geographical inequalities in EU are strongly rooted in the history of Europe

and can hardly be changed in such a short time;




• The main drivers of life style change work in the same direction and strength all over the

EU countries leaving geographical inequalities similar;

Documents

Enhancing the Delphi method in Health decision- making: … · among them the Delphi Method. In this method, worldwide participants can advocate for their points of view and discuss