Embed Size (px)
Citation preview
Mental models for decision-making in remote healthcare services
MASTER THESIS WITHIN: Informatics
NUMBER OF CREDITS: 30
PROGRAMME OF STUDY: Information Architecture & Innovation
AUTHOR: Wilian Molinari
JÖNKÖPING May 2020
A case study
several rows
i
Master Thesis - Informatics
Title: Mental models for decision-making in remote healthcare services – A case study
Authors: Wilian Molinari
Tutor: Andrea Resmini
Date: 19.05.2020
Key terms: Information architecture; decision-making; categorization; classification; mental
models.
Abstract
Mental models are an important theme within information systems. They show how people
understand reality, make decisions and how the information flows in order for them to do so. It
is particularly challenging to make such decisions when it comes to determining the health of
someone, making it a particularly delicate matter.
In that context, this thesis takes BetterDoc as object of study. It is an organization situated in
Cologne, Germany, with an increasingly positive record of providing remote healthcare
services and allowing patients to have the adequate treatment for their condition.
The study was based on the theory of mental models to bring to light the implicit patterns
present in making decision in that context. That was done by conducting qualitative interviews
with the staff of the organization, across different teams, and synthetizing the findings in a
common model that shows points of decision and the supporting information.
Those findings are useful for identifying points that need to be structured to provide clarity and
understanding, increasing the synergy and transparence of a socio-technical system that can
influences the outcomes of healthcare for many people.
ii
Acknowledgements
It is impossible to deliver this thesis without having, as casually called, “mixed feelings”. While
it is a proud moment that marks the completion of a degree, it also means the end of a period.
A period that lasted 2 years, being full of mind-opening experiences, joyful moments and
uncountable learnings that will for sure play a major role in shaping the rest of my life.
Therefore, one of the good feelings of the moment is the immense gratitude to many people.
My first thoughts go to my family in Brazil, my home country. It is hurtful not to be able to
share this moment with them as I would have expected, given the current pandemic. Without
their absolute support, completing a master’s degree in Sweden would be nothing but a dream.
For that, I owe them my most sincere gratitude.
Second, I would like to thank Andrea Resmini, who provided invaluable guidance throughout
the whole program. Also, I would like to praise his unconditional commitment to his vocation
as a professor and mentor, even in face of organizational challenges.
Next, I thank my colleagues who fostered an environment that can only be defined as “learning
made fun” from the very start. To Wiebke Henke and Rients van Blanken, who were crucial in
daily providing the support I needed to overcome social isolation in a foreign country, I have
no words but: thank you “fam”.
In conclusion, I want to thank all colleagues at BetterDoc who welcomed me, already as an
intern, and showed great collaboration. Specially, I thank Konstantin Weiss who saw potential
in me and enabled me to have such opportunity.
iii
Table of Contents
1. Introduction ........................................................................................................................ 1
1.1 Background ................................................................................................................ 2
1.2 Problem Discussion ................................................................................................... 4
1.3 Purpose ....................................................................................................................... 6
1.4 Research Question ..................................................................................................... 7
1.5 Research aim .............................................................................................................. 8
1.6 Delimitations .............................................................................................................. 8
1.7 Definitions .................................................................................................................. 8
1.7.1 Borg ........................................................................................................................ 8
1.7.2 PARC ................................................................................................................... 10
1.7.3 Case ...................................................................................................................... 11
1.7.4 Expert network ..................................................................................................... 11
2. Theoretical framework ..................................................................................................... 12
2.1 Socio-technical systems ........................................................................................... 12
2.2 Categorization and controlled vocabularies ............................................................. 14
2.2.1 ICD codes ............................................................................................................. 19
2.2.2 OPS codes ............................................................................................................ 19
2.3 Mental models .......................................................................................................... 19
2.4 Place-making ............................................................................................................ 21
2.5 Containerist .............................................................................................................. 21
2.6 Agile software development .................................................................................... 22
3. Methods ............................................................................................................................ 23
3.1 Research settings ...................................................................................................... 23
3.2 Research approach ................................................................................................... 24
3.3 Methods of data collection ....................................................................................... 25
iv
3.4 Data analysis ............................................................................................................ 26
3.5 Credibility and generalizability ................................................................................ 27
3.6 Ethics ........................................................................................................................ 28
4. Results .............................................................................................................................. 29
4.1 Communication ........................................................................................................ 30
4.1.1 Process ................................................................................................................. 31
4.1.2 Intent .................................................................................................................... 31
4.1.3 Empathy ............................................................................................................... 32
4.2 Health ....................................................................................................................... 33
4.2.1 Condition .............................................................................................................. 33
4.2.2 Treatment ............................................................................................................. 34
4.2.3 Medicines ............................................................................................................. 34
4.2.4 Biometrics ............................................................................................................ 35
4.2.5 Documentation ..................................................................................................... 35
4.2.6 Insurance .............................................................................................................. 36
4.3 Medical doctors ........................................................................................................ 36
4.3.1 Specialization ....................................................................................................... 36
4.3.2 Workplace ............................................................................................................ 37
4.3.3 Conflict of interest ............................................................................................... 37
4.3.4 Expertise .............................................................................................................. 38
4.3.5 Feedback .............................................................................................................. 38
4.4 Tools ........................................................................................................................ 39
4.4.1 Database ............................................................................................................... 39
4.4.2 Questionnaire ....................................................................................................... 39
4.4.3 Borg ...................................................................................................................... 40
4.4.4 PARC ................................................................................................................... 40
4.4.5 Phone calls ........................................................................................................... 40
v
4.4.6 External sources ................................................................................................... 41
5. Analysis ............................................................................................................................ 43
5.1 Mental Model ........................................................................................................... 43
5.1.1 Admit a patient ..................................................................................................... 45
5.1.2 Perform research .................................................................................................. 46
5.1.3 Follow-up case ..................................................................................................... 52
5.2 Patterns of decision and categorization .................................................................... 53
5.3 Difference analysis ................................................................................................... 57
6. Conclusion ....................................................................................................................... 59
7. Discussion ........................................................................................................................ 61
7.1 Methods and results discussion ................................................................................ 61
7.2 Implications for research and practice ..................................................................... 62
7.3 Future research ......................................................................................................... 66
8. Reference List .................................................................................................................. 68
9. Appendix .......................................................................................................................... 73
9.1 Interview guide ........................................................................................................ 73
9.2 Semi-structured questionnaire ................................................................................. 74
vi
Figures
Figure 1 - BetterDoc operational process .................................................................... 3
Figure 2 - Box diagram showing the data components of a case ................................. 5
Figure 3 - Inquiry workflow in Borg ........................................................................... 9
Figure 4 - PARC springboard. ................................................................................... 10
Figure 5 - Relationship between terms within a controlled vocabulary .................... 14
Figure 6 - Taxonomies, facets and tags as categorization schemes ........................... 17
Figure 7 - Example of mental model diagram ........................................................... 20
Figure 8 - The mental model diagram ........................................................................ 44
Figure 9 - The “admit patient” mental space ............................................................. 45
Figure 10 - First set of task towers in “perform research” mental space ................... 47
Figure 11 - Second set of task towers in “perform research” mental space ............... 50
Figure 12 - Task towers in “follow-up case” ............................................................. 52
Figure 13 - Network diagram of decision points. ...................................................... 54
Figure 14 - M3 hierarchy of inquiring systems ......................................................... 64
Tables
Table 1 - Overview of themes and topics extracted from interviews ........................ 30
1
1. Introduction
Technology is transforming the healthcare industry. The availability of features such as
electronic records, enhanced monitor and care and telemedicine, reduce waiting times and
provide patients with faster recovery (Health Europa, 2019).
With each time more technology and devices being available, health professionals have
increasingly numerous and dynamic sources to look for information. With mobile,
connected devices any information they might need is a fingertip away, pretty most
anywhere and anytime – drug information, research, studies, patient history and records,
to name a few (Healthcare Business & Technology, 2019). Simultaneously, initiatives to
use artificial intelligence and machine learning are developed each day. Identifying
diseases, issuing diagnosis, developing treatment plans and monitoring epidemics are
some of the applications that should make health data double every 73 days during 2020
(Marr, 2019).
Despite the increasing amount of data, decision making in healthcare is often
characterized by the lack of transparency. That is due to the multiplicity and complexity
of dimensions involved in assessing a determinate situation (Mühlbacher & Kaczynski,
2016). That brings back a challenge forecasted already decades ago. Wurman (1989)
wrote about information anxiety, a condition when an individual struggles to draw
knowledge from data; either by being overwhelmed or not having enough information.
To improve clarity and understanding of the information flows in products and services,
information architecture is part of the development. It goes beyond designing particular
artifacts, into understanding systems in their entirety to draw meaningful semantic
relations to structure systems with understandable and findable information within
complex environments (Rosenfeld et al., 2015). In order to have an effective design
process, there is great value in including the community of users. The knowledge gained
through their living experience must be respected by the designers (D. Cagle, 2019).
It is in this context that this research was carried at BetterDoc, a startup specializing in
providing healthcare services and accomplishing rising numbers throughout the last
2
years. The focus was defined as the understanding of the decision-making throughout the
service, aimed at directing patients to the adequate professionals who can provide the
treatment they need; avoiding unnecessary consultations and procedures that costs the
patients’ time, money and wellbeing.
1.1 Background
BetterDoc is a German company founded in 2012, headquartered in Cologne. Its service
is dedicated to matching patients who have trouble dealing with a health condition with
an adequate specialist to provide the right treatment, avoiding unnecessary appointments
or procedures. Up to this day, BetterDoc employs around 70 people allocated in different
teams, each responsible for different aspects of the business. When it comes to delivering
value to patients, there is a well-established process flow in which the different teams
have their own roles. Some as a touch point for the patient, others as a support to assure
it runs smoothly. Currently, the employees are allocated in the following teams:
• Operations
• Business Development
• Medical Development
• Product Development
• Human Resources & Finances
When it comes to the process itself, there are distinguished phases during which the main
players are the patient care and medical development, part of operations and medical
development, respectively. These are the main touch points, responsible for direct contact
with the patient to deliver the expected results. It is possible to describe the process in 5
different phases:
3
Figure 1 - BetterDoc operational process
Admission: In this phase, the patient is admitted to the service. This is the moment where
the patient explains their initial concern and the staff confirms the alignment to what the
organization offers. Information here is recorded into an information system that serves
as input for the following phases. The main goal here is to narrow down the kind of
ailment suffered by the patient so a structured questionnaire can be sent, containing
questions that are meant to lead to a clear diagnose.
Anamnesis: Following admission, there is a timeslot schedule where a medical
professional calls the admitted patient, having available the information they provided
before. This moment is meant to clarify any doubts regarding the questionnaire filled by
the patient, as often they are unable to describe it in a medical terminology. The
conclusions from this diagnosis is also recorded in the same information system.
Matching: The core of the value of the service offered by the organization. Based on the
diagnosis from the previous phase, a team of researches takes over the case. According
to the identified ailment, they define the medical criteria they would need a professional
to have and therefore recommend to the patient. This phase the information flow is not as
clearly defined as the previous stages, being determined by each individual. They may
use different sources to seek the information they find necessary to make a decision.
Follow-up: After making a recommendation, there is a follow-up of 24 month to monitor
the condition of the patient after having the appointment for the requested physician.
Archive: The case is considered completed and no further action is taken, keeping the
data for analytical purposes.
Admission Anamnesis Matching Follow-up Archive
4
1.2 Problem Discussion
At BetterDoc, the diagnostic process takes into consideration the answers of a patient for
a questionnaire designed to provide initial information about their clinical condition in a
structured manner. In some cases, the information provided is enough to start the research
process. Although, the answers from the questionnaire still have to be processed manually
by the research team even if they follow implicit patterns. These patterns are subject to
the interpretation of the staff when performing the research, meaning that different people
working on the same case may have different conclusions from it.
When matching a case to a physician, there are two dimensions of information regarding
the case that need to be considered. First, there is the personal information regarding the
patient. Such as their home address, which will be the base location to define a search
radius, and also any particular preferences specified by the patient. Example, a woman
that does not want to go to a male gynecologist. On the other side, there is information
regarding the symptoms of the ailment, provided by the patient in the format of structured
questionnaires. There are different categories of questionnaires, having been written by
the Medical staff at BetterDoc. When receiving the answers of a questionnaire, the staff
can decide if there is enough information regarding the diagnosis to proceed with the
matching or if there is the need to clarify it further. In case there is the need for
clarification, that is done during an appointment where the staff makes a phone call with
the patient to ask the pertinent questions for the case. Each case varies in complexity and
area of expertise, for which throughout its operations BetterDoc has managed to use
patterns to adapt and optimize its operations. Figure 2 shows a representation of how this
data is contained within a case.
5
Figure 2 - Box diagram showing the data components of a case
The figure shows a concept of how the data within a case is compartmented. It starts by
the patient providing his data in two aspects, personal and medical. In a personal manner,
some examples of data collected is the name, address, age, and radius the person is willing
to travel. These relate mostly to data privacy, as they are needed for identity validation.
In a medical aspect, the patient is asked to fill a questionnaire which collects data
regarding symptoms, previous diagnoses, and recommended therapies. It is also possible
to have medical documents such as examinations. All these data points are collected as
data concerning the patient to who the case is related, and the base for research defining
the physicians who constitute the recommendation. It is only an illustrative diagram and
does not illustrate all the data points collected throughout a case.
6
Having enough information for a clear diagnosis of the problem and therefore the
adequate treatment, it is possible to start the matching process. For that purpose, there is
a search system involving tags. Based on the diagnosis, the researcher determines a set of
tags and that is called a “search profile”, which can be any combination of tags in the
system. This search profile then returns the entries of physicians in the database that
match the given combination of tags. The tags are completely independent from each
other and there is no logic system in place for validating conditions. For example,
contradictory tags can be put in the same profile although there will be no matches for the
query. Also, tags can be freely written and created by any researcher, anytime one
believes that there is not yet a tag for the description they want. These tags are used to
describe competences, roles, specialization and pretty much anything that comes to mind.
In the future, there is an alternative being developed and tested to replace this matching
system by using an architecture of attributes, having a hierarchy which means that
detailing can be added with a reduced number of tags. For example, a physician having
the attribute “knee surgeon” needs above all to be categorized as “surgeon”, and even if
not displayed in the interface this data structure concept automatically respects this
hierarchy and avoids redundancy. These attributes will establish a controlled vocabulary
according to which it will be possible to describe in a structured manner the profile of
each physician in the database, what is known and also what is not unknown about them.
Based on this, there is the goal to also determine what is called specialist types. These
specialist types consist in a combination of attributes that is needed for a physician to be
able to perform certain treatments. The next step is then to have this new architecture to
replace the search profiles, being then possible to map the different diagnoses to the
according specialist type desired to perform the adequate treatment. Combining the
specialist type and the location specifications, it would then be possible to narrow the
results down to what would be suitable recommendations for the patient.
1.3 Purpose
BetterDoc is a data-driven organization that is yet to develop large scale analytical
systems. Most of the analyses currently performed are descriptive, essentially
summarizing in different ways how the company performs over time and the
7
characteristics of the cases it runs. There is a large database to be worked on, which can
serve as material to draw patterns to leverage operations, speeding up the processes by
gradually automating parts of it.
The application of such methods can potentially bring the organization benefits such as
increased capacity, while also reducing the waiting time for patients by quickening the
matching process. The most interesting for this research although would be to identify the
patterns based on which recommendations are made. Currently, they are intrinsic and
subjective to the understanding of each researcher in the Medical Development at
BetterDoc. This means that different researchers may have different interpretations on the
diagnosis of a case, having divergences on the outcome. As of today, there is no way to
assess that. The intention of this study is to analyze the thinking process of the staff and
draw the relations for the information flow when it comes to making decisions. This way
it would be possible to identify personal biases of the staff when doing so. Also, it would
be possible to use these logical patterns to facilitate the onboarding of new staff.
Considering BetterDoc richness of data, there are many areas to act in terms of performing
data analysis. To conduct an experiment, it is therefore necessary to define the scope and
limitations of this study. Given the high complexity of the information systems already
in place, it makes sense to start with a small delimited part of it, which, if successful, can
be used as a reference to progress further later on.
The scope of the project is from this data and from the users at BetterDoc, depict what
mental models are present in the matching process. These mental models are what defines
the decisions made in terms of analyzing the relevant medical information from the
patient and determining what attributes a physician would need to give the adequate
treatment.
1.4 Research Question
1. What decision-making patterns can be identified in BetterDoc's matching process?
2. What are the particular perspectives from different individuals during the matching
process?
8
1.5 Research aim
The research aim is to the evidence the information flow supporting decision-making,
identifying patterns and individual difference, during the specialist matching process at
BetterDoc.
1.6 Delimitations
Considering the delimitations for this project, they concern mostly the complexity of the
whole operation of the company and also the technical aspects regarding the architecture
of the production systems. To ensure an adequate conduction of the research, it is
therefore valuable to also clearly state what is not going to be addressed.
First, this study has by no means the intention to propose solutions for a production
system ready to be escalated and used in daily operations. Also, it will consist of a picture
of a given point in time and should not be generalized as the source of truth for the
company’s operations. When it comes to classification schemes, this work will focus on
analyzing the interconnections between different schemes rather than proposing specific
taxonomies. At last, the proposed outcome of this study is not to deliver any kind of
onboarding documentation based on possible patterns identified. The study is limited
itself to provide insights that, if desired, can serve as a basis for iteration and design of
the adequate information systems.
1.7 Definitions
In this section, there are a few definitions very proper to the organization that need to be
clarified for the proper understanding of the study in its entirety.
1.7.1 Borg
Borg is an ERP system developed at the early years of BetterDoc, named after the alien
species from Star Trek. While it started as an office joke, due to the way it evolved trying
to incorporate every single aspect of the business process, the naming ended becoming
definitive. Its architecture is based on the premise that there are two main data artefacts
generated during the process: leads and inquiries. Leads are the records of any contact
9
between a potential patient and the staff at BetterDoc. The goal is to keep data of
motivations for the contact and possibly declinations, as well as effectiveness of
campaigns and communications channels. A lead contains only elementary contact
information. When admitted as a patient, more detailed data is collected and added,
starting an inquiry.
In Borg, a case goes through different phases of the process by moving through different
step, called buckets.
Figure 3 - Inquiry workflow in Borg
The screenshot shows the phases defined during the development of Borg, in this example
Admission, Pre-research and Research. This defines a very strict workflow, establishing
10
requirements and dependencies for the user to be able to move the inquiry from one
bucket to another so the features necessary for each become available.
As the company progressed the service was being iterated and adjusted, while the strict
series of rules imposed initially in Borg incurred in a software hard to maintain and also
to develop further, as it the complexity of dependencies hinders the work of developers
and designers. Currently, the research feature is ultimately dependent on Borg.
1.7.2 PARC
Due do the accumulated complexity of implementing new solutions in Borg, a new ERP
system came to life. Named PARC, it is based in a much different architecture that aims
for flexibility, low maintenance and quick iterations. PARC is based on the containerist
architecture by Konstantin Weiss, that will be further elaborated in the theoretical
framework.
Figure 4 - PARC springboard.
11
As visible in the screenshot, PARC is organized in different tools for a range of tasks. In
PARC, there is no distinction between leads and inquiries, there are only cases, which is
opened at the contact from a patient regardless of how it proceeds. Each tool has their
supporting services, which are completely independent and do not interfere in each other,
allowing them to be rearranged and deployed if and when necessary; being maintained
individually. In that way, the removal or addition of other services does not affect the
performance of the remaining ones.
1.7.3 Case
In the previous definitions, there were a few related terms: leads, inquiries and cases. For
this work the used term will be case. A case is opened the moment the contact with a
patient is recorded, regardless of how far it proceeds in the workflow. It is the artefact
that binds the related data from different tools together.
1.7.4 Expert network
The expert network, also called expert panel, is a group of medics of remarkable
professional expertise and proven record of significant contribution in their areas. Such
medical doctors when identified are invited to join the expert network of BetterDoc,
meaning they make themselves available for providing their recommendations on
unprecedent cases that cannot be solved by the staff alone. The product of their view on
how to proceed with those cases is called “expert recommendation”.
12
2. Theoretical framework
Having BetterDoc’s process as the central object of study in this thesis, there are a series
of concepts relevant in the scientific literature. This chapter will elaborate on socio-
technical systems, categorization and mental models to understand how people
understand and interconnect to systems. In addition, concepts relevant to the context of
software development will be elaborated, such as agile projects, place-making and the
containerist framework.
2.1 Socio-technical systems
The most important concept for the context of this dissertation is the concept of a system.
A system is a group of items that are interdependent or regularly interact with each other,
forming a unified whole that is more than the simple sum of its parts (Arnold & Wade,
2015). While some of these system interdependencies and interactions are actually
physical interconnections (Morville 2014), many are, or can be represented as,
information flows. Information flows from one element part of the system to another, like
water in a tree’s trunk, enabling actors to “achieve a desired future state” and contributing
greatly to the overall resilience of the system (Resmini & Lacerda, 2016).
Socio-technical systems exist on the premise that both social and technical factors cannot
be dissociated. They emerge from the complex integration of humans, machines, and their
environment. Neglecting how these elements interact with each other can lead to failures
or to an inefficient system (Baxter & Sommerville, 2011). This in turn brings along the
need to approach research from multiple points of view that can both accommodate the
quantitative precision of machine-related operation and the qualitative and contextual
performance of humans in the process (Vermaas et al., 2011, p. 73).
In the BetterDoc case, the physical flow in the system is minimal and essentially
irrelevant. It is dominated by information flows, being collected and moved on the process
previously described to determine the outcome of a case.
There is a problem in discussing systems only with words. Words and sentences must, by necessity, come only one at a time in linear, logical order. Systems happen
13
all at once. They are connected not just in one direction, but in many directions simultaneously. To discuss them properly, it is necessary to use a language that shares some of the same properties as the phenomena under discussion. (Morville, 2014 p. 20)
Therefore, we can infer that written language has its limitations when it comes to
describing complex systems and the need to explore them in diverse manners to
consolidate an adequate understanding.
Regarding the development of technologies, Bijker (1995) writes that technology is not
shaped only by societal structures and power relations. It involves also the commitment
of individuals in their emotional and ingenuity aspect, considering how the aspects of the
individuals in question are shaped, are also a product of society. Local cultures form
values, skills and goals. In consequence, to understand technological creativity is
necessary to relate to their historical and sociological stories.
Information architecture is the Information architecture is “the practice of deciding how
to arrange the parts of something to be understandable”(Information Architecture
Institutte, n.d.). Pioneered by American architect Richard Saul Wurman in the 1970s, it
became a mainstream field of practice and research in the 1990s, with the introduction of
a core body of knowledge and expertise borrowed from library and information science
(Rosenfeld et al, 2015).
As of today, information architecture is “steadily growing into a channel- or medium-
aspecific multidisciplinary framing, with contributions coming in from architecture,
urban planning, design and systems thinking” (Resmini, 2014). The practice has been
heavily reshaped into a strategic craft dealing with sense-making and place-making in
blended environments (Benyon & Resmini, 2017). Given the eminently digital, bounded
nature of the BetterDoc case discussed in this dissertation, methods and tools used to
explore the problem space comprise both strategic-level or system-level insights
introduced in “contemporary information architecture” in the past few years, and more
traditional tools and methods of “classical information architecture” such as labeling,
categorization systems, and controlled vocabularies (Resmini & Luca, 2012).
14
2.2 Categorization and controlled vocabularies
In order to visualize the network of relationships between socio-technical systems,
metadata and controlled vocabularies are a fascinating lens to look through. In many
projects related to metadata, the controlled vocabulary is an essential element to hold the
between socio-technical systems, together. The structure of a thesaurus in the back-end,
can provide the user a more satisfying and seamless experience during the usage of the
product (Rosenfeld et al., 2015).
Establishing a controlled vocabulary can happen in different ways. When doing it more
vaguely, it is simply a subset of natural language. It can also be a list of equivalent terms
that form a synonym ring, or an authority file that lists the preferred term. Although,
putting a classification scheme together happens when the relationship between those
terms are defined, (Rosenfeld et al., 2015). The described relationship is shown in the
figure below.
Figure 5 - Relationship between terms within a controlled vocabulary (Rosenfeld et al.,
2015)
15
The diagram shows how the terms constituting a controlled vocabulary relate to each
other. The central element is a preferred term, which means that within a range of options
that term is selected in detriment of others. These others, which were equal options in
terms of meaning, they are kept as variant terms in an equivalence relationship, working
as synonyms. In a hierarchical relationship, which shows the relationship between terms
that can be contained within each other although they do not mean the same. At last, there
are terms that have a different meaning although they are related to the preferred term,
establishing an associative relationship.
Strictly defined, an authority file is a list of preferred terms or acceptable values. It does
not include variants or synonyms. Authority files have traditionally been used largely by
libraries and government agencies to define the proper names for a set of entities within
a limited domain. In practice, authority files are commonly inclusive of both preferred
and variant terms. In other words, authority files are synonym rings in which one term
has been defined as the preferred term or acceptable value.(Rosenfeld et al., 2015)
Rosenfeld et al. (2015) synthetized the core terminology to include the following:
• Preferred Term (PT): Also known as the accepted term, acceptable value, subject
heading, or descriptor. All relationships are defined with respect to the Preferred
Term.
• Variant Term (VT): Also known as entry terms or non-preferred terms, Variant
Terms have been defined as equivalent to or loosely synonymous with the
Preferred Term.
• Broader Term (BT): The Broader Term is the parent of the Preferred Term. It’s
one level higher in the hierarchy.
• Narrower Term (NT): A Narrower Term is a child of the Preferred Term. It’s one
level lower in the hierarchy.
• Related Term (RT): The Related Term is connected to the Preferred Term through
the associative relationship. The relationship is often articulated through use of
See Also. For example, Tylenol See Also Headache.
16
• Use (U): Traditional thesauri often employ the following syntax as a tool for
indexers and users: Variant Term Use Preferred Term. For example, Tilenol Use
Tylenol. Many people are more familiar with See, as in Tilenol See Tylenol.
• Used For (UF): This indicates the reciprocal relationship of Preferred Term UF
Variant Term(s). It’s used to show the full list of variants on the Preferred Term’s
record. For example, Tylenol UF Tilenol.
• Scope Note (SN): The Scope Note is essentially a specific type of definition of
the Preferred Term, used to deliberately restrict the meaning of that term in order
to rule out ambiguity as much as possible.
In addition to controlled vocabularies, another important topic for the case is
categorization. Morville (2014) elaborates on categories as the root of organizing
information, although they should not be built before it is understood how they connect
to the systems as a whole. Having infinite ways to organize information, it is vital to
understand objectives beforehand as the context is the key to classification. The emphasis
in this message is the need to clearly have objectives when elaborating a categorization,
since there is not a single correct way of doing it. Therefore, the categorization needs to
be elaborated to fulfill an established objective. In the context of this case study, it is
noticeable that this goal was not established and the categorization by means of tagging
was fruit of circumstantial use cases.
Tagging flips the model in comparison to taxonomies. It consists of writing and attaching
labels to objects, aiming at describing it with different keywords. There is no hierarchy
between the objects in the scheme, as the creators of such categorization system describe
objects freely as the tags grow in number. Tagging is described as suitable for
idiosyncratic, bottom-up and object-centered use cases. The value of tagging resides in
the strange connections and descriptions that appear once a user finds an object, in
detriment to findability (Morville, 2014).
It clearly describes the use of tags at BetterDoc's process, being created on the go as the
users see the need for it in a rather unstructured way. In contrast, Morville emphasizes
the advantage of taxonomies:
17
Due to the weakness of words, it is hard for any label at the top of any taxonomy to stand on its own. We need the root categories of main navigation, so users understand the full scope, and so there is a place for all content, now and in the future. Breadth lets the system scale over time. (Morville, 2014, p. 47)
The comparison is further illustrated in the following diagram:
Figure 6 - Taxonomies, facets and tags as categorization schemes. (Morville, 2014)
The diagram clarifies how the relationship between types of categorization. In a
taxonomy, there is the establishment of a hierarchical relation, meaning a category can
branch out to sub-categories and so on. When it comes to facets, they share the hierarchy
with the taxonomy but there are several ways for rearranging which are independent from
one another. As the example illustrates, a product may be categorized by price or by its
color. At last, tags represent the loosest of the schemes. Tags are completely independent
from one another and their relationship is only to the object in question, not between other
tags.
What values are implicit in this scheme? What is the intent and impact? Who does it help? Who gets hurt? What are its alternatives? And why is this the one we use? Why does it endure? We must subject all taxonomies to such questions because their imprint belies their impact. (Morville, 2014, p. 42)
18
Tags illustrates the current situation, where a labeling system evolved over time according
to immediate needs without a perspective for its future sustainability. On the other hand,
a taxonomy defines the current goal where to shift this system to a categorization in which
the elements have hierarchies and logical relations between each other, using that in favor
to reduce complexity while solving a case.
When building a categorization system, there are two principles that need to be considered
according to Rosch (2013):
• The system should provide maximum information with the least effort.
• The system assimilates the perception of the world as structured information
instead of unpredictable attributes.
Given the applications of categorization, it is worth noting that a categorization system
can make use of one or more different schemes. Different combinations allow the objects
to be seen from different perspectives, enabling more options of navigation.
The combination of tags and facets allows for better management of both these axes: from the vertical or paradigmatic point of view, when a user is going to associate a keyword to a facet (in order to tag a resource), the system suggests similar tags or hierarchy of tags pertaining to the same facet; from the horizontal or syntagmatic point of view, at the same time, the system will allow the user to see all the other tags belonging to the same facet(s). (Quintarelli et al., 2006, p. 3)
Ultimately, the application of categorization systems often comes to establishing
ontologies. In the context of information science, ontologies establish a structured
representation of naming and categorization. They have characteristics from both faceted
taxonomies and thesauri, but there is the application of strict semantic relationships
between terms and attributes with the objective of structuring the information in a
machine-readable form. The characteristics of thesauri allow a structure suitable for
cataloging and retrieval (Harpring, 2013).
19
2.2.1 ICD codes
Commonly referred to as ICD codes, these codes are a controlled vocabulary used in the
organization in relation to categorize diagnostics. The ICD, standing for International
Classification of Diseases, is taxonomy is maintained by World Health Organization and
it standardizes codes for specifying diseases (Baumel et al., 2017). The purpose of the
ICD code is allowing for international tracking and comparison of health data,
establishing a common vocabulary that can be shared. Currently the ICD is in its 10th
revision, named ICD-10. The ICD-11 is already public and set to be take place as the
standard version in January 2022 (World Health Organization, 2019).
2.2.2 OPS codes
OPS codes in their turn, are a German standard for classifying medical procedures. OPS
stand for “Operationen und Prozedurenschlüssel”, translated as Operation and Procedure
Classification System. It is a taxonomy updated yearly and legally bound to be used as
the, national standard in German hospitals. Its main purpose is assuring the accountability
of medical procedures for reimbursement purposes (Graubner, 2007).
2.3 Mental models
In addition to categorization, mental models also contribute to the understanding of a
socio-technical system. Categorization enables individuals to establish a common
language for communication, a shared understanding of an information system. The
mental models facilitate the understanding of how people interact with such systems,
representing, for example, their motivations, intentions and tasks. Young (2008, p. 3)
states that “Mental models give you a deep understanding of people’s motivations and
thought-processes, along with the emotional and philosophical landscape in which they
are operating.” To create a mental model, it is necessary to talk to people about what they
are doing and look for patterns, gathering ethnographic data.
While interacting with the environment, with other people and also with technology,
people form mental models. These models are particular to each individual and depict
20
how they think of themselves and the interactions between them and all the things they
interact with in the given environment. It is noted that mental models are naturally
evolving models, and therefore impossible to be considered finalized (Gentner & Stevens,
2014).
By categorizing and organizing those patterns, it is possible to construct a diagram that
illustrates how tasks are performed and the supporting tools (Young, 2008). The
illustration of a mental model follows as such:
Figure 7 - Example of mental model diagram (Young, 2008, p. 11)
The upper part of the diagram is a representation of how a particular audience behaves,
bringing to light their motivations. The bottom part depicts various ways of supporting
those behaviors. In this case, the example is about how to get to work in the morning. The
columns being the tasks towards this goal while the boxes underneath represent the
supporting tools. When identifying tasks to draw the model, they are considered atomic
tasks. By grouping similar atomic tasks, they are put in top of each other building task
towers (Young, 2008).
21
2.4 Place-making
One of the approaches introduced by contemporary information architecture to help users
make sense of complex socio-technical systems relies on place-making, defined as:
The capability of a pervasive information architecture model to help users reduce disorientation, build a sense of place, and increase legibility and way-finding across digital, physical, and cross-channel environments. (Resmini & Rosati, 2011)
When it comes to the usage of digital products, people experience such products primarily
through devices and their screens, such as phones, tablets and computers. Nonetheless,
these products and services configure places that people inhabit in order to accomplish
their goals, whatever they are (Resmini & Lacerda, 2016). Therefore, users need to be
able to understand and make sense of the information environments they enter, as the
arrangement is vital for them to accomplish the tasks they want to accomplish, in a way
that is no different from the way we experience physical environments. As such, to be
able to benefit the most from these environments, users need to form mental models of
how to navigate them (Arango, 2017).
By understanding how the users think and expect the socio-technical system to work, it
is possible to not only develop features accordingly but also dedicated places. To design
digital places in which the users are pleased to inhabit, it is necessary to understand their
mental modes and how the given places should be arranged.
2.5 Containerist
Containerist is an architecture for the web that defines a page as a stack of autonomous
containers, that encloses content. This architecture enables content and features to be
rearranged in different places, remaining the same in their origin but being prioritized to
suit better the user’s intent in different contexts. This way, dependencies are avoided
reducing the maintenance of those places, while also facilitating the creation of a
multitude of places that are specific to particular tasks. Weiss (2012) defines 6 principles:
• Stacking: the possibility to stack containers on top of each other as desired.
• Position: the possibility of changing position within different stacks.
22
• Repeat: a stack may contain more than one container of the same type.
• Condition: containers may have conditions that determine if it’s shown or
not.
• Reuse: the same container can appear in different stacks, even in different
positions.
• Federation: the container can come from a different site than the one it is
being displayed, enabling it to be positioned in any place.
If you think of placemaking, what is you as an architect arranging a place populate with
tools and artifacts in order to have a person solve a task or a set of tasks. In the physical
world, you need to produce the amount of the desired tools. In digital environments it is
possible to reuse the same tool in different contexts, without having to duplicate it.
Navigating in different places incurs losing the context. The containerization simplifies
maintenance and data handling (Weiss, 2020).
2.6 Agile software development
When it comes to the development of information systems, the object of this study
contains a paradigm shift from the waterfall to the agile approach. The agile manifesto
emphasizes the characteristics from this shift. Agile favors interactions over processes
and responding to change over following a plan, focusing on delivering valuable software
in shorter periods of time (Beck et al., 2013). As a consequence of the way agile projects
are conducted, the product is more flexible as it is built with the perspective of constantly
iterating and evolving along with the people who use it.
23
3. Methods
To achieve the purpose of this thesis, research methods need to be applied. This chapter
elaborates on the research philosophy, approach, data collection methods and analysis
defined to pursue the answer for the research questions. In addition, the chapter also
brings ethical considerations to the research and limitations.
3.1 Research settings
The first definition needed to conduct the research; is the research philosophy it follows.
That is important because there is not a single correct view of the world, therefore
different perspectives can lead to different results and that needs to be accounted for.
The idea that there are different views of the world, and the processes that operate within it, is part of what is known as philosophy. Philosophy is concerned with views about how the world works and, as an academic subject, focuses, primarily, on reality, knowledge and existence. (Danson et al., 2018, p. 49)
When it comes to research philosophy, there are four main approaches: positivist research
philosophy, interpretivist research philosophy, pragmatist research philosophy, and
realistic research philosophy. The positivist research philosophy claims that it is possible
to understand the social world in an objective way. Researchers who uses this approach
must take the role of an objective analyst and be able to dissociate from personal values,
in order to be able to work independently (Žukauskas et al., 2018).
Interpretivism advocates that it is necessary for the researcher to understand differences between humans in our role as social actors. This emphasizes the difference between conducting research among people rather than objects such as trucks and computers. […] In the same way we interpret our everyday social roles in accordance with the meaning we give to these roles. In addition, we interpret the social roles of others in accordance with our own set of meanings. (Saunders et al., 2009, p. 116)
The central idea of interpretivism is to work with the subjective meaning that can be found
in social phenomena. It is possible to use them as building blocks in theorizing by
acknowledging their existence, reconstructing the, understanding them and even avoiding
to distort them (Goldkhul, 2012).
24
Although, there are a few remarks towards an interpretivist approach. As the data is highly
contextual, it is difficult to draw generalizations as they are very particular to the sample.
Also, relying on the researcher’s interpretation to draw conclusions it is hard to determine
validity criteria, as other researchers or reviewers would possibly have different
conclusions. Nevertheless, it can provide rich evidence and insights towards the
understanding of behavior (Scotland, 2012).
For this study, the favored perspective is the interpretivism despite the explained
limitations. That is due to the characteristic of having the object of study as a socio-
technical system, which is constantly evolving based on the interplay between the actors
of this system. Having a clear goal towards understanding the behavior of a particular
group, the approach is considered the most fitting.
3.2 Research approach
When it comes to the approach, there are mostly two ways as described by literature.
Deductive or inductive. In either process the importance of theory is fundamentally
important. The inductive process aims at creating theory from the studied situation. Even
if there is already a theory or knowledge prior to it, it is seen as beneficial to the
argumentation to approach the context without pre-conceptions in place. The deductive
process, on the other hand, has an expectation of what is more likely to occur (Johnston,
2014).
When conducted with an inductive approach, the research tends to be more concerned
with a particular context observed, in which the events take place. In this scenario, the
definition of a small sample that can be analyzed in depth can be more appropriate than
large samples pertinent to a deductive approach (Saunders et al., 2009).
In conclusion, it means that the deductive approach consists in having a hypothesis based
on an already existing theory, to be tested and confirmed or rejected according to the
results of the research. That defines a top-down approach. On the other hand, the inductive
research requires no hypothesis to be started. It consists in a bottom-up, using the results
from the data to formulate a theory.
25
In this research the approach will be inductive, as its goal is to find patterns which can be
formulated into a theory where there is currently none.
3.3 Methods of data collection
When it comes to collecting data, there are two main perspectives towards the scientific
method. One is largely objective, seeing only one or a limited number of universal truths.
Mostly everything can be quantified or measured, translating observations as numbers.
The other option is seeing the world as largely subjective and open to different
possibilities of interpretation. This second perspective emphasizes that quantitative
measuring is not always possible or not even desirable, as words can evidence details
more accurately (Danson et al., 2018).
Therefore, the use of texts as empirical material in detriment of numbers characterizes the
qualitative research. It relies on the notion of social construction of reality in the object
of study, having an interest on the perspective of participants, their everyday practices
and knowledge (Flick, 2007).
From that context it becomes clear that the characteristic of this study is qualitative, given
the vital role of the perspective of each of the actors in seeking understanding. Following
on that, the data collection method for the case leads to interviews.
To collect data, one of the possible methods is the use of interviews. They can be
unstructured or semi-structured interview. In an unstructured or semi-structured interview
there is an incomplete script. The researcher may have prepared some questions
beforehand, but there is a need for improvisation. The interviewer is the researcher or is
one of a team. Myers & Newman (2007) state that in either case (semi-structured or
unstructured interview), preparing the script should involve at a minimum:
• Preparing the opening – introducing yourself etc.
• Preparing the introduction – explaining the purpose of the interview.
• Preparing the key questions.
26
• Preparing the close – if needed, asking permission to follow-up, or asking who
else the interviewee recommends might be interviewed. This is a technique known
as snowballing, where interviewing one person leads to another which in turn
leads to another. Snowballing helps the researcher to obtain a critical mass of
interview data.
That said, the methodology for this study is defined as a series of semi-structured
interview. That choice comes from the benefit of having common points to guide the
interview, although leaving space to go deeper in the particularities of each case.
The sample was defined as 7 people that work at BetterDoc, having different positions on
the company and also different levels of experience. These people have roles in research,
patient care and product development. The experience at the company ranges from 6
months to 3 years.
3.4 Data analysis
Considering the define data collection method, the product of qualitative interview is
usually a substantial amount of text and therefore the analysis happens as textual analysis.
Textual analysis enables researchers to gathering information and understand how
humans make sense of the world. By seeing a variety of ways to possibility interpret
reality, it is possible to understand our own sense-making practices and how we are
inserted in our own culture (McKee, 2011).
When it comes to a more methodical approach, the unit of analysis can be identified as
words, sentences, grammatical structures, tenses, clauses, ratios, or even themes (Prior,
2014).
Thematic analysis is an approach that consists in scanning the concerning datasets to find
repeated patterns of meaning. There’s not fixed structured for the form and product of
such analysis, and therefore the questions to be answered during the study are vital both
before and during the analysis. When defining themes, they should capture what is
27
important in the data towards answering the research question and that are able to
demonstrate patterns and meaning within the data (Braun & Clarke, 2006).
In addition to that, the chosen mental models framework elaborates on the analysis of the
transcripts as “combing”. This combing consists in analyzing the transcripts repeated
times scanning for indications of tasks, sometimes explicit and sometimes not. Identifying
those tasks, it is possible to group them and create the diagram of the model (Young,
2008).
Considering the points mentioned above, the analysis for this study was defined as a
thematic analysis to identify the recurrent themes important for the decision-making, as
related to the research questions. The reason is that themes focus on a higher level
perspective of patterns that can be then explored deeper, revealing connections that could
be overlooked when using smaller units of analysis. Within the themes, sub-categories
will be labeled to illustrate in a more detailed manner the sense-making of the participants.
Simultaneously, tasks will be identified to serve as the basis for the mental model
diagram.
3.5 Credibility and generalizability
The usage of qualitative methods imply that that statistical generalization is not going to
be evidenced in the results. It is helpful therefore to understand other types of
generalization. Internal generalizability refers to the capability of the study to generalize
the conclusion within the defined group or set in which the study occurred. On the other
hand, external generalizability refers to the conclusions which can be generalized beyond
that defined group (Flick, 2007).
Considering the qualitative characteristic of this study and the limited, very specified
sample within a single organization, the generalizability is limited to be internal. Any
generalization is limited to professionals working in the same environment and having
contact with the same workflow.
28
3.6 Ethics
In the observation of ethics, there are two main points pertinent to this study. The first
one is the conduction of interviews. The second is the confidentiality of sensitive health
and personal records. Regarding ethical standards in research, Myers & Newman (2007)
list the following points:
1. (A) Permissions – obtaining ethics approval from the appropriate ethics
committees, obtaining permission from interviewees (and if appropriate, their
manager).
2. (B) Respect – treating people with respect (before, during, and after the
interview), respecting their time, respecting their position within the organization,
respecting their knowledge.
3. (C) Fulfilling commitments to individuals and organizations. This may involve
1. (a) Keeping confidences, keeping transcripts/records/and the technology
confidential and secure.
2. (b) Presenting findings and results – it may be advisable sometimes to
provide early feedback to subjects and organizations and to check with
them about factual matters if needed.
With that in mind, the data collection for this research has a concern specifically with the
confidentially. The interviewees are always asked for permission to have the given
answers recorded, as well as their manager in case it is necessary. Nonetheless, the
transcripts will de be kept confidential as they might contain information sensitive to the
business, although summaries will be appended after being presented to designated
reviewers in the organization. When it comes to database records, by no means they will
be exported or accessed without complying to the company’s confidentiality policies in
the conduction of this study.
29
4. Results
The results presented in this are regarding the semi-structured interviews performed with
employees of BetterDoc, in which 7 people were interviewed. These people vary in
experience at the organization, from 6 months to around 3 years. They also belong to
different roles, namely: 4 researchers, 2 product developers and 1 patient care
coordinator. The most relevant findings were categorized and grouped to be discussed in
this chapter. Regarding the collected themes, their relation to the underlying topics are
presented in the table below. Furthermore, each theme and topic are described as well as
exemplified by supporting quotes from the interviews.
Considering the frequency and relevance of common topics described by the participants,
they were listed on a table. Also, the topics that play a role on each other were categorized
as related to a particular theme to contextualize how the topics play a role on each other
and appear along the decision process.
30
Table 1 - Overview of themes and topics extracted from interviews
Theme Topic
Communication
Process
Intent
Empathy
Health
Condition
Treatment
Medicine
Biometrics
Documentation
Insurance
Medics
Specialization
Workplace
Conflict of interest
Experts
Feedback
Tools
Database
Questionnaire
Borg
PARC
Phone calls
External sources
4.1 Communication
The theme of communication was very present englobing the touchpoints between the
patient and BetterDoc. Communication also incurs in the channels that are used to
exchange information between user and organization.
31
4.1.1 Process
The business process was the only topic mentioned outside the context of contacting the
patient. It was mentioned the existence of so called “Leitfäden”, which translated freely
from German can mean guidelines. Those are written documents that describe the process
and how it should be done, although the volatility of a start-up environment proved along
the years that it is a challenge to keep those up to date according to the evolving
workflows. In addition, it is also mentioned as a burden when onboarding new colleagues
in the teams. The workflows are described as hard to grasp and time consuming to explain
to teach someone.
On another aspect, communication about the process is essential when receiving contacts
of new patients, to clarify what is the company’s offering and what they expect. The
potential patients must be aware of the time it takes to run a case and the kind of results
they can expect.
“So, we have for each process which is important for us, especially for the new people, such Leitfaden in which all the information is written down.”(Participant-F, 2020)
“And what we have to do is we have to explain the whole process which is a work by itself, it's a lot of work so you have to explain the process you have to explain we use different systems and we have to use this and then that.” (Participant-A, 2020)
4.1.2 Intent
This topic refers to the intent of the patient when communicating with BetterDoc. It is
understood by the interviewees that an alignment is needed between what the organization
can offer as a service and what they would like to achieve when contacting the company.
The reason is that for the research to proceed, the intent stated by the patient during
admission must be feasible for the research. For example, the intent is usually captured
within options such as: conservative therapy; surgery; second opinion. Although, there
might be other possibilities. It was mentioned the patient may be concerned about finding
a doctor who can speak their language of preference, in case they are not fluent in German.
That shifts the focus from the health condition to being able to provide the patient with
32
adequate medical understanding of their situation, even if not severe. Another aspect of
the intent is the distance the patient is willing to go to attend the recommendation.
Both those aspects are taken into consideration, meaning for example that a surgeon won’t
be chosen for a patient who stated his intent as a conservative treatment. In case either is
not feasible, the patient needs to be informed and the divergence clarified in other to
proceed. That can happen in cases such as the condition is more severe than initially
thought and a surgery is needed even though the intent was a conservative therapy; or if
the case is that the patient leaves in a small urban area and the suitable specialists are not
within the range desired at first.
“And then he [the patient] can tell us his wishes for our research again, so if he's looking for another treatment, another surgery and other conservative therapy.” (Participant-D, 2020)
“[We have to consider] if the PC team has to call the patient again because we cannot do the things the things they wished us to.” (Participant-B, 2020)
“For me one of the most important things is what the patient is looking for, why he has contacted better doc and what his wishes for our research are. Then I try to see if the research makes sense with what he has as a health issue, if they are connected with each other or if it's completely random or impossible to fulfill.” (Participant-A, 2020)
“[…] for this patient the most important thing is that the doctors speak the language he or she is speaking maybe, if it's possible. So, it's not the health problem itself but it just the understanding which is the main problem of this patient.” (Participant-B, 2020)
4.1.3 Empathy
Empathy is related to the ability of understanding or sharing the feelings of the patient.
In the context of this work, it means being able to understand the struggle and concern
they have with their medical condition, and dedicating efforts to find the best possible
solution for it. It indicates a belief, specially from the researchers, that the positive
outcome of the recommendations is connected directly to the empathy, being able to earn
their trust to discuss sensitive topics comfortably and confiding information over the
phone or e-mails. It is also mentioned that sometimes that the researcher needs to be able
33
to “read between the lines” of patients’ communication due to the fact that sometimes
they lack the medical understanding to express their concern in technical terms, and
therefore what they intent. It is seen, therefore, that the ability to understand the affliction
and the situation from the patients’ perspective is fundamental to reach the best results.
“My way of researching is: I always imagine the patient being one of my family members.” (Participant-A, 2020)
“Being a researcher is being a person who looks after people and has that kind of feeling that you have to look after people, and you want to help.”(Participant-A, 2020)
“You have to learn to come to see behind what he's writing and what he might wish for and, it's not really explainable but the more you read questionnaires and the more you get into contact with the patient the more you can see what the patient is really wishing for.” (Participant-A, 2020)
4.2 Health
Health as a theme comes as the topics concerning the patients’ condition and situation
when being admitted. It englobes the diagnosis, treatment, medicines, biometrics,
documentation and health insurance.
4.2.1 Condition
The health condition of the patient is essential to understand what problem is afflicting
the person and therefore what would be the adequate solution for it. That naturally incurs
the diagnosis as vital, as the disease itself is the main determinant of the treatment that
should be performed. Although it is important to consider secondary conditions to the
current diagnosis, for example heart diseases or psychologic conditions that could affect
how treatment procedures are performed.
“I think maybe [the most important] is what the current diagnosis is, and what else they have so, side issues, side medical issues. I don't know, someone has an issue with their knee but also has a heart problem.” (Participant-C, 2020)
“What is his current status, is he a worker, is he jobless or something like that. Is he sick right now and can’t go to work? […] and there's another question how bad his
34
health problem is affecting him on his general life, his daily life.” (Participant-D, 2020)
4.2.2 Treatment
The treatment topics means the kind of treatments the patient has already performed in
the past. That has shown to be decisive because it is related to the current condition of the
patient, as it might be consequence of a procedure that was not performed adequately, or
it was not really necessary at all. It also indicates the likelihood procedures can be
successful, as it is less likely treatments will be recommended again if the patient already
undergone similar procedures and has not achieved the expected results.
“I’m looking what treatment did you already get. If the patient already got a lot of different conservative treatments, physiotherapy, some injections or acupuncture, I don't know, something like that. And he has pain over a lot of many years then maybe it's better to see a real specialist, not only conservative treatment because you already did a lot of conservative treatment.”(Participant-D, 2020)
“And we are also looking for did the patient had surgeries in the past. Because sometimes the problems are more related to the surgery. Really a lot of back patients already had a five or more, only two or one surgery. And then they often have problems because of the surgery, and it was not useful, or not good to do the surgery.” (Participant-E, 2020)
4.2.3 Medicines
Medicines show as a possible indicator of the severity of the case. That is because some
medicines are restricted and only recommended for patients who have a particular
situation, becoming a way to derive information of the actual health condition or
secondary concerns even when not mentioned directly by the patient. For example,
patients that describe taking strong pain medicine or daily medicaments for high blood
pressure, they provide additional facts that support the assessment of their conditions.
“And the patient takes a lot of medicine, pain medicine for example. Then we say ok it's better for the patient that we look for a physician with the profile he knows a lot of pain and treats only pain patients, for example.” (Participant-E, 2020)
“And we have other we have different facts, which tell us if it is a serious problem or it's not that serious. And that is for example how many surgeries did the patient
35
have in the past, or how many pain medicines that you take or whatever.” (Participant-E, 2020)
4.2.4 Biometrics
Biometrics are not described often as a decisive factor, but they can provide important
information. That is the case of assessing the BMI of the patient, what is only possible
when knowing also the height and weight of a person. The age is also described as an
influent variable. Both of the mentioned examples indicate potential risk factors that
should be considered in the decision of a treatment or facility. For example, a high BMI
indicate that risks associated to obesity must be accounted for; just as an advanced age
brings the warning for some procedures that have increased risk when performed on
elders.
4.2.5 Documentation
The diagnosis is not always possible to be done accurately only with the patient’s own
description to the symptoms. In those cases, it is asked that the patient sends possible
documentation that can clarify technicalities of the condition. Those documents can be
such as pictures and reports of previous examinations, such as x-rays, and also
recommendations for treatments. It is described that in case of most orthopedic conditions
they are not necessary as the symptoms tend to be clear enough, with the exception of
conditions affecting the back. For most of non-orthopedic cases they are requested,
particularly with severe diseases such as cancer.
“I think most of the cases 80% you can do without the medical findings in orthopedics, but it is also depending on not only on if it's a complex case or not.” (Participant-B, 2020)
“For many easy medical problems like - I do have knee pains - we did not request any of those documents but if it was something more complicated usually, it's for - I do have a tumor - or - I do have great back injury-.” (Participant-G, 2020)
36
4.2.6 Insurance
The health insurance of the patients has a role in telling which professionals are available
to them without the incurrence of extra costs. Although the statutory health insurance in
Germany is very comprehensive, there are clinics and medics who only admit privately
insured patients. Therefore, these options must be avoided in case of sending
recommendations to a patient who has a statutory insurance.
4.3 Medical doctors
Medics are the physicians in BetterDoc’s database that serves as the source of
recommendations and the core of the value offered to the patients. A medic entry in the
database has several kinds of data related, which help clarify whether that professional is
a good fit or not for the desired outcome of a case. This information can be regarding the
specialization, workplace, conflicts of interest, expertise and feedback.
4.3.1 Specialization
The specialization is the first factor to come up regarding the medics. It defines the focus
of the medical expertise of the professional and it is directly related to the diseased
suffered by a patient. It is worth mentioning that interviewees emphasized that researchers
must know as much as possible about the range of specializations available to make a
recommendation, as medical specializations can get extremely granular. Naturally, the
more specialized the target profile is, the less numbers are available and that needs to be
taken into consideration. So, it is possible to say that it is always a comparison between
how severe the condition is that it needs to be treated by a highly specialized professional,
in relation to the convenience of the patient to visit such professionals and undergo the
necessary treatment.
“Another point is you have to know that there are kind of doctors who have multiple specializations and there's some kind of doctors who are very very very very specialized in one little tiny bit of the body.” (Participant-A, 2020)
37
“And then I will look into the into what the doctor has in his skills, like if he is really specialized into it, if he just an all-round specialist or if he really is into that kind of disease that the patient has.” (Participant-A, 2020)
“If you're looking for the second opinion or looking for a surgeon, is the most important point when we are looking for when looking for the profile, the physician profile.” (Participant-E, 2020)
4.3.2 Workplace
The workplace of a medic adds information to the case when it is necessary to decide the
conditions of the adequate treatment to be performed. There are either medical practices
or clinics (also englobing hospitals). The main difference is that while medical practices
can be very suitable for second opinions or conservative treatments, clinics are favored
to more intensive procedures such as surgeries. The reason for that is the facilities tend to
be more comprehensive, also having not only a few professionals, but bigger teams with
several specializations that allow the handling of potential complications, stationary
treatment and intensive care units.
It is also mentioned that clinics can commit to the expertise of the professionals working
there. In some cases, the medic as an individual may not have an individual record that
brilliant but works in a clinic that has a lot of prestige as an organization.
“[…] if you want to recommend a clinic for surgery and the patient is quite ill with his heart for example. It would be important then that in the clinic there is cardiology.”(Participant-F, 2020)
“Criteria can be objective, like […] or the clinic the specialist is working has high case numbers, on the clinic has some special certificate.” (Participant-B, 2020)
4.3.3 Conflict of interest
Conflicts of interest are described as when it is seen that the professional potentially has
individual interests in proceeding with a particular treatment, most often a surgery. The
most common occurrence for that is the case of having a consultation with a surgeon that
in face of the potential earnings of performing it, recommends the procedure even if it
could be avoided. One of the aims of the researchers is to provide a professional who is
38
free of such conflicts and can provide a rational assessment purely regarding the health
conditions of the patient.
“[…] for a second opinion it's not good to recommend a surgeon. Because surgeons want to make money and of course they will recommend a surgery.” (Participant-D, 2020)
“we are looking for a physician doesn’t have any conflicts of interest. I don't know if this is the right word. And then it's better that the physician does not make surgeries. Because then the chance that he says you need a surgery is higher than the physician who doesn't do any surgeries at all.” (Participant-E, 2020)
4.3.4 Expertise
Expertise concerns the knowledge and experience of a given medical in dealing with a
particular disease or set of diseases. Since that is not always easy to measure and compare,
the interviewees described that they look for evidences that can show a professional’s
expertise in a certain topic. Those evidences can be, for example, publishing scientific
work and being a member of the expert panel, or recommended by it, and also
professional associations.
“Maybe we have done and asked our own experts before, and one of the doctors one of the physicians is recommended for this specific diagnosis.” (Participant-F, 2020)
“If I really need an expert in a specific field, and it's much more important than he is an expert than that he has an NPS.” (Participant-F, 2020)
4.3.5 Feedback
The feedback consists the recorded opinion of previous patients that visited a medic.
During the follow-up phase, they are asked to rate and comment on the medic that was
recommended to them by BetterDoc. This feedback is also stored in the database as is
shown to the researchers in the medic search, so they are able to compare the
professionals.
“I will look for a patient feedback if the doctor has good patient feedback and I will be more inclined to recommend them again.” (Participant-A, 2020)
39
4.4 Tools
The last of the themes introduced are regarding tools. They are mostly information tools
that steer how the tasks are performed in the workflow. The most discussed tools were
database, questionnaires, Borg, PARC and external sources.
4.4.1 Database
The database is the most important tool, keeping all the records that fuel the business and
enables recommendations to be done in a timely manner. Although the database can be
seen as a tool itself as a storage, it needs the support of other tools to be reached by the
users, in this case the staff at BetterDoc.
“I think it was just also depending on what you research because you know orthopedic is more in the database and I think multi (non-orthopedic) research is more in free search…” (Participant-B, 2020)
“I can have our database in which I can already have look - ok, these physicians we might have recommended before already quite often for this kind of purpose.” (Participant-F, 2020)
4.4.2 Questionnaire
The questionnaire is a defined format for several kinds of medical fields. The staff decides
which questionnaire is the appropriate one for the case during the admission and it is the
main tool to gather the supporting information that enables a case to be solved. The
questionnaire has several sections that follow a standard format and they serve the
purpose of collecting information on the patients’ health situation, in the form already
described previously in this chapter.
The questionnaire is loaded automatically into Borg and the researcher reads it,
composing a summary that emphasizes the main conclusions of it and serves as the main
input for the research to proceed.
“All the questions that we ask in the questionnaire and the patient wish and our intelligence, knowledge, about what's necessarily comes together into the research profile.” (Participant-A, 2020)
40
“First thing is I make a summary of it, so we have an overview, so I'll summarize the health questionnaire and medical findings.” (Participant-B, 2020)
4.4.3 Borg
In case of Borg, the most mentions were the strict workflow incurred for the lengthy set
of rules and dependencies within the information system. It was mentioned that it fitted
well for the originally defined workflow at the moment it was developed, although when
the workflow evolved in the daily operations it incurred in several restrictions and
additional work to the adaptation of the teams.
“That makes it easier for us. To be honest we had to work for Borg. We had to adjust to Borg and now PARC is adjusting to us.” (Participant-D, 2020)
4.4.4 PARC
Although the case process still can only be solved from beginning to end with the use of
Borg, it is notable the support towards the replacement with PARC. The new ERP already
proved in the daily operations that the more flexible architecture can provide significant
results in terms of adapting and evolving together with the organization, keeping a closer
pace.
“I think the transition and the shift to PARC is a really good thing. It would be better if it would be quicker obviously, but I really think that with the new system we can make our service more individualized for the patient.” (Participant-A, 2020)
“In this case I would also look for case numbers in PARC.” (Participant-F, 2020)
4.4.5 Phone calls
Phone calls are the main communication channel between the staff and the patient. That
also incurs in it being the main source of all the information, although recently initiatives
to enable other channels were also developed in the organization. As of today, contact by
phone is the most intensive and that incurs in the need of staff to answer all those calls,
limiting the capacity and also working hours in which is possible to start a case at
BetterDoc. In recent years communication was also steered to be done by e-mail as often
as possible, reducing the amount of calls needed to solve a case.
41
“I'm creating inquiries for the patient, together with the patients on the phone. I ask them about the anamnesis and try to find out what kind of disease they have, and I document everything they say and so I create this inquiry.”
“It is possible to send the patient some results and then the patient gets a phone call from one of our doctors or medical researchers explaining the results.” (Participant-A, 2020)
“Not everybody needs to be called and not everybody wants to be called and I think with the new system we can individualize it more.” (Participant-A, 2020)
4.4.6 External sources
When research needs information missing in the database, it is time to reach out for
external sources. These are defined according to convenience, usually the internet being
the first choice of source. When needing additional information regarding the expertise
of doctors or clinics, search engines can point to pages that elaborate on publications,
certificates and professional associations. The findings are then added to the database to
expand it and keep the records for future use.
Other sources can include external public databases that can be used to complement the
internal one. It is the case of review portals such as Jameda, a German website that allows
users to rate health professionals all over the country. It is not seen as the most reliable
source of information as the curation of it cannot be assessed. Another important source
is the “Weisseliste”, or White List in free translation. It is a web portal that displays
extensive information on hospitals and doctors, this one curated by the “Beauftragte der
Bundesregierung für die Belange der Patientinnen und Patienten”. In free translation
meaning: The Federal Commission for Patients’ Affairs, an institution filiated to the
German Federal Government.
Although the external sources can be helpful, they are not the primary source as the
reliability can be unsure and the effort to extract information is much higher. Therefore,
they are used as complements to the ERPs in place.
“I could just look this code in PARC, is it's working or so in the case of the “Weisseliste”, you cannot do that.” (Participant-F, 2020)
42
“[…] or if you don't have any feedback from the BetterDoc patients then we look for the Jameda feedback.” (Participant-D, 2020)
“I look into his [the medic’s] home page for that and I will look into publications or if he is a member of a certain association or if he has other strengths.” (Participant-A, 2020)
43
5. Analysis
This chapter is dedicated to the application of the mental models framework on the results
of the interviews. It puts the findings from the previous chapter in the context of the
workflow performed, bringing to light the patterns identified with this model, as well as
the differences spotted between individuals. The patterns are described in the form of
categorization schemes that influence the decisions made along the process.
5.1 Mental Model
The mental model is arranged in three different levels: mental space, task tower and
atomic tasks. The plotted diagram consists of three mental spaces, which are discussed
separately in this analysis. The diagram was designed with the online tool Whimsical. In
adaption to the original template mentioned in the theoretical framework, in addition to
the tools’ categorization schemes were also added to the bottom half of the diagram to
show how tasks are supported by the flow of that information.
44
Figure 8 - The mental model diagram
45
5.1.1 Admit a patient
The first mental space is dedicated to the phase of admitting a patient. This consists in the
tasks involving the contact and collecting information that is needed for the research.
Figure 9 - The “admit patient” mental space
The task towers for this mental space are “Start a new case” and “Clarify information
with patient”, the first one being the most prominent.
To start a new case there are a series of atomic tasks that can be performed. While most
tasks are rather trivial and mandatory, there are two in particular that involve some
decisions. The first is “send health questionnaire”. For the adequate questionnaire to be
sent, the staff must ask for what kind of concern the patient has and that follows a
classification scheme of its own, developed according to the process. The first
categorization is if the concern is orthopedic or not. In case of orthopedic, then the
46
affected body part is the category that determines the questionnaire. In case not, it is
determined by the concerning medical field.
As to the clarify information with the patient, it is not a task essential to the process as it
only happens when it turns out the first stage of admission was not enough to provide the
necessary information, or it could not be provided at the moment of the first contact.
5.1.2 Perform research
The next mental space is dedicated to the research itself. This is the most cognitive
intensive process, as it requires several decisions to be made that determine the outcome
of the case. There is a total of 12 task towers in this mental space. For better visualization
of the diagram during the analysis, it is divided into each half at a time. For the first half
the tasks are the following:
47
Figure 10 - First set of task towers in “perform research” mental space
• Go into patients' questionnaires
• Assure sufficiency of information for research
• Define target medical profile
• Perform clinic search
• Perform medic search
• Start expert question round
48
The first, “Go into a patients’ questionnaire” is a task that consists in manually processing
the answers provided by the patient during the questionnaire. This means that the noise is
removed, for example in case the patient writes the story about how they became sick.
The key information is highlighted so it is more findable to run the rest of the process
quicker. It is also at this moment that an ICD code is defined, together with the
confirmation of the diagnosis. Depending on the case, OCD codes might be suggested
already or not.
Next, after going through the information provided in the questionnaire it is possible to
say if it is clear enough for a diagnosis or more investigation is needed. In case further
clarification is needed, additional contact is done, and possible medical documents are
requested.
Following, the task tower is defining the target profile that needs to be researched. That
is done by assessing a variety of data that becomes categorized in ways that it steers the
decisions. For example, the first, and possibly most important variable, is the patient’s
intent for the service, or they what they would like to achieve. In addition to that, there
are factors such as biometric data; how debilitated they are due to the illness; how the
patient was diagnosed in the past; what treatments were already performed or
recommended; who were the medics visited previously; what is the patients’ health
insurance. All those data points are thought through by the research staff to reach what
would be the desired professional to handle the case. Although, only the biometrics have
a defined structure, as they follow numeric ranges. The remaining points have no
controlled vocabulary.
After defining the target for the research, there are two possibilities. Either the medics are
located by performing a clinic search, or directly a medic search. Starting by the clinic
search, it is possible, through PARC, to use ICD and OPS codes to find data relevant for
clinics. Mostly, how many cases were treated within those codes. That way it is possible
to compare how many similar cases the site has treated before and therefore the expertise
of the personnel in performing the given procedures. In order to decide for the clinic
search, there are variables that need to be present in the target profile. It is favored in
49
cases that require invasive treatments, possibly surgeries, when there’s a higher presence
of risk factor and in consequence how the facilities are able to deal with the potential
complications.
When it comes to the looking directly for a medic, the main factors are having the target
profile defined as a second-opinion or conservative therapy. Having that, the main factor
would be the specialization of the required medic to be able to perform the treatment.
In the occasion the identified options do not seem to provide a satisfactory solution to the
case, there is another resource that can be used. The staff can “Start experts question
round”, which consists in activating the expert network by sending questions regarding
the case to adequate medics, who are members of the expert panel. This means that those
professionals will answer to the case, assessing the condition and providing their own
recommendation on how to proceed with it; possibly also directly recommending other
professionals in case they know medics who have the desired experience. This action is
only performed in the case of rare conditions which have low or few samples to derive
information from, or complex cases which consist in a combination of different factors
that make the treatment more challenging than usual.
The second half of task towers can be visualized in the following diagram:
50
Figure 11 - Second set of task towers in “perform research” mental space
• Compare options for recommendations
• Expand medic database
• Expand medical knowledge
• Start experts question round
• Send recommendations to patient
• Discuss recommendations with patient
After the possible medics for a recommendation are selected, the next task is to compare
then so only the best 2 or 3 options are sent to the patient. The main classifier for that is
the presence of feedback from previous patients, measured by the NPS (Net Promoter
Score) technique. This metric is favored because it is cured by the own organization.
Textual feedback is also considered, as it is possible to see written comments from
patients who visited that medic in the past. In addition, it is possible that factors that
evidence the expertise of the medic are used. For instance, if the medics have an expert
recommendation or if they are part of the expert network himself. Also, having records
51
of scientific publications and being part of professional societies count in their favor. To
support the research, it is necessary to have an extensive database. This database is
maintained by the staff when looking for information on medics and updated according
to their findings during the work routine. Naturally for incomplete or new records, the
information must be acquired through external sources. This means mostly free searching
in the internet for the websites of clinics, medics or third-party portals. The relevant
findings are then added to the database as attributes to the medics’ records.
Another supporting task tower is “Expand medical knowledge”. It consists in the group
of tasks performed by the researcher in order to understand better how to deal with
different health conditions. The main tools mentioned for those were medical books and
searching for information on the web. It is also notable that need of study is related to the
experience of the worker, meaning that it is more intensive during the first moments of
the career; and also to their specific specialization, meaning that medical areas that have
more samples that can be related, do not require intensive effort.
Following, another task tower is “Discuss case with colleagues”. This one is dedicated to
the iteration between colleagues when a case seems unclear individually. There are daily
sessions in which the team has the opportunity to clear doubts with one another in order
to proceed with a case.
The next task tower is “Send recommendation to patients”. It is formed by the tasks of
sending the selected doctors to the patients’ e-mail. That is done through Borg, which
generates an e-mail according to a defined template that is then edited by the researcher
to suit the results of the particular case. That means explaining the qualities of each of the
selected medics, as well as the differences between them. In this action there is the
possibility of sending the recommendation as a preliminary or final result. That depends
on whether a medic from the staff has talked to a patient and the condition has been
clarified and confirmed, or cases that have are straightforward enough to be resolved
without any contact at all.
In either of the mentioned cases, it leads to the last task tower in this mental space:
“Discuss recommendations with patient”. Preliminary results are always discussed with
52
the patient, with the intention to clarify any pending details regarding the condition and
confirm or rework them. When the results are final, explaining them over the phone is not
a mandatory step. It is done mostly when it is noted that the case requires special care,
either for being a severe condition or if it is understood that the patient needs direct
guidance.
5.1.3 Follow-up case
The last mental space is “Follow-up case”. It consists in the process of following the
progress of the patient when seeking the treatment for their condition. There are only two
task towers:
Figure 12 - Task towers in “follow-up case”
Regarding the first tower, it is a quite trivial task. The follow-up questionnaires have no
discerning and are the same for all cases. The essence of it is to collect the patients’
feedback if they actually visited one of the recommended medics and how satisfied they
were with it. Also, there is a follow-up up to two years to assess how the patient condition
progressed during therapy.
53
The second delicate group of tasks is regarding the resolution of complaints. It
summarizes the work done when the staff at Patient Care receive the contact of someone
who is unhappy with the provided solution. The complaint is then assessed to verify the
best solution, being the most drastic case restart the process on it and perform a different
research.
5.2 Patterns of decision and categorization
Going through the mental model, there are points that come to light where the researchers
has to make decisions that steer how the rest of the process is going to happen. In other
words, when making a choice for A or B, it is a form of categorization. This decision is
carried on, and it becomes the base for additional decisions to be made afterwards. There
are a few points in which these decisions need to be made, implicating in collecting data
and manually processing is to define what is the suitable course of action. The diagram
below illustrates the relationship between these elements.
54
Figure 13 - Network diagram of decision points.
55
The diagram aims to shift from the task centered perspective in the previously shown
mental model diagram to focus on the connections between the different categorization
schemes and decisions that need to be made. In this diagram, the starting point of the
process is represented by the green box labelled as the case. The ending point is the results
to be sent to the patient, depicted as the red box. In between, the yellow boxes represent
the different attributes that are connected to different points of the information system,
while yellow means the decisions points. In other words, yellow boxes mean points where
attributes are merely captured but left untouched. The orange boxes mean where a
categorization task actively takes place.
By opening a case, the first information captured is if the case is either orthopedic or not.
Having that, that are only two possible ways to follow. First, in case it is orthopedic then
the related body part is recorded; in case it is not, it is asked for the related medical field
which the concern belongs to. In both cases, there is already a controlled vocabulary that
does not enable the user to freely choose for any option one would imagine, but in defined
categories that related to the frequency different kinds of cases are treated and pretty
mostly only meaningful in the workflow context of the staff. That categorization defines
in a 1:1 relation the format of the questionnaire sent to the patient that serves as the
touchpoint for capturing the next round of data.
The first decision that needs to be made by the staff is defining the diagnosis. That can be
confirming the previously suggested diagnosis or assessing a totally different condition.
Almost simultaneously, there is a decision for which options of treatment to look for.
Even being directly related to the diagnosis, it is not a simple 1:1 relation, but rather a 1
to many. Both decisions for the diagnosis and the recommended action are based on
debilitating conditions; biometrics; previous diagnosis and previous medic visits.
Although biometrics are a form of category in the way they are captured with the metric
system, the others do not have a scheme in place on how the input is treated and it is
mostly captured in its raw form. One exception is the debilitating conditions, which is
partially recorded by means of standardized questions that aim at measuring how much
pain the patient feels and how that hinders the daily life. In addition, the health insurance
and intent are only relevant for the recommended action and do not influence the
56
diagnosis. The health insurance has two stages. First is if the insurance is statutory or
private; second, the insurance company itself, as some companies have different services
available to the customers. Although the differences are minor and usually do not interfere
in the final results, it is something to be considered. At last, the intent of the patient is
validated. That means, if the intent is actually reasonable or if the case is considerably
different from what the person initially thought, and another course of action needs to be
taken.
In edge cases, there is also the possibility that the diagnosis falls under a range of complex
cases or rare conditions. Complex cases most often mean that not a single issue or disease
is afflicting the patient, but a combination of several that make the course of action
unclear. A rare condition is considered a diagnosis that has been seen few or not at all in
the past. In those situations, the expert panel plays a role in providing recommendations
to how to diagnose and proceed with such cases.
Having decided for the recommended action, the possibilities are identified as 3 major
categories. Either it is an invasive treatment; a conservative treatment; or a second
opinion. This is where the classification is the loosest, not having a defined structure.
Even though procedures are defined with OPS codes, the usage of those has been mostly
described to be relevant for reporting reasons and also the application in the clinic search.
Each of those options, except second opinion, serve as an umbrella for more specified
procedures. For example, all kinds of surgeries fall under invasive treatments while most
physiotherapy procedures go under conservative. It is worth noting that in cases of
invasive treatments, it is also important to consider potential complications and risk
factors.
Once the recommended action is defined, it is possible to look for the specialization of
medics that are performed to handle the recommendation. Some of those medics are them
selected for a final screening, during which they are compared through a sort of several
attributes. These fields also have no structured, except for the NPS. Being a continuous
numerical value that allows for easy comparation. The remaining points are evaluated
57
with not structure, being free text, which again depend on the interpretation of the
operator. Finally, according to these criteria the results are defined and sent to the patient.
5.3 Difference analysis
The final analysis derived from the mental model is how the individuals showed different
or individual beliefs. It is important to say that this analysis does not show in any form
that individuals disagree from each other regarding these topics. It only highlights the
points who were raised by only one or a few individuals, but not in by their majority.
Also, it does not mean that here are no other deviations, which can often be unconscious
and away of the awareness of the interviewed participants.
Different conflict of interests for children
It was mentioned once that the assessment of conflicts of interest is different when
regarding patients with a young age. The explanation was that it is considered that
surgeons have more empathy towards children, as unnecessary procedures may have an
impact on the rest of their lives. Also, it was mentioned that serious diseases in children
have a bigger impact in their lives for the possibility of affecting their growth process,
different from people who acquire them after an adult age.
Thinking of patient as a family member
Empathy was a theme that surfaced a vast number of times during the interviews, although
only once it was said directly that as part of their working process, they actively imagined
the patient as a family member as a way to increase empathy.
Discussing cases with colleagues
Another topic raised only once was discussing the case with colleagues, even though there
are regular meetings from the team. This practice allows for the share of opinions on a
case, in the situation that a particular researcher has reached a stalemate and does not
know how to proceed with it.
58
Internal vs external sources
The last difference noted was the usage of internal against external sources. The results
showed that researchers working in orthopedic cases relied much more intensively on the
internal sources of information. That is seen as logic, since orthopedic cases numerically
represent a bigger sample and a relatively narrower variance in the cases. Non-orthopedic
researchers described to use external sources more frequently, as they often find
themselves in front of cases without a significant number of samples within the
established database. That also plays a role in the learning curve for the different roles the
researchers may have. Having clearer patterns to follow, it is reasonable to assume the
curve is smoother for orthopedic researchers.
59
6. Conclusion
The aim of this study is defined as identifying patterns of decision making during the
research process at BetterDoc. Towards that aim, a literature review was conducted
involving concepts of information systems, controlled vocabularies, mental models and
software development.
In sequence, the methodology was defined as a deductive approach on an interpretivist
perspective. The chosen data collection method was semi-structured interviews with the
objective of gathering textual data from different employees across different teams. The
analysis was conducted as a textual analysis for identifying recurrent topics and the
thinking process of the interviewees, being done manually.
Towards the results, it was possible to identify four themes that brought to light 20
different topics relevant to the workflow pertinent to resolving a patient’s case in the
organization. Those results were synthesized in a mental model diagram, with the
objective of visually illustrating the beliefs, supporting tools and categorization schemes
inherent to the staff when executing their tasks. In addition to the mental model diagram,
another scheme was plotted to illustrate how the information flow interconnects in
different points. Based on that analysis, the research questions were answered.
RQ1: What decision-making patterns can be identified at BetterDoc's matching process?
There were four decision points identified, being them defining the diagnosis;
recommended action; required specialization; selection of medics. 22 information points
were evidenced and illustrated in relation to these decisions. Also, it was elaborated on
how these points of information are collected and structured.
RQ2: What are the particular perspectives from different individuals during the matching
process?
60
Regarding particular perspectives from individuals, they were not plotted as charts for
individual representation. Although it was possible to highlight points mentioned only by
one person or a certain segment. These points mean decisions or criteria that are
considered only by some of the interviewees, not being part of the standard workflow.
Those were: conflict of interests regarding children; thinking of a patient as a family
member; discussing cases with colleagues; using internal or external sources of
information.
The findings of this research can be used to further develop information systems to
provide the information needed to make decisions, by designing controlled vocabularies
that allow users to better dwell in the complexity of the system. The adequate
understanding of the users’ perspective can grant insights for the development cycles and
how to structure those information environments.
The dynamics in the decision-making theory that relate to the existence of mental models
imply that the decision relies on the perception of reality from the individuals, recognizing
there might be differences between perception and reality (Capelo & Ferreira Dias, 2015).
It is, therefore, beneficial to the user of information systems to have them designed as
places that work towards closing that gap. The elements of a place should be thought as
a way to facilitate the tasks and understanding of information pertinent to the context,
considering, for example, the naming, grouping of elements, hierarchy and the meaning
of the possibilities that are presented to the user.
Mental models are therefore highly valuable in the research and practice of system
dynamics. They need to be worked by the designers. A proper understanding of what they
are, what properties and characteristics they have, how they relate to learning and
decisions; is crucial for an enterprise to succeed (Doyle & Ford, 1998).
61
7. Discussion
This chapter consists in the discussion of this study. In a first moment, it brings
considerations and limitations regarding the methods applied as well as the results
achieved. Second, what the conclusions can lead to both in terms of research and practice
and how they are connected to each other. At last, there a few suggestions to be considered
in the development of further research that were identified.
7.1 Methods and results discussion
Regarding the methods applied in this study, there are a few remarks. First, the number
of interviews performed was shorter than the targeted number. While the target was set
from 8 to 10 interviews, 7 were fulfilled. There were two reasons for that. First, the study
took place during the peak of the COVID-19 breakout in Europe, which brought severe
measures of social isolation and changed the daily life for most of the inhabitants. That
was no exception to the staff at BetterDoc, who faced challenges adapting to the
circumstances, and as they were essential to the realization of the study, the initial
interview schedule was delayed. Second, while carrying the interviews and perceiving
the gain of unprecedent data was gradually reducing, the author opted to conduct no
additional interviews and avoid any possible inconveniences in such turbulent moment.
In addition, while the number of participants was limited also the sampling could be more
diversified. 7 people from the staff were selected, being from 3 different teams: research,
patient care and product development. It would have been valuable to also add the
perspective from medics who work in direct contact with the patient, discussing
diagnostics and results over the phone.
Regarding the analysis, the coding of the qualitative data was done manually given the
need of the refinement to synthesize data relevant for the mental model diagram. That
choice was made by the lack of experience of the author when using coding software, and
the unwillingness to dwell in uncertain territory within a limited timeframe. The analysis,
however, proved to be extensively time consuming.
62
It is noted that electronic coding can make the process smoother once the learning curve
is flattened, providing more extensive results in terms of depth and also more varied
reports (Basit, 2003). Having more preparation in advance to work with electronic coding,
the results could possibly be broader or better documented.
Also, the mental model diagram was synthetized as a whole, as proposed by the chosen
framework. Although, there are other authors that suggest an approach of depicting the
mental model of each separate individual and then analysis the common and different
points. That approach could provide a more in-depth analysis on the deviation between
individuals.
The results of this study are considered within the expected towards the initial goal. It
was possible to bring to light patterns experienced daily by the staff in the organization
and influence the health outcome of many people. However, it is important to note that
the results are a picture of the data collected through the interviews and may not be
representative to all employees at the organization or even external professionals.
It is evident that the results carry a lot of the workflow performed daily in the
organization. This limitation was recognized and expected since the beginning and
therefore, the results should not be seen as a guideline or immutable. They should be a
component of an iterative process, and although valuable for placemaking in information
systems, they are subject to further validation and refinement.
7.2 Implications for research and practice
Even though the outbreak of COVID-19 disrupted the daily life of people all around the
world during this study, it evidenced more than ever the relevance of the topic. Still, in
2019, InTouch Health (2019) expected the telehealth industry to reach 40 billion dollars
worldwide.
When writing for the World Economic Forum, Kalenzi (2020) describes hospitals
becoming risky places both for the patients and professionals during the pandemic. Also,
he highlights the rise of telemedicine as one of the possible pillars for a solution in such
63
crisis, but not without challenges such as: cost structures; application of AI technologies;
data sharing regulations. However, the benefits of remote screening and triage, either by
the application of video calls or bots, are considered essential for reducing the risks of
infection in patients and healthcare professionals. Also, by enabling quarantined
professionals to work remotely, it eases the workload on front-line personnel where the
effort is critical for tending the severely ill (Hollander & Carr, 2020).
It is stated that the global healthcare system struggles to fulfil a quadruple aim: improving
experience of healthcare; improving the health of populations; reducing per capita costs
of healthcare; improving the work life of healthcare professionals. When addressing the
application of artificial intelligence to healthcare, there are several concerns such as
achieving a solid control and assuring that technical accuracy translates to clinical
efficacy (Kelly et al., 2019).
Although BetterDoc does not define itself as a telehealth provider, its services have
unquestionably been successful in providing valuable health service to its users. In this
context, the study at the organization proves to be valuable. According to the M3 model,
research and practice are not dissociated and play a role on each other. The M3 naming
stands for Meta-Modelling Methodology. It describes three hierarchical layers, respective
to epistemology, science, and practice. The meta level comprises the conceptual
framework of a scientific community. The object level comprises the theories and models
that attempt to explain and predict problems and solutions. The practice in its turn apply
the tools to solve ordinary problems. In return, the application of those solutions in the
practice level incur in evidence that can oxygenate the upper layers in an innovative
progression. By acquiring new evidences of practical solutions, updated theories and
models emerge, possibly being incorporated in the meta level (Lacerda & Lima-Marques,
2014).
64
Figure 14 - M3 hierarchy of inquiring systems (Lacerda & Lima-Marques, 2014, p. 2
adapted from Van Gigch and Pipino 1986)
As the analyzed workflow showed to be very intensive in labelling and categorization
schemes, it is notable that such tasks can be done by algorithms. There is a wide range of
machine learning techniques that can be applied to make decisions in an automated way.
Classifiers are becoming a widely used application of machine learning, applying
algorithms to sets of data in order to predict the class (category) that entry belongs to (Pak
& Lee Teh, 2016). Although, this workflow has the characteristic of being more delicate
than other applications seen nowadays for machine learning, because it concerns the
health and well-being of people.
65
There is the possibility to experiment such algorithms with free software, such as the
Scikit-learn library for the Python programming language. This software already enables
many techniques to be applied, such as decision trees and neural networks.
Decision trees work in a way that all the relevant data points are used to find relevant
patterns towards a target variable. Naturally, all the values from the values come from the
dataset used. What the algorithm does is finding statistically the most likely splitting
points for decisions, following an if-then logic. Although, this logic is binary (either the
condition is true or false) and the algorithm tends to lose accuracy dealing with variables
that have a wide range of options. One of the advantages, although, it is the possibility of
a graphical representation showing what the splitting points are (Pak & Lee Teh, 2016).
Neural networks, in their turn do not allow for a graphical representation. They are
structured in a way that mimics human neurons in a series of layers. Considering that,
they proved to achieve good accuracy in dealing with complex problems. Although the
reasoning behind the achieved predictions are not explicit (Nielsen, 2015).
In addition, another machine learning application is the usage of Natural Language
processing (NLP). The use of NLP has been increasing rapidly in the past decade, due to
its capability of processing vast amounts of unstructured text and retrieve central pieces
of information. The application of NLP is also vital to features involving chatbots, who
need to be able to accurately understand the user to trigger the adequate response from
the computation point of view (Davenport & Kalakota, 2019).
The mentioned applications are interesting in the point that they allow for scaling
operations while easing the cognitive load on the human workforce. That would allow,
for example to reach results faster than manual processing or simply more accurate by
drawing patterns that can be missed by the human eye.
The potential of artificial intelligence is promising, although also controversial. The
impact of an incorrect diagnosis is too great of a risk. Therefore, there are a few
recommendations in case BetterDoc as an organization wants to develop initiatives in that
direction.
66
First of all, the usage and accuracy of prediction models depend on the consistency of the
dataset used to train the model. Therefore, there is an intense work of structuring
controlled vocabularies that needs to be done. That comes in particular with the complex
variables that are in place. While a controlled vocabulary gives the advantage of reducing
the complexity in question, there is a trade-off in the effort of designing and maintaining
such structured and that needs to be weighted.
As the information system’s capillarity can be very high, it is reasonable to say that a
single classifier or model is not able to tend to a case from beginning to end. At least in
the first moment. Therefore, a chain of classifiers would need to be in place. To further
reduce complexity, a compartmented architecture of classifiers would be likely to achieve
better results. For example, instead of having a single classifier diagnosing all orthopedic
cases, having different classifiers diagnosing the cases for each body part separately. That
would limit the range of both input and output variables, likely deriving a more accurate
model. Also, no prediction should be treated as final and should be subject human
inspection.
At last, the application of NLP can be fruitful in two ways. The first would be in the
processing of texts to derive information that can be used by the classifiers. This
application would be very related to the controlled vocabularies application. As another
option, the deployment of NLP as chatbots could lead to an automated communication
channel that can serve patients constantly anytime of the day or year, being able to collect
the necessary information directly and easing the burden of answering questionnaires.
It is worth noting that this section suggests a few options of artificial intelligence
applications and techniques, it is by no means a qualified experimentation and does not
mean that other possibilities should not be considered.
7.3 Future research
At last, there are three suggestions regarding further research that were considered when
discussing the results of this work. First, considering the already mentioned connection
between science and practice relating to each other in an innovative process, it would be
67
interesting to verify the results that can be achieved experimenting predictive models
deriving from a consolidated practice in remote health services. This could indicate a way
to address the challenge of having algorithms that are technically efficient but not
clinically effective.
Another possibility for research would be regarding the ethical considerations when it
comes to health diagnostics assisted by AI. The empathy could be a decisive factor for
people to accept the usage of telehealth services, and that is strongly related to the safety
of collecting and processing delicate data regarding health.
In conclusion, there is also opportunity to conduct further research regarding the mental
models of healthcare professionals in other organizations. There are a series of delicate
decisions those professionals need to address daily, that being at BetterDoc, at clinics,
hospitals, medical practices, triage facilities and even emergency hotlines. Although these
decisions might be very well documented in medical science and protocols, putting them
into the light of a design process focusing on the experience of the users is an interesting
prospect. Be it in small or big organizations, each day healthcare is one of the most
intensive uses of information systems and possibly one that affects people the most
directly. There can be fruitful results when using understanding better the work of those
professional in the process of placemaking in information environments, that
consequentially reflect on the outcome of those service for the users.
68
8. Reference List
Arango, J. (2017). 3 Placemaking Lessons From the Magic Kingdom - UX Collective. UX Collective. https://uxdesign.cc/3-placemaking-lessons-from-the-magic-kingdom-4263deb29b2f
Arnold, R. D., & Wade, J. P. (2015). A definition of systems thinking: A systems approach. Procedia Computer Science, 44(C), 669–678. https://doi.org/10.1016/j.procs.2015.03.050
Basit, T. N. (2003). Manual or electronic? The role of coding in qualitative data analysis. Educational Research, 45(2), 143–154. https://doi.org/10.1080/0013188032000133548
Baumel, T., Nassour-Kassis, J., Cohen, R., Elhadad, M., & Elhadad, N. (2017). Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignment. 409–416. http://arxiv.org/abs/1709.09587
Baxter, G., & Sommerville, I. (2011). Socio-technical systems: From design methods to systems engineering. Interacting with Computers, 23(1), 4–17. https://doi.org/10.1016/j.intcom.2010.07.003
Beck, K. M., Beedle, M., Bennekum, A. van, Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R. C., Mellor, S. J., Schwaber, K., Sutherland, J., & Thomas, D. (2013). Manifesto for Agile Software Development. Undefined.
Benyon, D., & Resmini, A. (2017). User experience in cross-channel ecosystems. HCI 2017: Digital Make Believe - Proceedings of the 31st International BCS Human Computer Interaction Conference, HCI 2017, 2017-July. https://doi.org/10.14236/ewic/HCI2017.38
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. https://doi.org/10.1191/1478088706qp063oa
Capelo, C., & Ferreira Dias, J. (2015). Exploring the role of mental models in strategic decision making. January 2008.
Covert, A. (2014). How to Make Sense of Any Mess. http://www.howtomakesenseofanymess.com/
D. Cagle, R. (2019). How Design Research Applies to Public Space - Medium. Medium. https://medium.com/@ryandcagle/how-design-research-applies-to-public-space-b5f2bc30e15c
Danson, M., Taheri, B., & Richards, J. (2018). Research Philosophy and Research
69
Design. 2014.
Davenport, T., & Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future Healthcare Journal, 6(2), 94–98. https://doi.org/10.7861/futurehosp.6-2-94
Doyle, J. K., & Ford, D. N. (1998). Mental models concepts for system dynamics research. System Dynamics Review, 14(1), 3–29. https://doi.org/10.1002/(SICI)1099-1727(199821)14:1<3::AID-SDR140>3.0.CO;2-K
Flick, U. (2007). Designing Qualitative Research. SAGE Publications, Ltd. https://doi.org/10.4135/9781849208826
Gentner, D., & Stevens, A. L. (2014). Mental Models. Taylor and Francis.
Goldkhul, G. (2012). Pragmatism vs interpretivism in qualitative information systems research. European Journal of Information Systems, 21(2), 135–146.
Graubner, B. (2007). ICD und OPS. Historische Entwicklung und Aktueller Stand. Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz, 50(7), 932–943. https://doi.org/10.1007/s00103-007-0283-x
Harpring, P. (2013). What Are Controlled Vocabularies? Introduction to Controlled Vocabularies: Terminology for Art, Architecture, and Other Cultural Works, 12–26. https://www.amazon.com/Introduction-Controlled-Vocabularies-Terminology-Architecture/dp/160606150X/
Health Europa. (2019). Technology and healthcare: working together for a healthier future. https://www.healtheuropa.eu/technology-and-healthcare-working-together/94898/
Healthcare Business & Technology. (2019). Medical Technology. http://www.healthcarebusinesstech.com/medical-technology/
Hollander, J. E., & Carr, B. G. (2020). Virtually Perfect? Telemedicine for Covid-19. The New England Journal of Medicine, 382(18), 1679–1681. https://doi.org/10.1056/NEJMp2003539
Information Architecture Institutte. (n.d.). What is Information Architecture? https://www.iainstitute.org/what-is-ia/
InTouch Health. (2019). What’s Driving Telehealth Growth in 2019? https://intouchhealth.com/whats-driving-telehealth-growth-in-2019/?gdprorigin=true
Johnston, A. (2014). Rigour in research: Theory in the research approach. European Business Review, 26(3), 206–217. https://doi.org/10.1108/EBR-09-2013-0115
70
Kalenzi, C. (2020). Telemedicine can be a COVID-19 game-changer. Here’s how | World Economic Forum. 13 May 2020. https://www.weforum.org/agenda/2020/05/telemedicine-covid-19-game-changer/
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. 1–9.
Lacerda, F., & Lima-Marques, M. (2014). Information Architecture as a Discipline—A Methodological Approach. In A. Resmini (Ed.), Reframing Information Architecutre (pp. 1–10). Springer. https://doi.org/10.1007/978-3-319-06492-5_1
Marr, B. (2019). The 9 Biggest Technology Trends That Will Transform Medicine And Healthcare In 2020. Forbes. https://www.forbes.com/sites/bernardmarr/2019/11/01/the-9-biggest-technology-trends-that-will-transform-medicine-and-healthcare-in-2020/#7d1acb4872cd
McKee, A. (2011). What Is Textual Analysis? In Textual Analysis (pp. 2–33). SAGE Publications Ltd. https://doi.org/10.4135/9780857020017.n1
Morville, P. (2014). Intertwingled.
Mühlbacher, A. C., & Kaczynski, A. (2016). Making Good Decisions in Healthcare with Multi-Criteria Decision Analysis: The Use, Current Research and Future Development of MCDA. In Applied Health Economics and Health Policy (Vol. 14, Issue 1, pp. 29–40). Springer International Publishing. https://doi.org/10.1007/s40258-015-0203-4
Myers, M. D., & Newman, M. (2007). The qualitative interview in IS research: Examining the craft. Information and Organization, 17(1), 2–26. https://doi.org/10.1016/j.infoandorg.2006.11.001
Nielsen, M. A. (2015). Neural Networks and Deep Learning. Determination Press.
Pak, I., & Lee Teh, P. (2016). Machine Learning Classifiers: Evaluation of the Performance in Online Reviews. Indian Journal of Science and Technology, 9(45). https://doi.org/10.17485/ijst/2016/v9i45/100703
Participant-A. (2020). Semi-structured interview.
Participant-B. (2020). Semi-structured interview.
Participant-C. (2020). Semi-structured interview.
Participant-D. (2020). Semi-structured interview.
Participant-E. (2020). Semi-structured interview.
71
Participant-F. (2020). Semi-structured interview.
Participant-G. (2020). Semi-structured interview.
Prior, L. (2014). Content Analysis. In P. Leavy (Ed.), The Oxford Handbook of Qualitative Research (pp. 358–379). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199811755.013.008
Quintarelli, E., Resmini, A., & Rosati, L. (2006). FaceTag: Integrating bottom-up and top-down classification in a social tagging system. CEUR Workshop Proceedings, 201(January).
Resmini, A. (2014). Reframing Information Architecture. Springer.
Resmini, A., & Lacerda, F. (2016). The architecture of cross-channel ecosystems. Proceedings of the 8th International Conference on Management of Digital EcoSystems - MEDES, 17–21. https://doi.org/10.1145/3012071.3012087
Resmini, A., & Luca, R. (2012). A Brief History of Information Architecture. Journal of Information Architecture, 3(2). http://journalofia.org/volume3/issue2/03-resmini/
Resmini, A., & Rosati, L. (2011). Place-making. In Pervasive Information Architecture (Issue Greenaway 1978, pp. 63–87). Elsevier. https://doi.org/10.1016/B978-0-12-382094-5.00004-6
Rosch, E. (2013). Principles of Categorization. Readings in Cognitive Science: A Perspective from Psychology and Artificial Intelligence, 312–322. https://doi.org/10.1016/B978-1-4832-1446-7.50028-5
Rosenfeld, L., Morville, P., & Arango, J. (2015). Information architecture : for the Web and beyond.
Saunders, M., Lewis, P., & Thornhill, A. (2009). Research Methods for Business students.
Scotland, J. (2012). Exploring the philosophical underpinnings of research: Relating ontology and epistemology to the methodology and methods of the scientific, interpretive, and critical research paradigms. English Language Teaching, 5(9), 9–16. https://doi.org/10.5539/elt.v5n9p9
Vermaas, P., Kroes, P., van de Poel, I., Franssen, M., & Houkes, W. (2011). A Philosophy of Technology: From Technical Artefacts to Sociotechnical Systems. Synthesis Lectures on Engineers, Technology and Society, 6(1), 1–134. https://doi.org/10.2200/s00321ed1v01y201012ets014
Weiss, K. (CPO at B. (2012). Containerist. http://containerist.org/
72
Weiss, K. (CPO at B. (2020). Private conversation.
World Health Organization. (2019). WHO | International Classification of Diseases, 11th Revision (ICD-11). WHO.
Wurman, R. S. (1989). Information Anxiety.
Young, I. (2008). Mental Models - Aligning Design Strategy with Human Behavior. http://rosenfeldmedia.com
Žukauskas, P., Vveinhardt, J., & Andriukaitienė, R. (2018). Philosophy and Paradigm of Scientific Research. In Management Culture and Corporate Social Responsibility. InTech. https://doi.org/10.5772/intechopen.70628
73
9. Appendix
9.1 Interview guide
Opening (10 min)
• Introduction
• Purpose of the research and
ethical commitments
• Interviewees background (role
and experience)
• The organization (process, vision,
goals)
Main body (15 min)
• Performing research
• Supporting information systems
• Medical documents
Closing (5min) • Summarizing
• Acknowledgment
74
9.2 Semi-structured questionnaire
Opening
- Introduction as an interviewer
- Clarify the purpose of research and confidentiality commitments
- Ask permission to record
- What is your role at BetterDoc? For how long do you work at the company?
- How you describe the process a case goes through at BetterDoc?
Main Body
- Are you able to perform research on your own?
- Can you demonstrate the steps in a test case?
- What is the most relevant information?
- What are the sources for information you need?
Closure
- Thank the interviewee for their time
- Summarize main points covered in the interview
- Is there any final consideration you would like to add?
