IN DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2017

Compliance with the General Data Protection Regulation: an exploratory case study on business systems’ adaptation

MIKAEL KNUTSSON

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF COMPUTER SCIENCE AND COMMUNICATION

English title

Compliance with the General Data Protection Regulation: an exploratory case study on business systems’ adaptation

Swedish title

Medgörlighet med Dataskyddsförordningen: en undersökande fallstudie av affärssystems anpassning

Author

Mikael Knutsson, mikknu@kth.se

Submitted for the completion of the KTH program; Media Technology, Master of Science in Computer Science and Engineering Supervisor: Rebekah Cupitt, KTH, School of Computer Science and Communications, Department of Media Technology and Interaction Design. Examiner: Henrik Artman, KTH, School of Computer Science and Communications, Department of Media Technology and Interaction Design. Date of submission: 2017-06-21

ABSTRACT Current moves into a heavily digitalized era has led to a phase where our privacy is being eroded as we hand over our personal data to organizations and their systems. At the same time, the applicable laws to give security to the individuals have failed to incorporate these legal developments. However, in April 2016 the European Union proposed a change to a new regulation called the General Data Protection Regulation (GDPR). The GDPR will be implemented and start to apply in May 2018, thus the main purpose of this study was to investigate how organizations can adapt to changing regulations on how personal data should be stored and managed, and what the key tension points are within specifically closed IT-systems. The goal of the GDPR and this study on its feature implementation is to guarantee the EU citizens their right to privacy. Through an exploratory case study involving an in-depth analysis of two closed IT-systems this study develops a broader understanding on how organizations should adapt their daily businesses in order to be fully compliant with the new bylaws. This study identifies four critical issues which are used to discuss how the new bylaws could affect the EU citizens’ privacy. To accomplish this and open up for further investigation within the field of data privacy laws - four different propositions to modifications were suggested.

SAMMANFATTNING Den aktuella övergången till en omfattande digitaliserad tid har lett till en fas där vår integritet går förlorad då vi överlämnar vår personliga information till organisationer och deras system. Samtidigt har de tillämpade datalagarna med syfte att skydda individen misslyckades med att införliva denna utveckling. Därför har den Europeiska Unionen i april 2016 föreslagit en förändring till en ny reglering som får namnet Dataskyddsförordningen. Dataskyddsförordningen kommer blir implementerad och börja gälla i maj 2018 och därav var huvudsyftet med den här studien att undersöka hur organisationer bör anpassa sig till de nya riktlinjerna för hur personlig information bör lagras och hanteras samt vilka spänningspunkterna är för slutna IT-system. Målet med Dataskyddsförordningen och vad den här studien beaktade i dess kommande utförande är att garantera EU-medborgare rätten till sin integritet. Genom att utföra en undersökande fallstudie innehållandes en djupgående analys av två slutna IT-system har den här studien bidragit med en bredare förståelse för hur organisationer bör anpassa sina dagliga verksamhet för att vara helt medgörliga med Dataskyddsförordningen. Studien har identifierat fyra kritiska problem som har legat till grund för att diskutera hur den nya förordningen kommer påverka EU-medborgarnas rätt till sin integritet. För att göra det möjligt samt öppna upp för framtida undersökningar inom ramen för dataskyddslagar föreslogs fyra förslag på generella förändringar.

Compliance with the General Data Protection Regulation: an exploratory case study on business 

systems’ adaptation 

Mikael Knutsson Interactive Media Technology 

Royal Institute of Technology, Sweden mikknu@kth.se 

 ABSTRACT Current moves into a heavily digitalized era has led to a phase where our privacy is being eroded as we hand over our personal data to organizations and their systems. At the same time, the applicable laws to give security to the individuals have failed to incorporate these legal developments. However, in April 2016 the European Union proposed a change to a new regulation called the General Data Protection Regulation (GDPR). The GDPR will be implemented and start to apply in May 2018, thus the main purpose of this study was to investigate how organizations can adapt to changing regulations on how personal data should be stored and managed, and what the key tension points are within specifically closed IT-systems. The goal of the GDPR and this study on its feature implementation is to guarantee the EU citizens their right to privacy. Through an exploratory case study involving an in-depth analysis of two closed IT-systems this study develops a broader understanding on how organizations should adapt their daily businesses in order to be fully compliant with the new bylaws. This study identifies four critical issues which are used to discuss how the new bylaws could affect the EU citizens’ privacy. To accomplish this and open up for further investigation within the field of data privacy laws - four different propositions to modifications were suggested.

Keywords Data protection laws, GDPR, privacy, the right to be forgotten, IT safety, data minimization

Abbreviations and definitions Data subject - An individual who is the subject of personal data Directive - Directive 95/45/EC EU - European Union GDPR - General Data Protection Regulation. Regulation 2016/679 ’Right to be forgotten’ - Article 17 in the GDPR

1. INTRODUCTION During the last decades as the numerous devices, sociotechnical systems (STS), Internet of Things (IoT) and endless databases storing and managing the flow of personal data has increased, the privacy conundrum and anxiety among the public has arisen.[35] Despite the enormous possibilities from the rise of Information Technology (IT) where we can share information with our friends via social networks or store family pictures in “the cloud”, new threats to privacy have emerged. The possibilities to keep track of citizens, practise tremendous surveillance or constitute a threat to privacy by breaking into computer systems has increased which has given rise to problems concerning law, policy and ethics.[3][49] An obvious response to these new threats to our privacy has been data protection laws that almost all countries to some extent possess today.[49]

The European Union has a long history of involvement in

data protection and the most known contribution to this has been the European Union Data Protection Directive from 1995 which is the current directive in use by EU countries.[30] It is this directive that each country in the EU should relate to when treating cases concerning protection of personal data and the fundamental right to privacy.[48][11] However, as the new digital society has evolved the regulations that were self evident 20 years ago are not as relevant today which is the primary reason why the European Parliament, European Commission and the European Council has proposed an upgrade of the directive from 1995. The new regulation, called the General Data Protection Regulation (GDPR), seeks to ensure a harmonized and unified approach to the protection of the data subjects (the EU-citizen) personal data. The aim of the GDPR is also to give back the data subjects control over their personal data and create a sustainable approach to data protection laws all over the European Union.[12] GDPR was adopted on 27 April 2016 and after a two-year transition it will enter into effect on 25 May 2018 where new concepts are going be introduced that will make the data protection more substantial and clear - such as data portability, privacy by design and the “right to be forgotten”.

One might claim that the principle underpinning the “right to be forgotten” already was included in the directive from 1995 which is true to a certain extent. Article 12 in the Directive from 1995 stated that an EU-citizen could ask to have their personal data deleted from an IT-system if the data was no longer necessary for the data controller. The question is, therefore, in what way does the old directive differ from the new and what parts have to be updated and clarified to meet the new demands of protection of privacy in the digital age? When comparing the two principles regarding the “right to be forgotten”, Article 12 in the Directive from 1995 and Article 17 in the GDPR there are some distinct amendments. For instance, a new rule is the insistence that even non-European companies must comply to the EU rules, as described in Article 3 in the GDPR, when they offer services to their customers in Europe. The GDPR also requires that when a request is made to delete all personal data from a database, the data controllers must then notify the third parties with whom they have shared this personal data. Furthermore, once the GDPR enters into force it will be up to the companies, and not the data subject, to prove that the personal data cannot be erased because it is needed in a lawful manner and this was not a matter of cause before. However, there is one distinct exception phrased in the new regulation which states that personal data may be stored for a longer period of time if the data will be used solely for historical, statistical or scientific research purposes.

Since the new regulation will require all organizations holding personal data on EU residents to delete the information upon request or when it is no longer required by the organization it will induce comprehensive modifications in IT-systems.[3] If organizations breach the GDPR (this could mean they are

violating the Privacy by Design concept by not holding the tools to erase personal data upon request or not having sufficient consent from the customer to process its data) they can be fined up to €20 million or 4% of the annual global turnover.

The question is how much control, power and data organizations are willing to give up in order to become fully compliant with the GDPR. These changes will affect data privacy in several ways. Currently, consumers and citizens are relatively unaware of issues relating to data privacy but also in many cases unable to comprehend their complexity and the various regulations and ways in which their data is collected, stored and used as capital.[42] The aim of this research was therefore to identify current issues within organizations and their systems of storing personal data and come up with relevant solutions to these problems. These solutions will hopefully be useful for organizations’ upcoming working procedure upon being compliant with GDPR.

2. BACKGROUND In this section the central concepts will be introduced and explained which is necessary in order to comprehend the following results. This will include general definitions of important concepts concerning the GDPR such as defining personal data and controllers, but also by introducing the idea behind data privacy and different kinds of IT-systems. Lastly the research question which I strive to answer throughout my study will be stated. 2.1 GDPR definitions

The GDPR defines different key concepts that the new regulation predicates all its subsequent ordinances upon. In order to understand its import, the most relevant definitions need to be defined.

2.1.1 Personal data Personal data is a central notion when speaking about data protection and the GDPR has defined it in Article 4, with some slight changes from the directive from 1995 due to the ubiquitous Information Technology society we have today. In this study, I will use the new definition as follows:

‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person; (Regulation 2016/679, note 1, Article 4)

This definition of personal data is, however, very broad and can be seen as vague since it leaves it up to the interpreter to understand things such as whether a picture of a person can be considered as sufficient to identify a person reliably. Furthermore, for some organizations such as those who work with online business, this might lead to even more compliance obligations. This is especially the case for companies that use cookies which, under the GDPR, will be considered as personal data because they identify the data subject through their internet provider address (IP address).

2.1.2 Data subject A data subject is an individual who is alive, to whom the personal data stored in an IT-system relates to.[12] For instance, if an organization holds personal data about their customers, each customer is a data subject. Within the scope of the GDPR, a data subject is provided with several rights which can be enforced against the organizations that holds the personal data. Examples of rights which may have an important impact upon how organizations should process the personal data of data subjects’ in a lawful manner are; the right to be forgotten, right of data portability and right of access. 2.1.3 Controller A controller or Data Controller which it is often mentioned as in literature and articles about data protection laws is a term that will be widely used in the context of the GDPR as well as in my study. It is therefore essential to have a clear understanding of its meaning. The European Commission and the other involving parts define controller in the GDPR as follows:

‘controller’ means the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data; where the purposes and means of such processing are determined by Union or Member State law, the controller or the specific criteria for its nomination may be provided for by Union or Member State law; (Regulation 2016/679, note 7, Article 4)

The Data Controller is a person, or a group of single individuals joined together, who decides the way personal data should be processed and are required to provide any data subject with access to their own personal data which they store in a system. They must also provide ways to update, modify and delete data without undue delay or any sorts of financial charges of the user unless it conflicts with other applicable rules in the GDPR. 2.1.4 Processor The Data Processor is under obligation to keep record of all kind of processing activities and is any person - except an employee of a Data Controller - who processes data on the behalf of a Data Controller.[20] The general definition of a Data Processor is stated in the GDPR as:

‘processor’ means a natural or legal person, public authority, agency or other body which processes personal data on behalf of the controller; (Regulation 2016/679, note 8, Article 4)

The processor concept is very similar to the one from the last Directive from 1995 and an entity that already is the Data Processor today will most likely continue to be that when the GDPR starts to apply next year.[14] 2.1.5 Third party Third party means any legal entity or individual who is not a Data Controller. In the GDPR, a third party definition is included and is declared as:

‘third party’ means a natural or legal person, public authority, agency or body other than the data subject, controller, processor and persons who, under the direct

authority of the controller or processor, are authorised to process personal data; (Regulation 2016/679, note 10, Article 4)

Third parties are often, when it comes to data security and the applicable laws regarding privacy, the biggest area of risk when it comes to exposing sensitive data and in 63% of cases where data breaches in an organization the third party is the one the be responsible.[34]

2.1.6 The right to be forgotten Before mentioning the exact details of Article 17 in the GDPR - ’right to be forgotten’ - it is essential to understand if the organization in question is within the general scope of the GDPR. In Article 3 of the regulation the territorial scope that the regulation affects and what it applies to is explained as follows: ● the processing of personal data in the context of the activities of an establishment of a controller or a processor in the EU, regardless of whether the processing takes place in the Union or not. ● the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to; (a) the offering of goods or services, irrespective of whether a payment of the data subject is required, to such data subjects in the Union; or (b) the monitoring of their behaviour as far as their behaviour takes place within the Union. ● the processing of personal data by a controller not established in the Union, but in a place where Member State law applies by virtue of public international law.[12] Once an organization complies with these parts they are within the scope of urging to be compliant with all the bylaws which the new regulation brings, where Article 17 plays a central role with its purpose to give individuals better control of their own data.[44] Article 17, also known as the ’right to be forgotten’ is mainly about the right for an individual to request either removal or deletion of their own personal data if the data controller finds no reason for any further processing of it as it was intended when it firstly was collected and processed. The ’right to be forgotten’ can be divided into three different important sections. The first part states that data subject shall:

have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay (Regulation 2016/679, note 1, Article 17)

and includes circumstances where individuals have the right to prevent any further data processing such as if the personal data was made public while he or she was a child. The second part of Article 17 covers the responsibility of the controller where:

the controller has made the personal data public and is obliged pursuant to paragraph 1 to erase the personal data (Regulation 2016/679, note 2, Article 17)

but also, to inform controllers who are processing personal data, such as third parties, about the data subject's data erasure request.[32] This includes copies, links or any replications that can connect the personal data to the data subject to be erasure. Finally, in the third and last section of Article 17 several different exceptions are stated where the Data Controller can refuse to comply with the erasure request. This could occur if the personal data is processed:

● to exercise the right of freedom of expression and information. ● to comply with legal obligation for the performance of a task carried out in the public interest or exercise of official authority. ● for public health reasons in the public interest. ● to achieve purposes in the public interest, scientific or historical research or for statistical purposes. ● to exercise or defend legal claims. 2.2 Related research Below, previous research relating to privacy will be introduced which is the foundational concept and reason why the new regulation, GDPR, will be implemented. 2.3 Privacy Privacy is a term or concept that commonly appears in several different occasions and contexts, such as discussions about politics, philosophy or legal purposes and therefore does not have a single unified definition yet.[8] Privacy is frequently used to denote various amount of interests regarding the control, secrecy and development of personal information and for this study, focus has been on discussing privacy from a perspective of how data is collected and later on being visible and maintained in IT-systems.[27]

Depending on if we consider a virtual space or the shared physical space the way we deal with privacy is usually different.[23] In the everyday physical sphere, an individual usually have a clear understanding of how to separate private and public space based on well-recognized social protocols but in the media spaces these well-established protocols are often vague and absent. Therefore, the latter is the most topical in this area of research and will be discussed throughout this paper.

Throughout the years there have resided many different defences and motivations on how we should perceive the value of privacy protection. Some defend it as intimate relationships and privacy goes hand in hand and some as privacy is mainly about not having undocumented knowledge on a personal level about one possessed by others. Rachels on the other hand defend privacy as it is necessary if we want to maintain the variety of social relationships with people in our surrounding.[18]][25][37][39] However, something almost all theorists agree on when it comes to privacy is the importance and great value of it even though they have clearly distinguished how privacy should play its role in legal theory, moral philosophy and public policy.[8]

To evaluate and discuss the upcoming results in a distinct way a delimited approach angle was chosen - where the term privacy stands in relation with the Information and Communication Technology (ICT) and is referred as data privacy.

2.3.1 Data privacy Data privacy, also known as informational privacy, can be considered as a type of privacy that refers to all the data about a person and more generally things that other people know about one person, especially when individual related data is included.[50] Data privacy is commonly used when discussing data protection and the legal, ethical and political issues where data is collected and disseminated. This is topical right now as the increment of new technology as well as the declining clarity on our privacy raises anxiety among many.[49] As today’s digital ecosystem grows where large amount of personal data can be collected and analyzed through complex search engines, in mobile devices and through visualization - distinct questions how this affect human security and rights emerge.[29] Who has access to

the data? Has someone accessed it without our knowledge? Can it be used in a context where it abuses other fundamental rights?

This leads us to a big challenge of data privacy - how can one, for instance a data processor, collect and utilize personal information and at the same time protect the data subject’s privacy preferences. The distinct predicament and issue from the proliferation of personal data in the light of the ethical aspect of sustained privacy is that courts often begins cases regarding information security and privacy with the assumption that information should be open for exchange.[31] On the other hand Floridi claims that data privacy should refer to the requirement that the personal information is stored and shared on the benchmark of a consent or necessity.[13] Furthermore Floridi continues with arguing that we should not focus too much on what informational privacy actually is. Rather we should think about how the new technologies impact the contextual privacy and what the acceptance level should be because this is what we can adjust to, for instance by laws and regulations regarding data protection as GDPR brings.

2.3.2 The right to privacy The right to privacy is often said to have its origin in Warren’s and Brandeis’ book “The Right to Privacy” in 1890 where they claimed that political, social and economic changes entailed a way to meet the new demands of the society and protect privacy of individuals.[51] At that time their approach was a response to recent technology development, such as sensationalist journalism and photography, that had changed the focus on how we should protect individuals. This is similar to the new threats that innovations within IT poses today and Westin argues that the underlying reasons why privacy is important is due to:

The claim of individuals, groups, or institutions to determine for themselves when, how, and to what extent information about them is communicated to others.[52]

Most people believe that privacy is a general human right due to its enshrinement in many of international human rights forums and together with data protection it constitutes to the core values of individuals together with the aim to reach democratic societies.[36][29] However, this right, when looking on cases concerning privacy issues that lawyers and philosophers has dealt with, has occasionally been falling under this purview. This has according to Marmor led to that some philosophers even have started doubting the underlying interest that the right to privacy should protect.[33]

What is also noticeable with the rapidly growth of today’s digital society is the fact that technology is not privacy neutral; technology has the potential to protect privacy but the digital development tends to do the opposite.[16] Garfinkel claims that it is more difficult, mostly because it is expensive, to build and maintain services which strive to protect individuals’ privacy than to disfigure it. According to Garfinkel there are two different choices on how the society can handle the current development where we can: 1) Do not allow networks, smartphones or credit cards to basically become hermits or 2) allow and share our personal data to the rest of the public domain.[16] However, Garfinkel argues that the first option is not an option we should consider due to the advantages the Internet gives us. That leaves society with one option - to regulate and consider the means through which we can benefit from increased data collection and connectivity while protecting the individual from exploitation for profit.

2.3.3 Loss of privacy To consider privacy as perfect, that is when an individual is totally inaccessible by other human beings, is unobtainable in any society according to Gavison.[17] On the other hand, the same applies on a total loss of privacy which is undesirable for anyone - in order to keep up relationships with other individuals there should be a balance between privacy and human interaction. This leaves us, Gavison means, with one important concept that is loss of privacy. A loss of privacy occurs constantly and an individual always loses privacy when he becomes the subject of attention. Even no new information has to become apparent since attention alone is the primary reason for losing privacy. For example, imagine a famous celebrity walking down a street with the ambition to be anonymous. The consequence of someone yelling, “There is X!”, pointing and referring to this celebrity would cause attention by the surrounding being focused on him. This would also make the temporary anonymity taken away and a loss of privacy to occur.

2.3.4 Privacy by Design (PbD) As the GDPR soon will start to apply a concept called Privacy by Design (PbD) has found its way into this new legislation. PbD bases in the stance of policy where if we look at a final product or services entire life cycle - privacy should be an integrated part in every single one of the different technological processes.[7][22] Privacy by design was first mentioned in the 90’s by Cavoukian but embedding privacy into all phases of a product's design, from the initial design process to the final prototype, was not that popular.[6] The privacy by design concept involves seven different principles on how a privacy-preserving design should look. These steps are summarized as follows: 1. Proactive not Reactive; Preventative not Remedial - PbD aims to prevent privacy risk to occur, not resolve them once they have occurred. 2. Privacy as the Default Setting - No kind of action should be required by the individual to protect their personal data, it should be built into IT-systems or business practises by default. 3. Privacy Embedded into Design - PbD should be embedded into - and be a core component - of the system and not any kind of add-on. 4. Full Functionality — Positive-Sum, not Zero-Sum - PbD seeks to demonstrate the importance of equating the need of both security and privacy, not set them into opposition (i.e. privacy vs security) of each other. 5. End-to-End Security — Full Lifecycle Protection - PbD ensures a strong security of personal data, from start to finish. 6. Visibility and Transparency — Keep it Open - Transparency should be present to the users and providers in all the different operations and components of a system. 7. Respect for User Privacy — Keep it User-Centric - PbD requires the design process of a system to keep the privacy aspect of the individual uppermost, all the time. 2.4 Information technology systems In order to look at organizations’ compliance and how modifications might affect the data privacy of a data subject it is necessary to examine the systems that collect and aggregate the personal data. This is often known as information technology systems. Information technology systems is any kind of system where information is either retrieved, stored or communicated to some extent, has been around for centuries. As far as back in 3000 B.C the Sumerians in Mesopotamia developed cuneiform, an

ancient writing system which was, in essence, a system for the communication of information.[5] However, during the last years a newcomer to this scene has arrived - namely the information technology (IT).[15] Information technology is a term that had its first appearance in 1958 when Leavitt and Whisler proposed this new notation and claimed that information technology is composed and includes three different parts which all should relate to each other: ● techniques that quickly processes large amount of information. This should subsume a high-speed computer. ● decision-making issues by taking mathematical or statistical methods into application. It could be techniques such as operations research and mathematical programming. ● simulation of high-order thinking through computer systems.[12] Information technology can be considered as a subset of Information and Communications Technology (ICT) and there are today digital systems of conveying information that operates on two levels - the machine code level and the interface level. Behind the interface level, each information system is different but in general there are two main types of information systems relevant for today's electronic age; open and closed systems. Moreover, Bellotti argues that the connectivity and sharing of personal data in the current information age is contributing to privacy concerns.[2] Bellotti says that the way sensitive or personal data is kept and used in systems poses an unethical threat to those people - which is almost anyone who uses computers. 2.4.1 Open system An open IT-system can be considered as the public portion of today’s internet where personal data is stored and managed in social networks sites, home-pages and tweets etc. and where the online identities on these sites struggle to be trustworthy linked to a natural person.[9] Moreover, Druschel et al. claims that an open system is a type of IT-system where it is unfeasible for the data subject who originated some sort of data, such as uploading a photo to a social media site, to keep it under control due to the widespread accessibility of information online.[9] One reason for this is that personal data in open systems easily can be digitally copied (digital copying is a big issue and generally cannot be completely prevented by technical means) or stored on local hard drives and therefore, often in different locations, be re-entered into the internet where the data subject did not expect to locate information about him/her. Druschel claims that there is no technical way to make personal data completely forgotten in an open system since data duplication, for instance by taking a photo of some personal data displayed on a screen, will always be impossible to elude. 2.4.2 Closed system A closed IT-system is a system-model where all users and controllers that access personal data to some extent, as well as all operators of a system that retrieves, manages and communicates personal data must fully respect the laws and regulations concerning management of personal information.[9] Druschel argues that implementing the ’right to be forgotten’ in a closed system is much more operable compared to an open system. This since personal data in a closed system, such as a corporate network where personal data is managed and stored exclusively by hardware and software which the organization owns, is only accessible by the employees and no third party. A closed system involves a difficulty to ensure compliance with the ’right to be forgotten’ by looking at technical aspects alone - it requires that

all users and employees whom has access to personal data must respect the privacy laws that is applicable in the region where the system is managed which is difficult to enforce. This difficulty of enforcing privacy laws which hinge upon people complying and incorporating the legal requirements relating to privacy in their everyday work points to a critical need to consider work practices and organizational contexts. 2.5 Research question The main purpose of this research project is to come up with a suggestion on how to implement the new requirements of the GDPR, with focus on the Article 17 concerning erasure of personal data in IT-systems by observing already existing systems. This is done by asking the question:

What are the key tension points in closed IT-systems that need to be considered when adopting the General Data Protection Regulation in order to protect data subjects’ privacy?

3. METHOD The research project is based on an exploratory case study methodology and is primarily conducted in two phases using an inductive approach. An inductive research approach is the opposite of a deductive approach and provides a procedure to analyze qualitative data in order to provide trustworthy and valid findings. This is done by starting of with the final stage of a scientific research, observing the empirical data, and work backwards to form hypotheses and theories.[47][45] This approach was applied when conducting my study where today’s situation concerning data protection laws firstly was elucidated and comprehended. Thereafter two types of closed IT-systems were investigated in-depth by an examination of the software system. The data gathered from the in-depth analysis were notes taken when examining the code, architecture and framework of the systems. This was later on followed up with semi-structured interviews in order to identify the key tension points according to the new regulation. By collecting data carefully in this phase, to

form a hypothesis (that is propositions to the deal with the tension points in my case) worked out much smoother. Figure 1 illustrates the entire research process which is described more in detail in section 3.1.

The reason why an exploratory case study was the most suitable approach is since it is a good tool to explore new fields within scientific investigation and where the data required to generate a hypothesis, which should be used for later investigation rather than just for illustrating, have not yet been obtained.[46][1] Furthermore, case studies provide a better understanding of different usability contexts when designing IT-systems and how new technology support these.[43]

Two different cases were chosen for the study. Just considering a single-case is vulnerable according to Yin since you will put “all your eggs in one basket” and even if two cases has the possibility to completely replicate each other the analytic conclusion coming from two cases is more powerful than just one case alone.[38] In this specific design, no direct replication of the two cases were striven to obtain. Rather the two cases would have a clear contrast from each other because if the findings support a hypothesized contrast, the following results are a foundation towards a theoretical replication.[10]

According to Resnik taking ethics into consideration when conducting researches is necessary due to many reasons.[40] For instance, adhering ethical norms is important since it promote trustworthiness and avoidance of error and at the same time limits the misrepresenting research data. Also, it provides mutual respect and credibility which are values that are essential when working collaboratively.

An ethical focus has been active throughout the entire study in order to assure the data collected to be as reliable as possible. This has mainly been accomplished while conducting the interviews where all interviewees will be anonymously represented in the report with a pseudonym based on if it as a man or women and no information can to any extent be connected with a subject. This confidential approach has been even more important due to the field of my research involving data security and ethics. The informants were assured anonymity and agreed upon this proceeding.

Moreover, since gaining access to the source code repositories for the both systems was essential for the analysis this was done in a carefully prepared way by signing a non-disclosure agreement with the company that maintained each system (that is the service provider). I only had the authorization to view the code but this could still be discussable from an ethical perspective since this in theory could lead to spreading information that is supposed to stay within the organization.

3.1 Case data collection and analysis In order to answer my research question the exploratory case study contained of four different phases which all contributed to some separately necessary findings. These stages, including a pilot study, an observation of two closed IT-systems, interviews and finally an analysis process are mentioned briefly below: Stage 1: Pilot study A pilot study was carried out to examine and test the methods which were going to be used in the upcoming observation. The pilot study was conducted in a way where I had two initiating meetings with the persons responsible for each of the two systems. At these meetings, they explained the purposes of the systems, showed me the architecture/schematics and demonstrated how personal data was handled today. The outcome of the pilot study was that some of the bullet-points used as a guide during the observation phase were removed and re-written as well as some interview questions were modified. Stage 2: Observation & in-depth analysis

The main study involved an in depth analysis and observation of the systems during two weeks. For this, a document which had been carefully developed and iterated a couple of times, containing different bullet points was mapped upon the systems to make sure it would give me enough material to answer my research question. The data collected from this stage were later on used to compare and analyze different tension points together with the data received from stage 3. Stage 3: Interviews To follow up the analysis of the systems, eight interviews were carried out to ensure the secondary data collection. The interviewees were all individuals who have worked within IT for many years and have to some extent been involved in one or in some cases both of the systems that I did analyze. The way the interviews were carried out were in a semi-structured manner mainly involving open questions since that is most suitable when the goal of the sessions is exploratory.[43] Semi-structured interviews is the most appropriate interview when conducting a qualitative research method and are considered as non-standardized where a list of themes with different questions are covered.

These questions might differ a bit depending on the organizational context and the flow during the interview.[41] To obtain an even balance, four people from each of the two cases

were interviewed and a gender equality of the interviewees was taken into consideration. This resulted in that 3 out of the 8 interviewees were women which can be considered as similar proportion on how the equality between women and men is today within the IT-sector in Sweden where one fifth are women.[28] Table 1 shows information about the interviewees, each with a pseudonym, describing which case they represent, their gender and role within the case. Stage 4: Qualitative analysis for exploring possible tension points To increase the validity of the results, data triangulation was used which means that the same information is cross validated as multiple theories and methods are used to explain the same phenomenon.[26] This was done by taking into consideration the data obtained from the pilot study as well as the analysis of stage 2 triangulated with the respondents’ data from the interviews together with the earlier conducted literature study. All data collected was thereafter used to quantify the findings which in addition led to that 4 different issues were discovered. These issues were finally used to evaluate and support an attempt to come up with the key tension points in closed IT-system that can affect the data subject's privacy in a negative way.

4. RESULTS In this segment the results from the exploratory case study will be presented, including data from the in-depth analysis of two

different closed IT-systems and relevant interview-material which were collected to ensure secondary data collection. 4.1 The cases The case study was mostly performed at the office of Agero Innovation AB which has its office located in the central parts of Stockholm city and the two cases that were analyzed and observed will be given the pseudonyms Case 1 and Case 2. Case 1 is a system where the users have the opportunity to take part of either a personal test or cognitive tests and is well used by human resources during recruitment processes. Case 2 on the other hand is a system used by customers to visualize different processes of companies. The criteria for selecting cases were based on the research question where only two systems were selected and required each of the systems under consideration to be: ● Closed systems; ● A system storing and managing personal data.

4.1.1 Case 1 Case 1 is a web based test system owned by Customer Company A but developed and administered by the service provider. The system is used for intelligence- and personality tests which registered candidates has to complete as a part of a recruitment process for a job. The outcome of the tests is for example used to visualize statistics on how well candidates performed and the system is used in several countries and is therefore translated to many different languages. In this case, Agero is the data processor for the system which means that they also have expanded obligations within the GDPR and may be exposed with the sanctions the new regulation brings. They must also proceed technical and organizational provisions and secure an appropriate security level. The data subjects in this case are the administrative users and the registered candidates. 4.1.2 Case 2 Case 2 is a Service as a Software (SaaS) categorized cloud system which offers their customers, both private persons and companies, a way to visualize processes such as showing different maintenance work for housing cooperatives. Another typical customer who uses the service are golf clubs, such as "Green Hills Golf Club" for example who uses the function in order to document and plan groundskeeping. This could range from tracking and visualizing work done on the grounds, the golf course and buildings, carried out previously by employees and contractors. This would help Green Hills to form an overall understanding of the necessary future maintenance work needed on the course and in the club buildings. The company that owns the SaaS is Customer Company B but it is being developed and maintained by the service provider. In this case, as well as for Case 1, the service provider is the data processor for the system and have therefore expanded obligations within the GDPR and may be exposed with the sanctions as well. The data subjects are all the customers who have registered to the service.

4.2 Data Privacy Related Issues The exploratory case study resulted in four different main issues which were identified after completing the in-depth analysis and interviews. These issues are based on how the two different systems are defective in relation to the GDPR and are supported from the outcome when checking the bullet points upon the two systems as well as quotes from the semi-structured interviews.

The issues will thereafter be the foundation when discussing the key tension points in closed systems in the upcoming discussion section.

The four different issues are; Data minimization, control over personal data, dealing with the right to be forgotten and uncertainty of the new regulation. Some of these issues were mostly noticed when doing the in-depth analysis and some while making the interviews. The findings and outcome from the in-depth analysis (see appendix 1) was useful in order to obtain a comprehensive picture of the result and discussion.

4.2.1 Data minimization One issue that was observed during the case study was the way the systems dealt with the principle of “data minimization” which derives from Article 5.1(b) that says that:

personal data should be collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes; further processing for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes shall, in accordance with Article 89(1), not be considered to be incompatible with the initial purposes (‘purpose limitation’);

The principle is about the collection of personal data should be limited to the necessary amount which is needed to fulfill the system’s purpose and no more. This involves saving data as long as it is really needed, and thereafter it should be automatically removed. This issue was noticed mostly when doing the analysis of the two systems but some questions from the interviews strengthen the clear difference between the systems.

For the system in Case 1 which is about creating intelligence and personality tests, working towards a fully data minimization has been an important task. The system does only store enough personal data which is necessary to identify who the candidate is and have a well working depersonalizing function. Moreover, people working within the system are aware of the importance minimizing personal data and have implemented an automated function that deletes unnecessary personal data regularly. The depersonalization function runs every midnight and depending on different criterias, such as if a candidate has obtained a job yet or not, and the amount of time he/she has been in the system. If a candidate is within a recruitment process at the time this user stays within the system for 6 months and if the candidate gets a job the user data is stored for 24 months. If the candidate does not get any job at all, he/she stays in the system for 6 months before the depersonalization process incorporates the user. Two of the experts interviewed who both are system developers within the system of Case 1 said the following:

My job as an IT-supplier is to help our customer to not save too much - Interviewee Ben

Interviewee Patrick said the following about the depersonalization process that facilitates the system to minimize personal data:

We have a job that runs everyday which takes into account a couple of parameters and removes the data that should not remain. Parameters which are included in the decision of policy - Interviewee Patrick

Case 2, the SaaS system working with visualization of different processes on the other hand is not working towards an active automatic removal of personal data today. The way they save

personal data, is already in contravention of the benchmarks discussed in the GDPR, and more than is needed to for the system to function and achieve its purpose. Even if some developers are aware of the defective work towards data minimization, nothing is actively changing at the moment.

There is no routine now on how to remove personal data. Neither do I know what counts as a reason to keep them in the system - Interviewee Chris

Like Chris, interviewee Tom found the lack of measures in place to protect sensitive data;

...However there is no control check when you write an address in the external documents for example. If you would like you could write hi my name is x and sensitive data - Interviewee Tom

Case 2 does not have an automatically deletion function which means that personal data might be stored in the system even if there is no legitimate reason to keep it.

4.2.2 Control over the personal data Having full control over personal data is an important topic in the new regulation and something organizations need to be aware of in order to be fully compliant. This since the GDPR involves new arrangements on what an organization must do if any data breach occurs or if they to any extent loses control of the personal data they store and manage. If any incident where personal data may reach an unauthorized person this must be reported to a regulatory authority within 72 hours by the data controller according to Article 33. None of the representatives from the closed systems had clear directions on how to handle any incident like this.

...since it has not been a discussion about that, there is no routine for it - Interviewee Chris

Representatives working with Case 1 all answered differently on the question who would be informed if a data breaches occurs which shows a clear obscure on how to deal with this issue. Furthermore, the representatives from Case 1 are aware of the risks which a trespass may involve, such as individuals’ answers on personality and intelligence tests which is considered as sensitive data according to the GDPR.

...people have answered on personality tests and intelligence test which include names and such. You don’t want that kind of information to reach the wrong person - Interviewee Sofia

Interviewee Ben was also aware of the potential data privacy issues which a weak management of personal data might lead to - for instance as a consequence of a data breach;

...since it manages personality tests, it is possible that answers regarding these tests and intelligence tests might leak out. And privacy problem there… - Interviewee Ben

The analysis of Case 2 and the outcome from the interviews with representatives from the system showed a clear obscurity on where the personal data is stored and who might be able to reach it. System representatives have the possibility to upload their own documents which can cause undesired information to spread. Interviewee Tom says that:

It could in the theory be company information that leak which is most likely what someone is interested in.

4.2.3 Dealing with the right to be forgotten

In Case 1 a request of being removed would result in that the user and its personal data is set with the flag depersonalization. However, someone with enough knowledge could use the raw data that come from the depersonalization process and project the data onto a reading module.

When a data subject requests to be removed from the system of Case 2, the personal data is removed but if the data subject has performed an action before, such as uploading any document to the system, the author of the document still contains the personal information which conflicts with the GDPR. This since there is enough information left in order to link it to a natural person. None of the systems in Case 1 or 2 log every data request concerning personal data which might obstruct the possibility to ensure a completely removal process of personal data.

4.2.4 Uncertainty of the new regulation Even if it is not a demand in the new regulation that everyone involved in a system has to have knowledge about the new bylaw, a fundamental knowledge is essential in order to work towards technical and organizational goals in a correct manner. A commonality between the Case 1 & 2 was that the majority of the interviewees are unsure and knew relatively little about the GDPR. When asked what they knew about the new regulation, the common answer was that they have heard about the ’right to be forgotten’ but not much more.

Not that much since I do not work with personal data in a wider extent, but I understood it has implied some juridical consequences for system that has to do modifications - Interviewee Chris

Furthermore, interviewee Emelie had a vague understanding of the new regulation but had heard of some key concepts briefly;

You have a right to be forgotten and can demand it, the company must deal with it - Interviewee Emelie

Interviewee Sofia was even more concerned about the upcoming regulations and felt unprepared towards the new policies;

I feel like we don’t have a project about it [GDPR] yet. I don’t know about it, it is confusing where the responsibility lies - Interviewee Sofia

Looking further, GDPR says it is important to determine the legal basis for processing personal data and document this, suggesting using a consent form which states that the data not be used for other purposes than what the consent form says. The analysis and interviewing shows that Case 1 has a consent form but the system representatives are unsure if the personal data is used for other purposes and exactly how they should comply with the new demands.

We should obey the bylaws there is today but have not made any standpoint. I think we obey the old directive, PUL, but do not obey with the new bylaws. - Interviewee Sofia

Case 2 does not have an active consent form today. This conflicts the new bylaw when it comes to a lawful processing mentioned in Article 6 which includes the demand of a data controller to inform the data subject what their personal data will be used for, often via a consent form or contract. Interviewee Carl, who is the product manager of the system analyzed in Case 2, said that there has been a consent form before but it was removed in a system update since they did not find it important to maintain. Carl says that a customer of the system is responsible for its own account. 5. DISCUSSION This report aimed to locate some deficiencies in closed IT-systems and together with the theoretical framework find out how defective systems and organizations handling them can affect the privacy of a data subject in a bad manner. To do this an attempt to answer the research question regarding what the key tension points in closed IT-systems that needs to be considered when adopting the GDPR in order to protect the data subjects’ privacy was done based on the findings during the exploratory case study by suggesting propositions to the discovered issues. These propositions are equate to the hypothesis as mentioned in the method section as being the final part of an inductive research approach and hopefully they can provide a starting point for possible future research within the field. Due to the relatively small scope of the study, issues regarding how to operate applications within open systems were not addressed but one should understand the serious challenges of upholding data privacy within open environments such as Facebook and Twitter. 5.1 Privacy deficiencies in closed IT-systems The discussion below is based on the collected data from the result section in relation to how these findings will affect the data subjects’ data privacy. 5.1.1 Data minimization and encryption The empirical findings from the case study shows that data minimization as well as a strict storage limitation of personal data is elaborated with but can be crucial for organizations working with closed IT-systems and that stores too much data. There is a subtle difference on how to judge if too much personal data is saved or not and this is something which organizations really have to put effort into in order to be fully compliant with the GDPR. For example, systems handling unencrypted sensitive data such as Case 1 has to be even more careful and only retain the minimum information needed to fulfill the purpose of the service.[24] Even if it seems like the system is totally data minimized, it is in some cases enough with only three attributes - age, gender and zip-code in order to uniquely identify a natural person.[21] If there then is unencrypted sensitive information stored about this person, data breaches can distinctly provoke our right to privacy.

Bellotti means that the need of system security and software protection is important in those cases and when the system is at the foundation of information sharing between individuals (customers) and companies (service providers), not only a data breach can have serious consequences on our privacy.[2] Since the closed systems studied in this project involve a lot of different people with insight in the information sharing and to whom it might reach it is important to ensure that the involved individuals are aware of to whom, when and how the personal information is accessible by others.[2]

Furthermore, Bellotti means that the risks when systems in a deliberate or unwitting manner are exposing the data subject’s privacy is crucial - although it sometimes occurs inadvertently. Since the new regulation requires organizations to implement technical measures to provide the appropriate protection of the

stored personal data, one possible solution to this could be an implementation of a more commonly used software protection model. This could for example be, as Bowyer proposed, things like private and public-key encryption or mechanism protecting certain files.[4] If organizations are unsure if they store too much information, one suggestion is to encrypt as much information as possible just to make sure the natural person’s privacy is fully protected - and to avoid sanctions.

Therefore, a substantial way for organizations to comply with these parts of the GDPR is to analyze and map out which kind of personal data that is being managed and rather encrypt more data than nothing at all. Especially if it is sensitive data.

5.1.2. Uncertainty of the new bylaw How can the data subject be assured a fully protection of their privacy if the people working to protect is do not know which bylaws to follow? Since the GDPR was designed to protect the data subjects’ privacy and redo the way organizations approach questions about data privacy, the fact that all the interviewees indicated that they only had a vague or no knowledge at all about the new regulation can cause troubles ahead. This finding is in line with a comprehensive survey which Dell conducted in October last year which shown that more than 82 % of global IT and business professionals only know a few or nothing at all about GDPR.[19] Also, the survey showed that 97 % of all companies do not have a plan when the GDPR starts to apply next year.

This connects with one big issue noticed during the in-depth analysis that was that no data controller was assigned or planned to be in neither of the systems. Since companies has to appoint an individual or a joint of individuals within the organization with a specific task to look after data protection questions this can be an issue if nothing is done. For example, the GDPR says that when a breach of security occurs this has to be reported to the supervisory authority within 72 hours and having an educated data controller knowing the bylaws will most likely assure this to happen. Even more critical is if the security breach most likely could result in a high privacy risk for the individuals exposed - then these individuals also have to be informed within the time limit of 72 hours. But how can one know if a loss of privacy is occurring? Loss of privacy is according to Gavison happening when other people gain information, pay attention or gain access about an individual and when the data subject’s personal data is considered less valuable than the corporates’ personal data as in the SaaS system of Case 2, this will most likely cause implementation issues of the GDPR.[51] Therefore this will probably be a question of interpretation for the data controller to realize what kind of measures that should be taken and a well educated data controller/s should be of high prioritization in order to protect the involved individuals privacy.  

Furthermore, as the results showed that the usage of valid consent form to explicit demonstrate how data is collected and for what purposes the data is used was not that obvious in neither of the cases this could lead to ambiguities that could affect the privacy of an individual in a negative way. Firstly, it is the data controller according to Article 7 who must be able to prove consent according to GDPR and also the consent has to be withdrawn if an individual requires it. But if there is no data controller assigned, this will break. Secondly, as Gavison claims that a legal protection for privacy involves the reasons advanced for its protection based on a coherence of privacy as a value - without informing how the personal data is used and for what purposes, how can one take liability for invasions of it? A suitable response to this in order to be fully compliant with the GDPR is educate the employees within the organizations and assign a data controller to deal with questions regarding the new regulation.

Finally, since a consent form on how personal data is managed is required - all systems should include one. Unless they will most likely be penalized for it. 5.1.3. Approaching the right to be forgotten One of the most common concerns is how Article 17 will come into play, and how a data subject will have the right to request erasure of all the personal data which can be linked to them. A clear pattern from the study of both Case 1 & 2 was that organizations working with closed IT-systems have implemented different deletion functions but will that be enough to comply with the GDPR? What about the human factor?

The ’right to be forgotten’ is a principle which enables an individual to request deletion of personal data if there is no longer compelling reason for further processing of it. Both Case 1 and Case 2 have routines to manually remove personal data if an individual would request it, but all the personal data is not being completely removed and it is still possible to track an individual after a deletion - which might conflict the new bylaw. That the ’right to be forgotten’ can be ensured by technical means alone seems unlikely which therefore contributes to an even more unwieldy situation for systems embedded in complex workplace organizations that often rely on the individual to gain access to the shareable data as well.[2] Even if data subjects within closed systems should not be concerned about issues that can affect their privacy in open systems such as someone taking a photograph of their personal data while it is being displayed on a screen, they cannot be completely sure that an authorized person would not do it. This connects to Garfinkel’s view on privacy and the role technology plays - where technology itself is not the element that violates our privacy; it is the individuals who use that technology in a conflicting and unethical manner who does.[16] This potential issue could be remedied by, for example, including a consent form during registration, that the employees with access to the personal data that ensures they respect the applicable privacy laws within the GDPR.

Other problems detected during the case study were that none of the systems logged every data request concerning personal data. While not doing this, the reliability of assuring a right to be forgotten towards the data subject obstructs. What also was questionable was the fact of uncertainty and arguable methods on how all personal information and derived information such as backup copies were tracked and later on removed. For instance, the system of Case 1 had a depersonalizing procedure, but it uses a functional database called Event Store where all events that occurs within the systems is stored. Anyone who might reach this functional database has the opportunity to project the raw data stored onto a reading model - and the personal data that is supposed to be completely deleted can be visible. If organizations do not have a full understanding about this, then they can never ensure their customers’ the ’right to be forgotten’. This since the individual will not know if their personal data is stored on an archival storage platform or on an employee’s computer for example. This predicament will most likely become intensified as the new bylaws start to apply and will probably contribute to the privacy anxieties as well as a widely doubtfulness whether if the organizations storing your personal data can assure the ’right to be forgotten’ or not. In fact, as Latonero and Gold points out, it is not the technology itself that will provide a sustainable answer to the social, structural and contextual problems - it is the technologists and organizations that should “do good” and for example make sure to implement safeguards like Privacy by Design.[29] Latonero and Gold claims that if the service providers lack knowledge within an area, for instance on how to deal with the new regulation, they should make sure to understand the ethics and obligations on how to apply technologies in a correct manner.

5.2 Future work Given that an organization are soon legally obliged to do what they can to set up plans to be fully compliant with the GDPR it would be interesting to see if a fully compliant system actually can assure the data subject’s right to their own personal data and privacy. This could for instance be done by doing a more comprehensive case study, with more than just two cases, and locate additional privacy-related issues from these.

Moreover, it would be interesting to investigate the difference between open and closed IT-systems since open system face considerably more issues when it comes to mapping out all locations where personal data is stored and therefore a fully compliance with the GDPR will most likely be harder to achieve.

As we have moved into a heavily digitalized era involving a constant connectivity it would be interesting if future research could also investigate what the most critical points might be when open and closed systems meet. 6. CONCLUSION As threats to the data subjects’ privacy nowadays can be reality as soon as you register an account onto a social network or signing up to a cloud service the upcoming GDPR will strive to give back the individual control over its own personal data. But individual control over data does not happen independently from the organizations that manage, store, and handle that data. these organizations storing and managing this information face the hard reality that just implementing a deletion function and minimizing the amount of data will not be enough to fully comply with the new regulation.

This paper has tried to identify critical issues in closed IT-systems today and suggest a couple of propositions, which are general in its context and therefore can be applicable by organizations working with similar systems - on how to face these challenges. To conclude the findings, the key tension points observed in Case 1 was that the system representatives were unsure on how to deal with data breaches which could lead to a loss of privacy for the data subject, they were not that well informed of the new demands that accompanies the new regulation and finally - it is still possible to acquire personal data through a projection procedure, even if it is said to be deleted. The key tension points in Case 2 was the the system did not have any automatic deletion function which means that personal data may be stored in the system forever, even if there is no legitimate reason to keep it. Moreover, the system representatives were unsure where all the personal data was stored and who might reach it and finally but probably most concerning towards the new regulation - no consent form was actively used today.

The four propositions suggested to be taken into consideration when adopting to the GDPR and to guarantee the data subject its right to privacy are: Proposition 1 - Analyze which kind of personal data that is being managed and rather encrypt more data than nothing at all, especially if it is sensitive. Proposition 2 - Educate the organization and assign a data controller. Proposition 3 - An updated consent form on how personal data is managed is required and all systems should include one. Proposition 4 - Ensure a complete deletion if a user request his right to be forgotten and make sure every employee respect the applicable laws. The outcome from this was that an active data minimization and encryption of personal data is a safety measure for organizations

who are unsure if they store too much data. Furthermore, education within the organization will be a key for those who wants to be fully compliant and finally - the ’right to be forgotten’ cannot be assured by only technical means. Therefore, it is critical that organizations find ways to assure this and let the customers be aware of how they achieve this.

As the different issues were discovered and supported from the exploratory case study it is obvious that the new bylaws most likely will concern and affect how organizations set up their budgets and plans on working towards data protection laws in the nearest future unless they want to be afflicted by the high penalties as a non-compliance can cause.

Finally, as the rapid changes in technology create new privacy risks it is essential for the organizations to keep a sustainable social approach on how they work towards applying data protection laws. The importance of a social trustiness towards the source who collects, stores and manages the personal data is fundamental whether a data subject decides to pass away their personal information to one another or not. 7. REFERENCES

[1] Baškarada, S. 2014. Qualitative Case Study Guidelines. Qualitative Report, 19(40), 1–25. [2] Bellotti, V. 1996. What you don’t know can hurt you: Privacy in collaborative computing. In People and Computers XI (pp. 241-261). Springer London. [3] Benjamin, L. M. 1991. Privacy, computers and personal information: Towards equality and equity in an information age. Communications and the Law, 13 (2): 3-16. [4] Bowyer, K. W. 1996. Ethics and Computing: Living Responsibly In A Computerized World, rev. ed. New York, NY: IEEE/Wiley. [5] Butler, J. G. 1997. A History of Information Technology and Systems. Retrieved from https://tcf.ua.edu/AZ/ITHistoryOutline.htm [Accessed: 2017-03-07] [6] Cavoukian, A. 2009. Privacy by Design - The 7 foundational principles - Implementation and mapping of fair information practices. Information and Privacy Commissioner of Ontario, Canada, 3(2), 247-251. [7] Danezis, G., Domingo-Ferrer, J., Hansen, M., Hoepman, J. H., Metayer, D. L., Tirtea, R and Schiffner, S. 2015. Privacy and Data Protection by Design-from policy to engineering. arXiv preprint arXiv:1501.03726. [8] DeCew, J. Privacy. The Stanford Encyclopedia of Philosophy (Spring 2015 Edition), Edward N. Zalta (ed.). Retrieved from https://plato.stanford.edu/archives/spr2015/entries/privacy/ [Accessed: 2017-03-10] [9] Druschel, P., Backes, M and Tirtea, R. 2012. The right to be forgotten-between expectations and practice. Report of the European Agency for Network and Information Security (ENISA). Retrieved from http://www.enisa.europa.eu/activities/identity-and-trust/library/deliverables/the-right-to-be-forgotten [Accessed: 2017-03-09]

[10] Eilbert, K. W. and Lafronza, V. 2005. Working together for community health—a model and case studies. Evaluation and Program Planning, 28(2), 185-199. [11] European Parliament. 1995. Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data. Official Journal of the European Union, L281/31: 31–50. [12] European Union. 2016. Regulation 2016/679 of the European parliament and the Council of the European Union. Official Journal of the European Communities 2014, April: 1–88. [13] Floridi, L. ed. 2014. Protection of Information and the Right to Privacy-A New Equilibrium? Vol. 17. Springer. [14] Gabel, D and Hickman, T. 2016. Key definitions - Unlocking the EU General Data Protection Regulation. Retrieved from https://www.whitecase.com/publications/article/chapter-5-key-definitions-unlocking-eu-general-data-protection-regulation [Accessed: 2017-04-06] [15] Galliers, R and Leidner, D. 2009. Strategic Information Management: Challenges and Strategies in Managing Information Systems, fourth ed. Routledge, United Kingdom. [16] Garfinkel, S. 2000. Database nation: The death of privacy in the 21st century. O'Reilly Media, Inc. [17] Gavison, R. 1984. Privacy and the limits of law. Philosophical Dimensions of Privacy. An Anthology, 89(3), 346–402. [18] Gerstein, R. S. 1978. Intimacy and privacy. Ethics, 89(1), 76-81. [19] Global Survey, Data Privacy Professionals, and European Customers September. 2016. GDPR : Perceptions and Readiness. September: 1–35. [Accessed 10 April 2017] [20] Heywood, D . 2016. Obligations on data processors under the GDPR. Retrieved from https://www.taylorwessing.com/globaldatahub/article-obligations-on-data-processors-under-gdpr.html [Accessed: 2017-04-06] [21] Hochbein, K. 2016. Can We Trust the Systems? Retrieved from https://www1.lehigh.edu/news/can-we-trust-systems [Accessed: 2017-04-20] [22] Hoepman, J. H. 2014, June. Privacy design strategies. In IFIP International Information Security Conference (pp. 446-459). Springer Berlin Heidelberg. [23] Hudson, S. E and Smith, I. 1996, November. Techniques for addressing fundamental privacy and disruption tradeoffs in awareness support systems. In Proceedings of the 1996 ACM conference on Computer supported cooperative work (pp. 248-257). ACM. [24] Information Commissioner's Office. 2017. The Guide to Data Protection. Retrieved from https://ico.org.uk/media/for-organisations/guide-to-data-protection-2-7.pdf [Accessed: 2017-03-22]

[25] Inness, J. C. 1996. Privacy, intimacy, and isolation. Oxford University Press on Demand. http://www.oxfordscholarship.com/view/10.1093/0195104609.001.0001/acprof-9780195104608 [Accessed: 2017-04-21] [26] Jick, T. 1979. Mixing Qualitative and Quantitative Methods : Triangulation in Action Mixing Qualitative and Quantitative Methods : Triangulation in Action. Qualitative Methodology 24(4), 602–611. [27] Kemp, R and Moore, A. D. 2007. Privacy. Library Hi Tech, 25(1), 58–78. [28] Larsson, L and Delin, M. 2015. IT-branschen hotas av brist på kvinnor. Retrieved from http://www.dn.se/ekonomi/it-branschen-hotas-av-brist-pa-kvinnor/ [Accessed: 2017-04-22] [29] Latonero, M and Gold, Z. 2015. Data , Human Rights & Human Security. 2012: 1–14. [30] Leavitt, H. J and Whisler, T. L. 1958, Management in the 1980s. Harvard Business Review 11. Retrieved from http://hbr.org/1958/11/management-in-the-1980s [Accessed: 2017-03-21] [31] Levesque, R. J. 2016. Adolescence, Privacy, and the Law: A Developmental Science Perspective. Oxford University Press. [32] Mantelero, A. 2013. The EU Proposal for a General Data Protection Regulation and the roots of the ‘right to be forgotten’. Computer Law & Security Review, 29(3), 229-235. [33] Marmor, A. 2015. What is the right to privacy? Philosophy & Public Affairs, 43(1), 3-26. [34] Mazzone, A. 2017. The EU GDPR & Third Party Risk. Retrieved from http://blog.aravo.com/the-eu-gdpr-third-party-risk [Accessed: 2017-04-07] [35] Nissenbaum, H. 2009. Privacy in context: Technology, policy, and the integrity of social life. Stanford, CA: Stanford University Press. [36] Nwauche, E. S. 2007. The Right To Privacy in Sweden, 1(1), 1-7. [37] Parent, W. A. 1983. Privacy, Morality, and the Law. Philosophy & Public Affairs, 12(4), 269–288. [38] Phelan, S. 2011. Case study research: design and methods. Evaluation & Research in Education, 24(3), 221–222. [39] Rachels, J. 1975. Why privacy is important. Philosophy & Public Affairs, 4(4), 323-333. [40] Resnik, D. B. 2011. What is ethics in research & why is it important. In The national. Retrieved from https://www.niehs.nih.gov/research/resources/bioethics/whatis/index.cfm?links=false [Accessed: 2017-03-21] [41] Saunders, M., Lewis, P and Thornhill, A. 2009. Research methods for business students 5th Edition. [42]Schwartz, P. M. 2003. Property, privacy, and personal data.

Harv. L. Rev., 117, 2056. [43] Sharp, H., Rogers, Y and Preece, J. 2007. Interaction design: Beyond human-computer interaction. Chichester, Wiley. [44] Shoor, E. 2015. Narrowing the Right To Be Forgotten: Why the European Union Needs To Amend the Proposed Data Protection Regulation. Brook Journal of International Law, 39(1), 487–519. [45] Social Research Methods. 2006. Deduction & Induction Retrieved from http://www.socialresearchmethods.net/kb/dedind.php [Accessed: 2017-03-15] [46] Streb C. K. 2010. Exploratory Case Study. Encyclopedia of Case Study Research. 373-375. [47] Thomas, D. R. 2006. A general inductive approach for analyzing qualitative evaluation data. American journal of evaluation, 27(2), 237-246. [48] UNCTAD. 2016. Data protection regulations and international data flows: implications for trade and development. Retrieved from http://unctad.org/en/PublicationsLibrary/dtlstict2016d1_en.pdf [Accessed: 2017-03-10] [49] Van den Hoven, J., Blaauw, M., Pieters, W and Warnier, M. 2016. Privacy and Information Technology. The Stanford Encyclopedia of Philosophy (Spring 2016 Edition), Edward N. Zalta (ed.). Retrieved from https://plato.stanford.edu/archives/spr2016/entries/it-privacy/. [Accessed: 2017-03-12] [50] Von Locquenghien, K. 2006. On the potential social impact of RFID-containing everyday objects. Science, Technology & Innovation Studies, 2(1), 57-77. [51] Warren, S. D and Brandeis, L. D. 1890. The Right to Privacy. Harvard Law Review, 4(5), 193-220. [52] Westin, A. F. 1968. Privacy and freedom. Washington and Lee Law Review, 25(1), p.166. 8. APPENDIX Appendix A - Comparison of the two systems according to the bullet points Below follows the 12 different bullet points that were checked upon the systems and the finding for each of the two systems.

Bullet point Finding Case 1

Finding Case 2

Which kind of personal data is being stored and managed in the system today?

Email-address as username. First name and surname is also filled in when a candidate is being

In this system, a username contains first name and a surname and the user is also filling in e-mail, organization

registered. Password, but not considered as personal data. ID might indirectly identify a natural personal and is therefore a personal data object.

and a password (the latte is not considered as personal data)

How is the personal data collected and who might be reached with it?

The personal data is collected when a candidate of the system is creating a user. The database storing the information can be reached by the developers, the management organization, a couple of people at the IT-department of Mercuri Urval who has authorization to the data. Finally, the normal users (such as an employee of Agero) has access to the system through the authorization model that exists.

The personal is collected when a person is creating an account in the system. In order to get access to the personal data you need authority to the code which you only can get if you work at Agero and get it by the system administrator.

Where is the personal data stored in the system?

There are three ways where personal data is coming into the system and thereafter stored. Firstly, it is through a role called

The personal data is stored in a database and to some extent information is also managed in external word/excel-files.

candidate when a user log into the systems using his/her id and password. Furthermore, the admin role stores all users including their personal data. Finally the systems has a integration module including clients, projects and participants where personal data is stored in.

How long is the personal data stored and when is automatically removement occurring?

Delete function is implemented through a depersonalize method which runs every midnight. Depending on different criterias, a candidate might remain in the system for different amount of times. If the candidate is in a recruitments process he/she remains in the system for 6 months. If he/she gets a job the user remains in the system for 24 months and if the candidate doesn't get any response, he/she stays in the system for 6 months before a depersonalize process

There is no time limit when personal data is automatically removed. Therefore personal data can be stored in an infinity.

includes the candidate.

How is all exact or derived copies of a personal data item removed in a manually manner?

This is handled by mercuri urval which owns the system - the candidate calls them and they use a delete function to remove the data object from the database.

If a user wants to get removed, he/she contacts the product owner who manually removes the user object containing personal data from the database.

Is there a data controller appointed today or planned to be?

No. No. It will be offered to the customer from the service provider and depends on the customer if it should be appointed or not.

Is every data request (concerning personal data) authenticated in the system?

Yes. Yes. A data request is always validated upon the user’s rights in the system. Validation happens against the user’s role.

Is every data request (concerning personal data) logged in the system?

No. No. The only time personal data is logged is when a user is logging into the system.

Is it possible to track and remove all copies of personal information and any derived information (i.e. backup copies stored on archival storage platform or cached copies stored on local disks of any employee's computer etc) in the system today?

Yes and no. In theory it is possible to do this but the system is using a functional database called eventstore where all events that occurs within the system is stored which makes this

Backup copies of the server and the content in the database is occurring. However, personal data stored is not searchable unless they are reinstalled on the server.

hard. In order to understand the raw data in the event store you need to project the data onto a reading model.

Is personal data stored on any discarded storage equipment (i.e. any magnetic and/or flash disk devices in discarded or recycled USB-sticks, smartphones or notebooks etc.)?

No. No personal data is being sent to other system. The only person data the systems stores is for the log-in purpose and for object and document ownership.

Is personal data transferred to other systems, parts or to countries outside the European Union?

No. No. Personal data is remained in the system only and the only country which the system is active in is Sweden.

Is personal data handled in searchable external files such as written document software’s (word- or excel files etc)?

No personal data is handled in these types of external files.

No. However, customers can add word and excel documents containing personal information but it is not searchable from outside. Only users in the system can access these documents.

Appendix B - Questions asked during the interview ● Grundläggande information: ○ Namn? ○ Ålder? ○ Yrkesroll/titel? ○ Företag/kund? ● Dataskyddserfarenhet: ○ Har ni någon erfarenhet gällande dataskydd, exempelvis genom personuppgiftslagen (PUL)? ■ Behandlar du personuppgifter på daglig basis? ■ Vilka personuppgifter?

○ Vad finns det för riktlinjer och policies för hantering av personuppgifter idag? ■ Följer ni dem? ○ Hur hanterar ni dataintrång idag? ■ Vilka informeras? ■ Vilka risker skulle ett dataintrång kunna ha för verksamheten? ○ Hur jobbar ni med transparens mot kunder? ■ Har en kund möjlighet att få tillgång till all information som lagras om en? ○ Har ni varit med om att straffas för bristande dataskydd tidigare? ■ Om ja - vad hände? ○ Finns det någon personuppgiftsansvarig idag? (kunnig inom juridik och it, mer ansvar och skyldigheter) ■ Om inte - kommer det tillsättas någon inom det närmaste året? ○ Varför anser du det vara viktigt att spara data? ○ Vem ska ha åtkomst till uppgifterna inom organisationen? ● Dataskyddsförordningen: ○ Vad känner du till om den nya förordningen? ■ Har du fått tillräcklig information om hur du ska jobba med de nya reglerna? ■ Upplever du det vara svårt att veta vilka riktlinjer man ska följa? ○ Vad tycker ni om dataminimering och hur arbetar ni med det? (inte samla in mer persondata än nödvändigt) ■ Vad skulle ni kunna förbättra? ■ Föreligger några särskilda risker för den personliga integriteten hos de registrerade? ○ Känner ni till de kännbara straffen för bristande uppfyllelse av kraven från GDPR? ○ Håller ni alla personuppgifter uppdaterade i förhållande till dess ändamål? ○ Tror ni att ni är helt redo för alla de krav som dataskyddsförordningen för med sig? ■ Om inte, vilka saker tror ni att ni måste förbättra? ○ Vad har ni för strategier när ni jobbar mot den nya dataskyddsförordningen? ■ Vilka? ■ Varför? ■ Finns det inbyggt skydd i systemet/n idag? (Privacy by design) ■ Krypteras känslig data? ○ Har ni full koll på vart alla personuppgifter kommer ifrån och hur de uppdateras? ■ Behandlar ni uppgifter om barn/minderåriga? ■ Hur inhämtar ni samtycke? ● Används personuppgifterna till ändamål utöver det som finns i samtycket? ○ Vad tror du den nya dataskyddsförordningen kommer få för effekt för privatpersoner? ○ Vad är ni villiga att ge upp i systemen idag för att vara helt medgörliga med dataskyddsförordningen? ○ Hur tror ni dataskyddsförordningen kommer ändra ert dagliga arbetssätt? ● Rätten att bli bortglömd: ○ Hur hanterar ni idag en förfrågan från kund att bli borttagen? ■ Hur lång tid brukar det ta? ■ Till vem skickas förfrågan? ■ Hur väl uppfyller ni en snabb och total borttagning av persondata?

○ Kontaktar ni en tredje part idag om att persondata ska tas bort? ■ Om inte, finns det någon plan på att göra det? ○ Tas personuppgifter bort om du inte har en laglig grund att behålla dem? ○ Hur säkerställer vi att informationskrav och anmälningskrav uppfylls?

 

 

 

 

 

 

 

 

 

www.kth.se