D6 - PACT Project · for the system and individual components. ... 2. Define the preliminary DSS architecture and subsequently define the technical and functional

D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria

PACT - Public perception of security and privacy: Assessing knowledge, Collecting evidence, Translating research into action Project co-funded by the European Commission within the 7th Framework Programme – SECURITY theme

2nd Reporting period WP6 Decision Support System (DSS) Responsible Partner: NCSRD Contributing partners: ATOS, CSSC, CIES, KEMEA, HUJI, RAND, PRIO, UOW Due date of the deliverable: February 28th 2014 Actual submission date: October 16th 2014

Dissemination level: PU

D6.1 DSS Architecture, Technical Requirements and Definition of Validation Criteria PACT project – GA: 285635

2

Document Management

D6.1 “DSS Architecture, Technical Requirements and Definition of Validation Criteria” Task: 6.1 Leader: NCSRD – Other contributors: ATOS, CSSC, CIES, KEMEA, HUJI, RAND, PRIO, UoW Authors: NCSRD: Dimitris M. Kyriazanos, Olga E. Segou, Anastassios Bravakis, Constantinos Rizogiannis, Stelios C. A. Thomopoulos. ATOS: Jaime Martin Perez, Kaan Dulgar, Adem Yasar Mulayim. CIES: Daniel Deering.

PROJECT FULL TITLE

Public Perception of Security and Privacy: Assessing Knowledge, Collecting Evidence, Translating Research into Action

PROJECT ACRONYM PACT

Collaborative Project funded under Theme SEC-2011.6.5-2 “The relationship between human privacy and security”

GRANT AGREEMENT 285635

STARTING DATE 01/02/2012

DURATION 36 months


3

Executive summary

The current report is the output of PACT task T6.1 “DSS Architecture Technical Specifications and System

Validation definition” and the first deliverable of WP6 “Decision Support System (DSS)”. Following

assessment of theoretical knowledge and collection of empirical evidence, PACT project proceeds with

analysing the main factors that affect public assessment of the security and privacy implications of given

security technology. By integrating the results of this analysis into a software tool, WP6 is responsible of

developing a prototype DSS, which may help end users to evaluate pros and cons of specific security

investments also on the basis of the societal perception of privacy and liberty. T6.1 begins by collecting user

requirements, defining the DSS architecture, functional and technical requirements and corresponding

validation methodology and criteria.

The PACT DSS is a tool for assessing the impact of future security technology investments and deployments

in terms of privacy, ethics, social acceptance and public perception. It aims at assisting the decision process

of experts through efficient visualisation and user-DSS interaction powered by the software

implementation of PACT’s theoretical and empirical findings. The PACT DSS is a core tangible output of

PACT research activities and it implements as a software tool the Privacy Reference Framework for Security

Technologies (PRFST) toolbox described in D5.2 “PACT Privacy Reference Framework for Security

Technology”. The PACT DSS is context-specific and it is based on the three contexts specified in the PACT

survey, namely the Travel, Healthcare and Internet. However, the PACT DSS is modular and extensible,

having the ability to integrate future empirical studies in other contexts, or enrich the existing contexts

without overhead or need for extended offline time for the PACT DSS system. This is achieved through

optimised database design, an extensible data model and a modular architecture, which allow quick

inclusion of additional datasets and knowledge input.

Task 6.1 was responsible for kick starting WP6 activities during the joint PRFST-DSS workshop held in

Madrid in October 2013. A key event in the WP6 timeline, the workshop confirmed the alignment of

research activities between WP5 and WP6 and established the smooth inclusion of PACT’s PRFST into the

DSS design. Moreover, during the workshop end user perspective was integrated into the DSS design

through round table discussions, dedicated presentations and questionnaires. Following the workshop,

user requirements were extracted and documented in this report, based on feedback and in collaboration

with all PACT experts groups (Stakeholders Advisory Group - SAG, Privacy Advocate Panel – PAP, and High

Level Policy Panel - HLPP) and PACT End User Partners. In parallel, T6.1 also reviewed and assessed relevant

work in the field, establishing connections and organising online meetings with relevant EU research

projects such as PRISMS1, PARIS2 and VALUESEC3.

Building on the results of these activities, T6.1 defined the preliminary PACT DSS architecture based on user

needs, state of the art techniques and best practises. The PACT DSS architecture follows a three-tier

approach and is broken down into the following components: (i) User Layer, Human Machine Interface

(HMI), Dialogue Management & Visualisation Module, (ii) Decision Trees & Engine Module and (iii)

Knowledge Management Module & Database Server. Based on the user requirements and the design

considerations expressed through the DSS architecture, T6.1 proceeded with defining the functional and

1 PRISMS website: http://prismsproject.eu/ 2 PARIS website: http://www.paris-project.org/ 3 VALUESEC website: http://www.valuesec.eu/


4

technical requirements of the PACT DSS. The requirements were documented in a template and format

convenient for use and reference by WP6 activities that follow, building the consolidated Requirements

Traceability Matrix which will feed the forthcoming software engineering and prototype development

activities.

T6.1 concludes its design activities by defining the validation criteria for the target PACT DSS. They include

user requirement based validation and acceptance as well as performance and technical success measures

for the system and individual components. Once completed and delivered, the successful operation of the

PACT DSS software tool will be validated against the criteria defined in this document, including user

perspective criteria, success measures for each component, functionality and performance validation

criteria.

Finally, the legal, technical and ethical scope of the PRFST and the DSS have been considered by consortium

partners and PACT’s external actors for a prolonged period of time. It is important again to reiterate that

the PRFST tool and the DSS – just as with any decision support tool – are designed to support and not

displace legitimate, democratic decision-making and validation processes. Decisions are to be taken by the

users themselves, with the system supporting the process through multiple modalities and presenting

quantitative and qualitative information regarding the impact of each decision path.


5

Table of contents Executive summary ------------------------------------------------------------------------------------------------------3

Table of contents ----------------------------------------------------------------------------------------------------------5

Table of figures ------------------------------------------------------------------------------------------------------------6

List of acronyms used in the document -------------------------------------------------------------------------------7

1. Introduction ---------------------------------------------------------------------------------------------------------8

1.1 Scope ----------------------------------------------------------------------------------------------------------------8

1.2 Deliverable Structure ---------------------------------------------------------------------------------------------9

1.3 Methodology -------------------------------------------------------------------------------------------------------9

2. Decision Support Systems: State of the Art and relevant work carried on in EU projects ----------- 12

2.1 DESSI ---------------------------------------------------------------------------------------------------------- 12

2.2 PARIS ---------------------------------------------------------------------------------------------------------- 13

2.3 PRISMS -------------------------------------------------------------------------------------------------------- 15

2.4 SURPRISE ----------------------------------------------------------------------------------------------------- 16

2.5 VALUESEC ---------------------------------------------------------------------------------------------------- 17

2.6 Summary ----------------------------------------------------------------------------------------------------- 20

3 PACT DSS Stakeholders Report and User Requirements -------------------------------------------------- 21

3.1 PRFST-DSS Workshop report --------------------------------------------------------------------------------- 21

3.2 DSS User Requirements ---------------------------------------------------------------------------------------- 25

4 DSS Architecture, Functional and Technical Requirements ----------------------------------------------- 29

4.1 DSS Architecture ------------------------------------------------------------------------------------------------ 29

4.2 Functional and Technical Requirements -------------------------------------------------------------------- 36

5 Requirements Traceability Matrix ----------------------------------------------------------------------------- 42

6 DSS Validation Criteria ------------------------------------------------------------------------------------------- 48

6.1 User Perspective Criteria -------------------------------------------------------------------------------------- 48

6.2 Component Specific Validation Criteria and Success Measures ---------------------------------------- 48

6.3 Functionality Validation Criteria ----------------------------------------------------------------------------- 50

6.4 Performance Validation Criteria ------------------------------------------------------------------------------ 50

7 Conclusions -------------------------------------------------------------------------------------------------------- 51


6

Table of figures Figure 1 A screenshot of the DESSI web-based application. ..................................................................................... 14

Figure 2 Structure of PRISMS DSS. ....................................................................................................................................... 16

Figure 3 Building blocks of the PRISMS DSS .................................................................................................................... 16

Figure 4 ValueSec Tool screenshot ...................................................................................................................................... 18

Figure 5 ValueSec DSS technical architecture. ................................................................................................................ 19

Figure 6: PACT DSS Context .................................................................................................................................................... 22

Figure 7 Mean and Median User Scores for each HMI design criterion .................................................................... 25

Figure 8 PACT Preliminary DSS Architecture .................................................................................................................. 30

Figure 9 Sample PACT bar chart ........................................................................................................................................... 32

Figure 10 Sample PACT spider diagram ............................................................................................................................ 33

Figure 11 Privacy Threat Index, Colour Scheme representation ............................................................................ 33

Figure 12: Sample decision tree ............................................................................................................................................ 34

Figure 13 Survey Data Table Sample for Travel context ............................................................................................ 35


7

List of acronyms used in the document

DSS: Decision Support System

E-R: Entity-Relationship

HLPP: High Level Policy Panel (external experts’ advisory group of PACT)

HMI: Human Machine Interface

PRFST: Privacy Reference Framework for Security Technologies (described in D5.1, D5.2)

PTI: Privacy Threat Index (described in D5.1, D5.2)

UI: User Interface


8

1. Introduction

The current document reports the activities of Task T6.1 “DSS Architecture Technical Specifications and

System Validation definition” and it is the first deliverable of WP6 “Decision Support System (DSS)”. The

report feeds into T6.2 “DSS Knowledge Base, Decision Context and Engine Functional Design” which

initiates software engineering activities and provides the DSS tool specifications to T6.3 for DSS prototype

development and implementation. T6.4 will finally perform the “Validation and End User Evaluation”,

basing the validation on defined criteria and methodology delivered by this document. therefore, this

deliverable D6.1 is the foundation stone of the overall WP6 work plan. For this reason T6.1 followed a

clearly specified step-by-step methodology (as described in brief in section 1.2 below) in order to achieve

the necessary milestones and the needed outputs, while engaging the PACT experts community and

collecting valuable end user perspective and experience4.

1.1 Scope

The main scope of this deliverable is to:

1. Define the user requirements by engaging and collecting feedback from PACT End User partners

and all PACT external experts groups.

2. Define the preliminary DSS architecture and subsequently define the technical and functional

requirements providing the functionality and technical aspects necessary to meet the user

requirements and DSS architecture design needs, taking also into account the state of the art and

relevant research work.

3. Define the measurable criteria which will validate the successful operation of the PACT DSS.

4. Report and document all relevant definitions in a template and format convenient for use and

reference by WP6 activities that follow (i.e. software engineering and prototype implementation).

It is important to remind readers that this deliverable intends to define the technical architecture,

requirements and validation criteria of the DSS model. As the scope of this deliverable sits within a

uniquely technical context, thematic considerations around fundamental rights, ethics, and data

protection are to be considered in the deliverables to follow; primarily in D6.2 where the functional

specifications will be set, in D6.3 embedded in the DSS prototype and in D6.4 which includes the

evaluation and user acceptance report.

Relevant previous PACT deliverables and results referenced throughout this document include:

D1.2 “Report on the review of DSS for decision making on security issues” referred to as PACT

D1.25

D1.3 “Report on security technology taxonomy and mapping” referred to as PACT D1.36

D1.6 “Use Case definition report” referred to as PACT D1.67

4 Key events: Joint PRFST-DSS workshop, 17-18 October 2013, Madrid, Spain and 2nd HLPP meeting, 18 November 2013, Brussels, Belgium. 5 Confidential, executive lay report to be made available soon on PACT website: http://www.projectpact.eu/deliverables/wp1-root-branch-review. 6 http://www.projectpact.eu/deliverables/wp1-root-branch-review/d1.3 7 http://www.projectpact.eu/deliverables/wp1-root-branch-review/d1.6


9

D4.2 “Survey Report” referred to as PACT D4.28

D5.1 “Report on the Definition and Design of the PRFST” referred as PACT D5.19

D5.2 “PACT PRFST” referred to as PACT D5.210

D7.5 “Updated plan for user and dissemination of foreground” referred to as PACT D7.511

1.2 Deliverable Structure

This document is organized in seven chapters.

Chapter 1 - Introduction: the current chapter presents the scope and objectives of the deliverable, briefly

explaining the methodology adopted in Task 6.1.

Chapter 2 – DSS State of the Art and connection with relevant work includes a review and an assessment

of the relevant work, collecting input from other relevant EU research projects PRISMS, PARIS, SURPRISE,

VALUSEC and DESSI.

Chapter 3 – PACT DSS Stakeholders report and User Requirements presents the collected and extracted

user requirements based on the results of the PRFST-DSS joint workshop as well as on the feedback

gathered from all PACT stakeholder groups.

Chapter 4 – DSS Architecture, Functional and Technical Requirements includes the description of the

preliminary DSS architecture along with the relevant functional and technical requirements.

Chapter 5 – Requirements Traceability Matrix: including for reference and convenience of future WP6

activities the consolidated, numbered table of Requirements reported in this document.

Chapter 6 – DSS Validation Criteria introduces the validation criteria against which the successful operation

of the PACT DSS software tool will be evaluated and validated.

Chapter 7 - Conclusions summarises the work reported in this document and highlights the more salient

PACT DSS features.

1.3 Methodology Task T6.1 methodology starts with the extraction of PACT DSS user requirements, which is followed by the

specification of the functional and technical requirements with respect to the PACT DSS Architecture as this

was defined and broken down into the following components: User Layer, HMI, Dialogue Management &

Visualisation Module, Decision Trees & Engine Module and Knowledge Management Module & Database

Server. Requirements are documented based on the template provided for each type of requirements and

consolidated in a Requirements Traceability Matrix, for reference and convenience of future activities and

deliverables of WP6. By analysing user requirements and technical aspects of the system design,

measurable success criteria are extracted, forming the validation approach for PACT DSS. These include the

8 http://www.projectpact.eu/deliverables/wp4-data-analysis/d4.2 9 http://www.projectpact.eu/deliverables/wp5-new-conceptualization-and-framework/d5.1 10 http://www.projectpact.eu/deliverables/wp5-new-conceptualization-and-framework/d5.2 11 To be published on PACT Website on: http://www.projectpact.eu/deliverables/wp7-dissemination-and-stakeholder-involvement


10

user perspective criteria along with the success measures for each DSS component, functionality and

performance validation. We present here the template used for each type of requirement.

User Requirements

User Requirements are extracted based on feedback from all PACT experts groups: the Stakeholder

Advisory Group (SAG), the Privacy Advocates Panel (PAP) and the High Level Policy Panel (HLPP). The

feedback was collected through questionnaires, round-table discussions and during dedicated presentation

& workshop sessions at the joint PRFST-DSS workshop (Madrid on 17th and 18th, October 2013, in the

premises of Atos Spain) as well as during the 2nd HLPP meeting in Brussels (18th November 2013).

In this context, user requirements extracted follow the specific table-row format, including the following

fields:

ID (ID): A unique ID for the specific user requirement, to be used for reference by following WP6 deliverables. The ID will follow the format common in all PACT WP deliverables: <document code>.<Type U/B/F/T (User/Business/Functional/Technical)>-<increasing number>, e.g. for this document D61.U-15.

Brief Summary: a summary, title-like of the requirement, so as to easily identify what this requirement is about.

Requirement: a full description of the requirement together with explanatory text and rationale, if necessary.

Source: the Experts Group responsible for contributing this requirement, i.e. Stakeholder Advisory Group, the Privacy Advocates Panel and the High Level Policy Panel.

Functional / Non-Functional: Whether the requirement is functional or non -functional.

Domain: General / Security/ Privacy / Ethics / etc…

Template:

ID Brief Summary

Requirement Source Functional / Non-Functional

Domain

D61.U-1

Functional and technical requirements

Functional and technical requirements are extracted taking into account a) the end user's perspective and

needs as encapsulated inside the user requirements, and b) the state of the art, best practices and market

trends. In this context, requirements extracted follow the specific table-row format, including the following

fields:

ID: A unique ID for the specific requirement, to be used for reference by following deliverables. The ID will follow the format common in all PACT WP deliverables: <document code>.<Type U/B/F/T (User/Business/Functional/Technical)>-<increasing number>, e.g. for this document D61.F-15.

Brief Summary: a summary, title-like of the requirement, so as to easily identify what this requirement is about.

Requirement: a full description of the requirement together with explanatory text and rationale, if necessary.


11

Requirement Source Ref: A reference to the user requirement ID or market analysis requirement ID that leads to this requirement –if applicable. The identification code corresponding to identifies used in the respective deliverables, e.g. D61.B-12 for a market analysis requirement and D61.U-9 for a user requirement.

Mandatory/Optional: The corresponding box is checked if the requirement is mandatory or optional in the actual PACT solution being implemented during the project. Mandatory requirements are essential for the PACT implementation while optional are usually reserved for future extensions and design considerations.

Finally, after all requirements have been extracted, they are also systematically grouped and inserted inside

the Requirements Traceability Matrix.

Template:

ID Brief Summary Requirement

Require

ment

Source

Ref.

Man

dator

y

Optio

nal

D61.F-1 X

D61.F-2 X


12

2. Decision Support Systems: State of the Art and relevant work carried on in EU

projects

In the context of PACT, the main aim of the Decision Support System (DSS) is to function as a tool

contributing to the assessment of public perceptions of security and privacy by end users and policy makers

when they deal with specific security investment issues. This chapter provides a state of the art of Decision

Support Systems as currently developed in relevant European projects. Each examined project is briefly

analysed to understand its goal and approach, the main design aspects and the possible contribution(s) or

input(s) for the design of the PACT DSS system itself. This brief comparative review is based on publicly

available information (websites, deliverables, etc.). The following projects are examined while collaboration

was established in case of ongoing projects:

DESSI: Decision Support on Security Investment)

PARIS: PrivAcy pReserving Infrastructure for Surveillance)

PRISMS: PRIvacy and Security MirrorS: Towards a European Framework for Integrated Decision

Making)

SurPRISE: Surveillance, Privacy and Security: A large scale participatory assessment of criteria and

factors determining acceptability and acceptance of security technologies in Europe)

VALUESEC: Mastering the Value Function of Security Measures)

2.1 DESSI12 DESSI project aims at developing a DSS system which will contribute to a transparent and participatory

decision making to specific security investment issues. The main goal is to overcome the exclusively

technology-driven solutions and to offer a versatile approach which takes into account the political, ethical

and social dimensions of society while at the same time offers alternative ways of addressing a specific

threat.

The DSS system that has been developed is a web-based application which guides the user in the analysis of

the security related issue through predefined questions and organisational steps. Specifically, the following

three phases are included (Fig. 1): i) Security problem ii) Security investment and alternatives, and iii)

Multi-criteria assessment workshop. Finally, a report that collects all phases of DESSI DSS system can be

produced.

Input for PACT

A relevant input for PACT DSS could be the architecture of the developed DESSI DSS, the technologies

adopted for the web application development and the knowledge database definition, as well as the

reasoning process of the decision-making system.

12 http://securitydecisions.org/, https://dessitool.org/en/

http://securitydecisions.org/

https://dessitool.org/en/


13

2.2 PARIS13 In the surveillance infrastructure design area PARIS project aims at the development of a methodological

approach which by design enforces the right of citizens for privacy, justice and freedom. At the same time,

the ethical and social aspects of such rights are taken into account, since the perception of such rights

varies between countries. Special consideration is also given to the adjustability of the proposed

methodology to any future changes in the legal framework.

The methodological approach will be based on two pillars; a theoretical framework which balances

surveillance with privacy, and a design process where privacy and accountability are considered by design.

13 http://www.paris-project.org/

http://www.paris-project.org/

Figure 1 A screenshot of the DESSI web-based application.

Specialized conceptual frameworks, called SALT frameworks, will also be defined and a framework

management tool will be developed to help surveillance system designers. The tool allows the creation,

digital reference and representation, and updating of a SALT framework. It can also allow for reasoning by

integrating information (such as rules, constraints), and guidelines on the relation between surveillance,

privacy, and accountability capability.

Input for PACT

An insight into the analysis, design and development phase of SALT framework management tool could be

of special interest for the PACT DSS design. Especially, the technologies used, the architecture, the updating

and representation of the theoretical framework and the integration of information and guidelines could

offer useful knowledge for the implementation of PACT DSS system.

2.3 PRISMS14 A close collaboration was established with PRISMS given the parallel activities, similar nature and objectives

with PACT. This was achieved through teleconferences, exchange of information and views on important

design aspects concerning the DSS and overall methodology.

The objectives of PRISMS project are to analyse the trade-off model relation between privacy and security,

investigate how surveillance technologies usage may infringe citizens’ privacy and fundamental rights, and

determine the factors that affect the public assessment of the security and privacy implications of a specific

security technology.

The outcomes of the aforementioned analysis will be used for the development of a DSS system which will

provide all stakeholders with an insight into the advantages, disadvantages, constraints, and conflicts of a

specific security investment as opposed to other alternatives.

In conceptual level, the DSS system of PRISMS project (Fig. 2) will take input from a knowledge database

which can comprise all the evidence and heuristics as well as input from a model-based management

system which contains possible scenarios and the necessary decision rules and hierarchy while the output is

depicted in the user interface. The building blocks of the PRISMS DSS are depicted in Fig. 3 where only the

impact assessment is to be implemented as a software tool.

Input for PACT

A source of useful input for PACT could be any information on the software implementation and the

architecture adopted but even any conceptual framework regarding the other building blocks of the

PRISMS DSS.

14 http://prismsproject.eu/

http://prismsproject.eu/


16

Figure 2 Structure of PRISMS DSS.

Figure 3 Building blocks of the PRISMS DSS

2.4 SURPRISE15

Another European project which has similar objectives of PRISMS and PACT is SurPRISE. This project will

deliver a large scale participatory assessment of criteria and determinants of acceptability of security

technologies in Europe.

In that context user requirements (factors influencing acceptance of these security technologies),

functional requirements (technical design, legal options, non-technical alternatives, and models about

15 http://surprise-project.eu/

http://surprise-project.eu/


17

privacy and security relationships) and non-functional requirements (empirical findings and testing of

models) will be identified for decision support on security technologies.

Input for PACT

SurPRISE can constitute a source of useful information for PACT in terms of identification and analysis of

the aforementioned requirements, models development, and technologies used as well as validation and

testing on DSS systems.

2.5 VALUESEC16

With ATOS being a common and key partner in both PACT and VALUESEC, relevant knowledge and useful

insight was provided:

“The vast majority of decisions within the security sector are characterized by a complex set of parameters

and uncertainties. They range from quantitative factors like costs, investment budgets and benefits such as

the intended reduction of damages to highly uncertain and qualitative factors such as political objectives or

societal acceptance. Funded by the European Commission’s 7th Framework Program, the ValueSec project

is intended to support and enhance the decision making process with a toolset so that security measures

reflect the interests of all stakeholders.

Throughout the last three years this project aimed to develop the means to make more transparent and

systematic assessments of the wide array of costs incurred by security decisions as well as the potential and

expected benefits linked to such decisions. It will thus facilitate better and more effective decision making.

Decisions must be based on transparent criteria and a rigorous cost-benefit assessment, incorporating

quantitative and qualitative factors such as social, cultural, ethical and economic implications in the overall

analysis.

Thus, the toolset functionalities are grouped by three pillars:

Risk Reduction Assessment, to evaluate how good a security measure is in terms of mitigating a threat and its corresponding impact on the organization assets.

Cost-Benefit Assessment, to calculate quantitative implications of decisions on security measures, focusing on monetary costs and benefits.

Qualitative Criteria Assessment, to integrate different non-tangible decision parameters into an evaluation process, such as societal or ethical factors.

The toolset aims to support rational decision-making by establishing methodologies to increase the

transparency of the decisions-making process and of the driving decision parameters”.17

16 http://www.valuesec.eu/ 17 http://www.valuesec.eu/sites/default/files/2014.02.27._VALUESEC_D7.5_Exploitation_Plan_FINAL.pdf

http://www.valuesec.eu/

http://www.valuesec.eu/sites/default/files/2014.02.27._VALUESEC_D7.5_Exploitation_Plan_FINAL.pdf


18

Figure 4 ValueSec Tool screenshot

Considering the system architecture (Fig. 5), the toolset comprises a set of centralized components and

tools as well as external components. Centralized components consist of the methodological framework

functions (Qualitative Criteria Assessment, Cost-Benefit Assessment), the knowledge database and other

common functionalities. They are all incorporated in the same web application and hosted in the same

server. The Risk Reduction Assessment functionality is handled by components placed on external servers

communicating with the central server using dedicated API.


19

Figure 5 ValueSec DSS technical architecture.

Contexts and use cases

The methodology and toolset were assessed in the context of realistic use cases based on explicit

requirements of end users that represent pre-defined decision-making contexts and processes. Its target

group of stakeholders is that of public decision makers and the project aims to understand their needs and

under which framework conditions they make decisions regarding security measures.

The selected decision making contexts for ValueSec are: Public mass events, Public transportation, Airport

security, Communal security planning and Cyber security. These are the use cases where the project was

tested for each of the contexts:

- Use Case 1: Mass Event - Formula One Race Track, Valencia´s Street Circuit: Improved surveillance

and detection systems

- Use Case 2: Public Transportation - Train Infrastructure: Improved Intrusion Detection and Damage

Prevention for Passenger Trains in a Depot

- Use Case 3: Aviation - Airport Security: New generation Liquid Aerosol Gel (LAG) Scanners


20

- Use Case 4: Communal Security - Flood Protection: Improvement of infrastructure and water

management

- Use Case 5: Cyber Security - Smart Grid attack: Improving Security of Energy Smart Grids from

Targeted Viruses Attacks

Input for PACT

Some of the questions and approaches of ValueSec, in which PACT partners Atos and PRIO also

participated, are relevant for PACT too i.e. how is the decision making process structured? What are

current parameters that influence the decision-making process and which ones are necessary to make

suitable decisions? What kind of decision support is needed to achieve trade-offs? ValueSec findings are to

be evaluated from the different perspectives of security policy makers and the perspectives of policy,

economics and society.

During the first phase of PACT a collaboration was already established i.e. results from ValueSec on the

analysis of available methodologies and tools for decision analysis and support have been considered for

the review of DSS for decision making on security issues.

In later phases of PACT, especially for the DSS design, aspects addressed by ValueSec can prove to be of

relevance, for instance for the definition of a decision support tool that satisfies stakeholders’ requirements

in a security context and the corresponding technical architecture. The fact that “decisions must be based

on transparent criteria and a rigorous cost-benefit analysis, incorporating quantitative and qualitative

factors such as social, cultural, ethical and economic implications in the overall analysis” matches well the

objective approach of PACT’s PRFST and highlights the heterogeneous mixture of dimensions and factors

both projects consider.

Moreover, information on the definition of decision support tool, the corresponding technical architecture,

the knowledge database schema definition, and the technologies adopted for central server and web

application development can be relevant for PACT DSS system as well.

2.6 Summary

A number of ongoing European projects, related with PACT, have been examined in the context of their

potential use as input for the PACT’s DSS design. Summarizing the findings of the above study, the main

points from each project where special attention should be paid to, are the theoretical framework and the

decision-making and reasoning process, the models development process, the technical system

architecture or the architecture of any relevant component (e.g. SALT framework management tool), the

technologies adopted for server, database and tools development, as well as validation and testing of DSS.

Exploiting the existing repository of knowledge, PACT can accomplish its goals to go beyond a mere risk and

impact assessment analysis but deliver an innovative DSS which is context-aware and adaptive, supports

multi-criteria decision making and collaborative decision, as well as considering crowdsourcing and social

media empirical knowledge collection. However, concerns with regard to crowdsourcing and social media

perceptions need to be taken into account (issues explained in more detail in section 4.1 “The User Layer”).


21

3 PACT DSS Stakeholders Report and User Requirements

User requirements for the PACT DSS have been extracted from the feedback collected from PACT End User

partners participating in the task and from all PACT stakeholder groups: the Stakeholder Advisory Group

(SAG)(PAP) and the High Level Policy Panel (HLPP). The key event to achieve this was the joint PRFST-DSS

workshop, which took place in Madrid on 17th and 18th October 2013, in the premises of Atos Spain. The

DSS concept was also presented and discussed during the 2nd HLPP meeting in Brussels (18th November

2013). This chapter offers a review of the PRFST-DSS workshop, and a consolidated table of extracted User

Requirements.

3.1 PRFST-DSS Workshop report

As explained in D5.2, under the prism of the PRFST and DSS objectives, the joint PRFST-DSS Workshop was

considered an excellent opportunity to present the key design elements of these key PACT contributions to

the PACT expert groups (including the Stakeholder Advisory Group, Privacy Advocates Panel and High Level

Policy Panel). The main goal of the PRFST-DSS workshop was to collect and compile valuable expert

feedback and translate it into user requirements for the PACT DSS. Active contributions during the

workshop were provided from all participants. The multidisciplinary expertise represented in the workshop,

by the participation of the different expert groups, was instrumental to help the consortium translate the

theoretical and empirical approaches into a useful tool for the project’s target stakeholders.

Discussion on Context and DSS functionalities

During the workshop, dedicated presentations were given concerning the PACT DSS, its concept of

operations and the consistency/relationship with the work carried on in WP5 (in particular the

development of the PRFST). The workshop was indeed an opportunity to harmonize the efforts within WP5

and WP6, given also the input and feedback from stakeholders at the audience and in accordance to the

PRFST-DSS connection well established and presented in PACT D5.1.

Figure 6 explains the Context framework which feeds PACT DSS engine and functionalities, which was

discussed during the workshop.

The key discussion points during the workshop focused on analysing the security decision making context

and on the PACT DSS functionalities. In detail, the session on analysing context included the following

questions towards the potential DSS users:

Based in your experience, how can intangible elements be valuated? (such as the reputation, the

corporative image, the corporative deontological code, and others)

Please rate the contextual elements for their usefulness and importance during security decision

making and with respect to privacy issues.

Suggest additional contextual elements

Please rate the identified key choice points in terms of how challenging they can be from your

experience.

Suggest challenging key choice points


22

Figure 6: PACT DSS Context

With regards to DSS functionalities, participants were asked to respond to the following questions:

Given the presentation, picture your “ideal” PACT DSS. Please write here up to five (5) example

questions that would help your decision making and you would really like to “ask” PACT DSS. Feel

free to write either simple or more complex questions.

What kind of historical data would you find useful in the context of PACT DSS?

What kind of projected figures would you expect from PACT DSS? What kind of figures or charts

would you find useful in terms of estimating public acceptance and trust?

Please describe a typical decision cycle in your organisation for privacy related decision making,

describing in a few words the role of each person in the cycle.

In these sessions, context and scope were important points discussed during the workshop, expressing the

requirement that the DSS should be context-aware and context-driven, adapting to particular scenario, use

case and application. The output for the users was also a key issue, where users would expect a Data

Protection or Privacy Impact Assessment as a provided output to users.

Discussion on Human-Machine Interaction (HMI) design

A session dedicated to the HMI design was also held during the PACT Workshop which included the

collection and analysis of HMI design questionnaires. The main goals of the HMI session were to provide an

understanding of the basic concepts of efficient HMI design. This subsection provides an overview of the

discussion that took place during the HMI session of the PACT Workshop.

The basic concept of Human-Machine18 Interaction was first introduced, as a multidisciplinary field

combining computer science, psychology, behavioural science, design and ergonomics. Efficient HMI design

is necessary in order to provide an easy-to-use user interface, which will minimize the time and effort

needed to complete a task. Interaction Styles 19 are defined as ways of communication between the user

18 Stuart K. Card, Thomas P. Moran, Allen Newell (1983): The Psychology of Human–Computer Interaction. Erlbaum, Hillsdale 1983 ISBN 0-89859-243-7 19 Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human-computer interaction. Wokingham,UK.


23

and the computer system. The session briefly discussed the most common interaction styles, enabling

dialogue between the PACT consortium and the end-users, towards an efficient design of the user interface

for the PACT DSS.

The pros and cons of the most common Interaction Styles were discussed, including:

The basic Command Line Interface: it provides a powerful environment but has a steep learning curve.

The Window, Icon, Menu, Pointer (WIMP) style: the most common style of interaction and one of the easiest to use, since most users are already familiar with it.

Form fill-in: Easy to use, but considered to be tedious.

Direct Manipulation: The user interacts with icons that map real-world objects to user actions.

Crossing Interfaces: Gestures are integrated with conventional click-drag-drop functionalities

Zooming: Allows zooming in and out of a focus area.

Focus plus Context: Gives an overview of the whole system and allows user to focus to a specific area of the screen.

Brushing and Linking: Only subsets of information are shown the user, by applying appropriate filters to control visibility.

Various examples of Decision Support Systems were then discussed, focusing on the Interaction Styles that

their respective user interface implements, in order to provide a more clear understanding of the way

interaction styles are mixed in a real world implementation of a system. This “exercise” also served to

illustrate which interaction styles are more common in implementations of Decision Support Systems.

The discussion also gravitated towards the design of efficient and usable User Interfaces, listing the

Nielsen20 usability heuristics, which include:

Visibility of System Status

Match Between System and the Real World

User Control and Freedom

Consistency and Standards

Error Prevention

Recognition Rather Than Recall

Flexibility and Efficiency of Use

Aesthetic and Minimalist Design

Easy Recognition, Diagnosis, and Recovery from Errors

Help and Documentation.

Following the HMI session, a discussion on the heuristic criteria was held and a questionnaire was handed

to the session participants. The questionnaire was compiled by NCSRD and illustrated some examples of

DSS User Interface design and asking for participants’ input. Specifically, it featured:

Examples of DSS UI design

Examples of 2D data visualisations

The Weinschenk/Barker21 design evaluation criteria

The participants were then asked to complete the questionnaire, indicating their personal opinion on the

20 Nielsen, J. and Mack, R.L. (eds) (1994). Usability Inspection Methods, John Wiley & Sons Inc. 21

Weinschenk, Susan, and Dean T. Barker. Designing Effective Speech Interfaces. New York: Wiley, 2000.


24

presented UIs and ranking the Weinschenk/Barker criteria according to their importance (as viewed by

each participant). When the HMI session concluded, sixteen (16) answered questionnaires were collected.

The results of this activity provided some insight on the user requirements for the DSS user interface, which

will be discussed in a following section.

The collection of the HMI design questionnaires during the PACT Workshop, provided some useful insight

on the user requirements for the efficient and usable UI design of the PACT DSS. According to participants’

input and their comments on the presented DSS UIs the required characteristic and functionalities for the

user interface can be summarized in the following list:

The user interface should not be cluttered.

Data visualizations should not be complex.

The HMI should give a minimalistic overview of the data also allowing the user to expand upon the visualizations when necessary, in order to obtain more information.

The HMI could resemble a Wizard-like process and provide the ability to process the data in an appropriate number of steps.

Condensation of graphics should be avoided.

Abbreviations should be avoided.

Cluttered Columns, lists and menus should be avoided.

Icons need to be meaningful and map to specific contexts.

The participants were also asked to rank the following design criteria22 according to their perceived

importance in UI design, by assigning a score (from 1-10) to each one:

1. User Control: Amount of user control available through the interface design. 2. Human Limitations: Taking into account human limitations, cognitive and sensorial, to avoid

overloading the user. 3. Modal Integrity: use of the most suitable modality for each task: auditory, visual, or

motor/kinesthetic. 4. Accommodation: the design is adequate to fulfil the needs of each user group. 5. Linguistic Clarity: the language used to communicate with the user is understandable, efficient and

adequate. 6. Aesthetic Integrity: the design is visually appealing to the user. 7. Simplicity: the design should not be unnecessarily complex. 8. Predictability: users will be able to form a mental model of how the system will respond to taken

actions and predict their outcomes. 9. Interpretation: the definition of a set of codified rules that interpret user actions to system

functions. 10. Accuracy: taken actions have the expected outcome and no errors are made 11. Technical Clarity: the concepts represented in the interface are clearly mapped to the domain they

are modeling. 12. Flexibility: the design can be adjusted to the needs of its user. 13. Fulfillment: the user has a fulfilling experience in using the system. 14. Cultural Propriety: user's cultural and social expectations are met. 15. Suitable Tempo: the pace at which users works with the system is adequate. 16. Consistency: different parts of the system have the same style, so that there are no different ways

to represent the same information or behaviour. 17. User Support: the design will support learning and provide the required assistance to usage. 18. Precision: the steps and results of a task will be what the user wants. 19. Forgiveness: the user will be able to recover to an adequate state after an error.

22 Weinschenk, Susan, and Dean T. Barker. Designing Effective Speech Interfaces. New York: Wiley, 2000.


25

20. Responsiveness: the interface provides enough feedback information about the system status and the task completion.

Figure 7 shows the results of the end-user evaluation.

Figure 7 Mean and Median User Scores for each HMI design criterion.

According to participants’ input, the criteria that received the highest ranking are:

Simplicity,

Responsiveness,

Forgiveness,

Linguistic Clarity, and

Aesthetic Integrity.

Therefore, it is safe to conclude that PACT Decision Support System is expected to provide:

Minimalistic visualizations in a simple, non-cluttered environment,

Dialogue with the user and useful feedback about its status,

Easy tracking of and recovery from errors,

Clarity of language, with no abbreviations and unnecessary jargon, and

An aesthetically pleasing and cohesive user interface, without cluttered menus, lists etc.

3.2 DSS User Requirements

Following the discussions and feedback collected and analysed from all PACT expert groups and as

described in the previous section, a set of user requirements were extracted. The following table

consolidates the user requirements in the format defined in Chapter 1.

Each requirement has a unique ID and is briefly summarized within the table. Requirements are either

marked as functional or non-functional, depending on their scope. Functional requirements define

functionalities of a given system, including the inputs and outputs. Therefore, functional requirements


26

focus on describing what tasks a system should perform. Non-functional requirements set constraints on

the behaviour of a system and the resources that it utilizes. In essence, non-functional requirements focus

on how a system performs a specific task. The combination of functional and non-functional requirements

can be utilized to distil the system architecture and functional design.

The key points that were raised by the workshop participants, and that permeate the collected user

requirements are summarized as follows:

The main functionality of the PACT DSS is to provide an impact assessment that is context-

sensitive and flexible in its definition: The PACT DSS should provide an impact assessment taking

into account the user’s choice of scenario. This may include utilizing data from multiple sources, as

well as taking into account the legal, ethical implications of the scenario. This is particularly

apparent in D61.U-1, D61.U-3, D61.U-4, and D61.U-6 to D61.U-10 (see table below).

The PACT DSS should be easy-to-use and non-confusing to the user: This key point was raised

several times, regarding not only the HMI design but also the use of technical language etc. The use

of simple language is necessary and the system should not require a high level of ICT experience.

This is evident within D61.U-5, D61.U-7 and D61.U-11 to D61.U-17 (see table below).

The PACT DSS should allow the user to provide feedback in every stage of its use, while showing

fault-tolerant behaviour towards the user and ensuring the reliability of the presented data: The

users require a high level of customization to be available as well as the ability to provide feedback

to the system in every part of the process. The user must then be able to retrace their steps and

easily recover from errors. The information retrieved and processed by the PACT DSS should always

be reliable. This is a common theme, apparent in D61.U-2, D61.U-15, D61.U-17 and D61.U-18 (see

table below).

The user requirements can then be used to extrapolate the technical requirements and functional design of

the system, both essential parts of a robust engineering process. The Requirements Traceability Matrix that

follows in Chapter 5, provides further insight on how the collected user requirements are linked to the

individual technical requirements.


27

ID Brief Summary Requirement Source Functional / Non-

Functional

Domain

D61.U-1 Privacy Impact Assessment

The system should provide a tangible report in the form of a privacy impact assessment for security technologies

Security Stakeholders/ Privacy Advocate

Functional General, Privacy

D61.U-2 Impact assessment assistance

Given that 1/3 impact assessment reports are sent back to be re-worked, the PACT DSS should offer personalised support, assistance and informal validation for impact assessment procedures.

High Level Policy Panel

Functional General, Security, Privacy

D61.U-3 Context-specific

The PACT DSS should focus in the particular context and scope of the scenario/technology under investigation.

Security Stakeholders/ High Level Policy Panel

Functional General, Security

D61.U-4 Consolidated knowledge

The system should consolidate theoretical and empirical data from multiple sources.


Non Functional

General, Security

D61.U-5 Security Language

The system should avoid negative connotations connected with specific terms, and should “talk” the same language with the users.



D61.U-6 Laws and Ethics

The system should highlight case-specific connected legal and ethical issues, along with any potential tension between those.


Functional Legal, Ethical

D61.U-7 Measurable Index

The system should use whenever possible measurable index, applied e.g. to PTI.



D61.U-8 Intangible assets

The system should take into account intangible assets

Security Stakeholders

Functional Privacy, Security

D61.U-9 Compliance The system should check if proposed security solution complies with current standards, practices and legislations


Functional Legal, Privacy, Security

D61.U-10 Update Knowledge Base

Ability to update the knowledge base including results and standards taken into account etc.


Functional General


28

D61.U-11 Data Visualisations

The system should provide useful data visualizations to the user.


Functional General

D61.U-12 Data Input The HMI must provide a way to input scenario-specific data


Functional General

D61.U-13 Non-cluttered visualizations

The system’s HMI should not be cluttered with too much data to visualize simultaneously.


Non-Functional

General

D61.U-14 Simple visualizations

Data visualizations should be simple, non-complex with the ability to expand into more complex views if necessary


Non-Functional

General

D61.U-15 Wizard-like interface

The HMI should allow the data input and data processing in clearly defined steps, making it easy for the user to track back their steps if required.


Non-Functional

General

D61.U-16 Direct Manipulation Icons

Use of icons in HMI design, with clear mapping to real-world concepts


Non-Functional

General

D61.U-17 Easy Access Access to the PACT DSS should be easy and shouldn’t require too much ICT skills on behalf of the users


Non-Functional

General

D61.U-18 Reliability PACT DSS should provide reliable information at all times.

Security Stakeholders, High Level Policy Panel

Non-Functional

General


29

4 DSS Architecture, Functional and Technical Requirements

In this chapter the core output of T6.1 is presented: the DSS Architecture and the Functional and Technical

Requirements for the system and individual components. These include namely the requirements

describing the behaviour and functions of the system and the technical aspects that the system must fulfil

respectively, such as performance related issues, reliability and availability issues.

The architecture is defined on the backdrop of a study of the state of the art (cf. PACT D.1.2), of relevant

ongoing research (as presented in Chapter 2) and of optimal design patterns suitable for the general needs

of the PACT DSS. Functional and technical requirements are extracted from the user requirements

presented in chapter 3 and the expressed needs of the wide range of stakeholder groups engaged in PACT

project.

4.1 DSS Architecture

Having gone through the State of the Art for relevant systems and given the user requirements and joint

WP5/WP6 activities, T6.1 produced a preliminary architecture for the PACT DSS. The preliminary

architecture sets the frame in which functional and technical requirements for the DSS will be populated

and will be finalised with the specifications definition in D6.2. The architecture (depicted in Figure 8) is

optimized for a web-based application and thus follows the design pattern for a three-tier architecture,

with distinct tiers for each of: data, application logic and front-end/user side. A three-tier architecture is

favoured for web applications due to its modularity and flexibility, as it creates a logical separation

between the presentation layer, the business logic layer, and the database layer. The separation of data

from the application (or business) rules and presentation creates a flexible environment where changes in

each layer leave other layers unaffected. The modular design enables the possibility of updating data

seamlessly into the system, or creating a multi-user environment with changing application rules without

disrupting presentation or causing extra development overhead. An analysis of each architecture layer

follows.


30

Figure 8 PACT Preliminary DSS Architecture

The User Layer

Being a web-based application, the user layer includes a so-called web “thin client”. Namely the user can

access the application through this web-browser without the need of deploying or running a specific client

application on his computer. This comes with several advantages:

1. Convenience and accessibility: The user can access the PACT DSS anywhere, anytime. The

application is not bound physically to specific computer or device while as long as the server is

up and running, the user can access the application on a 24-7-365 basis.

2. No installation and maintenance effort: The web application does not come with extra effort or

skills required from the user to install it or maintain it, such as e.g. installing additional batches and

updates. Instead the application is deployed and maintained on the server-side, leaving this effort

to the particular application/service provider.

3. Interoperability: As a web application PACT DSS will run across all operating systems and platforms

equipped with a web browser and compliant with relevant internet presentation standards.

4. Cost-effective: Web applications are in general more cost-effective than desktop applications, given

the amount of users they can hold without the costs of individual maintenance fees.


31

5. User Training: With the application being online and easily supervised remotely or connected with

an eLearning module, user training can be done far more efficiently, anywhere, anytime and at the

user’s pace of convenience.

Besides the general advantages that come with opting for a web-based approach for the PACT DSS, the

PACT application will also benefit from Web 2.0 state of the art techniques and patterns such as social

media, online collaboration, interactive interfaces and dynamic, user driven and user responsive content.

Typical UIs and DSS “look and feel” can be found in PACT D1.2, with PACT DSS aiming to provide a context

optimised and state-of-the-art UI. Visualisation components for the PACT DSS are managed and generated

on the server side, via the HMI, Dialogue Management & Visualisation module.

A small, but not insignificant, caveat here about social media and crowdsourcing perceptions must be

considered when the functional specifications are set in D6.2. Using data – particularly on user

perceptions - generated on social media and other crowdsourcing platforms is fraught with ethical and

data protection challenges. Furthermore, doubts remain about the research value of those data. These

should be borne in mind as we move to setting the functional specifications in D6.2 and further into

the WP. While the project recognises that the research tool of the project is looking to be context

relevant in the future, the ethical, data protection and value of using the input data remain.

The Application Server

The application server is where the PACT DSS Application Logic resides. The rules and algorithms

implementing DSS functionalities are in this layer and divided into three modules, each one grouping a

particular set of functionalities: (i) the HMI, Dialogue Management & Visualisation module, responsible for

handling the presentation and interaction with the User; (ii) the Decision Trees & Engine Module,

responsible for the creation of Decision Support Trees and graphs according to the specific context,

technologies and case under investigation by the user, based on knowledge and data provided by the (iii)

Knowledge Management Module, a group of services responsible for extraction of knowledge and data

management provided by the Database.

The PACT DSS Application Server is the overall responsible layer for implementing the needed intelligence

for the PRFST toolbox as described in PACT D5.2. Required data is retrieved from the database, fed into the

decision support process and presented accordingly to the user through multiple visualisation modes (such

as charts, spider diagrams and colour coded tables) as rational arguments towards supporting the user’s

decision process. As a decision support tool however, it should be noted that the PACT DSS is not a legally

bound tool responsible of taking or validating legally the user’s decisions. The user is the expert responsible

of taking decisions, and the PACT DSS is a convenient and user-friendly tool to help him reach to the

decision faster and with a better view of all associated parameters, knowledge and arguments.

HMI, Dialogue Management & Visualisation Module

This module is responsible for the HMI, managing the dialogue and interaction space with the user, while

implementing visualisation components from the data and knowledge provided by the Knowledge

Management Module and from the graphs and conceptual trees provided by the Decision Trees & Engine

Module. The user dialogue with the PACT DSS will promote an interface users will find friendly and intuitive.

To achieve this, the user will perform security technologies selection and context parameterisation through

assistive UI components which will minimise strenuous UI activities like excessive clicks and complicated

structures navigation, by applying optimal design patterns including “one-click-away”, auto completion

input forms, drop-down selection lists and personalisation capabilities according to the specific user. User


32

Profile management is therefore also part of this module, adapting the interface to the particular user

needs and preferences, as well as to user decision and access level.

Regarding visualisation components, the module will provide convenient visualisation modalities in order to

provide the user with the necessary insight on decision-affecting factors. According to the nature and

structure of knowledge/data the optimal representation changes according to the complexity of the

knowledge structure and the relationships between multiple criteria, with a key visualisation mode being

the presentation of a user friendly representation of decision trees (full blown trees may be complicated as

in Figure 12) managed by the Decision Trees & Engine Module. E.g. in PACT D5.2 data and knowledge

correlations are visualised using the sample bar chart (Figure 9) and spider diagram (Figure 10). For each

technology two colours are used - positive effect is represented using the colour on right and negative

effect is represented using the colour on left Refer to PACT D5.2 for better understanding colour

representation and semantics.

Figure 9 Sample PACT bar chart


33

Figure 10 Sample PACT spider diagram

As explained in PACT D5.2: “Apart from decision trees there are other kind of charts that are useful in order

to make decision makers aware of how suitable are the options in accordance with their compliance to

privacy criteria. The objective of charts are to show decision makers in a clear way a comparison of the

possibilities according to the weights assigned to show how much they are privacy respecting”. For more

information on the charts and their use within the PRFST methodology please refer to section 4 of PACT

D5.2.

Knowledge and assessment of values will in some cases be presented using easy to understand colour

coding, as in the case of the Privacy Threat Index (PTI) of the PRFST (Figure 11), with low, medium and high

threat-impact combinations are coloured from green to yellow to red, depending on the level of attention

needed by the user in terms of privacy issues and corrective actions needed.

Threat Likelihood Impact

Low Medium High

High

Medium

Low Figure 11 Privacy Threat Index, Colour Scheme representation

The UI dialogue will go through the specific security technology case study through a guided step-by-step

structure, which will allow fast and convenient transition to any step needed to be revisited or edited. After

completing all steps, the final assessment, all user selections, input and annotations will also be provided in

an exportable formal (e.g. PDF file) as a consolidated impact assessment report.

Decision Trees & Engine Module

The Decision Trees & Engine Module is responsible for receiving relevant data and building the sequential

decision process in the form of a conceptual decision tree, “including decision alternatives, events (states of

nature), probabilities/weights and outcome of alternatives related with the sequential decisions” (PACT


34

D5.2). The module therefore builds the tree hierarchy including multiple available attributes such as factors

affecting the public perception and trust.

SUR

VEI

LLA

NC

E TE

CH

NO

LOG

Y SE

LEC

TIO

N

DA5

DA4

DA3

DA2

DA1

E1

E2

E3

E1

E2

E3

E1

E2

E3

E1

E2

E3

E1

E2

E3

O51

O52

O53

O41

O42

O43

O31

O32

O33

O21

O22

O23

O11

O12

O13

Figure 12: Sample decision tree

As shown in Figure 12, the full blown decision tree shows the overall picture when there are several

options to choose. It is the Visualisation’s module task to ensure efficient and user-friendly visualisation of

the knowledge encapsulated in the decision trees, while the decision trees module manages relevant

knowledge structures and hierarchies, offering the necessary intelligence and application logic to guide the

user through the optimal decision path. The module supports the user’s decision and does not by any

means enforce decisions or drive the decision process in an automated way. It is important to clarify that

the design of the PACT DSS is user-driven, and decisions are taken by the user himself, with the system

supporting the process through multiple modalities and presenting quantitative and qualitative information

regarding the impact of each decision path.

Knowledge Management Module

The Knowledge Management Module is responsible for managing and feeding the empirical and theoretical

knowledge to all other modules of the application server, namely for presentation, visualisation and

decision process building purposes. The module establishes the connection with the database server,

extracts needed data and provides it in a convenient format through a set of embedded web services. The

services will provide necessary data formatting as well as any necessary additional mathematical

functionality not covered by the stored data results.

Given the large PACT dataset which includes the whole survey results, the Knowledge Management Module

will focus on the efficient management and building of relevant and focused context around the decision

process and user selected case/technology under investigation, enabling a context-driven and specific

approach. To achieve this, additional context parameters will either be provided through the HMI dialogue,

user profiling and/or location-aware application settings.


35

The Database Server

The data layer consists of the PACT DSS database and database server, which includes all data stored in a

convenient structure for the PACT system to use. Data stored in the database will include both theoretical

and empirical PACT findings. The former will contain theoretical input assigned to specific use cases as in

PACT D1.6, as well as risks and privacy threats associated with particular technologies as per the mapping in

PACT D1.3. The data base E-R (Entity Relationship) diagram will interconnect the risk mapping and

technology taxonomy with key decision-influencing knowledge elements such as related regulatory

frameworks and laws, associated ethical values, privacy goals & assets and other attributes.

The E-R diagram will extend and include relationships to the empirical set of data, encapsulating the public

perception and opinion into the overall picture and associating multiple attributes and parameters that

may be affecting it. For this purpose, task 6.1 and partner NCSRD has collaborated with RAND and T4.2 and

produced a data template and format convenient for porting the PACT survey results into the DSS

database. The template consists of a single 2-dimensional table per country per context/scenario, with each

row corresponding to a different technology choice/attribute (e.g. standard CCTV, CCTV with embedded

intelligence, etc) and each column corresponding to a different demographic or other parameter (e.g. age,

income, education etc). Figure 13 shows an extract of the survey data table.

Figure 13 Survey Data Table Sample for Travel context

Attribute Observed Predicted Std. dev nσ Observed Predicted Std. dev nσ

Standard CCTV – working like a television 4.173 4.213 48,9 -0,8 4.919 4.904 52,4 0,3

Advanced CCTV that can detect abandoned bags 4.801 4.789 51,8 0,2 5.638 5.641 55,9 -0,1

Advanced CCTV that can recognise suspicious movements of people 4.603 4.582 50,6 0,4 5.196 5.219 53,9 -0,4

Advanced CCTV that can recognise faces 4.583 4.602 50,2 -0,4 5.335 5.306 53,5 0,5

No CCTV cameras 2.798 2.800 44,7 0,0 2.840 2.831 45,8 0,2

NA 0 0 0,0 0,0 0 0 0,0 0,0

CCTV information not stored for future use-only used in real time monitoring 3.058 3.000 43,3 1,3 3.402 3.474 46,5 -1,6

CCTV information stored for 3 days 3.124 3.102 41,3 0,5 3.555 3.604 44,2 -1,1

CCTV information stored for 7 days 4.367 4.429 48,4 -1,3 5.110 5.041 51,3 1,3



No CCTV cameras 2.798 2.800 44,7 0,0 2.840 2.831 45,8 0,2

Only police departments in the [UK] have access to the camera information 6.609 6.579 59,4 0,5 7.656 7.697 63,9 -0,6

All European police departments have access to the camera information 6.292 6.328 58,8 -0,6 7.335 7.302 62,8 0,5

All police departments worldwide have access to the camera information 5.259 5.279 56,2 -0,3 6.097 6.070 60,0 0,4

No CCTV cameras 2.798 2.800 44,7 0,0 2.840 2.831 45,8 0,2

NA 0 0 0,0 0,0 0 0 0,0 0,0

NA 0 0 0,0 0,0 0 0 0,0 0,0

No security personnel 3.532 3.460 47,7 1,5 3.879 3.960 50,9 -1,6

Unarmed security personnel employed by a private company 4.546 4.633 50,5 -1,7 5.339 5.285 53,8 1,0

Armed security personnel employed by a private company 4.173 4.186 49,3 -0,3 4.792 4.760 52,3 0,6

Unarmed police 4.578 4.603 50,1 -0,5 5.267 5.264 53,4 0,1

Armed police 4.129 4.104 48,7 0,5 4.651 4.631 51,5 0,4

NA 0 0 0,0 0,0 0 0 0,0 0,0

People randomly selected for physical search and bag check 5.614 5.710 60,5 -1,6 6.597 6.521 64,4 1,2

People randomly selected to go through metal detector or full body scanner 7.318 7.228 64,8 1,4 8.285 8.327 69,3 -0,6

No physical security checks 8.026 8.047 65,4 -0,3 9.046 9.053 69,1 -0,1

NA 0 0 0,0 0,0 0 0 0,0 0,0

NA 0 0 0,0 0,0 0 0 0,0 0,0

NA 0 0 0,0 0,0 0 0 0,0 0,0

10 seconds 3.477 3.318 44,5 3,6 4.083 3.834 47,5 5,2

30 seconds 1.768 1.765 32,8 0,1 1.993 2.025 34,9 -0,9

1 minute 2.104 2.151 35,6 -1,3 2.287 2.405 37,5 -3,1

2 minutes 1.975 2.090 35,2 -3,3 2.254 2.365 37,4 -3,0

5 minutes 3.608 3.614 48,0 -0,1 4.265 4.219 51,7 0,9

None 8.026 8.047 65,4 -0,3 9.046 9.053 69,1 -0,1

None of these 7.247 7.219 71,7 0,4 8.339 8.367 76,9 -0,4

Total 28.205 28.205 32.267 32.267

Male Female


36

As described previously the PACT DSS is context-specific and the data model of the PACT data base builds

upon the context as specified in the PACT survey, namely the Travel, Health and Internet scenarios.

However, the design will follow optimal patterns in order to be modular and extensible, having the ability

to integrate future empirical studies in other context, or enrich the existing scenarios without overhead or

need for extended offline time for the PACT DSS system.

4.2 Functional and Technical Requirements

Based on the user requirements presented in chapter 3 and the design considerations expressed through

the DSS architecture in section 4.1, T6.1 proceeded with defining the Functional and Technical

Requirements of the PACT DSS. Functional requirements describe the functions of the DSS, namely the

inputs, the behaviour and the outputs of the system. On the other hand, technical requirements describe

quality traits of the system, characterising its operation and pertaining its technical aspects. Both constitute

essential elements and results of requirement analysis, an important step in software engineering.

Following analysis, functional and technical requirements are described in a way that the needs of the users

are translated into system functions, behaviours and operational characteristics, which are in turn

understandable by developers in order to proceed with system specifications and implementation. For this

purpose, functional and technical requirements are clearly linked to user requirements in order to ensure

all identified real needs are being met.

The requirements are organised according to the Architecture to overall requirements and to individual

component requirements.

4.1.1 Overall System Requirements

In this section the functional and technical requirements corresponding to general functionalities and

technical traits of the system as a whole are presented.

Functional Requirements

This section is dedicated to the activities and processes the system must perform in general and specifically

for information retrieved (input) and delivered (output) as well as for management purposes. Tables with

corresponding requirements follow respectively.


Requirem

ent

Source

Ref.

Ma

nd

ato

ry

Opti

onal

D61.F-

19

Context-specific decision and

assessment case

The user will select the context

and specific security technology

application to be assessed among

available options of the PACT DSS.

D61.U-3 X

D61.F-

20

Assets The DSS will connect with selected

case related tangible and

intangible assets.

D61.U-8 X


37

D61.F-

21

Decision-making process support The user will be offered with

visualised rationale arguments at

each key decision choice point to

support the decision-making

process.

D61.U-11 X

D61.F-

22

Multiple factors assessment The user will assess the impact

through evaluation of multiple

factors: associated risks, privacy,

ethics, social acceptance and

public perception.

D61.U-2 X

D61.F-

23

Theoretical and empirical

knowledge

Relevant theoretical and empirical

knowledge will be provided to the

user in an efficient, relevant and

context-driven manner.

D61.U-4 X

D61.F-

24

Measurable index A measurable index for

assessment will be offer to the

User, the Privacy Impact (or

Threat) Index.

D61.U-7 X

D61.F-

25

Laws and ethics The PACT DSS will provide quick

references to related laws,

regulations, standards as well as

ethics issues, highlighting any

known tension or controversies.

D61.U-6,

D61.U-9

X

D61.F-

26

Storing and retrieving multiple

assessment cases

The user shall be able to save his

work and resume it at a later

point. The User will be able to

hold multiple ongoing assessment

cases.

D61.U-2 X

D61.F-

27

Security Terminology PACT DSS will use a standard

security language based on

relevant European

recommendations (e.g. STACCATO

security taxonomy).

D61.U-5 X

D61.F-

28

Input PACT DSS will be able to receive

input from the user for a

particular security

technology/scenario under

investigation.

D61.U-12 X


38

D61.F-

29

Output: Impact assessment report PACT DSS will produce an impact

assessment report which he can

save it for future reference, print

it or produce an exportable file

and sharable result.

D61.U-1 X

D61.F-

30

Online collaboration PACT DSS will enable multiple user

collaboration on a single decision

problem, offering sharing/inviting,

parallel working and

synchronization functionalities.

D61.U-2,

D61.U-10

X

D61.F-

31

User Profile Management PACT DSS will support user

registration and use via login. This

will effectively support profile

management, personalised user

preferences and personal

assessment cases portfolio.

D61.U-2 X

D61.F-

32

Anonymous login PACT DSS will also operate for

anonymous users, i.e. without

need for login.

D61.U-2 X

D61.F-

33

Social Media –crowdsourcing The PACT DSS will be able to

receive updates regarding social

acceptance and public perception

via social media and

crowdsourcing techniques.

D61.U-4 X


39

Technical Requirements

This section includes the requirements encapsulating the technical aspects the overall system must fulfil in

general, for targeting usage environment and platform, achieving satisfactory response times, scalability

purposes and meeting security, privacy and ethics requirements. Tables with corresponding requirements

follow respectively.


Requirem

ent

Source

Ref.

Ma

nd

ato

ry

Opti

onal

D61.T-34 Online access The PACT DSS will be a web

application and will be accessed

online.

D61.U-17 X

D61.T-35 Platform independence The PACT DSS will be able to

operate smoothly across all

popular platforms and web

browsers.

D61.U-17 X

D61.T-36 System response time The DSS shall be able to respond

to any user request within a

specified period of time.

D61.U-17 X

D61.T-37 Concurrent users support The DSS shall be able to operate

smoothly under increased load.

D61.U-17 X

D61.T-38 Extensibility The PACT DSS shall be able to

interface and use data from

multiple sources.

D61.U-4 X

D61.T-39

Protected Access Access for registered users will be

protected through use of

password.

D61.U-2 X

D61.T-40 Integrity of information The integrity of information

processed by the system must be

checked.

D61.U-18 X

D61.T-41 Disclaimer A disclaimer will be provided to

the users explaining the use of

profile data as well as the

boundaries of the PACT DSS tool.

D61.U-18,

D61.U-2

X


40

4.1.2 Individual Component Requirements

User Layer, HMI, Dialogue Management & Visualisation Module


Requirem

ent

Source

Ref.

Ma

nd

ato

ry

Opti

onal

D61.F-

42

Visualisation Modes Visualisation modes used by PACT

DSS include bar charts, spider

diagrams and colour coded tables.

D61.U-11,

D61.U-14,

D61.U-16

X

D61.F-

43

Minimum effort selection Use of autocomplete and selection

forms will be used to minimise

user effort and overhead.

D61.U-13,

D61.U-12,

D61.U-14

X

D61.F-

44

Assistive Step-by-step procedure The HMI dialogue will be

structured in an assistive step by

step procedure, with the ability of

re-visiting any step at any time

without losing any data or

progress.

D61.U-15,

D61.U-13

X

D61.T-

45

UI fast navigation Navigation throughout the PACT

DSS UI shall be reduced to the

minimum number of clicks.

D61.U-15,

D61.U-13

X



Requirem

ent

Source

Ref.

Ma

nd

ato

ry

Opti

onal

D61.F-

46

Decision Engine The PACT DSS Decision Engine will

be an implementation of

algorithms and structures that will

utilize the existing knowledge to

apply criteria and offer advice on

the scenarios specified.

D61.U-2 X

D61.F-

47

Sequential decision process The user decision process for the

specific case/security technology

will be modelled as a conceptual

decision tree.

D61.U-2,

D61.U-15

X


41

D61.F-

48

Tree optimal path The system will calculate at all

times the tree optimal path for

minimum impact, posing relevant

arguments and indicators.

However the user is responsible

for driving the decision process

throughout the tree at his own

free will and choice.

D61.U-2 X

D61.F-

49

Decision sequence transition Transition through decision

sequence should be allowed

without losing progress or data.

D61.U-2,

D61.U-15

X

Knowledge Management Module & Database Server


Requireme

nt Source

Ref.

Ma

nda

tory

Opti

onal

D61.T-

50 Knowledge Database PACT DSS will hold an updatable

database containing models and

statistics pertaining to privacy

and security concerns, including

fine- and coarse-grained

information and theoretical

knowledge based on context.

D61.U-10,

D61.U-18,

D61.U-4,

D61.U-5,

D61.U-6,

D61.U-8

X

D61.T-

51

Knowledge management Knowledge and data will be

managed through a set of web

services, offering efficient

management.

D61.U-10,

D61.U-18,

D61.U-12,

D61.U-4

X

D61.T-

52

Data model extensibility The data model will be easily

extensible in order to support

updates from future empirical

research.

D61.U-10 X

D61.T-

53

Internal data format An internal data model will be

applied to the knowledge

supplied so it can be

meaningfully stored and

become available for use.

D61.U-10 X


42

5 Requirements Traceability Matrix

The following table, called the Requirements Traceability Matrix, consolidates all requirements defined in

this document, for purposes of easy reference and convenient reading by future WP6 activities.

User Requirements

ID Brief Summary

Requirement Source Functional / Non-

Functional

Domain

D61.U-1 Privacy Impact Assessment

The system should provide a tangible report in the form of a privacy impact assessment for security technologies


Functional General, Privacy

D61.U-2 Impact assessment assistance

Given that 1/3 impact assessment reports are sent back to be re-worked, the PACT DSS should offer personalised support, assistance and informal validation for impact assessment procedures.



D61.U-3 Context-specific

The PACT DSS should focus in the particular context and scope of the scenario/technology under investigation.



D61.U-4 Consolidated knowledge

The system should consolidate theoretical and empirical data from multiple sources.


Non Functional

General, Security

D61.U-5 Security Language

The system should avoid negative connotations connected with specific terms, and should “talk” the same language with the users.



D61.U-6 Laws and Ethics

The system should highlight case-specific connected legal and ethical issues, along with any potential tension between those.


Functional Legal, Ethical

D61.U-7 Measurable Index

The system should use whenever possible measurable index, applied e.g. to PTI.



D61.U-8 Intangible assets

The system should take into account intangible assets


Functional Privacy, Security

D61.U-9 Compliance The system should check if proposed security solution complies with current standards, practices and legislations


Functional Legal, Privacy, Security


43

D61.U-10 Update Knowledge Base

Ability to update the knowledge base including results and standards taken into account etc.


Functional General

D61.U-11 Data Visualisations

The system should provide useful data visualizations to the user.


Functional General

D61.U-12 Data Input The HMI must provide a way to input scenario-specific data


Functional General

D61.U-13 Non-cluttered visualizations

The system’s HMI should not be cluttered with too much data to visualize simultaneously.


Non-Functional

General

D61.U-14 Simple visualizations

Data visualizations should be simple, non-complex with the ability to expand into more complex views if necessary


Non-Functional

General

D61.U-15 Wizard-like interface

The HMI should allow the data input and data processing in clearly defined steps, making it easy for the user to track back their steps if required.


Non-Functional

General

D61.U-16 Direct Manipulation Icons

Use of icons in HMI design, with clear mapping to real-world concepts


Non-Functional

General

D61.U-17 Easy Access Access to the PACT DSS should be easy and shouldn’t require too much ICT skills on behalf of the users


Non-Functional

General

D61.U-18 Reliability PACT DSS should provide reliable information at all times.


Non-Functional

General

Functional and Technical Requirements

Overall System Functional Requirements


Requirem

ent

Source

Ref.

Manda

tory

Opti

onal

D61.F-19 Context-specific decision

and assessment case

The user will select the context and

specific security technology

application to be assessed among

available options of the PACT DSS.

D61.U-3 X

D61.F-20 Assets The DSS will connect with selected

case related tangible and intangible

assets.

D61.U-8 X


44

D61.F-21 Decision-making process

support

The user will be offered with

visualised rationale arguments at

each key decision choice point to

support the decision-making

process.

D61.U-11 X

D61.F-22 Multiple factors

assessment

The user will assess the impact

through evaluation of multiple

factors: associated risks, privacy,

ethics, social acceptance and public

perception.

D61.U-2 X

D61.F-23 Theoretical and empirical

knowledge

Relevant theoretical and empirical

knowledge will be provided to the

user in an efficient, relevant and

context-driven manner.

D61.U-4 X

D61.F-24 Measurable index A measurable index for assessment

will be offer to the User, the Privacy

Impact (or Threat) Index.

D61.U-7 X

D61.F-25 Laws and ethics The PACT DSS will provide quick

references to related laws,

regulations, standards as well as

ethics issues, highlighting any known

tension or controversies.

D61.U-6,

D61.U-9

X

D61.F-26 Storing and retrieving

multiple assessment cases

The user shall be able to save his

work and resume it at a later point.

The User will be able to hold

multiple ongoing assessment cases.

D61.U-2 X

D61.F-27 Security Terminology PACT DSS will use a standard

security language based on relevant

European recommendations (e.g.

STACCATO security taxonomy).

D61.U-5 X

D61.F-28 Input PACT DSS will be able to receive

input from the user for a particular

security technology/scenario under

investigation.

D61.U-12 X

D61.F-29 Output: Impact assessment

report

PACT DSS will produce an impact

assessment report which he can save

it for future reference, print it or

produce an exportable file and

sharable result.

D61.U-1 X

D61.F-30 Online collaboration PACT DSS will enable multiple user

collaboration on a single decision

problem, offering sharing/inviting,

parallel working and

synchronization functionalities.

D61.U-2,

D61.U-10

X


45

D61.F-31 User Profile Management PACT DSS will support user

registration and use via login. This

will effectively support profile

management, personalised user

preferences and personal

assessment cases portfolio.

D61.U-2 X

D61.F-32 Anonymous login PACT DSS will also operate for

anonymous users, i.e. without need

for login.

D61.U-2 X

D61.F-33 Social Media –

crowdsourcing

The PACT DSS will be able to receive

updates regarding social acceptance

and public perception via social

media and crowdsourcing

techniques.

D61.U-4 X

Overall System Technical Requirements


Requirem

ent

Source

Ref.

Manda

tory

Opti

onal

D61.T-34 Online access The PACT DSS will be a web

application and will be accessed

online.

D61.U-17 X

D61.T-35 Platform independence The PACT DSS will be able to operate

smoothly across all popular

platforms and web browsers.

D61.U-17 X

D61.T-36 System response time The DSS shall be able to respond to

any user request within a specified

period of time.

D61.U-17 X

D61.T-37 Concurrent users support The DSS shall be able to operate

smoothly under increased load.

D61.U-17 X

D61.T-38 Extensibility The PACT DSS shall be able to

interface and use data from multiple

sources.

D61.U-4 X

D61.T-39

Protected Access Access for registered users will be protected through use of password.

D61.U-2 X

D61.T-40 Integrity of information The integrity of information

processed by the system must be

checked.

D61.U-18 X

D61.T-41 Disclaimer A disclaimer will be provided to the

users explaining the use of profile

data as well as the boundaries of the

PACT DSS tool.

D61.U-18,

D61.U-2

X

User Layer, HMI, Dialogue Management & Visualisation Module Requirements


46


Requirem

ent

Source

Ref.

Manda

tory

Optio

nal

D61.F-42 Visualisation Modes Visualisation modes used by PACT

DSS include bar charts, spider

diagrams and colour coded tables.

D61.U-11,

D61.U-14,

D61.U-16

X

D61.F-43 Minimum effort selection Use of autocomplete and selection

forms will be used to minimise user

effort and overhead.

D61.U-13,

D61.U-12,

D61.U-14

X

D61.F-44 Assistive Step-by-step

procedure

The HMI dialogue will be structured

in an assistive step by step

procedure, with the ability of re-

visiting any step at any time without

losing any data or progress.

D61.U-15,

D61.U-13

X

D61.T-45 UI fast navigation Navigation throughout the PACT DSS

UI shall be reduced to the minimum

number of clicks.

D61.U-15,

D61.U-13

X

Decision Trees & Engine Module Requirements


Requirem

ent

Source

Ref.

Manda

tory

Optio

nal

D61.F-46 Decision Engine The PACT DSS Decision Engine will

be an implementation of algorithms

and structures that will utilize the

existing knowledge to apply criteria

and offer advice on the scenarios

specified.

D61.U-2 X

D61.F-47 Sequential decision

process

The user decision process for the

specific case/security technology

will be modelled as a conceptual

decision tree.

D61.U-2,

D61.U-15

X

D61.F-48 Tree optimal path The system will calculate at all times

the tree optimal path for minimum

impact, posing relevant arguments

and indicators. However the user is

responsible for driving the decision

process throughout the tree at his

own free will and choice.

D61.U-2 X

D61.F-49 Decision sequence

transition

Transition through decision

sequence should be allowed without

losing progress or data.

D61.U-2,

D61.U-15

X


47

Knowledge Management Module & Database Server Requirements


Requirem

ent

Source

Ref.

Manda

tory

Option

al

D61.T-50 Knowledge Database PACT DSS will hold an updatable

database containing models and

statistics pertaining to privacy and

security concerns, including fine-

and coarse-grained information and

theoretical knowledge based on

context.

D61.U-10,

D61.U-18,

D61.U-4,

D61.U-5,

D61.U-6,

D61.U-8

X

D61.T-51 Knowledge management Knowledge and data will be managed

through a set of web services,

offering efficient management.

D61.U-10,

D61.U-18,

D61.U-12,

D61.U-4

X

D61.T-52 Data model extensibility The data model will be easily

extensible in order to support

updates from future empirical

research.

D61.U-10 X

D61.T-53 Internal data format An internal data model will be

applied to the knowledge supplied so

it can be meaningfully stored and

become available for use.

D61.U-10 X


48

6 DSS Validation Criteria

This chapter defines the validation criteria for the target PACT DSS, including user requirement based

validation and acceptance as well as performance and technical success measures for the system and

individual components. Following completion and delivery of the PACT DSS software tool, its successful

operation will be validated against the criteria defined in this chapter. Depending on the criteria type,

validation will be performed by either by technical experts in case e.g. of performance validation or by end

users in case of user perspective criteria. The latter case will include volunteers from the combined PACT

experts groups. T6.4 “Validation and User Evaluation” is responsible for planning and performing relevant

validation tests and acquiring “hands-on” user evaluation feedback through the T6.4 cluster of end-user

participants and volunteers from external experts groups. This is planned to be conducted during the

combined key event consisting of the PACT consortium meeting, SAG and PAP annual meeting and HLPP

final event, 18-19 December 2014 in Brussels, Belgium.

6.1 User Perspective Criteria Being a user-driven system, the PACT DSS User Perspective Validation Criteria are of outmost importance

for the successful validation of the system. By analysing the user feedback reported in chapter 3, the

following user perspective validation criteria are identified for the PACT DSS:

Reduction on average decision process time (including collaboration with other users)

Improving decision specific context visibility & access to necessary knowledge for related security

technologies decision making

Reduction on average time for compiling a privacy impact assessment report

Convenience and acceptance of PACT impact assessment report

Usability & User friendliness

Cost effectiveness (value for money)

6.2 Component Specific Validation Criteria and Success Measures

User Layer, HMI, Dialogue Management & Visualisation Module

HMI validation will feature qualitative and quantitative evaluation criteria. This goal can be achieved by

combining elements from various Usability Inspection paradigms, focusing on Cognitive Walkthroughs. A

cognitive walkthrough is essentially a qualitative task-driven process, where the end user is requested to

perform specific tasks. During the process, the user is encouraged to comment on the system’s ease of use,

responsiveness etc. Common questions during Cognitive Walkthroughs are, for example23:

Does the user understand which subtasks are needed to reach the user's goal?

Is the correct action visible and available?

Does the user get feedback?

Will the user know if their actions are correct or not?

Etc.

23 Blackmon, M. H. Polson, P.G. Muneo, K & Lewis, C. (2002) Cognitive Walkthrough for the Web CHI 2002 vol.4 No.1 pp463–470


49

By providing the appropriate set of questions during the PACT DSS Validation, the end-users should be able

to offer insight on the efficiency of the HMI design, especially in terms of the main design criteria that they

identified during the PACT workshop (i.e. simplicity, responsiveness, forgiveness, linguistic clarity, aesthetic

integrity, as mentioned in Section 3.2). Apart from providing a questionnaire to the user, it is also possible

to implement a Think-Aloud protocol 24 during system validation. During “Think-aloud” evaluation,

participants are encouraged to provide real-time verbal reports of their user experience.

During the Cognitive Walkthrough, the users will be asked to answer appropriate questions on the look-

and-feel of the HMI design thus providing a qualitative evaluation of the user interface. In addition to a

Cognitive Walkthrough questionnaire, a number of free usability tools can be used, such as:

Mouse tracking software: By tracking mouse movement and click patterns it is easy to estimate user effort in performing a scenario, as well as discovering possible bottlenecks.

Eye-tracking software: By tracking eye movement, it is possible to identify which UI elements dominate the user’s attention and if they become distracting.

Examples of free tools for usability inspection include Simple Mouse Tracking25 and Usabilla26. In cases

where human observation is required, the users will be debriefed and asked for their consent prior to their

participation in the validation process.

A more quantitative approach may also be implemented, by utilizing Fitt’s Law and the Keystroke-level

model (KLM) to estimate user effort (without requiring human observation). Fitts's law (often cited as Fitts'

law) is a theoretical model of human movement that predicts the time required to move to a target area, as

a function of the distance to the target and the target size. Fitts's law is thus used to model the act of

pointing and/or moving a mouse. KLM provides a time estimate for the completion of a task, by breaking it

down in subtasks (pointing, clicking, typing etc.) and taking into account the mental preparation of the user

as they plan their actions. It can be summarized in the following steps27:

Step 1 — “Obtain a working prototype of computer interface or a step-by-step operational description of a

task.”

Step 2 — “Identify the goals or the desired outcome of work”

Step 3 — “For each of these goals, find subgoals or tasks that achieve the main goals.”

Step 4 — “Identify methods to main goals and all subgoals.”

Step 5 —“Convert description of methods to pseudo-code (the terminology that is described above).”

Step 6 — “State any and all assumptions used in the making of pseudo-code and goals.”

Step 7 — “Determine appropriate mental or keystroke operators for each step.”

Step 8 — “Assign time values to mental or keystroke operators.”

Step 9 — “Add up execution times for operators.”

Step 10 —“Adjust total time of task to be sensitive by age of expected.”

By combining both qualitative and quantitative measures, a comprehensive evaluation of the HMI design

will be achieved.

24

Lewis, C. H. (1982). Using the "Thinking Aloud" Method In Cognitive Interface Design (Technical report). IBM. RC-9265. 25 Simple Mouse Tracking http://smt.speedzinemedia.com (accessed February 2014) 26 Usabilla https://usabilla.com (accessed February 2014) 27 Kieras, D. (1993, 2001), "Using the Keystroke-Level Model to Estimate Execution Times", (On-line handout).

http://en.wikipedia.org/wiki/Conceptual_model

http://smt.speedzinemedia.com/

https://usabilla.com/


50


Validation of the Decision Trees & Engine Module will be based on the following criteria:

Extensible design and compliance with any valid tree structure

Generated tree consistency

Tree optimal path consistency

Knowledge Management Module & Database Server

Validation of the knowledge and data management services will be based on the following criteria:

Services failures should occur only in communication failure cases

Integrity checks of data

Data consistency and PACT DSS E-R compliance

Availability of services

6.3 Functionality Validation Criteria The PACT DSS functionalities will be validated according to the functional requirements provided in section

4.2 and to the use cases that will be extracted during the software engineering process (task 6.2). Namely

functionality validation will occur by successfully completing a checklist of all required system

functionalities as described by the requirements in this deliverable.

6.4 Performance Validation Criteria

Given its architecture, the PACT DSS in general inherits the performance traits of the online platform it is

deployed upon. The overall system performance will be validated based on the key operations and

performance traits of the system, as these are identified in the preliminary architecture, user and technical

requirements. The performance criteria involve assessment report creation time, concurrent users support

and system response time. Namely, the following criteria will be used for performance validation:

Assessment report creation time

Maximum concurrent users

Average system response time


51

7 Conclusions

This deliverable presented the output of Task “T6.1 DSS Architecture Technical Specifications and System

Validation definition”. This included a review and assessment of the relevant state of the art of current DSS

development as carried on by relevant EU research projects - PRISMS, PARIS, SURPRISE, VALUSEC and DESSI

– so to identify which input can be drawn from these projects. In parallel, T6.1 partners collected and

extracted user requirements based on the PRFST-DSS joint workshop results as well as feedback gathered

from all PACT stakeholder groups: the Stakeholder Advisory Group, the Privacy Advocates Panel and the

High Level Policy Panel. Following these tasks, the preliminary DSS architecture was defined and described,

producing also the relevant functional and technical requirements and consolidated Requirements

Traceability Matrix for reference. The architecture will be finalised in the task T6.2 software engineering

activities. Finally, the PACT DSS validation criteria were defined including the user perspective criteria along

with the success measures for each DSS component, functionality and performance validation. Following

completion and delivery of the PACT DSS software tool, its successful operation will be validated against the

criteria defined in this document.

To conclude this document, this section sums up the following important points and DSS features:

The PACT DSS is a decision support software tool implementing the PRFST toolbox as described in

PACT D5.2. Required context data is retrieved from the system database, fed into the decision

support and decision-tree process and presented accordingly to the user through multiple

visualisation modes (such as charts, spider diagrams and colour coded tables) as rational arguments

towards supporting the user’s decision process.

The PACT DSS aims to support decision process for security technologies investments and

assessment of impact in terms of privacy, ethics, social acceptance and public perception, assisting

the decision process of experts through efficient visualisation and user-DSS interaction based on

PACT’s theoretical and empirical findings.

The PACT DSS is context-specific and the data model of the PACT data base builds upon the context

as specified in the PACT survey, namely the Travel, Health and Internet scenarios. However, the

design will follow optimal patterns in order to be modular and extensible, having the ability to

integrate future empirical studies in other context, or enrich the existing scenarios without

overhead or need for extended offline time for the PACT DSS system.

PACT DSS aims at the following user-perspective success criteria:

o Reduction on average decision process time (including collaboration with other users)

o Improving decision specific context visibility & access to necessary knowledge for related

security technologies decision making

o Reduction on average time for compiling a privacy impact assessment report

o Convenience and acceptance of the produced PACT impact assessment report

o Usability & User friendliness

o Cost effectiveness (value for money)


52

The PACT DSS is not a legal instrument responsible of validating legality of the user’s decisions. The

user is the expert responsible of taking decisions, and the PACT DSS is a convenient and user-

friendly tool to help him reach to the decision with a better view of associated parameters,

knowledge and arguments. It is important to highlight that the design of the PACT DSS is user-

driven, and decisions are taken by the users themselves, with the system supporting the process

through multiple modalities and presenting quantitative and qualitative information regarding the

impact of each decision path.

o During T6.1 by analysing user needs and defining the DSS behaviour and characteristics (as

these are expressed in terms of functional and technical requirements), we have placed a

significant effort to clearly define the DSS scope and to strengthen appropriate premises

within legal and ethical boundaries.

o PACT will also provide a socio-economic impact study and a political impact study (cf.

activities ST7.3.1 and ST7.3.2, PACT DoW, to be included in PACT D7.5) that will further

clarify the risks and limits of a DSS operating beyond constraints and limitations

expressed here.

Documents

D6 - PACT Project · for the system and individual components. ... 2. Define the preliminary DSS architecture and subsequently define the technical and functional