Deliverable 2.1 ADAS&ME Architectural Framework and System ... · Angelos Bekiaris, CERTH Anna Anund, VTI Christer Ahlström, VTI Christophe René Joseph Ecabert, EPFL Daniel Teichmann,

Adaptive ADAS to support incapacitated drivers Mitigate Effectively risks through tailor

made HMI under automation

This project has received funding from the European Union’s Horizon 2020 research

and innovation programme under grant agreement No 688900

Deliverable 2.1 – ADAS&ME Architectural

Framework and System Specifications

Deliverable Identity

Work Package No. WP2

Work Package Title System Architecture and Specification

Activity No. A2.1 & A2.2

Activity Title System Architecture & Technical Specification

Dissemination level CO

Main Author(s) Sri Venkata Naga Phanindra Akula (TUC)

Luca Zanovello (DUCATI)

File Name ADASANDME_Deliverable_2.1.doc

Online resource http://www.adasandme.com

Ref. Ares(2017)5763972 - 24/11/2017

http://www.adasandme.com/link-to-deliverable

ADAS&ME (688900) D 2.1 – ADAS&ME Architectural Framework and System Specifications

November 2017 Page 2 of 133 Version 1.2

Version History

Date Version Comments

08.05.2017 0.1 First proposal for TOC

07.07.2017 0.2 First draft of system architecture and system

specifications

15.08.2017 0.3 Modified the content as per review comments by

partners from Ford.

04.09.2017 0.41

Update on framework for documenting system

requirements and functional requirements (Tabular

format)

14.09.2017 0.51 Update as per review comments

18.09.2017 0.60 Updated the document with contributions received

from partners

13.10.2017 0.70 First draft of consolidated requirements for review

16.10.2017 0.80 Document shared for Internal Review

26.10.2017 0.90 Updated document as per Internal Review comments

01.11.2017 0.91 Change in the document layout as per review

comments

10.11.2017 1.0 Document with updated layout

21.11.2017 1.1 Consolidation of contributions received from

partners

23.11.2017 1.2 Final version of all contributions from partners

24.11.2017 1.3 Final version of the deliverable submitted to EC

Authors (full list)

Alessia Knauss, AUTOLIV

Alicia Lotz, OVGU

Alexander Burmeister, DLR

Angelos Bekiaris, CERTH

Anna Anund, VTI

Christer Ahlström, VTI

Christophe René Joseph Ecabert, EPFL

Daniel Teichmann, RWTH

Dimitris Gavrilis, UPATRAS

Emmanuel Doucet, VEDECOM

Evangelia Chrysochoou, CERTH

Frederik Diederichs, FRAUNHOFER

George Georgoulas, UPATRAS

Hamid Sanatnama, SEYE

Ivan Dmitriev, VEDECOM

Karel Kreuter, DENSO

Luca Zanovello, DUCATI

Marc Wilbrink, DLR

Marcel Mathissen, FORD

Manelli Matteo, SCANIA

Michael Jüttner, NAVENTIK

Nikica Hennes, FORD

Nikos Dimokas, CERTH

Pantelis MAROUDIS, VALEO

Paul Schouten, TOMTOM

Sri Venkata Naga Phanindra Akula, TUC

Stas Krupenia, SCANIA



Stella Nikoloau, CERTH

Svitlana Finér, SEYE

Yannis Lilis, FORTH

Project Coordinator

Dr. Anna Anund

Research Director / Associate Professor

VTI - Olaus Magnus väg 35 / S-581 95 Linköping / Sweden

Tel: +46-13-20 40 00 / Direct: +46-13-204327 / Mobile: +46-709 218287

E-mail: [email protected]

Legal Disclaimer

The information in this document is provided “as is”, and no guarantee or warranty is given that the information

is fit for any particular purpose. The above referenced authors shall have no liability for damages of any kind

including without limitation direct, special, indirect, or consequential damages that may result from the use of

these materials subject to any liability which is mandatory due to applicable law.

The present document is a draft. The sole responsibility for the content of this publication lies with the authors. It

does not necessarily reflect the opinion of the European Union. Neither the INEA nor the European Commission

is responsible for any use that may be made of the information contained therein.

© 2016 by ADAS&ME Consortium

mailto:[email protected]



Table of Contents

TABLE OF CONTENTS ........................................................................................................................................ 4

INDEX OF FIGURES ........................................................................................................................................... 6

INDEX OF TABLES ............................................................................................................................................. 6

GLOSSARY ........................................................................................................................................................ 8

1 INTRODUCTION ..................................................................................................................................... 11

1.1 SCOPE OF THIS DOCUMENT ............................................................................................................................ 11 1.2 DELIVERABLE STRUCTURE .............................................................................................................................. 12

2 METHODOLOGY .................................................................................................................................... 13

2.1 SYSTEM ARCHITECTURE DEFINITION ................................................................................................................. 13 2.2 ARCHITECTURE DEVELOPMENT PROCESS ........................................................................................................... 13 2.3 FRAMEWORK FOR ADAS&ME SYSTEM SPECIFICATION ....................................................................................... 14

3 ADAS&ME SYSTEM CONCEPT ................................................................................................................ 17

3.1 GENERAL SYSTEM ARCHITECTURE ................................................................................................................... 17 3.2 SYSTEM ARCHITECTURE IN DETAIL ................................................................................................................... 19

3.2.1 Sensor Subsystem (SS) ................................................................................................................... 19 3.2.2 Driver State Monitoring Subsystem (DSMS) .................................................................................. 23 3.2.3 Environmental Situation Awareness Subsystem (ESAS) ................................................................ 35 3.2.4 ADAS&ME Core (ADAS&ME Core) ................................................................................................. 41 3.2.5 Vehicle Automation Subsystem (VAS) ........................................................................................... 47

4 ADAS&ME SYSTEM STRUCTURE ............................................................................................................ 49

4.1 OVERVIEW ................................................................................................................................................. 49 4.2 USE CASE A: ATTENTIVE LONG HAUL TRUCKING – FUNCTIONAL SYSTEM STRUCTURE ................................................ 50

4.2.1 System Requirements .................................................................................................................... 50 4.2.2 Functional Architecture ................................................................................................................. 52

4.3 USE CASE A: ATTENTIVE LONG HAUL TRUCKING – BEHAVIOURAL SYSTEM STRUCTURE .............................................. 53 4.3.1 Physical Architecture ..................................................................................................................... 53 4.3.2 Communication Architecture......................................................................................................... 54

4.4 USE CASE B: ELECTRIC VEHICLE RANGE ANXIETY - FUNCTIONAL SYSTEM STRUCTURE ................................................ 55 4.4.1 System Requirements .................................................................................................................... 55 4.4.2 Functional Architecture ................................................................................................................. 57

4.5 USE CASE B: ELECTRIC VEHICLE RANGE ANXIETY - BEHAVIOURAL SYSTEM STRUCTURE .............................................. 58 4.5.1 Physical Architecture ..................................................................................................................... 58 4.5.2 Communication Architecture......................................................................................................... 59

4.6 USE CASE C: DRIVER STATE BASED SMOOTH & SAFE AUTOMATION TRANSITIONS- FUNCTIONAL SYSTEM STRUCTURE ....... 60 4.6.1 System Requirements .................................................................................................................... 60 4.6.2 Functional Architecture ................................................................................................................. 62

4.7 USE CASE C: DRIVER STATE BASED SMOOTH & SAFE AUTOMATION TRANSITIONS- BEHAVIOURAL SYSTEM STRUCTURE ..... 63 4.7.1 Physical Architecture ..................................................................................................................... 63 4.7.2 Communication Architecture......................................................................................................... 64

4.8 USE CASE D: NON-REACTING DRIVER EMERGENCY MANEUVER .............................................................................. 65 4.9 USE CASE E: LONG RANGE ATTENTIVE TOURING WITH MOTORBIKE – FUNCTIONAL SYSTEM STRUCTURE ........................ 66

4.9.1 System Requirements .................................................................................................................... 66 4.9.2 Functional Architecture ................................................................................................................. 70

4.10 USE CASE E: LONG RANGE ATTENTIVE TOURING WITH MOTORBIKE – BEHAVIOURAL SYSTEM STRUCTURE .................. 71 4.10.1 Physical Architecture ................................................................................................................ 71 4.10.2 Communication Architecture .................................................................................................... 72

4.11 USE CASE F: RAIDER FAINT ....................................................................................................................... 73 4.11.1 System Requirements ............................................................................................................... 73

4.12 USE CASE G: PASSENGER PICK UP/DROP OFF AUTOMATION FOR BUSES-FUNCTIONAL SYSTEM STRUCTURE ................ 74 4.12.1 System Requirements ............................................................................................................... 74



4.12.2 Functional Architecture ............................................................................................................ 76 4.13 USE CASE G: PASSENGER PICK UP/DROP OFF AUTOMATION FOR BUSES - BEHAVIOURAL SYSTEM STRUCTURE ............ 77

4.13.1 Physical Architecture ................................................................................................................ 77 4.13.2 Communication Architecture .................................................................................................... 78

5 SYSTEM SPECIFICATIONS ....................................................................................................................... 79

5.1 SYSTEM SPECIFICATIONS ............................................................................................................................... 79 5.1.1 Specifications template ................................................................................................................. 79 5.1.2 Sensors .......................................................................................................................................... 80 5.1.3 HMI Modules ................................................................................................................................. 81 5.1.4 Interfaces ....................................................................................................................................... 82 5.1.5 Elaboration Units ........................................................................................................................... 84

6 CONCLUSIONS ....................................................................................................................................... 85

7 REFERENCES .......................................................................................................................................... 86



Index of Figures

FIGURE 1 FRAMEWORK FOR ADAS&ME SYSTEM SPECIFICATIONS ...................................................................................... 14 FIGURE 2 ADAS&ME GENERAL SYSTEM ARCHITECTURE ................................................................................................... 18 FIGURE 3 SENSOR SUBSYSTEM ...................................................................................................................................... 19 FIGURE 4 MESSAGE ROUTING USING RABBITMQ ............................................................................................................. 20 FIGURE 5 DIRECT MESSAGE EXCHANGE ROUTING USING RABBITMQ .................................................................................... 21 FIGURE 6 DRIVER STATE MONITORING SUBSYSTEM .......................................................................................................... 23 FIGURE 7 SLEEPINESS DRIVER STATE DETERMINATION ....................................................................................................... 24 FIGURE 8 REST DRIVER STATE DETERMINATION ............................................................................................................... 25 FIGURE 9 STRESS DRIVER STATE DETERMINATION FOR UC A, C, D, AND G ........................................................................... 26 FIGURE 10 STRESS DRIVER STATE DETERMINATION FOR UC E ............................................................................................ 27 FIGURE 11 EMOTIONS DRIVER STATE DETERMINATION FOR USE CASE A ............................................................................... 29 FIGURE 12 EMOTIONS DRIVER STATE DETERMINATION FOR USE CASE B ............................................................................... 29 FIGURE 13 EMOTIONS DRIVER STATE DETERMINATION FOR USE CASE C AND D ..................................................................... 30 FIGURE 14 DISTRACTION DRIVER STATE DETERMINATION FOR UC A, C, D, AND G ................................................................. 31 FIGURE 15 DISTRACTION DRIVER STATE DETERMINATION FOR UC E .................................................................................... 32 FIGURE 16 PHYSICAL FATIGUE DRIVER STATE DETERMINATION FOR UC E&F ........................................................................ 34 FIGURE 17 ENVIRONMENTAL SITUATION AWARENESS SUBSYSTEM ...................................................................................... 36 FIGURE 18 SENSOR DATA FUSION MODULE .................................................................................................................... 37 FIGURE 19 COMMUNICATION MODULE ......................................................................................................................... 38 FIGURE 20 ENVIRONMENTAL STATE DETERMINATION MODULE .......................................................................................... 40 FIGURE 21 ADAS&ME CORE SUBSYSTEM ..................................................................................................................... 41 FIGURE 22 CONCEPTUAL DIAGRAM OF THE DRIVER/RIDER IDENTIFICATION MODULE (DIM). ................................................... 42 FIGURE 23 PM INTERACTION WITH OTHER SUB-SYSTEMS USING RABBITMQ BROKER .............................................................. 43 FIGURE 24 DECISION SUPPORT MODULE ........................................................................................................................ 45 FIGURE 25 HMI CONTROLLER MODULE ......................................................................................................................... 47 FIGURE 26 VEHICLE AUTOMATION SUBSYSTEM................................................................................................................ 48 FIGURE 27 VEHICLE AUTOMATION SUBSYSTEM FOR UC E/F .............................................................................................. 48 FIGURE 28 ADAS&ME TRUCK (USE CASE A) FUNCTIONAL ARCHITECTURE........................................................................... 52 FIGURE 29 ADAS&ME TRUCK (USE CASE A) PHYSICAL ARCHITECTURE ............................................................................... 53 FIGURE 30 ADAS&ME TRUCK (USE CASE A) COMMUNICATION ARCHITECTURE ................................................................... 54 FIGURE 31 ADAS&ME ELECTRIC CAR (USE CASE B) FUNCTIONAL ARCHITECTURE ................................................................. 57 FIGURE 32 ADAS&ME ELECTRIC CAR (USE CASE B) PHYSICAL ARCHITECTURE ..................................................................... 58 FIGURE 33 ADAS&ME ELECTRIC CAR (USE CASE B) COMMUNICATION ARCHITECTURE ........................................................ 59 FIGURE 34 ADAS&ME NORMAL CAR (USE CASE C AND D) FUNCTIONAL ARCHITECTURE ....................................................... 62 FIGURE 35 ADAS&ME NORMAL CAR (USE CASE C AND D) PHYSICAL ARCHITECTURE ............................................................ 63 FIGURE 36 ADAS&ME NORMAL CAR (USE CASE C AND D) COMMUNICATION ARCHITECTURE ................................................ 64 FIGURE 37 ADAS&ME MOTORBIKE (UC E AND F) FUNCTIONAL ARCHITECTURE .................................................................. 70 FIGURE 38 ADAS&ME MOTORBIKE (UC E AND F) PHYSICAL ARCHITECTURE ....................................................................... 71 FIGURE 39 ADAS&ME MOTORBIKE (UC E AND F) COMMUNICATION ARCHITECTURE ........................................................... 72 FIGURE 40 ADAS&ME BUS SIMULATOR (USE CASE G) FUNCTIONAL ARCHITECTURE ............................................................. 76 FIGURE 41 ADAS&ME BUS SIMULATOR (USE CASE G) PHYSICAL ARCHITECTURE ................................................................. 77 FIGURE 42 ADAS&ME BUS SIMULATOR (USE CASE G) COMMUNICATION ARCHITECTURE ..................................................... 78

Index of Tables

TABLE 1 SENSOR SUBSYSTEM INTERFACES ....................................................................................................................... 20 TABLE 2 SENSOR INTERFACE DESCRIPTION WITH INTERFACE MODULE (INPUT) ....................................................................... 22 TABLE 3 DRIVER STATE MONITORING SUBSYSTEM INTERFACES ........................................................................................... 23 TABLE 4 SLEEPINESS DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS .......................................................... 24 TABLE 5 REST DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS ................................................................... 25 TABLE 6 STRESS DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS FOR UC A, C, D, AND G ............................... 26 TABLE 7 STRESS DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS FOR UC E .................................................. 27 TABLE 8 EMOTIONS DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS ........................................................... 30



TABLE 9 DISTRACTION DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS FOR UC A, C, D, AND G ...................... 31 TABLE 10 DISTRACTION DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS FOR UC E ....................................... 32 TABLE 11 PHYSICAL FATIGUE DRIVER STATE DETERMINATION MODULE I/O DATA PARAMETERS UC E&F .................................. 34 TABLE 12 ENVIRONMENTAL SITUATION AWARENESS SUBSYSTEM INTERFACES ....................................................................... 36 TABLE 13 SENSOR DATA FUSION I/O DATA PARAMETERS ................................................................................................. 37 TABLE 14 COMMUNICATION MODULE I/O DATA PARAMETERS .......................................................................................... 38 TABLE 15 ENVIRONMENTAL STATE DETERMINATION MODULE I/O DATA PARAMETERS........................................................... 40 TABLE 16 ADAS&ME CORE INTERFACES ....................................................................................................................... 41 TABLE 17 DRIVER/RIDER IDENTIFICATION MODULE I/O DATA PARAMETERS ......................................................... 42 TABLE 18 PERSONALISATION MODULE I/O DATA PARAMETERS .......................................................................................... 43 TABLE 19 DECISION SUPPORT MODULE I/O DATA PARAMETERS ........................................................................................ 45 TABLE 20 HMI CONTROLLER MODULE I/O DATA PARAMETERS ......................................................................................... 47 TABLE 21 VEHICLE AUTOMATION SUBSYSTEM I/O DATA PARAMETERS ................................................................................ 48 TABLE 22 VEHICLE AUTOMATION SUBSYSTEM FOR UC E&F - I/O DATA PARAMETERS ............................................................ 48



Glossary

ADAS&ME ADAPTIVE ADAS TO SUPPORT INCAPACITATED DRIVERS & MITIGATE

EFFECTIVELY RISKS THROUGH TAILOR MADE HMI UNDER AUTOMATION

ADAS&ME C ADAS&ME CORE

ADAS&ME FA ADAS&ME FUNCTIONAL ARCHITECTURE

ADAS&ME PA ADAS&ME PHYSICAL ARCHITECTURE

CoE CONFIDENCE OF ESTIMATION

CM COMMUNICATION MODULE

DID DIGITAL INFRASTRUCTURE DATA

DSDM DRIVER STATE DETERMINATION MODULE

DSM DECISION SUPPORT MODULE

DSMS DRIVER STATE MONITORING SUBSYSTEM

SDSM SENSORS FOR DRIVER STATE MONITORING

ECG ELECTROCARDIOGRAM

EDA ELECTRO DERMAL ACTIVITY

ESAS ENVIRONMENTAL SITUATION AWARENESS SUBSYSTEM

ESDM ENVIRONMENTAL STATE DETERMINATION MODULE

FSR FORCE SENSITIVE RESISTOR

GPS GLOBAL POSITIONING SYSTEM

GSR GALVANIC SKIN RESPONSE

HMI HUMAN MACHINE INTERACTION

HMI E HMI ELEMENTS

HMI FM HMI FRAMEWORK MODULE

HR HEART RATE

HRV HEART RATE VARIABILITY

IM INTERFACE MODULE

IMU INERTIAL MEASUREMENT UNIT

PM PERSONALIZATION MODULE



PPG PHOTOPLETHYSMOGRAPHY

SEM SENSORS FOR ENVIRONMENT MONITORING

SDFM SENSOR DATA FUSION MODULE

SDSM SENSORS FOR DRIVER STATE MONITORING

SS SENSOR SUBSYSTEM

SSD SOFT SENSOR DATA

SVM SENSORS FOR VEHICLE MONITORING

UC USE CASE

VAS VEHICLE AUTOMATION SUBSYSTEM

VSDM VEHICLE STATE DETERMINATION MODULE

WPCU WEARABLE PLATFORM CONTROL UNIT



Executive Summary

ADAS&ME is a use case-oriented project, which aims to combine driver/rider state with the

situation and environmental context knowledge to manage specific interaction strategies,

which include both HMI and automated functions, to ensure safer and more efficient road

usage.

The deliverable defines a common technical framework, which can guide the overall systems

development, taking into account communication and integration issues, which often

represent threats especially in large projects, where different partners develop their modules

separately.

This deliverable includes:

The General System Architecture: all UC architectures will be based on this high-

level architecture. The UCs will include the subsystems and modules here defined,

also the communication flow between these will be substantially the same;

The System Architecture in detail: all the subsystem and modules specified in General

system architecture are described in terms of the I/O interfaces and I/O data.

The System Requirements are specified based on the UC descriptions for priority

scenarios identified at ADAS&ME deliverable 1.2 “Driver/ Rider models, Use Cases

implementation scenarios”

The UC Functional Architecture: the requirements are used to adapt the general

system architecture for each UC, taking into account its peculiarities, the different

states, and functions to be implemented;

The UC Physical Architecture: in this sections, the architecture for each use case is

designed by considering the existing hardware available able at each demonstrator

and adapting it to the needs of UC.

The UC Communication Architecture: in this section, the deliverable concentrates on

the communication interfaces between the different modules. Data formats, input, and

output interfaces are defined here for each UC;

Specifications: all the relevant hardware modules are here specified. Their level of

detail depends on the different module, i.e. modules already integrated and tested will

not be fully specified.

To sum-up, Deliverable 2.1 is aimed at creating a common structure guiding the work

performed in the other technical work packages (i.e. WP3, WP4, WP5, WP6, and WP7) with

which WP2 is strictly correlated, but many activities have not been finalized yet or have just

started. As a consequence, the data contained here may be incomplete and/or become obsolete

in the future. Therefore an update for this document is expected for M30 when it will be able

to provide the exact information.



1 Introduction

ADAS&ME (“Adaptive ADAS to support incapacitated drivers &Mitigate Effectively risks

through tailor made HMI under automation”) will develop adapted Advanced Driver

Assistance Systems that incorporate driver/rider state, situational/environmental context, and

adaptive interaction to automatically transfer control between vehicle and driver/rider and

thus ensure safer and more efficient road usage. To achieve this, a holistic approach will be

taken which considers automated driving along with information on driver/rider state.

The work is based on 7 identified Use Cases for cars, trucks, buses, and motorcycles, aiming

to cover a large proportion of driving on European roads.

Work package 2 is responsible for the design of an Architectural Framework for ADAS&ME

system requirements and specifications for all ADAS&ME Use Cases.

1.1 Scope of this document

The aim of this document is to make the reader, i.e. every technical resource working on

topics included in WP2, WP3, WP4, WP5, WP6, and WP7, understand functional, technical

requirements, specifications of the to-be-developed ADAS&ME system and the final

architecture of the demonstrators without ambiguity. The confidential nature of the document,

combined with the reader profile, led the authors to create a highly technical and detailed

deliverable, which may be difficult to understand for people not involved in the project.

The document describes a framework to achieve final system specifications from system

requirements and is necessary to accomplish the following tasks:

WP2: perform the risk analysis, for what concerns potential technical risks

WP3: develop the Vehicle State Determination Module (VSDM), the Communication

Module (CM) and the Environmental State Determination Module (ESDM)

WP4: develop the Driver State Determination Module (DSDM) and the

Personalisation Module (PM)

WP5: develop the Decision Support Module (DSM) and the HMI and Automation

strategies

WP6: to ensure the smooth integration of the demonstrators of all the above-

mentioned hardware and software modules and to be an aid while performing their

technical verification before the final evaluation

WP7: support the data collection and the data analysis activity

Considered the variety of these tasks, different partners will not need all the data with the

same level of detail, and besides, will make different uses of the deliverable content. For

example:

Driver state algorithms developers need to have very precise information about the

monitoring sensors performances, while for an a-priori risk analysis a more basic level

of detail is sufficient.

On the other hand, driver state algorithm developers are not interested in physical

specifications (dimensions, operating temperature range, power supply,) which are

very important to perform risk analysis and to execute the items integration in the

demonstrators.

The document is intended to be a living tool and it shall be kept updated until M30, especially

the requirements and specifications sections will be made subject to changes. Both



requirements and specifications are grouped across UCs since the project is UC driven.

Specific work will be needed in the next phase to complete the requirements definition,

identify possible conflicts among them and solve these issues harmonizing them or, if not

possible, making a prioritization of the conflicting ones and keeping only the main ones.

1.2 Deliverable Structure

This document is structured in five chapters explaining the ADAS&ME system architectural

framework and system specifications.

Chapter 1: Introduction outlines the idea of the ADAS&ME project and guides the reader

by detailing the key points present in other chapters of this document.

Chapter 2: Methodology describes the overall process followed to achieve ADAS&ME

System Specifications. The first part of the chapter lays the theoretical foundation and

roadmap to achieve them. The later part describes the general framework to achieve system

specifications and then describes the development process of ADAS&ME system architecture

in high level.

Chapter 3: ADAS&ME System Concept describes the high-level ADAS&ME system

architecture and the system architecture in detail for each building block of ADAS&ME

system architecture.

Chapter 4: ADAS&ME System Structure details on the design approach considered and

followed to realize the ADAS&ME System Concept mentioned in Chapter 3 specific to each

use case. This chapter presents the requirements of each subsystem for each ADAS&ME use

case. These system requirements form the base for design decisions in order to derive

functional, physical and communication architectures for each use case.

Chapter 5: System Specifications quantitatively represents the realized ADAS&ME systems

as specified in Chapter 4.



2 Methodology

This chapter describes the development process followed for ADAS&ME system architecture.

2.1 System Architecture definition

System Architecture diagram is the foundation and fundamental diagram for any project. This

diagram mainly depicts the key systems or subsystems along with their interactions required

to achieve any given system requirements. As time progress, during the various design phase

of the project, this diagram will form the base in designing the further key diagrams like

General Functional Architecture, Functional Architecture for each key module, Physical

Architecture, and Communication Architecture. This approach helps the different persons

(Developers, Testers, Integration Team, and Project and Technical Management team)

involved in the project to communicate end goal to be achieved without any confusion. Also,

this method helps to identify any risks involved during any phase of the project. The

concerned team can take proper actions to eliminate all possible risks even before they

happen.

2.2 Architecture development process

ADAS&ME is a use case driven project. It has 30 different partners working on different

aspects to realize its end goal. Hardware and Software components designed and developed in

this project will be tested on different classes of transport vehicles (Ex: Truck, Electric Car,

Normal Car, Motorbike, and Bus) depending on the use case. Hence architecture design

present in this document should be flexible, configurable, and manageable to realize the goal

of each use case at the respective demonstrator.

In this project, architecture development to realize the identified use cases happen in the

following stages.

1. General System Architecture

2. Functional Architecture per key module

3. General Functional Architecture for each use case

4. Interoperability Information

5. Physical Architecture

6. Communication Architecture

In general, for any project, a use case diagram describes the complex task to be realized in the

project. This complex task is divided into several subtasks. Different tasks of similar

functionality are grouped together to form a subsystem. General System Architecture presents

the high-level view of different subsystems along with the data interactions between other

subsystems. Each subsystem consists of several modules. Hardware and Software components

built up these modules.

Functional architecture for each key module describes the internal details of each subsystem

present in the General System Architecture diagram that is common to most of the use cases

realized in the project. Internal details include a brief description of the subsystem, modules

present in the subsystem, input and output interfaces available to send or receive data.

Information on actual input and output data that is sent using at these interfaces is also defined

here.



Based on the use case description functional architecture for each subsystem is as such or

modified as per the needs of the use case, combined with other subsystems. The combined

representation of all the required subsystems forms the General System Architecture for each

use case. Until this stage, the design decisions are more abstract and logical in nature.

Unlike the previous stages, the last three stages, i.e. interoperability information, physical and

communication architecture, are more focused on the actual implementation aspects of the use

case. Interoperability information describes the actual data that is being communicated

between subsystems inside the overall architecture. Physical and communication architectures

describe the physical hardware and the communication stack used for the realization of the

use case.

This document is a living document and will be updated during the course of the project. So

this document contains information on the tasks that have been already done prior to writing

this report, information about the tasks that are currently in progress and information about

the tasks that will be performed in the future.

2.3 Framework for ADAS&ME System Specification

System Design

ADAS&ME Project Proposal Idea

Survey

State of the Art List of Users User wishes Stakeholder wishes

Needs and Limitations

User needs Stakeholder needs Planned Innovation Limitations Use Cases

System Requirements

Tasks Communication Timing Priorities

Functional Blocks

ADAS&ME System Specifications

Input Data Output Data

Error Handling

Figure 1 Framework for ADAS&ME System Specifications

Figure 1 describes the high-level framework followed to achieve system specifications for

ADAS&ME. At first, the ADAS&ME project proposal idea is looked upon for similar

solutions existing in the literature. Second, opinions of end users and stakeholders were



enquired by an online survey as part of Activity A1.2. Results of this activity are presented at

Deliverable 1.2 (D1.2) “Driver/ Rider models, Use Cases implementation scenarios”

D1.2 also specifies the final use cases that will be realized as part of ADAS&ME based on the

various surveys, workshops conducted and analysis done during the course of the project. As

per the D1.2, all the 7 Use Cases will be selected for implementation, with a focus on the

following priority scenarios:

Use Case A: Attentive long haul trucking (truck)

1st Priority: Safe Stop

2nd Priority: Traffic Jam negotiation

3rd Priority: Overtaking

4th Priority: Handovers

For UC A, although these were the priorities identified by Stakeholders and Users, it is not

certain that such self-driving functionality (SAE level 5 of Automation) will be implemented

within the duration of the project. Given that ADAS&ME is not, in and of itself, intended to

develop self-driving technology, the results described D1.2 will guide future development

within Scania. In contrast, the work completed within ADAS&ME will focus on the driver

monitoring and adaptive HMI components and will be implemented in near-horizon self-

driving capability. Consequently, in the Use Case Details (see Table 46 of D1.2), a more

accurate description of the self-driving functionality (SAE Level 3, ex: the truck will drive

autonomously within a lane and follow traffic at varying speeds. If no traffic is present, it will

drive at the required speed) is presented. Within this description, there is a focus on five

phases:

1. Personalization at start-up

2. Manual driving

3. Handovers

4. Takeovers

5. Safe stop

Use Case B: Electric Vehicle range anxiety (e-car)

1st Priority: EV range problem appears due to a traffic jam

2nd Priority: Driver trusts the system and follows indications to power charging station

For UC B, based on user and stakeholders feedback the scenarios have been re-adopted in

comparison to the first description, thus the following phases will be implemented:

1. Range Anxiety Mitigation

2. Range Incident Mitigation

3. 5km Protection

Use Case C: Drive state-based smooth and safe automation transitions (conventional car)

1st Priority: Unsuccessful handover due to the non-reacting driver and safe stop

2nd Priority: Controlled (automation initiated) handover transitions taking the driver state for

the interaction design into account

3rd Priority: Unsuccessful handover due to the non-reacting driver and ‘follow-me’ connected

driving activation (V2X) in case safe stop is not feasible (follow connected vehicle until a safe



stop is possible)

Use Case D: Non-reacting driver emergency maneuver (conventional car)

1st Priority: Unsuccessful handover and take over the transition

2nd Priority: Controlled (automation initiated) take over transitions based on driver state

3rd Priority: V2V communications about the planned maneuver of the ego vehicle to vehicles

close behind or beside the ego vehicle (enable cooperative maneuver)

Use Case E: Long range attentive touring with motorbike (motorbike)

1st Priority: Assistance, during long range touring, in case of tiredness

2nd Priority: Assistance, during long range touring, in case of inattention

Use Case F: Rider Faint (motorbike)

1st Priority: Activation of active systems if the rider is fainting

2nd Priority: Activation of active systems if the rider is going to faint and is ignoring

assistance

Use Case G: Passenger pick-up/drop-off automation for buses (city bus)

1st Priority: System initiated takeover

2nd Priority: Driver initiated take over



3 ADAS&ME System Concept

End system of ADAS&ME shall realize the priority scenarios for all the seven use cases as

mentioned in the previous chapter. The First part of the chapter concentrates on the general

system architecture. While the second one describes the functional architecture of each

induvial subsystem present in the General System Architecture.

3.1 General System Architecture

Figure 2 shows the general System Architecture of ADAS&ME. The whole ADAS&ME

system is divided into several subsystems and each subsystem is further divided into modules.

Hardware and Software components build up the modules.

The problem that ADAS&ME solves can be well understood from the title of ADAS&ME

project which is “Adaptive ADAS to support incapacitated drivers Mitigate Effectively

risks through tailor made HMI under automation”. The subsystems inside the General

System Architecture are designed to solve this overall problem.

In order to “support incapacitated drivers”, it is necessary to constantly monitor the driver

and know exactly when the driver is incapacitated. DSMS solves the problem of identifying

the driver state. The problem of “Mitigate Effectively risks through tailor-made HMI” is

solved by the ADAS&ME Core Subsystem which is capable of making decisions based on

the driver state and environment situation. Not only under vehicle Automation mode but also

in manual mode, the General ADAS&ME System architecture is an “Adaptive ADAS” as the

ADAS&ME Core can adapt its output based on the input coming from DSMS, ESAS.

Sensor subsystem is placed separately to all the subsystems, thereby enabling all the

subsystems to access the sensor data of their choice. During the development stage of the

project, if there happens to be any request for change of the sensor or addition of a new

sensor, this can be done with a very minimal effort as the software algorithms present at

different subsystems are abstracted from the sensor interface. This design choice was made to

provide a flexible and extensible system, forming a future-proof platform and allowing for

algorithm re-use in case of changes in the system architecture. This also simplifies adapting

the ADAS&ME system to the different platforms provided for the individual use cases.

General system architecture has the following subsystems and modules in it as shown in

Figure 2 (from bottom to top) 1. Sensor Subsystem (SS): This subsystem contains input/output, hardware and software

interfaces to receive data from different classes of sensors namely

a. Sensors for Driver State Monitoring (SDSM),

b. Sensors for Environment Monitoring (SEM),

c. Sensors for Vehicle Monitoring (SVM),

d. Digital Infrastructure Data (DID),

e. HMI Data (HMI D) and

f. Software algorithm outputs executed at other subsystems referred as Soft Sensor Data

(SSD).

Interface Module (IM) present at this subsystem receives the sensor data, processes it

and analyses it in order to be sent to the subsequent layers via publish-subscribe

mechanism is facilitated by the Interface Module. All the Subsystems or Modules in

need of input data can subscribe to sensor data of their interest. In order to synchronize

the data received from different classes of sensors, this module also adds a predefined

header and footer to the actual sensor data when sending to the above layers.



End results of the intermediate software algorithms executed at any subsystem can

share their outputs to the IM. These intermediate results of software algorithms are

referred to as another class of sensor data i.e. Soft Sensor Data (SSD). In order to

differentiate that the SSD shared back to the SS may contain the intermediate results

or the final outputs of software algorithms present in at the subsystem, the data flow

for SSD between different subsystems to the sensor subsystem is shown with a dotted

line in Figure 2.

Environmental Situation Awareness Subsystem (ESAS)

Algorithms for Environmental Situation Determination

ADAS&ME Core (ADAS&ME C)

Personalization Module (PM)Decision Support Module (DSM)

HMI Controller Module (HMI CM)

Vehicle Automation Subsystem (VAS)

Vehicle Automation Module (VAM)

Soft Sensor data

Driver monitoring Data

Environmental/Vehicle Data

Digital Infrastructure

DataHMI Data

Interface Module (IM)

Sensor Subsystem (SS)

Driver State Monitoring Subsystem (DSMS)

Algorithms for Driver State Determination

Figure 2 ADAS&ME General System Architecture

2. Driver State Monitoring Subsystem (DSMS): This subsystem contains a software

input interface to receive data from the SS. On receiving all the required sensor data,

software algorithms present at Driver State Determination Module (DSDM) will

assess the driver state (sleepiness, distraction, etc.). It also has an output interface to

share the combined driver state information of all the software algorithms as an array

to above layers and to SS.

3. Environmental Situation Awareness Subsystem (ESAS): This subsystem contains a

software input interface to receive sensor data from the SS. Vehicle State

Determination Module (VSDM), Communication Module (CM), Environmental State

Determination Module (ESDM) together determine the environment situation. This

computed information is shared via an output interface to ADAS&ME server

subsystem and to SS.

4. ADAS&ME Core (ADAS&ME C): This subsystem contains an input software

interface to receive the direct sensor information and the first level of processed

information from DSMS and ESAS. Personalization Module (PM), Decision Support

Module (DSM) and HMI Controller Module (HMI CM) present at this subsystem



process the received data and come up with a decision strategy. This decision strategy

information is shared via an output interface to VAS and SS.

5. Vehicle Automation Subsystem (VAS): This subsystem contains a bi-directional

interface to receive the decision strategy information. Based on the received input

strategy it may trigger the necessary automation function present at the demonstrator

also inform the status of deployed automation function to ADAS&ME C.

3.2 System Architecture in detail

The following section describes the functionality of each subsystem present in ADAS&ME

General System Architecture in detail. At first, each subsystem is described in terms of its

functionality, different modules present inside the subsystem, input/output interface

descriptions of each subsystem and module. Followed by this each module inside the

subsystem is described in terms of its functionality, input and output data parameters.

All the modules inside the subsystem can access the data available at the input interface of the

subsystem. Similarly, the output data of the subsystem is the combination of output data of all

the modules present in the subsystem.

The naming convention for each interface inside any subsystem is done in a specific order

(Camel case format). For example consider the interface name IReadSoftSensorData_SS,

First letter “I” stands for Interface, next four letters “Read” specify the type of the interface

(Read for input and Write for output, ReadWrite for Bi-directional) Next set of letters

“SoftSensorData” describes the data communicated at this interface and last set of letter after

the underscore character “SS” stands for the name of the subsystem system in short (SS for

Sensor Subsystem)

Note: The functionality of each subsystem described in this subsection will be similar for all

use cases of ADAS&ME unless and until specified explicitly.

3.2.1 Sensor Subsystem (SS)

Sensor Subsystem comprises of all the sensors that are used for sensing the physical quantity

of interest specific to the ADAS&ME project in real time. All sensors are interfaced to IM to

receive sensor data as input. Figure 3 shows the internal arrangement of modules inside SS.

Apart from the interfaces to receive the sensor data from respective sensor output interfaces,

this subsystem has two other interfaces one IWriteSensorData_SS (Output Interface) to

share the input sensor data to subsequent subsystems and two IReadSoftSensorData_SS

(Input Interface) to receive the intermediate results of software algorithms from other

subsystems referred as SSD.

Soft Sensor data (SSD)

Sensors for Driver State Monitoring

(SDM)

Sensors for Environment/Vehicle Monitoring (SEVM)

Digital Infrastructure Data (DID)

HMI Data



IReadSoftSensorData_SS

IWriteSensorData_SS

Figure 3 Sensor Subsystem



Table 1 Sensor Subsystem Interfaces

S No Name of the Interface Interface

type

Connection

From To

01. IWriteSensorData_SS Software

---

IReadSensorData_DSMS,

IReadSensorData_ESAS,

IReadSensorData_Core

02. IReadSoftSensorData_SS Software

IWriteSoftSensorData_DSMS,

IWriteSoftSensorData_ESAS,

IWriteSoftSensorData_Core

---

3.2.1.1 Interface Module (IM)

Interface Module acts as a gateway between the ADAS&ME System and the outside world.

This module has a hardware/software interface for each ADAS&ME sensor. As different

sensors have different update rates, this module retrieves the data from each induvial sensor

interface, generalizes the retrieved sensor data into ADAS&ME specific data format by

adding a predefined header and footer. This generalized data in ADAS&ME format is shared

with the subsequent subsystems. IM module also facilities for other subsystems subscribe to

sensor data of interest. On receiving any new sensor data this module would publish it to all

the interested subsystems.

In order to de-couple the complexity of interfacing all sensors between each other or through

a central integration point, a message broker will be employed to facilitate information

exchange between sensors/subsystems. This message broker (e.g. RabbitMQ) will make

available a number of exchanges under which messages will be published. Depending on the

exchange protocol, a number of possibilities for routing will be possible. In the simplest

scenario that can be seen in the figure below (according to the AMPQ model), one sensor can

publish a message to an exchange. The exchange, in turn, will distribute this message to one

or more queues in which other sensors have subscribed to and will eventually receive the

message.

Source: https://www.rabbitmq.com/tutorials/amqp-concepts.html

Figure 4 Message Routing using RabbitMQ

Direct communication is also feasible through the use of routing keys. The exchange

publishes the message in queues according to a routing key (see figure below). It also possible

to route messages to all queues (broadcast).

Other protocols include STOMP or MQTT where the latter targets specifically devices with

limited resources and are significantly simpler. The choice behind RabbitMQ lies in the wide

number of protocols and clients it supports.

https://www.rabbitmq.com/tutorials/amqp-concepts.html



There are many ways to implement the integration through the use of a message broker. In

one scenario, each sensor binds to a specific queue (direct communication of information to

the sensors that require that information), in another, the sensor binds to multiple topics

(information is published to topics according to each type and sensors consume information

according to their need). It is always possible to set up an RPC interface for the direct

synchronous sensor to sensor communication.

Source: https://www.rabbitmq.com/tutorials/amqp-concepts.html

Figure 5 Direct message exchange routing using RabbitMQ

The information being exchanged is transmitted as the message's payload. It can be any byte

stream (except STOMP which requires text format). It is proposed to use lightweight JSON or

XML formats so that it is easier to encode/parse them and be backward compatible in cases

where the format of the message needs to be amended.

The table given below describes major sensors along with their interface descriptions that are

specifically used in ADAS&ME project for estimating driver state

https://www.rabbitmq.com/tutorials/amqp-concepts.html



Table 2 Sensor Interface Description with Interface Module (Input)

Sensor

class

Name of the

sensor Measured parameter Interface description

Specifications S

DS

M

SEYE Pro

Head position

Head rotation

Gaze direction

Eyelid opening

Pupil diameter

Blink detection

Fixation detection

Saccade detection

Driver identity

Video signal

Link

E4

Bio Signal Heart Rate, Blood Volume Pulse,

Galvanic Skin Response,

Skin Temp, Acceleration

Link

Steering Wheel Driver Hand Position

Link

UWB

Heart Rate, Respiration

Rate, and Heart Rate

Variability Link

RWTH Seat

Capacitive

Electrocardiography, Magnetic Induction,

and Photoplethysmography

Link

Mic + Interface Voice Signal

Link

Wearable

Sensors Galvanic Skin Response,

Electrocardiography etc.

Link



Output data of Interface Module:

Interface module converts the input sensor data into ADAS&ME Standard format and then

publishes it to the subsequent modules for processing. Details about the ADAS&ME Standard

format, metadata of actual communication data between any two modules of the architecture

will be detailed in the next version of this deliverable.

3.2.2 Driver State Monitoring Subsystem (DSMS)

The Driver State Monitoring Subsystem consists of the Driver State Determination Module

for assessment of the current state of the driver. This subsystem receives the sensor data of

interest form SS via the interface IReadSensorData_DSMS (Input). All the modules inside

this subsystem can access the data present at IReadSensorData_DSMS interface. There is

one software algorithm per driver state that processes the received data and derive a criticality

level and a confidence level. This data is shared with other subsystems via

IWriteSoftSensorData_DSMS (Output) and IWriteDriverState_DSMS (Output).

Figure 6 Driver State Monitoring Subsystem

Table 3 Driver State Monitoring Subsystem Interfaces


type

Connection

From To

01. IReadSensorData_DSMS Software IReadSensorData_SS ---

02. IWriteSoftSensorData_DSMS Software --- IWriteSoftSensorData_SS

03. IWriteDriverState_DSMS Software --- IReadDriverState_Core

3.2.2.1 Driver State Determination Module

The Driver State Determination Module consists of several software algorithms aiming at the

detection of the current driver state. There is one software algorithm per driver state that

processes the received data and derive a criticality level and a confidence level. This data is

meant to be used as an input parameter for the Decision Support Module (DSM).

Sleepiness:

Sleepiness has been defined as a physiological drive to fall asleep (Dement and Carskadon,

1982), and driver sleepiness has consequently been defined as when a driver has to make an

effort to remain awake while driving (Anund et al., 2008). Sleepiness algorithm will be

developed using Supervised Learning and will be based on analysing data for a certain period



of time. Data like head nodding frequency, blinking frequency and/or blinks are too slow,

eyes are closed for too long, gaze fixated for too long, constant acceleration for too long,

sharp wheel corrections frequency, etc. Different indicators will probably have different time

periods. In addition to that, it will be decided later whether the set of indicators will be the

same or different for manual and automated driving. If it will be the same, then the algorithm

will be based on the information coming only from the Smart Eye Systems. If it will be

different, then the algorithm will use additional information coming from other

sensors/systems. Sleepiness algorithm will be validated using KSS (Karolinska Sleepiness

Scale) driver’s self-rating performed every 5 min of driving.

Figure 7 Sleepiness Driver State Determination

Table 4 Sleepiness Driver State Determination Module I/O Data Parameters

Type Data Parameters From Measure Unit Relevance

Inp

uts

Video (Driver):

· Head movement

· Gaze direction

· Eyelid opening (Blinks)

SDSM Refer SDSM I/O Must Have

ECG:

· Heart rate (HR)

· Heart rate variability

(HRV)

SDSM Refer SDSM I/O Optional

Real Time Clock:

· Time

SDSM - Must Have

Kinematics:

· Speed

· Acceleration

· Steering wheel angle

· Braking

ESDM Refer ESDM I/O Optional

Lane keeping ESDM - Optional

Global Positioning System:

· Longitude and Latitude


Automated or manual driving TBD TBD Optional

Outp

u

ts

Severity levels of sleepiness

state:

1, 2, 3

DSMS Enumerated Data type




Quality factor of sleepiness

estimation: (CoE)

· Confidentiality level

DSMS Percentage (0-100%)

* Note! Here we mark data as “must have”if it is the same for the real-time automated and not

automated driving. “Optional” data is the data only available in the manual driving mode or

not finalized for the real-time use.

Rest:

Rest is a state characterized by relaxation, and in most cases, mental and physical inactivity.

Thus, the definition of rest means that no work is undertaken. Rest typically involves getting a

break from driving, either by stopping and getting out of the vehicle or by means of

automation. The physiological correlates of successful rest are decreased physiological

activation, e.g. lower heart rate, blood pressure and reduced muscle tension. If fatigue is

mainly related to sleep and sleepiness, one may assume that eye blink indicators should

respond to the resting activity.

Figure 8 Rest Driver State Determination

Table 5 Rest Driver State Determination Module I/O Data Parameters


Inputs

ECG/PPG: Heart rate

Heart rate variability SDSM Refer SDSM I/O Must Have

Real Time Clock:

time SDSM - Must Have

Electrodermal Activity:

Phasic and tonic activity SDSM Refer SDSM I/O Must Have

Accelerometer (wrist) SDSM Refer SDSM I/O Must Have

Video (Driver):

Head rotation

Eye movements

Blink duration


Personal data:

Gender Pers. system

Refer Pers. System

I/O Optional

Rest

Video (Driver) ECG EDA

Accelero-meter GPS

Real Time Clock

Automated/manualdriving

Personal data

Severity level of rest (1 -Rest, 0 – Not rest)

Quality factor for restestimation (confidence level)




Age

Driving experience

Defective vision

Outp

uts

Levels of rest state: Yes/No DSMS 0 or 1, yes or no

Quality factor of rest

estimation: Confidentiality

level: CoE


Stress:

Stress is the “Anticipation of a situation that cannot be mastered using available

resources” which is characterized by a feeling of strain and pressure. In this project

Workload is understood as a situation rather than a state: “In a simplistic way,

workload concept can be defined and perceived as a demand placed on the man”.

According to this definition, the workload can be cause for stress. Indicators for

stress identification vary for different ADAS&ME use cases, whereas for the

motorcycle Use Case E a dedicated diagram and table apply due to the different

sensor setup (this is an optional state though for UC E according to D1.2).

Figure 9 Stress Driver State Determination for UC A, C, D, and G

Table 6 Stress Driver State Determination Module I/O Data Parameters for UC A, C, D, and G


Inputs

Video (Driver):

Dilated pupils

Tunnel vision

Head rotation

Eye movements

Blink duration

SDSM Refer SDSM I/O

Must Have

(UC CD)

Optional

(UC A/G)

ECG waveform

Heart rate SDSM Refer SDSM I/O Must Have




Heart rate variability

PPG waveform SDSM Refer SDSM I/O Must Have

Respiration SDSM Refer SDSM I/O

Must Have

(UC CD)

Optional

(UC A/G)

Electro dermal Activity:

(EDA)

Phasic and tonic activity


Personal data:

Gender

Age

Driving experience

Defective vision

Pers. system Refer Pers. System

I/O

Must Have

(UC CD)

Optional

(UC A/G)

Vehicle data:

Steering wheel angle


rate

Vehicle speed (lat.,

long.)

Vehicle acceleration

(lat, long)

Braking

Throttle pedal position

Yaw rate

ESDM Refer ESDM I/O Must Have

Outp

uts

Levels of stress state:

1, 2, 3 DSMS

Enumerated data

type:

Normal, increased,

high

Quality factor of rest

estimation:

Confidentiality level (CoE)

DSMS scale 0.0-1.0

Figure 10 Stress Driver State Determination for UC E

Table 7 Stress Driver State Determination Module I/O Data Parameters for UC E



Type Data Parameters From Measure Unit Relevance Alias In

puts

FSR pressure sensors:

Pressure on

footrests and

handlebar

(stretching

activity)

SS Refer SS I/O Optionals

Skin temperature sensors:

Skin temperatures

(in different body

regions)

SS Refer SS I/O Optional

ECG:

Heart rate, Heart Rate

Variability

SS Refer SS I/O Must

Have

PPG:

Heart rate SS Refer SS I/O

Must

Have

Respiration sensor:

Respiratory rate SS Refer SS I/O

Must

Have

Blood pressure (indirect

measure using ECG and

PPG)

SS Refer SS I/O TBD

EDA (GSR) sensor:

electrodermal

activity SS Refer SS I/O

Must

Have

GPS/Motorcycle:

Motorcycle speed SS Refer SS I/O Optional

GPS:

Statistical data SS Refer SS I/O Optional

Kinematics:

speed acceleration lean angle accelerator and

throttle position brakes use

(wheel/master

pressure)

SS Refer SS I/O Optional

Personal Data:

age

sex physiological

parameters

PM Refer PM I/O Optional

Outp

uts

Levels of stress state:

1,2,3 DSMS

Enumerated

Data type

Quality factor of stress

estimation:

Confidentiality level

DSMS Percentage (0-

100%)



Emotions:

In psychological research it is common state of the art that emotions reflect short-term states,

usually bound to a specific event, action, or object that is expressed by measurable bodily-

reaction patterns (facial expressions, voice characteristics, bio physiological parameters).

With respect to the desired Use Cases (UC) this driver state focuses on four emotional states:

Neutral, Positive, Frustrated and Anxious with a particular focus on Range Anxiety for UC B.

Range anxiety is defined as the unpleasant feeling occurring when not being able to reach a

desired destination or the next charging spot while traveling in an EV.

There will be no severity levels of the emotional states. The driver is either in one of the

critical states (positive, frustrated and anxious) or in neutral state.

For different UCs there will be different parameters to indicate the emotional state. For UC B

all indicators will be used (cf. Figure 10b). UC C/D will only use indicators extracted from

audio and video signal (cf. Figure 10c) and UC A will only use indicators from audio signal

(cf. Figure 10a). Additionally, for UC B particular vehicle data is needed to clearly distinguish

between anxiety and range anxiety.

Figure 11 Emotions Driver State Determination for use case A

Figure 12 Emotions Driver State Determination for use case B



Figure 13 Emotions Driver State Determination for use case C and D

Table 8 Emotions Driver State Determination Module I/O Data Parameters


Inputs

Audio:

Voice signal / acoustic

characteristics

SDSM

Refer SDSM I/O

Must Have

Audio:

Linguistic information of

Speech:

Word frequency

Word sequences

Emotional key words

Speaking rate


Video:

Video signal / action units SDSM Refer SDSM I/O Must Have

Video:

Head rotation

Eye movement


Bio signals:

Heart rate

Respiration rate

Galvanic skin response

Heart Rate Variability

SDSM Refer SDSM I/O Must have

Personal data:

Gender

Age

Pers. system Refer Pers. System

I/O Must Have

Vehicle data:

Range level

Acceleration patterns

ESDM Refer ESDM I/O Must Have

Outp

uts

Emotions:

Neutral

Positive

Frustration

Anxiety/Range anxiety

DSMS Binary (active – 1;

inactive - 0)

Quality factor of emotion

estimation:





Distraction:

A driver is considered attentive when he or she is able to form and maintain a mental

representation of the traffic situation. Ideally, a driver should only be considered inattentive or

distracted when information sampling is not sufficient to form this mental representation. In

ADAS&ME, foveal vision is used as a proxy for attention, and distraction is, in all but UC E,

quantified based on glances away from a so called Field Relevant for Driving (FRD) that are

too frequent or too long. In UC E, visual distraction is quantified based on surrogate

measures, such as head position. Due to the major differences between UC E and the rest of

UCs (A, C/D, G) two completely different algorithms will be developed leading to different

architectures for the distraction module.

Figure 14 Distraction Driver State Determination for UC A, C, D, and G

Table 9 Distraction Driver State Determination Module I/O Data Parameters for UC A, C, D, and G


Inputs

Video

Gaze direction

Head rotation


Driving performance:


Steering wheel angle rate

Vehicle speed (lat., long.)

Vehicle acceleration (lat,

long)

Braking

Throttle pedal position

Yaw rate


Kinematics

Braking

Acceleration


Environment:

GPS coordinates

Upcoming intersections

Traffic intensity





Surrounding vehicles position

Etc.

ECG/PPG:

Heart rate

Heart rate variability


Real Time Clock:

time


Video (exterior):

Video in front of the vehicle


Video (interior):

In cabin video


Outp

uts

Levels of distraction state:

1, 2, 3

DSMS

Quality factor of distraction

estimation:



Figure 15 Distraction Driver State Determination for UC E

Table 10 Distraction Driver State Determination Module I/O Data Parameters for UC E


Inputs

Head and Torso IMUs:

Head position (relative position

with regard to torso)

SDSM Refer SDSM I/O Must have

ECG:

Heart Rate



PPG:

Heart rate SDSM Refer SDSM I/O Optional

Respiration sensor:

Respiratory rate


Blood pressure (indirect measure using

ECG and PPG)


Kinematics

speed SDSM Refer SDSM I/O Must Have




acceleration lean angle accelerator and throttle position brakes use (wheel/master

pressure)

Real Time Clock:

time


Outp

uts

Levels of distraction state:

0-1 (not detected , inattentive)

DSMS

Quality factor of distraction

estimation:



Physical Fatigue:

Physical fatigue comprises two sub – states: muscular fatigue and thermal impairment, both

strictly related to the motorcycle riding experience. Muscular fatigue due to riding can be

broadly divided into two major activities: maintaining the body posture and generating the

required forces to control the vehicle. Thermal impairment of motorcycle riding task is a state

of physiological impairment caused by long-term exposure to extreme temperatures (only hot

temperatures are examined) and/or impeded thermal regulation. In ADAS&ME project

thermal impairment comprises the following three sub – states including: thermal discomfort,

hyperthermia and thermal faint (heat syncope). Thermal impairment is often initially

experienced with simple discomfort, then hyperthermia (that is coupled with dehydration)

and subsequently in extreme cases the rider may faint. Hyperthermia is an elevated body

temperature due to failed thermoregulation that occurs when a body produces or absorbs more

heat than it dissipates. The term dehydration refers to a deficit of total body water, with an

accompanying disruption of metabolic processes. Heat syncope, also known as thermal

fainting, is defined as a short loss of consciousness and muscle strength, characterized by a

fast onset, short duration, and spontaneous recovery.



Figure 16 Physical Fatigue Driver State Determination for UC E&F

Table 11 Physical Fatigue Driver State Determination Module I/O Data Parameters UC E&F


Inputs

Gloves IMUs:

Hands vibrations SS Refer SS I/O TBD

FSR pressure sensors:

Pressure on footrests

and handlebar

(stretching activity)

SS Refer SS I/O TBD

Riding time SS Refer SS I/O Must Have

Skin temperature sensors:

Skin temperatures (in

different body regions)

SS Refer SS I/O Must Have

ECG:

Heart rate,


SS Refer SS I/O TBD

PPG:

Heart rate SS Refer SS I/O TBD

Respiration sensor:

Respiratory rate SS Refer SS I/O TBD

Blood pressure (indirect

measure using ECG and PPG) SS Refer SS I/O TBD

EDA (GSR) sensor:

electrodermal activity SS Refer SS I/O TBD

Air Temperature Sensor:

Air temperature


Air Humidity Sensor:

Air relative humidity SS Refer SS I/O Must Have

UV sensor:

Daylight (UV radiation) SS Refer SS I/O Must Have

GPS/Motorcycle:

Motorcycle speed SS Refer SS I/O Must Have

GPS: SS Refer SS I/O Must Have




Statistical data

Kinematics:

speed acceleration lean angle accelerator and throttle

position

brakes use

(wheel/master pressure)


Personal Data:

age

sex physiological

parameters

PM Refer PM I/O TBD

Outp

uts

Levels of physical fatigue state:

for UC E : 0, 1, 2

(uncritical, critical,

risky) for UC F : 0,1, 2, 3 (not

detected, predicted

risky, predicted

imminent, detected)

DSMS Enumerated Data type

Quality factor of physical

fatigue estimation:



3.2.3 Environmental Situation Awareness Subsystem (ESAS)

Environmental Situation Awareness Subsystem consists of three major modules for predicting

the current environment situation around the demonstrator vehicle

1. Sensor Data Fusion Module

2. Communication Module

3. Environmental State Determination Module

This subsystem receives the sensor data of interest form SS via the interface

IReadSensorData_ESAS (Input). All the modules inside this subsystem can access the data

present at IReadSensorData_ESAS interface. Software algorithms process the received data

and share it to other subsystems via IWriteSituationState_ESAS (Output) and

IWriteSoftSensorData_ESAS) (Output).




Veh

icle State

IReadSensorData_ESAS

IWriteSituationState_ESAS

Sensor Data Fusion Module (SDFM)

Particle filtering techniques

Environmental State Determination Module (ESDM)

Veh

icle

context

Situational Awareness

Emergen

cy In

form

ation

List of fused

ob

jects

Veh

icle State

Enviro

nm

ent

Situatio

n

IWriteSoftSensorData

Communication Module

Digital Infrastructure V2X Data Sharing V2X Data Integration

Co

E

Figure 17 Environmental Situation Awareness Subsystem

Table 12 Environmental Situation Awareness Subsystem Interfaces


type

Connection

From To

01. IReadSensorData_ESAS Software IReadSensorData_SS ---

02. IWriteSoftSensorData_ESAS Software --- IWriteSoftSensorData_SS

03. IWriteSituationState_ESAS Software --- IReadSituationState_Core

3.2.3.1 Sensor Data Fusion Module (SDFM)

The sensor data fusion module contains algorithms for the vehicle state estimation (ego-

localization).

The software handles GNSS Signal Processing, probabilistic error modeling and sensor data

fusion using particle filter techniques. Other than legal GNSS Systems, NAVENTIK‘s

Software enables GNSS satellite navigation for safety-critical ADAS applications by

replacing the traditional black box satellite receiver. All possible error sources from signal

source to the sensor fusion layer are modelled statistically.

Additional to GNSS raw radio samples captured by a low-cost frontend (the front end

hardware is delivered by NAVENTIK) the sensor data fusion module gets measurements

from sensors present in the demonstrator vehicle (DLR) such as inertial sensors or vehicle

odometry. The outputs of the module are related to the vehicle state: position (x, y, and z),

heading and speed.



Sensor Data Fusion Module (SDFM)


Veh

icle Speed

Yaw R

ate

Wh

eel Speed

NA

VEN

TIK End

urance

Lon

gitud

ina

l A

cceleration

Latera

l A

cceleration

Veh

icle P

ositio

n

Veh

icle H

ead

ing

Veh

icle Spee

dVehicle State

Co

E

Figure 18 Sensor Data Fusion Module

Table 13 Sensor Data Fusion I/O Data Parameters

Type Data Parameters From Measure

Unit Relevance Alias

Input Vehicle Speed SS (SVM) m/s Must Have

Yaw Rate SS (SVM) Rad/s Must Have

Wheel Speed SS (SVM) m/s Must Have

Outp

ut

Vehicle Position ESAS

(SDFM) Lat/Lon Must Have

Vehicle State

Vehicle Heading ESAS

(SDFM) Rad/s Must Have

CoE ESAS

(SDFM) % value Must Have

Vehicle Speed ESAS

(SDFM) m/s Must Have

3.2.3.2 Communication Module (CM)

The communication module consists of three sub-modules, namely the Digital Infrastructure,

V2X Data Sharing and the V2X Data Integration.

The Digital Infrastructure together with its cloud backend provides information on the road

ahead, routing information and points of interest (POI). The V2X Data Sharing handles the

information exchange with other vehicles to enlarge the understanding of the local, dynamic

environment and to coordinate maneuvers. The V2X Data Integration module then integrates

data with information coming from on-board sensors, which detect other road participants.

ESDM receives the information from the Digital Infrastructure and the enhanced list of road

objects. More detailed information about the messages can be found in Table 14

Communication Module I/O Data Parameters.



Communication Module

Digital Infrastructure V2X Data Sharing

Percep

tion

(O

n-b

oard

sen

sors)

Veh

icle State

List or M

ap o

f Fu

sed O

bjects

Vehicle Context

Digital

Infrastru

cture

Data

V2X Data Integration

Digital Infrastructure

Remote Vehicles

V2

X

Messages

Figure 19 Communication Module

Table 14 Communication Module I/O Data Parameters

Type Data

Parameters From

Measure

Unit Relevance

Name or

Alias

Input

Vehicle State &

uncertainty

ESAS

(SDFM)

Refer SDFM

I/O Must Have

Spatial

dimensions &

uncertainty

SS (SVM) m Optional Perception

(on-board

radar,

LIDAR,

camera

sensors)

Object type SS (SVM) -- Optional

Detection

confidence SS (SVM) Double [0,1] Optional

Input/output

Current Location ESAS (DI) Lat/Lon Must Have Location

Destination

Location ESAS (DI)

Lat/Lon or

Address Must Have

Driving Task

Intensity ESAS (DI)

Integer

[0,100] Must Have

Road

Information

Amount of lanes ESAS (DI) Integer Must Have

Lane curvature ESAS (DI) Boolean Must Have

Lane changes ESAS (DI) Boolean Must Have

Roadwork ESAS (DI) Lat/Lon,

Polygon Must Have

Speed limit ESAS (DI) Km/h Must Have

Traffic flow ESAS (DI) % of free

flow Must Have

Weather

precipitation ESAS (DI)

Lat/Lon +

mm/h Must Have

Severe weather ESAS (DI) Lat/Lon +

Scale Must Have

Route ESAS (DI) Polyline Must Have

Routing

Traffic Incident

Data ESAS (DI) Polyline Must Have

Map ESAS (DI) JSON Must Have



Type Data

Parameters From

Measure

Unit Relevance

Name or

Alias

POI ESAS (DI) Lat/Lon Must Have POI

Collaborative

Awareness

Message (CAM)

ESAS (RV) ETSI EN

302 637-2 Must Have

V2X

Messages

(Highlights)

Decentralized

Environmental

Notification

Message

(DENM)

ESAS (RV) ETSI EN

302 637-3 Must Have

Collective

Perception

Message (CPM)

ESAS (RV) Proposal in

D3.1 Must Have

Coordinated

Maneuver

Message (CMM)

ESAS (RV) Proposal in

D3.1 Must Have

V2X Message Descriptions:

CAM (Cooperative Awareness Message): contains position, speed, heading, and some

other useful data of the originating vehicle. Standardized in ETSI as ETSI EN 302

637-2.

DENM (Decentralized Environment Notification Message): contains event

information like broken down vehicle, construction site etc. Standardized in ETSI as

ETSI EN 302 637-3.

CPM (Collective Perception Message): contains local sensor detected objects shared

between vehicles. In drafting stage in ETSI.

CMM (Coordinated Manoeuvre Message): contains handshakes, parameters exchange

for carrying out coordinated manoeuvre. Not standardized. Message format not

specified yet.

3.2.3.3 Environmental State Determination Module (ESDM)

Environmental State Determination Module determines the current environment situation and

assesses it by taking into account the current vehicle state and vehicle context information.

Situation Awareness mainly happens on three levels:

1. perceiving elements in the environment within a volume of space and time;

2. comprehending what they mean in context; and

3. predicting their status in the near future

This predicted information is then correlated with the actual environment for assessing the

current situation.





Veh

icle State

Veh

icle C

ontext

No

rmal

Critical

Dangerou

s

Environmental Situation State

Situational Assessment

Co

E

Figure 20 Environmental State Determination Module

Table 15 Environmental State Determination Module I/O Data Parameters

Type Data

Parameters From Measure Unit Relevance Alias

Input/Output

Vehicle State ESAS

(SDFM)

Refer SDFM

I/O Must Have

Vehicle Context ESAS

(CM) Refer CM I/O Must Have

Output

Situation States:

Normal, Critical,

Dangerous

ESAS

(ESDM)

Enumerated

Data type Must Have

Environmental

Situation State

CoE ESAS

(ESDM) % value Must Have

Note: for UC E&F the ESAS is a simplified subsystem, where the ESDM delivers only a

broad weather estimation. For further information, please refer to UC E&F requirements and

functional architecture (Sec.4.10 and 4.11).


November 2017 Page 41 of 133

Version 1.2

3.2.4 ADAS&ME Core (ADAS&ME Core)

ADAS&ME Core subsystem consists of Personalization System, Decision Support Module, and HMI Controller Module. These modules address

the core functionality of ADAS&ME. This subsystem consists of five interfaces. IReadDriverState_Core (Input) to read the current driver state,

IReadSensorData_Core (Input) to read the sensor data coming from SS, IReadSiutationState_Core (Input) to read the current environment

situation, IRead/WriteDecisionStrategy_Core (Input/ Output) to inform the decision strategy with other subsystem and also to get a response

from another subsystem about the computed decision strategy. IWriteSoftSensorData_Core (Output) share the necessary outputs of modules

to SS.

ADAS&ME Core (ADAS&ME C) IReadDriverState_Core IReadSituationState_CoreIReadSensorData_Core

IRead/WriteDecisionStrategy_Core

Personalization Module (PM) Decision Support Module (DSM)

Fuzzy Rules


HMI Elements Personalized HMI

Driver behavioral profile

Driver decisions and actions

Dri

ver

Pref

eren

ces

System decisions and driver actions Historical data from storage

Environm

ent

Cond

ition

Veh

icle State

Driver p

rofile

Oth

er

Pers

ona

lized

H

MI d

ata

Driver Profile and State

Decisio

n Strategy

Emergency

Inform

ation

Environmental Situation


Decisio

n Strategy

Emergency

Inform

ation

List of fused

objects

Vehicle State

Environm

ent

Situatio

n

Inattention

Sleepiness

Emotio

n

Stress

Figure 21 ADAS&ME Core Subsystem

Table 16 ADAS&ME Core Interfaces


type

Connection

From To

01. IReadDriverState_Core Software IWriteDriverState_DSMS ---

02. IReadSensorData_Core Software IWriteSensorData_SS ---

03. IReadSituationState_Core Software IWriteSituationState_ESAS ---

04. IRead/WriteDecisionStrategy_Core Software --- IRead/WriteDecisionStrategy_VAS

05. IWriteSoftSensorData_Core Software --- IReadSoftSensorData_SS



3.2.4.1 Personalisation System

The “Personalisation System”1 consists of two modules; Identification Module (IM) and

Personalisation Module (PM) which will be interconnected with the ADAS&ME

infrastructure for data collection.

3.2.4.2 Driver/Rider Identification Module (DIM)

The Driver/Rider identification module will identify the current driver. This identification will

be achieved via the driver’s facial image or detected by other sensors. A procedure to enroll a

new driver/rider ID to the database will be also provided by this module. The outward

interface will be unified across all UC: s and will not depend on whether the identification

happens inside (by face identification) or outside (by Wi-Fi/Mobile ID) of the vehicle. DIM

will communicate with other sub-systems through the Broker (RabbitMQ). ID for car and bus

drivers will come from Smart Eye system (SEP) via Ethernet, ID for truck drivers will come

from mobile/tablet device via Wi-Fi, rider ID will come from his/her wearable equipment.

Figure 22 Conceptual diagram of the Driver/Rider Identification Module (DIM).

Table 17 Driver/Rider Identification Module I/O Data Parameters


Inp

ut

FaceID Smart Eye Pro Refer SS I/O Must Have

Rider_ID Wearable Equipment Refer SSI/O Must Have

TruckDriver_ID Mobile/Tablet Refer SS I/O Must Have

Ou

tpu

t

New_Driver/Old_

Driver/No_Driver:

1,2,3

Enumerated Data type

ID Double

1 The word Personalisation is referred as Personalization at some instants of this document. Both words mean same and represent same functionality.



3.2.4.3 Personalization Module

Personalisation Module provides and maintains static and dynamically updated driver/rider

specific parameters to other ADAS&ME system components (Driver state algorithms,

Decision Support Module, Personalised HMI). This may also include calculated statistics

about driver/rider parameters such as “normal heart rate”. PM provides a Remote Procedure

Call (RPC) mechanism that enables other subsystems (Decision Support Module, Algorithms,

Personalised HMI, Sensors, etc.) to persistently store and retrieve specific parameter values.

Additionally, it supports notifying other system components when a parameter value changes

using a Public-Subscribe mechanism. Both functionalities are supported using a RabbitMQ

message broker (Figure 23). The full list of the parameters constituting the driver profile is

currently under discussion.

PM also receives input from DIM to load the appropriate profile information for the identified

driver/rider. For previously enrolled drivers/riders (or new drivers/riders with associated

preloaded information), PM loads the corresponding profile from the profile database when it

gets a driver/rider ID. For new drivers/riders, PM starts building the profile by collecting

information from various system components. Initial values for certain profile parameters

(e.g. driver/rider preferences) may be explicitly entered by the driver/rider through interactive

GUI dialogs (Personalised HMI).

Figure 23 PM interaction with other sub-systems using RabbitMQ broker

Table 18 Personalisation Module I/O Data Parameters


Unit Relevance

Category Parameter name

Inp

uts

Driver ID Identity Number

Driver/Rider

Identification

Module

Double Must Have

Interface

preferences Sounds

Personalized

HMI String Must have




Unit Relevance

Category Parameter name

Themes Personalized


Language Personalized


Demographics Age Driver/Rider

Profile Integer Must have

Gender Driver/Rider

Profile String Must have

Height Driver/Rider

Profile Double Optional

Spoken languages Driver/Rider

Profile

Vector of

Strings Must have

Driving style

Number of hands

on the wheel (most

of the time)

Driver/Rider

Profile

Integer

(1 or 2) Optional

Critical

disabilities

Deafness (yes/no,

1/0)

Driver/Rider

Profile

Integer

(0 or 1) Optional

Colour-blindness

(no/type)

Driver/Rider

Profile String Optional

Physiological

data Average Heart Rate SS I/O Double Optional

Heart rate history

for the last X days SS I/O

Vector of

Doubles Optional

Driving history

Year when got

driving license

(first time)

Driver/Rider

Profile Integer Must have

Kilometers drove

for the last X days SS I/O

Vector of

Doubles Optional

Kilometers drove

for the last year SS I/O Double Optional

Driver state

history

Driver states with

critical (highest)

level detected for

the last month

(state, number of

times)

Driver state

algorithms

Vector

(String,

integer)

Optional

… … … … …

Ou

tpu

ts

(parameter_id,

old_value,

new_value)

Parameter id

refers to age,

gender, etc.

Struct data

type



3.2.4.4 Decision Support Module

The decision support module (DSM) aims to validate the information of driver monitoring

module and the environmental awareness in a way to trigger the appropriate ADAS &ME

interaction strategy. The DSM is an expert system that incorporates concepts derived from

experts of WP4 and WP5 in the field of driver behavior and human-machine interface (HMI)

and uses their knowledge to provide problem-solving to the interaction of driver and

automates vehicles of the system. Experts rely on common sense when they solve decision

problems. Fuzzy logic resembles expert’s decision making with an ability to generate precise

solutions from certain or approximate information. Therefore DSM will be developed as a

Fuzzy Expert System. Its architecture comprises three main components that also identify the

three processing phases.

Fuzzification which refers to the process of transforming the input into fuzzy sets.

In this process, we determine the degree of membership of crisp input in the

appropriate fuzzy sets.

Expert System main component comprises the linguistic logic of HMI interaction

strategy and can be divided into two sub-processes of rules – knowledge base and

inference engine.

o Knowledge base is the phase required to store the entire rule system that is

given by the experts.

o Inference process simulates the experts reasoning process by making fuzzy

inference on the inputs by construction the IF-THEN rules. Within this

process, the rule basis is created and evaluated and combinations of results

are revealed from each rule.

Defuzzification process consists the transposed transformation of fuzzy set

obtained by the inference engine into an actual value – action crisp value.

The DSM has input interface with the precedent modules that are providing the input data

/information. An output interface is also required to communicate with the subsequent

modules of automated functions and HMI elements of the automated vehicle.

DECISION SUPPORT MODULE (DSM)

DSS Expert System

Fuzzificatio

n

Defu

zzification

Linguistic Logic of

HMI Interaction

Strategy

Inference Engine

Knowledge Base

Inputs Outputs

Figure 24 Decision Support Module

Table 19 Decision Support Module I/O Data Parameters


Inp

uts

Vehicle State ESAS Refer ESAS I/O UC specific

Emergency Information ESAS Refer ESAS I/O UC specific




List of fused objects ESAS Refer ESAS I/O UC specific

Environmental situation ESAS Refer ESAS I/O UC specific

Emotion DSMS Enumerated Data

type UC specific

Inattention DSMS Enumerated Data

type UC specific

Physical fatigue DSMS Enumerated Data

type UC specific

Stress DSMS Enumerated Data

type UC specific

Sleepiness DSMS Enumerated Data

type UC specific

Rest DSMS Enumerated Data

type UC specific

Vehicle speed/acceleration SS Real value UC specific

Driver profile PM Refer PM I/O UC specific

Output

Decision:

Normal, Information,

Warning, Intervention

DSM Enumerated Data

type Must Have

HMI Elements selection DSM Enumerated Data

type Must Have

Vehicle Automation

selection DSM

Enumerated Data

type Must Have

3.2.4.5 HMI Controller Module

The HMI Controller Module (HMI CM) is responsible for controlling HMI elements towards

interacting with the driver/rider. Based on the HMI elements of each demonstrator, all

available input devices, either explicit (e.g., touch, gestures, speech, dashboard controls,

hardware buttons) or implicit (gaze/head tracking, physiological parameters, sequential

actions) will be considered for receiving driver/rider input. Similarly, all available output

devices (e.g. touch screens, speakers, vibration motors, dashboard notification components,

AR displays) and their respective modality types (i.e. auditory, haptic and visual) will be

considered for providing output to the driver/rider.

The central component of the module is the Personalized HMI (see

Figure 25), an expert system that will apply rule-based reasoning to produce the

personalization and adaptation decisions for realizing personalized interaction with the

driver/rider according to the strategy decided by the DSM, while taking into account the

driver/rider profile and state, the vehicle state and the environmental situation. It consists of

the two following sub-components:

The Inference Engine, that uses a set of personalization and adaptation rules

provided by domain experts (i.e. driver behaviour and HMI experts) in the form of

IF-THEN rules and evaluates them according to the current input data to produce

HMI activation decisions.

The HMI Activator, that realizes these decisions by activating the corresponding

HMI elements with the appropriate data and personalizing and adapting the visual



appearance of any GUI applications (e.g. increasing font size, changing colour

contrast, simplifying the user interface).

Apart from activating HMI elements, the Personalized HMI will also produce output

regarding the driver/rider response to system decisions and actions (e.g., driver/rider

dismissed a system issued notification). Finally, any preferences set explicitly by the

driver/rider (typically through some GUI HMI element) will be communicated to the

Personalisation Module to be integrated into the driver/rider profile information. To provide

this functionality the Personalized HMI module has input and output interfaces, interacting

with other subsystems and modules as shown in Table 20.

Figure 25 HMI Controller Module

Table 20 HMI Controller Module I/O Data Parameters


Input

Vehicle State ESAS (SDFM) Refer SDFM I/O UC specific


State ESAS (ESDM) Refer ESAS I/O UC specific

Emergency Information ESAS Refer ESAS I/O UC specific

Driver Profile ADAS&ME C (PM) Refer PM I/O Must Have

Driver State DSMS Refer DSMS I/O Must Have

Decision Strategy ADAS&ME C

(DSM) Refer DSM I/O Must Have

Output

Driver/Rider Preferences ADAS&ME C (HMI

CM) Refer PM I/O Must Have

Driver/Rider response

actions

ADAS&ME C (HMI

CM)

Enumerated Data

type Must Have

3.2.5 Vehicle Automation Subsystem (VAS)

The Vehicle Automation Subsystem (VAS) comprises the Automation Functions, which carry

out all necessary functionalities to enable automated driving at different SAE levels of

automation (0-4). These functionalities include driving assistance at a lower level of

automation, as well as fully system controlled steering and acceleration/deceleration at a

higher level of automation. The VAS is a standalone system that adapts the

activation/deactivation of Automation Functions and different interaction strategies owing to

the input of the Decision Support Module (DSM). The VAS may provide a set of available

Automation Functions to the DSM, as well as the current state of automation

(activated/deactivated etc.) and possible upcoming system limits concerning the Automation

Functions.

Personalization and Adaptation Rules

Input Data Inference Engine

Personalized HMI

HMI Elements


Inputs OutputsDecisions

HMI ActivatorGUI applications



Figure 26 Vehicle Automation Subsystem

Table 21 Vehicle Automation Subsystem I/O Data Parameters

Type Data Parameters From/To Measure Unit Relevance

Input Personalized Interaction

Strategy DSM

Enumerated Data

Type Must Have

Output

Available Automation

Functions DSM

Enumerated Data

Type Optional

Distance to System Limit DSM Integer Optional

Automation State DSM Enumerated Data

Type Optional

Automation Functions are a completely new feature for motorcycles and represent a real

challenge because of the complex 3D dynamics of two-wheelers. Therefore the VAS (and the

automation functions it contains) is at an early stage of development and is simplified, if

compared with the one defined for cars and trucks. This is also reflected in the low TRL level

of UC E/F.

In these use cases, the ADAS&ME Core Subsystem decides which automation function has to

be triggered, after having checked that the function activation implies no major safety issues

(e.g. activation in curve).

Figure 27 Vehicle Automation Subsystem for UC E/F

Table 22 Vehicle Automation Subsystem for UC E&F - I/O Data Parameters

Type Data Parameters From/To Measure Unit Relevance Alias

Input Personalized

Interaction Strategy DSM

Enumerated Data

Type Must Have

Output Automation State DSM Enumerated Data

Type Optional



4 ADAS&ME System Structure

4.1 Overview

Previous chapter ADAS&ME System Concept has laid a good foundation on the logical

functionality of the overall system in general and each key module. At this chapter,

ADAS&ME System Structure we build up the use case on the existing foundation by

Describing the requirements for each subsystem. System requirements are specified in terms

of what is the exact functionality needed for the subsystem? How is the required functionality

realized? (Ex: software, hardware).

Defining the functional, physical and communication architecture diagrams for each use case

based on the system requirements

Functionality realized at each ADAS&ME use case can be broadly classified into three

categories:

1. Driver state detection

2. Environment situation assessment

3. Decide upon an HMI/Automation strategy in order to adapt to the current driver state and

environment situation

In ADAS&ME project six different driver states (Sleepiness, Stress, Distraction, Emotion,

Physical Fatigue, and Rest) are detected over seven identified use cases. At previous chapter,

each driver state and physiological indicators to detect the driver state are described in detail.

Different sensors (some are already existing and some are newly designed in the project)

which measure the required physiological indicators for each driver state are chosen for

realizing the respective use case. Software algorithms are designed to receive the

physiological indicators of the driver as input data in real time and process them to detect

current driver state.

Demonstrators used in the ADAS&ME project have several sensors inbuilt in them to monitor

the environment around the vehicle. Software algorithms are designed to receive the physical

indicators of environment measured by these inbuilt sensors as input and process them to

assess the current environmental situation.

The core functionality of the use case lies in taking a decision based on the computed driver

state and environment situation to trigger an appropriate HMI/Automation strategy. Software

algorithms are designed to receive current driver state and environment situation to arrive at a

decision strategy.

Functional architecture diagram shows the logical flow of input and output data for the above-

explained three key functionalities. Physical architecture describes the physical interface

between any two modules of the functional architecture that will be used in realizing the use

case at the demonstrator. During the integration phase of the project, physical architecture

assists in the integration of different subsystems into one final ADAS&ME System.

Communication architecture diagram depicts the different communication layers present in

the ADAS&ME system. From this diagram one can deduce key information’s like one

different types of interfaces required for the end system, two the structure of the actual data

that is being communicated between layers, three the way different layers execute at the end

system (ex: sequential, parallel) and finally it gives a big picture of different transformation

that input data undergoes till it arrives at the final output.



In this project as after interfacing the sensors to processing comment, most of the processing

happens using software algorithms. Hence after the interface layer communication between

different layers happens in the software domain. In some cases, software algorithms are

present at different and PC’s and these PC’s communicate via TCP/IP using Ethernet

connection type.

As this deliverable is an initial version system specification of End ADAS&ME system

Functional, Physical and Communication architectures present in this chapter are the first

versions of the end system. During the course of the project, these diagrams will be modified.

This version of the deliverable underlines the input and output data parameters for each key

module. This information forms the base of the specification of actual data that is

communicated between any two modules, which will be the next major focus of work.

4.2 Use case A: Attentive Long Haul Trucking – Functional System

Structure

4.2.1 System Requirements

4.2.1.1 Driver State Monitoring Subsystem

SR.A1. The driver state monitoring is based upon the following sensors:

o Smart Eye Tracker

o Empatica E4

o Autoliv ZForce Steering Wheel

Measuring the following parameters:

o Gaze direction

o Facial expressions

o Heart rate

o Heart rate variability

o Respiration rate

o Galvanic Skin Response

SR.A2. Sensor outcomes are fed to the Driver State Monitoring subsystem; that is responsible for all

metadata development and analysis.

SR.A3. The system should detect in less than 5 sec driver sleepiness, inattention, stress, rest and

emotions; while the driver is driving.

SR.A4. The system predicts in less than 1 minute driver state (sleepiness, inattention, stress, rest and

emotions); while in automation mode.

SR.A5. Sleepiness, distraction and stress have each 3 criticality levels: ‘uncritical’, ‘critical’, risky’.

SR.A6. Rest has 2 criticality levels: “Rest” or “Fatigued” (on/off)

SR.A7. Emotions considered are 4: “Neutral”, “Positive”, “Fear”, “Anger”

SR.A8. All single detections/ predictions above should have a sensitivity over 90% and a specificity

over 98%.

SR.A9. All single detection/ prediction module outcomes are provided in a vector together with an

estimated confidence level (%).

4.2.1.2 ADAS&ME Core

SR.A10. ADAS&ME Core Personalisation module takes into account key driver characteristics (age,

gender, experience, specific physiological parameters such as key anthropometrics, specific behavioural



parameters, such as average lane position) to improve the above detection/ prediction level of single

algorithms (to reach the SR.A6 level; if not achieved; or above).

SR.A11. ADAS&ME Core DSM detects the course of the problem, discerning from the different states;

with an overall sensitivity over 90% and specificity over 95%.

SR.A12. ADAS&ME Core DSM decides on the overall criticality level and the HMI strategy to be

followed (i.e. of which single state), by prioritising or combining single state detections and considering

the input of Environmental Situation Awareness Subsystem.

SR.A13. ADAS&ME Core DSM outcome to the HMI Subsystem has 4 HMI levels: “Normal”,

“Information”, “Warning”, and “Intervention”.

SR.A14. ADAS&ME Core outcome to the HMI subsystem includes the vector: {priority driver state,

HMI level, driver identification ID}

4.2.1.3 Environmental Situation Awareness Subsystem

SR.A18. It bases its verdict on the existing Scania Interface Module (external module to the project,

“Black Box”).

4.2.1.4 HMI Subsystem

SR.A19. Based upon ADAS&ME Core subsystem input, this subsystem realises the decided strategy,

utilising the following HMI devices: Autoliv Steering Wheel, ICL, Secondary Display, Sound Display,

or LED Light Strip or (for “intervention” strategy) activating “Scania Automation Platform”.

SR.A20. Based upon ADAS&ME Core subsystem driver identification ID, the subsystem personalises

the HMI devices/ elements to the specific driver in terms of: modality (visual, acoustic, haptic), device

used, intensity, and timing.

4.2.1.5 SCANIA Automation Platform Subsystem

SR.A21. The driver will be always informed on automation status and will be informed on his/her own

status in case any intervention is activated.

SR.A22. It has bidirectional link to the ADAS&ME Core subsystem, being activated upon request and

also requesting permission to be enabled/ disabled.

SR.A23. Overall integration will be performed on one SCANIA NGS truck, to be evaluated in the

IDIADA test track (simulating highway, merging to highway and normal environments) and in a real

high strategy section in Barcelona.



4.2.2 Functional Architecture

HMI Subsystem


Scania Automation Platform (SAP)

Scania Environmental Situation Awareness Subsystem (SESAS)

Decisio

n Strategy

ADAS&ME Functional Architecture (Truck)

IReadPublicSensorData_DSMS IReadSensorData_SESAS

IWriteDriverState_DSMS

IWriteSituationState_SESAS

IRead/WriteDecisionStrategy_SAP


IWritePublicSensorData_SS

Driver State Determination Module (DSDM)

Sleepiness Inattention Emotion

Sensor Data Fusion

Environmental State Determination Module

ADAS&ME Core (ADAS&ME C) IReadDriverState_Core IReadSituationState_CoreIReadSensorPublicData_Core


Personalization Module (PM)

Decision Support Module (DSM)

HMI Framework Module (HMI FM)HMI Elements Personalized HMI

Driving strategy and Path Planning

Public Interface Module (PIM)

Sensor for Vehicle or Environment Monitoring (SVEM)

20.11.2017

Emergency

Inform

ation

List/Map

of

fused objects

Vehicle State

Environm

ent

Cond

ition

Stress

Profile Selection

1. Vehicle

Dynam

ic Sen

sors

Sensor for Driver State Monitoring (SDSM)

1. Empatica

E4

2. Smart

Eye Pro

multi

camera

3. Au

toliv R

GB

steering w

heel

Inattention

Sleepiness

Emotio

n

Stress

Rest

Scania Interface Module (SIM)

2. Environ

me

nt M

onitorin

g Sen

sors

IWriteScaniaSensorData_SS

Sensors for driver monitoring

Data exchange between subsystems or modules

Sensors for Enviornment/Vehicle monitoring

Sub System

Emergency data exchange between subsystems or modules

Module

Algorithms

HMI Data

Legend

Automation Mode Selector Automation Mode Safety Check

Positioning

3. CA

N SA

E J1939

Visual Displays Sound Engine

IReadDecisionStrategy_HMI

HMI Controller

Scania Proprietary

Vehicle Controller

HMI Elements

1. H

andheld

devices

2. IP sw

itches

3. steering w

heel

switch

es

4. Hands o

n steering w

heel

detection

Soft Sensor Data (SSD)

1. Perso

nalised data

2. Driver

State inform

ation

3. Driver

Sensors

data from

storage

4. Decisio

n Strategy

Real Time Clock

IReadSoftSensorData_SS

IWriteSoftSensorData_DSMS


Low level data exchange between subsystems or modules

Maps

1. CA

N SA

E J1939

Sensor for VehicleMonitoring (SVM)

IWriteDecisionStrategy_Core

Transitions

Profile Database

Scania Off-board Automation Platform

1. Rou

te Planning

2. Map

s

1. Clu

ster D

isplay

3. Au

dio

5. Am

bient R

GB lights

2.Handled

device

4. RG

B lights on steering w

heel

6. Seat V

ibrators

Figure 28 ADAS&ME Truck (Use case A) Functional Architecture



4.3 Use case A: Attentive Long Haul Trucking – Behavioural System Structure

4.3.1 Physical Architecture

Smart Eye Pro Cameras

Empatica E4

Autoliv RGB Steering Wheel

Steering Wheel Switches

Instrument Panel Switches

Handled Devices

PCEthernet Ethernet

Public Interface Module

Bluetooth

Sensors and Vehicle Dynamic

CAN

CAN

WiFiAP

WiFi 802.11 Ethernet

CAN

EthernetSwitch

Ethernet

Sensor Data

SAE J1939

Vehicle and Environment Monitoring Sensors

Scania Off-Board Automation Platform

4G Modem

4G / LTE Ethernet

Scania Interface Module

Rab

bit

MQ

RTI

DD

S

Environment Situation Awareness Subsystem

Linux Industrial PC

Ethernet

Vehicle, Environment and Routing data

Rab

bit

MQ

Rab

bit

MQ

ADAS & ME Core

RTI

DD

S

Personalization Module

Decision Support Module

Time Synchronization ServiceR

abb

it M

Q

Rab

bit

MQ

Driver State Monitoring Subsystem

Sleepiness

Inattention

Ethernet

EthernetSwitch

Ethernet

Ethernet

Driver State Information

Decision Strategy

Ethernet

Situation Information

Ethernet

Soft Sensor Data

Emotion

Stress

Rest

RTI

DD

S

Linux Industrial PC

RTI

DD

S

Automated Functions

Scania Automation Platform

Ethernet

Decision Strategy

HMI PC

Rab

bit

MQ

Personalized HMI

HMI Subsystem

Visual Displays

Sound Engine

Element Controllers

Cluster Display

Audio System


Ambient RGB lights

Seat Vibrators

DVI / HDMI

Handled Devices

USB

CAN

WiFiAP

Core PC

Ethernet

Linux Real Time PC

[ TBD ]

[ TBD ]

Driver State Monitoring PC

Algorithms / Data

Middleware

Hardware Interface

Components Name

Legend

Network equipment

PC

Algorithms, Modules

HMI elements

Input sources

Scania Proprietary

PCEthernet

Figure 29 ADAS&ME Truck (Use case A) Physical Architecture



4.3.2 Communication Architecture

ADAS & ME Interface layer

Sensor Layer

Scania Proprietary Interface layer

Input Data Format Input Data Format

Protocol layer

Input of Interface layer

Driver Monitoring Vehicle Monitoring Environment Monitoring HMI Elements

SAE J1939 AMQP Proprietary API RTI DDS Proprietary API

USB CANTCP/IP UDP/IP

Bluetooth

Ethernet

3G/LTE

Output of Interface layer

RabbitMQ RTI DDS

Processing layer

Personalization layer

Driver State DetectionEnvironment Situation

AwarenessSensor data

Protocol layer

Input of Interface layerEthernet

CANTCP/IP UDP/IP

Proprietary

Output of Interface layer

RTI DDS Proprietary API

Driver ID Driver State

Decision layerDecision Strategy

HMI Layer Vehicle Automation layerPersonalized HMI HMI Elements Automation Functions

SAE J1939

Communication between layers after Interface layer happen over Ethernet and TCP/IP Figure 30 ADAS&ME Truck (Use case A) Communication Architecture



4.4 Use case B: Electric Vehicle Range Anxiety - Functional System

Structure



SR.B1. The driver state monitoring is based upon the following sensors:

o Smart Eye Tracker

o Array Microphones

o UWB Radar

o Empatica E4;

measuring the following parameters:

o Gaze direction

o Facial expressions

o Voice characteristics

o Voice quality

o Heart rate

o Heart rate variability

o Respiration rate

o Galvanic Skin Response.

SR.B2. Sensor outcomes are fed to the Driver State Monitoring subsystem; that is responsible for all


SR.B3. The system should detect in less than 30 sec driver emotions; while the driver is driving.

SR.B4. Emotions considered are 2: “Neutral”, “Anxious”

SR.B5. All single detections/ predictions above should have a sensitivity over 90% and a specificity

over 98%.

SR.B6. All single detection/ prediction module outcomes are provided in a vector together with an



SR.B7. ADAS&ME Core Personalisation module takes into account key driver characteristics (age,



algorithms (to reach the SR.B5 level; if not achieved; or above).

SR.B8. ADAS&ME Core DSM detects the course of the problem, discerning from the different states;


SR.B9. ADAS&ME Core DSM decides on the overall criticality level and the HMI strategy to be

followed (i.e. of which single state), by prioritising or Emotions state detection with vehicle data and

behavioural indicators (in order to confirm that detected state is linked to EV range) and considering the

input of Environmental Situation Awareness Subsystem.

SR.B10. ADAS&ME Core DSM outcome to the HMI Subsystem has 4 HMI levels: “Normal”, “Range

Anxiety”, “Range Incident”, “Intervention”.

SR.B11. ADAS&ME Core outcome to the HMI subsystem includes the vector: {HMI level, driver

identification ID}


SR.B12. It bases its verdict on the existing Vedecom Interface Module (external module to the project,

“Black Box”).




SR.B19. Based upon ADAS&ME Core subsystem input, this subsystem realises the decided strategy,

utilising the following HMI devices: Central Stack Screen, A-Pilar LED strips and Sound System or

(for “intervention” strategy) activating “Vedecom Automation Platform”.

SR.B20. Based upon ADAS&ME Core subsystem driver identification ID, the subsystem personalises

the HMI devices/ elements to the specific driver in terms of: modality (visual, acoustic), intensity, and

timing.

4.4.1.5 VEDECOM Automation Platform Subsystem

SR.B21. The driver will be always informed on automation status and will be informed on his/her own


SR.B22. It has bidirectional link to the ADAS&ME Core subsystem, being activated upon request and

also requesting permission to be enabled/disabled.

SR.B23. Overall integration will be performed on one VEDECOM Renault Zoe EV, to be further

discussed, for a possible evaluation in the public roads around IDIADA test track (simulating traffic and

normal environments). The main objective is an evaluation in the VEDECOM and Valeo facilities area.




Environmental Situation Awareness Subsystem (ESAS)Driver State Monitoring Subsystem (DSMS)


Decisio

n Stra

tegy

IReadSensorData_DSMSIReadSensorData_ESAS

IWriteDriverState_DSMSIWriteSituationState_ESAS

IRead/WriteDecisionStrategy_VAS



Sensor for Driver State Monitoring (SDSM) ADAS&Me Digital Infrastructure Data (DID)

IWriteSensorData_SS


Emotion

Situation Modelling (SM)

Scenario Assessment

Vehicle Mode ControllerError Handler


Sensor for Vehicle or Environment Monitoring (SVEM) HMI Elements

Context Data

2. T

raffic

1. H

D M

ap

s

3. R

oa

d

Ch

are

cterist

ics

4. W

eath

er

5. H

istoric

Da

ta

6.R

elava

nt

PO

I s

Soft sensor data (SSD)

1.

Perso

na

lised

da

ta

2. D

river State

info

rmatio

n

3.

En

viorm

ent

al situ

ation

in

form

ation

4.

Sen

sord

ata

fro

m d

ata sto

rage

5. D

ecision

Stra

tegy

IWriteSoftSensorData_ESAS

Driving Strategies and Path Planning

Safe Corridor Plan Safe Maneuver

Real Time Clock




Sub System


Module

Algorithms

HMI Data

Low level application output data

Low level application output data exchange between subsystems or modules

Legend

IRead

SoftSen

sorData_SS

State Machine

Em

otio

n

1. C

entra

l Stack

Disp

lay

2. A

ud

io

Wa

rnin

g So

un

ds

3. A

ud

io

Spe

ech

4. A

-Pilla

rs R

GB

LED

Strip

5. C

luster

Disp

lay

3. H

ea

dset

micro

ph

on

e

1. Sm

art

Eye P

ro

mu

lti cam

era

2. A

rray o

f tw

o

sho

tgu

n

micro

ph

on

es

4. U

WB

R

adar

5. E4

E

mp

atica

1. Lid

ar system

IB

EO

3. Lid

ar sy

stem

Velo

dyn

e

4. Fro

nt

Cam

era a

dva

nseE

(x2

)

5. IM

U G

PS

RTK

2. R

adars

6. O

ther

low

level sen

sors

from

the

vehicle

6. R

ear cam

era a

dva

nsee

Map Data

Scenario Classification

Execute Maenuver

Veh

icle State

Co

mm

un

ication

to D

river

ADAS&ME Core (ADAS&ME Core) IReadDriverState_Core IReadSituationState_CoreIReadSensorData_Core



Fuzzy Rules





Dri

ver

Pre

fere

nce

s


En

viron

men

t C

on

ditio

n

Veh

icle State

Driver p

rofile

Oth

er

Per

son

aliz

ed

HM

I dat

a


Decisio

n

Strateg

y

Em

ergen

cy Info

rma

tion



Figure 31 ADAS&ME Electric Car (Use Case B) Functional Architecture



4.5 Use case B: Electric Vehicle Range Anxiety - Behavioural System Structure


Automation PC

AdvanSee MipSee (x3)

GNSS reciever

Correvit

IBEO HAD Fusion Box

Ethernet

CAN

Atlans C

ARS308

Velodyne VLP16

IBEO Lux (x5)

Ethernet

Ethernet

MicroAutoBox (Low level control)

CAN

Environment Perception Module

Automation Functions

Trajectory Planner

Smart Eye Pro Camera

<<TBD>>

USB

ADAS&Me PC

A-pillar RGB strips

Central Stack Touchscreen

Audio System

<<TBD>>

WiFi RouterWiFi

Ethernet switch

Ethernet Ethernet

Ethernet switch

Ethernet Ethernet

VALEOSteering Wheel

Microphones withAudio Interface

Empatica E4

PCEthernet


MicroPC

Ethernet switch

Ethernet

WiFi

ADAS & ME Core



HMI Module

Bluetooth

WiFi

<<TBD>>

NTP Time Synchronisation

Service


Emotions

Ethernet

Ethernet

<<TBD>>

<<TBD>>

Ethernet

Algorithms / Data

Middleware

Hardware Interface

Components Name

Legend

Network equipment

PC

Algorithms, Modules

HMI elements

Input sources

VEDECOM Proprietary

Figure 32 ADAS&ME Electric Car (Use Case B) Physical Architecture




ADAS&ME Interface layer

Protocol layer

Ethernet

CAN

TCP/IP UDP/IP

USB Bluetooth

Input Data Format

REST Proprietary APINMEA 2000/0183 MQTT

802.11p 3G/LTE

Audio

WAVE ITS-G5


process input data and append context info

Context Data

Output of Interface layerRabbitMQ

Processing layer


Environment Situation awareness

Sensor data

Driver Profile Driver State Environment Situation

Sensor layer

Driver monitoring Vehicle monitoring Environment monitoring HMI Elements

Decision layer

Vehicle Automation layer

HMI layer

Decision Strategy

Personalized HMI


YoGoKo websocket

Driver State detection

Figure 33 ADAS&ME Electric Car (Use Case B) Communication Architecture



4.6 Use case C: Driver State based smooth & safe automation transitions-

Functional System Structure



SR.C1. The driver state monitoring is based upon the following sensors: o Microphone

o Driver seat

o Cameras

o Steering wheel

Measuring the following parameters: o ECG

o Head position and rotation

o Eyelid blinking

o Heart rate

o Pitch of voice

o Intensity of voice

o Spectral information of voice

SR.C2. Sensor outcomes are fed to the Driver State Monitoring subsystem; that is responsible for all


SR.C3. The system should detect in less than 5 sec driver sleepiness, distraction, stress, and emotions;

while the driver is driving.

SR.C4. The system predicts in less than 1 minute driver state (sleepiness, distraction, stress and

emotions); while in automation mode.

SR.C5. Sleepiness, distraction and stress have each 3 states: ‘uncritical’, ‘critical’, risky’.

SR.C6. Emotions considered are 4: “Neutral”, “Positive”, “Fear”, “Anger”

SR.C7. All single detections/ predictions above should have a sensitivity over 90% and a specificity

over 98%.

SR.C8. All single detection/ prediction module outcomes are provided in a vector together with an



SR.C9. ADAS&ME Core Personalisation module takes into account key driver characteristics (age,



algorithms (to reach the SR.C7 level; if not achieved; or above).

SR.C10. ADAS&ME Core DSM detects the course of the problem, discerning from the different states;


SR.C11. ADAS&ME Core DSM decides on the overall criticality level and the HMI strategy to be



SR.C12. ADAS&ME Core DSM outcome to the DLR Interaction Controller is an Enumeration value

which is input for the HMI Controller module inside the DLR Framework.


SR.C13. It provides ADAS&ME Core with 3 traffic environmental risk levels: “Normal”, “critical”,

“Dangerous”.



SR.C14. It accompanies each level with confidence level (%) estimation.

SR.C15. It performs its function for a test track environment.

SR.C16. It bases its verdict on the existing DLR Sensor Fusion output, the communication module with

its sub-modules and the Naventik Sensor Fusion Module.

SR.C17. Digital Infrastructure provides environmental data components (e.g. traffic, weather, POI’s) to

ADAS&ME Core.

SR.C18. Digital Infrastructure back-end derives a driving task intensity prediction for a planned route.

SR.C19. Relevant vehicles have to be connected and support the messages proposed in deliverable D3.1

SR.C20. V2X Data Integration sub-module combines detections from on-board sensors (DLR Sensor

Fusion) with information from the Communication Module and Sensor Data Fusion Module

SR.C21. Communication Module provides ADAS&ME Core with a list of detected vehicles (location

and confidence information).

SR.C22. The DLR Automated Functions use the communication module to exchange messages

between vehicles for coordinated manoeuvres.

SR.C23. The Sensor Data Fusion Module is based upon the following sensor inputs: GNSS raw data

(mandatory) and vehicle speed, yaw rate, wheel speeds, longitudinal & lateral acceleration (optional

extensions). These signals must be cyclic with a maximum cycle time of 10ms.

SR.C24. Each measurement of vehicle speed, yaw rate, wheel speeds, longitudinal & lateral

acceleration contains a time stamp indicating the time of measurement/validity. For better state

estimation preferably accurate at least to 0.1ms.

SR.C25. The Sensor Data Fusion Module will estimate the vehicle state (position, heading and speed)

together with the corresponding standard deviations with a cycle time of 100ms.


SR.C25. DLR uses the DLR Interaction Controller to perform the tasks of the HMI Subsystem.

SR.C26. Based upon ADAS&ME Core subsystem input, the DLR Interaction Controller realises the

decided strategy, utilising the following HMI devices: Speaker, Ambient light Display, Handheld

device, cluster display or (for “intervention” strategy) activating DLR Automation functions.

SR.C27. Based upon ADAS&ME Core subsystem driver identification ID, included in the personalized

interaction strategy form DSM, the subsystem personalises the HMI devices/ elements to the specific

driver in terms of: modality (visual, acoustic, haptic), device used, intensity, and timing.

4.6.1.5 DLR Automation Functions

SR.C28. The driver will be always informed on automation status and will be informed on his/her own


SR.C29. The automation functions will be activated and deactivated by the DLR Interaction Controller,

based on information from the ADAS&ME Core subsystem.

SR.C30. Overall integration will be performed on the DLR research vehicle FASCar II, to be evaluated

and demonstrated on the IDIADA test track.







Decisio

n Strategy

Veh

icle State

IReadSensorData_DSMS IReadSensorData_ESAS

IWriteDriverState_DSMS

IWriteSituationState_ESAS

IRead/WriteDecisionStrategy_VAS



Sensor for Driver State Monitoring (SDSM) ADAS&Me Digital Infrastructure Data (DID)

IWriteSensorData_SS


Sleepiness Inattention Emotion Sensor Data Fusion Module (SDFM)



ADAS&ME Core (ADAS&ME C) IReadDriverState_Core IReadSituationState_CoreIReadSensorData_Core



Fuzzy Rules





Sensor for Vehicle or Environment Monitoring (SVEM) HMI Elements

6. H

eadset m

icrop

ho

ne

1. cECG

3. PP

G

inside seat

4. Sm

art Eye Pro

m

ulti cam

era

5. A

rray of tw

o sh

otgun

micro

ph

on

es

2. M

IM

7. Z-force stee

ring w

heel

1. V

isual D

isplays

2. A

udio

Warn

ing So

un

ds

3. A

udio

Spee

ch

4. R

GB Strip

on stee

ring

wh

eel

5.

Entertainmen

t System

6.

Ha

ndheld

dev

ices

7. C

luster

Display

Context Data

2. Traffic

1. H

D M

aps

3. R

oad C

harecteristics

4. W

ea

the

r

5. H

istoric

Data

6.R

elavant P

OI s

Stress


1.

Personalise

d d

ata

2. D

river State

info

rmatio

n

3.

Enviorm

ent

al situatio

n in

form

ation

4.

Sensordata

from

data

storage

5. D

ecisio

n

Strate

gy

IWriteSoftSensorData_ESAS


Veh

icle con

text

Communication with Digital Infrastructure Module

Digital Infrastructure reporting V2X Data Sharing

Real Time Clock




Sub System


Module

Algorithms

HMI Data



Legend

IRead

SoftSen

sorData_SS


Inattentio

n

Sleepiness

Emotio

n

Stress

1. lbeo laser

scan

ners

3. M

obile

eye camera

4. N

OV

ATEL

SPA

N-C

PT

5. V

2X

dow

n

channel

2. SM

S R

adars

7. O

ther low

level sen

sors fro

m the

vehicle

8.

NA

VEN

TIK End

urance

6. V

2X U

p

channel



Dri

ver

Pref

eren

ces


Environm

ent

Cond

ition

Veh

icle Sta

te

Driver p

rofile

Oth

er

Pers

ona

lized

H

MI d

ata


Decisio

n Strateg

y

Emergen

cy In

form

atio

n

Emergen

cy Info

rmation

List of fused

ob

jects

Veh

icle State

Environm

ent

Situatio

n


Figure 34 ADAS&ME Normal Car (Use case C and D) Functional Architecture



4.7 Use case C: Driver State based smooth & safe automation transitions- Behavioural System Structure


Smart Eye Pro Camera

Driver Seat

AutolivSteering Wheel

Mikrophones withAudio Interface

PCEthernet

Sensors andVehicle Dynamics

CAN/Ethernet

EthernetSwitch

Ethernet

Sensor Data

DLR Interface Module

Rab

bit

MQ

DrivePXVehicle and environment Data

ADAS & ME Core



Rab

bit

MQ


Sleepiness

Distraction

EthernetSwitch

Ethernet

Ethernet

Situation Information

Ethernet

Soft Sensor Data

Emotion

Stress

Linux PC


DLR Automation Functions

Ethernet

Personalized Interaction Strategy

HMI PC

Rab

bit

MQ

Personalized HMI

DLR Interaction Controller

Visual Displays

Sound Engine

Element Controllers

Cluster Display

Audio System


Ambient RGB lights

DVI / HDMI

Handled Devices

USB

PC

WiFi 802.11PC

[ TBD ]

Algorithms / Data

Middleware

Hardware Interface

Components Name

Legend

Network equipment

PC

Algorithms, Modules

HMI elements

Input sources

PC[TBD]

[TBD]

PCUSB

DLR Proprietary

DENSO WSU Ethernet

NAVENTIK Endurance

Ethernet

PC

Handled DevicesWiFi and UMTS AP

WiFi 802.11

Eth

ern

et

NTP Time Synchronisation

Service

USB

EthernetSwitch

Ethernet

Eth

ern

et

Eth

ern

et

USB

Ethernet


Environment Situation Awareness Subsystem

Ethernet

WiFi and UMTS AP

Ethernet

Rab

bit

MQ

PC

Figure 35 ADAS&ME Normal Car (Use case C and D) Physical Architecture





Protocol layer

Ethernet

CAN

TCP/IP UDP/IP

USB Bluetooth

Input Data Format

REST Proprietary APINMEA 2000/0183 MQTT

802.11p 3G/LTE

Audio

WAVE ITS-G5



Context Data


Processing layer



Sensor data


Sensor layer

Driver monitoring Vehicle monitoring Environment monitoring HMI Elements

Decision layer


HMI layer

Decision Strategy

Personalized HMI



Figure 36 ADAS&ME Normal Car (Use case C and D) Communication Architecture



4.8 Use case D: Non-reacting driver emergency maneuver

System Requirements, Functional Architecture, Physical Architecture,

Communication Architecture specified for UC C hold true for UC D as

well.



4.9 Use case E: Long range attentive touring with motorbike – Functional

System Structure


4.9.1.1 Driver (Rider) State Monitoring Subsystem

SR.E1. The Rider State Monitoring is based upon the following sensors:

o ECG (Electrocardiogram)

o PPG (Photoplethysmogram)

o Temperature Sensors

o Respiratory Sensor

o EDA/GSR (Electro dermal activity or Galvanic Skin Response) Sensor

o Inertial Measurement Units

o FSR (Force Sensitive Resistor)

o UV sensor

o Humidity Sensors

o GPS

o Altimeter

o Barometer

o Vehicle dynamics sensors (ABS, BBS)


o Heart Rate (HR)

o Heart Rate Variability (HRV)

o Pulse Arrival Time (PAT) for blood pressure (BP) estimation

o Respiratory Rate

o Skin temperature in different body regions (head, torso, hands)

o Skin conductance

o Relative head position (with regard to torso position)

o Vibrations induced by the vehicle on the hands

o Pressure on handlebar and footrests

o UV index

o Air relative humidity

o Air temperature

o Atmospheric pressure

o GPS coordinates (including altimetry)

o Vehicle dynamics (speed, acceleration, lean angle, accelerator and throttle position, brakes

use)

SR.E2. The system should be able to derive the following parameters through indirect

measurements: core temperature, blood pressure, head tracking.

SR.E3. Rider State Monitoring Subsystem should take into account also the broad weather

estimation made by the Environmental Situation Awareness Subsystem, for the physical fatigue

detection.

SR.E4 The system should detect in less than 5 sec rider’s distraction and in less than 5 min rider’s

physical fatigue, and stress; while the rider is riding.

SR.E5. Physical fatigue and stress have each 3 criticality levels: ‘uncritical’, ‘critical’, risky’.

SR.E6. Distraction has 2 states: ‘not detected’, ‘inattentive’

SR.E7. Inattention detection should have a sensitivity over 80% and a specificity over 90%.



SR.E8. Stress and Physical fatigue detections should have a sensitivity between 60-70% and a

specificity between 75-85%, since the algorithms developed are fully innovative and start from

TRL 1 reaching max TRL 5.

SR.E9. All single detection module outcomes are provided in a structure together with an estimated

confidence level (%).

SR.E10. The input interface of the Driver (Rider) State Monitoring Subsystem should communicate

with the output interface of Sensors Subsystem, ADAS&ME Core and Environmental Situation

Awareness Subsystem.

SR.E11. The output interface of the Driver (Rider) State Monitoring Subsystem should

communicate with the input interface of ADAS&ME Core.

SR.E12. Driver (Rider) State Monitoring Subsystem is responsible for all metadata development

and analysis, unless data pre-processing is necessary to be performed by the Wearable Platform

Control Unit (WPCU); due to communication constraints between the vehicle and the wearable

platform.


SR.E13. ADAS&ME Core stores and handles riders' profile.

SR.E14. ADAS&ME Core Personalisation Module (PM) takes into account key rider

characteristics (age, gender, riding history such as kms per day, outside temperature/ humidity, time

of day, specific physiological parameters, etc.) to improve the above detection levels of single

algorithms (to reach the SR.E5 and SR.E6 levels, if not achieved, or above).

SR.E15. ADAS&ME Core PM should be able to create a new profile, based on rider ID

(New/Old/None), to load an existing rider profile and to broadcast an event to other subsystems

when receive a driver/rider ID (New/Old/None). The profile will be stored inside the WPCU and

communicated to the Vehicle Control Unit (VCU) once a wireless communication is established.

SR.E16. ADAS&ME Core Decision Support Module (DSM) detects the course of the problem,

discerning from the different states; with an overall sensitivity between 70-80% and specificity

between 80-90%.

SR.E17. ADAS&ME Core DSM decides on the overall criticality level and the HMI and/or

automation strategy to be followed (i.e. of which single state), by prioritising or combining single

state detections and considering the input of Environmental Situation Awareness Subsystem.

SR.E18. ADAS&ME Core DSM output to the HMI Subsystem has 4 HMI levels: “Normal”,

“Information”, “Warning”, “Intervention”.

SR.E19. ADAS&ME Core DSM output to the HMI Subsystem includes the vector: {priority driver

state, HMI level, rider identification ID}.

SR.E20. ADAS&ME Core DSM should have an interface, to send/receive data from/to the Driver

(Rider) State Monitoring Subsystem, Environmental Situation Awareness Subsystem, Sensor

Subsystem, HMI Framework Module, Personalization Module and Vehicle Automation Subsystem.

SR.E21. ADAS&ME Core DSM should be able to process the data coming from the Driver (Rider)

State Monitoring Subsystem, Environmental Situation Awareness Subsystem, Sensor Subsystem,

HMI Framework Module, Personalization Module and Vehicle Automation Subsystem.

SR.E22. ADAS&ME Core DSM should have a sub-module for Pre Automation Activation Check.

SR.E23.The input interface of the ADAS&ME Core should communicate with the Output interface

of Sensors Subsystem, Driver (Rider) State Monitoring Subsystem and Environmental Situation

Awareness Subsystem.

SR.E24.The output interface of the ADAS&ME Core should communicate with the input interface

of Vehicle Automation Subsystem, HMI subsystem and Driver (Rider) State Monitoring

Subsystem.




SR.E25. The Environmental Situation Awareness Subsystem (ESAS) is based upon the following

measurements:

o Dynamic map

o Weather information

o Traffic information

It is underlined that the ESAS in the motorcycle platform does not include vehicle-based

environmental sensing.

SR.E26. ADAS&ME ESAS should be able to build a broad representation of weather and traffic, in

a preselected route, and give a synthetic estimation of the current risk. The relevant estimation is

obtained, only for the zone where the vehicle is currently moving, via Sensors Subsystem inputs

like Air Temperature, Air Relative humidity, Atmospheric Pressure and UV index.

SR.E27. ADAS&ME ESAS should continuously update its estimation.

SR.E28. The input interface of the Environmental Situation Awareness Subsystem should

communicate with the output interface of Sensor Subsystem.

SR.E29. The output interface of the Environmental Situation Awareness Subsystem should

communicate with the input interface of Driver (Rider) State Monitoring Subsystem and

ADAS&ME Core.


SR.E30. Based upon ADAS&ME Core Subsystem input, this Subsystem realises the decided

strategy, utilizing the following HMI devices:

o Info Helmet (wireless device with LEDs, integrated at helmet) visual feedback to the

rider

o Headphones acoustic feedback to the rider

o Helmet vibramotors haptic feedback to the rider

o Gloves vibramotors haptic feedback to the rider

o Dashboard visual feedback for the rider

o Navigation unit visual feedback for the rider

o Helmet Rear LED strip visual feedback for surrounding traffic

o (Vehicle) Hazard lights visual feedback for surrounding traffic

SR.E31. Based upon ADAS&ME Core subsystem and Personalisation Module, the HMI

Subsystem personalises the HMI devices/ elements to the specific rider in terms of: modality

(visual, acoustic, haptic), device used, intensity, and timing.

SR.E32. The input and output interfaces of the HMI Subsystem should communicate with the input

and output interfaces of ADAS&ME Core.

4.9.1.5 Vehicle Automation Subsystem

SR.E33. The driver will be always informed on automation status and will be informed on his/her

own status in case any intervention is activated.

SR.E34. The automation functions will be activated and deactivated by the Vehicle Automation

Subsystem (VAS), based on information from the ADAS&ME Core subsystem. It is underlined

that the automation functions capability and maturity are limited, due to the low TRL level of the

use case.

SR.E35. Overall integration will be performed on Ducati motorcycle, to be evaluated in the

IDIADA test facilities.

SR.E36. For the case that the rider is detected in a state that requires an “intervention” strategy, the

ADAS&ME Core subsystem activates the “Motorcycle Recovery Mode” function.



SR.E37. Once the emergency automation function is activated it can only be deactivated by

stopping and re-starting the vehicle. Longer constrained resting periods are still under evaluation.




Figure 37 ADAS&ME Motorbike (UC E and F) Functional Architecture



4.10 Use case E: Long range attentive touring with motorbike – Behavioural System Structure


Vehicle

Vehicle

Vehicle

Back Protector

dSpace – Vehicle Control Unit

Helmet

TMP36 – Temperature Sensor (4x)

Sensor Data

Teensy 3.6 – Wearables Platform Control Unit

(WPCU)

I2C

BT Module #1

XBEE - RF Module #1

RF

Rider State Monitoring Subsystem

Physical Fatigue

Inattention

Stress

Pre-Processed Sensor Data

Pre-Processed Sensor Data

Sensors Data Pre-Processing (TBC)

CAN channel #2

CAN channel #1

Environmental Situation Awareness Subsystem

Weather Estimation

ADAS & ME Core



Personalized HMI

HMI Elements

HMI Subsystem

Recovery Mode

Capsize Control

Vehicle Automation SubsystemDecision Strategy

Rider State InfoPersonalisation Info

CAN channel #1

Analogic Channel

Engine Management System

Engine Control

Capsize Control Hardware

Dashboard

Hazard Lights

TomTom Bridge – Satellite Navigator

CAN channel #1

BT Module #1

BT 4.0

Right & Left Gloves

Vibramotors

Helmet

Vibramotors

Rear LED Strip

Info-Helmet

Headphones

BT Module #2

TBD

Back Protector

Teensy 3.6 – Wearables Control Unit

Wearable HMI control

XBEE – RF Module #2

XBEE – RF Module #1

RF I2C

I2C

I2C

I2C

CAN channel #2

Sensor Data

Serial

TMP36 – Temperature Sensor (4x)


PersonalData

XBEE - RF Module #2

Teensy 3.6 – Data

Converter

Data Conversion (To CAN)

BT 4.0

Teensy 3.6 – Data

Converter

Data Conversion (From CAN)

Sim

ulin

kSi

mu

link

Simu

link

Simulink Simulink

Simulink

Simulink

Back Protector

FGPMMOPA6H - GPS Unit

Alt-IMU-10V5 – Barometer, Magnetometer, Acceletometer, IMU and Temperature Sensor

Underwear

PZT - Respiration Sensor

HDC1080 – Temperature and Relative Humidity Sensor

ADS1015 – A/D converter


SENS-ECG-UCE6 – 3 el. ECG – aggiungere

Right Glove

SENS-EDA-UCE6 - EDA SensorADS1015 – A/D

converter

LSM6DS33 – Accelerometer and 6 axis IMU

VEML6075 – UV Sensor

HDC1080 – Temperature and Relative Humidity Sensors




Vehicle

Vehicle Dynamics Sensors

TomTom Bridge – Satellite Navigator

Left Glove

MAX30102 – PPG Sensor

VEML6075 – UV Sensor


HDC1080 – Temperature and Relative Humidity Sensors

FSR Sensor

TBD

TBD TBD

Figure 38 ADAS&ME Motorbike (UC E and F) Physical Architecture




ADAS&ME Interface Layer

Input of Interface Layer 2CAN

Bluetooth

Input Data Format(CAN) Standard Frame Format

Process input data and append context info Context Data

Output of Interface Layer Simulink

Algorithm Processing Layer

Personalization Layer

Decision Layer

Vehicle Automation LayerHMI Layer

Sensor Layer

Binary Format

Radio Frequency

Input of Interface Layer 1

Sensor Pre-Processing Layer (if needed) NMEA

Generic Format (multiple choices

available)

Sensor Data Rider State Detection

Rider ID Rider State

Decision Strategy

Personalised HMI Automated Function

Rider Monitoring Vehicle Monitoring Environment Monitoring

Data Pre-Processing

I2C

Figure 39 ADAS&ME Motorbike (UC E and F) Communication Architecture



4.11 Use case F: Raider Faint


NOTE: UC E Requirements apply to UC F unless it is specified here below

4.11.1.1 Driver (Rider) State Monitoring Subsystem

SR.F1.The system should detect faint or predict high probability to faint in less than 5 sec; while

the rider is riding (it substitutes SR.E3).

SR.F2. The system should prevent risk of fainting within the next couple hours; while the rider is

riding (it substitutes SR.E4).

SR.F3. Faint has 4 states: “Not Detected”, “Predicted Risky”, “Predicted Imminent”, “Detected” (it

substitutes SR.E5).

SR.F4. Faint prediction/ detection should have a sensitivity over 75% and a specificity over 85% (it

substitutes SR.E8).

SR.F5. Faint prevention should have a sensitivity over 70% and a specificity over 90% (it

substitutes SR.E8).




4.11.1.5 Vehicle Automation Subsystem

SR.F6. For the case that the rider is detected in a state that requires an “intervention” strategy, the

ADAS&ME Core subsystem activates the “Capsize Control” function (it substitutes SR.E37).

Functional Architecture, Physical Architecture, Communication Architecture specified

for UC E hold true for UC F as well.



4.12 Use case G: Passenger pick up/drop off automation for buses-Functional

System Structure



SR.G1. The driver state monitoring is based upon the following sensors:

o SmartEye Pro

o Empatica E4

o Autoliv Steering Wheel


o Gaze direction

o Head rotations

o ECG

o PPG

o EDA

o Driver movements (accelerometer)

SR.G2. Sensor outcomes are fed to the Driver State Monitoring subsystem; that is responsible for all


SR.G3. The system should detect driver inattention in handover situations in less than 2 sec, driver

stress and sleepiness in less than 5 minutes while the driver is driving.

SR.G4. The system predicts driver stress and sleepiness in less than 5 minute (distraction is not

relevant) while in automation mode.

SR.G5. Sleepiness, distraction and stress have each 3 states: ‘uncritical’, ‘critical’, risky’.

SR.G6. Risky sleepiness detections/ predictions should have an accuracy above 85% (above 75% for

uncritical and risky). Distraction detection in handover situations should have a sensitivity above 95 %

and a specificity above 75 %. Stress detection/prediction should have a sensitivity above 80 % and a

specificity above 75 %.

SR.G7. All single detection/ prediction module outcomes are provided in a vector together with an



SR.G8. ADAS&ME Core Personalisation module takes into account key driver characteristics (age,


parameters, such as average lane position and time driven) to improve the above detection/ prediction

level of single algorithms (to reach the SR.A6 level; if not achieved; or above).

SR.G9. ADAS&ME Core DSM detects the course of the problem, discerning from the different states.

Driver distraction in handover situations have the same requirements as in SR A6. Driver sleepiness and

stress should have an overall sensitivity over 80% and specificity over 75%.

SR.G10. ADAS&ME Core DSM decides on the overall criticality level and the HMI strategy to be



SR.G11. ADAS&ME Core DSM outcome to the HMI Subsystem has 4 HMI levels: “Normal”,

“Information”, “Warning”, and “Intervention”.

SR.G12. ADAS&ME Core outcome to the HMI subsystem includes the vector: {priority driver state,

HMI level, driver identification ID}




SR.G13. It provides ADAS&ME Core with 3 traffic environmental risk levels: “Low”, “Medium”, and

“High”.

SR.G14. It accompanies each level with confidence level (75%) estimation.

SR.G15. It performs its function for real (in this case simulated) traffic environments (urban).

SR.G16. It bases its verdict on the Environmental Situation Awareness Subsystem.


SR.G17. Based upon ADAS&ME Core subsystem input, this subsystem realises the decided strategy,

utilising the following HMI devices: Cluster display in the bus, Audio system, Autoliv RGB steering

wheel, ambient RBG lights and seat vibrators or (for “intervention” strategy) activating the Vehicle

Automation Module in the bus driving simulator.

SR.G18. Based upon ADAS&ME Core subsystem driver identification ID, the subsystem personalises

the HMI devices/ elements to the specific driver in terms of: modality (visual, acoustic, haptic),

confirmation, device used, intensity, and timing.

4.12.1.5 Bus simulator Automation Platform Subsystem

SR.G19. The driver will be always informed on automation status and will be informed on his/her own


SR.G20. It has bidirectional link to the ADAS&ME Core subsystem, being activated upon request and

also requesting permission to be enabled/ disabled.

SR.G21. Overall integration will be performed and evaluated in VTI: s bus driving simulator in a

simulated urban environment.




Environmental Situation Awareness Subsystem (Simulation)

Simulation of real world situation information


Vehicle Automation

Decisio

n Strategy

IReadSensorData IReadSensorData

IWriteDriverState

IWriteSituationState

IRead/WriteDecisionStrategy



Sensor for Driver State Monitoring (SDSM)

IWriteSensorData

Determination of Driver State Module

Sleepiness Inattention

ADAS&ME Core IReadDriverState IReadSituationStateIReadSensorData

IRead/WriteDecisionStrategy

Personalization ModuleSensor data Environment Situation

Decision Support ModuleFuzzy Rules

HMI Framework ModuleHMI Elements Personalized HMI


ADAS&ME Interface Module

Sensor for Vehicle Monitoring (Simulated data) HMI Elements

Context Data

Em

erg

en

cy Info

rmatio

n

List of fu

sed

ob

jects

Veh

icle Sta

te

Enviro

nm

ent

Co

nd

ition

Stress


1.

Perso

nalise

d d

ata

2. D

river State

informatio

n

3.

Envio

rmen

tal situ

ation

in

form

ation

4.

Senso

rdata

from

data

storage

5. D

ecision

Stra

teg

y



Real Time Clock




Sub System


Module

Algorithms

HMI Data



Legend

IRead

SoftSen

sorData

Inattentio

n

Sleep

iness

Stress

Driver Profile

1. V

ehicle

senso

r data

1. Sm

art Eye P

ro

mu

lti cam

era

2. Em

patica

E4 w

atch

Driver State

Situational Demand Estimate (Simulated Data)

2. T

raffic

1. H

D M

aps

3. R

oad

C

harecterist

ics

4. W

eath

er

5. H

istoric

Data

6.R

elavant

PO

I s

1. Scree

n in

th

e Clu

ster

2. A

uto

liv Steerin

g W

hee

l

3. Sp

eakers

3. Stee

ring

Wh

eel

Senso

r Zfo

rce

4. LED

Ligh

ts

Figure 40 ADAS&ME Bus Simulator (Use Case G) Functional Architecture



4.13 Use case G: Passenger pick up/drop off automation for buses - Behavioural System Structure


Smart Eye Pro Cameras

Emaptica E4


PCEthernet Ethernet


Bluetooth

EthernetSwitch

Ethernet

Sensor Data

Rab

bit

MQ

Environment Situation (Simulated)

Ethernet

ADAS & ME Core



Rab

bit

MQ

Rab

bit

MQ


Sleepiness

EthernetSwitch

Ethernet

Ethernet

Soft Sensor Data

Stress

Inattention

Ethernet

Decision Strategy

HMI PC

Rab

bit

MQ

Personalized HMI

HMI Subsystem

Visual Displays

Sound Engine

Element Controllers

Cluster Display

Audio System


Ambient RGB lights

Seat Vibrators

DVI / HDMI

USB

CANCore PCSimarch2

USB

USB

Algorithms / Data

Middleware

Hardware Interface

Components Name

Legend

Network equipment

PC

Algorithms, Modules

HMI elements

Input sources

Sensor Data

PCEthernet

Figure 41 ADAS&ME Bus Simulator (Use Case G) Physical Architecture





Protocol layer

Ethernet

CAN

TCP/IP UDP/IP

USB Bluetooth

Input Data Format

Proprietary API MQTT

802.11p Input of Interface layer


Context Data


Processing layer



Sensor data


Sensor layer

Driver monitoring Environment monitoring HMI Elements

Decision layer


HMI layer

Decision Strategy

Personalized HMI



Figure 42 ADAS&ME Bus Simulator (Use Case G) Communication Architecture



5 System Specifications

5.1 System Specifications

Technical specifications are the quantitative translation of the requirements above defined and

are in compliance with the functional and communication architecture. They represent a

valuable tool for other activities within the ADAS&ME project, such as risk analysis,

algorithms development for driver/rider and environment states recognition, integration of the

various systems in the demonstrators and technical verifications of the same.

Here and in Annex 1, information about the different modules is gathered as they are at the

moment: since several activities have just started or still have to start, these specifications will

be updated and integrated in the next deliverable/document, expected for Month 30.

This section and Annex 1 have the following structure:

template illustration

sensors specifications

HMI specifications

interfaces communication

processing unit communication

For the sake of simplicity, not all the items are specified with the same level of detail. As a

matter of fact, some of them are already integrated in the demonstrators, so there is no need to

consider aspects such as weight, power supply, etc.

5.1.1 Specifications template

The template is in a tabular form and is divided into 4 sections:

Introduction: including information about the manufacturer, the responsible partner,

processing units and communication (I/O ports and communication network)

Performance section: giving details about the item performances (e.g. for a sensors:

physical quantity measured, along with accuracy, frequency,)

Physical specifications: including information about integration aspects (weight, size,

operating temperature range, power supply, …)

Environmental Specifications: in case the item has or should have specific

characteristics in terms of water resistance, vibration resistance,

Module Name

Module ID

Manufacturer

Responsible Partner

General Description

Micro

Memory

Battery

Supported OS

I/O Ports

Communication network details

Module Performances Definition Value Unit

Resolution



Refresh rate

…

Physical Module Specifications

Definition Value Unit

Dimensions mm

Weight g

Operating Temperature Range °C

Temperature Gradient °C/min

Max humidity @40 °C % r.h.

Supply Voltage V

Peak Voltage V

Supply Current A

Battery (Type /Ah) Ah

Environmental Specifications

Functional Safety

Environment

Vibration, Shock, Bump

EMI/EMC

Load Dump

Note

5.1.2 Sensors

Sensors

ID Name Type Responsible

Partner

Used in

UC

S1 Smart Eye Pro

Multicamera System

Multi camera system for eye-

tracking. Number of cameras is UC

dependent (3-5 for truck, 2 for car

and bus).

Smart Eye A,B,C,D,G

S2 Empatica E4 wristband

Wristband that measures skin

conductance (EDA), heart rate

(PPG), temperature, movements

(acc)

Scania, Valeo A,B,G

S3 zForce SW z-Force Steering Wheel Autoliv A,C,D,G

S4 UWB Radar Sensor Ultra-Wide Band Radar Valeo /

NUIG B

S5 Zephyr Bioharness 3

Chest strap providing ECG,

Breathing, Posture and Temperature

monitoring

Valeo B

S6 ibeo LUX and ibeo LUX

Fusion System LIDAR and fusion system Vedecom B

S7 ARS 408-21 Radar 77 GHz Radar Vedecom B



Sensors


Partner

Used in

UC

S8 Velodyne Lidar Puck

VLP-16 Roof-mounted LIDAR Vedecom B

S9 Shure VP82 Shotgun Microphone OVGU B,C,D

S10 Sennheiser HSP 4 EW-3

beige Headset/Neckband Microphone OVGU B,C,D

S11 cECG

Seat-integrated capacitive ECG,

Magnetic Induction and Photo

plethysmography module

RWTH C,D

S12 FGPMMOPA6H GPS (Coordinates including

Altitude, Universal Time, Speed) DAINESE E,F

S13 altIMU-10 V5 Gyro, Accelerometer, Compass,

Barometer and Temperature sensor DAINESE E,F

S14 MAX30102 Low power, optical heart-rate

module PPG and Oximeter DAINESE E,F

S15 VEML6075 UV/ Ambient Light sensor DAINESE E,F

S16 SENS-ECG-UCE6 Electrocardiography bipolar

differential measure DAINESE E,F

S17 SENS-EDA-UCE6 Electro dermal Activity Skin

Conductance Sensor DAINESE E,F

S18 RESPIRATION PZT Respiration Piezoelectric film

technology DAINESE E,F

S19 LSM6DS33 Accelerometer and 6 axis IMU DAINESE E,F

S20 HDC1080 Digital Temperature and Humidity

Sensor DAINESE E,F

S21 TMP36 Low Voltage Analogic Temperature

Sensor DAINESE E,F

5.1.3 HMI Modules

HMI modules


Partner

Used in

UC

HMI1 Scania Instrument Cluster

(ICL) Dashboard display Scania A

HMI2 Scania handheld device

(Surface Pro 4) Windows tablet Scania A

HMI3 Seat Vibrators Five vibromotors for placement

within a Scania seat Scania A

HMI4 LED Light Strip Light Strip for Instrument Panel Scania A

HMI5 UCB Central Stack HMI Tablet PC used to emulate a central

stack screen Valeo B



HMI6 IPAD 2 Integrated column

Display HMI Column Display Vedecom B

HMI7 IAO simulator Screen

Array 5" RGB screen array FhG/IAO C,D

HMI8 IAO simulator Touch

Screen 7" RGB Touch-screen FhG/IAO C,D

HMI9 IAO simulator Cluster

Display Screen Cluster Display RGB Screen FhG/IAO C,D

HMI10 IAO simulator LED Array LED array FhG/IAO C,D

HMI11 STR-DH550 AV receiver FhG/IAO C,D

HMI12 DLR Cluster Display

Screen Cluster Display RGB Screen DLR C,D

HMI13 DLR LED Ambient Light LED Stripe DLR C,D

HMI14 Nexus 7 Android Tablet DLR C,D

HMI15 Flat Coreless Vibration

Motor DC Vibration Motor Dainese E,F

HMI16 TomTom Bridge

Navigation and Digital Infrastructure

unit

TomTom and

Ducati E,F

HMI17 Multistrada Dashboard 5" TFT Dashboard Ducati E,F

HMI18 Galaxy Note 10.1 SM-

P600 10" tablet VTI G

HMI19 MO-159-001-EW-1000 LED backlight LCD display VTI G

5.1.4 Interfaces

UC A Interfaces

CAN

Working Frequency N/A GHz

Data Rate 500 kbps

Ethernet


Data Rate 1 Gbps

Wi-Fi 802.11 2.4 Ghz

Working Frequency 2.4 GHz

Data Rate 6 kbps

Bluetooth 4.0


Data Rate 1 to 3 Mbps

UC B Interfaces

CAN


Data Rate 500 kbps

Ethernet




Data Rate 100 Mbps

USB 2.0


Data Rate 480 Mbps

Bluetooth 4.0



LTE

Working Frequencies 700, 800, 900, 1800, 2600 MHz

Data Rate up to 326.4 (down)

up to 86.3 (up) Mbps

802.11p


Data Rate 6 Mbps

UC C/D Interfaces

CAN


Data Rate 500 kbps

Ethernet


Data Rate 1 Gbps

USB 2.0


Data Rate 480 Mbps

Bluetooth 4.0



LTE

Working Frequencies 700, 800, 900, 1800, 2600 MHz

Data Rate up to 326.4 (down)

up to 86.3 (up) Mbps

802.11p


Data Rate 6 Mbps

UC E/F Interfaces

CAN


Data Rate 500 kbps

I2C (fast)




Data Rate 400 kbps

Radio Frequency


Data Rate 250 kbps

Bluetooth 4.0


Data Rate (real, not theoretical) ~50 kbps

UC G Interfaces

CAN


Data Rate 500 kbps

Ethernet


Data Rate 1 Gbps

Bluetooth 4.0 L.E.


Data Rate 1 Mbps

Interface Components


Partner

Used in

UC

IM1 UR44 2.0 USB Audio-

Interface

Audio/MIDI Interface OVGU B, C, D

IM2 ADS1015 Analog-to-Digital Converter Dainese E, F

IM3 OPA333AIDCKR Operational Amplifier Dainese E, F

IM4 XBEE S1 802.15.4 RF module Dainese E, F

5.1.5 Elaboration Units

Elaboration Units


Partner

Used in

UC

E1 Nuvo-3100VTC Industrial computer Scania A

E2 NISE 3600E Series Fan-less computer Vedecom B

E3 DENSO WSU 5900+RPI Communication Unit DENSO C,D

E4 Teensy 3.6 Development Board Dainese E,F

E5 MicroAutoBox II dSPACE prototyping system Ducati E,F



6 Conclusions

In order to enable ADAS&ME ("Adaptive ADAS to support incapacitated drivers & Mitigate

Effectively risks through tailor made HMI under automation") to develop adapted Advanced

Driver Assistance Systems that incorporate driver/rider state, situational/environmental

context, and adaptive interaction to automatically transfer control between vehicle and

driver/rider and thus ensure safer and more efficient road usage, this document lays a sound

technical foundation for easy development and implementation of all the Use Cases identified

in ADAS&ME Deliverable 1.2 "Driver/ Rider models, Use Cases implementation scenarios".

Major task accomplished in Work package 2 as part of this deliverable, is transforming Use

Case descriptions in natural language to functional tasks to be realized in hardware and

software. This document presents details of each and every step followed during this process

of transformation. To summarize the whole process followed in short:

Step 1: General System Architecture is designed by considering different ADAS&ME

use case descriptions.

Step 2: Each key module represented in the General System Architecture is looked in

detail in terms of its interfaces across different ADAS&ME demonstrator platforms,

possible input, and output data

Step 1 and Step 2 are designed in general for ADAS&ME Project and not specific to

any particular ADAS&ME use case. This design choice was made to provide a

flexible and extensible system, forming a future-proof platform and allowing for

algorithm re-use in case of changes in the system architecture. This also simplifies

adapting the ADAS&ME system to the different platforms provided for the individual

use cases.

Step 3: Each key module designed in Step 2 forms the basic build block of the Use

Cases. Hence system requirements for each key module across different use cases are

specified in terms of:

Tasks to be performed

Data Interactions

Output Performance

Step 4: Functional Architecture for each Use Case is derived by adapting all key

modules designed in Step 2 to requirements specified for each Use Case at Step 3 and

combining them.

Step 5: Physical Architecture is achieved by considering the existing architecture for

each ADAS&ME demonstrator vehicle and deriving the new hardware required to

realize the Use Case(s) at the corresponding demonstrator.

Step 6: Information presented in Functional and Physical architecture for each Use

Case is combined to arrive at the different layers of the Communication architecture.

Followed by the system architecture all the physical hardware components (Sensors, HMI

elements, Interfaces, Elaboration units) are quantitatively specified.

The process to describe interoperability information between any two key modules is not

described in this deliverable, since it is a preliminary version of the ADAS&ME architecture.

Furthermore, the logic of the preliminary version is to be ‘all inclusive’, meaning that all

considered potentials are included in order to be technically verified in terms of feasibility,



performance, interoperability and final integration. Input and Output data parameters for each

key module, architecture diagrams specified in this preliminary version, paves the way for

arriving at communications specifications between key modules, as well as selecting the final

components and subsystems that will actually serve the targets of each Use Case. This will be

the next major focus of our work and will be described in detail in next version of the

deliverable, which will include the final ADAS&ME Architecture and Specifications.

7 References

ANUND, A. K. (2008). Driver sleepiness and individual differences in preferences for

countermeasures. Journal of Sleep Research, 17, 16-22.

DEMENT, W. C. (1982). Current perspectives on daytime sleepiness: the issues. Sleep, 5, 56-

66.

Ganssle, J. (2008). The Art of Designing Embedded Systems, Second Edition. Elsevier Inc.

Noergaard, T. (2012). Embedded Systems Architecture, 2nd Edition, A Comprehensive Guide

for Engineers and Programmers. Newnes, Elsevier Inc.

Tania Dukic Willstrand, A. A. (June 2017). Deliverable 1.2 – Driver/Rider models, Use Cases

and implementation scenarios. ADAS&ME Project Work Package 1.



Annex 1 – Modules Specifications

Module Name Smart Eye Pro (SEP) Multi Camera

Module ID S1

Manufacturer Smart Eye

Responsible Partner Smart Eye

General Description Multi camera system for eye-tracking. Number of cameras is UC

dependent (3-5 for truck, 2 for car and bus).

Micro N/A

Memory N/A

Battery N/A

Supported OS Windows 7, 10

I/O Ports Ethernet (TCP/UDP)

Communication

network details Protocol: Proprietary API

Module Performances


The multi camera a system is used to detect the following parameters: - Head Position <x,y,z> - Head Rotation <φ,θ,ψ> - Gaze direction <x,y,z>

- Eyelid Opening - Pupil Diameter

- Blink - Fixation - Saccade

- Driver Identity

Data Acces Information for other modules: The system will process the video stream on a dedicated system in order

to compute various features (i.e. listed above). Those features will be

accessible to the next module thought dedicated API shared by SEYE.

It is not possible to directly specifies the system performances in

measuring them, because they depend on the number of cameras used, the

camera placement (which is different for every kind of vehicle), on the

continuous update of the profile during tracking and on the specific

situation. The only thing that can be specified are the cameras themselves,

and the lenses used.

Camera

Max Image Circle 1/2 in

Sensor Type CMOS -

Sensor Size 6.1 x 4.9 mm

Resolution (H x V) 1280 x 1024 px

Pixel size (H x V) 4.8 x 4.8 μm

Mono/Color Mono -

EMVA quantum

efficacy 55 (typical) %

Dark noise 10.7 (typical) e¯



Saturation capacity 7.8 (typical) ke¯

Dynamic range 57.3 (typical) dB

SNR 38.9 (typical) dB

GM26014MCN

Focal length 6 mm

Iris Range f1.4 - close -

Angle of view (H x V x

D) 1/2" 54.58 x 42.32 x 65.65 deg

MOD 0.2 m

Filter Thread M27.0 x P0.5 -

GMHR38014MCN

Focal length 8 mm

Iris Range f1.4 - 16 -


D) 2/3" 56.6 x 43.95 x 67.22 deg

MOD 0.1 m

Filter Thread M27 x P0.5 -

GM21214MCN

Focal length 12 mm

Iris Range f/1.4-Close -


D) 1/2" 28.58 x 21.56 x 35.49 deg

MOD 0.3 m

Filter Thread M27 x P0.5 -

Physical Module

Specifications


Dimensions 42x29x29 mm

Weight 90 g

Operating Temperature

Range 0 to 50 °C


Max humidity @40 °C % r.h.

Supply Voltage 12 to 24 (in alternative also PoE is

available) V

Peak Voltage 24 V

Power Consumption 3.5 (typical) W

Battery (Type /Ah) N/A Ah

Environmental

Specifications

Functional Safety N/A

Environment IP30

Vibration 10-58 Hz / 1.5 mm_58-500 Hz / 20 g_1

Octave/Minute (10 repetitions)

Shock 20 g / 11 ms / 10

shocks

Bump 20 g / 11 ms / 100

shocks positive

EMI/EMC N/A

Load Dump N/A



Note

Module Name Empatica E4 wristband

Module ID S2

Manufacturer Empatica

Responsible Partner Scania, Valeo

General Description Wristband that measures skin conductance (EDA), heart rate (PPG),

temperature, movements (acc)

Micro N/A

Memory Flash (no info about size, but corresponding to 60h of measurements)

Battery Streaming mode: 20+ h, Memory mode: 36+ h, Charging time: < 2 h

Supported OS The API support both OSX and Windows

I/O Ports Bluetooth Smart (for streaming), USB for downloading already recorded

data

Communication

network details

The Watch streams data via Bluetooth to a dedicated processing

unit hosting a data server. The data can be accessed then by the next

module using proprietary API linked above.

Module Performances


Blood Volume Pulse, Photo plethysmography

Range N/A

Accuracy

Resolution 0.9 nW/digit

Frequency 64 Hz

Electro dermal Activity

Range 0.01 to 100 µS

Accuracy

Resolution 1 digit ~900 pS

Frequency 8 Hz

Skin temperature + Heat flux

Range

-40 to 85 for ambient temperature (if

available) -40 to 115 for skin temperature

°C

Accuracy ±0.2 (in the 36 to 39 °C interval) °C

Resolution 0.02 °C

Frequency 4 Hz

3-axis accelerometer

Range ±2 (ranges of ±4g or ±8g are

selectable with custom firmware) g

Accuracy g

Resolution 8 bits of the range g

Frequency 32 Hz

Physical Module

Specifications


Dimensions 44x40x16 (case)

110 to 190 (band length) mm

Weight 40 g




Range °C


Max humidity @40 °C 0 to 100 % r.h.

Supply Voltage 3.7 V

Peak Voltage V

Supply Current A

Battery (Type /Ah) Li-Ion / 260 mAh

Environmental

Specifications

Functional Safety

CE Cert. No. 1876/MDD (93/ 42/EEC Directive,

Medical Device class 2a), FCC CFR 47 Part

15b, IC (Industry Canada), RoHS

Environment IP22

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name zForce SW

Module ID S3

Manufacturer Autoliv

Responsible Partner Autoliv

General Description z-Force Steering Wheel

Micro External processing unit for gesture identification needed. Possible

solutions are Raspberry Pi v2B and Intel NUC

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports CAN

Communication

network details

The steering wheel is connected to its own processing unit (i.e. Intel

NUC) for processing events happening on the steering wheel. That

information is provided to the next module through MQTT

protocol.

Module Performances


Response Time 80 ms

Maximum distance

from steering wheel for

detection

50 mm

Minimum size of a

detectable object 0.8 mm

Physical Module

Specifications


Dimensions TBD, it depends on the steering

wheel where the sensors will be

integrated

mm

Weight g


Range N/A °C



Temperature Gradient N/A °C/min

Max humidity @40 °C N/A % r.h.

Supply Voltage 12 V

Peak Voltage N/A V

Supply Current 3 A


Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

- It is not still clear if the steering wheel will have the z-Force sensor (at

the moment the most probable option) or physiological sensors (ECG,

EMG, and EDA). - It is not still not clear if an Autoliv steering wheel or the current steering

wheels in the demonstrators will be used.

Module Name UWB Radar Sensor

Module ID S4

Manufacturer Valeo

Responsible Partner Valeo / NUIG

General Description Ultra-Wide Band Radar

Micro

Memory

Battery

Supported OS Windows 7/10

I/O Ports connector, 3 wires (battery voltage, GND, LIN)

Communication

network details

Protocol: LIN (slave)

The radar will have its own dedicated processing unit to handle

measurement. Then data will be accessible through UDP Packet to

the next module. Software interface (API) will be developed in

parallel to the sensor and should be ready by May 2018.

Module Performances


Heart Rate

Range 0 to 240 bpm

Resolution 1 bpm

Frequency 250 (up to 10kHz under evaluation) Hz


Range 0 to 100 ms

Resolution 8 (0.2) ms


Respiration Rate

Range 0 - 80 rpm

Resolution 1 rpm




Physical Module

Specifications


Dimensions 80 x 50 x 30 mm

Weight t.b.d. g


Range t.b.d. °C

Temperature Gradient t.b.d. °C/min

Max humidity @40 °C t.b.d. % r.h.

Supply Voltage t.b.d. V

Peak Voltage t.b.d. V

Supply Current t.b.d. A

Battery (Type /Ah) t.b.d. Ah

Environmental

Specifications

Functional Safety t.b.d.

Environment t.b.d.

Vibration, Shock,

Bump t.b.d.

EMI/EMC t.b.d.

Load Dump t.b.d.

Note Sensor is under development, all values could change after validation

Module Name Zephyr Bioharness 3

Module ID S5

Manufacturer Zephyr Technology

Responsible Partner Valeo

General Description Chest strap providing ECG, Breathing, Posture and Temperature

monitoring

Micro N/A

Memory Will be used in streaming mode

Battery 24 hrs Standby Mode, 18 hrs Active Mode

Supported OS Data are streamed over BT on virtual COM port. BT protocol provided

I/O Ports Bluetooth Smart (for streaming), USB for downloading already recorded

data

Communication

network details Protocol: Proprietary BT API

Module Performances


Heart Rate

Range 25-240 BPM

Resolution 1 BPM

Frequency 60 Hz

Breathing Rate

Range [3-70]

Breaths

per

minute

Resolution 1

Breaths

per

minute

Frequency 25 Hz

Skin temperature

Range [10,60] °C



Resolution 2 °C

Frequency 1 Hz

3-axis accelerometer

Range [-16,16] g

Resolution g

Frequency 100 Hz

Physical Module

Specifications


Dimensions 28 (Diam) x 7 mm (1.84 x 0.44

inches) mm

Weight 71 g


Range -30 to 60 °C



Supply Voltage N/A V

Peak Voltage N/A V

Supply Current N/A A


Environmental

Specifications

Functional Safety FCC Part15, RTTR 1999/4/EC, RoHS

2002/95/EC

Environment ESD: IEC 801-2KV

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name ibeo LUX and ibeo LUX Fusion System

Module ID S6

Manufacturer ibeo

Responsible Partner Vedecom

General Description LIDAR and fusion system

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports CAN, Ethernet(IP, FTP), power

Communication

network details

IP, default addresses 192.168.0.1:12002 for LUX and

192.168.0.100:12002 for ICU and Fusion System

Module Performances


Distance

Range (10% remission) up to 200 (50) m

Accuracy 0.1 m

Resolution 0.04 m

Frequency 12.5 / 25 / 50 Hz



FOV

Horizontal FOV range 110 ( -60 to +50) deg

Vertical FOV range 3.2 deg

Horizontal Angular

Resolution up to 0.125 deg

Vertical Angular

Resolution 0.8 deg

Frequency 12.5 / 25 / 50 Hz

Physical Module

Specifications



Weight approximately 1 kg


Range -40 to +185 °C



Supply Voltage 9 to 27 V

Peak Voltage 27 V

Power Consumption 8 (average), < 10 (max) W


Environmental

Specifications


Environment IP68

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name ARS 408-21 Radar

Module ID S7

Manufacturer Continental


General Description 77 GHz Radar

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports CAN

Communication

network details

Module Performances


Distance

Range

- 0.20 to 250 (far range) - 0.20 to 70m/100 (@0…±45° near

range) - 0.21 to 20 (@ ±60° near range)

m



Accuracy ± 0.4 (far range)

± 0.1 (near range) m

Velocity (relative velocity of the detected vehicle)

Range -400 to +200 km/h

Accuracy ± 0.1 km/h

Physical Module

Specifications


Dimensions 137.25 x 90.8 x 30.66 mm

Weight app. 320 g g


Range -40to +85 °C



Supply Voltage 8 V

Peak Voltage 32 V

Supply Current 550 mA (@ 6.6 W, typ.) and 1.0 A

(@ 12 W, max. peak power) A

Battery (Type /Ah) - Ah

Environmental

Specifications


Environment

IP 6k 9k (dust, high-pressure cleaning) IP 6k7 (10 cm under water), ice-water shock

test, salt fog resistant, mixed gas EN 60068-2-60

Vibration, Shock,

Bump

500 m/s2@6 ms half-sine (10 x shock each in

+/-X/Y/Z dir.); 20 [(m/s2)2/Hz]@10 Hz / 0,14

[(m/s2)2/Hz]@1000Hz (peak)

EMI/EMC N/A

Load Dump disconnection >60 V and re-start returning to

<60 V

Note

Module Name Velodyne Lidar Puck VLP-16

Module ID S8

Manufacturer Velodyne


General Description Roof-mounted LIDAR

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports Ethernet, GPS, power

Communication

network details

Module Performances


Distance

Range up to 100 m

Accuracy ± 0.03 m

FOV



Horizontal FOV range 360 deg

Vertical FOV range 30 (-15to +15) deg

Horizontal Angular

Resolution 0.1 to 0.4 deg

Vertical Angular

Resolution 2 deg

Physical Module

Specifications


Dimensions 103 x 71.7 (φ,h) mm

Weight 830 g






Peak Voltage 18 V

Supply Current N/A N/A

Battery (Type /Ah) N/A N/A

Environmental

Specifications


Environment IP67

Vibration, Shock,

Bump

- Shock: 500 m/s2 Amplitude, 11 ms Duration - Vibration: 5 Hz to 2,000 Hz, 3 G_rms

EMI/EMC N/A

Load Dump N/A

Note

Module Name Shure VP82

Module ID S9

Manufacturer Shure

Responsible Partner OVGU

General Description Shotgun Microphone

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports XLR-3

Communication

network details

Module Performances


Pick-up Pattern Super cardioid / lobar -

Frequency Response 90 - 20000 Hz

Physical Module

Specifications


Dimensions (L x Ø) 195 x 22 mm

Weight 76 g


Range -18 to 57 °C




Max humidity @40 °C 0 to 95 % r.h.

Supply Voltage 11 to 52 VDC V

Peak Voltage N/A V

Supply Current < 2.0mA A


Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note The sensor needs Sennheiser MZA 900P Phantom Power Adapter

Module Name Sennheiser HSP 4 EW-3 beige

Module ID S10

Manufacturer Sennheiser


General Description Headset/Neckband Microphone

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports 3.5 mm mini-jack

Communication

network details

The audio signal is transmitted to the next module through USB

connection and can be access through proprietary API.

Module Performances


Pick-up Pattern cardioid -


Physical Module

Specifications


Dimensions (Width) 110 mm

Weight 4.8 g


Range -10 to 50 °C



Supply Voltage 4.5 to 15 V

Peak Voltage N/A V

Supply Current 250 µA A


Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A



EMI/EMC N/A

Load Dump N/A

Note The sensor needs Sennheiser MZA 900P Phantom Power Adapter (both

for power supply and wired application)

Module Name cECG

Module ID S11

Manufacturer RWTH

Responsible

Partner RWTH

General

Description

Seat-integrated capacitive ECG, Magnetic Induction and Photo plethysmography

module

Micro N/A

Memory no considerable memory included inside the sensor, memory of a PC is used

Battery (not included at the moment)

Supported OS Windows 7/10

I/O Ports Virtual Com and USB connected to a PC, Ethernet (TCP/UDP) to external PC

Communication

network details

Protocol: Proprietary

The seat will include a dedicated processing unit, then data will be available

through Ethernet to the next module. The processing unit will have a power

consumption much lower than a PC.

Data Packet

Structure

Bit stuffing

Bit stuffing with 0xAA (for this example) as an escape sequence is used

throughout all packages. The only case where 0xAA appears in the data

stream is at the beginning of a package and of a sample. If data with 0xAA

as a value must be transmitted, the values is escape with a 0xAA byte. The

following description is based on the original / restored stream.

Package Header

Byte 0 1 2

Data Preamble1 Preamble Number of samples in this package

Example 0xAA 0xCC 1,2,3, …

Explanation Package start Raw data or rate package

Raw Data Byte 0 1 2 3 4 5 6 7

Data Preamble1 Preamble Data

identifier

Sensor

/channel

identifier

Data0 Data1 Data2 Data4

Example 0xAA 0xDD 0x01,

0x02

0x01,

0x02 3.52, 5.00, 5e 10-3, ...

Explanation sample start

PPG,

MIM,

cECG

Up-left,

up-right,

middle-

left, …

32-bit values, floating-point

number

Heart Rate Data Byte 0 1 2 3 4 5 6

Data Preamble1 Preamble Data

identifier Data0 Data1 Data2 Data4

Example 0xAA 0xEE 0x01, 0x02 3.52, 5.00, 5e 10-3, ...

Explanation sample start HR, RR 32-bit values, floating-point number



Module

Performances


capacitive ECG

Range 20 V

Accuracy 1.3 mV

Frequency 1000 Hz

Magnetic Induction

Range 15-25 MHz

Accuracy 5-10 (delta in Frequency) HZ

Frequency ~100 Hz

Photo plethysmography

Range 0 - 3.3 V

Accuracy 50 µV

Frequency ~100 Hz

Physical

Module

Specifications


Dimensions measurement box

260 x 210 x 100

+ PC

mm

Dimensions sensor head 100 x 100 x 100 mm

Weight t.b.d. g

Operating Temperature Range t.b.d. °C

Temperature Gradient t.b.d. °C/min

Max humidity @40 °C t.b.d. % r.h.


Peak Voltage 18 V

Supply Current 2.5 A


Environmental

Specifications


Environment N/A

Vibration, Shock, Bump N/A

EMI/EMC N/A

Load Dump N/A

Note Sensor prototype is under development. Given specification are estimations.

Module Name FGPMMOPA6H

Module ID S12

Manufacturer Global Top Technology

Responsible Partner DAINESE

General Description GPS (Coordinates including Altitude, Universal Time, Speed)

Micro ARM

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports UART LVTTL



Communication

network details

UART NMEA sentences as standard. Detailed information should be

found in PMTK_A11.pdf

Module Performances


Coordinates

LatitudeRange -90 to +90 deg

Longitude Range -180 to +180 deg

Course Range 0 to +360 deg

Altitude Range 0 to +18000 m

Position Accuracy 2,5 with DGPS ; 3 without DGPS m

Quality PDOP, HDOP, VDOP number

Time

Date YYYY/MM/DD

Time HH:MM:MM:mmm

Accuracy 10 ns

Velocity

Range 0 to 515 m/s

Accuracy 0,05 with DGPS ; 0.1 without DGPS m/s

Update Rate (for all measurements)

Frequency 1 to 10 Hz

Note above 5 Hz, DGPS cannot work m

Physical Module

Specifications


Dimensions 15x15 embedded mm

Weight 15 g


Range -40° to +85°C °C

Temperature Gradient 10 °C/min

Max humidity @40 °C 95 % r.h.

Supply Voltage 3V3 V

Peak Voltage 4,3 V

Supply Current 0,025 A

Battery (Type /Ah) -- Ah

Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name altIMU-10 V5

Module ID S13

Manufacturer POLOLU


General Description Gyro, Accelerometer, Compass, Barometer and Temperature sensor

Micro N/A



Memory N/A

Battery N/A

Supported OS N/A

I/O Ports I2C

Communication

network details

I2C communication, see LIS3MDL.pdf, LPS25H.pdf, LSM6DS33.pdf for

details

Module Performances


Linear Accelerations ( along x,y,z)

Range ±16 g

Accuracy ±0.04 g

Frequency 1600 Hz

Angular Rate (along x,y,z)

Range ±2000 deg/s

Accuracy ±10 deg/s

Frequency 800 Hz

Magnetic Field

Range ±16 G

Accuracy ±1 G

Frequency 100 Hz

Environmental Pressure

Range 260 to 1260 hPa

Accuracy ± 0.1 hPa

Temperature

Range -30 to 105 °C

Accuracy ± 2 °C

Physical Module

Specifications


Dimensions 25,4 x12,7 mm

Weight 5 g






Peak Voltage 4 V



Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name MAX30102



Module ID S14

Manufacturer Maxim Integrated


General Description Low power, optical heart-rate module PPG and Oximeter

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports I2C

Communication

network details

I2C communication, see maximintegratedproducts_MAX30102 DS-

917698.pdf for details

Module Performances


Heart Rate

HR 0 to 300 Bpm


Oxygen Saturation

SPO2 0 to 110 %


Physical Module

Specifications


Dimensions 12,7 x12,7 mm

Weight 5 g






Peak Voltage 4 V



Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name VEML6075

Module ID S15

Manufacturer Vishay


General Description UV/ Ambient Light sensor

Micro N/A

Memory N/A

Battery N/A



Supported OS N/A

I/O Ports I2C

Communication

network details I2C standard (16bit)

Module Performances


UV Index

Range 0 to 11 -

Accuracy 1 -

Frequency 10 Hz

Physical Module

Specifications


Dimensions 2 x 1.25 x 1 mm

Weight N/A g





Supply Voltage 0 to 3.6 V

Peak Voltage 3.6 V

Supply Current 480 μA


Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name SENS-ECG-UCE6

Module ID S16

Manufacturer Bitalino


General Description Electrocardiography bipolar differential measure

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports Analog to I2C through ads1015

Communication

network details I2C standard accordingly to ads1015.pdf

Module Performances


Gain 1100 -

Bandwidth 0.5 to 40 Hz

Range ± 1.5 mV

Frequency defined by ADC converter Hz



Bipolar differential

Measure 3 leads

Physical Module

Specifications


Dimensions 27x12 + Wires and stickable

electrodes mm

Weight 10 g






Peak Voltage 3.5 V

Supply Current 0.17 mA


Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name SENS-EDA-UCE6

Module ID S17



General Description Electro dermal Activity Skin Conductance Sensor

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A


Communication


Module Performances


Gain 2 -

Bandwidth 0 - 5 Hz

Range 0 to 13 μS


Bipolar differential

Measure 3 leads

Physical Module

Specifications


Dimensions 27x12 + Wires and stickable

electrodes mm

Weight 10 g








Peak Voltage 5.5 V



Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name RESPIRATION PZT

Module ID S18



General Description Respiration Piezoelectric film technology

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A


Communication


Module Performances


Gain 1 -

Bandwidth 0 to 15 Hz

Range -50 to +50 %


Physical Module

Specifications


Dimensions Thoracic Strap mm

Weight 20 g






Peak Voltage 4.3 V

Supply Current 4 mA


Environmental Functional Safety N/A



Specifications Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name LSM6DS33

Module ID S19

Manufacturer ST


General Description Accelerometer and 6 axis IMU

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports Digital I2C

Communication

network details I2C standard

Module Performances


Linear Accelerations ( along x,y,z)

Range ±16 g

Accuracy ±0.04 g

Frequency 1600 Hz

Angular Rate (along x,y,z)

Range ±2000 deg/s

Accuracy ±10 deg/s

Frequency 800 Hz

Physical Module

Specifications


Dimensions 3 x 3 x 0.86 mm

Weight 1 g


Range -40 to 85 °C



Supply Voltage 1.7 to 3.6 V

Peak Voltage 3.6 V



Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A



Note

Module Name HDC1080

Module ID S20

Manufacturer Texas Instruments


General Description Digital Temperature and Humidity Sensor

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports I2C

Communication

network details I2C standard

Module Performances


Temperature

Range -40 to 70 °C

Accuracy ± 0.2 (in the 5 to 60 °C range) °C

Frequency 1 (typical) Hz

Relative Humidity

Range 0 to 100 %

Accuracy ± 2 %

Frequency 1 (typical) Hz

Physical Module

Specifications



Weight N/A g


Range -40 to 70 °C




Peak Voltage 5.5 V

Supply Current 1.3 μA


Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name TMP36



Module ID S21

Manufacturer Analog Devices


General Description Low Voltage Analogic Temperature Sensor

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A


Communication


Module Performances



Accuracy ± 1 (@ 25 °C)

± 1 (in the full range) °C


Physical Module

Specifications


Dimensions 3.5 x 4.6 x 19 mm

Weight 0.2 g



Temperature Gradient + 3 (ramp up)

- 6 (ramp down) °C/s


Supply Voltage -2.7 to 5.5 V

Peak Voltage 5.5 V



Environmental

Specifications


Environment N/A

Vibration, Shock,

Bump N/A

EMI/EMC N/A

Load Dump N/A

Note

Module Name Scania Instrument Cluster (ICL)

Module ID HMI1

Manufacturer Volkswagen Group

Responsible Partner Scania

General Description Dashboard display

Micro N/A

Memory N/A

Battery N/A



Supported OS N/A

I/O Ports HDMI

Communication network

details N/A

Module Performances


Resolution 1920 x 720 px

Screen Size N/A in

Pixel Density N/A ppi

Physical Module

Specifications


Dimensions 237 x 147 x ?? mm

Weight N/A g


Range N/A °C



Supply Voltage 12 V

Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name Scania handheld device (Surface Pro 4)

Module ID HMI2

Manufacturer Microsoft


General Description Windows tablet

Micro Core i5 8GB

Memory 256GB SSD

Battery

Supported OS Windows

I/O Ports

Full-size USB 3.0 microSD card reader

Headset jack Mini DisplayPort

Cover port Surface Connect


details

802.11ac Wi-Fi wireless networking; IEEE 802.11a/b/g/n

compatible Bluetooth 4.0 wireless technology



Module Performances


Resolution 2736 x 1824 pixels

Screen Size 12.3 in

Pixel Density 267 ppi

Physical Module

Specifications


Dimensions 292.1 x 201.42 x

8.45 mm

Weight 786 g


Range N/A °C




Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Module Name Seat Vibrators

Module ID HMI3

Manufacturer Precision Microdrives


General Description Five vibromotors for placement within a Scania seat

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports N/A


details N/A

Module Performances


Rated Vibration Speed 5000 rpm

Typical Vibration

Amplitude 1 G

Minimum Vibration

Amplitude 0.7 G

Typical Normalised

Amplitude 10 G

Vibration Frequency 25 to 95 Hz

Physical Module Definition Value Unit



Specifications Dimensions (φ,l) 24.4 x 30.8 mm

Weight 48 g


Range -20 to 65 °C




Peak Voltage 3.6 V

Supply Current 350 mA


Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name LED Light Strip

Module ID HMI4

Manufacturer Denso


General Description Light Strip for Instrument Panel

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports USB


details Serial Communication

Module Performances


Number of LEDs 100 -

Number of Colours 16777216 -

Brightness Values 256 -

Physical Module

Specifications



Weight 2 kg


Range N/A °C



Supply Voltage AC100 V

Peak Voltage N/A V

Supply Current 0.5 (50W) A




Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name UCB Central Stack HMI

Module ID HMI5

Manufacturer Microsoft

Responsible Partner Valeo

General Description Tablet PC used to emulate a central stack screen. The module

will be used for UCB related information and interactions

Micro Intel Core i7

Memory 1512 Gb HDD, 6Gb RAM

Battery 5 Ah

Supported OS Windows

I/O Ports USB 3.0, Mini DisplayPort, Audio Jack (input/output)


details Wi-Fi

Module Performances


Resolution 2736x1824 pixels

Refresh rate 60 Hz

Screen Size 12,3 in

Aspect Ratio "3:2" -

Physical Module

Specifications


Dimensions 292.1x201x42x8.45 mm

Weight 786 g


Range 5 to 40 °C



Supply Voltage 15 V

Peak Voltage V


Battery (Type /Ah) Li-Ion/5 Ah

Environmental

Specifications


Environment Interior use


EMI/EMC N/A

Load Dump N/A

Note



Module Name IPAD 2 Integrated column Display HMI

Module ID HMI6

Manufacturer Apple


General Description The integrated HMI in the Vedecom Zoe 1

Micro A5 double core

Memory 16 GB

Battery 10 hours, 25Wh

Supported OS OS X 10.5.8

I/O Ports Apple port, WiFi, BT 2.1


details No change possible. Third-party turnkey equipment

Module Performances



Screen Size 12,3 in


Physical Module

Specifications



Weight 601 g


Range 0 to 35 °C


Max humidity @40 °C 95% % r.h.

Supply Voltage 12 V

Peak Voltage 12 V


Battery (Type /Ah) Li-Ion / 2.1 Ah

Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name IAO simulator Screen Array

Module ID HMI7

Manufacturer Adafruit

Responsible Partner FhG/IAO

General Description 5" RGB screen array

Micro N/A

Memory N/A

Battery N/A

Supported OS Mac, Windows 7, and Debian Linux



I/O Ports HDMI


details N/A

Module Performances


Active area 108 x 64.8 mm


Brightness 200 cd/m2

Screen size 5 in

Pixel pitch 0.135 x 0.135 mm2

Physical Module

Specifications


Dimensions 124x76x7 mm

Weight g


Range -20 to + 70 °C




Peak Voltage 3.6 V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Already integrated in the simulator. Possible to control three displays at the same time with

Matrox TripleHead2Go and a Displayport Input

Module Name IAO simulator Touch Screen

Module ID HMI8

Manufacturer Adafruit


General Description 7" RGB Touch-screen

Micro N/A

Memory N/A

Battery N/A

Supported OS Mac, Windows 7, and Debian Linux

I/O Ports HDMI, USB


details N/A




Active area 154.08 x 85.92 mm



Screen size 7 in

Pixel pitch N/A mm2

Physical Module

Specifications



Weight 241 g


Range -20 to + 70 °C




Peak Voltage 3.6 V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name IAO simulator Cluster Display Screen

Module ID HMI9

Manufacturer N/A


General Description Cluster Display RGB Screen

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports VGA


details

Module Performances


Active area N/A mm


Brightness N/A cd/m2

Screen size 12 in

Pixel pitch N/A mm2

Physical Module

Specifications





Weight N/A g


Range N/A °C




Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name IAO simulator LED Array

Module ID HMI10

Manufacturer Fraunhofer Vehicle Interaction Lab


General Description LED array

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports Remote control or CAN


details For CAN a SUB-D 9 connector

Module Performances


3 partitions with 4 red LEDs each

Audio Sounds possible possible from SD card

Physical Module

Specifications


Dimensions 280 x 40 x26 mm

Weight N/A g


Range N/A °C




Peak Voltage N/A V





Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name STR-DH550

Module ID HMI11

Manufacturer Sony


General Description AV receiver

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports HDMI, Coaxial, Analog


details

Module Performances


Traffic Sounds

Audio Sounds possible

Physical Module

Specifications



Weight 7800 g


Range N/A °C




Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name DLR Cluster Display Screen

Module ID HMI12

Manufacturer N/A.

Responsible Partner DLR



General Description Cluster Display RGB Screen

Micro N/A

Memory N/A

Battery N/A

Supported OS Windows 7

I/O Ports N/A


details

Module Performances


Active area N/A mm


Brightness N/A cd/m2

Screen size N/A in

Pixel pitch N/A mm2

Physical Module

Specifications



Weight N/A g


Range N/A °C




Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name DLR LED Ambient Light

Module ID HMI13

Manufacturer N/A


General Description LED Stripe

Micro N/A

Memory N/A

Battery N/A

Supported OS Windows 7

I/O Ports Arduino, USB


details



Module Performances


LED Stripe 120 LEDs

LED density 1 LED every 2.4cm

Physical Module

Specifications


Dimensions 2880 mm

Weight N/A g


Range N/A °C



Supply Voltage 5 V

Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name Nexus 7

Module ID HMI14

Manufacturer Asus


General Description Android Tablet

Micro Krait Quad-Core, 1.5 GHz

Memory 16 Gb HDD / 2 Gb RAM

Battery 9.5 hours / 3950 mAh

Supported OS Android

I/O Ports micro USB


details

WiFi (802.11 a/b/g/n) Bluetooth (4.0 with A2DP/LE)

micro USB 2.0

Module Performances



Screen Size 7 in


Physical Module

Specifications



Weight 290 g




Range N/A °C




Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name Flat Coreless Vibration Motor

Module ID HMI15

Manufacturer Seeed Technologies

Responsible Partner Dainese

General Description DC Vibration Motor

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports only DC power source


details


RPM 10000 rpm

Physical Module

Specifications


Dimensions (Ø x H) 10 x 2.7 mm

Weight 0.9 g


Range -20 to 70 °C




Peak Voltage 3.5 V



Environmental

Specifications


Environment N/A


EMI/EMC N/A



Load Dump N/A

Note

Module Name TomTom Bridge

Module ID HMI16

Manufacturer TomTom

Responsible Partner TomTom and Ducati

General Description Navigation and Digital Infrastructure unit

Micro Qualcomm Snapdragon [email protected] (APQ8026), Quad-

core, 1MB L2 cache

Memory 16 GB

Battery 2100mAh

Supported OS Android 4.3

I/O Ports USB 2.0 - OTG


details

Bluetooth 4.0 WiFi IEEE802.11 a/b/g/n (2.4 GHz and 5 GHz)

GSM/UMTS

Module Performances


Active area N/A mm



Screen size 7 in

Pixel pitch N/A mm2

Physical Module

Specifications



Weight 480 g


Range -20 to 60 °C



Supply Voltage 10 - 32 V

Peak Voltage N/A V


Battery (Type /Ah) Li-Ion / 2.1 Ah

Environmental

Specifications


Environment N/A

Vibration, Shock, Bump

Shock: 50g with a timebase of 6ms

in each axis

Vibration: 10 - 1000 Hz, with max

PSD = 0.208 g^2/Hz (full details



are available if needed)

EMI/EMC N/A

Load Dump N/A

Note

Module Name Multistrada Dashboard

Module ID HMI17

Manufacturer MAE

Responsible Partner Ducati

General Description 5" TFT Dashboard

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports CAN


details CAN bus

Module Performances


Active area 108 x 64.8 mm


Brightness 1000 minimal cd/m2

Screen size 5 in


Physical Module

Specifications


Dimensions N/A mm

Weight N/A g


Range N/A °C




Peak Voltage N/A V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Physical Module Specification, Environmental Specifications,

I/O and network details are not necessary, since the item is

already fully integrated and working on the motorcycle

Module Name Galaxy Note 10.1 SM-P600



Module ID HMI18

Manufacturer Samsung

Responsible Partner VTI

General Description 10" tablet

Micro 1.9 GHz octa-core

Samsung Exynos 5420 SoC processor

Memory HDD 16 Gb, RAM 3 Gb

Battery 8220 mAh

Supported OS Android

I/O Ports microUSB 2.0


details

Wi-Fi 802.11 a/b/g/n/ac BT 4.0

microUSB 2.0

Module Performances


Active area 295.8 cm2


Screen size 10.1 in

Pixel density 299 ppi

Physical Module

Specifications



Weight 540 g


Range N/A °C




Peak Voltage N/A V


Battery (Type /Ah) Li-Po/ 8220 mAh

Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name MO-159-001-EW-1000

Module ID HMI19

Manufacturer Crystal Display Systems Ltd



Responsible Partner VTI

General Description LED backlight LCD display

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports N/A


details N/A

Module Performances


Active area 376.3 x 150.5 mm



Screen size 15.9 in


Physical Module

Specifications



Weight 2.5 kg


Range N/A °C




Peak Voltage N/A V

Power Consumption 23 W


Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name UR44 2.0 USB Audio-Interface

Module ID IM1

Manufacturer Steinberg


General Description Audio/MIDI Interface

Micro N/A

Memory N/A

Battery N/A

Supported OS Windows XP and later

Mac OS X 10.5.8 and later



I/O Ports - 6x4 USB 2.0 audio interface

- 4x D-PREs

Communication

network details

Module Performances


Frame Format according to 2.0 USB specifications -

Sampling Rate 192 KHz

Resolution 24 bit

Physical Module

Specifications



Weight 1.6 kg


Range 0 to 40 °C



Supply Voltage 12 V

Peak Voltage N/A V

Supply Current 0.7 A


Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name Sennheiser MZA 900 P

Module ID IM2

Manufacturer Sennheiser


General Description Phantom power adapter

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports 3.5 mm mini-jack / XLR-3

Communication

network details

Module Performances



pre-attenuation,

switchable 0 / -12 dB

Roll-off filter,

switchable 125 Hz (-3 dB), 12db/oct



Physical Module

Specifications


Dimensions (L x Ø) 100 x 19 mm

Weight 60 g


Range -20 to 60 °C


Max humidity @40 °C < 95 % r.h.

Supply Voltage 10 - 52 VDC V

Peak Voltage N/A V

Supply Current 2.6 - 2.8 mA A


Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name ADS1015

Module ID IM3



General Description Analog-to-Digital Converter

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports I2C interface

Communication

network details

Module Performances


Frame Format dedicated format -

Sampling Rate 0.128 to 3.3 KHz

Resolution 12 bit

Physical Module

Specifications


Dimensions 2 x 1.5 x 0.4 mm

Weight kg






Peak Voltage 5.5 V





Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name OPA333AIDCKR

Module ID IM4



General Description Operational Amplifier

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports I2C interface

Communication

network details

Module Performances


Slew Rate 0.16 V/µs

Typical Gain Bandwidth

Product 350 KHz

Minimum CMRR 106 dB

Minimum PSRR 120 dB

Physical Module

Specifications



Weight < 1 g





Supply Voltage (Single) 1.8 to 5.5 V

Supply Voltage (Double) ± 0.9 ~ ± 2.75 V

Peak Voltage 5.5 V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A



Note

Module Name XBEE S1 802.15.4

Module ID IM5

Manufacturer DIGI


General Description RF module

Micro N/A

Memory N/A

Battery N/A

Supported OS N/A

I/O Ports 8 pin digital I/O

Communication

network details

Module Performances


Data rate 250 kbit/s

Indoor/Urban range 30 m

Outdoor/Line of Sight

range 100 m

Transmit Power Output 1 mW (0

dBm)

Recevier Sensitivity -92 (1% packet error rate) dBm

Physical Module

Specifications



Weight N/A. g


Range -40 to 85 °C




Peak Voltage 3.4 V



Environmental

Specifications


Environment N/A


EMI/EMC N/A

Load Dump N/A

Note

Module Name Nuvo-3100VTC

Module ID E1

Manufacturer Neousys Technology




General Description Industrial computer

CPU Intel® Core™ i7-3610QE (2.3/3.3 GHz, 6 MB cache)

Memory

8 GB DDR3

1333/1600 MHz

SDRAM

2.5” HDD/SSD

OS Linux Debian

I/O Ports

4 x Gigabit Ethernet

1 x DVI

2 x Display port

4 x USB

3.5mm mic-in Speaker-out

2 x Seriel port RS-232

Specific SW installed N/A

Physical Module

Specifications



Weight 2,8 kg

Power Consumption

8~35V DC input

via 3-pin pluggable

terminal block

W

Environmental

Specifications

Temperature -40°C ~ 85°C

Humidity 10%~90% , non-condensing

Vibration

Operating, 5 Grms, 5-500 Hz,

3 Axes(w/ SSD, according to

IEC60068-2-64)

Shock

Operating, 50 Grms, Half-sine

11 ms Duration (w/SSD,

according to IEC60068-2-27)

Note

Module Name NISE 3600E Series

Module ID E2

Manufacturer Neovo AG


General Description Fan-less computer

CPU Support 3rd generation Intel® Core™ i5/ i3 rPGA

socket type processor

Memory 2x DDR3 SO-DIMM socket, supports up to 8GB

DDR3/DDR3L 1333/1600 SDRAM



OS Windows XP/7/8.1 32bits and 64bits

I/O Ports

I/O Interface Front:

ATX power on/ off switch

HDD Access/ Power status LEDs

2x USB3.0 ports (Blue Color)

2x Display Port (Can be converted to DVI-D or HDMI

via cables)

2x Antenna holes

1x external CFast (optional)

1x SIM card socket

I/O Interface Rear:

2x DB9 for COM5 & COM6 (RS232)

1x DB44 Serial Port, 4x COM port - - COM1/COM3/COM4: RS232 - - COM2: RS232/422/485

2 x Intel® GbE LAN ports (Intel® 82574L and

82579LM); Support WoL, Teaming and PXE

2x USB2.0 ports

2x USB3.0 ports (Blue Color)

1x DB15 VGA port

1x DVI-D port

1x Line-out and 1x Mic-in

2-pin Remote Power on/ off switch

9~30V DC input


Physical Module

Specifications



Weight N/A kg

Power Consumption N/A W

Environmental

Specifications

Shock

- HDD: 20G, half sine, 11ms,

IEC60068-2-27 - CFast: 50G, half sine, 11ms,

IEC60068-2-27

Vibration

- Random: 0.5Grms @

5~500Hz according to

IEC60068-2-64 - Sinusoidal: 0.5Grms @

5~500Hz according to

IEC60068-2-6

EMC N/A

Note

Module Name DENSO WSU 5900+RPI

Module ID E3

Manufacturer DENSO



Responsible Partner DENSO

General Description Communication Unit

CPU IMX.6 / ARM CortexA53

Memory 256 MB RAM / 1GB RAM

OS QNX / LNX

I/O Ports

1x 2-wire Ethernet

2x HS Can

4x GPIO

2x FAKRA 5.9 GHz V2X

1x FAKRA GNSS

Specific SW installed DENSO CAD Platform

Physical Module

Specifications


Dimensions N/A mm

Weight N/A kg


Environmental

Specifications

Shock N/A

Vibration N/A

EMC N/A

Note

Module Name Teensy 3.6

Module ID E4

Manufacturer NXP


General Description Development Board

CPU 180 MHz ARM Cortex-M4 with Floating Point Unit

Memory

1 MB Flash

256 KB RAM

4 KB EEPROM

micro SD card port included for memory expansion

OS N/A

I/O Ports

1 x USB High Speed (480Mbit/sec) Port

1 x USB Full Speed (12Mbit/sec) Port

2 x CAN Bus Ports

32 x General Purpose DMA Channels

22 x PWM Outputs

4 x I2C Ports

11 x Touch-Sensing Inputs

62 x I/O Pins (42 breadboard friendly)

25 x Analog Inputs to 2 ADCs with 13-bit resolution

2 x Analog Outputs (DACs) with 12-bit resolution

Ethernet mac, capable of full 100Mbit/sec speed

1 x Native (4-bit SDIO) micro SD card port



1 x I2S Audio Port, 4-Channel Digital Audio Input &

Output

6 x Serial Ports (2 with FIFO and Fast Baud Rates)

3 x SPI Ports (1 with FIFO)


Physical Module

Specifications


Dimensions 62.3 x 18 x 4.2 mm

Weight 4.9 g


Environmental

Specifications

Vibration N/A

Shock N/A

EMC N/A

Note

Module Name MicroAutoBox II

Module ID E5

Manufacturer dSPACE

Responsible Partner Ducati

General Description dSPACE prototyping system

CPU IBM PPc 750 GL, 900 MHz (incl. 1 MB level2 cache)

Memory

16 MB main memory

6 MB memory for communication between

MicroAutoBox and PC

16 MB non volatile flash memory

OS N/A

I/O Ports

1 x 100/1000 Mbit/s Ethernet connection

(UDP/IPTCP/IP on request)

1 x USB 2.0

4 x CAN channels

2 x RS232 interface

2 x serial interface usable as K/L-Line or LIN interface

Specific SW installed Simulink

Physical Module

Specifications



Weight N/A kg

Power Consumption Max 50 W

Environmental

Specifications Vibration

- ISO 16750-3:2007 / 4.1.2.4

Test IV - DO-160F.8 / B1 Test

Conditions - EN 60068-2-6



Shock

- ISO 16750-3:2007 / 4.2.2 - RTCA / DO-160F Section 7

Test 7.2 Category A and D, Test type R

EMC

- EN 61326-1, Table 2 - ISPR 11, EN 55011 Group 1,

Class A - TCA/DO160G: Dec. 2010: Section 21.4 and 21.5

Note

Documents

Deliverable 2.1 ADAS&ME Architectural Framework and System ... · Angelos Bekiaris, CERTH Anna Anund, VTI Christer Ahlström, VTI Christophe René Joseph Ecabert, EPFL Daniel Teichmann,