HERA slnt wp2 d22 final - helios.mi.parisdescartes.frhelios.mi.parisdescartes.fr/~nikos/HERA_slnt_wp2_d22_final.pdf · AAL-45061 Deliverable D 2.2 Title: HERA Specifications and Validation

AAL-45061

Deliverable D 2.2 Title: HERA Specifications and Validation plan

Deliverable Type: PU*

Nature of the Deliverable: R** Date: 15/07/10

Distribution: WP2 Code: HERA/SLNT /WP2/D22-final

Editor: SOLINET Contributors: All Partners

*Deliverable Type: PU= Public, RE= Restricted to a group specified by the Consortium, PP= Restricted to other program participants (including the Commission services), CO= Confidential, only for members of the Consortium (including the Commission services)

** Nature of the Deliverable: P= Prototype, R= Report, S= Specification, T= Tool, O= Other

Abstract: This document represents the deliverable D2.2 – HERA Specifications and Validation plan. The aim of this deliverable is to select and specify the services to be implemented, to select the equipment and the SW technologies required for the implementation and deployment of the services, to specify the HERA network and Application Server architecture and its components and to define the test and validation plan.

© Copyright by the HERA Consortium.

The HERA Consortium consists of:

TELEKOM AUSTRIA Project Coordinator Austria ALCATEL-LUCENT Partner Germany SINGULAR LOGIC Partner Greece PARIS DESCARTES UNIVERSITY Partner France SOLINET Partner Germany HYGEIA Partner Greece ROTES KREUZ Partner Austria

D2.2: HERA Specifications and Validation Plan Page 2 of 95

HERA/SLNT/WP2/D22-final © HERA Consortium – July 2010

This page has been intentionally left blank



Table of Contents

GLOSARY ................................................................................................................................ 7

1. INTRODUCTION............................................................................................................. 9 1.1 SCOPE ........................................................................................................................... 9 1.2 DELIVERABLE’S STRUCTURE ...................................................................................... 10 1.3 METHODOLOGY .......................................................................................................... 11

2. HERA NETWORK ARCHITECTURE ....................................................................... 12

3. SELECTION OF THE SERVICES TO BE IMPLEMENTED ................................. 15

4. HERA SERVICES DETAILED SPECIFICATION ................................................... 18 4.1 GAMES ........................................................................................................................ 18

4.1.1 Cognitive Reinforcement Service ....................................................................... 18 4.2 INFORMATIONAL SERVICES ........................................................................................ 26

4.2.1 Passive Communication Service ........................................................................ 26 4.3 MEDICAL MEASUREMENTS ......................................................................................... 30

4.3.1 Blood Pressure Monitoring Service ................................................................... 30 4.3.2 Body Weight Monitoring Service ....................................................................... 34

4.4 NUTRITION AND HEALTH ............................................................................................ 38 4.4.1 Nutrition Counseling Service ............................................................................. 38

4.5 ORGANISER ................................................................................................................. 41 4.5.1 Reminders Service .............................................................................................. 41

5. EQUIPMENT SELECTION AND SUPPORTED FEATURES ................................ 44 5.1 SET-TOP-BOX/TVSET SELECTION AND SUPPORTED FEATURES .................................. 44 5.2 MEDICAL DEVICES SELECTION AND INTERFACES ....................................................... 47

5.2.1 A&D blood pressure monitor ............................................................................. 49 5.2.2 A&D weight scale ............................................................................................... 49

5.3 AONTV SERVICE PLATFORM TECHNICAL DETAILS AND INTERFACES ........................ 50 5.4 APPLICATION SERVER SELECTION AND TECHNICAL DETAILS ..................................... 51 5.5 OTHER DEVICES SELECTION AND INTERFACES ........................................................... 52

5.5.1 ADSL Router ...................................................................................................... 52 5.5.2 Equipment for Connecting Bluetooth Devices to the Internet ........................... 52

6. SW TECHNOLOGIES SELECTION .......................................................................... 55 6.1 HUMAN MACHINE INTERFACES TECHNOLOGIES ......................................................... 55

6.1.1 Capability exchange ........................................................................................... 55



6.1.2 Remote control / TV-based interaction .............................................................. 56 6.1.3 Two-way communication .................................................................................... 56 6.1.4 Voice-based User Interfaces .............................................................................. 56

6.2 COMMUNICATION TECHNOLOGIES .............................................................................. 58 6.3 SERVICE EXECUTION PLATFORM AND DATABASE TECHNOLOGY ................................ 59 6.4 AGENT PLATFORM ...................................................................................................... 60 6.5 DATA MODELING ....................................................................................................... 62

7. HERA APPLICATION SERVER ARCHITECTURE AND FUNCTIONAL SPECIFICATION .................................................................................................................. 65

7.1 ARCHITECTURE OVERVIEW ........................................................................................ 65 7.2 NOTIFICATIONS MODULE............................................................................................ 67 7.3 MEASUREMENTS MODULE .......................................................................................... 67 7.4 AONTV INTERFACE MODULE ..................................................................................... 68 7.5 AUTHENTICATION MODULE ........................................................................................ 69 7.6 POLICY MANAGEMENT MODULE ................................................................................ 69 7.7 THE MULTI-AGENT SYSTEM MODULE (MAS) ........................................................... 70

8. TEST AND VALIDATION PLAN ................................................................................ 74 8.1 SYSTEM VALIDATION TEST PLAN ............................................................................... 74

8.1.1 Unit Testing ........................................................................................................ 75 8.1.2 Integration Testing ............................................................................................. 75 8.1.3 System Testing .................................................................................................... 75

8.2 DEPLOYMENT PLAN .................................................................................................... 76 8.2.1 Austrian Pilot Site .............................................................................................. 76 8.2.2 Greek Pilot Site .................................................................................................. 80

8.3 USER ACCEPTANCE TEST PLAN .................................................................................. 81 8.3.1 Diabetes and Cardiac Diseases User Group ..................................................... 82 8.3.2 MCI and AD user group ..................................................................................... 83

9. CONCLUSIONS ............................................................................................................. 85

REFERENCES ....................................................................................................................... 86

ANNEX A - CANDIDATE SET-TOP-BOXES ................................................................... 87



List of Figures

Figure 1: Methodology followed for definition of the HERA specification and validation plan11 Figure 2: HERA Architecture Overview .................................................................................. 12 Figure 3: HERA home environment ........................................................................................ 13 Figure 4: Cognitive reinforcement service architecture diagram and stakeholders ................. 19 Figure 5: The cognitive reinforcement service ontology ......................................................... 21 Figure 6: The cognitive reinforcement services interfaces ...................................................... 22 Figure 7: Usage of the cognitive reinforcement service by a medical expert (Use Case 2.2) . 23 Figure 8: Usage of the cognitive reinforcement service by a HERA end user (Use Case 2.3) 24 Figure 9: Passive Communication service architecture diagram and stakeholders ................. 26 Figure 10: The HERA user data for the reality reinforcement service in UML class diagram

format. .............................................................................................................................. 27 Figure 11: Push information interfaces .................................................................................... 27 Figure 12: The Information push service ontology .................................................................. 28 Figure 13: Push information service operation (Use Case 2.8) ............................................... 29 Figure 14: Blood pressure monitoring architecture diagram and stakeholders ........................ 30 Figure 15: Weight monitoring architecture diagram and stakeholders .................................... 34 Figure 16: Nutrition counseling architecture diagram and stakeholders ................................. 38 Figure 17: Nutrition counselling pyramid ................................................................................ 40 Figure 18: Reminders architecture diagram and stakeholders ................................................. 41 Figure 19: Austria Telecom ADB-3800W ............................................................................... 45 Figure 20: UA-767PBT blood pressure monitor ...................................................................... 49 Figure 21: UA-321PBT weight scale ....................................................................................... 49 Figure 22: AonTV platform architecture ................................................................................. 50 Figure 23: Thomson 585 v7 ADSL router ............................................................................... 52 Figure 24: “Bluetooth Access Point + Router Class 1” ........................................................... 53 Figure 25: BlueRadios BR-BG-01 ........................................................................................... 53 Figure 26: Asus WL-500g Premium router .............................................................................. 54 Figure 27: The Web4CE framework ........................................................................................ 55 Figure 28: The HERA data model (part 1) ............................................................................... 63 Figure 29: The HERA data model (part 2) ............................................................................... 64 Figure 30: HERA Application Server Architecture ................................................................. 65 Figure 31: Notifications Module .............................................................................................. 67 Figure 32: Measurements Module ............................................................................................ 68 Figure 33: Authentication Module ........................................................................................... 69 Figure 34: Policy Management Module ................................................................................... 70 Figure 35: The Information push service ontology .................................................................. 71 Figure 36: The MAS module interface .................................................................................... 73 Figure 37: Remind the user about his tasks (Use Case 2.1) ..................................................... 73 Figure 38: HERA testing and validation methodology ............................................................ 74 Figure 39: Overall HERA testing and validation plan ............................................................. 75 Figure 40: Service process for HERA ordering and installation .............................................. 79



List of Tables

Table 1: Home Environment Equipment.................................................................................. 13 Table 2: Consolidated services needs per user group and services prioritisation for

implementation ................................................................................................................. 16 Table 3: Candidate HERA IPTV STB ..................................................................................... 46 Table 4 Candidate HERA digital blood pressure monitor ....................................................... 48 Table 5: Candidate HERA weight scale ................................................................................... 48 Table 6: Windows 2008 server system requirements ............................................................... 51 Table 7: IBM System x3650 M2 technical characteristics ...................................................... 51 Table 8: Communication Technologies ................................................................................... 58 Table 9: Web Application Technology stacks ......................................................................... 59 Table 10: Application Server Licensing Scheme ..................................................................... 60 Table 11: Database Technologies ............................................................................................ 60 Table 12: AonTV interface module Web Services .................................................................. 68



GLOSARY 3G 3rd Generation, a family of standards for mobile telecommunications AAL Ambient Assisted Living ACL Agent Communication Language (agent systems context) ACL Access Control List (user authentication/authorization context) AD Alzheimer’s Disease ADSL Asymmetric Digital Subscriber Line AJAX Asynchronous JAvascript and XML API Application Programming Interface AS Application Server DVB Digital Video Broadcasting DVB-T Digital Video Broadcasting–Terrestrial EJB Enterprise Java Beans FIPA Foundation for Intelligent Physical Agents, http://fipa.org GUI Graphical User Interface HDMI High-Definition Multimedia Interface HMI Human-Machine Interface HTML Hyper Text Markup Language HTTP Hyper-Text Transfer Protocol HTTPS Hyper-Text Transfer Protocol Secure IEEE Institute of Electrical and Electronics Engineers IP Integrated Project IP Internet Protocol JADE Java Agent Development Environment, http://jade.tilab.com/ MAS Multi-Agent System MCI Mild Cognitive Impairment MHP Multimedia Home Platform PA Personal Assistant PC Personal Computer



RS-232 Recommended Standard 232 SOAP Simple Object Access Protocol STB Set-Top-Box

TCP Transmission Control Protocol, one of the core protocols of the Internet Protocol

USB Universal Serial Bus WAP Wireless Access Point Wi-Fi Wireless network technology WP Work Package XML eXtensible Mark-up Language XSD XML Schema Document



1. INTRODUCTION The HERA project aims to provide a platform with cost-effective specialised assisted living services for the elderly people suffering from MCI or mild/moderate AD or other diseases (diabetes, cardiovascular) with identified risk factors, which will significantly improve the quality of their home life, extend its duration and at the same time reinforce social networking. HERA will apply technological solutions for aiding users managing their daily lives. Thus, by using the HERA system, the time to be at home, rather than in an institution, will be prolonged and relieve them from visiting the specialists often, while keeping them able to perform their daily activities and social interactions. This document represents the deliverable D2.2 “HERA Specifications and Validation Plan”, which is the result of Task 2.3 “HERA Architecture Specification” and Task 2.4 “Test and Validation plan” in WP2. The aim of this deliverable is to select and specify the services to be implemented, to select the equipment and the SW technologies required for the implementation and deployment of the services, to specify the HERA network architecture and its components and to define the test and validation plan.

1.1 SCOPE The general scope of this deliverable is to provide the HERA specifications and the test and validation plan. Following the state-of-the-art and requirements analysis made in the deliverable D21 “State-of-the-art and Requirements Analysis” and based on its results, the main objectives of this deliverable are: • To select and specify in detail the services to be implemented, deployed and piloted. • To select the equipment required for the implementation and deployment of the services. • To select the SW technologies for the implementation of services application logic, HMIs

and communication interfaces. • To specify the HERA network architecture and its components, with particular focus on the

HERA application server functional specification, which is the main component of the HERA system.

• To define the test and validation plan for the HERA services. This deliverable serves to facilitate the future work in the HERA project related to: • The detailed design and development of the HERA network infrastructure, the Application

Server and its components. • The detailed design and development of the HERA services. • The deployment and validation of the HERA services in the Greek and Austrian Pilot sites.



1.2 DELIVERABLE’S STRUCTURE The deliverable is structured in 9 chapters: • Chapter 1 provides the introduction of the document, defines its scope and describes the

methodology followed. • Chapter 2 presents the HERA architecture, its main components and stakeholders. • Chapter 3 describes the methodology for the selection of the services to be implemented and

outlines the selected services to be implemented and deployed. • Chapter 4 provides the services detailed specification. • Chapter 5 provides the details with respect to the equipment selection, which will be used

within HERA including STBs/TVsets, medical devices, ADSL routers, Bluetooth access points etc.

• Chapter 6 provides the details with respect to the SW technologies selection in terms of HMIs, application and web server, database, agent platform, communication technologies etc.

• Chapter 7 provides the specification of the HERA Application Server and its modules. • Chapter 8 defines the test and validation plan for both Greek and Austian pilot sites. • Chapter 9 draws conclusions.



1.3 METHODOLOGY The following figure depicts the methodology that was followed towards the definition of the HERA specifications and validation plan.

HERA Equipment

STB/Tvset selection

HERA Services

HERA AS Architecture specification

HERA Use Cases

HERA SW Technologies

HMIsWeb and

application server, Database

HERA equipment

HERA Specifications

HERA network architecture and

stakeholders

HERA AS modules

specification

Equipment for Bluetooth devices

bridging

Medical Devices selection

Application Server selection

Communication technologies

selectionAgent Platform

HERA Test and Validation Plan

HERA Deployment Plan

Services Selection

HERA Deployment and Validation

User AcceptanceSystem

deployment and validation

HERA services specification

Figure 1: Methodology followed for definition of the HERA specification and validation plan

Based on the user requirements and use cases defined in D21, the HERA services to be implemented were selected and specified in detail. Before specifying the overall network architecture it was necessary to identify and select the involved devices and equipment in the selected services. Additionally, the SW, communication and HMI technologies have also been selected. With this background information and the results of the INHOME and ASK-IT projects, the HERA network architecture and the application server architecture were specified. Having this information and specifications available, the test and validation plan was defined including the services deployment plan, the user acceptance test plan and the system validation plan.



2. HERA NETWORK ARCHITECTURE The following figure provides an overview of the HERA network architecture.

Figure 2: HERA Architecture Overview

It consists of 3 main sites including the home environment, the service provider premises and the medical center. It involves the following stakeholders:

• Elderly persons or users, which want to manage their daily activities in an easy manner, prolonging the time to be at home rather than in an institution and relieve them from visiting the specialists often. The main user categories of HERA are elderly people suffering from MCI or mild/moderate AD or other diseases (diabetes, cardiovascular) with identified risk factors.

• Health service providers (Hospitals, Specialised centers) including doctors, which need to be assisted in order to better monitor their patients and to be able to take care of a large number of patients.

• Service providers, which need to offer new products such as AAL services and expand their client base.

The main components of the HERA network architecture are described below and grouped in three main sites:



Home environment The home environment includes the following equipment. Equipment Product Interfaces TVset/STB - Austria ADB-3800W HDMI/SCART, RJ-45 TVset/STB – Greece ADB-3810-TW or Philips

NetTV HDMI/SCART, RJ-45

Blood Pressure device A&D UA-767PBT Bluetooth Weight scales device A&D UA-321PBT Bluetooth ADSL Router – Austria Thomson 585 v7 RJ-45 ADSL Router – Greece It will be provided by a Local

ISP RJ-45

Bluetooth Access Point Asus WL Bluetooth, RJ-45 Table 1: Home Environment Equipment

A connection diagram about the home environment equipment is depicted in the following figure.

Blue

toot

h en

able

d de

vice

s

Figure 3: HERA home environment

The home architecture shall be provided with 2 options to the elderly users: • Option 1: The Bluetooth Access Point close to the STB, so measurements shall be taken near

the STB. • Option 2: The Bluetooth Access Point in any other room as preferred by the user, so

measurements shall be taken in the preferred room. The home environment and the medical center are communicating with the service provider over the Public Internet. In cases that the HERA AS is part of an IPTV infrastructure, the communication between the home environment and the service provider is done over a private IP network.



Medical Center The medical center includes standard PCs with Internet connections. The medical center is communicating with the service provider over the Public Internet. Service Provider The service provider hosts the HERA AS and the corresponding database with user data. The home environment and the medical center are communicating with the service provider over the Public Internet. In cases that the HERA AS is part of an IPTV infrastructure, the communication between the home environment and the service provider is done over a private IP network.



3. SELECTION OF THE SERVICES TO BE IMPLEMENTED Towards selecting the HERA services to be implemented, a methodology has been defined consisting of the following steps: • Make a service prioritisation for implementation according to the following criteria:

o The ranking of the services according to the needs of the HERA user groups (D21). o The market needs and the exploitation potential of the services.

• Make the selection based on this prioritisation, taking into account the technological constraints for services implementations identified by the technology partners.

Towards services prioritization for implementation, the following priorities have been considered: • Priority “1”: Services considered to be implemented in very short term (immediately and

during the project course). • Priority “2”: Services considered to be implemented in short to medium term (after the end

of the project). • Priority “3”: Services considered to be implemented in medium to long term (after 3 years

counting from the end of the project). Based on the services needs defined in D21 (Table 14), an initial selection/filtering was made including the services that marked as “very important” from at least one user group. All the services that were not marked at least by one user group as “very important” were given priority “3”. As far as the ones marked as “very important” from at least one user group are concerned, the following priorities were given: • Priority “1” for the services that have high exploitation potential based on the market needs. • Priority “2” for the services that have not so high exploitation potential based on the market

needs. Based on this analysis, the following table has been prepared.



Service type Mild AD Moderate AD MCI Elderly Cardiovascular Diabetes Priority for service

implementation

Cognitive Reinforcement Very important Very important Very important Important Not so important Not so important 1

Physical Reinforcement Important Important Important Important important Very important 2

Chatting Not so important Irrelevant Not so important Very important Not so important Not so important 2

Sharing photos Not so important Irrelevant Not so important Very important Not so important Not so important 2

Getting support and advisory information from specialists Irrelevant Irrelevant Irrelevant Irrelevant Very important Very important 2

Maintaining a family calendar Very important Very important Very important Very important Not so important Not so important 2

Maintaining a phone book Important Important Very Important Very Important Not so important Not so important 2

Passive Communication (reality orientation) Very important Very important Very important Irrelevant Irrelevant Irrelevant 1

EGC monitoring Irrelevant Irrelevant Irrelevant Irrelevant Important Important 2

BP monitoring Irrelevant Irrelevant Irrelevant Irrelevant Very important Very important 1

Heart rate monitoring Irrelevant Irrelevant Irrelevant Irrelevant Very important Very important 2

Oxygen saturation monitoring Irrelevant Irrelevant Irrelevant Irrelevant Not so important Irrelevant 3

Body weight monitoring Irrelevant Irrelevant Irrelevant Irrelevant Very important Very important 1

Pill reminder Very important Very important Very important Very important Not so important Not so important 1

Other reminders Very important Very important Very important Very important Not so important Not so important 1

Alarm Irrelevant Irrelevant Irrelevant Important Very important Very important 2

Open pharmacies Not so important Not so important Not so important Not so important Not so important Not so important 3

Weather forecast Important Not so important Important Not so important Not so important Not so important 3

Simplified appliances manuals Not so important Irrelevant Not so important Not so important Not so important Not so important 3

Digital memory book Very important Very important Very important Irrelevant Irrelevant Irrelevant 3

Nutrition counseling Irrelevant Irrelevant Irrelevant Irrelevant Irrelevant Very important 1

Table 2: Consolidated services needs per user group and services prioritisation for implementation


HERA/SLNT /WP2/D22-final © HERA Consortium – July 2010

Based on an initial analysis made for the implementation of the priority “1” services, it came out that all priority “1” services can be implemented within the project limits. The following section provides the detailed specification of the selected services.



4. HERA SERVICES DETAILED SPECIFICATION The following generic service categories were defined, with a view to expand the services under each category in future activities, beyond HERA. • Games • Informational Services • Medical Measurements • Nutrition and Health • Organiser The detailed specification of the HERA services is provided in the following section

4.1 GAMES

4.1.1 COGNITIVE REINFORCEMENT SERVICE

4.1.1.1 SERVICE OVERVIEW This service is a web application based on a repository of cognitive exercises useful for the following HERA user groups:

o Mild AD > 65 years

o Moderate AD > 65 years

o MCI > 65 years The service includes a back-office application where doctors (or authorized medical personnel) can suggest mental games to the elderly. Thus, this service is related to the use cases 2.2 and 2.3 of D2.1. This service stakeholders are the service provider that hosts the service, the doctors and Medical personnel at a medical center that define what games are useful for each HERA user who reinforces his cognitive skills using the service (see Figure 4).



Figure 4: Cognitive reinforcement service architecture diagram and stakeholders

4.1.1.2 SERVICE CONFIGURATION System administrator / Service provider The system administrator belongs to the service provider. The system administrator registers the patient profile to the Application Server, which includes the information shown in Figure 5 for the HERAUser class. Moreover, he adds medical personnel including the information shown in Figure 5 for the Professional class. The system administrator provides professionals and HERA users with a login and password for service access. The system administrator tags available exercises (Exercise concept) as applicable to one or more of the HERA user groups. It has a specific type and a maximum difficulty level. Its type can be one of the following:

o Selective and Focused Attention

o Divided Attention and Attentional Switch

o Automatic and Voluntary Processing

o Sustained Attention and Vigilance

o Learning

o Implicit versus Explicit processes

o Working Memory

o Prospective Memory

o Executive functions

o Language



o Space-Time Orientation The class diagram depicted in Figure 5 shows the ontology related to this service. There are the Professional and HERAUser user types having information such as their contact information (address, telephones, etc), date of birth, name and surname. The Professional has a type property indicating whether he is a doctor, a nurse or administrative personnel and an organization property indicating the institution to which he/she is employed. The HERAUser includes a property for assigning the user’s group (userGroup). The Professional and the HERAUser both have access to their prescriptions (the former prescribes and the latter executes). The Prescription has the untilDate property that shows when this prescription terminates. More than one exercise may be prescribed to follow a specific schedule and for a specific difficulty level (if there exists such).). The Exercise has a type (see the list above), a maximum level, a name and a number of user groups to which it applies. Finally, the exerciseInvocationURL property shows the internet address for invoking the specific exercise. This address can be a link to a HERA defined service or a link to an external service.



-type : string-maxDifficultyLevel : int-name : string-applicableToUserGroups : string-exerciseInvocationURL : string

Exercise

ExercisesPrescription

-day_of_week : string-hour_of_day : int-difficultyLevel : int

ExerciseSchedule

-exercises1..*

-targetScore : stringCognitiveExercise

-name-surname-dateOfBirth : Date-fixTelephone : string-mobileTelephone : string-fax : string-email : string-web : string

Person

-streetAndNumber : string-postalCode : string-city : string-country : string-description : string

Address

-address

1

-userGroup : stringHERAUser

-userInformation 1

-myPatients

0..*

-attendingMedicalPersonnel

1..*-type : string-organization : string

Professional

-userInformation 1

-myPrescriptions

0..*

-subscribedBy 1

-myPrescriptions

0..*

-patient 1

-untilDate : DatePrescription

-exercise1

-dateTaken : Date-score : string

ExerciseScore

UserExercisesResults

-exercises0..*

-myExerciseResults 1

-patient 1

-exercise

1

Figure 5: The cognitive reinforcement service ontology

The service is deployed as a web application. It has two main components (see Figure 6), the user interface (UserInterface) and the service main module (CognReinfService). The interfaces



usage is explained in the following paragraph.

+getAvailableExercises(in usergroup : string)+initializeService(in username : string)+viewExerciseInstructions()+playExercise()+sendEmailToProfessionalWithResults()

CognReinfService

Figure 6: The cognitive reinforcement services interfaces

Doctors / Medical Center The HERA professional authenticates to the Application Server using his/her login and password. The HERA professional can enter/configure the patient data (for the patients under its supervision). A HERA professional can use the service back office to assign games to a user. He/she will be able to assign a subset of the five games with a difficulty factor. The HERA professional can change the games assignment whenever he wants, the user will find the new set of games the next day when he opens the service. The HERA professional will propose a specific time of day for the user to do his exercises. Patient / Home environment The service provider makes the appropriate equipment installations to the patient home environment. The patient should be provided with a username and password for being able to access the service through his STB/TVset.

4.1.1.3 SERVICE OPERATION The service includes two web interfaces for the HERA users and the professionals. Moreover, it includes a data base where the different exercise information is stored and also where the user scores are stored. The users play the assigned games by invoking the web interface with the right parameters and it can be only in a part of the browser screen allowing other menus of the HERA application to be visible. As soon as the game finishes the results are written for the specific user to the database. The service allows other modules (i.e. the intelligent module) to be informed about the user’s results through web services. In Figure 7 the reader can see the elaboration of the Use Case 2.2 in a sequence diagram. This diagram shows the medical expert using his HMI for selecting a user (making a relevant query to the HERA policy module). If he cannot find his user he can add a new one using the HERA MAS module. When he has selected the user for which he wants to add exercises he selects them among a list of relevant ones and then does his prescription. Note that the prescription is done through the MAS module because the agents need to be aware of the user’s schedule.



MedicalExpert CognReinfServiceMASModule

userSelectQuery(username)

addUser(new_username, password, usergroup, medical_pers_username)

newUserInsertQuery(username, password, usergroup, medical_responsible)

user creation results

If the username exists the user must select anotherand repeat the procedure.

{userExists==false}

getAvailableExercises(usergroup)

selectExercises()

prescribeExercises(patient_username, medical_pers_username, prescription)

newExercisePrescriptionInsertQuery(username, prescription, medical_responsible, sessionID)

prescription results

PolicyModule

Figure 7: Usage of the cognitive reinforcement service by a medical expert (Use Case 2.2)

In Figure 8 the reader can see the elaboration of the Use Case 2.3 in a sequence diagram. This diagram shows the HERA user using his HMI for cognitive training. The use case starts before the usage of the cognitive reinforcement service to show that the MAS module can inform the user that it is time to have his exercises and even remind him later. The cognitive reinforcement service starts when the HERA HMI either redirects the user to the cognitive reinforcement service URL or opens it in a window of the overall HERA application using the initializeService invocation. The service HMI allows the user to select among his prescribed exercises, view their instructions and play them. As soon as he is finished an email is sent to his medical responsible with the results and the results are also saved in the HERA database.



MASModule NotificationModule

displayMessage(message, possibleResponse, username, timeout)

userRespondsToDialog()

userAction(response, message, value, username)

sendEmailToRelative(message)

«precondition»{(user response==null || timeout exceeded) && hasPassed(one_day)}

MCI_MildAD_OR_ModerateAD_User

showDialogToUser()

user response

CognReinfService

displayMessage

«precondition»{(user response==null || timeout exceeded) && hasPassed(one_hour)} userRespondsToDialog()

showDialogToUser()

user response

userAction(response, message, value, username) initializeService(username)

PolicyModule

userExercisesSelectQuery(username, sessionID)

viewExerciseInstructions()

playExercise()

userExerciseResultsInsertQuery(username, sessionID, exerciseResults)

user continues by playing all available exercises and, optionally, viewing their instructions.

user quit exercises

userFinishedCognReinfService(username)

userExerciseResultsSelectQuery(username)

sendEmailToProfessionalWithResults()

sendEmailToRelative(message)

«precondition»{user did not finish all exercises && hasPassed(one_day)}

displayMessage

«precondition»{user did not finish all exercises && hasPassed(one_hour)} userRespondsToDialog

showDialogToUser

user response

userAction(response, message, value, username)

Figure 8: Usage of the cognitive reinforcement service by a HERA end user (Use Case 2.3)

The following games have been selected for implementation: Game Title Remember The Story? Congitive skill (s) reinforcement

Attention and Memory (Learning)

Description The user is asked to carefully watch the video presented. Then, a) he has to answer multiple choice questions on the story, b) is shown various objects/faces and asked to identify the objects/faces presented in the video and c) is asked to give an end to the story by choosing between two different options.



Game Title Hide And Seek? Congitive skill (s) reinforcement

Attention and Memory (Learning)

Description The user is shown a fully-furnished and fully-decorated room (e.g. living room). The game shows valuable objects (e.g. money, watch, granddaughter’s present, car keys etc) one by one and reveals were the owner of the house have hidden them before leaving for summer vacations. After a 5 and a 15 minutes delay, the user is asked to recall where each valuable object has been hidden.

Game Title Real Life Shopping Cart Congitive skill (s) reinforcement

Executive functions

Description The system presents the elderly with an amount of euro (e.g. 100) and a rich set of items found in supermarkets (in pictures). Each item has a price associated with it, which corresponds to a realistic market price. The elderly must fill-in a shopping cart with the appropriate items so as to reach a specific goal (e.g. prepare a birthday party for his grandson) without exceeding the given amount of money.

Game Title Find The Differences Congitive skill (s) reinforcement

Attention

Description The user is asked to find the differences in two apparently identical images. Game Title Hangman Congitive skill (s) reinforcement

Language

Description Hangman is a paper and pencil guessing game for two or more players. One player thinks of a word and the other tries to guess it by suggesting letters. The word to guess is represented by a row of dashes, giving the number of letters. If the guessing player suggests a letter which occurs in the word, the other player writes it in all its correct positions. If the suggested letter does not occur in the word, the other player draws one element of the hangman diagram as a tally mark. The game is over when:

• The guessing player completes the word, or guesses the whole word correctly

• The other player completes the diagram: This diagram is, in fact, designed to look like a hanging man. Although debates have arisen about the questionable taste of this picture, it is still in use today. The players draw the gallows before play and draw parts of the man's body (traditionally the head, then the torso, then the left arm, then the right arm, then the left leg, then the right leg). The amount of detail on the man can vary, affecting the number of chances (difficulty level).



4.2 INFORMATIONAL SERVICES

4.2.1 PASSIVE COMMUNICATION SERVICE

4.2.1.1 SERVICE OVERVIEW This service is a passive communication web application based on a repository of information, which realizes HERA use case 2.8, which involves the following HERA user groups:

o Mild AD > 65 years

o Moderate AD > 65 years

o MCI > 65 years

o Elderly – persons > 65 years This information can include helpful videos about the management of their disease, their age or changes in everyday life, like the possibility to pay a bill using home banking, etc. It may also be important information on how to handle a local hazard in a specific area. The user will have the possibility to enable or disable this feature by editing his/her profile. The default choice will be to receive the information. This service stakeholders are the service provider that streams video to the HERA user, the doctors and Medical personnel at a medical center that create helpful videos for the HERA users and the HERA user who can select the type of videos that he prefers to watch (see Figure 9).

Figure 9: Passive Communication service architecture diagram and stakeholders

4.2.1.2 SERVICE CONFIGURATION



System administrator / Service provider The system administrator belongs to the service provider. The system administrator registers the patient profile to the Application Server, which includes the information shown in Figure 10.


Person


Address

-address

1


-userInformation 1

Figure 10: The HERA user data for the reality reinforcement service in UML class diagram

format.

The system administrator provides professionals and HERA users with a login and password for service access. The service is deployed as a web application and is just a video cast. It also supports a web service for giving information about existing videos. In Figure 11, the two interfaces are shown, the first (getAvailablePushInformation is an HTTP SOAP service, while the showInformation is an HTTP service).

+getAvailablePushInformation(in userGroup : string, in effectiveDateFrom : Date, in effectiveDateTo : Date, in forArea : Address)+showInformation(in informationID : int)

PushInformationService

Figure 11: Push information interfaces

Doctors / Medical Center The HERA professional authenticates to the Application Server using his/her login and password. The Medical personnel creates videos and tags them as applicable

1. to one or more of the user groups, 2. to a specific physical area (country, city, or even neighborhood, see the Address concept)

and also The videos will belong to two (2) categories:

1. General information videos (they are inserted in the system by a doctor and can be shown at any time to any user group). These never expire.

2. Videos related to the current news (they are inserted in the system by a doctor and can be shown to any user group, however they expire after one week from their upload)

Whenever the user has opened his TV and the day is Sunday and the hour 10 a.m. or after and there are available videos that the user has not watched then he will be prompted to watch one



using the following algorithm: If there are videos from the category two (2) that have not expired and that the user has not watched, then the user is prompted to watch the more recent one among them. Else if there are videos from the category one (1) that the user has not watched, then the user is prompted to watch one of them at random. The class diagram in Figure 28 shows the ontology related to this service with the PushInformation main concept. The applicable season is defined using the applicableDateFrom and applicableDayTo properties. The videoFile property can be used as a link to a URL or to a file in the file system. Thus, this service needs a database infrastructure in which to store the information for the available videos, and also a link to the video files. Finally, using the applicableArea association the area can be defined by an instantiation of the Address concept, by assigning values only to the fields needed. For example, if a piece of information is relevant to a whole country only the country field (field or property can both be used to describe the attributes of a concept) may be given a value. If the information is relevant only to a specific city then aside from the country field, the city field must be defined as well. If the event is relevant only to a specific week then the applicableDateFrom and applicableDateTo are defined as the starting day and the ending day respectively.

Figure 12: The Information push service ontology

Patient / Home environment The service provider makes the appropriate equipment installations to the patient home environment. The patient should be provided with a username and password for being able to view the videos through his STB/TVset.

4.2.1.3 SERVICE OPERATION All videos have IDs with which the service can be invoked from a browser and show the video. The calling application (HERA) can select to show this video in a specific area of the browser window. The service also supplies as a web service its list of videos with their meta-tags to any requester application. The videos are annotated with the following information:

• Which geographical area they apply to. This can be as general as be relevant to a whole country or can be further refined to a specific city or address (see the applicableArea association of the class PushInformation whose fields are all optional)



• The date since which they are applicable (applicableDateFrom field) and the date until which they are valid (applicableDateTo field)

• The HERA user groups for which this information is useful Figure 13 shows how the service operates. The MAS module can ask for available information and select to present it to a specific user through the notification module. The latter must call the showInformation service. Actually, the service will be initialized either by redirecting the user to the push information service site, or by showing the content in a part of the HERA application window (this seems the better solution as, for example, the user may be able to use other HERA menus and also retain his session).

Figure 13: Push information service operation (Use Case 2.8)

The user’s experience will be the following: A new video will be available every Sunday morning at 10.00 am. When the user is at home using the HERA system:

1. The system makes a specific audible sound to alert the user 2. A message box indicating that a new video is available will be presented on the screen,

accompanied by a short description of the story. 3. The user presses the “watch it now” button and watches the video. The message box

disappears from the screen. If the user fails to respond to the system message, the message box remains on the screen. If the user presses the “watch it later” button. The message box disappears from the screen and appears again at 8.00 pm. If the user presses the “this video is not of interest to me” button the message box disappears from the screen and no further action is done by the system regarding this video. Another functionality of the passive communication service is that based on the user’s type, i.e. a) Mild AD > 65 years, b) Moderate AD > 65 years and c) MCI > 65 years the date, time and weather report will appear on the TV screen occasionally. The frequency of the appearances will be fine tuned during the first service testing period. The medical personnel can choose to individually set the frequency of these appearances for each HERA user.



4.3 MEDICAL MEASUREMENTS

4.3.1 BLOOD PRESSURE MONITORING SERVICE

4.3.1.1 SERVICE OVERVIEW This service provides the capability to monitor the blood pressure of patients suffering from cardiac diseases, under the supervision of their doctor. The patients are capable to send blood pressure measurements from their home environment over the Internet to an Application Server, which belongs to a service provider, using appropriate medical devices. The Application Server can provide notifications to a Medical Center and to the patient on its STB/TVset, in case of patient’s critical or emergency situations. Doctors in the Medical Center can view patient’s profile and blood pressure measurements history through a standard PC. Patients can also view their blood pressure measurements history through an STB/TVset. The following figure shows an architecture diagram depicting also the stakeholders of this service.

Figure 14: Blood pressure monitoring architecture diagram and stakeholders

The following sections provide in detail the required service configuration and the service operation.

4.3.1.2 SERVICE CONFIGURATION System administrator / Service provider The system administrator belongs to the service provider. The system administrator registers the patient profile to the Application Server, which includes the following information: • ID • Name • Age



• Address • Diseases • Allergies • Relative: (name, telephone, reach ability) • Attending physician: (name, reach ability) The system administrator associates the patient with a medical device Identifier (ID) and provides him/her with a Personal Identification Number (PIN) for service access through the STB/TVset. The system administrator also registers the doctor profiles to the Application Server, which includes the following information: • ID • Name • Name of organization • Area of expertise The system administrator provides doctors with a login and pswd for service access through a standard PC. Doctors / Medical Center The doctor authenticates to the Application Server using its login and password. The doctor enters/configures the patient data (for the patients under its supervision). The doctor enters the details about the standard patient data as described above (Diseases, Allergies, Relative, Attending physician). The doctor also configures the following details for the patient: • the frequency and time for the measurements taking (measurement should take place once to

three times a day). • the time limit for the patient to be notified for a measurement that has not been taken on

time. • the optimal upper and lower limits of the blood pressure. • the emergency limits of the blood pressure i.e when a notification to the patient and the

central care center has to take place. Patient / Home environment The service provider makes the appropriate equipment installations to the patient home environment. The patient should be provided with its own blood pressure medical device by the service provider. The patient should be provided with a PIN for being able to view his blood pressure measurements history through its STB/TVset. Emergency Center The emergency center is not directly connected with the service. The medical center or the patients or caregivers are responsible to contact (perhaps with a normal phone call) the emergency center in case of emergency situations, thus it does not require any particular configuration.

4.3.1.3 SERVICE OPERATION Measurement taking The patient may take the blood pressure measurement from any place/room inside the home environment. An arm type blood pressure device is used. The patient puts the blood pressure device on his/her arm and takes a measurement. The measurement is securely transmitted to the



Application Server over the Public Internet. The measurements includes the following information: • systolic blood pressure (mmHg) • diastolic blood pressure (mmHg) • pulse rate The patient is identified through the medical device ID, which is transmitted together with the measurement to the Application Server. Measurement analysis by the Application Server and notifications The Application Server analyses the measurements received for all patients, according to the data entered by the doctors to the system. The Application Server sends notifications to the medical center and the patients when required. Notifications to the medical center (doctors) are provided in a form of pop-up windows on a standard PC. Notifications to the patients are provided in a form of pop-up windows on their STB/TVset. Notifications remain on the PC or STB/TVset screen until confirmed by the doctor or the patient and do not disappear automatically. If a measurement has not been taken as scheduled, a notification is sent to the patient according the time limit scheduled by the doctor. The notification text should be “You forgot to measure your blood pressure”. The patient does not take as feedback a notification for informatory reasons immediately after each measurement. If no measurement has been entered into the system for 1 day (patient forgot to take his measurements or is out of his house or due to a system error), a notification is sent to the health center. The notification text should be “No measurements were taken between xx and xx”. If a measurement received by the Application Server is not valid (corrupted data), a notification is not sent to the patient. If a measurement is valid (not corrupted), the Application Server operates as follows: • If measurement data are critical (out of upper and lower limits), the patient is notified to

repeat the measurement after 3 minutes. The notification text should be “Please repeat measurement”.

• If measurement data are critical (out of upper and lower limits) for a second time, a notification is sent to the patient recommending to consult its medical expert. The notification text should be “Your blood pressure is out of normal limits - if you feel insecure please consult your medical expert”.

• If measurement data are out of emergency limits, the patient is notified to repeat the measurement after 3 minutes. The notification text should be “Please repeat measurement”.

• If measurement data are out of emergency limits for a second time, a notification is sent to the medical center recommending to contact the emergency center for this patient. A notification is also sent to the patient recommending to contact the emergency center. The notification text should be “Your blood pressure is out of normal limits – if you feel insecure please contact the emergency center”.

Measurements history viewing The patient can view the measurements history through its STB/TVset. The patient authenticates to the application server through the STB/TVset by entering its PIN using the remote control. The patient views (in both tabular and graphical formats) the blood pressure measurements trend over the last day, week or month including the following information:



• date • hour • systolic blood pressure • diastolic blood pressure • pulse rate The patient can also view average values of the measurements per month for the past 12 months. The doctor can view the measurements history for its patients through a standard PC. The doctor authenticates to the application server through the PC by entering its login and password. The doctor views (in both tabular and graphical formats) the patient profile and the blood pressure measurements trend over the last day, week or month including the following information: • date • hour • systolic blood pressure, • diastolic blood pressure • pulse rate The doctor can also view average values of the patient measurements per month for the past 12 months. The doctor can view the generated notifications (if any) for each patient. All measured data shall be stored for 1 month, and then they should be archived. However average values per month shall be available.



4.3.2 BODY WEIGHT MONITORING SERVICE

4.3.2.1 SERVICE OVERVIEW This service provides the capability to monitor the weight of patients suffering from diabetes type 2, high blood pressure and/or other cardiac diseases or overweight, under the supervision of their doctor. The patients are capable to send weight measurements from their home environment over the Internet to an Application Server, which belongs to a service provider, using appropriate medical device (scales). Doctors in the Medical Center can view patient’s profile and weight measurements history through a standard PC. Patients can also view their weight measurements history through an STB/TVset. This service shall be able to be also used without the involvement of doctors, which means that the users or relatives shall be provided with access to make the configuration by themselves. The following figure shows an architecture diagram depicting also the stakeholders of this service.

Public Internet

System Administrator /Service Provider

HERA AS

Database

Doctors / Medical Center

Patients / Home environment

STB/TVset

Medical Device

Figure 15: Weight monitoring architecture diagram and stakeholders


4.3.2.2 SERVICE CONFIGURATION System administrator / Service provider The system administrator belongs to the service provider. The system administrator registers the patient profile to the Application Server, which includes the following information: • ID • Name • Age



• Address • Diseases • Allergies • Relative: (name, telephone, reach ability) • Attending physician: (name, reach ability) The system administrator associates the patient with a medical device Identifier (ID) and provides him/her with a Personal Identification Number (PIN) for service access through the STB/TVset. The system administrator also registers the doctor profiles to the Application Server, which includes the following information: • ID • Name • Name of organization • Area of expertise The system administrator provides doctors with a login and pswd for service access through a standard PC. Doctors / Medical Center The doctor authenticates to the Application Server using its login and password. The doctor enters/configures the patient data (for the patients under its supervision). The doctor enters the details about the standard patient data as described above (Diseases, Allergies, Relative, Attending physician). The doctor also configures the following details for the patient: • the frequency and time for the measurements taking (measurement should take place once a

day in the morning). • the time limit for the patient to be notified for a measurement that has not been taken. • the current weight value of the patient and the goals for weight reduction on weekly and

monthly basis. The margins (+ x Kgr) of the goals’ values are also configured. Patient / Home environment The service provider makes the appropriate equipment installations to the patient home environment. The patient should be provided with its own weight scales device by the service provider. The patient should be provided with a PIN for being able to view his weight measurements history through its STB/TVset.

4.3.2.3 SERVICE OPERATION Measurement taking The patient takes the weight measurements close to the Bluetooth Access Point. The patient gets on the weight scales device and takes a measurement. The measurement is securely transmitted to the Application Server over the Public Internet. The measurements include a weight measurement value in Kgr. The patient is identified through the medical device ID, which is transmitted together with the measurement to the Application Server. Measurement analysis by the Application Server and notifications The Application Server analyses the measurements received for all patients, according to the data entered by the doctors to the system. The Application Server sends notifications to the patients when required. Notifications to the



patients are provided in a form of pop-up windows on their STB/TVset. Notifications remain on the PC or STB/TVset screen until confirmed by the patient and do not disappear automatically. If a measurement has not been taken as scheduled, a notification is sent to the patient according the time limit scheduled by the doctor. The notification text should be “You forgot to measure your weight”. The patient does not take as feedback a notification for informatory reasons immediately after each measurement. If a measurement received by the Application Server is not valid (corrupted data), a notification is not sent to the patient. If a measurement is valid (not corrupted), the Application Server operates as follows: • If measurement data are dated as a weekly goal is dated, then:

o if the measured weight value is above the weekly goal value plus the defined margin, the patient is notified that the weekly goal has not been reached and consultation of the medical expert is recommended. The notification text should be “You did not reach your weekly goal of weight reduction, please contact your doctor” or in other case “You did not reach your weekly goal of weight reduction”.

o if the measured weight value is below the weekly goal value plus the defined margin, the patient is notified that the weekly goal has been reached. The notification text should be “You reached the goal of weight reduction”.

• If measurement data are dated as a monthly goal is dated, then: o if the measured weight value is above the monthly goal value plus the defined

margin, the patient is notified that the monthly goal has not been reached and consultation of the medical expert is recommended. The notification text should be “You did not reach your monthly goal of weight reduction, please contact your doctor” or in other case “You did not reach your monthly goal of weight reduction”.

o if the measured weight value is equal or below the monthly goal value plus the defined margin, the patient is notified that the monthly goal has been reached. The notification text should be “You reached the goal of weight reduction”.

Measurements history viewing The patient can view (in both tabular and graphical formats) the measurements history through its STB/TVset. The patient authenticates to the application server through the STB/TVset by entering its PIN using the remote control. The patient views the goals defined by the doctor and the weight measurements trend over the last day, week or month including the following information: • date • hour • weight measurement The patient can also view (in both tabular and graphical formats) the latest valid weight measurement per month for the past 12 months. The doctor can view the measurements history for its patients through a standard PC. The doctor authenticates to the application server through the PC by entering its login and password. The doctor views the patient profile, the defined goals and the weight measurements trend over the last day, week or month including the following information: • date • hour



• weight measurement The doctor can also view the latest valid weight measurement per month for the past 12 months.



4.4 NUTRITION AND HEALTH

4.4.1 NUTRITION COUNSELING SERVICE

4.4.1.1 SERVICE OVERVIEW This service provides nutrition counseling to patients suffering from diabetes type 2 and/or have overweight or in general people that would like to have a nutrition tracking service. The service is used without the involvement of doctors. The following figure shows an architecture diagram depicting also the stakeholders of this service.

Public Internet

System Administrator /Service Provider

HERA AS

Database

Patients / Home environment

STB/TVset

Figure 16: Nutrition counseling architecture diagram and stakeholders


4.4.1.2 SERVICE CONFIGURATION System administrator / Service provider The system administrator belongs to the service provider. The system administrator registers the patient profile to the Application Server, which includes the following information: • ID • Name • Age • Address The system administrator provides him/her with a Personal Identification Number (PIN) for service access through the STB/TVset. The system administrator also maintains in the system database a variety of nutrition groups as well as ingredients, quantities and daily portions belonging to each group, as follows: • Drinks 1,5 l per day (one glass contains 250 ml), which corresponds to 6 glasses (portions):

o Glass of mineral water o Unsweetened fruit or herbal tea o Thinned vegetable juice or fruit juice

3-4 cups of coffee or 3-4 cups of black tea should not be counted, but are allowed. • Vegetables, legumes and fruits, 5 portions per day, which corresponds to 3 portions of



vegetables and /or legumes and 2 portions of fruits per day and include: o 200-300 gr refined vegetable o 100-200 gr uncooked raw vegetables o 75-100 gr salad o 150-200 gr legumes cooked (raw about 70-100 gr) o 125-150 gr fruit o 250ml vegetable or fruit juice o Etc

• Grain and potatoes, 4 portions per day, which may include: o 50-70 gr bread and pastry o 50-60 gr of musli or grain flakes o 200-250 gr cooked pasta (raw 65-80 gr) o 150-180 gr cooked rice or grain (raw 50-60 gr) o 3 – 4 middle sized potatoes (cooked 200-250 gr) o Etc

• Milk and milk products, 3 portions per day, which may include: o 200 ml milk o 180-250 gr yoghurt o 200 gr quark o 200 gr cottage cheese o 50-60 gr cheese

• Fish, Meat and Eggs, 1 portion per day, which may include: o 150 gr fish (one max. two portions per week are ok). o Fatless meat or fatless sausage (max. 300- 400gr) (max. 3 portions per week are ok) o Egg (3 eggs per week are ok)

• Fats and oils, 1 portion per day, which include: o 1-2 soup spoons oils

• Fat, sweet and salty, maximum 1 portion per day. The system administrator is responsible for the ingredients database maintenance. Patient / Home environment The patient should be provided with a PIN for being able to get nutrition counseling through its STB/TVset.

4.4.1.3 SERVICE OPERATION The patient authenticates to the application server through the STB/TVset by entering its PIN using the remote control. The patient can create a new session (when he/she first accesses the service for the day), continue a previous session (which corresponds to the current day’s session) or close the current day’s session. The patient can also view the history of the closed sessions for each day for the last 30 days (maximum). The patient views all possible groups, ingredients and quantities as well as the allowed portions



per group. The patient can mark the ingredients that he/she has consumed and a nutrition pyramid is started to be built or updated accordingly. In the best case – at the end of the day – a full pyramid is constructed, similar to the one depicted in the following figure.

Figure 17: Nutrition counselling pyramid

Each nutrition group corresponds to a different pyramid level, starting from Level 1, which corresponds to “Drinks” until Level 7, which corresponds to “Fat, sweet and salty”. If the patient has consumed less, the pyramid is not fully contracted. If the patient has consumed more ingredients belonging to several nutrition groups, the corresponding block of the pyramid is highlighted and a textual clarification is provided informing the patient about his/her overconsumption, for instance “You have consumed more in ‘Milk and milk products’ nutrition group”.



4.5 ORGANISER

4.5.1 REMINDERS SERVICE

4.5.1.1 SERVICE OVERVIEW This service provides the capability to remind patients about certain activities that they have to perform including for instance reminders for getting their medication as scheduled. Doctors in the Medical Center can configure patient reminders through a standard PC. Patients can view these reminders as notifications on the STB/TVset. This service shall be able to be used without the involvement of doctors. The following figure shows an architecture diagram depicting also the stakeholders of this service.

Figure 18: Reminders architecture diagram and stakeholders


4.5.1.2 SERVICE CONFIGURATION System administrator / Service provider The system administrator belongs to the service provider. The system administrator registers the patient profile to the Application Server, which includes the following information: • ID • Name • Age • Address



• Diseases • Allergies • Relative: (name, telephone, reach ability) • Attending physician: (name, reach ability) The system administrator also registers the doctor profiles to the Application Server, which includes the following information: • ID • Name • Name of organization • Area of expertise The system administrator provides doctors with a login and pswd for service access through a standard PC. Doctors / Medical Center The doctor authenticates to the Application Server using its login and password. The doctor enters/configures the patient data (for the patients under its supervision). The doctor enters the details about the standard patient data as described above (Diseases, Allergies, Relative, Attending physician). The doctor also creates, configures and saves the following type of reminders for the patient: • Reminders triggered on specific dates: The doctor may specify a specific date and time(s) for

the reminder. In this case the reminder is triggered only once on that specific date and time(s).

• Reminders triggered on a weekly schedule: The doctor may specify a specific time(s) and the days of the week for the reminder. In this case the reminder is triggered on the selected days of the week and on the specific time(s).

• Reminders triggered on daily basis: The doctor may specify a specific time(s) for the reminder. In this case the reminder is triggered on daily basis on that specific time(s).

The doctor also enters the reminders text that should be displayed on the patient STB/TVset screen. The value of this service for medication reminder is mainly to support users (elderly) that leave on their own or in cases that there is a caregiver not to be necessary for users’ compliance with the drug therapy. The degree of intervention from a caregiver would depend on the needs of the user. The notification text should be “Take your <morning> <afternoon> <night> pills”. The user could have a pill box organized with the respective pills for everyday. The pill box could be refilled and checked every week from the user’s caregiver, relative or a nurse that visits the user for this purpose.

All reminders have a Priority which can be selected by the doctor. The following priorities can be selected: • Low priority reminder: By selecting this type of priority, a notification message in displayed

on the STB/TVset screen.

• High priority reminder: By selecting this type of priority, a notification message is displayed on the STB/ TVset screen and a beeper sound is generated.

Another possibility for the doctors is to assign the pills before or after a specific meal (breakfast, lunch, dinner). In that case the time when the patient takes his meals must be recorded and the reminder must be set some minutes before or after the meal. Especially for the MCI and AD



patients the doctors may need to assign specific colors to the boxes of pills that they take (one box color for breakfast, one for lunch and one for dinner). The patient must be reminded to take the specific box color. Finally, the doctors may need to assign some specific cases for specific patients. This will be related to the pills for blood pressure conditioning. Thus, the doctor may dictate that if the blood pressure is measured lower or higher than a specific threshold two times in a row or for two or three times in the same day then the pill dosage must be moderated.

4.5.1.3 SERVICE OPERATION The Application Server analyses the reminders data for all patients, according to the data entered by the doctors to the system. If a reminder is triggered, the Application Server sends notifications to the patients. Notifications to the patients are provided in a form of pop-up windows on their STB/TVset. Notifications remain on the STB/TVset screen until confirmed by the patient and do not disappear automatically. If a reminder is marked as “high priority”, a beeper sound may accompany the notification.



5. EQUIPMENT SELECTION AND SUPPORTED FEATURES

5.1 SET-TOP-BOX/TVSET SELECTION AND SUPPORTED FEATURES In order to achieve user acceptance and usability HERA services must deploy mainstream technologies and simple intuitive interfaces that even elderly people are well familiarized with. This is possible with the use of TV sets (through Set-Top-Boxes), which are among the most commonly used technologies, present in almost all houses and most of the elderly users are familiar with, not other complicated devices. In order to select the appropriate IPTV STB we had to set some basic requirements/criteria. To start with the STB has to provide some basic interfaces, in order to connect to the TV set (HDMI, SCART) and the internet (RJ-45). Moreover the STB has to offer a DVB interface. This was deemed necessary due to the fact that in Greece digital television switchover is at its final phase and widely advertised while there is no IPTV provider from Greece participating in the HERA consortium. Hence a common STB could be used both for DVB decoding and also as a client machine for the HERA services. Keeping in mind that the user services will NOT be accessed from the traditional PC domain, but rather from the TV domain, new technologies that cross different domains are required. Several major companies within these different domains have decided to work together on these issues. One of the results is a framework for remote user interfaces for both UPnP networks and the Internet. This framework is called Web4CE (a.k.a. CEA-2014) [9] and has been accepted as the baseline remote user interface technology within the Digital Living Network Alliance (DLNA) [10], which is a large industry-wide effort for creating true interoperability between network-enabled devices. The HERA application relies on stimuli from the client side, with the form of medical data being sent to the application server and responses from the HERA application server. These responses include reporting of abnormal or out-of-boundaries measured medical data. As such they have to be immediately reported to the user of the service. This is done via pop-up windows that interrupt what the user is currently viewing on the TV set. This mechanism will also be used to support the “Reminders Service”. Moreover, in case the user has the STB in stand-by-mode we need a mechanism to relay important information from the server to the client. It is evident that stand-by-mode activation must be supported by the STB. Due to the nature of the transmitted data, we need to establish a secure channel over an insecure network. This is achieved through the usage of HTTPS (HTTP over SSL or HTTP Secure), that provides adequate protection against eavesdropping and man-in-the-middle attacks. All these are summarized bellow:

• Web4-CE compliance • Interfaces (USB, HDMI, SCART, RJ-45) • DVB-T option • Pop-up windows • Stand-by-mode activation • HTTPS • JavaScript 1.5

Regarding the Austrian pilot site, that will use the AonTV platform by Telekom Austria, the AonTV commercial STB will be used. The device is the one provided by Advanced Digital



Broadcast (ADB), namely ADB-3800W, shown bellow.

Figure 19: Austria Telecom ADB-3800W

The ADB-3800W is an advanced, interactive, digital set-top box optimized for IP-based telecommunications networks. The product is compatible with high definition (HD) and standard definition (SD) television transmissions and is compatible with the industry’s leading Conditional Access (CA), Digital Rights Management (DRM) and middleware technologies. The product has an advanced, single chip microprocessor, providing performance enhancements such as fast channel change and swifter rendering of applications, including the Electronic Programme Guide. The unit is compatible with HD and SD television transmissions across all industry standard compressions, including MPEG-2, MPEG-4/H.264 and VC-1 Advanced Video Coding (AVC). ADB’s advanced and state-of-the-art HD technology, coupled with a High Definition Multimedia Interface™ (HDMI™), ensure crystal clear viewing. The advanced, on-screen navigation of the ADB-3800W can be achieved by utilizing both MHP or JavaScript applications, and a dedicated remote control unit allows complete control of the user interface. Its datasheet can be found at: http://www.adbglobal.com/files/ADB_datasheet_3800W_HD_USqtrfinal_0.pdf A market survey, using sources from web and Telekom Austria relative input, resulted in an extensive list of candidate IPTV STBs (found in ANNEX A - Candidate Set-Top-Boxes). However, in order to assist with the selection of a product, the basic requirements/criteria presented before were augmented with non-technical requirements, such as: communication support, minimum order requirements, EU reseller/technical support, resulting in the table bellow:



IP-TV STB

Requirements

Product 3800W

3810TW

SceneGate 8800

Mood400-020/22

STMC-XL

N8000

N7000

Fontana Model

T 502

Web4-CE compliance

Interfaces (USB, HDMI, SCART, RJ-45)

SDK

DVB-T option

Pop-up windows

Stand-by-mode activation

JavaScript API

HTTPS

JavaScript 1.5

Table 3: Candidate HERA IPTV STB

As shown in the table both the ADB and the Telergy products comply with all requirements. However, due to the fact that the ADB IPTV STB is used by Telecom Austria users we have initially selected ADB-3810-TW for the Greek Trial, which also includes an additional front-end DVB-T decoder. Its datasheet can be found at http://www.adbglobal.com/files/ADB_datasheet_3800_10TW.pdf . The reason for that is that there is no IPTV provider from Greece participating in the HERA consortium, which would allow the integration of HERA AS in its IPTV platform. Thus, ADB-3810-TW with DVB-T decoder was initially decided to be used. During the next period extensive experimentation and early prototyping will be performed in order to assess its feasibility for services development. On the other hand, the latest trend in TV manufacturers is the Internet-enabled TV. Various companies have already released their products including:

Philips Net TV Sony Internet TV Samsung Internet@TV Panasonic Viera Cast

Internet-enabled TVs constitute a strong candidate in order to be used as a terminal instead of an STB in the Greek Trial, with the most candidate one the Philips NetTV due to synergies with other projects (AAL-Call-2 HOMEdotOLD project). The final decision on the user terminal to be used in the Greek Trial will be made in September 2010 after the experimentation and early prototyping with both ADB-2810-TW and Philips NetTV, by always respecting the compatibility with Web4CE standards in both platforms.



5.2 MEDICAL DEVICES SELECTION AND INTERFACES The selection of the medical devices to be used within the HERA platform is imposed by the selected services. Since these services include:

a) Monitoring of the blood pressure of patients suffering from cardiac diseases and b) Monitoring of the weight of patients suffering from diabetes type 2, high blood pressure

or overweight This results in two medical devices:

1. A digital blood pressure monitor and 2. A digital weight scale.

Keeping in mind that these devices should present a wireless connectivity option, our search resulted in the following options, as provided in the following tables. Company Product Special Characteristics Connectivity Certification Battery Life

UA-767PBT-C (Upper arm blood pressure monitor)

Higher compatibility with various Bluetooth receivers Irregular Heart Beat Indicator One-touch measurement

Bluetooth Class 1

CE03661 CE06782 IC: 4250A-WMLC403 FCC

~300 measurements

UA-767PBT (Upper arm blood pressure monitor)

128bit encryption of data for protecting patients' privacy Higher compatibility with various Bluetooth receivers Irregular Heart Beat Indicator One-touch measurement

Bluetooth Class 1

CE0366 CE0678 IC: 4250A-WMLC40 FCC

~300 measurements

Company Product Special Characteristics Connectivity Certification Battery Life

708-BT (Upper arm blood pressure monitor)

IEEE11073-10407 Data transmission : Measurement date/time, measurement values, model, and serial number

Bluetooth Class 2

Continua ~300 measurements


Boso-medicus prestige

Irregular Heart Beat Indicator One-touch measurement

Bluetooth Class 2

~300 measurements


UA-851 THW Irregular Heart Beat Indicator One-touch measurement

Proprietary protocol

EN 1060-14

EN 1060-35

~300 measurements

1 CE0366: European Directive 93/42 EEC for Medical Products. This is made evident by the mark of conformity. (0366: The reference number to the involved notified body) 2 The device complies with the statutory EMC (Electromagnetic Compatibility) directive 89/336/EEC. The WML-40AH is approved in accordance to R&TTE directive transmitter module marked by CE0678, manufactured by MITSUMI incorporated to OEM product. 3 IC: 4250A-WMLC40: Compliance with Industry Canada. 4 EN 1060-1: Non-invasive sphygmomanometers – Part 1: General requirements 5 EN 1060-3: Non-invasive sphygmomanometers – Part 3: Supplementary requirements for electro-mechanical blood pressure measuring systems



Table 4 Candidate HERA digital blood pressure monitor


UC-321PBT-C Precision Personal Health Scale series

Bluetooth Class 1

89/336/EEC, CE0678

~ 1,000 measurements

UC-321PBT Precision Personal Health Scale series

Bluetooth Class 1

89/336/EEC, CE0678



BF-206BT Precision Scale Bluetooth Class 2

IEEE11073-10415


UC-324THW

Complete record of your weight and BMI

Proprietary protocol



BC-590BT Weight, Body fat %, Body water %, Muscle mass, Physique rating, Daily Caloric Intake, Metabolic age, Bone mass, Visceral fat, 4 users

Bluetooth Class 2

BC-502 Bluetooth Class 2

HD-351 BT Bluetooth Class 2


WiFi ~ 500

measurements

Table 5: Candidate HERA weight scale

At the time of the survey information regarding costs, was not possible to obtain. As seen in the above tables, there are more than one wireless connectivity options. However, the solution from LifeSource was not accepted since they use a proprietary protocol that does not meet our requirements in terms of security. This left us with Bluetooth and WiFi connectivity options. The Withings solution was also dropped since their product cannot be configured to transmit measured data to any application server. Instead data are being sent and stored in the Withings application server whereby a specific API can be used for data extraction, so that any other application can manipulate them. Consequently the only wireless connectivity left was Bluetooth. Here, we had two possibilities: either Class I or Class II, with their difference being in their effective range, the former being at 100m while the latter at 10m. A selection for Class I Bluetooth connectivity was deemed appropriate resulting in the selected devices being the A&D UA-767PBT blood pressure monitor and A&D UA-321PBT weight scale that have also been approved both by HYGEIA and ROTES KREUZ. A short description on these devices is presented in the 5.2.1 and 5.2.2 .



5.2.1 A&D BLOOD PRESSURE MONITOR The selected blood pressure monitor is the UA-767PBT shown bellow.

Figure 20: UA-767PBT blood pressure monitor

A&D Medical UA-767PBT digital blood pressure monitor can send real-time communication of blood pressure measurements to the Access Point immediately. The device can also operate in a batch-mode to send the measurements stored in the memory to the Access Point. The communication packet include systolic, diastolic, pulse rate, measurement time, transmission time, battery status, device ID and other device information. A&D Medical UA-767PBT digital blood pressure monitor uses A&D Medical HDP/SPP communication unit which is a Class 1 Bluetooth 2.1 Embedded board. UA-767PBT supports SSP (secure simple pairing) so user does not need to enter PIN during the pairing process. The complete datasheet can be found at http://www.telemedicine.jp/product_ua767pbt.html

5.2.2 A&D WEIGHT SCALE The selected weight scale is the UA-321PBT shown bellow.

Figure 21: UA-321PBT weight scale

A&D Medical UC-321PBT digital weight scale can send real-time communication of weight measurement to the Access Point immediately. The devices can also operate in a batch-mode to send the measurements stored in the memory to the Access Point. The communication packet include weight in either lb or kg, measurement time, transmission time, battery status, device ID and other device information. A&D Medical UC-321PBT digital weight scale uses A&D Medical HDP/SPP communication unit which is a Class 1 Bluetooth 2.1 Embedded board. UC-321PBT supports SSP (secure simple pairing) so user does not need to enter PIN during the pairing process. The complete datasheet can be found at http://www.telemedicine.jp/product_uc321pbt.html



5.3 AONTV SERVICE PLATFORM TECHNICAL DETAILS AND INTERFACES

The Telekom Austria IPTV provides a foundational platform (OMP) and a consistent set of fully customizable software applications that offer advanced video services, subscriber features and operations systems to support fully bundled, multimedia broadband entertainment services. OMP relies on a Unix base, with an Apache server to deliver applications to clients. All service provider data is kept in an Oracle database. The Apache server is coupled with a Java application server - this application server runs applications that allow the client applications to communicate with the database. Features like Broadcast TV, Video on Demand, Web browsing, Email, and others, can be added to the system.

Figure 22: AonTV platform architecture

The system consists of two application servers. These Sun V240 servers are responsible for running OMP, DHCP and DNS. These servers are each load balanced, however due to the requirement of OMP to maintain proper associations between client and a particular OMP server, the load balancing must be sticky to force the client to always use the same (although it can be either) OMP server. This server platform is used to host a variety of applications that deliver the User Interface to the set-top-box and provides all of the management information necessary for it to operate correctly. The system is also used to store billing information. The system consists of dual load balanced Web Cache servers,. These reduce the load on the Application servers where static HTML objects are cached. These are based on two HP DL360 servers. The system consists of two database servers. These Sun V240 servers run Veritas cluster software to provide disk management functionality. They also run Oracle 9i RAC. RAC



provides Oracle functionality across multiple front-end Oracle database servers.

5.4 APPLICATION SERVER SELECTION AND TECHNICAL DETAILS Regarding the application server the requirements are:

• 35 clients/users to be served at the pilot phase • ~ 100KB database size • MySQL support • Apache web-server • Windows 2008 server operating system

Having a Windows 2008 server operating system imposes some requirements on the server. These are:

Component RequirementProcessor • Minimum: 1GHz (x86 processor) or 1.4GHz (x64 processor)

• Recommended: 2GHz or faster Memory • Minimum: 512MB RAM

• Recommended: 2GB RAM or greater Available Disk Space • Minimum: 10GB

• Recommended: 40GB or greater Drive DVD-ROM drive Display and Peripherals

• Super VGA (800 x 600) or higher-resolution monitor • Keyboard • Microsoft Mouse or compatible pointing device

Table 6: Windows 2008 server system requirements

As a result the selected Application Server is the IBM System x3650 M2. A short overview of the server technical characteristics is depicted in the following table.

Manufacturer IBM

Form Factor 2U Rack

Processor Intel XEON Quad Core

Processor Code Name E5520

Processor speed (GHz) 2.26

Number of processors 1

Memory Capacity 6 GB / (128GB max)

Video graphics card Matrox G200eV (16MB)

Hard Disk Drive 2 X SAS 146GB / (6TB max)

Network Interface SERDES to I/O Bay (1000Mbps,100Mbps,10MBps)

Controllers ServeRAID M5014 SAS/SATA Controller

Table 7: IBM System x3650 M2 technical characteristics



Service providers powerful Application Servers will be used, when these services will become commercially available.

5.5 OTHER DEVICES SELECTION AND INTERFACES

5.5.1 ADSL ROUTER The main function of an ADSL router is to provide the internet ADSL connection to multiple computers/CE devices in your home. In the case of the HERA environment it is needed in order to connect the IPTV STB and the medical devices to the internet. In the Austrian Pilot Site, the Thomson 585 v7 ADSL router provided by Telecom Austria will be used, which is supports ADSL, ADSL2, and ADSL2+. It supports powerful security mechanisms such as Wi-Fi Protected Access (WPA™), Wired Equivalent Privacy (WEP™) encryption, and a physical user registration button, which allow users to communicate and access data with efficient link quality and the highest level of network security.

Figure 23: Thomson 585 v7 ADSL router

The complete data sheet can be found at http://medialibrary.technicolor.com/CommunsImagesEnLigne/Download/2570662_333_1_277_0-2B940409CBBE7466BD93675E211A2D45/DS_Technicolor_TG585v7.pdf In the Greek Pilot Site, the ADSL router that will be provided by the local Internet Service Provider will be used.

5.5.2 EQUIPMENT FOR CONNECTING BLUETOOTH DEVICES TO THE INTERNET

5.5.2.1 OFF-THE –SHELF BLUETOOTH ACCESS POINT Since our medical devices support Bluetooth connectivity we need a mechanism to connect these devices to the Internet. This is achieved via a Bluetooth Access Point. The reason why we need an access point and not a bridge, is that an access point connects wireless devices to a wired network at the data-link layer. Two wireless bridges may be used to connect two wired networks over a wireless link, useful in situations where a wired connection may be unavailable, such as between two separate homes. The proposed devices are the USBGear “Bluetooth Access Point + Router Class 1” and the BlueRadios BR-BG-01, shown in the following figures.



Figure 24: “Bluetooth Access Point + Router Class 1”

Figure 25: BlueRadios BR-BG-01

5.5.2.2 CUSTOMIZED BLUETOOTH ACCESS POINT An alternative to the devices listed above, would be to use a Linux based wireless access point. This would be an OpenWRT based device such as a router where we add a USB Bluetooth dongle to one of the available USB input slots. We decided to use OpenWRT (a Linux distribution for embedded devices), because it has a good documentation (a Wiki), is well supported and maintained, it has a big repository and it is well customizable. On top of that a Bluetooth protocol stack for Linux (BlueZ) would provide the needed functionality. Finally a driver/application running on the Linux based router would allow for forwarding of Bluetooth data to a specific IP address and TCP/UDP port. A description of Open WRT can be found here: http://openwrt.org/ while a description for BlueZ can be found here: http://www.bluez.org/ The proposed router to build such a system is the Asus WL-500g Premium, since it’s the device used widely by developers across the OpenWRT community. Regarding the USB Bluetooth dongle the proposed one is a Class I Belkin Bluetooth dongle.



Figure 26: Asus WL-500g Premium router

Due to the high price of the off-the-shelf Bluetooth access point and routers, we consider to implement a customized Bluetooth access point, as described above.



6. SW TECHNOLOGIES SELECTION

6.1 HUMAN MACHINE INTERFACES TECHNOLOGIES Accessing web-based applications served by devices in the home and on the Internet on a variety of different CE-devices has a lot of interesting opportunities. The Web4CE framework (which is supported by important industry consortia such as DLNA and CEA) has all the mechanisms and ingredients in place to achieve this, and provides a sound technical foundation to build upon. The Web4CE framework (as depicted bellow) adopts a number of technologies for both in-home and the Internet that span multiple domains. These technologies are: XHTML, DOM level 2, CSS TV Profile [11] (which is a superset of the CSS Mobile Profile) and UPnP [12]. It also supports AJAX (i.e. XMLHttpRequest) [13] to support the latest web applications, such as GoogleMaps, and to enhance interactive applications such as Electronic Program Guides (EPG). The collection of technologies to describe the user interface (i.e. XHTML, CSS TV profile, AJAX, etc) is called CE-HTML.

Figure 27: The Web4CE framework

6.1.1 CAPABILITY EXCHANGE In order to deal with a wide variety of CE-devices, i.e. ranging from TVs to mobile phones, a mechanism is needed for exchanging UI and A/V capabilities. This mechanism allows a server to adapt the UI to the capabilities of a client device. In case of Web4CE, also resource-constrained devices (e.g. a mobile phone) may act as a server. Existing technologies such as CC/PP are too demanding for resource-constrained server devices. Therefore, Web4CE defines a lightweight capability exchange mechanism, which is based around a few base abstract profiles:



• “SD_UIPROF” (i.e. default profile), which describes clients with a 640x480 screen, a base set of keys and fonts that are readable for different screen sizes and viewing distances.

• “HD_UIPROF”, which is used for 1280x720 displays.

• “MD_UIPROF”, which is used for lower-end devices with at least a 208x208 display, with 4096 colors and fewer keys.

Web4CE defines an XML language to describe extensions on top of these profiles. Servers may take these extensions into account for adapting the UI. Web4CE also defines a mechanism for a server to list the capabilities that it requires to correctly render the UI. This allows clients to filter out compatible UIs before a connection is made.

6.1.2 REMOTE CONTROL / TV-BASED INTERACTION An important part of the Web4CE framework is to provide support for showing applications on a TV, and to interact with these applications by using a remote control (incl. text-entry and focus navigation). Web4CE defines the necessary extensions to handle remote control keys.

6.1.3 TWO-WAY COMMUNICATION Two-way communication between a client and a server is an important requirement for interactive consumer services. The currently de-facto XMLHttpRequest mechanism [5] can be used in most cases. However, use cases require key events and/or page updates to be forwarded to the client in a timely manner, without introducing significant delays (which may be introduced when using 'polling-based' methods, such as XMLHttpRequest). Therefore, we introduce the so-called "NotifSocket" scripting object, which can be used for setting up a TCP/IP connection between a client and a server through JavaScript and to send/receive data over this connection.

6.1.4 VOICE-BASED USER INTERFACES Multimodal interfaces allow users to interact with computers using multiple different modes or channels of communication (e.g. speaking vs. clicking a button vs. writing). Different modes are best suited for different kinds of input or outputs. The most effective multimodal interfaces enable more natural and effective interaction by allowing users to interact using whichever mode or combination of modes are most appropriate given the situation and their preferences and abilities. The following paragraphs provide a short overview of the state-of-the-art in technologies and web browsers for voice-based user interfaces design and development. VoiceXML VoiceXML is a markup language for creating voice-user interfaces. It uses speech recognition and/or touchtone (DTMF keypad) for input, and pre-recorded audio and text-to-speech synthesis (TTS) for output. It is based on the Worldwide Web Consortium's (W3C's) Extensible Markup Language (XML), and leverages the web paradigm for application development and deployment. By having a common language, application developers, platform vendors, and tool providers all can benefit from code portability and reuse. With VoiceXML, speech recognition application development is greatly simplified by using familiar web infrastructure, including tools and Web servers. Instead of using a PC with a Web



browser, any telephone can access VoiceXML applications via a VoiceXML "interpreter" (also known as a "browser") running on a telephony server. Whereas HTML is commonly used for creating graphical Web applications, VoiceXML can be used for voice-enabled Web applications. X+V VoiceXML supports efficiently the development of voice applications. However the combination of visual and voice input would be able to provide enriched services to the users with extended capabilities. This type of applications is called multimodal. In this context XHTML was modularized and extended in order to support both voice and multimodal applications. X+V was proposed by a consortium of IBM, Opera and Motorola to the World Wide Web Consortium (W3C). The most recent version of XHTML+Voice specification is XHTML+Voice Profile 1.2. Modularization of VoiceXML produced several modules which were integrated to XHTML along with Speech Synthesis Markup Language (SSML) and Speech Recognition Grammar Format (SRGF). In this context, speech dialogs in X+V are implemented by the use of VoiceXML 2.0 form elements which are assigned specific ids. Forms respond to XML Event events. XML Event specification defines several attributes which can be used along with different elements in order to provide an efficient event processing model. The events are assigned event handlers which reference to the id of the forms. SALT Speech Application Language Tags (SALT) is a new standard for the development of multimodal applications proposed by a consortium of Cisco, Comverse, Intel, Philips, Scansoft, Microsoft and a number of additional contributors. It is a speech interface which provides a set of extensions to existing presentation languages such as HTML and XHTML. With an enhancement of a small number of new elements, SALT provides the necessary components for the development of voice enabled and multimodal applications. In general SALT is supported by different devices ranging from PCs to Personal Digital Assistants (PDAs) and tablet PCs. Multimodal applications are very attractive for small devices (PDAs, Set-top Boxes, tablet PCs) due to their small sized input and output hardware equipment (screen, keypad). In addition SALT is compatible with a number of different standards such as SRGS, SSML, HTML, XML and HTTP. Therefore portability of SALT applications at different voice and multimodal systems is guaranteed. X+V Browsers The most popular X+V browsers currently available are the Opera Web Browser and the NetFront Web Browser. The most currently available version of the Opera web browser (version 9.x.x) supports the XHTML+Voice Profile 1.2. The standard is supported in all the three versions of the browser (Opera for Desktop, Opera for Mobile, Opera for devices). NetFront web browser is also available for different types of devices such as set-top boxes (STB), PDAs, web phones, game consoles and e-mail terminals. The most updated versions of the browser support XHTML+Voice Profile 1.1. In general Opera is more attractive than NetFront due to the fact that it is provided for free. In addition it complies with most of the W3C standards and can be adopted for different device types. SALT Browsers In the case of SALT standard different applications and tools have been developed from different vendors. We focus on free browser based solutions. In this context we came up with two available solutions:



• Microsoft Speech Application Software Development Kit (SASDK) Version 1.1 enables the development of multimodal applications. The kit provides an extension to Internet Explorer in order to support speech-only and multimodal services.

• The OpenSALT project at Carnegie Mellon University provides a web browser based on the open source Mozilla web browser enhanced with the open source Sphinx recognition and Festival synthesis software.

In both cases SALT is a new standard and therefore has not been adopted in a large scale yet. However, the following issues do not make the corresponding technologies suitable for application within the HERA project: • IPTV STB manufacturers do not provide support for voice-based interaction with the STB

since special installation of microphone arrays is required within the home environment. • The special installation of microphone arrays is something that the pilot sites cannot easily

and properly support during the project in the end user houses due to cost, installation and calibration issues.

• Relevant surveys and experience of the project’s participants in the area showed that there aren’t mature TTS/SR engines available that can be easily used / integrated in the STB.

Due to these limitations voice-based multimodal interfaces will not be supported in HERA and the associated effort will be used for the implementation of Web4CE HMI libraries.

6.2 COMMUNICATION TECHNOLOGIES The following table provides the technologies used for the communication between the different elements of the HERA network architecture.

Interface Technologies/Protocols

Blood pressure monitor device <-> Bluetooth Access Point Bluetooth Weight scale <-> Bluetooth Access Point Bluetooth Bluetooth Access Point <-> ADSL Router LAN IPTV STB <-> ADSL Router LAN TV set <-> IPTV STB SCART/HDMI ADSL Router <-> HERA Application Server IP HERA Application Server <-> Medical Center PCs IP HERA Application Server <-> AonTV Application Server IP

Table 8: Communication Technologies



6.3 SERVICE EXECUTION PLATFORM AND DATABASE TECHNOLOGY Web applications are popular due to the ubiquity of web browsers, and the convenience of using a web browser as a client. The ability to update and maintain web applications without distributing and installing software on potentially thousands of client computers is a key reason for their popularity, as is the inherent support for cross-platform compatibility. In earlier type of client–server computing, each application had its own client program which served as its user interface and had to be separately installed on each user's personal computer. An upgrade to the server part of the application would typically require an upgrade to the clients installed on each user workstation, adding to the support cost and decreasing productivity. In contrast, web applications use web documents written in XHTML, which are supported by a variety of web browsers. Each individual web page is delivered to the client as a static document, but the sequence of pages can provide an interactive experience, as user input is returned through web form elements embedded in the page markup. During the session, the web browser interprets and displays the pages, and acts as the universal client for any web application. Browser applications typically require little or no disk space on the client, upgrade automatically with new features. They also provide cross-platform compatibility in most cases (i.e., Windows, Mac, Linux, etc.) because they operate within a web browser window. However, browser applications rely on application files accessed on remote servers through the Internet. Therefore, when connection is interrupted, the application is no longer usable. The following table summarizes the current technology stacks, showing their differences in terms of:

• Programming languages, • Web server • Database support

Web Application Technologies

Web Server

Application Server Web Services Support

Programming Languages / Technologies

Database Support

PHP Apache Not applicable. Can be combined with JSP (Glassfish, JBOSS, Resin, Tomcat)

Yes Java, EJB Oracle, MySQL, MS SQL Server

ASP IIS ADO.NET AS, SharePoint Yes C# Oracle, MySQL, MS SQL Server

JSP Apache Glassfish, JBOSS, Resin, Tomcat Yes Java, EJB Oracle, MySQL,

MS SQL Server

Table 9: Web Application Technology stacks

These technologies are quite different, which means that someone who's familiar with one approach would have a high learning curve to use a different one. Once an application is developed using one technology, it is difficult and expensive to convert it to a different one. Regarding the licensing scheme this depends on the selected application server. The table bellow illustrates that.



Application Server OS Vendor License Glassfish v.2.1 Windows/Linux SUN SUN Mozilla public license (GPL, Commercial) JBOS v.5 Windows/Linux JBOS AS / Red Hat LGPL and Proprietary (Java) SharePoint Windows Microsoft Microsoft ADO.NET AS Windows Microsoft Microsoft Tomcat v.6 Windows/Linux Apache Apache Software License (GPL, Commercial) Resin v.3 Windows/Linux Caucho GPL and Proprietary (Java)

Table 10: Application Server Licensing Scheme

Keeping in mind that the agent platform is implemented in Java, to cater for easy integration a decision to use PHP or JSP was made. In addition to that JSP offers full compatibility with TAAG not only for exploitation but for implementation of the services as well. The JSP architecture is also used by Telecom Austria, thus providing the needed compatibility for immediate exploitation. Finally the selected application server is the Tomcat 6.x version. Regarding database technologies these are summarized below:

Database Technologies

Web Application Technologies Support

OS License Stored Procedures

MySQL DB v5 PHP/ASP/JSP Windows / Linux

GPL and Commercial (MySQL enterprise subscription)

Very basic features, runs interpreted in session threads. Limited scalability.

Oracle DB 11g PHP/ASP/JSP Windows / Linux

OTN Limited License for Developing: Oracle 10gR2 Express Edition and OTN License per person and per PC

Advanced features, runs interpreted or compiled. Lots of built in packages add significant functionality. Extremely scalable.

SQL server PHP/ASP/C# Windows Microsoft License Advanced Features

Table 11: Database Technologies

The selected database is MySQL since it provides a commercial license and has all the necessary features/procedures needed by our application. In terms of performance we don’t expect any issues, since the trial periods include only 20 clients/users. We should also mention that when any of the HERA services will become commercially available as product we will move to Oracle.

6.4 AGENT PLATFORM JADE6 (Bellifemine et al, 2001) is a software development framework fully implemented in Java language aiming at the development of multi-agent systems and applications that comply with FIPA standards for intelligent agents. JADE provides standard agent technologies and offers to the developer a number of features in order to simplify the development process:

6 JADE (Java Agent DEvelopment Framework) is a software Framework fully implemented in Java language. It simplifies the implementation of multi-agent systems through a middle-ware that complies with the FIPA specifications and through a set of graphical tools that supports the debugging and deployment phases. URL: http://jade.tilab.com



a) Bordini et al. (2006), in their survey of programming languages and platforms for Multi-

Agent Systems, found that the most popular agent platform is JADE

b) It is open source, thus available to anyone, a fact which increases the potential visibility of this work

c) It has an excellent users community that helps new users to achieve their goals quickly (minimizing learning time) and that continuously provides new add-ons

d) It is compliant with the FIPA standards

e) It can be compiled for use on devices with limited resources such as PDAs and mobile phones

For the agents decision making we chose argumentation as it allows for decision making using conflicting knowledge, thus different experts can express their knowledge without needing to be consistent with the opinions of the others. Argumentation can be abstractly defined as the formal interaction of different conflicting arguments for and against some conclusion due to different reasons and provides the appropriate semantics for resolving such conflicts. Thus, it is very well suited for implementing decision making mechanisms dealing with the above requirements. Moreover, the dynamic nature of those conflicting decisions due to different situations or contexts needs a specific type of argumentation frameworks (Kakas and Moraitis, 2003, Prakken and Sartor, 1996). These frameworks are based on object level arguments representing the decision policies and then they are using priority arguments expressing preferences on the object level arguments in order to resolve possible conflicts. Subsequently, additional priority arguments can be used in order to resolve potential conflicts between priority arguments of the previous level. Therefore, we are concerned with argumentation frameworks that allow for the representation of dynamic preferences under the form of dynamic priorities over arguments. In this work we are using the framework proposed by Kakas and Moraitis (2002, 2003). This framework has been applied in a successful way in different applications (see e.g. Moraitis and Spanoudakis, 2007, Spanoudakis et al., 2009) involving similar scenarios of decision making and it is supported by an open source software called Gorgias7,. It allows to define nonstatic priorities between arguments, which means that the priorities of rules can depend on context. Finally, the modularity of its representation allows for the easy incorporation of views of different experts.

7 Gorgias is an open source general argumentation framework that combines the ideas of preference reasoning and abduction (http://www.cs.ucy.ac.cy/~nkd/gorgias)



6.5 DATA MODELING Data will first be described in UML class diagrams so that the different communication interfaces can be described adequately. Later they should also be defined in ER diagrams as they will be inserted in the HERA database. They overall HERA data model is presented in Figure 28 and. It includes the data used for describing the various services in a previous chapter. There are the Professional and HERAUser concepts defining the HERA user types having information such as their contact information (address, telephones, etc), date of birth, name and surname. The Professional has a type property indicating whether he is a doctor, a nurse or administrative personnel and an organization property indicating the institution to which he/she is employed. The HERAUser includes a property for assigning the user’s group (userGroup). The Professional and the HERAUser both have access to their prescriptions (the former prescribes and the latter executes). The Prescription has the untilDate property that shows when this prescription terminates. More than one exercise may be prescribed to follow a specific schedule and for a specific difficulty level (if there exists such). The PhysicalExercise prescribed can have a specific number of repetitions for the user to achieve (numberOfRepetitions property). The CognitiveExercise prescribed can have a target score for the user to achieve (targetScore property). The Exercise has a type (see the list above), a maximum level, a name and a number of user groups to which it applies. Finally, the exerciseInvocationURL property shows the internet address for invoking the specific exercise. This address can be a link to a HERA defined service or a link to an external service. The PushInformation concept is used in the reality orientation push information service and tags available videos and information as applicable to one or more of the HERA user groups, to a specific physical area (country, city, or even neighborhood) and also to the season in which it is helpful (using the applicableDateFrom and applicableDayTo properties). Thus, this service needs a database infrastructure in which to store the information for the available videos, and also a link to the video files. There are also two types of data related to reminders, the first is the rendezvous with the doctors (depicted with the Rendesvous concept) and the second is the pill prescription. The PillSchedule concept allows the doctor to prescribe pills either for a specific time of the day or with a specific lunch type (e.g. with breakfast, lunch or dinner). Each time one or more pills are to be taken in a specific dosage (see PillDosage concept) and the information stored for each pill is its name, a file with its picture and some instructions related to its use (see the Pill concept and its properties). The following figure depicts the HERA data model based on the above description and analysis.



-type : string-maxDifficultyLevel : int-name : string-applicableToUserGroups : string-exerciseInvocationURL : string

Exercise

ExercisesPrescription

-pill1

-day_of_week : string-hour_of_day : int-difficultyLevel : int

ExerciseSchedule

-exercises1..*

-targetScore : stringCognitiveExercise


Person


Address

-address

1


-userInformation 1

-myPatients

0..*

-attendingMedicalPersonnel

1..*-type : string-organization : string

Professional

-userInformation 1

-myPrescriptions

0..*

-subscribedBy 1

-id : int-videoFile : string-applicableToUserGroups : string-applicableDateFrom : Date-applicableDateTo : Date

PushInformation

-applicableArea

1

PillPrescription

-name : string-imageFile : string-usefulInformation : string

Pill

-hour_of_day : int-withLunchType : string

PillSchedule

-pillTakeOnHourOfDay1..*

-pills1..*

-numberOfPills : floatPillDosage

-myPrescriptions

0..*

-patient 1

-untilDate : DatePrescription

-exercise1

-dateTaken : Date-score : string

ExerciseScore

UserExercisesResults

-exercises0..*

-myExerciseResults 1

-patient 1

-exercise

1

-frequencyAppearMinutes : int-durationOfAppearanceMinutes : int-showTime : bool-showWeatherReport : bool-showDate : bool

RealityOrientationPrescription

Figure 28: The HERA data model (part 1)

The following figure depicts the HERA data model based on the blood pressure monitoring, weigh monitoring and reminder services specification in chapter 3.



Figure 29: The HERA data model (part 2)



7. HERA APPLICATION SERVER ARCHITECTURE AND FUNCTIONAL SPECIFICATION

7.1 ARCHITECTURE OVERVIEW The following figure depicts an overview of the HERA Application Server Architecture.

Figure 30: HERA Application Server Architecture

As already presented before, the selected web server is the Apache web server, an open source http server executed both on Linux and Windows. The selected database is MySQL Community Server since it is generally available (no license required) and has all the necessary features/procedures needed by our application, plus the Java Database Driver Connector (JDBC Driver - Connector/J) that enables developers to build database applications in Java. The main modules constituting the HERA AS are:

• The “Notifications Module”, which is responsible for sending text based notifications to a specific HERA user.

• The “Measurements Module”, which is responsible for receiving measurements from the medical devices and store them to the database while at the same time forward these measurements to the appropriate application logic module, for further processing. Additionally it handles history measurement requests from the application logic.

• The “AonTV Module”, which is the main interface with the Telekom Austria commercial IPTV platform (AoTV) responsible for sending notification requests, measurements history and other data upon request to AonTV clients.

• The “System Administration Module”, which is responsible for: o Handling of creation/removal of user/doctor entries in the database and setting of

the respective policies. o Maintenance of nutrition counseling database.



• The “Authentication Module”, which is responsible for authenticating a HERA user. • The “Policy Management Module”, which is responsible for accessing the database, with

the purpose to report to the application logic in which services a specific user has access. Additionally it manages which fields, in the database, the doctor/administrator can access while replying to the application logic in which group the application user belongs.

• The “Multi-Agent System Module”, which is responsible for learning the user’s habits, personalizing several services such as the pill reminder and the passive communication and for reminding him to do his daily tasks through the notifications module.

In our architecture there is an application logic layer that hosts the logic needed by the supported services. These are:

• “Blood Pressure Monitoring Application Logic”, which is responsible for analyzing received measurements, from the “Measurements Module”, and make decisions based on the optimal upper and lower limits, as set from the attending physician and sent notifications to the patient and the Medical Center when measurements are critical or out of emergency limits. Additionally it’s responsible for sending notifications to the patient regarding missed measurements and in case no measurement has been sent for 1 day, to notify the health center. Finally it’s responsible for handling “Measurements History Requests” from the overlying HMIs (user or doctor) and reply with measurements list.

• “Weight Monitoring Application Logic”, which is responsible for analyzing received measurements, from the “Measurements Module”, and make decisions based on the current weight value and the goals for weight reduction on weekly and monthly basis. A notification is sent if the measured weight value is above the weekly or monthly goal value plus the defined margin. Finally it’s responsible for handling “Measurements History Requests” from the overlying HMIs (user or doctor) and reply with measurements list.

• “Nutrition Counseling Application Logic”, which provides the user with nutrition counseling and tracing of consumed ingredients and portions.

• “Reminders Application Logic”, which is responsible for analyzing the reminders data for all patients, according to the data entered by the doctors to the system and send notifications to the patients if a reminder is triggered.

• “Passive Communication Logic”, which is responsible for pushing information to the patients. This information can be videos which help them manage their disease, inform them about recent developments or about recent events. These videos are selected and uploaded to the application by the doctors. Moreover, it can push information such as the date and time which will be visible on the user’s screen.

• “Cognitive Reinforcement Logic”, which is responsible for helping the patients reinforce their cognitive skills through playing specific games. These are assigned to the patient by the doctors.

Finally the Human Machine Interfaces (HMIs) provide all the necessary graphical elements so that HERA platform users can access, view, modify their stored profiles and data. These are:

• “User HMIs” that include a login page, a “Service Overview” page showing the user prescribed services and a number of application/service specific pages (weight history, blood pressure history, nutrition counseling etc.).

• “Doctor HMIs” that include a login page, patient profile configuration pages as well as notifications and alarm pages.



• “Administration HMIs” that include a login page and a “HERA Users” page, allowing for creation/removal of HERA users (patients/doctors) and their respective policies.

All communication interfaces are based on Internet technologies supporting standard security mechanisms including SSL, HTTPS and Secure Web Services.

7.2 NOTIFICATIONS MODULE This module is responsible for handling requests towards displaying pop-up windows on the STB. The following figure provides the block diagram of the notifications module.

IPAddressResolver

STBRemoteNotificationHandlerName, Text, Type

UserID

IPAddress

UserID, Name, Text, Type

Encoded string that is decoded at the client side using NotifSocket scripting object

Figure 31: Notifications Module

The module passes the UserID to the IPAddressResolver. If a valid IP address exists it is passed to the STBRemoteNotificationHandler module that is then responsible to build the request towards displaying on the specific user STB the specified text. This is possible through the STB JavaScript API function calls (Web4CE standard supports outgoing TCP connections via the Web4CE specific NotifSocket mechanism). Based on the information type (warning, error, information), the look and feel of the pop-up window changes.

7.3 MEASUREMENTS MODULE This module is responsible for updating the database with measures from the medical devices and retrieve data (Systolic/Diastolic blood pressure (mmHG), pulse rate, Kgr, Timestamp) needed by the application logic, while at the same time forward these measurements to the appropriate application logic module, for further processing. The following figure provides the block diagram of the measurements module.



UserIDResolver

DatabaseWarehouse

Data

DeviceID

UserID

DataForward

DeviceID, Data

Data

UserID

Data

UserID, Period

MeasurementsList

Figure 32: Measurements Module

In order to write data to the database the module uses the device id. Since the device id is unique and associated with a unique user it’s possible to find that user (UserIDResolver) and update their measured data (DatabaseWarehouse), while forwarding the measurements to the appropriate application logic module (DataForward). On the other direction when the application requests for measured data, as in the case of the measured blood pressure and weight history, the module uses the user id in order to access the database stored measurements and the period of measurements in order to retrieve the requested measurements list (DatabaseWarehouse). It is then up to the application how to display this information.

7.4 AONTV INTERFACE MODULE This section illustrates the interface between AonTV and the HERA AS, which is based on Web Services (WSDL, SOAP) using Apache Axis2. A summary of the Web Services is shown on Table 12.

Return Value Web Service Name Input Parameters String MeasureTypes MeasureTypesWS int stbid String MeasuredValues MeasuredValuesWS int stbid

String StartDate String EndDate String MeasureType

String Notifications NotificationsWS int stbid Table 12: AonTV interface module Web Services

The first web-service, namely MeasureTypeWS. returns a list of measure types that are available for an AonTV customer (i.e. “Blood Pressure”, “Body Weight”). The AonTV customer will be selected via Set-Top-Box ID (stbid). If there are more than one customers/patients behind the same STB it is necessary to define several criteria to unique identify the patient. The web-service queries the database and returns an XML-encoded string with the available measure types. The second web-service, namely MeasuredValuesWS, returns measurement values in a specified



time range. The patient selects the type (blood pressure, weight, …) and the time range on TV-screen and the AonTV platform makes a call to the web service containing all parameters which are necessary to collect the appropriated data. The HERA AS generates the response message in an XML encoded string. All measurement values in the appropriated time range have to be included in the response message and the graphic for display on television will be generated on the AonTV site. The same scenario is necessary for notifications. In this case the AonTV platform calls the NotificationsWS in regular intervals. The HERA AS collects the notifications of the patients and transfers the data (notification type, notification message …), which should be transferred to the patients, back to the AonTV servers. The AonTV platform does not store any data, it only displays the requested data from the user behind the television.

7.5 AUTHENTICATION MODULE This module receives data from the HMI in the form of a username and a password, queries the database (DatabaseWarehouse) and replies back to respective application module (CredentialsComparison). The following figure provides the block diagram of the authentication module.

Figure 33: Authentication Module

This is part of the HERA login functionality. Specific HTML pages (Web4CE for users, plain HTML for administrator/doctors) will allow for population of the username and the password that are compared to the ones stored in the database (CredentialsComparison). If they match the module replies with a successful authentication message, otherwise an authentication failure message is issued.

7.6 POLICY MANAGEMENT MODULE This module exists in the same business logic layer as the profile management module and it is accessing the database. The HERA foreseen policies are:

• Services (such as Blood Pressure Monitoring, Weight Monitoring, other) • Fields (name, address, diseases, other) • Group (patient, doctor, administrator)

The administrator through the “Administration HMI” can update the respective policies. The following figure provides the block diagram of the policy management module.



Figure 34: Policy Management Module

The module uses the UserID and depending on the request it returns the services that the user has access, which fields the doctor/administrator can access and which group the application user belongs.

7.7 THE MULTI-AGENT SYSTEM MODULE (MAS) The intelligent agent module accesses information in the HERA database and initiates user interaction with all services by reminding him on the tasks he has to achieve. It retains a calendar for each user’s tasks and on one hand resolves conflicts when more than one things are scheduled in the same time according to the information provided in D2.1 use case 2.3.

The architecture of the Multi-Agent System module (or MAS module) is depicted in

Figure 35. There is the web server (Apache) running the Tomcat module which allows Java applications to execute. The JADE platform is launched in Tomcat and the MAS module exposes its services through a Java interface or a web service interface. Using similar interfaces it can connect to the other modules with which it interacts.



HERA Server

Apache Web server

Apache Tomcat

Authentication moduleJSP or Java servlets

HTTP interface

HERA DatabaseMySQL

JDBC interface

MySQL DBMS

FIPA Agent Platform (jade.tilab.com) Bundle

Interface Agent

Personal Assistant Agents

ACL

SOAP or Javainterface

Notification moduleJava


Huma-Machine InterfaceJSP or Java servlets

HTTP interface

AJAX interface

Privacy control moduleJava


Push information moduleJava


Figure 35: The Information push service ontology

The MAS module uses a specific type of agent, an Interface (INT) agent for connecting with other modules. This happens because it is not practical for every agent to launch his own interfaces for other modules to send him a message. Thus, the interface agent is the only one exposing the MAS services to the other modules. This agent gets all requests for service and delegates them to the interested personal assistant agent.

The Personal Assistant (PA) agent is the most complex role and it serves a registered user, stores and manages his/her profile and personal data and uses the requests’ history in order to adapt the services to his/her habitual patterns. Service adaptation is achieved by hour of the day dependent profile management and continuous refinement of user selected actions.

A different PA agent will be active for each user. The PA is persistent and retains a schedule for the user’s tasks. These tasks are assigned by the doctors that monitor the specific user using the services’ backoffice system. The following tasks will be achieved by the PA:

• When the user’s scheduled time for taking pills arrives the PA reasons on the quantity to be taken. The doctors will be able to assign specific conditions when assigning pills to a patient. For example, if the blood pressure exceeds a limit then he has to take two pills (while normally he is assigned one pill)

• The agent learns the user’s referred time for cognitive training. Thus, in the case that the user selects a specific time (but different than the assigned one) for at least three times in



a row the agent learns that the patient wants to have his exercises at that time every day.

• When an item is inserted or is rescheduled the agent reasons on the priority of possibly conflicting tasks. Specifically, when the user has been assigned more than one tasks for the same time (e.g. by different caregivers) or that he has specific preferences (e.g. to watch a TV series at a particular time of day the following priorities will hold:

o Priority no1: take the assigned pills

o Priority no2: watch his favourite TV series

o Priority no3: engage with the cognitive reinforcement exercises

o Priority no4: engage with the physical reinforcement exercises

• Whenever the user switches on his TV and the day is Sunday after 10 a.m. the agent will select a piece of information to broadcast using the passive communication service.

The MAS module participates in use cases 2.1, 2.2, 2.3, 2.4, 2.5, 2.8. The interfaces that it realizes are shown graphically in Figure 36. Specifically:

• The addUser method is used by the HERA HMI to add a new user to the system. The MAS creates a new personal assistant agent and associates him to the specific new user. Then the user data is written to the HERA database.

• The userOpenedTV message informs the PA (to whom the INT forwards it) that his user just opened the TV and thus is supposed to be watching. The PA uses this time to forward to the user any reality orientation piece of information available.

• The userAction method is used by the Notification module to inform the MAS that a specific user that was shown a specific dialog responded in a specific manner. This message is forwarded by the interface agent to the interested PA who will respond according to his beliefs about the user actions.

• The prescribeExercises interface allows a doctor to inform (through the HMI) the PA (to whom again the INT forwards the message) that a new prescription is available for his patient. The PA will add the prescription items to his calendar and inform the user accordingly when needed.

• The userFinishedCognReinfService and userFinishedPhysReinfService interfaces are used to alert the MAS that a specific user just finished his exercises. The PA of that user will check the database for seeing the user’s results and figure out when he will be needed reminding for doing his exercise.

• Finally, the SendEmailToRelative interface allows the PA to inform a relative about the HERA end user’s actions if these seem to be abnormal.



Figure 36: The MAS module interface

Figure 37 shows how the use case 2.1 of D2.1 is defined in a UML sequence diagram. The MAS module asks the Notification module to deliver a message to the user with multiple possible responses and a timeout. The notification module is expected to inform the MAS about the user’s choice (or even that it timed out) at the latest when the timeout is exceeded.

Figure 37: Remind the user about his tasks (Use Case 2.1)

For the other interfaces usage in the different HERA use cases the user can see the relevant sequence diagrams in Figure 13, Figure 7, and Figure 8.



8. TEST AND VALIDATION PLAN

8.1 SYSTEM VALIDATION TEST PLAN The testing and validation of the HERA platform and services will be performed at different levels involving the complete system or parts of it. It will include four types:

1. Unit level testing

2. Integration level testing

3. System level testing

4. Acceptance level testing

The first three levels of testing will be mainly conducted by the technology partners of the HERA consortium, whereas acceptance testing will be primarily performed by the user partners.

Figure 38: HERA testing and validation methodology

Activities in Unit testing, Integration testing, and System testing are generally considered as Verification activities. These will enable the evaluation of the HERA platform and services components, by determining whether the outputs of a given development phase satisfy the requirements established before the start of that phase. On the other hand, Validation activities as in Acceptance testing will confirm that the HERA platform and services meets its intended use. In other words, these will focus on the final services, which will be extensively tested from the users’ point of view.



M13 M14 M15 M16 M17 M18 M19 M20 M21 M22 M23 M24Unit TestingIntegration TestingSystem TestingAcceptance Testing

Unit TestingIntegration TestingSystem TestingAcceptance Testing

HERA platform and services

v1

HERA platform and services

v2

Figure 39: Overall HERA testing and validation plan

As far as the system validation test plan is concerned, the following sections provide the nature of the tests involved in the different testing levels.

8.1.1 UNIT TESTING This first level of testing refers to the evaluation of each HERA unit in isolation. A unit is assumed to be any software component that implements a well-defined functionality providing a certain level of abstraction to the implementation of higher level functionalities (see also the HERA architecture and modules). There are two reasons for testing a unit in a stand-alone manner:

• Any errors can be specifically attributed and thus, can be easily fixed. • It is straightforward to verify that each distinct execution of a unit produces the expected

result.

8.1.2 INTEGRATION TESTING During integration, the HERA modules will be assembled together in an incremental manner, thus forming higher-level modules and sub-systems. All these will be further interconnected (in a certain way) in order to accomplish the complete HERA platform. In this context, integration testing provides a systematic technique for uncovering errors associated with interfacing. It will ensure that the independently-tested units will operate correctly after their combination.

Integration testing will be performed in phases of increasing complexity for better efficiency and effectiveness. Hence, it will include:

• Interface Integrity Tests are designed to evaluate internal and external interfaces, as each HERA module is integrated into the structure. An important part of interface testing is to map an incoming message format to the expected message format of the receiving unit, and ensure that the two match.

• Functional Validity Tests are designed to uncover functional errors in each unit, after its integration.

• End-to-End Validity Tests are performed to ensure that a completely-integrated module works efficiently from end to end.

• Pair-wise Validity Tests are performed to ensure that any two modules / sub-systems work properly when connected together by a communications method.

8.1.3 SYSTEM TESTING



As the HERA platform consists of both hardware devices and software modules, there is a need to have a much broader view of its behaviour. The objective of the system-level testing is to establish whether the implementation of the HERA platform and services conforms to the requirements specified. Thus, a variety of tests will be executed in order to meet a wide range of established expectations, as well as unspecified ones. These will include:

• Basic tests - providing evidence that the system can be installed, configured, and brought to an operational state.

• Functionality tests - providing comprehensive testing over the full range of the requirements within the capabilities of the HERA platform and services. Particular attention will be provided to the user interfaces and the overall usability. Thus, the following characteristics will be evaluated:

o Accessibility: Can users enter, navigate, and exit with relative ease?

o Responsiveness: Can users do what they want and when they want in a way that is clear?

o Efficiency: Can users do what they want with a minimum number of steps and time?

o Comprehensibility: Do users understand the HERA services with a minimum amount of effort?

• Robustness tests - determining the recovery process from various error conditions or failure situations (e.g. connection loss with STB/TVset). These will verify how gracefully the HERA platform behaves in error situations and in a changed operational environment.

• Performance and stability tests - measuring the performance characteristics of the HERA platform, and if the latter remains stable for a long period of time.

8.2 DEPLOYMENT PLAN

8.2.1 AUSTRIAN PILOT SITE The first step selecting the test users is to establish a contact to an Austrian Health Insurance Company (WGKK). This company offers various ambulatory treatment services for their clients which are under a physician’s care. The plan is, to select users together with the physicians, as we believe that they obtain profound knowledge about the health circumstances and personal circumstances of their patients. The general profile of test-users will be:

• Living in urban areas • Above 60 years • Still mobile • Female or male (balanced share) • With one or more of the following health problems:

Main categories of health problems will be:



• high or low blood-pressure • overweight • problems with healthy nutrition

The field trial will be divided into three parts and conducted in the following way. Initial phase To improve the understanding of the participants all of them will be invited for a presentation of the system. This presentation will be performed by TAAG and FRKR. The plan is to show the functions of the HERA system with the help of a mock-up. At this stage the participating test-users will sign the informed consent. Phase I For the first phase of the trials at the site of the Red Cross, the goal is to select a number of 10 -15 test-users. The health situation of the selected users will match into the elaborated use cases. Even a balanced sample will be intended. At his phase, the system will be installed at the Red Cross. Phase II For the second phase, when the users try out the system and the equipment at their homes, the goal is to select a number of 10 test-users. The timeplan towards the first trials phase is as follows:

• April and May 2010: find an approach to the Health insurance Company of WGKK (Wiener Gebietskrankenkasse) and first discussions with care department.

• June 2010: further discussions with the medical department of the Health insurance Company and preparation of information leaflets for doctors and test-users

• July, August, September 2010: selection of test-users and preparation of evaluation questionnaire, equipment ordering.

• October 2010: Workshop with TAAG and FRKR. The HERA system will be presented to the test-users). The informed consent forms will be signed.

o November and December 2010: finalization of evaluation guidelines, training material preparation and user training.

The training material will be developed within November (after the first prototype of the services is released). The service developers will provide the end users (both doctors and elderly) manuals to FRKR (in both word files and powerpoint presentations). FRKR will then train the users with respect to the service usage in its premises. Phase 1: Installation of 1st prototype (including all services) centrally In phase one, a HERA server will be installed at the TAAG premises and one STB at FRKR premises. Then, the patients will be invited, so they will also have the chance to experience the look and feel of the services. The equipment will be one set of STB and TVset, one weight measuring device and one blood pressure measuring device. They will be purchased in October 2010. Phase 2: Installation of final system at individual houses After a successful prototype period the HERA System can be used from individual persons in individual houses. Therefore it is necessary to complete an order form and send it to TAAG. The order form includes different packages what the user can select

• different speeds of the internet access • one media box and optional a second



• optional LCD television • different television packages (Basic, Premium and HD Quality) • wireless or cable connection to the media box • HERA components: BT Access point, BT BP meter, BT Scale, ….

TAAG checks if the user already is an aonTV customer or not. In case that the user is a new customer the possibility of manufacturing at the ordered location will be checked. Some criteria must be fulfilled before the ordering process can go into the next step. One criterion is the range from the switching centre to the customer and the second criteria is the availability of free channels. If an aonTV manufacturing is not possible it is possible to use the HERA system without it. In this case a simple Internet access with HERA services will provided to the customer. Otherwise the internal provisioning of the user profile, aonTV profile will be started. At the end of the provisioning process a logistics centre gets the following information to send an installation package to the customer:

• User data • Login credentials • PIN • Optional TV package + add-on • HERA Components (BT BP meter, BT Scale, …)

The logistic centre replies with a confirmation to TAAG and announce the date of delivery. Depending on that date the customer service technician plans the installation date at the customers site. He installs all the components for internet access and aonTV. After this components are tested and working correct the additional Bluetooth set top box will be installed. This box connects the HERA AS with the medical components at the users home environment. These components could be a BT BP meter, BT Scale, … . The customer service technician setup the coupling between medical devices and BT set top box and configures the user relevant informations. If the user is already an aonTV customer the installation of internet access and media box could be canceled. In this case the task of the customer service technician is to setup the HERA environment und make an enrollment with the patient. A flow chart which displays the process as described above can be found in the following figure.



Figure 40: Service process for HERA ordering and installation



Helpdesk Telekom Austria will offer 7x24 support for technical issues related to AonTV and equipment malfunctions through its standard customer care service. As far as support regarding the HERA applications operation and functionality is concerned, Telekom Austria will provide a helpdesk for the 2nd trials phase that will be operational for specific hours per day (e.g. 2 hours/day).

8.2.2 GREEK PILOT SITE User Selection A grand total of 25 individuals (15 healthy elderly and 10 MCI patients, mild and moderate AD patients) will be selected to participate in the project trials phase. The user selection process will be performed on-site by the medical experts comprising the clinical staff of HYGEIA. The initial visit will require approximately 1-1 ½ hour to complete. At this visit, thorough history-taking will be performed -including medical, psychiatric and social background information- along with a routine physical examination. Demographics and other non-health info about both the patient and his/her primary caregiver will be taken. The potential user will then participate in a comprehensive neuropsychological assessment performed by a trained medical expert to define the level of his/her cognitive functioning. The tests used will be standardized and provide normative data representing the contribution of demographic variables for the discrimination between normal and pathological performances. Next, the user’s affective status will be assessed, followed by assessment of his/her functional abilities. Functional assessment will be performed in the form of semi-structured interview with the user’s primary caregiver to detect difficulties in carrying out basic and instrumental activities of daily living. Subsequently, appropriate blood tests and brain imaging will be requested by the clinician in order to establish the diagnosis of MCI, mild or moderate AD (this last step will be omitted in the case of users who prove to be cognitively intact on the basis on the neuropsychological assessment results and patients who have already been diagnosed) and the next appointment will be scheduled. At the second visit, the medical experts will have an overall look of the available information (medical and social history, results of the neuropsychological, affective and functional assessment, blood tests, brain imaging), on the basis of which they will decide if the user fulfils the inclusion criteria and determine which target group she/he belongs to. In case the user fulfils the above criteria, details about the HERA project will be described to the user and his/her primary caregiver (rational behind the project, procedures, anticipated results and benefits, confidentiality, rights and obligations). The user and the caregiver will be given ample time to completely read the Inform Consent Form and ask project related questions. Before signing the forms, the clinician will establish that they fully understand the study and agree with the study related procedures. A copy of the singed Informed Consent Form will be given to the user and his/her caregiver and another will be retained on-site. User selection schedule The user selection process will be carried out from September 2010 to first half of January 2011. In the first month the process will be defined in detail including the questionnaires. During October there will be the first visit and during November the results will be evaluated and the successful individuals will proceed to the second visit. During December also the second visit results will be evaluated and the selected individuals will be asked to fill the relevant forms. Within December and January (up to 14th of January) the users will be trained in the premises of Hygeia.



Training material and users training The training material will be developed within November (after the first version of the services is released). It will be ready by the end of December when the integrated system first version will be ready. The service developers will provide the end users manual to HYGEIA in editable form (e.g. word files). The medical personnel will then be able to change the layout and wording of the documents related to the services for the HERA end users so that it is readable and compatible to their cognitive level. HYGEIA will then train the users with respect to the service usage in its premises. Phase 1: Installation of 1st prototype (including all services) centrally Hygeia has a number of demented patients who are followed up at its premises. The medical personnel also works there. In phase one, a HERA server will be installed at the Singular Logic premises and one or two terminals at the Hygeia premises. There the medical personnel will have a chance to be acquainted with the services and the backoffice and comment on the layout and functionality. Then, when the patients come for their appointments they will also have the chance to experience the look and feel of the services. The equipment will be two sets of STBs and TVs possibly different models. Also two for weight measuring devices and two blood pressure measuring devices. They will be purchased in October 2010. The installation will be done by SingularLogic, with the consulting support and supervision of Telecom Austria which is leader in the respective WP. It is possible to test the applicability of more than one STBs and / or TV sets and/or sensors. The best combination will be selected for the users’ homes for phase 2. Phase 2: Installation of final system at individual houses Installation SingularLogic shall check and make sure that the user owns an internet connection. If not, in cooperation with HYGEIA will purchase and install an internet connection for the user. SingularLogic, with the consulting support and supervision of Telecom Austria which is leader in the respective WP, will also be responsible for installing the HERA equipment at the user homes and test the connection to the server and good operation of the system. They will instruct the user and a possible care giver about the usage of the equipment. Helpdesk There will be two helpdesks working from M20 to M24 (i.e. from May 2011 to September 2011). The main one will be the helpdesk at Hygeia hospital that is already operational. The Hygeia help center will be equipped with trouble shooting steps to be followed at each kind of problem. If the problem is not resolved then it will be forwarded to the Singular Logic helpdesk either through email or voice mail. If the call is made during the specific hours then it can be forwarded to the Singular help center. The Singular Logic help center will be able to resolve questions regarding the web applications. It will also be able to identify the malfunctioning component and send bug reports to the developing partners responsible for the failure. The Singular Logic helpdesk will be operational for specific hours per day (e.g. 2 hours/day).

8.3 USER ACCEPTANCE TEST PLAN



8.3.1 DIABETES AND CARDIAC DISEASES USER GROUP The user evaluation will include the following major points:

• the usage of the system by test users • the usage during the day

The test-users shall use the system frequently. The evaluation will take place after the last usage with the help of a usability/evaluation questionnaire. Person Phase 1 (initial test phase) Evaluation

1 Phase 2 (at home) Evaluation

2 Usage

1 Usage 2

Usage 5

Usage 1

Usage 2

P1 2.11. 5.11. 13.12. 13.12. P2 3.11. 7.11. 31.12. 31.12. P3 P15

The evaluation will be conducted at the end of the initial test phase and at the end of the second testing phase at the patients’ homes. Both evaluations will be conducted by the FRKR personnel with the help of a standardized quantitative evaluation protocol. The aim of the evaluation is to systematically observe and document user needs and requirements of older people to use the technical device. The criteria of the user evaluation questionnaire will be:

• Accessibility (Is the system easy to start up/use/exit?) • Responsiveness (Can users do what they want? How useful was the feedback provided

by the system?) • Efficiency (Can users do what they want with a minimum number of steps and time? Do

users feel like this system is easier to use than their current protocol?) • Comprehensibility (Do users understand the HERA system with a minimum amount of

effort?) • Visibility of system status • User control / flexibility of usage (How possible/user-friendly is it, to use the system

during my day (with my special health needs, working conditions etc.?) • Design (Is the system clearly structured/navigable/nicely designed?) • Sense of security and data protection (Do you feel safe when using the system and

entering my personal health data?) • Impact (Do you feel like your way of coping with your illness has changed with or after

using the system?) Satisfaction should be measured twice – after the initial test phase and after the second test phase at home. Subjective satisfaction rates can be compared after these two measurements. The evaluation will be conducted at the end of the initial test phase and at the end of the second



testing phase at the patients’ homes. Both evaluations will be conducted by the FRKR personnel with the help of a standardized quantitative evaluation protocol. Planned analysis of trials in Phase I: At the end of the last usage of the user, the user is personally questioned about satisfaction and the above mentioned criteria with the help of a standardized protocol. The measurement will take about 15 minutes after the last test trial and will be documented by the researcher. In this period even the attending doctors will be questioned with a standardized protocol. Planned analysis of trials in Phase II: At the end of the last usage of the user at home, the user is personally visited and questioned about satisfaction and the above mentioned criteria with the help of a standardized protocol. In the end satisfaction is measured

• about the system • about participation in the trials • about the impact of participating in the trials on their health/compliance

As already performed at Phase I the attending doctors will be involved again in the evaluation. The results of both evaluations will be collected and analysed by the FRKR with a simple statistical analysis. The analysis will be reported and information will be gained about the overall satisfaction of the test users with the technical device and the impact of the usage on their wellbeing. The aim of the evaluation is to gain knowledge about the possible adaptations of the technical device for the HERA consortium in order to be able to construct a user-friendly device.

8.3.2 MCI AND AD USER GROUP The evaluation will be conducted at the end of the initial test phase and at the end of the second testing phase at the patients’ homes. In the case of MCI patients or patients with mild/moderate AD, both the patient and the caregiver will be questioned. The evaluations will be conducted by medical experts of the HYGEIA Memory Clinic using a standardized quantitative evaluation protocol. The aim of the evaluation will be to systematically document the users’ evaluation of the HERA services and the devices used for the purposes of the project. The draft contents of the user evaluation questionnaire will include among others:

• Accessibility (Is the system easy to start up/use/exit?) • Responsiveness (Can users do what they want? How useful was the feedback provided

by the system?) • Efficiency (Can users do what they want with a minimum number of steps and time? Do

users feel like this system is easier to use than their current protocol?) • Comprehensibility (Do users understand the HERA system with a minimum amount of

effort?) • Visibility of system status? • User control / flexibility of usage (How possible/user-friendly is it, to use the system?) • Design (Is the system clearly structured/navigable/nicely designed?) • Sense of security and data protection (Do you feel safe when using the system and

entering my personal health data?) • Impact (Do you feel like your way of coping with your illness has changed with or after

using the system?)



Apart from these, the evaluation questionnaire will also include AD/MCI specific questions including:

• Passive communication service evaluation (Are the videos presented useful/interesting/easy enough for the patient to follow?)

• Cognitive reinforcement service evaluation (Are the mental games suggested fun/interesting? Is their difficulty level adjusted to the patient’s needs? Have the patient and his/her caregiver notice any improvement in the patient’s cognitive abilities that could be attributed to this service?)

• Reality orientation service (subcategory of the cognitive reinforcement service) evaluation (Has the reality orientation service proved to be useful? Has it helped the patient orient in time?)

• Independence (Have the HERA services helped the patient be more independent?) • Caregiver burden (Have the HERA services reduced the burden of caring for a

demented patient?) • Quality of life (Have the HERA services improved the overall quality of life of the

family?) The evaluation will take place twice – after the initial test phase and after the second test phase at home. Subjective satisfaction rates can be compared after these two measurements. Planned analysis of trials in Phase I: At the end of the last usage, the user will be asked to answer in the predefined questions mentioned above. In the case of MCI and AD patients, the caregiver will also be asked to undergo the same procedure. The whole process will take about 15-30 minutes to complete. Planned analysis of trials in Phase II: At the end of the last usage, a medical expert will visit the user’s house and ask him/her to answer in the predefined questions mentioned above. In the case of MCI and AD patients, the caregiver will also be asked to undergo the same procedure. The whole process will take about 15-30 minutes to complete. The results of both evaluations will be collected and analyzed using simple statistical analysis methods. The analysis will be reported and information will be gained about the overall satisfaction of the users and the impact of the HERA services on their wellbeing. The evaluation criteria will be based on the evaluation questionnaire contents, as described above.



9. CONCLUSIONS The aim of this deliverable was to select and specify the services to be implemented, to select the equipment and the SW technologies required for the implementation and deployment of the services, to specify the HERA network and Application Server architecture and its components and to define the test and validation plan. The following services were selected to be implemented:

• Passive Communication Service • Cognitive Reinforcement Service • Blood Pressure Monitoring Service • Body Weight Monitoring Service • Nutrition Counseling Service • Reminders Service

These services will be deployed in two (2) pilot sites at Austria and Greece. In order to achieve user acceptance and usability HERA services will deploy mainstream technologies and simple intuitive interfaces, through the utilization of Set-Top-Boxes connected to TV sets, which are among the most commonly used technologies present in almost all houses. Based on the requirements/criteria set for the selection of the IPTV STB the selected devices both for the Austrian and Greek pilot sites are the:

• ADB-3800W and • ADB-3810-TW or Philips NetTV respectively

The selection of the medical devices to be used within the HERA platform is imposed by the selected services. Based on a market and technology survey the following devices were selected:

• UA-767PBT blood pressure monitor and • UA-321PBT body weight scale

Regarding communication of the Bluetooth enabled medical devices with the HERA AS, this will be done through the realisation of a custom Linux based Wireless Access Point. As far as the HMIs for TV-based applications are concerned, the Web4CE framework (which is supported by important industry consortia such as DLNA and CEA) will be used providing a sound technical solution. Additionally, the HERA network architecture (Figure 2) was specified and its components were presented (Figure 3), focusing on the HERA AS functional specification, which is the main component of the HERA system. Finally the test and validation plan has been defined. Verification activities such as Unit testing, Integration testing, and System testing enable the evaluation of the HERA components, by determining whether the outputs of a given development phase satisfy the requirements established before the start of that phase. On the other hand, Validation activities as in Acceptance testing will confirm that the HERA system meets its intended use.

ANNEX A: Page 86 of 95


REFERENCES 1. Bellifemine, F., A. Poggi, and G. Rimassa. Developing Multi-Agent Systems with

JADE. In C. Castelfranchi and Y. Lespérance, editors, Intelligent Agents VII. Agent Theories, Architectures, and Languages-7th. International Workshop, ATAL-2000, Boston, MA, USA, July 7–9, 2000, Proceedings, Lecture Notes in Artificial Intelligence. Volume 1986/2001, Springer-Verlag, Berlin, 2001.

2. Bordini R., Braubach L., Dastani M., El Fallah Seghrouchni A., Gomez-Sanz J.J., Leite J., O’Hare G., Pokahr A., Ricci A., « A Survey of Programming Languages and Platforms for Multi-Agent Systems », Informatica, No. 30, 2006, p. 33-44.

3. Kakas A, Moraitis P, Argumentation based decision making for autonomous agents. In Proc of the 2nd Int Conf on Auton Agents and Multi-Agent Syst, Melbourne, Australia, July 14-18, 2003

4. Kakas A, Moraitis P, (2002) Argumentative Agent Deliberation, Roles and Context. Electronic Notes in Theoretical Computer Science 70(5):39-53

5. Moraitis P, Spanoudakis N (2007) Argumentation-based Agent Interaction in an Ambient Intelligence Context. IEEE Intell Syst 22(6):84-93

6. Prakken, H. and Vreeswijk, G.: Logics for Defeasible Argumentation, in: Handbook of Philosophical Logic, volume 4, D. Gabbay and F. Guenthner, eds., Kluwer Academic Publishers, 2002, pp. 218-319.

7. Deliverable D 2.1 “State-of-the-art and Requirements Analysis” HERA Project (AAL-45061) February 2010

8. Spanoudakis N., Pendaraki K. and Beligiannis G.. Portfolio Construction Using Argumentation and Hybrid Evolutionary Forecasting Algorithms. International Journal of Hybrid Intelligent Systems (IJHIS), Vol. 6, No. 4, 2009, pp. 231-243

9. CEA, ANSI/CEA-2014 Web-based Protocol and Framework for Remote User Interfaces on UPnP Networks and the Internet (Web4CE), June 2006, http://www.ce.org/standards/StandardDetails.aspx?Id=2865&number=CEA-2014

10. Digital Living Network Alliance, http://www.dlna.org/en/industry/home 11. W3C, CSS TV Profile 1.0, May 2003, http://www.w3.org/TR/css-tv 12. UPnP Forum, http://www.upnp.org/ 13. McLellan D., Very Dynamic Web Interfaces,

http://www.xml.com/pub/a/2005/02/09/xml-http-request.html



ANNEX A - CANDIDATE SET-TOP-BOXES

Company Product OS Web4CE (CEA-2014)

Browsers Interface Pop-up windows

Activation from stand-

by mode

site

ADB-2500W (Advanced, standard

definition IPTV set-top box)

Linux, ADB JavaScript

API for IPTV

Mozilla RJ-45, HDMI, USB,

S-Video, RCA x 3

ADB-2810W (High definition AVC IPTV

set-top box with home networking capabilities)


API for IPTV


SCART, RCA,

S-Video(O)

ADB-3800/10TW (Advanced, high

definition IPTV set-top box with home

networking capabilities)


API for IPTV


SCART, S-Video(O) RCA x 3,

DVB

ADB-3800 W (Advanced, high

definition IPTV set-top box with home

networking capabilities)


API for IPTV, Compatible with MS Windows CE

5.0 BSP


RCA x 3




by mode

site

STB-77xx series Linux 2.6

Opera Nokia Siemens Network (IPTV

solution)

RJ-45, USB,

S-Video, SCART, HDMI,

IR-Remote

STB-7710 and STB-7711

Linux 2.6

Opera SCART, HDMI, USB,

RJ-45, IR-Remote

SceneGate 8000 Linux 2.6

Nokia Siemens Network- IPTV

solution Dreampark – Dreamgallery

Zappware – iView Tara Systems -

TVolution

USB, HDMI, RJ-45,

IR-Remote

SceneGate 8500 (HD IPTV with PVR)

Linux 2.6



Zappware – iView Tara Systems -

TVolution

USB x 2, HDMI, RJ-45,

IR-Remote

SceneGate 8800 (HD DVB/IPTV hybrid

STB)

Linux 2.6



Zappware – iView

USB x 2, HDMI, RJ-45,

IR-Remote



Tara Systems - TVolution




by mode

site

A110 (SD MPEG-2 IP-STB)

A110H (under desk)

Linux

RJ-45, RGB,

S-Video, USB,

Remote Control

A125 (Multi-codec IP-STB)

Linux

RJ-45, RGB,

S-Video, USB,

Remote Control

A130 (HD IP-STB)

A130H (under desk)

A130M (Digital HD IP-STB)

Linux

RJ-45, HDMI,

S-Video, USB,

Remote Control

A132 (HD IP-STB) Linux

RJ-45, HDMI,

S-Video, USB,

Remote Control

Mood400-020/22 Linux

RJ-45, HDMI,

SCART, S-Video, SPDIF,

USB




by mode

site

IP 89

IP 8950

IS 8920

IS 8936

IC 8935

IA 8925

Embedded Linux

Embedded Browser based on ANT

RJ-45, USB, HDMI,

Video RCA, Audio RCA,

S/PDIF

External IR Box and Remote

Control

IP 8956 Embedded Linux Embedded Browser based on ANT

USB




by mode

site

STMC-EC ( IPTV set top media center)

Linux 2.6.12

WebKit RJ-45, HDMI,

SCART, USB, RCA

STMC-XL ( Hybrid IPTV and DVB-T set

Linux 2.6.12

WebKit Rj-45, HDMI,



top media center) USB, SCART




by mode

site

IPT-110VHA Linux Embedded Browser based on ANT and Etcs. dHTML 4.01,

CSS /DOM, JavaScript

HDMI, SPDIF, RJ-45

IPT-600VHS2 Linux

Ant Galio or Opera HDMI, S-Video, SPDIF, RJ-45, Scart, USB




by mode

site

IME20 Embedded LINUX

Embedded Web Browser (ANT Galio)

for Linux HTML4.0

SSL Java script ver 1.5

RJ-45, Composite

IME34 Windows CE.NET 5.0

Internet Explorer for Windows CE 5.0,

HTTP 1.0/1.1 (http, https)

FTP, HTML 4.0, Cascade Style Sheet support

Jscript 5.5, VB Script 5.5

Active X control support, CSS1 &

CSS2

RJ-45, Composite,

USB, Serial

IME50 Embedded Linux / WinCE 5.0

Internet Explorer for Windows CE 5.0,

HTTP 1.0/1.1 (http, https)

FTP, HTML 4.0, Cascade Style Sheet support

Jscript 5.5, VB Script 5.5

Active X control support, CSS1 &

CSS2

VGA, HDMI, S-VHS,

Component, Composite,

SPDIF, RJ-45, USB


Embedded Web Browser (ANT

Browser) for Linux HTML 4.0 HTTP 1.1

JavaScript 1.5 CSS 1.0/2.0

HDMI, Component, Composite,

SPDIF, RS232,

USB

IME300 Customized WinCE.NET 4.2

Customized Brower / ?Internet Explorer

RS232, RGB,

Composite, Component,

RJ-45

IME600 WinCE 5.0 or Embedded XP

Internet Explorer V6.0

VGA, DVI,

Component, Composite,

SPDIF, RJ-45, USB


Embedded Web Browser (ANT

Browser) for Linux

Composite, Component,

HDMI,



HTML 4.0 HTTP 1.1

JavaScript 1.5 CSS 1.0/2.0

SPDIF, RJ-45

USB x 2




by mode

site

N8000 series Linux

Netgem TV Browser : HTML 4.01, ECMAScript

(JavaScript 1.5), CSS1&2, DOM1,

DOM2 Events

RJ-45, WiFi (O), USB x 2,

HDMI, DIN To SCART,

SPDIF, IR Controller

N7000 (Hybrid MPEG4 HD receiver)

Linux

Netgem TV Browser RJ-45, HDMI,

SCART, USB x 2,

IR Controller

N5000 (IP Player) Linux

Netgem TV Browser RJ-45, HDMI, SPDIF, USB x 2




by mode

site

Fontana Model MS CE 5.0, HTML 4.01 embedded Internet Explorer

browser

Internet Explorer 6.0 HDMI, Component,

S-Video, RF,

USb x 2, IR Controller


Browsers Interface

site

MediaPro IP3000 Linux

Opera RJ-45, Serial port,

HDMI

IP5000HD (old product),

IP6000HD

Windows XP

USB x 6, HDMI,

RCA SPDIF, RJ-45 x 2,

Mic in




by mode

site

DIT-Series Microsoft Mediaroom, SoftAtHome, Miview TV

HDMI, USB,

RJ-45, RCA Audio,

SPDIF, SCART

DIP-Series Microsoft Mediaroom, SoftAtHome, Miview TV

HDMI, USB,

RJ-45, RCA Audio,

SPDIF, SCART

DIS-Series Microsoft Mediaroom, SoftAtHome, Miview TV

HDMI, USB,

RJ-45, RCA Audio,

SPDIF, SCART






by mode

site

PeerStation 220 Linux

Extensive JavaScript based APIs & SDK

for GUI development & integration with

custom middleware

HDMI, Component,

SPDIF, S-Video,

Composite, Stereo RCA,

RJ-45, USB x 2 WiFi (O)


Extensive JavaScript based APIs & SDK

for GUI development & integration with

custom middleware

HDMI, Component,

SPDIF, S-Video,




HDMI, Component,

SPDIF, S-Video,






by mode

site

P.VU2000 Linux 2.6.x Windows CE.Net

option

Midlleware options: Pirelli OpenSet SDK

to let operators develop advanced

and customised services. Alacatel-Lucent OMP/OMC

and MiView TV. Microsoft

Mediaroom and Ericsson IAP

RJ-45, USB x 2, SCART, S-Video, HDMI, SPDIF,

2 x Stereo RCA, WiFi (O)

IP100 Linux 2.4.x Windows CE.Net

option

Opera 8.5, HTML, Cookies, HTTPS

RJ-45, RJ-11,

USB x 2, IR Controller,

WiFi (O), SCART x 2,

HDMI, S-Video,

RCA CVBS, SPDIF,

Stereo RCA




by mode

site

ITAD81 HD RJ-45, HDMI, SPDIF,

Component, Stereo RCA,

USB, SCART x 2

ITAD84 HD RJ-45, HDMI, SPDIF,

Component, Stereo RCA,

USB, SCART x 2

ITAD83 SD

RJ-45, HDMI, SPDIF,

Component,



Stereo RCA, USB,

SCART x 2




by mode

site

s-box 6600HD

built-in WEB Browser

S-Video, Composite,

HDMI, SPDIF, USB, LAN, WAN,




by mode

site

Tormado M53 Linux

WAN, S-Video, USB x 2,

Composite

Tormado M55 Linux

RJ-45, USB x 2,

HDMI, Composite, IR Controller

Tornado M60 Digital Media Center PC

Linux

Firefox and Konqueror

IR Controller, S-Video,

VGA, Stereo RCA,

RJ-45, USB x 6,

PCI, SATA,

ATA IDE, Serial,

Parallel, WiFi (O)




by mode

site

T 401 Linux 2.6

Ant Galio v2.2 S-Video, SPDIF, RJ-45, USB

T 402 Linux 2.6

HTTP 1.1, HTML 4.01, JavaScript

SCART, SPDIF, RJ-45, USB

T 501 Linux 2.6

Ant Galio v2.2 HDMI, S-Video, SPDIF, RJ-45, USB

T 502 Linux 2.6 HTTP 1.1, HTML 4.01, JavaScript




by mode

site

STB F8 Linux 2.4 Opera RJ-45, SCART, HDMI, SPDIF, USB x 2

Hybrid IP STB FS6 Linux 2.4 Opera SCART, RJ-45, SPDIF, USB x 2 DVB-T,C



HDMI Sh@rk SCART,

RJ-45, USB, HDMI, SPDIF

F7 WebTVBox MS OS Firefox, IE, Opera HDMI, USB




by mode

site

DBI2210 HTML 4.01, Javascript 1.3, JS extensions APIs SSL, CSS, DOM, Cookies

S-Video, SPDIF, HDMI, RJ-45,

USB x 2

DBI8500 HTML 4.01, Javascript 1.3, JS extensions APIs SSL, CSS, DOM,

Cookies

S-Video, SPDIF, HDMI, RJ-45,

USB x 2




by mode

site

MC-1088 Linux

Composite, S-Video,

Component, HDMI, SPDIF, RJ-45, USB




by mode

site

IPTV-100 (vPortal VP-

10)

Linux (MontaVista) Built-In Web Browser

RJ-45, Composite,

S-Video, Component,

HDMI, USB x 2

IPTV-120 (vPortal VP-

20)

Linux (MontaVista) Built-In Web Browser

RJ-45, Composite,

S-Video, Component,

HDMI, USB x 2 RS-232




by mode

site

WE-300 Windows CE.Net 5.0

Internet Explorer 6.0 RCA, Component,

SPDIF, USB x 2, RJ-45 x 2

WE-310 Embedded Linux

RCA, Component,

SPDIF, USB,

RJ-45 x 2

WE-700 Windows CE.NET Windows XPE

Embedded Linux

Internet Explorer 6.0

RCA, Component,

VGA, SPDIF,

USB x 2, RJ-45 x 2




by mode



site

Wyplay HD IPTV Box Linux

XHTML 1.0 , HTML 4.01,

CSS 2 Dom 1,2 Cookies

Javascript 1.5 (ECMAScript 262)

AJAX SVG, Canvas

XML Data parsing Javascript API for

application framework

RJ-45, USB,

WiFi (O), Bluetooth (O),

HDMI




by mode

site

Pisces101-E2 Linux

Oregan embedded web browser

RJ-45 x 2, USB x 2, S-Video, SPDIF,

Component, HDMI

Pisces102 Linux, Java


USB x 2, S-Video,

Composite, SPDIF,

RJ-45 x 2, HDMI

Pisces111W Linux 2.6.x.x


USB x 2, Component, Composite,

S-Video, SPDIF, HDMI,

RJ-45 x 2, 802.11n

Pisces121W Linux 2.6.x.x


USB x 2, Component, Composite,

S-Video, SPDIF, HDMI,

RJ-45 x 2, 802.11n

XTV125 Linux


USB x 2, S-Video,

Composite, SPDIF,

RJ-45 x 2, HDMI




by mode

site

STB-1001H Linux

XHTML 1.1, HTTPS

HTML 4.01 Javascript 1.5

Dom level2 CSS level2

Cookie Javascript API for


RJ-45, USB, HDMI, SPDIF

STB-1001S3 Linux XHTML 1.1, RJ-45, SCART,



HTTPS HTML 4.01

Javascript 1.5 Dom level2 CSS level2



SPDIF, USB

STB-1003 Linux

XHTML 1.1, HTTPS

HTML 4.01 Javascript 1.5

Dom level2 CSS level2



RJ-45, USB x 2,

Smartcard, SPDIF, CVBS,

S-Video, HDMI

Documents

HERA slnt wp2 d22 final - helios.mi.parisdescartes.frhelios.mi.parisdescartes.fr/~nikos/HERA_slnt_wp2_d22_final.pdf · AAL-45061 Deliverable D 2.2 Title: HERA Specifications and Validation