Nikolaos Kaklanis - PhD presentation

University of Surrey Department of Computing

A COUPLED USER AND TASK MODELLING

METHODOLOGY FOR ACCESSIBLE PRODUCT DESIGN

PhD Thesis

Nikolaos Kaklanis

Information & Communication Systems Engineer, MSc


• Disability is part of the human condition– Almost everyone will be temporarily or permanently impaired

at some point in life. • Aging is strongly connected with difficulties in

functioning• The lowest estimate, based on the currently defined

disablement categories, estimates their total number at around 74 Million persons, in the European Union.

• Many products & services are inaccessible for people with functional limitations

The need: Accessible & ergonomic products and services for the elderly and disabled

THE NEEDTHE NEED


TYPICAL DEVELOPMENT CHAINTYPICAL DEVELOPMENT CHAIN


DEVELOPMENT CHAIN USING DEVELOPMENT CHAIN USING VIRTUAL USER MODELSVIRTUAL USER MODELS

The

Context of the introduced research


A new coupled user-task modeling methodology

is needed, which will enable– the detailed description of users with disabilities– the development of Virtual User Models (VUMs)

representing specific population groups

PROBLEM IDENTIFICATIONPROBLEM IDENTIFICATION


• A novel Virtual User Model (VUM) enabling the detailed description

of the physical, cognitive & behavioural user characteristics with special focus on the elderly and people with functional limitations.

• An innovative methodology (based on statistical analysis) for the development of Virtual User Models that represent specific disabled population groups.

• A framework for the adoption of virtual user models within accessibility simulation and ergonomy assessment processes of virtual designs.

• Design and development of tools for supporting and putting the aforementioned methodologies and frameworks into practice.

RESEARCH CONTRIBUTIONRESEARCH CONTRIBUTION


STATE OF THE ART IN VIRTUAL USER MODELINGSTATE OF THE ART IN VIRTUAL USER MODELING

• Ontology-based models

– OntobUM (Razmerita et al., 2003)

– GUMO (Heckmann, 2006)

– VICON User Model (Pierre et al., 2011)

• XML-based models

– UserML (Heckmann, 2003)

– GUIDE User Model (Biswas & Langdon, 2012)

– User Models using relational databases

– MyUI User Model (Peissner et al. 2012)

• Personas (Cooper, 1999; Goodwin, 2002; Nielsen, 2002 & 2003; Pruitt & Grudin, 2003)

• Task modeling approaches

• ConcurTaskTree (Paternò, 1999)

• Hierarchical Task Analysis (HTA) (Annett & Duncan, 1967)

• UsiXML (Vanderdonckt, 2005)


LIMITATIONS OF EXISTING APPROACHESLIMITATIONS OF EXISTING APPROACHES

Existing approach/methodology Limitations Personas Not expressed in a machine-readable format

Focused on specific body parts

Cannot represent in detail physical, cognitive & behavioural characteristics

VUMs in machine-readable format

Disabilities not supported

Focus on the progression of the disability User modelling methodologies Cannot create VUMs representing specific population groups User is defined independently from tasks User & task modelling

methodologies It’s not clear which tasks are affected by the disabilities


• Introduction

•An innovative user modeling methodology focused on the elderly and disabled

• A methodology for creating virtual user models representing specific population groups

• The proposed user modeling methodology in practice

• Conclusions

An innovative user modeling methodology focused on the

elderly and disabled


The proposed framework is based on the following seven

major building blocks:

• Abstract User Models– high level description of potential user models

• Generic Virtual User Models– a Generic Virtual User Model (GVUM) describes a set of users having a

specific set of disabilities

• Instance of a Generic Virtual User Model– an instance of a virtual user (e.g., Persona).

AN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLEDAN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLED


• Primitive Tasks

– primitive human actions

• Task Models – actions that are being systematically performed in the context of the virtual

prototype to be tested

• Multimodal Interaction Models – alternative ways of a primitive task’s execution

• alternative modalities • assistive devices

• Simulation Models– simulation scenario

AN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLEDAN INNOVATIVE USER MODELING METHODOLOGY FOCUSED ON THE ELDERLY & DISABLED


ARCHITECTURE OF THE PROPOSED FRAMEWORKARCHITECTURE OF THE PROPOSED FRAMEWORK


• Primitive tasks

– define primitive human actions – are related to the disability

category

• limited but sufficient number of primitive tasks

PRIMITIVE TASKSPRIMITIVE TASKS

Primitive task’s category

Primitive task

Motor Push

Motor Grasp

Cognitive Select

Cognitive Read


• User tasks are divided in two categories:

– a) primitive (e.g., grasp, pull, walk, etc.)– b) complex (e.g., driving, telephone use, computer use, etc.)

• For each complex task, a Task Model is developed– to specify how the complex task can be analyzed into

primitive tasks.

TASK MODELSTASK MODELS

COMPLEX TASK PRIMITIVE TASK BODY PART OBJECT

CLOSE CAR DOOR WHILE

SEATED

REACH ARM DOOR

GRASP HAND INTERIOR DOOR

HANDLE

PULL HAND INTERIOR DOOR

HANDLE

PUSH HAND LOCK BUTTON


• UsiXML’s taskModel

TASK MODELS - IMPLEMENTATIONTASK MODELS - IMPLEMENTATION

<?xml version="1.0" encoding="UTF-8"?>

<taskmodel>

<task id="st0task0" name="Close_car_door_while_seated" type="abstraction">

<task id="st0task1" name="Reach_door_with_arm" type="interaction"/>

<task id="st0task2" name="Grasp_interior_door_handle_with_hand" type="interaction"/>

<task id="st0task3" name="Pull_interior_door_handle_with_hand" type="interaction"/>

<task id="st0task4" name="Push_lock_button_with_hand" type="interaction"/>

</task>

……

</taskmodel>

UsiXML source code


• Alternative ways of a primitive task’s execution with respect to

• the different target user groups• the replacement modalities• the possible use of assistive devices.

MULTIMODAL INTERACTION MODELSMULTIMODAL INTERACTION MODELS

Task Body

part

Modality Task

object

Disability Alternative

task(s)

Body part Alternative

modality

Alternative

task object/

assistive

device

Pull Hand

Motor Interior

door

handle

Upper limb

impaired

Speak Mouth Voice control Voice

activated

doors

Push Hand/Elbo

w

Motor Button that

closes door



MULTIMODAL INTERACTION MODELS - IMPLEMENTATIONMULTIMODAL INTERACTION MODELS - IMPLEMENTATION

<?xml version="1.0" encoding="UTF-8"?><taskmodel> <task id="st0task0" name="Pull_interior_door_handle" type="abstraction"> <task id="st0task1" name="Pull(modality:motor)(means:hand)(object:interior_door_handle)" type="interaction"/> <task id="st0task2" name="Speak(modality:voice)(means:mouth)(object:voice_activated_doors)" type="interaction"/> <task id="st0task3" name="Push(modality:motor)(means:hand,elbow)(object:button_that_closes_door)" type="interaction"/> </task> <deterministicChoice>

<source sourceId="st0task1"/> <target targetId="st0task2"/>

</deterministicChoice> <deterministicChoice>

<source sourceId="st0task2"/> <target targetId="st0task3"/>

</deterministicChoice></taskmodel>

UsiXML source code


• A high level description

of potential user models

ABSTRACT USER MODELS ABSTRACT USER MODELS

Disability

category Disability

Short

description Quantitative disability metrics

Functional

limitations

(ICF

Classification)

Age-

relate

d

Motor Spinal

cord

injuries

(Thoracic

injuries)

Spinal cord

injuries

cause

myelopathy

or damage to

nerve roots

or

myelinated

fiber tracts

that carry

signals to

and from the

brain.

Temporal gait variables:

-Gait Cycle (sec):2.17 (1.05)

-Cadence (steps/min): 65.0 (23.1)

-Double support (%): 42.8 (10.2)

-Stride (m): 0.48 (0.13)

-Velocity ((m/sec)/height): 0.27 (0.13)

S120 Spinal cord and related structures,

S1200 Structure of spinal cord,

S12000 Cervical spinal cord,

s12001 Thoracic spinal cord,

s12002 Lumbosacral spinal cord,

s12008 Structure of spinal cord, other specified,

s12009 Structure of spinal cord unspecified,

Could

be

Kinematic variables:

-Hip excursion (˚): 39.3 (9.0)

-Knee excursion (˚): 38.1 (13.2)

-Ankle excursion (˚): 25.0 (4.9)

-Hip velocity (˚/sec): 38.2 (17.5)

-Knee velocity (flexion) (˚/sec): 64.1 (41.8)

-Knee velocity (extension) (˚/sec): 83.8 (54.2)

-Ankle velocity (˚/sec): 48.1 (30.8)


• Refers to a class of virtual users exhibiting one or more

specific disabilities

• Describes the tasks affected by the disabilities and their associated disability-related parameters.

GENERIC VIRTUAL USER MODELSGENERIC VIRTUAL USER MODELS

Disability

category

Disability Affected primitive

tasks

Affected primitive tasks’ parameters

Grasp The user is able to grasp objects, with

size <= 3cm x 3cm x 3cm

Pull The user can pull an object with max_Force: 5N

Gait velocity ranges from 0.18 to 1.03 m/sec

Motor Hemiplegia

Walk

Abnormal step rhythm


GENERIC VIRTUAL USER MODELS – SUPPORTED CHARACTERISTICSGENERIC VIRTUAL USER MODELS – SUPPORTED CHARACTERISTICS

Anthropometric

Upper Limb (left)

HandFinger

(thumb)

Wrist

Forearm

Elbow

Shoulder

Lower Limb (left)

Hip

Foot Toe

Neck

Spinal Colulmn

Gait

Finger (baby)

Finger (ring)

Finger (index)Finger

(middle)

Thigh

Knee

Ankle

Upper Limb (right)

HandFinger

(thumb)

Wrist

Forearm

Elbow

Shoulder

Finger (baby)

Finger (ring)

Finger (index) Finger

(middle)

Foot Toe

Foot Toe

Foot Toe

Foot Toe

Lower Limb (right)

Hip

Foot Toe

Thigh

Knee

Ankle

Foot Toe

Foot Toe

Foot Toe

Foot Toe


GENERIC VIRTUAL USER MODELS - IMPLEMENTATIONGENERIC VIRTUAL USER MODELS - IMPLEMENTATION


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BASIC CONTAINERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BASIC CONTAINERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEAD ELEMENT)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEAD ELEMENT)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (DISABILITY MODEL)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (DISABILITY MODEL)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PREFERENCES)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GENERAL PREFERENCES)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (ANTHROPOMETRIC PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (ANTHROPOMETRIC PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (UPPER LIMB PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (UPPER LIMB PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (LOWER LIMB PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (LOWER LIMB PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (NECK PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (NECK PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPINAL COLUMN PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPINAL COLUMN PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GAIT PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (GAIT PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (VISUAL PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (VISUAL PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEARING PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (HEARING PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPEECH PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (SPEECH PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (COGNITIVE PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (COGNITIVE PARAMETERS)


GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BEHAVIOURAL PARAMETERS)GENERIC VIRTUAL USER MODELS – IMPLEMENTATION (BEHAVIOURAL PARAMETERS)


INSTANCES OF GENERIC VIRTUAL USER MODELSINSTANCES OF GENERIC VIRTUAL USER MODELS

• A specific virtual user with specific disability related parameters including:

– disabilities

– affected primitive tasks

User ID Disability

category

Disability Affected primitive

tasks

Affected primitive tasks’ parameters

Grasp The user is able to grasp objects, with

size <= 2.5cm x 2.5cm x 2.5cm

Pull The user can pull an object with max_Force: 3N

Gait velocity : 0.9 m/s

User 1 Motor Hemiplegia

Walk

Abnormal step rhythm


INSTANCES OF GENERIC VIRTUAL USER MODELS - IMPLEMENTATIONINSTANCES OF GENERIC VIRTUAL USER MODELS - IMPLEMENTATION

<?xml version="1.0" encoding="UTF-8"?><userModel xmlns="http://www.usixml.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema- instance" xsi:schemaLocation="http://www.usixml.org/spec/UsiXML-ui_model.xsd" id="User_Model" name="Adam Brown" creationDate="2013/02/02 20:31:06" schemaVersion="1.8.0"> <head> <version modifDate="2010/10/17 20:31:06">0.0</version> <authorName>Automatically generated byethe User Model Generator</authorName> <comment>This model has been generated using the User Model Generator</comment> </head> <disabilityModel> <disability name="Spinal cord injuries" ageRelated="true" typesICF="b147, b760, b7603, b780"> <disabilityDetails>Spinal cord injuries cause myelopathy or damage to nerve roots or myelinated fiber tracts that carry signals to and from the brain. Depending on its classification and severity, this type of traumatic injury could also damage the grey matter in the central part of the cord, causing segmental losses of interneurons and motorneurons. </disabilityDetails> <affectedTasks> <affectedTask idTask="walking_ID" type="motor" name="walking"

details="inability to effectively transfer weight between legs, abnormal step rhythm, excessive plantar flexion during swing phase, falling during activities" />

<affectedTask idTask="driving_ID" type="motor" name="driving" details="automatic transmission and buttons on the steering wheel instead of pedals for brake and accelerator needed" /> </affectedTasks> </disability> </disabilityModel> <capabilityModel> <general> <gender>male</gender> <ageGroup>30-49</ageGroup> </general> <generalPreferences /> <anthropometric> <weight measureUnits="kgr" value="81.023056"/> <stature measureUnits="cm" value="174.586060"/> <headLength measureUnits="cm" value="19.533623"/>…

GVUM: range of values

Instance of a GVUM: absolute values

while…


COMPARISON BETWEEN THE PROPOSED VUM & EXISTING MODELSCOMPARISON BETWEEN THE PROPOSED VUM & EXISTING MODELS

Number of variables of the proposed VUM

Number of variables of the GUIDE VUM

Number of variables of the MyUI VUM

Number of variables of the VICON VUM

• total: over 250 • total: 27• included in the

proposed VUM: 21

• total: 30• included in the

proposed VUM: 10

• total: 33• included in the

proposed VUM: 11

Why some variables of the others VUMs do not exist in the proposed VUM?

• they describe qualitative (not quantitative) characteristics and, thus, cannot be directly simulated (e.g., ability to move the hand precisely)

• describe the result of the simulation process (e.g., number of successful interactions with the system)

• are not actually user characteristics (e.g., amount of ambient light at the users place)


• The scenario to be followed during the simulation process

• A Simulation Model may include three different types of tasks: – a) abstract tasks

– b) interaction tasks

– c) application tasks

SIMULATION MODELSSIMULATION MODELS

Scenario Main tasks Subtasks

Pull handbrake Use handbrake

Release handbrake

Open storage

compartment

Automotive simulation:

assess the accessibility of the

handbrake and the storage compartment Use storage

compartment

Close storage compartment



SIMULATION MODELS - IMPLEMENTATIONSIMULATION MODELS - IMPLEMENTATION


• Introduction




• Conclusions

• The proposed methodology offers• detailed definition of

• user (including disabilities)• user tasks• simulation scenarios

• Mapping/linkage between all these definitions


• Introduction




• Conclusions

A methodology for creating virtual user models

representing specific population groups


1. Selection of disability parameters

– trying to model the most characteristic functional limitations (Abstract User Model) caused by each disability

• e.g., disability parameters for Gonarthrosis (osteoarthritis of the knee) include: knee flexion/extension and gait parameters, like cadence, step length, etc.

2. Acquire raw data (real measurements)

3. Estimate the probability density function (pdf) of each disability parameter

• Parametric regression• Non-parametric regression• An innovative hybrid regression method able to handle small sample sizes• All the above regression methods are applied and the one with the smallest Mean

Squared Error (MSE) is chosen as the most suitable

4. Validation and optimization of the regression model– Bootstrap method (Adèr et al., 2008)

MODELING A DISABILITY – GENERAL STEPSMODELING A DISABILITY – GENERAL STEPS


STEP 2: MEASUREMENTS CONDUCTEDSTEP 2: MEASUREMENTS CONDUCTED

• Measurements were obtained from total 164 subjects

• Parkinson’s disease (17 subjects)

• Stroke (36 subjects)

• Multiple Sclerosis (20 subjects)

• Cerebral Palsy (17 subjects)

• Coxarthrosis (7 subjects)

• Gonarthrosis (5 subjects)

• Lower Limb Amputation (3 subjects)

• Elderly people (59 subjects)


STEP3: PARAMETRIC REGRESSIONSTEP3: PARAMETRIC REGRESSION

2

2

2

)(

22

1)(

x

exf

Distribution Type PDF Example

Gaussian Hip flexion of the elderly

Distribution Type PDF Example

Gaussian mixture Fingers’ extension of people with stroke

Distribution Type

PDF Example

Exponential Gait cycle of stroke patients

Distribution Type

PDF Example

Lévy Double support of stroke patients

Distribution Type

PDF Example

Uniform Gait velocity of patients with cerebral palsy

n

i

x

i

ii

i

ewxf1

2

)(

2

2

2

2

1)(

xk ex

kkxf

1

)()(

1)(

23

)(2

)(2)(

x

ecxf

x

c

elsewhere

baxabxf

,0

],[,1

)(


• PDF is unknown

STEP3: NON-PARAMETRIC REGRESSIONSTEP3: NON-PARAMETRIC REGRESSION

Kernel regression

Knee angular velocity of the stroke patients

Local polynomial multiple regression

Step width of the elderly

•Seems to follow a Lévy distribution. •This can be proved by examining the MSE resulting from parametric regression:

•Lévy: MSE: 0.00204•Gaussian: MSE: 0.00256•Gaussian mixture: MSE: 0.00290•Uniform: MSE: 0.00367•Exponential: MSE: 0.07297

•Seems to follow a Gaussian distribution.•This can be proved by examining the MSE resulting from parametric regression:

•Gaussian: MSE: 0.94464•Gaussian mixture: MSE: 1.02365•Uniform: MSE: 1.26263•Exponential: MSE: 1.29327•Lévy: MSE: 2.69961


• Sample size is important! • BUT…the task of taking real measurements from disabled people is

– very difficult – needs much resources

• Estimating the PDF of a disability parameter using a limited set of subjects could result in great variance.

We introduce the concept of hybrid regression that exploits– information found on the literature – real measurements– virtual measurements

STEP3: HYBRID REGRESSIONSTEP3: HYBRID REGRESSION

Solution


• The probability distribution followed by a disability parameter is

defined using an ε-contaminated class of priors, namely Γ. – The contaminated class controls the balance of the determinate

priors, , and other possible priors, , through an ε-level of weight.

• The ε-contaminated class is formulated as:

– where D is our initial data set and

– ε denotes the level of uncertainty

HYBRID REGRESSION – HYBRID REGRESSION – Ε-Ε-CONTAMINATED CLASS OF PRIORSCONTAMINATED CLASS OF PRIORS

s'0 sq'

)}(),,|()1()|({ 20 yqDyDy

Parametric regression on the original sample

User defined

We tested the following values:

0, 0.01, 0.05, 0.10, 0.20

Gaussian


• Let variable Y represent the real measurements

• Assumption: Y is be the response of the simple linear regression model

– where is the predictor value

– is the intercept,

– is the slope and

– are independent and identically distributed random variables from

HYBRID REGRESSION – LINEARITY ASSUMPTIONHYBRID REGRESSION – LINEARITY ASSUMPTION

iii XY 1

iX0

1

i ),0( 2N


• Our initial data set consists of measurements , where:

– each is a real measurement

– ‘s are calculated as follows:• from Gaussian pdf found in the literature

– to keep a relevance between ‘s and ‘s, we get random samples ( ‘s) that meet

• When there is no available information in the literature, we calculate the ‘s using the following formula:

With this process, ‘s are actually coming from a random movement (within a specific range) of the initial samples ‘s.

HYBRID REGRESSION – INITIAL DATA SETHYBRID REGRESSION – INITIAL DATA SET

D n niXY ii ,...1),,(

iY

iX),( 2

litlitN

iY iXiX

410 YYX litii

iX

kY

kYYX

ii

iXiY

Random value within [0.95, 1.05]


• A virtual data set consisting of m pairs is constructed

as follows:– ‘s are random samples from the pdf of ‘s.

– ‘s are random samples of , assuming that is a Gaussian distribution with

and

where and

HYBRID REGRESSION – VIRTUAL SAMPLESHYBRID REGRESSION – VIRTUAL SAMPLES

AD

AjX iXA

jY 0

AA

YXYA

01

00

n

iiiY

xyn

DMSEA

1

201

00

2 )(1

)(

n

i

n

iii

n

ii

n

ii

n

iii

Xn

X

YXn

YX

1

2

1

2

11101

1

1

XY 01

00

),( Aj

Aj XY

User-selected valueWe tested the following values:0, 0.5n, n, 2n, 3n


• At this stage, we have available:

– The initial data set

– A virtual data set

• Both data sets are used for the fine tuning of the parameters of

, which describes the distribution of disability parameter .

• Now, for the definition of , we perform Gaussian parametric regression on the real measurements (Yi’s).

HYBRID REGRESSION – FINE TUNING OF PARAMETERS OF HYBRID REGRESSION – FINE TUNING OF PARAMETERS OF ΓΓ

AD

D

y

0

This is what we are searching for…


• The maximum likelihood estimate taken into account both the

original and the virtual sample sets ( and , correspondingly) is given by:

• Using the EM algorithm for parameter estimation concludes after k iterations with and , which are used for the optimisation of the Gaussian distribution . – More specifically,

• now becomes

• and becomes

HYBRID REGRESSION – EM ALGORITHMHYBRID REGRESSION – EM ALGORITHM

D AD

mn

ni

Ai

n

iiMLE DyDy

11, )|()|(maxarg

k0 k

1

0

AA

YXYA

01

00 AkkA

YXYA 10

n

iiiY

xyn

DMSEA

1

201

00

2 )(1

)(

n

ii

kkiY

xyn

DMSEA

1

210

2 )(1

)(


• The combination of

– ε (uncertainty) and – m (number of virtual samples)

that results to the minimum MSE is used to form the final Γ.

HUBRID REGRESSION – RESULTED HUBRID REGRESSION – RESULTED ΓΓ

)}(),,|()1()|({ 20 yqDyDy


HYBRID REGRESSION EXAMPLE - STEP WIDTH OF THE ELDERLYHYBRID REGRESSION EXAMPLE - STEP WIDTH OF THE ELDERLY

Purple line: data taken from the MSP

Green line: virtual samples generated considering the MSP data.

Yellow line: virtual samples generated considering the data coming from the literature

Light blue line: pdf reported in the literature

Dark blue line: represents all the samples (both real and virtual)

Red line: final estimated pdf

Regression type MSE

Parametric – Gaussian 0.74377

Parametric – Lévy 2.73860

Parametric – Gaussian mixture

0.91292

Parametric – Exponential 1.46577

Parametric – Uniform 1.13034

Nonparametric – Kernel 0.5102

Nonparametric – Polynomial

0.6414

Hybrid 0.00642


• The definition of , enables the calculation of disability parameter values for different population groups. – e.g. , to find the value of wrist flexion that corresponds to the

90% of people with arthritis, we have:

CALCULATING THE DISABILITY PARAMETER VALUES FROM THE ESTIMATED PDFCALCULATING THE DISABILITY PARAMETER VALUES FROM THE ESTIMATED PDF

b

a

jijiijijji aFbFdxxf )()()( ,,,

jiF ,

9.0)0()( ,, onwristFlexiarthritisonwristFlexiarthritis FkF

PDF of parameter j of disability i

Finite integral:

The value of wrist flexion corresponding to the 90% of

people with arthritis


REGRESSION MODELS - INDICATIVE RESULTS (STROKE PATIENTS)REGRESSION MODELS - INDICATIVE RESULTS (STROKE PATIENTS)

Parameter Name

Before validation & optimisation After optimisation

Regression Type

Distribution Type PDF

Parameters Distribution Type

PDF Parameters

Gait Cycle Parametric

Levy

23

)(2

)(2)(

x

ecxf

x

c

μ = 0,918

c = 0,29893

Gaussian

2

2

2

)(

22

1)(

x

exf

μ = 1,81818

σ = 0,609692

Push Force

Hybrid )()()1()( 0 xqxxf

Π0 (Gaussian):

μ=65,47 σ=67,54

q (Exponential):

λ=0,02 ε = 0,05

Gaussian

2

2

2

)(

22

1)(

x

exf

μ = 78,5959

σ = 61,9811

Hip Extension

Right Parametric

Gaussian Mixture

n

i

x

i

ii

i

ewxf1

2

)(

2

2

2

2

1)(

μ1=32,81111 σ1=5,10935 w1=0,34615

μ2=15,82353 σ2=4,21936 w2=0,65385

Gaussian Mixture

n

i

x

i

ii

i

ewxf1

2

)(

2

2

2

2

1)(

μ1 = 32,597 σ1 = 5,682615 w1 = 0,343074

μ2 = 15,86191 σ2 = 4,316446 w2 = 0,656926


• Introduction




• Conclusions

The proposed user modeling methodology in practice


• VERITAS simulation

platform • Automatic simulated

accessibility and ergonomy assessment of designs

• Its development was based on the proposed user modeling methodology

ACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNSACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNSPhysical characteristics Normal values Elderly (60 to 84) Rheumatoid arthritis

Hand maximum pull force (N) 335 76.8

Wrist extension (°) 0 - 60

Wrist radial deviation (°) 0 - 27.5 0 – 19

Wrist ulnar deviation (°) 0 – 35 0 – 26

Forearm supination (°) 0 – 85 0 – 74

Forearm pronation (°) 0 – 85 0 – 71

Elbow flexion (°) 0 – 142.5

Elbow hyper-extension (°) 0 – 10 0 – 4

Shoulder flexion (°) 0 – 160 0 – 10

Shoulder abduction (°) 0 – 85 0 – 67 0 – 15

Shoulder internal rotation (°) 0 – 80 0 – 63

Shoulder external rotation (°) 0 – 45 0 - 15

Spinal column flexion (°) 0 - 90 0 – 23.6

Spinal column extension (°) 0 – 30 0 - 17

Spinal column left lateral flexion (°) 0 – 25 0 – 19

Spinal column right lateral flexion (°) 0 – 25 0 - 20


ACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNS – STAPLER USEACCESSIBILITY AND ERGONOMY EVALUATION OF DESIGNS – STAPLER USE

Stapler torque-resistance (Nm) Normal Elderly Rheumatoid arthritis

2.5 Pass Pass Pass

5.0 Pass Pass Pass

10 Pass Pass Pass

15 Pass Fail Pass

20 Pass Fail Fail

25 Pass Fail Fail

30 Fail Fail Fail


• DIAS tool: Impairments

simulator

• It has been extended to support the proposed VUMs.

IMPAIRMENT SIMULATION OVER JAVA, MOBILE AND WEB APPLICATIONSIMPAIRMENT SIMULATION OVER JAVA, MOBILE AND WEB APPLICATIONS


• ACT-R: a cognitive

architecture showing how human cognition works.

• The proposed VUM includes ACT-R parameters.

• Reading task example – a non-stressed user

• visual-attention-latency: 0.085 sec

• reading task – total time: 3.57 sec

– a user with acute stress• visual-attention-latency: 0.0978 sec

• reading task – total time: 3.726 sec

COGNITIVE SIMULATION – THE PROPOSED VUMs WITHIN ACT-RCOGNITIVE SIMULATION – THE PROPOSED VUMs WITHIN ACT-R


• Introduction

•A new User Model able to describe elderly and disabled people




• Conclusions

Conclusions – Future Work


1. A new Virtual User Model was introduced, able to describe a variety of

user characteristics, including physical, cognitive and behavioural.

2. This Virtual User Model is a part of an innovative coupled user and task modeling methodology that aims to be used mainly in simulation frameworks performing accessibility assessment of virtual designs.

3. A methodology for creating virtual user models able to represent specific disabled population groups

4. Tools for putting the proposed user modeling methodology into practice

CONCLUSIONSCONCLUSIONS


• The proposed VUM goes one step beyond the state-of-the-art, as it

enables the detailed description of the functioning of the elderly and disabled

• The proposed coupled user & task modeling methodology can be a valuable tool for the designers/developers towards the development of accessible products

• The proposed methodology for creating virtual user models able to represent specific disabled population groups offers designers with the opportunity to define an accessibility threshold for their designs.

IMPORTANCE OF THE RESEARCH CONTRIBUTIONIMPORTANCE OF THE RESEARCH CONTRIBUTION


• The accuracy of a VUM representing a specific population group

depends on the sample size.

• However, the proposed methodology can result to the production of very accurate personas.

CRITICAL DISCUSSIONCRITICAL DISCUSSION


• Fine-tuning of the resulted regression models using

• more data coming from the literature

• more real measurements

• More disabilities could be modeled

• A coupled physical-cognitive user modeling approach could be considered

• Continue the dissemination through– the VUMS cluster of projects (http://www.veritas-project.eu/vums/) – the activities of the W3C MBUI Working Group

FUTURE WORKFUTURE WORK


1. Kaklanis, N., Moschonas, P., Moustakas, K., Tzovaras, D., “Virtual User Models for the elderly and disabled

for automatic simulated accessibility and ergonomy evaluation of designs”, Universal Access in the Information Society, Special Issue: Accessibility aspects in UIDLs, Springer

2. Giakoumis, D., Kaklanis, N., Votis, K., Tzovaras, D., “Enabling user interface developers to understand accessibility limitations through visual, hearing, physical and cognitive impairment simulation”, Universal Access in the Information Society, Springer

3. Kaklanis, N., Votis, K., Tzovaras, D., ”Open Touch/Sound Maps: A system to convey street data through haptic and auditory feedback”, Computers & Geosciences, Elsevier (accepted for publication)

4. Kaklanis, N., Votis, K., Tzovaras, D., “An online multimodal audio-haptic framework for visually impaired users accessing OpenStreetMap data”, Universal Access in the Information Society, Special Issue: 3rd generation accessibility: Information and Communication Technologies towards universal access, Springer (under review)

5. Kaklanis, N., Biswas, P., Mohamad, Y., Gonzalez, M.F., Peissner, M., Langdon, P., Tzovaras, D., Jung, C, “Towards Standardization of User Models for Simulation and Adaptation Purposes”, Universal Access in the Information Society, Special Issue: 3rd generation accessibility: Information and Communication Technologies towards universal access, Springer (under review)

6. Koutkias, V., Kaklanis, N., Votis, K., Tzovaras, D., Maglaveras, N., “An Integrated Semantic Framework Supporting Universal Accessibility to ICT”, Universal Access in the Information Society, Special Issue: 3rd generation accessibility: Information and Communication Technologies towards universal access, Springer (under review)

7. Kaklanis, N., Stavropoulos, G., Tzovaras, D., “Modeling disabilities through regression analysis for the development of accurate virtual user models”, User Modeling and User-Adapted Interaction, The Journal of Personalization Research (under review)

PUBLICATIONS – SCIENTIFIC JOURNALSPUBLICATIONS – SCIENTIFIC JOURNALS


8. Kaklanis, N., Votis, K., Moustakas, K., Tzovaras, D. 3D HapticWebBrowser: Towards Universal Web

Navigation for the Visually Impaired, 7th International Conference on Web Accessibility, W4A, 26-27 April 2010, Raleigh, North Carolina, USA

This paper won the "Judges Award” at the “Web Accessibility Challenge” (sponsored by Microsoft)

9. Kaklanis, N., Moustakas, K., Votis, K., Tzovaras, D. A framework for accessibility testing of virtual environments based on UsiXML”, ACM SIGCHI Symposium on Engineering Interactive Computing Systems, UsiXML-EICS 2010, Berlin, June 2010

10. Kaklanis, N. Moschonas, P., Moustakas, K., Tzovaras, D., Enforcing accessible design of products and services through simulated accessibility evaluation, CONFIDENCE 2010, Jyväskylä, Finland, 9-10 December 2010

11. Kaklanis, N., Moustakas, K., Tzovaras, D. A framework for automatic simulated accessibility assessment in virtual environments, HCI International 2011, Orlando, Florida, USA, 9-14 July 2011

12. Kaklanis, N., Votis, K., Moschonas, P., Tzovaras, D. HapticRiaMaps: Towards Interactive exploration of Web World maps for the Visually Impaired”, W4A 2011, Hyderabad, India, March 2011

13. Oikonomou, T., Kaklanis, N., Votis, K., Kastori, G.E., Partarakis, N., Tzovaras, D. WaaT: Personalised Web Accessibility Evaluation Tool, W4A 2011, Hyderabad, India, March 2011

14. Oikonomou, T., Kaklanis, N., Votis, K., Tzovaras, D. An Accessibility Assessment Framework for Improving Designers Experience in Web Applications, HCII 2011, Orlando, Florida, USA, July 2011

15. Partarakis, N., Doulgeraki, C., Antona, M., Oikonomou, T., Kaklanis, N., Votis, K., Kastori, G.E., Tzovaras, D. A Unified Environment for Accessing a Suite of Accessibility Evaluation Facilities, HCII 2011, Orlando, Florida, USA, July 2011

16. Moschonas, P., Kaklanis, N., Tsakiris, A., Moustakas, K., Tzovaras, D., “An open simulation framework for automated and immersive accessibility engineering”, 6th Cambridge workshop on universal access and assistive technology, CWUAAT 2012, Fitzwilliam College, University of Cambridge, March, 2012

PUBLICATIONS – INTERNATIONAL CONFERENCESPUBLICATIONS – INTERNATIONAL CONFERENCES


17. Korn, P., Kaklanis, N., Votis, K., Tzovaras, D., “Using the AEGIS OAF: making an accessible RIA”, 27th

Annual International Technology and Persons with Disabilities Conference, CSUN 2012, March, 2012, San Diego, CA

18. Moschonas, P., Kaklanis, N., Tzovaras, D., “Novel Human Factors for Ergonomy Evaluation in Virtual Environments Using Virtual User Models”, 10th International Conference on Virtual Reality Continuum and its Applications in Industry, VRCAI’ 2011, December, 2011, Hong Kong, China

19. Kaklanis, N., Moustakas, K., Tzovaras, D., "An extension of UsiXML enabling the detailed description of users including elderly and disabled", in International Workshop on Software Support for User Interface Description Language, Interact 2011, Lisbon, September, 2011

20. Mohamad, Y., Biswas, P., Langdon, P., Peissner, M., Dangelmaier, M., Jung, C., Wolf P., Kaklanis, N., “Standardisation of user models for designing and using inclusive products”, Joint Virtual Reality Conference (JVRC), 20-21 September, 2011, Nottingham, UK

21. Kaklanis, N., Moustakas, K., Tzovaras, D., 2012. A methodology for generating virtual user models of elderly and disabled for the accessibility assessment of new products. In Proceedings of the 13th international conference on Computers Helping People with Special Needs - Volume Part I (ICCHP'12), Klaus Miesenberger, Arthur Karshmer, Petr Penaz, and Wolfgang Zagler (Eds.), Vol. Part I. Springer-Verlag, Berlin, Heidelberg, 295-302. DOI=10.1007/978-3-642-31522-0_44 http://dx.doi.org/10.1007/978-3-642-31522-0_44

22. Moschonas, P., Tsakiris., A., Kaklanis, N., Stavropoulos, G., Tzovaras, D., “Holistic accessibility evaluation using VR simulation of users with special needs”, UMAP 2012, Montreal, Canada, July, 2012

23. Kaklanis, N., Mohamad, Y., Peissner, M., Biswas, P., Langdon, P., Tzovaras, D., ”An Interoperable and Inclusive User Modelling concept for Simulation and Adaptation”, UMAP 2012, Montreal, Canada, July, 2012

24. Kaklanis, N., Votis, K., Tzovaras, D., “Touching OpenStreetMap Data in Mobile Context for the Visually Impaired”, CHI 2013 Mobile Accessibility Workshop, April 28, 2013, Paris, France (accepted)



25. Kaklanis, N., Votis, K., Tzovaras, D., “A Mobile Interactive Maps Application for a Visually Impaired

Audience”, The Paciello Group Web Accessibility Challenge - W4A 2013, 10th International Cross-Disciplinary Conference on Web Accessibility, 13-15 May 2013, Rio de Janeiro, Brazil (accepted)

26. Tsakiris, A., Kaklanis, N., Paliokas, I., Stavropoulos G., and Tzovaras, D. Cognitive Impairments Simulation in a Holistic GUI Accessibility Assessment Framework, 12th European AAATE Conference, 19-22 September 2013, Vilamoura, Algarve, Portugal (under review)



27. Kaklanis, N., Moustakas, K., Tzovaras, D. Haptic Rendering of HTML components and 2D maps included in

web pages, in G. Ghinea, F. Andres, S. Gulliver (Eds) Multiple Sensorial Media Advances and Applications: New Developments in MulSeMedia, 2010

28. Biswas, P., Kaklanis, N., Mohamad, Y., Peissner, M., Langdon, P., Tzovaras, D., Jung, C., “An Interoperable and Inclusive User Modeling concept for Simulation and Adaptation”, A Multimodal End-2-End Approach to Accessible Computing, Springer

PUBLICATIONS – BOOK CHAPTERSPUBLICATIONS – BOOK CHAPTERS


THANK YOU!

Next...

video presentations