A Transformational Approach for Multimodal Web User Interfaces based on UsiXML Adrian Stanciulescu, Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte and Francisco Montero Université catholique de Louvain, School of Management (IAG) {stanciulescu, limbourg, vanderdonckt, michotte, montero}@isys.ucl.ac.be bjectives: eveloping multimodal web UI based on UsiXML pproach based on model-to-model graph transformations rogresivelly moves from Task&Domain to FUI onsists on four steps divided into sub-steps Hypotheses: Four modalities that involve four devices: keyboard, mouse,microphone, loud speakers No fusion/fission needed: independent interpretation/rendering for each modality is performed Modality type is exclusive: sequential and independent interaction Supported CARE properties: assignement and equivalence Step 1: The Task & Domain Models Graphical description of the models and the mappings between them IdealXML tool: automatic generation of UsiXML specification Transformation engine (TransformiXML) Transformation engine (TransformiXML) Concrete user Interface #1 (graphical) Concrete user Interface editor (GrafiXML) XHTML code XHTML browser XHTML Generator Abstract user interface #1 Domain model Task model … … Abstract user interface #2 Abstract user interface #n Concrete user Interface #1 (multimodal) X+V code IBM X+V browser XSL Transformation … Step 1 Step 2 Step 3 Step 4 General development scenario of UI Task & Concepts Abstract UI Concrete UI Final UI Task & Concepts:interactive tasks carried out by the end user & domain objects AUI: UI definition independent of any modality of interaction CUI: concretizes an AUI into CIOs (widget sets found in popular graphical and vocal toolkits) Context of use: user, computing platform, enviroment FUI: operational UI that runs on a particular platform either by interpretation or by execution Step 2: From Task & Domain Models to AUI Model Defining transformation systems that realize the transition from Task & Domain Models to AUI Model LHS RHS NAC Resultant AUI fragment www.similar.cc BCHI www.isys.ucl.ac.be/bchi www.usixml.org LHS RHS NAC Step 3: From AUI Model to CUI Model Defining transformation systems applied to realize the transition from AUI Model to CUI Model Four variants of CUI are considered: total graphic, predominant graphic, predominant vocal, total vocal Resultant CUI fragment Step 4: From CUI Model to FUI Model Transforming each variant of CUI into their respective FUI: •Total graphic •Predominant graphic •Predominant vocal •Total vocal XHTML genarator of GrafiXML tool XSLT to produce VoiceXML code XSLT to produce X+V code XSLT to produce X+V code Transformation rule mechanism: G: initial UsiXML specification LHS matches into G NAC doesn’t match into G RHS replaces LHS G is transformed into G’ (resultant UsiXML specification Specification of transformation: TransformiXML tool: Choose input file (models to transform) Select development path (starting and destination point) Edit rules in an XML editor Save the result in a UsiXML file

A Transformational Approach for Multimodal Web User Interfaces based on UsiXML Adrian Stanciulescu, Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte

Download PPT Report

Upload
hugo-chandler
View
213
Download
0

Embed Size (px)

Citation preview

Page 1: A Transformational Approach for Multimodal Web User Interfaces based on UsiXML Adrian Stanciulescu, Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte

A Transformational Approach for Multimodal Web User Interfaces based on UsiXMLAdrian Stanciulescu, Quentin Limbourg, Jean Vanderdonckt, Benjamin Michotte and Francisco Montero

Université catholique de Louvain, School of Management (IAG){stanciulescu, limbourg, vanderdonckt, michotte, montero}@isys.ucl.ac.be

Objectives:Developing multimodal web UI based on UsiXMLApproach based on model-to-model graph transformationsProgresivelly moves from Task&Domain to FUIConsists on four steps divided into sub-steps

Hypotheses:Four modalities that involve four devices: keyboard, mouse,microphone, loud speakers No fusion/fission needed: independent interpretation/rendering for each modality is performedModality type is exclusive: sequential and independent interactionSupported CARE properties: assignement and equivalence

Step 1: The Task & Domain ModelsGraphical description of the models and the mappings between them IdealXML tool: automatic generation of UsiXML specification

Transformation engine (TransformiXML)

Concrete userInterface #1(graphical)

Concrete userInterface editor

(GrafiXML)

XHTML code

XHTML browser

XHTML Generator

Abstract user interface #1

Domain modelTask model

…

Abstract user interface #2 Abstract user interface #n

Concrete userInterface #1(multimodal)

X+V code

IBM X+V browser

XSL Transformation

…

Step 1

Step 2

Step 3

Step 4

General development scenario of UI

Task & Concepts

Abstract UI

Concrete UI

Final UI

Task & Concepts:interactive tasks carried out by the end user & domain objects

AUI: UI definition independent of any modality of interaction

CUI: concretizes an AUI into CIOs (widget sets found in popular graphical and vocal toolkits)

nte

use

r, c

ting

pla

tform

, en

viro

FUI: operational UI that runs on a particular platform either by interpretation or by execution

Step 2: From Task & Domain Models to AUI Model

Defining transformation systems that realize the transition from Task & Domain Models to AUI Model

LHS

RHS

NAC Resultant AUI fragment

www.similar.ccBCHI

www.isys.ucl.ac.be/bchi www.usixml.org

LHS

RHSNAC

Step 3: From AUI Model to CUI ModelDefining transformation systems applied to realize the transition from AUI Model to CUI ModelFour variants of CUI are considered: total graphic, predominant graphic, predominant vocal, total vocal

Resultant CUI fragment

Step 4: From CUI Model to FUI Model Transforming each variant of CUI into their respective FUI:

•Total graphic•Predominant graphic•Predominant vocal•Total vocal

XHTML genarator of GrafiXML tool

XSLT to produce

VoiceXML code

XSLT to produce X+V codeXSLT to produce X+V code

Transformation rule mechanism: G: initial UsiXML specification

LHS matches into GNAC doesn’t match into GRHS replaces LHS

G is transformed into G’ (resultant UsiXML specification

Specification of transformation:

TransformiXML tool:Choose input file (models to transform)Select development path(starting and destination point)Edit rules in an XML editorSave the result in a UsiXML file