UBIQUITOUS COMPUTINGLecture 13

Agenda• Model-based User Interface Design (MB-UID)

• User Interface Description Languages (UIDLs)

• MARIA (Modelbased lAnguage foR Interactive Applications)

• Ubiquitous computing• Migratory user interfaces• Plastic user interfaces

MODEL-BASED USER INTERFACE DESIGNMB-UID

MB-UID• Model-based user interface design - centered around a description

of application objects and operations at a level of abstraction higher than that of code

• A good model:• can be used to support multiple interfaces• help separate interface and application• describe input sequencing in a simple way,• check consistency and completeness of the interface• evaluate the interface’s speed-of-use• generate context-specific help and assist in designing the interface

• designers rarely use computer-supported application modeling today and prefer less formal approaches such as story boards of user interface prototypes

MB-UID• the quality of traditional UIs depends in a strong factor on the experience of the designers and

their skills in the platform and development tools

• the market is moving to web interfaces, ubiquitous systems, and wireless. More user interfaces

for every-day applications are needed in several devices (multiple user interfaces), also at the same time.

• Developing UIs of this kind of systems has extra constrains and constitutes a challenge per se.

• Different kinds of homogeneity among different platforms should be preserved in the User Interface. Otherwise, the user’s learning curve will force him to learn a new UI for each device he uses to access the system.

• the user interface can be specified once and then refined for each target devices until reaching the implementation

• Such approach can be supported by MB-UID methods and tools

• using an approach capable of generating the UI code, a high percentage of time and resources can be saved.

Advantages of MB-UID• The abstraction level is higher that working with development

environments. The specification is less dependent from the underlying technology.

• Better productivity. A percent of the final UI can be directly generated and used in the final system without any further changes.

• Better quality. Generated code has always the same elements in the same places. Repeating tasks exactly in the same way every time is easier for a machine than for humans, obtaining, in this way homogeneity. Conformity with standards is also easy to obtain by generating the code according to such standards.

• Less errors. Generated code contains zero generation errors (if the generator is robust and it has been thoroughly tested). Semantic errors can still appear but as a consequence of misunderstanding requirements or modeling misprints.

Advantages of MB-UID• Provides a precise Engineering process. Development

can be repeated, measured and tracked as a production line. Future projects can be estimated and scheduled with more precision based on previous data.

• Multiple device support: using generators for multiples devices, platforms and architectures.

• Less Time to Market. UIs can be built in less time allowing to put the product early in the market

Problems of M-BUID• Maintainability. Specifications and artifacts produced by

the tools should have easy maintenance: allowing to keep the changes in the specification or, alternatively, allowing a sort of reverse engineering from the code to the model.

• Scalability. Practitioners and designers need to apply these methods and tools to real-life problems. If the methods and tools have been poorly tested outside academic environments, is difficult to fine-tune the method and tools for scalability to guarantee the success in industrial scenarios.

Problems of MB-UID• Round Trip Problems. Code generated from a model has

more constrains and less choices that the allowed in the final target device. Frequently, some sort of tweaking or beautification must be applied to modify the final UI to fit the usage scenario. These out-of-the-model changes should be reapplied (ideally automatically) whenever the model changes and the UI needs to be regenerated accordingly.

• Integration with artists’ designs. Especially important for web sites and multimedia contents, artists have a relevant role in the developing of such systems. MB-UID artifacts must be integrated with this other source of elements to configure the final UI. To support this kind of development, special tools or conventions must be followed in both sides of the development.

Problems of MB-UID• Lack of standards like UML for software design or

architecture. There is no standard for UI development (something similar to UML)

• Lack of robust code generators. More code generators are needed to produce UIs for different kind of applications. Few tools are available on the market and are limited to specific context of usage.

Problems of MB-UID• Lack of integration with business logic. UIs per se are

not enough to build running systems. Good integration mechanisms with application’s functionality (business logic) are needed

• Lack of commercial tools supporting the methods. Few tools are available on the market. Practitioners interested in the field need to know them in first place, use and evaluate them to check if a tool fits their needs.

USER INTERFACE DESCRIPTION LANGUAGESUIDLs

UIDL (User Interface Description Languages)

• A User Interface Description Language (UIDL) is a formal language used in Human-Computer Interaction (HCI) in order to describe a particular user interface independently of any implementation

• meta-models in order to rigorously define the semantics of such a UIDL - adhere to the principle of separation of concerns

• meta-models: context of use (user, platform, environment), task, domain, abstract user interface, concrete user interface, usability (including accessibility), workflow, organization, evolution, program, transformation, and mapping

• a UI may involve different interaction modalities (e.g., graphical, vocal, tactile, haptic, multimodal), interaction techniques (e.g., drag and drop) or interaction styles (e.g., direct manipulation, form filling, virtual reality).

• A common fundamental assumption of most UIDLs is that UIs are modeled as algebraic or model-theoretic structures that include a collection of sets of interaction objects together with behaviors over those sets

UIDL• consists of a specification language that describes various aspects of a user

interface under development

• reduce the amount of time and effort needed to make user interfaces by providing useful abstractions and supporting automation of the design process

• allow the same interface description to be rendered on multiple platforms.

• beneficial for automatically generating interfaces that are personalized to individual users

• involves defining a• syntax (i.e. how these characteristics can be expressed in terms of the language)

• semantics (i.e., what do these characteristics mean in the real world)

• It can be considered as a common way to specify a UI independently of any target language (e.g., programming or markup) that would serve to implement this UI.

UIDLs• Development goals support:

• support for multi-platform • device independence,• modality independence ( it can be used not only for communication

through one channel or medium, but through several)• content delivery

UIDLs• Free

• eXtensible Interaction Scenario Language (XISL)• Device Independent Authoring Language (DIAL)• Extensible MultiModal Annotation Markup Language (EMMA)• Simple Unified Natural Markup Language (SunML)• TeresaXML• MariaXML• Software Engineering for Embedded Systems using aComponent-Oriented Approach (SeescoaXML)• User Interface Markup Language (UIML)• USer Interface eXtensible Markup Language (UsiXML)• eXtensible Interface Markup Language (XIML)

• Vendor:• MXML (Adobe) is used to describe UI layoutand behaviors, and Action Script for the FlexFramework.• Open Laszlo (Laszlo) is a XML-based languagefor rich Internet applications.• SISL (Lucent Technologies) is a XML-based language service logic that is shared across many different

user interfaces, including speech-based natural language interfaces.• XAML (Microsoft) is a markup language for declarative application programming for theWindows

Presentation Foundation.• XUL (Mozilla) is used to build feature-rich cross platform applications that can run connected

or disconnected from the Internet

Properties comparison of UIDLS

 

 Table 2 Properties Comparison of UIDLsUIL Models Methodology Tools Supported languages SupportedplatformsLevel Tags ConceptsDISL Presentation,dialogandcontrolSpecificationof a generic, platform-independentmultimodal UIRenderingengineVoiceXML, JavaMIDP, Java Swing,Visual C++Mobile andlimiteddevicesModellevel NotspecifiedHead element,interface classes(structure, style, behavior), state,generic widgetsGIML Presentation,dialog,anddomainSpecificationof a genericinterfacedescription.GITK (GeneralizedInterfaceToolkit)C++, Java, Perl NotspecifiedMeta-model15 tags Interface, dialog,widget, objectsISML Presentation, task,dialog,domainSpecificationof a generic UIdescriptionUnder constructionJava, Microsoftfoundation class, Javaswing classesDesktopPC, 3DscreenModellevel NotspecifiedMappings andconstrains, actionevents, meta-objects,display parts,controller parts,interaction definitionRIML There isnoinformationSpecificationof a generic UIdescriptionThere is noinformationXHTML, XFORMS,XEvents, WMLSmart phone, pda,Mobile,Desktop PcModellevelThere isnoinformationDialog, Adaptation,layout, elementSeescoaXMLTask,Presentation,dialogSpecificationof a generic UIdescriptionCCOM(BetaVersion1.0 2002)PacoSuiteMSC Editor Java AWT, Swing,HTML, java.microedition,applet, VoxML, WMLJuggler Mobile,desktop PC,Palm IIIModellevel NotspecifiedComponent, port,connector, contract, participant, blueprint,instance, scenario, pltform, user, deviceSunML Presentation,dialog,domainSpecificationof a generic UIdescriptionSunMLCompiler Java Swing,voiceXML, HTML,UIML,Desktop Pc, pdaModellevel14 tags Element, list, link,dialog, interface,generic events,synchronizationTeresaXMLPresentation, task,dialogSpecificationof a generic UIdescriptionCTTE Toolfor task ModelsTeresaMarkup: Digital TV,VoiceXML,XHTML/SVG, X+VProgramming: C#DigitalTV,Mobile,DesktopPC,Modellevel19 tags Mappings, models, , platform, task, input,outputUIML Presentation,dialog,domainSpecificationof a generic UIdescriptionUIML.net,VoiceXMLrenderer,WMLrenderer,VB2UMILHTML, Java, C++,VoiceXML, QT,CORBA,and WMLdesktop PC,a handhelddevice, tv,mobileModellevel50 tags interconnection of the user interface to business logic,servicesWSXL Presentation,dialog, domainSpecificationof a generic UIdescription Not specified HTML PC, Mobile phone,Modellevel12 tags CUI=XForms,WSDL,Mapping=XLangWorkflow=WSFL,Logic=XML eventXICL Presentation,dialog,Specificationof a generic UIdescriptionXICLSTUDIOHTML, ECMAScript,CSS e DOM.desktop PC Modellevel NotspecifiedComponent,structure, script,events, properties,interfaceXIML Presentation, task,dialog,domainSpecificationof a generic UIdescriptionXIMLSchemaHTML, java swing,WLMMobile,desktop PC,PDAModellevel32 tags Mappings, models,sub models,elements, attributesand relations betweenthe elements43 

43

                                                                                                                                                            

http://www.academia.edu/647242/A_theoretical_survey_of_user_interface_description_languages_Preliminary_results

UIDLs• UIML (www.uiml.org)• use ML (http:// www.uni-kl.de/pak/useML/)• MariaXML • Usi XML (www.usixml.org) • XIML (www.ximl.org).

MARIAModelbased lAnguage foR Interactive Applications

MARIA• MARIA XML:

• one language for the abstract description

• a number of platform-dependent languages that refine the abstract one depending on the interaction resources considered:• graphical form-based• graphical mobile form-based• vocal• digital TV• graphical direct manipulation• multimodal (graphical and vocal) for desktop and mobile• advanced mobile (with support for multi-touch and accelerometers)

MARIA • Data Model

• an abstract/concrete description of the underlying data model of the user interface, needed for representing the data (types, values, etc.) handled by the user interface

• The interactors composing an abstract (concrete) user interface can be bound to elements of a type defined in the abstract (respectively, concrete) data model

• The concrete data model is a refinement of the abstract one• The introduction of a data model also allows for more control over the admissible

operations that can be performed on the various interactors. • the data model is described using the XSD type definition language.• useful for: correlation between the values of interface elements, conditional

presentation connections, conditional layout of interface parts, and specifying the format of the input values.

• The dependencies between the state of an interactor and the data model imply that at runtime, if a data element bound to an interactor changes its value, this change has to be notified to the interactor for updating its state and vice versa

• The logic of this callback mechanism can be defined using the target technology constructs during the generation derived from the interface description

MARIA• Event Model.

• an event model has been introduced at different abstract/concrete levels of abstractions

• The introduction of an event model allows for specifying how the user interface responds to events triggered by the user• Property Change Events. These are events that change the status of some UI

properties. The handlers for this type of event are only change properties, which indicate in a declarative manner how and under which conditions property values change.

• Activation Events. These are events raised by activators, which are interactors with the purpose of activating some application functionality (e.g., access to a database or to a Web service). This type of event can have either change properties or script handlers (which have an associated generic script).

• The abstract definition of these events contains the information for the generation of the dynamic behavior of the final UI.

MARIA• Supporting Ajax Scripts, which Allow the Continuous Updating of

Fields.

• the possibility of supporting continuous fields updating at the abstract level. We have introduced an attribute to the interactors: continuosly updated= “true”[“false”]. The concrete level provides more detail on this feature, depending on the technology used for the final UI (Ajax for Web interfaces, callback for stand-alone application, etc.).

• It is worth noting that while at the abstract level a certain interactor has to support continuous dynamic updating of its values from a certain abstract data source, at the concrete level, the specific platform-dependent technology used to support such continuous updating of the interactor must be specified.

MARIA• Dynamic Set of User Interface Elements

• the possibility to express the need to dynamically change only a part of the UI - specified in such a way as to affect both how the UI elements are arranged in a single presentation, and how it is possible to navigate between the different presentations.

• The content of a presentation can dynamically change (this is also useful for supporting Ajax techniques)

• it is also possible to specify dynamic behavior that changes depending on specific conditions - use of conditional connections between presentations

MARIA• an interface is composed of :

• one data model • one or more presentations

• Each presentation is composed of:• Name• a number of possible connections• elementary interactors• interactor compositions

• The presentation includes a data model and a dialog model, which contains information about the events that can be triggered by the presentation in a given time.

• The presentation is also associated with a dialog model which provides information about the events that can be triggered at a given time.

• The dynamic behavior of the events, and the associated handlers, is specified using the CTT temporal operators (for example, concurrency, or mutually exclusive choices, or sequentiality, etc.)

MARIA – Abstract Description• describes a UI only through the semantics of the interaction,

without referring to a particular device capability, interaction modality or implementation technology

• an AUI - composed by various Presentations that groups model elements presented to the user at once.

• The model elements are of two types:•  Interactor - every type of user interaction object• InteractorComposition - groups together elements that have a logical

relationship

• two types of interactor compositions: • grouping • relation.

MARIA – Abstract Description• Interactor:

• an interaction object:• Selection: single choice and multiple choice• Edit: text edit, object edit, numerical edit, position edit• Control: navigator: navigate between different presentations; activator: trigger the activation of a

functionality

• only output object:• Object• Description• Feedback• Alarm• Text

• further refinement of each of these objects can be done only by specifying some platform-dependent characteristics, therefore it is specified at the concrete level

• all the interaction objects have associated events in order to manage the possibility for the user interface to model how to react after the occurrence of some events in their UI

MARIA – Concrete Description• provide a platform-dependent implementation language-

independent description of the user interface

• it assumes that there are certain available interaction resources that characterize the set of devices belonging to the considered platform

• It provides an intermediate description between the abstract description and that supported by the available implementation languages for that platform

MARIA – composing interfaces• Web services based applications - designers and developers often

have to compose existing functionalities and corresponding user interface specifications (annotations)

• Four main aspects have been identified:

• the abstraction level of the user interface description (abstract, concrete, implementation)

• the granularity of the user interface (single objects, groups of objects may be composed, presentations may be joined to obtain the UI)

• The types of aspects that are affected by the UI composition (the dynamic behavior, the perceivable UI objects or the data manipulated by the user)

• The time when the composition occurs (design time/runtime).

Case study• Home application - users can control some interactive

devices and appliances• Functionality:

• suggestions for searching a device through a text editing interactor (for example, the user enters a part of the device name and some suggestions for the completion appear)

• displaying information on a set of appliances in a part of the presentation while the user can dynamically add or remove elements from the appliance set

Case study• Specifying a web service

<aui:external_functions>

. . .

<aui:function name="getSuggestions" type="web_service">

<aui:output type="UserSession/suggestions" />

<aui:input type="xs:string" name="inputString"/>

</aui:function>

. . .

<aui:external_functions>

Case study• When the input string changes, the service must be called

<aui:text_edit id="device_search">

<aui:events>

<aui:value_change>

<aui:handler>

<aui:script>

<![CDATA[

data:UserSession/suggestions

=external:getSuggestions(ui:device_search.value); ]]>

[…]

<aui:handler>

<aui:change_property interactor_id="device_suggestions"

property_name="hidden" property_value="false" />

</aui:handler>

[…] </aui:text_edit>

The input text

The event

Web service call

Make results visible

Case study<aui:single_choice id="device_suggestions" data_reference="UserSession/suggestions">

<aui:events>

<aui:selection_change>

<aui:handler>

<aui:change_property interactor_id="device_select_activator" property_name="enabled"

data_value="true" />

<aui:change_property interactor_id="device_monitor_activator“ property_name="enabled" data_value="true" />

<aui:change_property interactor_id="device_search" property_name="value"

data_value="ui:device_suggestions.selected" />

<aui:change_property interactor_id="device_suggestions"

property_name="hidden" property_value="true" />

[…]

</aui:single_choice>

A single choice interactor is populated with the suggested devices

Case study• The concrete interfaces will be adapted to platform

capabilities (screen size, processor speed) and interaction technique (vocal, multi-touch, mouse)

• Differences:• Number of presentations• Different locations of the groups on the screen

Case study

Concrete interfaces for desktop and iPhone

MARIAE – MARIA Environment• provides a novel solution able to exploit task models

(represented in the ConcurTaskTrees notation) and user interface models (in the MARIA language) for the design and development of interactive applications based on Web services for various types of platforms (desktop, smartphones, vocal, multimodal, ...)

Gummy• a multiplatform graphical user interface builder that can

generate an initial design for a new platform by adapting and combining features of existing user interfaces created for the same application

• keeps all user interfaces consistent without requiring designers to considerably change their work practice

• can generate an initial design for a new platform by adapting and combining features of existing user interfaces created for the same application

• a computing platform - the combination of a hardware device, an operating system and user interface toolkit

Problems in multi-platform UI design approaches

• high cost incurred in adding a new target device and keeping all user interfaces consistent using manual approaches

• there is no clear separation between the user interface and the underlying application logic

• A common solution to these issues is to specify the user interface in an abstract way by means of high-level models such as task models and dialogue models

• The platform-specific user interfaces are then generated automatically from this abstract description

• The user interface has to be specified only once, which makes it easier to make changes or add a new target platform

Problems in multi-platform UI design approaches

• Issues:

• the resulting user interfaces usually still lack the aesthetic quality of a manually designed interface.

• The design process is not intuitive since designers have to master a new language to specify the high-level models and cannot accurately predict what the resulting user interface will look like

GUMMY• Gummy can generate an initial design for a new platform

from existing user interfaces created for the same application but for other platforms

• makes it it easy to target new platforms and keep all user interfaces consistent without requiring designers to considerably change their work practice

• Gummy allows for a true WYSIWYG4 multi-platform user interface design process.

GUMMY approach• Gummy combines the benefits of both the manual approach and model-

based techniques

• Designers create and perform prototyping of a multi-platform graphical user interface (GUI) in the same way as they do when dealing with traditional GUI builders such as Microsoft Visual Studio2 and Netbeans3.

• Gummy builds a platform-independent representation of the user interface and updates it as the designer makes changes

• This allows for an abstract specification of the user interface while keeping the design process intuitive and familiar

• An abstract user interface specification avoids a tight interweaving of application and presentation logic

GUMMY• The underlying language – UIML

• UIML contains:• a platform-independent user interface description - used to describe the structure,style,

content and behaviour of a user interface using platform-independent terms• a mapping vocabulary - contains mappings of the terms onto concrete widgets

• The traditional rendering step - translate the platform-independent user interface

• description into a concrete user interface using these mappings.• Gummy - the opposite approach: the concrete representations were used in

the tool and were internally mapped onto the associated abstractions.

• The designer works with the concrete graphical representations, avoiding the XML language, while the tool maintains a synchronised platform-independent UIML document for the concrete design.

GUMMY – mapping vocabulary

GUMMY – the process of creating UIs

GUMMY – the process of creating UIs

1. the user interface designer specifies the target platform for which they want to design a user interface (possible platforms are a mobile phone with the Compact .NET framework, a digital TV with DVB-MHP, etc.)

2. According to the specified platform, Gummy automatically loads a GUI builder workspace that is fully equipped for designing user interfaces for this platform.

3. From a set of existing user interfaces that are all created for the same application, Gummy automatically generates an initial design for the selected target platform – using a transformation engine component. When there are no existing user interfaces available, this step generates an empty user interface for the selected platform.

GUMMY – the process of creating UIs

4. The designer can refine the initial user interface until it fits their vision. The resulting design is then added to the set of existing user interface designs where it may serve as an input for the transformation engine. After this step, a new iteration is started.

5. When the designer is finished, Gummy exports all designs as one or more UIML user interface descriptions – can be rendered on the target platforms.

Damask• prototyping tool which targets web UIs that run on PCs and mobile

phones, and prompt-and-response style voice Uis

• designers sketch out their design for one device while using design patterns to specify higher-level concepts within their design

• Damask’s patterns include pre-built UI fragments that are already optimized for each device

• designers also use layers to specify which UI parts are common across devices and which are specific to one device

• Damask uses the sketches and patterns to generate designs for the other devices, which the designers can refine

UBIQUITOUS COMPUTINGUBICOMP

What is• "Ubiquitous computing names the third wave in

computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives." --Mark Weiser

Ubicomp• "a new way of thinking about computers in the world, one

that takes into account the natural human environment“

• Mark hoped to create a world in which people interacted with and used computers without thinking about them

• Ultimately, computers would "vanish into the background," weaving "themselves into the fabric of everyday life until they are indistinguishable from it.“

• ubiquitous computing demands that designers cultivate new skills, develop new ways of thinking about problems, and create new tools.

Basic concepts• More than 50 million personal computers have been sold, and

nonetheless the computer remains largely in a world of its own

• Computers - approachable only through complex jargon that has nothing to do with the tasks for which people actually use computers.

• The state of the art is perhaps analogous to the period when scribes had to know as much about making ink or baking clay as they did about writing

• a new way of thinking about computers in the world that takes into account the natural human environment and allows the computers themselves to vanish into the background

Basic concepts• "Ubiquitous computing" does not just mean computers that can be

carried to the beach, jungle or airport - still focuses attention on a single box (by analogy to writing, carrying a super-laptop is like owning just one very important book)

• ubicomp≠virtual reality (a world inside the computer)

• The goal for initially deploying the hardware of embodied virtuality: hundreds of computers per room

• these hundreds of computers will come to be invisible to common awareness.

• People will simply use them unconsciously to accomplish everyday tasks.

Ubicomp• Sociologically, ubiquitous computing may mean the decline of the

computer addict

• embodied virtuality will bring computers to the presidents of industries and countries for nearly the first time

• Computer access will penetrate all groups in society

• ubiquitous computers will help overcome the problem of information overload

• Machines that fit the human environment, instead of forcing humans to enter theirs, will make using a computer as refreshing as taking a walk in the woods

What is• "Ubiquitous computing names the third wave in

computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives." --Mark Weiser

Ubicomp• "a new way of thinking about computers in the world, one

that takes into account the natural human environment“

• Mark hoped to create a world in which people interacted with and used computers without thinking about them

• Ultimately, computers would "vanish into the background," weaving "themselves into the fabric of everyday life until they are indistinguishable from it.“

• ubiquitous computing demands that designers cultivate new skills, develop new ways of thinking about problems, and create new tools.

Basic concepts• More than 50 million personal computers have been sold, and

nonetheless the computer remains largely in a world of its own.

• Computers - approachable only through complex jargon that has nothing to do with the tasks for which people actually use computers.

• The state of the art is perhaps analogous to the period when scribes had to know as much about making ink or baking clay as they did about writing

• a new way of thinking about computers in the world that takes into account the natural human environment and allows the computers themselves to vanish into the background

Basic concepts• "Ubiquitous computing" does not just mean computers that can be

carried to the beach, jungle or airport - still focuses attention on a single box (by analogy to writing, carrying a super-laptop is like owning just one very important book)

• ubicomp≠virtual reality (a world inside the computer)

• The goal for initially deploying the hardware of embodied virtuality: hundreds of computers per room

• these hundreds of computers will come to be invisible to common awareness.

• People will simply use them unconsciously to accomplish everyday tasks.

Ubicomp• Sociologically, ubiquitous computing may mean the decline of the computer

addict

• embodied virtuality will bring computers to the presidents of industries and countries for nearly the first time

• Computer access will penetrate all groups in society

• ubiquitous computers will help overcome the problem of information overload. There is more information available at our fingertips during a walk in the woods than in any computer system, yet people find a walk among trees relaxing and computers frustrating

• Machines that fit the human environment, instead of forcing humans to enter theirs, will make using a computer as refreshing as taking a walk in the woods

Migratory User Interfaces• Migratory user interfaces are interactive applications that can

transfer among different devices while preserving the state and therefore giving the sense of a non-interrupted activity

• The devices that can be involved in the migration process should be able to run a migration client, which is used to allow the migration infrastructure to find such devices and know their features

• The client must be able to send the trigger event (to activate the migration) to the migration server, when it is activated by the user

• the state of the source interface will be transmitted to the server in order to be adapted and associated to the new user interface automatically generated for the target device

Migratory User Interfaces

The migration platform

Migratory User Interfaces

The migration process

The Migration Process

1. Device discovery

• concerns the identification of the devices that can be involved in the migration process and the attributes that can be relevant for migration (private or public device, their connectivity, their interaction resources, etc.).

• the device discovery has to be activated in every device that is involved in the migration (including the Migration Server). Its purpose is allowing the user to trigger the migration by selecting the migration target device - the device discovery module has to notify the presence of the associated device to a known multicast group. The list of the devices currently subscribed to such a group defines the list of devices that are available and could be involved in a migration process. In order to carry out this notification, the device discovery/migration client modules use multicast datagrams communications using the UDP/IP protocol.

The Migration Process 2. the user requests a page access from the current device

3. the request is transmitted to the application server

4. the application server provides the page - downloaded by the Migration Server (which works also as a proxy server) and then annotated by the Migration Server (the annotation is the automatical insertion of a script that is then used to transmit the state of the user interface)

5. deliver the page to the requesting device

6. the migration is triggered and the target device is identified

7. the current page along with its state is transmitted through an Ajax Script to the migration server (a version of the page adapted for the target device is created, with associated the state of the source version, and uploaded so that the user can immediately continue with all the data entered beforehand)

Migration issues• Adapting the UI to different devices (different size,

operating system, etc) - UI plasticity

• Plastic UI – the capability to withstand variations in both the device and its physical environment while preserving usability

• Should be able to adapt to different screen sizes and different input/output techniques (voice, gestures, multi-touch, stylus, mouse)