V2 – An Emerging Universal Remote Console Standard

V2 – An Emerging Universal Remote Console Standard

Gottfried ZimmermannUniversity of Wisconsin-Madison, USATechnical Editor, V2International Representative, V2

V2 - An Emerging Universal Remote Console Standard

2

Note for screen reader users

These slides contain graphics. Please refer to the notes section of a slide to find long descriptions of the graphics it contains.


3

Recent Technological Advancements

Moore’s Law Steady increase in speed and power, decrease in size and prize Mobile devices have computing power of big irons 15 years ago

Ubiquitous Computing Computers everywhere The “disappearing” computer Wired and wireless connection technologies Remote operation of networked devices and services

“Artificial Intelligence” Speech Recognition Natural Language Understanding Intelligent user agents


4

But There Are Challenges…

Existing user interfaces are not intuitive and easy to understand for many users

For remote control, networked devices have to provide separate user interface specifications for every controller platform

Intelligent user agents may be the remedy for tomorrow, but not mature enough for today


5

Isn’t it time to think about

more universal ways

to interact with the devices and services

around us?


11

… and at the same time facilitate

more natural ways

of interaction

with tomorrow’s technology


18

People with disabilities

wouldn’t be left out.

In fact, everybody would benefit.


19


20


21

Overview

What is the V2 Standard? Universal Remote Console Design goals Technical concepts Examples

Who is building on V2? Members Standardization roadmap Prototypes

Demo (screenshots) if time allows

Conclusion and where to get more information


22

Universal Remote Console

a device or software application, such as• a Personal Digital Assistant (PDA)• a cell phone• etc.

… through which a user accesses and controls

… any electronic & IT product, such as• a TV, a washing machine, a thermostat, an alarm clock• a telephone, a copy machine• an elevator, a ticket vending machine• etc.


23

Idea: Use any of these controllers…(“URCs”)


24

… to access any of these products (“Targets”)


25

Design Goals for the URC Framework

“Universal” Remote Bi-Directional Communication Middleware-Neutrality Device & Modality Independence URC-Specific Custom User Interfaces Direct Access to Target Functions Support for I18n Reuse Existing Standards and Technologies Easy Target Maintenance User Interface Extensions Beyond What’s Built Into the Target Extensibility for Emerging Technologies A11y? – implicit


26

Goal: “Universal” Remote

One URC can control all Target products Targets: devices and services Anywhere

at home at work at school in public places

But: Dedicated remotes for specific Targets possible e.g. TV

“Let the user decide”


27

Goal: Bi-Directional Communication

Bi-directional communication between URC and Target Universal Remote Console

Implicit synchronization of the Target’s status information

Push notifications onto URC


28

Con

cep

t: Com

positio

n o

f U

ser In

terfa

ce

User Interface

Generator

User Interface Socket

User Interface(on the Universal Remote Console)

+

=

Resources

Label Help… Accesskey

… Keyword… …

User Interface Implementation Description+


29

Socket: machine-operable access and control point for a Target (or a functional unit of a Target)

One Target has one or more sockets Socket elements

variables statics commands notifications

Socket elements are implicitly synchronized between URC and Target

Note: Actual socket implementation (with “living values”) is platform-specific

Concept: User Interface Socket


30

Describes the socket elements in a platform-neutral way, including Named elements Types (subset of XML Schema Definition types) Dependencies (XPath)

The socket description informs a URC about the functionality of a socket in terms of its socket elements

Socket and Socket Description is everything that’s needed to access a Target through a “bare-bone UI”

Concept: User Interface Socket Description (SD)


31

Exam

ple

: Socke

t D

escrip

tion

<uiSocket about="http://www.mycorp.com/thermometer/socket“id="socket" xmlns="http://www.incits.org/incits390-2005“xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<static id="modelNumber" type="xsd:double">570</static><variable id="temperature" type="xsd:double"><dependency write="false()"/></variable><variable id="maximum" type="xsd:double"><dependency write="false()"/></variable><variable id="minimum" type="xsd:double"><dependency write="false()"/></variable><variable id="scale" type="scaleType" /><command id="reset" /><notify id="checkReset" explicitAck="false" category="alert">

<dependency acknowledge="value('confirmReset')='done' orvalue('cancelReset')='done'"/>

</notify><command id="confirmReset" type="uisocket:basicCommand" >

<dependency read="value('checkReset')='active'“execute="value('checkReset')='active'"/>

</command><command id="cancelReset" type="uisocket:basicCommand" >

<dependency read="value('checkReset')='active'“ execute="value('checkReset')='active'"/>

</command><xsd:schema><xsd:simpleType name="scaletype" id="idScaleType">

<xsd:restriction base="xsd:string"><xsd:enumeration value="F"/><xsd:enumeration value="C"/>

</xsd:restriction></xsd:simpleType>

</xsd:schema></uiSocket>


32

Goal: Middleware-Neutrality

Implementation of standard should be possible on a variety of middleware platforms for discovery and control

Binding to platform should not affect user interface

Bridging between different middleware platforms


33

Concept: Target-URC Network (TUN)

TUN is middleware that connects the socket on the Target with its copy on the URC

TUN provides services for discovery, control & eventing Possible TUNs

Universal Plug and Play (UPnP) Jini/Java Web service technologies (UDDI, WSDL, SOAP, etc.) ZigBee? XForms data model? Others?

Note: Implementation guidelines necessary to ensure interoperability within TUN

Gateways can bridge between different TUNs


34

Layered Model for URC and Target

Universal Remote Console Target

URC Framework Layer(Socket, PreT, Resources)

Networking Environment Layer(UPnP, Jini, etc.)

Wireless Connection Layer(Bluetooth, 802.11b, etc.)

Target Framework Layer(Portals, Sockets)

Networking Environment Layer (UPnP, Jini, etc.)

Wireless Connection Layer(Bluetooth, 802.11b, etc.)

URC User Interface Layer Target Application Layer


35

Concept: Target Description (TD)

Description of a target for discovery purposes Identifier Location description Description of sockets

• Socket identifier

• Pointer to socket description

• Pointer to UIIDs

Directory of available resources Other (Dublin Core) properties…

One TD per Target TD is middleware-neutral

Can be used across multiple TUNs


36

Exam

ple

: Targ

et D

escrip

tion

<target about="http://www.myCompany.com/DigitalThermometer" xmlns="http://www.incits.org/incits392-2005" ...>

<location id="location"></location>

<locator type="audio" id="audio-locator"/><locator type="visual" id="visual-locator"/>

<dc:identifier xsi:type="myComp:companyCode">0123456</dc:identifier><dcterms:conformsTo>

http://www.incits.org/incits392-2005</dcterms:conformsTo>

<socket id="socket“name="http://www.myCompany.com/DigitalThermometer/socket"><socketDescriptionLocalAt>

socket/socketdescription.xml</socketDescriptionLocalAt><category

taxonomyName="http://www.unspsc.org">41.11.22.00</category></socket>



</target>


37

Goal: Device & Modality Independence

Target can be fully controlled by any URC, without user programming or downloading PDAs Cell phones Wrist watches Computers URC devices AT devices

Single-source authoring of remote user interface Modality independence of user interface

Separation of content and presentation Multi-channel delivery of user interface


38

Concept: User Interface Implementation Description (UIID)

“View” component for a socket UIID provides additional information how to build a “usable”

user interface Structure What user interface components (e.g. “widgets”) to use How to map components to socket elements (“binding”)

One socket may have multiple UIIDs for different purposes (i.e. multiple views)

UIIDs can be proprietary BUT: One UIID must be written in standard format


39

Concept: Presentation Template (PT)

Subtype of UIID, defined by V2 Modality-independent user interface description Defines UI components and structure for socket Elements (“Interactors”) similar to XForms:

<group> <input> & <secret> <textarea> <select1> & <select> <range> <trigger> <output> <modalDialog>

“Core Presentation Template”: reflects full functionality of a socket Mandatory for every socket


40

Exam

ple

: Pre

sen

tatio

n T

em

pla

te

<pret name="http://www.mycorp.com/thermometer/corepret"id="pret"xmlns="http://www.incits.org/incits391-2005">

<output id="modelNumber" ref="http://www.mycorp.com/thermometer/socket#modelNumber"/>

<group id="readings"><output id="temperature“

ref="http://www.mycorp.com/thermometer/socket#temperature"/><output id="maximum“

ref="http://www.mycorp.com/thermometer/socket#maximum"/><output id="minimum“

ref="http://www.mycorp.com/thermometer/socket#minimum"/></group>

<select1 id="scale" ref="http://www.mycorp.com/thermometer/socket#scale"/><trigger id="reset" ref="http://www.mycorp.com/thermometer/socket#reset"/>

<modalDialog id="checkReset" ref="http://www.mycorp.com/thermometer/socket#checkReset"><trigger id="confirmReset" ref="http://www.mycorp.com/thermometer/socket#confirmReset"/><trigger id="cancelReset" ref=" http://www.mycorp.com/thermometer/socket#cancelReset"/>

</modalDialog>

</pret>


41

Goal: URC-Specific Custom User Interfaces

Ability to implement URC-specific custom user interfaces (in addition to modality-independent user interfaces) take advantage of capabilities of URC platform

• e.g. special UI for iPAQ

branding through user interface can be important for differentiation when functionality is

mostly the same across vendors


42

UIID may be implemented in any technology: Presentation Template (defined by URC standard) Java/Swing Flash SVG Etc.

Note: All UIIDs are based on the same socket

Concept: UIID Format Open


43

Goal: Direct Access to Target Functions

Concept: Direct access to all socket elements at any time without user navigation (“browsing”) URC knows about dependencies

Concept: UIID may bind to multiple sockets (of multiple Targets) user doesn’t need to know what Target to pick for a specific function more usable user interface if task-oriented


44

Goal: Support for I18n

Easy to localize user interface even after Target installation

Localization work may be out-sourced by Target manufacturers to 3rd parties

Even graphics may be locale-specific


45

Concept: Atomic Resources & Resource Sheets

Atomic Resource definition: object within a user interface that is used as an atomic entity in the

construction of a concrete user interface An Atomic Resource can be specific to

Language/locale Runtime environment Audience etc.

Atomic Resources are grouped and described in Resource Sheets typically stored as files on the Target (or on Resource Services) typically one Resource Sheet per language/locale


46

Concept: Atomic Resources & Resource Sheets (cont’d)

V2 standard defines an Atomic Resource Description containing: Identifier for Atomic Resource Type (Text, Image, etc.) Format (MIME type) Use context

• Element reference (to a socket element, PreT element or TD element)

• Value reference

• Operation reference

• Role (label, help, access key, keyword, etc.)

• Language context

Other (Dublin Core) properties…


47

Exam

ple

: R

eso

urce

Sh

eet

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"xmlns:dc="http://purl.org/dc/elements/1.1/"xmlns:dcterms="http://purl.org/dc/terms/" xmlns="http://www.incits.org/incits393-2005#">

<ResourceSheet rdf:about="http://www.myCorp.com/therm/rsheet-en.rdf"><dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005" /><aResourceDescription>

<AResourceDescription rdf:about="http://www.myCorp.com/therm/rsheet-en.rdf#temp_label"><content rdf:parseType="Literal">

<span xml:lang="en">temperature</span></content><dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005" /><useContext>

<UseContext><elementRef

rdf:resource="http://www.myCorp.com/therm/socket#temp" /><role rdf:resource="http://www.incits.org/incits393-2005#label" /><languageContext>en</languageContext>

</UseContext></useContext><dc:publisher>MyCorp, Inc.</dc:publisher><dc:type>Text</dc:type>

</AResourceDescription></aResourceDescription>

</ResourceSheet></rdf:RDF>


48

Concept: Resource Directory

Resource Sheets are listed in a Resource Directory

Resource Directory is part of the Target Description

A Resource Directory contains links to any

Target Resources (and external Resources)

Tree structure with leaves providing “scents” up to the root


49

Exam

ple

: R

eso

urce

Dire

ctory

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"xmlns:dc="http://purl.org/dc/elements/1.1/"xmlns:dcterms="http://purl.org/dc/terms/" xmlns="http://www.incits.org/incits393-2005#">

<ResourceDirectory><dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005"/>

<resourceSheet><ResourceSheet

rdf:about="http://www.myCorp.com/thermometer/rsheet-en.rdf"><dcterms:conformsTo

rdf:resource="http://www.incits.org/incits393-2005"/><localAt>rsheet-en.rdf</localAt><containsType>Text</containsType><containsDomain

rdf:resource="http://www.myCorp.com/thermometer/socket"/><containsDomain

rdf:resource="http://www.myCorp.com/thermometer/cpt"/><containsRole

rdf:resource="http://www.incits.org/incits393-2005#label"/><containsLanguageContext>en</containsLanguageContext>

</ResourceSheet></resourceSheet>



</ResourceDirectory></rdf:RDF>


50

Goal: Reuse Existing Standards and Technologies

No need to re-invent the wheel when already existing

People are already familiar with existing technologies

Tools and code libraries already existing


51

Reused Standards/Technologies in V2 Standards

XML & XML Schema Definition

XPath & XPointer

XForms form controls

RDF & Dublin Core


52

Goal: Easy Target Maintenance

Target may be a “dumb” device

e.g. thermostat

In general, Targets will last longer than URCs

Target features can be added even after purchase

if either the Target or the URC has access to the Internet

No URC update or central registry necessary


53

Concept: “Dumb” Target and “Smart” URC

Target Resources static independent from middleware platform may be on ROM can be served by simple HTTP server

URC picks appropriate resources generates user interface by “mix-and-match”

URC could be composed of gateway and ultra-thin client


54

Target Components


55

URC Components


56

Goal: User Interface Extensions Beyond What’s Built Into the Target

Target manufacturer, URC manufacturer and 3rd parties may provide extensions to the Target’s user interface skins icons sounds etc.

Simplification hide complex functions use same UI for a class of Targets

Any special adaptation for a particular URC a particular audience etc.


57

Concept: Supplemental Resources


58

Goal: Extensibility for Emerging Technologies

Natural Language Interaction Intelligent user agents Semantic Web concepts applicable Emerging security & privacy mechanisms Future-proof

Buy products today, and use them with simple URCs In the future, you may use intelligent user agents to

control them


59

Concept: Open URC Framework

Use of XML, XML Schema Definition and RDF Atomic Resources are extensible Resource Services are versatile Model-based user interface generation

Semantic information available More semantic information can be added to socket

description as Supplemental Atomic Resources or as Supplemental UIIDs or in any other way facilitated by dedicated Resource Services

Security & Privacy services can be added as part of TUN as independent services


61

Toward an ANSI standard

V2 Technical Committee

… within the InterNational Committee for Information

Technology Standards (INCITS)

… under the American National Standards Institute (ANSI)

V2 comprises members from industry, user groups,

academia and government


62

Members of V2

… and others

…joining…


63

5 Draft ANSI Standard Documents

“Protocol to Facilitate Operation of Information and Electronic Products through Remote and Alternative Interfaces and Intelligent Agents: …” INCITS 389-2005: Universal Remote Console

INCITS 390-2005: User Interface Socket Description

INCITS 391-2005: Presentation Template

INCITS 392-2005: Target Properties Sheet

INCITS 393-2005: Resource Descriptions

2nd public review starting soon


64

Implementation Guidelines for TUNs

V2-UPnP Implementation Guidelines in development Looking for a home for publication V2-UPnP Modes:

RemoteUI mode Legacy mode

Other implementation guidelines will be developed as needed


65

Toward an ISO/IEC Standard

Submitted as “New Work Item Proposal” to ISO/IEC JTC1 SC35 on User Interfaces

Project has been approved by JTC1

Assigned to SC35 WG8 “User Interfaces for Remote Interactions”

Goal: Release as international standard International participation Continuity Improvements


66

Members of JTC1 SC35

P-Members: Austria Canada China Czech Republic Denmark Finland Germany Greece Italy Japan Korea Sweden United States

O-Members: Ireland Netherlands Poland Russian Federation


67

V2 Prototype Implementations

University of Wisconsin-Madison, USA Java / Web-based Simulation Environment, www.myurc.com Java / UPnP (SDK in development) Java / Jini (old)

Georgia Institute of Technology, Atlanta, USA C#/C++ / UPnP

National Institute of Standards and Technology (NIST), USA Java / Jini

Marquette University, Wisconsin Just starting


68

Demo (screen shots)

Web-based Prototypefrom University of Wisconsin

www.myurc.com/V2SimEnv


69


70


71


72


73


74


75

Conclusion

The V2 standard is close to 2nd public review under ANSI Watch www.incits.org/tc_home/v2.htm Contact [email protected] if you want to be added to URC-

SIG mailing list

ISO/IEC JTC1 SC35 is developing the V2 standard on an international level

White paper available at www.myurc.com Prototype and SDK developments under way

If this standard doesn’t succeed, proprietary technologies may prevail, trying to monopolize the market


76

Thank you for your Attention!

Work supported by…

U.S. Department of Education (NIDRR)National Science Foundation

through the National Computational Science Alliance (NCSA)

Microsoft

Sun Microsystems