Prepared for a course given during Spring 2009 semester at EPFL

ContactEPFL / IC / IIG / GR-VABâtiment BCStation 141015 Lausanne

Tel: +41 21 693 25 75

TeamChristine VANOIRBEEK

Stéphane SIREMathieu TAGGIASCOMartin VESELY

XML based user interaction through

Rich Interfaces- SVG -

MULTIMEDIA DOCUMENT

Stéphane Sire (speaking)

PLAN

• Part 1– Rich User Interaction Style

• Part 2– Iterative Design and supportive technologies

• Part 3– SVG and alike

2

MEDIA Research Group Spring 2009

PART 1

Rich User Interaction Style

EVOLUTION OF USER GRAPHICAL INTERACTION STYLES

4

Desktop Web Mobile

Command Line

Direct (1983)Manipulation

Point & Click(hypertext)

HTML Forms

WAP

Rich User Interaction

WIMP

Rich Desktop Application (RDA)

Rich Internet Application (RIA)

iPhone and alike (touch based)

CHARACTERISTICS OF RICH USER INTERACTION

• Better Graphics• Extensive use of Animations • Creative layout models• Borrowings from other interaction styles• Better feedback times (AJAX)

5

A new vocabulary for design

BETTER GRAPHICS (1)

• Layers– Painter's model of rendering– Compositing operators

• (Porter & Duff)– Clipping and masking

• Applications– Layout based on layers (see later)– Non rectangular windows

• Objects with holes– Layer's effects

6

Source: Digistrip (CENA)

BETTER GRAPHICS (2)

• Color models– Alpha channel (RGBA)– Gradient colors– Texture filling

• Applications– Light and shadows effects– Glossy effect– Reflection (cow.neondragon.net/stuff/reflection/)

– etc

7

BETTER GRAPHICS (3)

• Shapes & lines– Stroke styles

• Joints• extremities

– Bézier paths• Very efficient algorithm to draw it (De Casteljeau algorithm)

• Applications– Rounded shapes

8

BETTER GRAPHICS (4)

• Pixel Effects– Anti-aliasing

– Filters (blur, bump mapping, etc.)

9

Source www.corzo.com

Source www.treebuilder.de

EXTENSIVE USE OF ANIMATIONS

10

User's guidance

System state change

Visual state transition

Source www.joehewitt.com/iui/samples/music.html Source www.lemonde.fr(choose Journal Electronique)

Source www.panic.com/goods/

ANIMATION TRICKS

• "Exageration"

• Pace• Application: Kinetic Scrolling (video on youTube)

11

Author's made movie from www.laredoute.fr

Author's made movie from www.youarethemodel.com

NEW LAYOUT MODELS (1)

• Layered information

12

Source www.laredoute.fr Source www.natoora.com

Source maps.google.com

NEW LAYOUT MODELS (2)

• Contraction/dilatation– Accordion

• Continuous surfaces– Linear (ribbon) : www.gucci.com or www.christofle.com

– Two dimensions : www.goruneasy.com

– Unconventional : www.etsy.com (cf. Explore)

13

Source www.goruneasy.com Source www.yomiuri.co.jp (site design has changed)

NEW LAYOUT MODELS (3)

• Fixed layout + Animation– Scrolling Ribbon

• more information with constant space within the page

• Layers + Animation– Drawer windows– Docks

14

Source www.tf1.fr

BORROWINGS FROM OTHER STYLES

• Direct manipulation– Drag & drop but still marginal

• Post-WIMP techniques– Bifocal menus

• Video-games– Rotative menus

15

Source www.amazon.com

Source www.wat.tv

BETTER FEEDBACK TIMES WITH AJAX

• No more page reload and full page refresh– Increase reactivity– Requires special tricks to get user's attention

• Applications– Auto-suggestion input field (see Google Autosuggest)

– Dynamical queries• allows some forms with no "Submit" button

• or allows to pre-filter results before submitting

16

Sourcewww.trivop.com

Sourcewww.darty.com/nav/achat/telephonie/telephonie_mobile/telephone_portable/guide.html(site design has changed)

RICH USER INTERACTION (SUMMARY)

• Graphical design– To increase intuitivity– To create emotions– Emotions increase perceived usability

• See Don Norman last book "Emotional Design"

• Requires multi-disciplinary teams – User experience (interaction) designer– Visual designer, Motion designer– Developers– etc

17

(video on youTube)

MEDIA Research Group Spring 2009

PART 2

Iterative Design and Supportive

Technologies

PRODUCT DEVELOPMENT PROCESS

19

Reprinted from "Sketching User Experiences" (Buxton, 2007)

ITERATIVE DESIGN

20

CLOSE TO AGILE SOFTWARE DEVELOPMENT

• Agile Manigesto (source agilemanifesto.org)– Individuals and interactions over processes and tools – Working software over comprehensive documentation – Customer collaboration over contract negotiation – Responding to change over following a plan

• See also Extreme Programming• Web version

– the permanent Beta

21

EXAMPLES

22

Source: Nielsen

Source: Hiser

Source: Alex Poole

CLASSICAL DEVELOPMENT CYCLE

23

Users

SketchesStoryboardWireframesIA diagrams

Pro

gram

me

rs

Interaction,Visual,Motion,…Designers

ExecutableCode

BrowserPluginRuntime Env.

Feedback loop

How to increase the number of iterations ?

MODEL DRIVEN DEVELOPMENT CYCLE

24

Users

SketchesStoryboardWireframesIA diagrams Programmers

Interaction,Visual,Motion,…Designers

Graphical modelsAnimation modelsOther models…

XML

Browser, PluginRuntime Env.

Executable code

HOW TO EDIT A MODEL ?

• Specialized editors – Example for graphics: Adobe illustrator

25

SVG File

A SHORT EXAMPLE

26

Interaction Designer

Visual Designer

Programmer:+ Javascript

Graphical ModelFile(s)

Example from Adobe dev web site

ADDING MORE MODELS….

27

Interaction Designer

Visual Designer

Graphical Model

Behavior Modelling with State Machines

Finite State Machine Models

MODEL DRIVEN SUMMARY

• Models– Specialized editors– Easy to move from

one platform to another

• Code– Programming skills– Hard to maintain– Less portable

28

Models

Models(XML)

CodeCode

MEDIA Research Group Spring 2009

PART 3

Web Standards for Graphical Design:

SVG and alike

W3C STANDARDS (1)

• Mainly "Models" – XML based languages– From "Draft" to "Recommendation" status– Models both for output and input modalities– HTML, XHTML, XForms, SMIL, MathML, VoiceXML, SCXML, InkML,

EMMA, …, and SVG• Some "Code" too

– Application Programming Interfaces (APIs)– DOM (3 levels)

• API for manipulating models loaded into memory as trees• Many more APIs (events, load and save, progression, etc.)

– XMLHttpRequest• Ajax programming

30

WEB STANDARDS AND MDA

31

Markup Only

Code Behind

Markup Behind Code Only

XML

XML

ImperativeProg. Lang.

Imperative Prog. Lang.

Definition of Interactive ComponentsInstantiation

of Intera

ctive C

omp

one

nts

Today Web applications

Future of applications

Today Desktop applications

W3C AND RICH USER INTERACTION

32

(*) simplified with specialized toolkits (scriptaculous, jQuery, etc)

Characteristics Standard solution

Better graphics SVG or CSS3 (with HTML)

Animation SMIL/SVG animation module

DOM Manipulation with Javascript(*)

Creative layout SVG or CSS (with HTML)

DOM Manipulation with Javascript(*)

Better feedback time XMLHttpRequest

Cross domain XHR

Borrowing from other styles All of the above

SVG HISTORY

• Scalable Vector Graphic• Started in 1998• 1st draft: February 1999• SVG 1.1 recommendation 2003

– http://www.w3.org/TR/SVG11/

• SVG 1.2 Tiny candidate recommendation 2006– http://www.w3.org/TR/SVGMobile12/– SVG 1.2 Full to come later

• Working Group with main software editors– Sun, Adobe, Apple, Microsoft, Corel, Ilog, HP, Canon, Autodesk, etc.

33

SVG VS. FLASH

34

Property SWF SVG

XML - Std W3C No Yes

Metadata Yes, propietary Yes, RDF compatible

Format Binary Text

Size < SVG > SWF : but can be compressed with gzip

by about 80% (*)

Runtime Plugin (200K) Native (Opera, Safari, Firefox, …)

Source No Yes (text is searchable and selectable)

Audio/Video Yes Yes (SVG 1.2 Tiny)

(*) see http://www.w3.org/TR/SVG/minimize.html

AUTHORING TOOLS

• Graphical Editors– Sodipodi, Illustrator, Corel Draw, OmniGraffle Pro, …

• Specialized Editors– Ikivo Animator : animations

• Automatic generation – Client-side Javascript generation

• Example from http://www.destatis.de/

– Client or server size XSLT generation• demo from www.treebuilder.de/default.asp?file=250484.xml

• Automatic conversion from/to other formats– From MathML, AutoCAD, Visio, etc.– To PDF, PNG, etc.

• Manual authoring :)

35

PLAYERS

• Adobe to stop its plugin support in 2009…– Adobe is switching to Flash/AIR

• … but becoming native in Web browsers– Opera, Safari, Firefox, …– open source rendering libraries (Cairo)

• Not directly available in IE…– Microsoft is switching to XAML/WVG with silverlight plugin

• Standalone players– Batik (Java) - contains a useful utility to generate SVG fonts

• SVG Tiny 1.2 on mobile phones– www.bitflash.com/mobile_primer.html– www.ikivo.com/02player_svg.html– And more…

• See list of implementations www.svgi.org/

36

DRAWING BASIC SHAPES

• Shapes– Rectangles <rect> – Cercles <circle>– Ellipses <ellipse> – Lines <line>– Polylines <polyline>– Polygons <polygon> – Text <text>

37

• Attributes– x, y, width, height, rx, ry– fill, stroke, stroke-

linecap, stroke-linejoin, stroke-miterlimit, stroke-width, …

– opacity, stroke-opacity, fill-opacity

– ...

<rect x="200" y="300" width="90" height="40” fill="lime" stroke="blue" stroke-linejoin="bevel" stroke-width="3"/>

DRAWING ARBITRARY SHAPES

• Path <path d="path data" >– outline of a shape which can be filled, stroked, used as a clipping path – defined by connected lignes, arcs and curves.

• Base attribute– "path data" : complex vocabulary – "," and EOL are allowed in "path data", spaces are optional between a

command and a number (to compress data)

38

<path d="M 100 100 L 300 100 L 200 300 z"

fill="red" stroke="blue" stroke-width="1" />

PATH DATA VOCABULARY• Each command is named by a letter (M: moveto L: lineto, etc.) followed by a

coordinate, either– Absolute if capital (e.g. "M")– Relative to the previous point if lowercase (e.g. "m")

• M and m:– Move the point to the given coordinate– Create a new sub-path starting at the origin– Syntax: M|m (x y) (x y)*– The second group (x y)* is interpreted as lineto commands– Ex: M100 100 200 200

• L and l:– Draw a line from the current point to the given coordinate and draw lines between

subsequent given points– Syntax: L|l (x y) (x y)*

• H and h (V and v):– Draw a horizontal line (vertical) from the current point to the current point incremented with

the given number– Syntax: H|h (x) et V|v (y)

• Z and z:– Close current subpath with the last point defined with M|m

39

M 100,100

L 200, 200

x

Cur point

MORE PATH DATA VOCABULARY

40

Command Args (can be repeated)+ Description

A, a rx ry x-axis-rotation direction sweep x y

Elliptical arc

C, c x1 y1 x2 y2 x y Cubic Bézier curve from current point to x y with control points x1 y2 and x2 y2

S, s x2 y2 x y Cubic Bézier curve with control point as the reflection of the reflection of second control point of a C, c command

Q, q x1 y1 x y Quadratic Bézier curve from current point to x y with control point x1 y1

T, t x y Quadratic Bézier curve with control point as the reflection of previous control point of a Q, q command

See details on http://www.w3.org/TR/SVG/paths.html

DEFINING REUSABLE GRAPHICAL COMPONENTS

• Definition of a component– Group element: <g id="name">– Container element for grouping graphics elements– Can be nested– Its attributes are inherited by its descendants

• Instantiation of a component– <use xlink:href="#name"/>

41

<g id="shape" stroke="blue"> <path width="1cm" d="M 0 50 35 25 70 50 Z" stroke-width="0.1cm" fill="none"/> <rect x="10" y="50" width="50" height="1cm" stroke-width="0.1cm"/> </g><use xlink:href="#shape" x="140" y="20"/><use xlink:href="#shape" x="70" y="60"/>

DECLARING GRAPHICAL COMPONENTS WITHOUT DRAWING THEM

• Everything that appears in a <defs> section will not be drawn– It will be available to be reused in other parts– This is a way to group resources (at the beginning of the file)

42

<defs> <g id="shape" stroke="blue"> <path width="1cm" d="M 0 50 35 25 70 50 Z" stroke-width="0.1cm" fill="none"/> <rect x="10" y="50" width="50" height="1cm" stroke-width="0.1cm"/> </g></defs><use xlink:href="#shape" x="140" y="20"/><use xlink:href="#shape" x="70" y="60"/>

POSITIONING GRAPHICS

• Most elements can be positioned with x and y within their parent container

• Groups <g> are also used to apply transformations– <g transform="translate(x,y)">– Other transformations include rotate, skewX|Y, scale and matrix– This is equivalent to associating a transformation matrix with each group

<g> (and to redefine a new coordinate system for the descendants)• Transformation are cumulative in nested groups

– Current transformation matrix at each <g>– Obtained as the product of all transformation matrices up to the parent

<g>

43

<g transform="rotate(20, 200, 300)"> <rect x="200" y="300" width="90" height="40" fill="lime" stroke="blue" stroke-width="3"/></g>

PAINTING AND COLORING

• Colors applies with different attributes to– Strokes (attribute "stroke")– Shapes interior (attribute "fill")

• Colors are rendered by a paint server– The attribute URI reference selects a paint server– Solid colors (e.g. "orange" or "#50A619")– Gradients or Patterns (e.g. "url(#myGradient)")

44

<defs> <linearGradient id="Gradient01"> <stop offset="20%" stop-color="#39F" /> <stop offset="90%" stop-color="#F3F" /> </linearGradient> </defs><rect x="200" y="300" width="90" height="40” fill="url(#Gradient01)" stroke="blue” stroke-width="3"/>

CLIPPING

• clipPath element– any path, text or basic

shape– defines an outline with

inside/outside

• clip-path attribute– attach clipPath to an

element

45

Clip layerNot visible

Contentlayer

Result

<clipPath id="visContent"> <text x="0" y="80" font-family="Verdana" font-size="70" fill="blue" >Media</text> </clipPath> <g clip-path="url(#visContent)"> <rect x="0" y="0" width="200" height="100" fill="url(#grad)"/> </g>

MASKING

• Masking– Attribute

• mask="url(#mask)"– Draw #mask in an offscreen

buffer– Convert it to gray level

image– Composite with the masked

layer

46

Layer1

Layer2Masked by Layer3

Result

Layer3MaskNot visible

<g mask="url(#Mask)"> <use xlink:href="#logo" transform="scale(1 -1"/></g>

ANIMATING GRAPHICS (1)

• Each graphical attribute can be animated– animate element– Animation description based on SMIL2 (Synchronized Multimedia

Integration Language level 2), W3C: http://www.w3.org/TR/smil20/

• Animation describing– Trajectory

• Attribute value variation

– Pace• Time variation to produce effects such as slow-in

47

<rect x="10" y="10" width="200" height="20"> <animate attributeName="width" attributeType="XML" from="200" to="20" begin="0s" dur="5s" fill="freeze" /> </rect>

ANIMATING GRAPHICS (2)

• Other types of animation– Animation along a path (motion tweening)

• animateMotion– Special instruction for color animation

• animateColor

• More advanced animation can be built with more programming (e.g. Morphing)– Exemple: Dragicevic, Artistic Resizing (UIST, 2005)

48

METADATA

• Images can be describe with – <desc>

• Can contain any RDF data– <title>

• Standard element to give a title

• This can be useful for – Accessibility (screen readers, …)– Indexing image file (do not forget it's text files)

• SVG 1.2 also introduces attributes for RDFa & microformats

49

<g> <title> Company sales by region </title> <desc> This is a bar chart which shows company sales by region. </desc> <!-- Bar chart defined as vector data --> </g>

HOW TO USE SVG

• Within an SVG document (.svg : image/svg+xml)– You can nest several SVG fragments / views inside an SVG document– This is a way to define "elastic" layouts

• Apply % values to x, y and width, height attributes

– Don't forget viewBox attribute to scale to fit

• Within an HTML document– By reference with <embed>, <object>, <iframe> or <img> tags– See www.carto.net/papers/svg/samples/svg_html.shtml

• Within an XHTML document (.xhtml : text/xhtml+xml)– Using the namespace inclusion mechanism– Must be parsed as XML by the browser– For a local file ".xhtml" is a way to force it

50

EXTRA FEATURES

• Pixel filters • Markers• Glyphs• Text along a path• Adding Interactivity with Javascript Programming

– SVG APIs• SVG DOM API: for scripted interactivity• SVG Micro DOM (for SVG 1.2 Tiny)

• Much more … explore by yourself – Mozilla SVG home: developer.mozilla.org/en/docs/SVG– Lot of fascinating examples: www.treebuilder.de/– SVG-Wiki: wiki.svg.org/– W3C SVG Home: www.w3.org/Graphics/SVG/

51

52

X11

1987 1992 1994 2002 2004

ShockwaveHTML HTML 4.0

Flash/SWF

X-HTMLSVG

Javascript DOM

XMLHttpRequest Ajax

CSS

IFrame

1990 1996 1998 2000 2006

OpenGL

PDF

Direct 3D OpenGL2.0

Mosaic NetscapeIE

FirefoxKonfabulator AIR

Silverlight

XAMLFlex/MXMLXUL

WVG

CONCLUSION: RIA STANDARDS AND TECHNOLOGIES

CREDITSWeb sources indicated directly on the slides, otherwise (many thanks to them):

Illustrationsp7 CENA Digistrip www.tls.cena.fr/divisions/PII/toccata/composants/digistrips.html

Demosp10 SVG Slot Machine – www.treebuilder.de

p30 Adobe Airbus A321 Panel demo – www.adobe.com/svg/demos/main.html

BookSketching User Experience, Bill Buxton, Morgan Kaufman by Elsevier inc. 2007 – www.billbuxton.com

Article

Combining SVG and models of interaction to build highly interactive user interfaces,

S. Chatty, A. Lemort, S. Sire, J-L. Vinot, SVGOpen 2005 – www.svgopen.org/2005/papers/CombiningSVGModelsBuildInteractiveUserInterfaces/

MEDIA Research Group Spring 2009

EXTRA SLIDES

54

COMPLETE SVG FILE EXAMPLE

55

<?xml version="1.0" encoding="utf-8"?><!DOCTYPE svg [ <!-- entities etc. here --> ]><svg version="1.1" baseProfile="full" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="8cm" height="6cm"> <defs> <!-- resources to be reused --> <linearGradient id="Gradient01"> <stop offset="20%" stop-color="#39F" /> <stop offset="90%" stop-color="#F3F" /> </linearGradient> <rect id="shape" width="1cm" height="1cm" stroke="blue" stroke-width="0.1cm"/> </defs> <!-- content --> <rect x=".1cm" y=".1cm" width="7.9cm" height="5.9cm" fill="none" stroke="black" stroke-width="1px" /> <use x="1cm" y="1cm" xlink:href="#shape" fill="#BBB"/> <use x="4cm" y="1cm" xlink:href="#shape" fill="url(#Gradient01)"/> <use x="1cm" y="4cm" xlink:href="#shape" fill="url(#Gradient01)"/> <use x="4cm" y="4cm" xlink:href="#shape" fill="blue"/> </svg>