27
APPLICATION DEVELOPMENT WITH TOUCH AND SENSORS ON ULTRABOOK TM / WINDOWS 8 Copyright© 2012, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 1 Sulamita Garcia – [email protected] Technical Marketing Engineer Twitter @sulagarcia

Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Embed Size (px)

DESCRIPTION

Ultrabook Development Using Touch - presented by Sulamita Garcia

Citation preview

Page 1: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

APPLICATION DEVELOPMENT WITH

TOUCH AND SENSORS ON

ULTRABOOKTM / WINDOWS 8

Copyright© 2012, Intel Corporation. All rights reserved.

*Other brands and names are the property of their respective owners 1

Sulamita Garcia – [email protected]

Technical Marketing Engineer

Twitter @sulagarcia

Page 2: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

/me

Page 3: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

MAHJONG DEMO HERE

Copyright© 2012, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 3

Page 4: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Using touch in a desktop application

• Touch and gestures– Tap/Double tap

– Panning with inertia

– Selection/Drag

– Press and tap

– Press and hold

• Do’s and Don’t of gesture interfaces

• Demo: Use touch and gestures to select, move and manipulate three images

• The Windows Touch API

• Using touch with XMAL

4Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

– Zoom

– Rotate

– Two-finger tap

– Flicks

Page 5: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Adding touch to a desktop application

• Touch is not a mouse

• Make touch targets large enough

• Support standard gestures and behaviors

• Portrait vs. Landscape

• Do not use touch just for touch’s sake

• Touch should be forgiving

• Microsoft User Experience Guidelines– msdn.microsoft.com/en-

us/library/windows/desktop/cc872774.aspx

– blogs.msdn.com/b/ie/archive/2012/04/20/guidelines-for-building-touch-friendly-sites.aspx

5Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

Page 6: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Designing for Touch User Experience

• Design UI to include traditional laptop style and tablet mode usages

• Space and size controls to register fuzzy touch input– Common Controls: 23X23 pixels

– Command Controls: 40X40 pixels

• Use multiple form tabs for touch efficiency

• Consider the target size while designing your application UI

• Use natural and intuitive controls

• Ultrabook Optimal Resolution: 1366X768

Source: MSDN

http://msdn.microsoft.com/en-us/library/windows/desktop/cc872774.aspx#guidelines

Copyright(C) 2012 Intel Corporation. All rights reserved.

*Other brands and names are properties of their respective owners. 6

Page 7: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Designing for Touch User Experience

• Use common controls as much as possible

• Choose custom controls that support touch

• Prefer constrained controls to unconstrained controls

• Provide default values and auto text-completion

• Use check boxes instead of multiple selection lists

Source: MSDN

http://msdn.microsoft.com/en-us/library/windows/desktop/cc872774.aspx#guidelines

Copyright(C) 2012 Intel Corporation. All rights reserved.

*Other brands and names are properties of their respective owners. 7

Page 8: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Designing for Touch User Experience

• Place controls in areas better utilized for touch

• Command controls should be more easily

accessible

Copyright(C) 2012 Intel Corporation. All rights reserved.

*Other brands and names are properties of their respective owners. 8

Page 9: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Using Windows Touch API• Touch and gesture events are delivered via Windows

Touch messages– Windows must register for Windows Touch input

• WM_TOUCH reports action, position and identifier

• WM_GESTURE describes the gesture via GESTUREINFO structure

• Special interfaces to help process gesture messages – IManipulationProcessor

– IInertiaProcessor

• API reference at msdn.microsoft.com/en-us/library/windows/desktop/dd371406%28v=vs.85%29.aspx

9Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

Page 10: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Sensors and Windows Native

• Sensor manager controls sensors– Use sensor manager to get to sensors

– Notifies when a sensor connects

• Sensors report data, changes in state and disconnection

• Access both via COM interfaces– Use API to communicate to sensors

– Sensor events handled using callbacks

– API reference at msdn.microsoft.com/en-us/library/windows/desktop/dd318953%28v=vs.85%29.aspx

10Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

Page 11: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

WinNative: Receive gesture message

LRESULT CALLBACK WndProc(…)

{…switch (message){

case WM_GESTURE: // Call code to// interpret the// gesturereturn

DecodeGesture(…);…

}

11Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

(Source: Microsoft)

• Check for

WM_GESTURE message

in WndProc

Page 12: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

WinNative: Process gesture

LRESULT DecodeGesture(…){

…BOOL bResult = GetGestureInfo(…);

BOOL bHandled = FALSE;

if (bResult){

// now interpret the gestureswitch (gi.dwID){

case GID_ZOOM : // Code for zooming goes herebHandled = TRUE;break;

case GID_PAN : …

case GID_ROTATE : …

case GID_TWOFINGERTAP: …

case GID_PRESSANDTAP: …

default:// A gesture was not recognizedbreak;

} …

12Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• Retrieve the additionalgesture information from the GESTUREINFO structure

• Handle each of the possible gestures

(Source: Microsoft)

Page 13: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Demo

• New Windows*8 UI application

– Full screen

– Borderless window

• Desktop application

– Runs in desktop environment

– Multiple windows

– Can support touch and sensors

13Copyright© 2012, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners

Source: Microsoft

Page 14: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Using touch and XAML/.NET

• XAML provides access to WPF touch-enabled

UI components

– WPF touchs event are available in both Windows* 7 and

Windows* 8.

• Typical gesture events:

– ManipulationStarting

– ManipulationDelta

– ManipulationInertiaStarting

14Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

Page 15: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

XAML: TouchControl.xaml

<UserControl x:Class="Win8Demo.TouchControl" …>

<Grid x:Name="cont" ClipToBounds="True" ><!-- Set the IsManipulationEnabled totrue -->

<Image x:Name="photo_1"Source="/Assets/ultrabook1.png“IsManipulationEnabled="True"Width="500" />

<Image x:Name="photo_2“Source="/Assets/ultrabook2.png“IsManipulationEnabled="True“Width="500" Margin="125,-79,25,79"/>

<Image x:Name="photo_3“Source="/Assets/ultrabook3.png“IsManipulationEnabled="True“Width="500" Margin="0,-59,150,59" />

</Grid></UserControl>

15Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• Three images placed in

a Grid inside of a

TouchControl

• Each image has IsManipulationEnabledset to true

Page 16: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Touch Event State Diagram

Copyright© 2012, Intel Corporation. All rights reserved.

*Other brands and names are the property of their respective owners 16

Manipulation

Starting

Manipulation

Delta

Manipulation

Inertia Starting

Manipulation

Delta

w/ Inertia

Manipulation

Completed

Events begin with User

Touching the Image

User finger is

touching the

screen

User finger is lifted from the

screen

Page 17: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

XAML: TouchControl.xaml.cs

public TouchControl(){

//Register for manipulation eventsphoto_1.ManipulationStarting +=

m_rect_ManipulationStarting;photo_1.ManipulationDelta +=

m_rect_ManipulationDelta;photo_1.ManipulationInertiaStarting +=

m_rect_ManipulationInertiaStarting;

photo_2.ManipulationStarting += …

photo_3.ManipulationStarting += … }

17Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• Add event handlers to

each image for:•ManipulationStarting

•ManipulationDelta

•ManipulationInertiaStarting

Page 18: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Touch Event State Diagram

Copyright© 2012, Intel Corporation. All rights reserved.

*Other brands and names are the property of their respective owners 18

Manipulation

Starting

Manipulation

Delta

Manipulation

Inertia Starting

Manipulation

Delta

w/ Inertia

Manipulation

Completed

Events begin with User

Touching the Image

User finger is

touching the

screen

User finger is lifted from the

screen

Page 19: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

XAML: Manipulation starting event

handlervoid

m_rect_ManipulationStarting

(…)

{

e.ManipulationContainer = this;

}

19Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• ManipulationStarting event handler is called at start of touch event

• By setting Manipulation-Container to the touched image all subsequent manipulation events will be relative to that element

Page 20: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Touch Event State Diagram

Copyright© 2012, Intel Corporation. All rights reserved.

*Other brands and names are the property of their respective owners 20

Manipulation

Starting

Manipulation

Delta

Manipulation

Inertia Starting

Manipulation

Delta

w/ Inertia

Manipulation

Completed

Events begin with User

Touching the Image

User finger is

touching the

screen

User finger is lifted from the

screen

Page 21: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

XAML: Manipulation delta event

handlervoid m_rect_ManipulationDelta(…){

…//Get the manipulation’s deltavar delta = e.DeltaManipulation;

//Compute the transformed center pointPoint rectCenter = new Point(rect.ActualWidth *

0.5, rect.ActualHeight * 0.5);rectCenter = matrix.Transform(rectCenter);

//Adjust the element’s scale, rotation and translation

matrix.ScaleAt(delta.Scale.X, delta.Scale.Y,rectCenter.X, rectCenter.Y);

matrix.RotateAt(delta.Rotation, rectCenter.X, rectCenter.Y);

matrix.Translate(delta.Translation.X, delta.Translation.Y);

//Update the element’s render transformationrect.RenderTransform =

new MatrixTransform(matrix);e.Handled = true;

21Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• ManipulationDeltaevent happens as user moves fingers

• Compute new image center point, scale, and rotation using event data

• Set Handled to true so other handlers don’t process same event

Page 22: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Touch Event State Diagram

Copyright© 2012, Intel Corporation. All rights reserved.

*Other brands and names are the property of their respective owners 22

Manipulation

Starting

Manipulation

Delta

Manipulation

Inertia Starting

Manipulation

Delta

w/ Inertia

Manipulation

Completed

Events begin with User

Touching the Image

User finger is

touching the

screen

User finger is lifted from the

screen

Page 23: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

XAML: Inertia starting event handler

void m_rect_ManipulationInertiaStarting(…){

//Set the manipulations inertia valuese.TranslationBehavior = new

InertiaTranslationBehavior(){

InitialVelocity =e.InitialVelocities.LinearVelocity,DesiredDeceleration =…

};e.ExpansionBehavior = new

InertiaExpansionBehavior(){

InitialVelocity =e.InitialVelocities.ExpansionVelocity,DesiredDeceleration = …

};e.RotationBehavior = new

InertiaRotationBehavior(){

InitialVelocity =e.InitialVelocities.AngularVelocity,DesiredDeceleration = …

};e.Handled = true; …

23Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• Inertia actions are handled by the ManipulationInertiaStarting event handler

• Define the behavior of the inertia by specifying • Linear velocity• Deceleration• Expansion velocity (used in

pinch or spread)• Angular velocity (used in

rotation)

• Again, set Handled to true so other handlers don’t process same event

Page 24: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Touch Event State Diagram

Copyright© 2012, Intel Corporation. All rights reserved.

*Other brands and names are the property of their respective owners 24

Manipulation

Starting

Manipulation

Delta

Manipulation

Inertia Starting

Manipulation

Delta

w/ Inertia

Manipulation

Completed

Events begin with User

Touching the Image

User finger is

touching the

screen

User finger is lifted from the

screen

Page 25: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

XAML: Handle inertia manipulation

if (e.IsInertial){

//Get the containing element’s sizeRect containingRect = …

(e.ManipulationContainer).RenderSize);//Get the transformed element’s new//boundsRect shapeBounds =

rect.RenderTransform.TransformBounds(…);

//If element falls out of boundsif(!containingRect.Contains(shapeBounds)){

//Report boundary feedbacke.ReportBoundaryFeedback(…);//Stop any after inertiae.Complete();

}}

25Copyright(C) 2012 Intel Corporation. All rights reserved. *Other brands and names are properties of their respective owners.

• Check if inertia is moving the

image across the grid

• If so, make sure image does not

travel outside of the grid by

calling the Complete method to

end the inertia

Page 26: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

References

• Intel Ultrabook™ Community

• Intel® Software Network

– software.intel.com

• Windows Development Reference

– msdn.microsoft.com/en-us/library/windows/desktop/hh447209%28v=vs.85%29.aspx

• WinRT API Reference

– msdn.microsoft.com/en-us/library/windows/apps/br211377.aspx

26Copyright© 2012, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners

Page 27: Ultrabook Development Using Touch - Intel Ultrabook AppLab Berlin

Q & A

27Copyright© 2012, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners

Sulamita Garcia – [email protected]

Technical Marketing Engineer

Twitter @sulagarcia