Application Experiences at Your FingertipsVineet SardaSenior ConsultantMicrosoft Corporation

SESSION CODE: WCL302

Agenda

Touch HardwareKey Touch ScenariosWindows 7 Touch InvestmentsTouch Application Development Guidance

GestureManipulation and InertiaWPF development

Call To ActionResources

Windows Touch Is Here

Windows Touch is defined as two or more independently traceable touch points.

ConsumersTouch interfaces have proven successful Multi-Touch offers clear differentiation with a high “WOW” factor Multi-touch enhances on-the-go use and enable new scenarios

HardwareMulti-touch capable machines in market today in a broad set of form factors

Navigating and Consuming the Web

Playing Casual Games

Reading and Sorting Email

Viewing Photos

1

2

3

4

5

6 Navigating Files and Using Applications

Consuming Music and Video

Key Touch Scenarios Are Consumptive

Windows InvestmentsInternet Explorer®

Pan and zoom gestures, flick forward/back“Open link in new tab” gestureAddress bar drag menu

Windows® Media Center:Live gallery panningOn-screen keyboardLarger transport controls & seekNew Now Playing & details pages

Photo Viewer (& Windows Live Photo Gallery)Multitouch Zoom and rotation

XPS ViewerStretch zoom centered around gestureSmart zoom

Touch Pack for Windows 7

Windows 7 Principle: Be Great at “Look” and “Do”

Users have expectations for how an experience will “Look” and what it should “Do”Direct Manipulation demands more from “Look” and “Do”Touch Pack for Windows 7

Surface Globe: Immersive, 2-D and 3-D touch world exploration:

Zoom in on locationsPinpoint areas of interestGet local informationPersonalize map views

Development Tiers

Best

Better

Good

Raw data points from WM_TOUCH for custom gestures. Manipulation/inertia APIs

Multi-touch support – zoom, finger pan, rotate, panning with inertia, and 2 finger tap. WM_GESTURE

Maps to common Windows messages – pan, zoom, right click, and double click

Gestures: Consistency is key

Gestures: Touch actions that are one step removed from direct manipulations

Two finger tapPress and tap Pinch/stretch (zoom)Circular motion (rotate)Single and two-finger panningFlicks

Useful for shortcut-like efficiencyConsistent operations like zoom in, or navigateFocus on content : to surface extra information or invoke contextual action UI

Design ConsiderationsConsistency

Should work in the same way across Windows applicationsKeep it simple – users won’t discover or remember too many abstractions

Windows 7 Touch Gestures

WM_GESTURE

Windows Message and APIs:

WM_GESTURE, WM_GESTURENOTIFY

GESTURECONFIG struct

Set/GetGestureConfig()

GESTUREINFO

GetGestureinfo()

Setting Gestures and WM_GESTURE case WM_GESTURENOTIFY: { BOOL bResult;

GESTURECONFIG gcAll[] = {0,GC_ALLGESTURES,0}; // Enable all gestures bResult = SetGestureConfig(hWnd, 0, 1, gcAll, sizeof(GESTURECONFIG));

} case WM_GESTURE: {

GESTUREINFO gi; gi.cbSize = sizeof(gi); bResult = GetGestureInfo((HGESTUREINFO)lParam, &gi); switch (gi.dwID) { case GID_BEGIN:

break; case GID_END:

break; case GID_ZOOM: //GID_PAN, GID_ROTATE, GID_TWOFINGERTAP, GID_PRESSANDTAP

break; }

Gesture Demo

demo

WM_TOUCH

WM_TOUCH* family of messagesEnables touch-optimized experiencesProvides standard Win32 messages

WM_TOUCHDOWN, WM_TOUCHMOVE, WM_TOUCHUPSemantically similar to mouse messagesConveys raw touch data to Win32 apps

Scenario examples:Finger painting, custom gestures, feeding higher level controls, etc

ManipulationsManipulations are a great foundation for touch-optimized experiences

Examples of higher level APIs/controlsManipulations are:

2D affine transformations (translate, scale, rotate)Superset of supported gesturesSupports multiple concurrent manipulations

Require a source of raw data: WM_TOUCH or RTSSimilar to Surface APIs

Implemented with Interfaces:IManipulationProcessorIManipulationEvents

Typical FlowOn contact down:

Hit test against targets(Add to map of contact ids)MP->ProcessDown()

On contact move:(Look up mapping)MP->ProcessMove()

On contact up:(Look up mapping)MP->ProcessUp()(Kick off inertia)

Get callbacks on event interfaceApply transforms to targets

Manipulations and Inertia

demo

Surface HardwareWindows 7

Surface HardwareWindows VistaWindows 7

NativeWin32

Application

Touch Development Roadmap

WPF 3.5

Surface SDK 1.0

Managed Wrapper and Interop

Managed Wrapper and Interop

WinForms Application

Windows 7 ReleaseNET 4.0 / Surface 2.0 Release

WPF 3.5 SP1

WPF 4.0

Surface SDK 2.0WPF

Application

Surface Application

Multi-Touch Controls

Multi-Touch API

Surface Multi-Touch Controls & API

Multi-Touch API

Multi-Touch API and Controls

.NET 3.5 Multitouch library

Multi-touch for Windows 7

NativeWindows SDK

.NET 4WPF

SilverlightSilverlight3

GesturesAvailable via

manipulation

Raw Touch

Manipulation

Inertia

Multi-Touch and Multi-Capture

Multi-touch Manipulation and Inertia (Pan, Zoom, Rotate) events on UIElement Raw Multi-touch events (Up, Move, Down) on UIElement, UIElement3D and ContentElement Multiple capture supporting multiple active controls ScrollViewer enhancement to support multi-touch panning Touch device extensibility

WPF Touch

demo

Manipulation Container

Canvas

Manipulation Container

IsManipulationEnabled=trueIsManipulationEnabled=true

Update RenderTranform

Handle ManipulationDelta Event• Coordinates relative to

Manipulation Container

Handle ManipulationStarting Event• Set Manipulation Container

Manipulation Events

Starting

Started

Delta

Inertia

Starting

Delta

Complete

dStartInertia() StartInertia()

Touch Down (Initial)

Touch Move

Touch Up (All)

Initialize:•Mode• Container• Pivot

Completed()

Manipulations with WPFvoid image_ManipulationStarting(object sender, ManipulationStartingEventArgs e){ e.ManipulationContainer = canvas; }

void image_ManipulationDelta(object sender, ManipulationDeltaEventArgs e){ var element = e.Source as FrameworkElement; if ( element != null ) {

var deltaManipulation = e.DeltaManipulation; var matrix = ((MatrixTransform)element.RenderTransform).Matrix; Point center = new Point ( element.ActualWidth/2, element.ActualHeight/2) ;center = matrix.Transform(center); matrix.ScaleAt(deltaManipulation.Scale.X, deltaManipulation.Scale.Y, center.X, center.Y); matrix.RotateAt(e.DeltaManipulation.Rotation, center.X, center.Y); matrix.Translate(e.DeltaManipulation.Translation.X, e.DeltaManipulation.Translation.Y);((MatrixTransform)element.RenderTransform).Matrix = matrix;

}}

WPF Manipulations

demo

Call To Action

Consider the design principles in this presentationTouch experiences should be intuitive and not overly complexDesign scenarios that enhance your application productivity and ease of useLeverage the materials listed in the resources section

ResourcesSoftware Design

Windows User Experience Interaction GuidelinesMulti-Touch in Windows 7Developer Enhancements to Windows Touch and Tablet PC(User Perceptions of Response Time) Performance UX Patterns

Developer ResourcesCode Magazine: Multi-touch capabilities in Windows 7Windows Touch on MSDNWindows Touch Developer ResourcesTablet and Touch SDKIntroducing the Microsoft Touch Pack for Windows 7

Track ResourcesLearn about Windows Client Development: http://developforwindows.comTry out the Internet Explorer 9 Platform Preview: www.ietestdrive.comInterested in Windows hardware or software? Check it out: http://ReadySet7.com

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

Complete an evaluation on CommNet and enter to win!

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st

http://northamerica.msteched.com/registration

You can also register at the

North America 2011 kiosk located at registrationJoin us in Atlanta next year

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft

cannot guarantee the accuracy of any information provided after the date of this presentation.MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

JUNE 7-10, 2010 | NEW ORLEANS, LA