Natural user interface What is Next Gen? Instan t Multitasking

Gersh PayzerSenior Software DeveloperBuilding Xbox One apps using HTML and JavaScript

Building Xbox One apps using HTML and JavaScript

2-647

Meet the Xbox One App Platform10 foot experienceControllerGestureVoice

Agenda

Natural user interface

What is “Next Gen”?

InstantMultitasking

Multi-OS Architecture

Exclusive partition

Host OS

Shared partition (Windows)

apps

Xbox Shell

System Services

OS

HTML 5 and CSS 3 (IE 10) WinJS and the Windows Runtime Full-access to Xbox APIs from JavaScript:

Kinect SmartGlass Controller Etc.

C++ via WinRT

HTML 5 Platform

DirectX/C++

ListView - an example of porting to Xbox

Consider user input Controller Voice Gesture

“10 ft.” UX

What’s involved in porting from the web?

Controller

Use conventions and keep it simple: A – select B – cancel/go back Left thumbstick/DPad – directional navigation

Always set default focus Remember the user’s last focus Use onkeyup in most cases Only use onkeydown for repeatable

actions

Controller design considerations

Use automatic focus Respond to the click event for buttons

(oninvoked event for WinJS controls)

Reading controller input onkeydown / onkeyup events

Polling API (for games)

How to handle controller input

button.addEventListener("keyup", function (ev) { if (ev.keyCode === WinJS.Utilities.Key.gamepadA) { // Respond to the A button being pressed } }, false);

Automatic focus demo

Gesture

FEATURED MOVIES

Home Movies TVPOPULAR TV SHOWS

Gesture demo

Hard to hit targets Small buttons

Ideal size: 200px by 200px or greater Min size: 100px by 100px

Buttons on the edges of the screen. Worst is the bottom

Mouse !== Pointer/Touch/Gesture Strategies

Increase hittable area without increasing button size

Gesture design considerations

The user’s hand is a Pointer to your app

Touch interaction engine (direct manipulation)

Hand tracking

App code System rendered cursor

“Attraction”:

How to use gesture

button, input, select, a, .win-focusable { -ms-attraction: 0px 0px 0px 0px;}

button { -ms-attraction: "none";}

GestureRecognizer class Visual Gesture Builder Hand state Body data Depth data

Advanced topics

Voice

Media Tile856 x 424

Media Tile 424 x 208

Media Tile 424 x 208

More Popular TV Shows

FEATURED MOVIES

Home Movies TV

More Featured Movies

POPULAR TV SHOWS

VUI title text is Xbox Green, with a 64px high full-width background

Two rows are used for smaller tiles

Do not truncate titles as they are not sp…

Avoid S1E2 or v2 abbreviations

Hyphens, colons, and special characters should not be used for VUI

VUI Title

VUI Title

VUI TitleLong titles should be shortened for VUI

STOP LISTENING

Dynamic versus static phrases Too many phrases (greater than 20) Too many characters per phrase (more than 64)

Consider alternate phrases “Add to My Contoso Instant Queue” “Add to queue”

Words that are not actually pronounceable (P!nk, Apt.) Numbers are okay though Consider using “Item 1”, “Item 2”, etc

Is your target audience mostly kids?

Voice design considerations

Voice demo

Voice under the hood

<div data-win-voice="{ srcElement: select(‘.display-text’), targetElement: select(‘.win-voice-activetext’) }" data-win-control="XboxJS.UI.ItemContainer">

<div class="display-text">Play</div> </div>

<div data-win-voice="{ srcElement: select('.display-text'), targetElement: select('.win-voice-activetext'), handler: playArtist, pronunciation: ‘M N M' confidence: 0.7, }" data-win-control="XboxJS.UI.ItemContainer"> <div class="display-text">Play EMINEM</div> <div class="voice-text"></div> </div>

<button data-win-voice="{ phrase: 'play' }">play</button>

Voice under the hood

Text Text

XboxJS.UI.Voice

2. Grab source text 4. Style bling text

1. User says “Xbox…”

3. Register phrases & function callbacks

Windows.Xbox.Speech

Xbox One and Kinect sessions at //Build!Time Slot Room Presenter Code Session Title

Wed, April 2nd 1:00 - 2:00 3024

Nikola Metulev 2-649 Building Windows, Windows Phone, and Xbox One Apps with HTML/JS/CSS & C++

Wed, April 2nd 2:30 – 3:30 3024 Gersh Payzer 2-647 Building Xbox One Apps using HTML and JavaScript

Wed, April 2nd 4:00 - 5:00 3024 Chris White 2-514 Kinect 101: Introduction to Kinect for Windows

Thurs, April 3rd 1:00 - 2:00 2004 Rob Cameron 3-648 Leveraging Windows Features to build Xbox One App Experiences

Thurs, April 3rd 4:00 - 5:00 2004

Kevin Kennedy 2-532 Bringing Kinect into your Windows Store app

Thurs, April 3rd 5:30 - 6:30 3018 Alain Maillot 2-637 Extending Xbox Experiences to DevicesFri, April 4th 9:00 - 10:00 3014 Eddie Mays 2-650 Designing the Ultimate 10’ Experience

Fri, April 4th 10:30 - 11:30 Hall 1A Frank Savage 2-651 Understanding the Xbox One Game Platform Built on Windows

Fri, April 4th 12:30 - 1:30 3014 Bryan Nealer 2-652 Xbox Music API – Music for every app

Partners that offer Developer Services and Middleware are encouraged to join the Xbox Developer Partner Program

Email xboxdpp@microsoft.com for more information

Xbox Apps Developer Partner Program

Xbox One is built on Windows Core

Take advantage of the Windows platform today and build Windows Store/Windows Phone Apps

We have a room at Expert 1:1, come talk to the Xbox Apps team! :-)

Join the Xbox team for an informal social event in the Xbox Loft Thursday 6pm

Summary

Please fill out a session survey!

Your Feedback is Important

Fill out an evaluation of this session and help shape future events.

Scan the QR code to evaluate this session on your mobile device.

You’ll also be entered into a daily prize drawing!

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.