Graphics at the core ofWindows 8 and your appShai HinitzSr. Program Manager, Windows Games3-112

Windows 8 graphics overviewGraphics development language optionsMigrating your graphics to WinRTCase Study: Cat in the Hat by Oceanhouse Media

Session agenda

Windows 8 is fully graphics hardware accelerated to increase performance and reduce power consumptionThe Lock and Start screenPlatform for “fast and fluid” & “stick to your finger” experiencesInternet Explorer 10Rich visual and multimedia experiences

All Windows Store apps are hardware acceleratedAll rendering surfaces are built on DirectXUse DirectX for maximum control of GPU performance and extensibility

2D graphics 3D graphics

Segoe UIFonts, text analysis, layout Image & video decoding

GPGPU Too

HTML5, CSS3 & Direct2D Direct3D

Segoe UIHTML5, CSS3 & DirectWrite HTML5, Direct2D effects,

WIC & Media Foundation

DirectCompute & C++AMP

Graphics language of choiceWrite full-screen, chrome-free applications with smooth, flicker-free action using HTML5 and CSS3 Windows Store apps UI toolkitDevelop interactive 2-D applications and include 3-D graphics using XAML Windows Store apps UI toolkitAuthor fully immersive interactive 3-D experiences by taking advantage of the full processing capabilities of modern graphics hardware using HLSL with DirectX® 11.1Use hybrid models supporting XAML & DirectX if appropriate

Development language combinationsHTML/CSS for GUI/Markup and canvas for renderingLeverage hardware accelerated engine from IEWindows features (file I/O, events, notifications, etc.) are exposed in JavaScript

XAML can be used with C++ or C# or Visual Basic appsDirectX can be used with C++ (or C# via third-party wrappers)Hybrid models supportedDirectX and XAMLJavaScript wrappers for DLLs

Javascript // get the canvas and the 2D context for itvar canvasElement = document.getElementById("myCanvas");var context = canvasElement.getContext("2d");

// set the font and sizecontext.font = "20px Georgia";

// clear an area and render hello world at position (10,50)context.clearRect(0.0.300,150);context.fillText("Hello World!", 10, 50); 

HTML// define the canvas size with the tag in the HTML doc<canvas id="myCanvas" width ="720" height ="486"></canvas>

HTML5/JavaScript code snippet

XAML// Set rendering location and text style<StackPanel Grid.Row="1" Margin="120,30,0,0">

<TextBlock x:Name="greetingOutput" Style="{StaticResource ShyTextStyle}"/></StackPanel>

C#// Set textprivate void YourFunction(object sender, Windows.UI.Xaml.RoutedEventArgs e) {     greetingOutput.Text = "Hello World!"; }

XAML/C# code snippet

void DWriteHelloWorld::CreateWindowSizeDependentResources() { // Create the text string DirectXBase::CreateWindowSizeDependentResources(); Platform::String^ text = "Hello World!"; D2D1_SIZE_F size = m_d2dContext->GetSize(); // Create a DirectWrite Text Layout object DX::ThrowIfFailed( m_dwriteFactory->CreateTextLayout( text->Data(), // Text to be displayed text->Length(), // Length of the text m_textFormat.Get(), // DirectWrite Text Format object size.width, // Width of the Text Layout size.height, // Height of the Text Layout &m_textLayout ) );

DirectX/C++ code snippet

void DWriteHelloWorld::Render(){  // Clear the buffer and set up m_d2dContext->BeginDraw(); m_d2dContext->Clear(D2D1::ColorF(D2D1::ColorF::CornflowerBlue)); m_d2dContext->SetTransform(D2D1::Matrix3x2F::Identity());  // Update text position and style m_d2dContext->DrawTextLayout(       D2D1::Point2F(0.0f, 0.0f),       m_textLayout.Get(),       m_blackBrush.Get() );  m_d2dContext->EndDraw();}

DirectX/C++ code snippet

Communication and data

Devices and printing

WinRT APIsDirectX

graphics and media

Desktop apps

CC++

Win32

C#VB

.NET Silverlight

HTMLJavaScrip

t

Internet Explorer

Windows Store apps

Application model

JavaScriptCC++

Windows kernel services

Syst

em se

rvice

sVi

ewM

odel

co

ntro

ller

Kern

el

XAML

C#VB

HTML/CSS

Where’s

DirectX??

Graphics architecture in Windows 8

DXGI

VideoDirect3DDirectCompute

Direct2DC++ AMP Media Foundation

HTML, CSS, and XAML graphics

Graphics Processing Unit (GPU)

Windows 8 DirectX advantages

The new Windows 8 graphics stack is better integratedUnified graphics runtime avoids side by side installation issues Feature levels target the broadest range of GPU levels and vendorsFor the ultimate experience in graphics and video, use DirectX 11.1 features to bring tessellation and stereoscopic 3D to your apps

3D graphics via Direct3D 11Use the same API for Windows Store and desktop appsWinRT API supports the newest DirectX 11.1 featuresSupport for multiple hardware generations via Feature LevelsFeature_Level_9Feature_Level_10 (plus all of 9)Feature_Level_11 (plus all of 10)

Direct3D 11.1 has new updates for Windows 8Performance improvementsHigher quality visualsPower efficiencyEase of development

Check out 3-113 Graphics with the Direct3D11.1 API made easy

Porting your apps to the Windows Runtime

Porting graphics apps to WinRTDetermine source target technologyWeb HTMLSilverlight/WPF XAMLOpenGL Direct3D

Reuse art assets (images, textures, fonts) Get the uncompressed (un-flattened) sources (e.g. PSD)Source vector-based images

Preserve graphics code and mark up resourcesHTML, CSS, JavaScript, XAML, C#, VB, C++…

Start from your app or a template or sample?

General porting approachAdapt app’s UI to exploit Windows 8 features and follow design guidelinesMigrate your code from JavaScript/C#/Visual Basic/C++ to the corresponding Windows Runtime APIsStart with VS template that best matches your app layout and functionalityCopy folders, code and art assets to reuse to new projectInclude the files in the new Visual Studio ProjectCopy the reusable markup code (HTML, CSS, XAML, shaders) code into the new project

General porting approach

Search and replace source namespaces with target WinRT namespaces

Build the project to generate errors to locate code to update or replace Stub out any code that you cannot easily update until your project buildsReplace the commented-out code line by line until your port is complete

Web WinRT JavaScript/HTML

Key considerationsWWAHost.exe hosts and executes your appJavaScript content should just workUse JavaScript to call WinRT APIsiFrames supported – note local vs. hosted contentSome web technologies are not supportedActiveX controlsJava objects

See Migrating a web app using JavaScript and HTML on MSDN

Graphics considerations<canvas> for real-time graphics controlled by JavaScriptSupport for HTML5, CSS3 and SVG content for general graphics and UIMost video, audio media and image content/tags will be compatible2D and 3D transforms for 2D and 2½D graphics Concatenation of multiple transforms for improved performance

Some web graphics technologies are not supportedNon HTML5 standard objectsAdobe Flash Player

Silverlight content transitions best to XAML

Check out 3-110 From zero to hero! Building a Windows Store game in HTML5

Silverlight/WPF WinRT C#/XAML

Key considerationsLeverage your experience from Silverlight and WPFProgramming concepts and usage patterns influenced WinRT designPorting effort will depend on breadth of APIs usedWinRT APIs include most commonly used subset of .NET framework APIsMuch XAML and C# or Visual Basic will transition smoothlySearch and replace "System.Windows“ with (WinRT)"Windows.UI.Xaml"

Graphics considerationsCode runs directly on the runtime (no framework host)Input handled directlyMore secure than internet-centric modelLess storage limitationsWebView can host HTML/CSS content

Asynchronous approach for best performanceThe WinRT imaging API supports more image source formatsSome methods that would slow down rendering (like OpacityMask) were omitted Easy to use encoder/decoder types New theme and transition animationsSimilar media APIs with DRM plug-in options

Check out 3-116 Introduction to creating Windows Store apps using XAML

OpenGL|ES WinRT C++ /

OpenGL and Direct3D similaritiesBoth have similar rendering pipelines and graphics featuresDirect3D is a rendering implementation and API, not a spec Hardware accelerated at all times

OpenGL is a rendering spec and API, not an implementation Hardware accelerated where possible

General differencesDirectX

Graphics API and runtime library for Windows Store appsLower modules map directly to GPU componentsDirect access to hardware layout; app can precisely manage resources and processingHigher-level modules (Direct2D, MPT) built upon lower modules to simplify developmentIHV consistency via certification

OpenGL

Rendering spec and multi-vendor implementation for Desktop appsHardware-agnostic rendering pipeline abstraction Abstracted for hardware diversity, runtime manages most resourcesOpenGL provides higher-level modules via 3rd party libraries (e.g. SDL)IHV differentiation via extensions

Shader comparisonsGLSL (OpenGL)Procedural, step-centric (C like)Remain legible for compilation at run timeRow-major matrices (default)Typical vector type: vec2/3/4texture2D [function]sampler2D [datatype]lowp, mediump, highpuniform variablesVarying passes data between pipeline stages

HLSL (Direct3D)Object Oriented, data-centric (C++ like)Precompiled for performance and obfuscationColumn-major matrices (default)Typical vector type: float2/3/4texture.Sample [method]Texture2D [datatype]min10float, min16floatconstant buffersData transfers via input layout declarations

Optional DirectX componentsHigh-speed processing with DirectXMath:C++ SIMD graphics math library (calculating T, S, R transformation matrices)Includes SSE2 and ARM-NEON extensions

Common texture operations with DirectXTex:Leverages Windows Image Component (WIC) APIsResizing, format conversion, mipmap generationRead/write DDS filesEncode/decode Block compression (BC) textures for Direct3D runtimeHeight-map to normal-map conversion Simple .TGA reader/writer

Optional DirectX Took Kit (DirectXTK)SpriteBatch - simple and efficient 2-D sprite renderingSpriteFont – bitmap-based text renderingEffects – set of built-in shaders for common rendering tasksGeometricPrimitive – draws basic shapes such as cubes and spheresCommonStates – factory providing commonly used D3D state objectsVertexTypes – structures for commonly used vertex data formatsDDSTextureLoader – lightweight DDS file texture loaderWICTextureLoader – WIC-based image file texture loaderScreenGrab – lightweight screen shot saver

OpenGL/DirectX code comparison

//The incoming vertex' positionattribute vec4 position; //And its colorattribute vec3 color; //The varying statement tells the shader pipeline that this variable//has to be passed on to the next stage (to the fragment shader)varying vec3 colorVarying; //The shader entry point is the main methodvoid main(){colorVarying = color; //Pass the color to the fragment shadergl_Position = position; //Copy the position}

Vertex shader OpenGL example

// Define the vertex and pixel shader structsstruct VertexShaderInput{    float3 pos : POSITION;    float4 color : COLOR;}; struct PixelShaderInput{    float4 pos : SV_POSITION;    float4 color : COLOR;};

// Pass position and color values from Vert struct to Pixel struct (adding W and Alpha)PixelShaderInput SimpleVertexShader(VertexShaderInput input){    PixelShaderInput vertexShaderOutput;    vertexShaderOutput.pos = float4(input.pos, 1.0f);    vertexShaderOutput.color = float4(input.color, 1.0f);     return vertexShaderOutput;}

Vertex shader Direct3D example

varying vec3 colorVarying; void main(){//Create a vec4 from the vec3 by padding a 1.0 for alpha//and assign that color to be this fragment's colorgl_FragColor = vec4(colorVarying, 1.0);}

Fragment shader OpenGL example

// Collect input from vertex shader    struct PixelShaderInput{    float4 pos : SV_POSITION;    float4 color : COLOR;};

// Set the pixel color value for the specified Renter Target float4 SimplePixelShader(PixelShaderInput input) : SV_TARGET{    return input.color;}

Pixel shader Direct3D example

// Bind shaders to pipeline (both VS and FS are in a program)glUseProgram(m_shader->getProgram()); // (Input Assembly) Get attachment point for position and color attributesm_positionLocation = glGetAttribLocation(m_shader->getProgram(), "position“);glEnableVertexAttribArray(m_positionLocation);

m_colorLocation = glGetAttribColor(m_shader->getProgram(), “color“);glEnableVertexAttribArray(m_colorLocation); // Bind the vertex buffer objectglBindBuffer(GL_ARRAY_BUFFER, m_geometryBuffer);glVertexAttribPointer(m_positionLocation, 4, GL_FLOAT, GL_FALSE, 0, NULL);glBindBuffer(GL_ARRAY_BUFFER, m_colorBuffer);glVertexAttribPointer(m_colorLocation, 3, GL_FLOAT, GL_FALSE, 0, NULL); // Draw a triangle of 3 vertices!glDrawArray(GL_TRIANGLES, 0, 3);

Rendering OpenGL example

// Binding Shaders to pipeline (VS and PS)m_d3dDeviceContext->VSSetShader(vertexShader.Get(),nullptr,0);m_d3dDeviceContext->PSSetShader(pixelShader.Get(),nullptr,0); // Declaring the expected inputs to the shadersm_d3dDeviceContext->IASetInputLayout(inputLayout.Get());m_d3dDeviceContext->IASetVertexBuffers(0, 1, vertexBuffer.GetAddressOf(), &stride, &offset);

// Set primitive’s topologym_d3dDeviceContext->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);

// Draw a triangle of 3 vertices!m_d3dDeviceContext->Draw(ARRAYSIZE(triangleVertices),0); 

Rendering Direct3D example

Cat In The HatPorting Oceanhouse Media’s book engine to WinRT

Ken MaffeiLead Programmer

Dr. Seuss CollectionLittle Critter CollectionSmithsonian5 Little MonkeysBerenstain Bears

Oceanhouse Media book titles

Drag picture to placeholder or click icon to add

Android (Java), 2D Canvas WebOS (C++), OpenGL Ported the WebOS code to WinRTApp is matrix-driven for bitmaps and text

Click icon to add picture

Path to WinRT

Ramp-up to WinRT

1. Direct port elements

2. Obviously had to change

3. Potential re-write using new WinRT concepts

Three categories of effort

First things first – item #3

Identify where the answers were

Confidence we could implement everything when needed

Confidence in rewrite items

WinRT sample projects

The samples provided a great resource:Playing back multimediaReading up font collections from diskAdding effects, such as outlining, to text

Often, cut and paste just works!

The samples are invaluable

API documentation

Documentation includes sample code and descriptions of concepts such as:

The Asynchronous modelFile I/OMaking HTTP requests

Familiarize yourself with the API documentation

Some WinRT coding is low-level

WinRT API is powerful and flexible but…Certain tasks may require a fair amount of codeGood News! The sample code always worked and easily modified

Sometimes closer to the metal than you are used to

Beginning the port

Deferring some features allowed smooth C++ code transitionType conversions (Rects, Points and Bitmaps) were straightforwardSwitched from UTF8 and std::string to Unicode and Platform::String^ There were also some very unfamiliar types…

C++ code comes straight across, but there were lots of type conversions

// Get the user application data folderWindows::Storage::StorageFolder^ localFolder = ApplicationData::Current->LocalFolder;

// Create an asynch task to open the fileconcurrency::task<StorageFile^> getFileOperation(localFolder->GetFileAsync(fileName));

// Read the contents of the file asynchronouslygetFileOperation.then([this, m_d2dContext](StorageFile^ file){

return FileIO::ReadBufferAsync(file);}).then([this, m_d2dContext](concurrency::task<IBuffer^> previousOperation) {...}

Local storage and asynchronous file I/O

Some things were just easier in WinRT!

Memory: no code forking requiredLarge bitmaps are not a problemAnimated Text: high-quality direct manipulation with Direct2D and DirectWrite

Direct2D simplified many tasks for us

// In Direct2D/DirectWrite, manipulating text is // equivalent to manipulating bitmaps// Set up scale, rotation & translation transforms// These can be animated each frameD2D1::Matrix3x2F r = D2D1::Matrix3x2F::Rotation(angle);D2D1::Matrix3x2F s = D2D1::Matrix3x2F::Scale(scale.x, scale.y);D2D1::Matrix3X2F t = D2D1::Matrix3X2F::Translate(x, y);m_d2dContext.SetTransform(s*r*t);

// If bitmapm_d2dContext->DrawBitmap(bitmap.Get(), args);

// If textm_textLayout.Get()->Draw(m_d2dContext, args);

Text rendering was finally easy

WinRT design and UI guidelines

They will impact your architectureCriteria misses can disqualify your appAccommodate expected behaviors:App barsCharmsSwipes, right-clicks, etc.Icon/logo design

Don’t wait! Consider this early!

This is VERY important

SummaryDirect2D is powerful, great for graphics and text effectsThere will be a spectrum of challengesEmbrace the sample code and docsUnderstand and adhere to the UI guidelines

Next stepsMake your app visually pleasing, with fast & fluid, beautiful graphics!Learn from docs & samplesCheck out the tutorials & talksDesign key experiences firstGet started with the VS templatesParticipate in the Developer Center forums

Check out these other talks:3-115 Introduction to creating Windows Store apps using HTML and JavaScript3-001 Building a Windows Store app using XAML and C++3-109 Developing a Windows Store app using C++ and DirectX4-102 Performance tips for Windows Store apps using DirectX and C++

For more information visit: www.dev.windows.com

Thank You!

Q&A

• Develop: http://msdn.microsoft.com/en-US/windows/apps/br229512

• Design: http://design.windows.com/

• Samples: http://code.msdn.microsoft.com/windowsapps/Windows-8-Modern-Style-App-Samples

• Videos: http://channel9.msdn.com/Windows

Resources

Please submit session evals by using the Build Windows 8 appor at http://aka.ms/BuildSessions

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.