Hardware Acceleration on Mobile, Ariya Hidayat & Jarred Nicholls

Wednesday, November 2, 11

Ariya Hidayat & Jarred NichollsSencha

Twitter: @ariyahidayat @jarrednicholls

Hardware Accelerationon Mobile

Challenges

Game vs Web

Animation

Physics

Textured objects

Transformation

Web Pages

Images

How Browser Works

Layout engine

Network stack

GraphicsJavaScript engine

User interface

WebKit Components

WebKit Library

JavaScriptCore

WebCore

HTMLrendering

DOM CSS

Platform Abstraction

Network Unicode Clipboard

Graphics Theme Events

Thread Geolocation Timer

Graphics Abstraction

Mac Chromium Qt Gtk

CoreGraphicsSkia

QPainter

graphics stack

GraphicsContext

Components of Mobile

Touch interface

Graphics Processor

Divide and conquer

Very specific purpose

Optimized for Games

Transformation

Textured triangles

Fixed geometry

Parallelism

Challenges

Predictable contents (assets) ✔Mostly text ✔Mostly images ✔Expected run-time response immediate

Primitive Drawing

Path is Everything

Triangle Rectangle

PolygonEllipse

Stroke = Outline

Solid Dashed Dotted Textured

Brush = Fill

SolidNone Gradient Textured

Smooth via Antialiasing

Multiple levels of transparency

Perfect for parallelism

Path Approximation

Curves

Polygon

Gradient: ExpensiveCPU: sequential, tedious

GPU: parallel, still a lot of work

Blur Shadow: Really Posh

Involved pixel “blending”

GPU: parallel, still tedious

Transformation

Scaling

RotationPerspective

Text Rasterization

Font Atlas

Buffer

Simple to Complex

Hello!

Simple shapeSolid color

CurvesGradient brushTextured stroke

Blur shadowSerif textRotated

Make it as an image

HOW FAST?

Android Drawing

GraphicsContext

WebCore graphics

Log file

Example: BingplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff ffrestorePlatformStateplatformDestroyplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff ffrestorePlatformStateplatformDestroyplatformInitsavePlatformStatetranslate 0,0translate 0,0clip 1,0 0x6.95322e-310fillRect 0,0 800x556 color ff ff ff fffillRect 0,0 800x556 color ff ff ff ff

How Fast?

#include "TimeCounter.h"

bool WebViewCore::drawContent(SkCanvas* canvas, SkColor){ uint32_t timestamp = getThreadMsec();

.... painting code .... DBG_SET_LOGD("% ms", getThreadMsec() - timestamp);}

external/webkit/WebKit/android/jni/WebViewCore.cpp

Example: Google News

16:24:04.070 D/webcoreglue( 273): drawContent 11 ms16:24:04.110 D/webcoreglue( 273): drawContent 13 ms16:24:04.150 D/webcoreglue( 273): drawContent 13 ms16:24:04.190 D/webcoreglue( 273): drawContent 10 ms16:24:04.240 D/webcoreglue( 273): drawContent 10 ms16:24:04.280 D/webcoreglue( 273): drawContent 13 ms16:24:04.320 D/webcoreglue( 273): drawContent 13 ms16:24:04.360 D/webcoreglue( 273): drawContent 13 ms16:24:06.080 D/webcoreglue( 273): drawContent 12 ms16:24:06.140 D/webcoreglue( 273): drawContent 10 ms16:24:06.180 D/webcoreglue( 273): drawContent 13 ms16:24:06.230 D/webcoreglue( 273): drawContent 14 ms16:24:06.600 D/webcoreglue( 273): drawContent 26 ms16:24:06.640 D/webcoreglue( 273): drawContent 13 ms16:24:06.860 D/webcoreglue( 273): drawContent 33 ms16:24:06.890 D/webcoreglue( 273): drawContent 12 ms16:24:06.930 D/webcoreglue( 273): drawContent 13 ms16:24:06.960 D/webcoreglue( 273): drawContent 13 ms16:24:07.000 D/webcoreglue( 273): drawContent 13 ms

adb logcat -v time | grep drawContent

Backing Store

Pinch to Zoom

when you pinch...

Responsive UI

Processor

Rendering

User interaction

decoupled

Rendering vs Interaction

Web Page

Backing StoreScreen

Rendering

User interaction

Checkerboard Pattern

Progressive Rendering

Crisp but slow

Fast but blurry

Tiling System

CPU GPU

........

Texture upload

Tile Transformation

Panning = Translation Zooming = Scaling

Backing Store Support

Texture-based

Vector-based

Honeycomb and later

Y U NOposition:fixed

Layer & Compositing

Elements = Layers

Typical Animation

opacity, movement, scaling, rotation, ...

Immediate Mode

every animation tick

setInterval(function() { box.draw(x, y); x += 10;}, 20);

Scene Graph

box.drawInto(layer);setInterval(function() { layer.translate(10, 0);}, 20);

Change transformation matrix

Logical 3-D

Keep the Textures

Compositing Support

Honeycomb and later

Well-known Trick

-webkit-transform: translateZ(0)

forcing 3-D transform

Not needed for CSS animation!

Debugging in Safari

defaults write com.apple.Safari IncludeInternalDebugMenu

Compositing Indicators

Composited layer

Container layer

No texture

Overflow

Texture (number = upload)

Debugging in Chrome

about:flags

Avoid Texture Reupload

No reupload Upload

OpacityPosition

SizeAnimation

Everything else!

“Hardware accelerated CSS”

Avoid Overcapacity

GPU = Limited silicon spaceLarge layer ➔ broken into “tiles”

Prepare & Reuse

Viewport

Hide the layer offscreen

Wrap Up

Hardware Acceleration

Drawing primitives

Backing stores

Layer & compositing

THANK YOU!

QUESTIONS?

ariya @ sencha.com

ariyahidayat

ariya.ofilabs.com

jarred @ sencha.com

jarrednicholls

Hardware Acceleration on Mobile, Ariya Hidayat & Jarred Nicholls

Technology

idigf - pembangunan - mastel - kanaka hidayat

Hidayat Ut Talibeen Urdu translation

Jarred Dunn, Pianist McGill University Schulich School of

Grant Thornton Hendrawinata Gani Hidayat - Investor …investor.gmrgroup.in/AnnualAccountSubsidiariesPDF/2009-10/PT...Grant Thornton . Hendrawinata Gani & Hidayat ':.(J3!I'JJlnQ

The Louisiana Purchase By Ryan Bilger & Jarred Ring

Hidayat Al-Mutallimin Bodl. MS

An. HIdayat-Ismael_Kascil_dr.diding, Sp.pd, FINASIM

Bidayat ul hidayat

Skiripsiku (AMBOSA HIDAYAT, S. Hut)

Jarred Nov 22

Jarred Beall Landscape Portfolio

Landscape Architecture by Jarred Beall

Kuliah Dr Edi Hidayat

Faiz Syauqi Hidayat-Portfolio

Jarred Fischer About Carpentry

Budi Hidayat Kesinambungan Pendanaan JKN

Purnama Hidayat, PhD

Pipeline Processor Design Project Jarred Beck

Jurnal TA Amir Hidayat

The Coming Buddha Ariya Metteyya