HTML5 VIDEO FILTERSIN SEARCH OF THE DRAGON BALLS

Created by giorgio.beggiora@artigianidelweb.it

OBJECTIVEShow in a browser a video with the following controls:

GrayscaleBrightnessContrastRGB channels

LOOKING FOR AN IDEA1. The mysterious SVG filters?

✗ potentially the best option, but 2014 is too early because...✗ not so good browsers' support✗ IE9- (XP/Vista) don't support SVG at all

2. The lightweight CSS filters?✗ bad browsers' support✗ no support for feColorMatrix SVG filter

3. The powerful WebGL?✗ bad browsers' support✗ unfamiliar enviroment for most developers✗ not W3C, depends from graphic board's drivers

4. The hungry Canvas?✓ good browsers' support (IE9+)✓ polyfills for IE8 (but don't rely on them)✓ familiar enviroment for most developers

A VIDEO IN A CANVAS var videoElement = document.getElementById('myVideo') ; var canvasElement = document.getElementById('myCanvas') ; var canvasContext = canvasElement.getContext('2d') ; canvasContext.drawImage(videoElement,0,0,width,height) ; // loop

DEMO

VIDEO854 × 480 = 409.920 px× 4 channels = 1.639.680 byte

CANVAS

Red 80%

Green 80%

Blue 80%

Contrast 80%

Brightness 80%

Black & white

13.9FPS

BONUS

CHROMA KEY

LET'S TRY FULL HD

DEMO FULL HD

WHAT DO WE NEED?

MORE POWER!

PROGRAMMING TIPSconsider not using frameworks or polyfillsaccess DOM as little as possiblereduce the number of dotscache everything (i said EVERITHING!)don't write 2 times the same thinguse while(i--) insead of for(i++)multiply instead of dividerequestAnimationFrame()['a','b'].join('') instead of 'a'+'b'bitwise operations when convenient (i.e. floor if n>=0)O'Reilly - High Performance JavaScriptjsPerf, StackOverflow, Google

AND NOW, WHAT DO WE NEED?

MORE POWER!

TYPED ARRAYSDirect access to RAM (IE10+)

// Floating point arrays.var f64 = new Float64Array(8);var f32 = new Float32Array(16);

// Signed integer arrays.var i32 = new Int32Array(16);var i16 = new Int16Array(32);var i8 = new Int8Array(64);

// Unsigned integer arrays.var u32 = new Uint32Array(16);var u16 = new Uint16Array(32);var u8 = new Uint8Array(64);var pixels = new Uint8ClampedArray(64);

Examplevar a = new Uint8Array(64); // length = 64 // 0 < value < 2^8 ;a[0] = 300 ;console.log (a[0]) // 44 = 300 - 256

BUFFERS AND VIEWSvar buffer = new ArrayBuffer(16); // 16 byte = 128 bit of RAM// to access it, you need to create a viewvar int32View = new Uint32Array(buffer); // length = 4 (128/32) // each element = 0 < value < 2^32// you can create multiple views of the same buffervar int8View = new Uint8Array(buffer); // length = 16 (128/8) // each element = 0 < value < 2^8// what does it mean?int32View[0]=300;

console.log(int32View[0]); // 300console.log(int32View[1]); // 0

console.log(int8View[0]); // 44console.log(int8View[1]); // 1

// it means that the Endianess is "Little-endian" (first byte = less significant)

YOU ARE ALREADY USING THEMCanvas 2D

alert(canvasElement.getContext('2d').getImageData(0,0,w,h).data);// in most recent browsers, "data" is an [object Uint8ClampedArray]// in older browsers, "data" is an [object CanvasPixelArray]// the cause is an HTML5 spec change

var a = new Uint8ClampedArray(2); // length = 2 < value < 2^8 ;a[0] = 300 ;console.log (a[0]) // 255console.log (a[1]) // 0

XMLHttpRequest2

File APIs

Media Source API

WebGL

Transferable objects

Binary WebSockets

AND NOW, WHAT DO WE NEED?

MORE POWER!

WEB WORKERS (IE10+)Execute JavaScript in background (multithreading!!!)

Honestly, i had no time to try them :P

demo by Conor Buckley using getUserMedia()http://bit.ly/10YhHoK

LET'S TRY ALL THIS STUFF(except for web workers)

DEMO FULL HD OPTIMIZED

AND NOW, WHAT DO WE NEED?

MORE POWER!

PIXI.JS2D WebGL framework with canvas fallback O.O

pixijs.com

(video with audio will autoplay on next slide)

DEMO FULL HD WEBGL

WHAT IS THE (COLOR)MATRIX?var brightness_colorMatrix = [// r g b a /*R*/ 1, 0, 0, x,/*G*/ 0, 1, 0, x,/*B*/ 0, 0, 1, x,/*A*/ 0, 0, 0, 1];

// R = r*1 + g*0 + b*0 + a*x ;// G = r*0 + g*1 + b*0 + a*x ;// B = r*0 + g*0 + b*1 + a*x ;// A = r*1 + g*0 + b*0 + a*1 ;

THANK YOUBY

GIORGIO . BEGGIORA @ ARTIGIANI DEL WEB . IT