audio, video and canvas in HTML5 - standards>next Manchester 29.09.2010

audio video and canvas in HTML5

Patrick H Lauke standardsgtnext Manchester 29 September 2010

NATIVE MULTIMEDIA SUPPORT

HTML5ltDOCTYPE htmlgt

ldquoextending the language to better support Web applications [] This puts HTML in direct competition with other technologies[] in particular Flash and Silverlightrdquo

Ian Hickson Editor of HTML5httplistsw3orgArchivesPublicpublic-html2009Jan0215html

making your site Fonzie compliant

ltvideogt

Adobe Flash currently most commonvideo delivery mechanism

ltobject width=425 height=344gtltparam name=movie

value=httpwwwyoutubecomv9sEI1AUFJKwamphl=enampfs=1ampgtltparamgt

ltparam name=allowFullScreen value=truegtltparamgt

ltparam name=allowscriptaccess value=alwaysgtltparamgt

ltembed src=httpwwwyoutubecomv9sEI1AUFJKwamphl=enampfs=1amp type=applicationx-shockwave-flash allowscriptaccess=always allowfullscreen=true width=425 height=344gtltembedgtltobjectgt

ltvideo src=videowebm controls Autoplay Loop preload=none poster=posterjpg width=320 height=240gt lta href=videowebmgtDownload movieltagtltvideogt

video as native object ldquoplays nicerdquo with rest of the page keyboard accessibility built-in

video formatsthe big debate

MP4 H264

ubiquitous patent encumbered licensingroyalties

Ogg Theora

ldquofree and openrdquo no licensingroyaltiesnot many tools for it not web optimised

wwwwebmprojectorg

WebM

open and royalty-free web optimised

wwwyoutubecomhtml5

providing multiple sourcesltvideo controls autoplay poster=hellip width=hellip height=hellipgt

ltsource src=moviewebm type=videowebm gtltsource src=movieogv type=videoogg gtltsource src=moviemp4 type=videomp4 gtlt-- fallback content --gt

ltvideogt

still include fallback for old browsershttpcamendesigncomcodevideo_for_everybody

ltvideo controls autoplay poster=hellip width=hellip height=hellipgtltsource src=moviewebm type=videowebm gtltsource src=movieogv type=videoogg gtltsource src=moviemp4 type=videomp4 gtltobject width=hellip height=hellip type=applicationx-

shockwave-flash data=playerswfgtltparam name=movie value=playerswf gtltparam name=flashvars value= hellip file=moviemp4 gtlt-- fallback content --gt

ltobjectgtltvideogt

powerful (simple) APIto script your own controls

icantcoukeasy-youtube

wwww3orgTRhtml5videohtmlmedia-elements

controlling a ltvideogt element

var v = documentgetElementById(player)

vplay()vpause()vvolume = hellip vcurrentTime = hellip hellip

events fired by ltvideogt element

var v = documentgetElementById(player)vaddEventListener(loadeddata function() hellip true)vaddEventListener(play function() hellip true)vaddEventListener(pause function() hellip true)vaddEventListener(timeupdate function() hellip true)vaddEventListener(ended function() hellip true)hellip

ltaudiogt

audio exactly the same as video

ltaudio src=rdquomusicmp3rdquo controls autoplaygtltaudiogt[]ltaudio controls autoplaygt

ltsource src=musicmp3 type=audiompeg gtltsource src=musicoga type=audioogg gtlt-- fallback content --gt

ltaudiogt

same format debacle MP3 vs Ogg Vorbis (vs WAV)

ltcanvasgt

canvas = ldquoscriptable imagesrdquo

ltcanvas width=hellip height=hellipgtltcanvasgt

canvas has standard API methods for drawing

ctx = canvasgetContext(2d)ctxfillRect(x y width height)ctxbeginPath()ctxmoveTo(x y)ctxlineTo(x y)ctxbezierCurveTo(x1 y1 x2 y2 c1 c2)

canvas drawing ready-made images

ctx = canvasgetContext(2d)var logo = new Image()logosrc = logopngctxdrawImage(logox1y1w1h1x2y2w2h2)

or call in an existing image already on the page

canvas access to image data array

ctx = canvasgetContext(2d)canvasData = ctxgetImageData(xywh)[RGBARGBARGBARGBA hellip ]

canvas exporting image data as datauri

ctx = canvasgetContext(2d)hellipcanvastoDataURL(imagepng) returns dataimagepngbase64hellip

canvastoDataURL(imagejpg) returns dataimagejpegbase64hellip

Note do it on the canvas not the context

canvas also works with video

ctx = canvasgetContext(2d)v = documentgetElementById(player)ctxdrawImage(vx1y1w1h2x2y2w2h2)

grab currently displayed frame (update as appropriate)

canvas accessibility concerns

video audio and canvas on any devicewithout plugins

(Java Flash Silverlight not ubiquitous)

HTML5 (and friends) haslots more APIs for developers

(for powerful client-side apps)

isgeolocationpartofhtml5com

geolocationnavigatorgeolocationgetCurrentPosition(success error)navigatorgeolocationwatchCurrentPosition(success error)function success(position)

wheres Wally var lat = positioncoordslatitudevar long = positioncoordslongitude

offline detection

windowaddEventListener(online function() hellip true)windowaddEventListener(offline function() hellip true)

and application cachelthtml manifest=rdquoblahmanifestrdquogtCACHE MANIFEST send this with correct textcache-manifest MIMEimagesspritespngscriptscommonjsscriptsjqueryjsstylesglobalcssdataxml

and many more(even beyond HTML5)

is it all safe to use right now

dont do browser sniffing

httpwwwflickrcomphotostimdorr2096272747

feature-detectionprogressive enhancement graceful degradation ndash or use shims

httpdiveintohtml5orgeverythinghtml

feature-detection for audiovideo

if (documentcreateElement(video)canPlayType) hellip if (documentcreateElement(audio)canPlayType) hellip

feature-detection for audiovideo codecs

var v = documentcreateElement(video)if ((vcanPlayType)ampamp ((vcanPlayType(videowebmcodecs=vp8vorbis) == probably) || (vcanPlayType(videowebmcodecs=vp8 vorbis) == maybe))) hellip

ready-made HTML5 audiovideo players(for the lazy)

sublimevideonet

videojscom

wwwhappywormcomjqueryjplayer

HTML5 as Flashkiller

not a question of HTML5 replacing Flash

giving developers a choice

wwwoperacomdeveloperpeopleoperacompatricklpresentationsstandards-next_29092010standards-next_29092010pdf

patricklaukeoperacom