June 2010

Testing Mobile JavaScriptJohn Resig

I’ve been researching and trying to makesense of the mobile space recently.

Started preparations in 2009 (with the creation of TestSwarm).

Cross-browser mobile web development is crazy.(...but not nearly as crazy as it use to be.)

I’ve seen things...

A Simple Goal

✤ Started with a simple goal:

✤ To make sure that jQuery works on the most popular mobile platforms and browsers.

✤ As it turns out, it’s really hard to define the scope of the problem.

Questions: Answered

✤ Three questions that need to be answered before doing mobile development:

✤ What platforms and browsers are popular?

✤ What browsers are capable of supporting modern scripting?

✤ What devices and simulators do I acquire to test with?

✤ To answer those questions we need good data.

Best Stats?

✤ Who has the best statistics on the mobile market?

✤ StatCounter and Gartner seem to be tops.

✤ StatCounter covers billions of hits per month.

✤ Gartner is well-respected and knows sale information well.

✤ AdMob is decent for specific platforms (iPhone, Android).

✤ That being said: Very little actually-useful information leaks out.

Lack of Information

✤ Right now finding this information is a challenge.

✤ As a result, developers develop for what’s in front of them:

Platforms

Platform Sales

Hard Question

✤ What versions of those platforms are popular?

✤ No one seems to know, or isn’t talking about it.

Browsers

Hard Question

✤ What versions of those browsers are popular?

✤ No one seems to know, or isn’t talking about it.

✤ Supposedly Yahoo is going to be releasing some information soon, we’ll see.

✤ Right now it’s considered to be a competitive advantage to keep it private.

Testing Strategy

✤ Draw a line in the sand

✤ Buy Devices, Download Simulators

✤ Automate testing (TestSwarm!)

Drawing a Line

✤ Yahoo!s Graded Browser Support technique is optimal.

✤ Qualify which browsers you will support (actively test against) and give them a grade.

✤ A = Full support, C = Fall back to old site, Other = Assume full support.

Mobile Graded Browser Support: Market share + Browser quality

My God... it’s full of browsers...

Testing Browsers

✤ Two options:

✤ Buy devices and/or

✤ Download simulators

✤ Simulators are good for most automated testing.

✤ Any sort of interaction testing you’ll want to do on a physical device.

✤ Always good to do a sanity check on a physical device before going live.

Simulators

✤ Simulators are available for most platforms and browsers.

✤ Most simulators require Windows to run (some require Perl or Java)

✤ Some browsers even provide standalone executables (Opera, Fennec).

✤ Getting the simulators running can be a real bear.

Automated Testing

✤ Once you have simulators (or physical devices) up and running you’ll want to interact with them as little as possible.

✤ Automated test execution will be really important.

✤ TestSwarm was developed for this express purpose: Make it easy to push tests out to a large number of clients (even mobile).

Platforms

✤ Symbian (S60 / UIQ)

✤ iPhone OS

✤ Blackberry OS

✤ Android

✤ Windows Mobile

✤ WebOS

✤ Maemo / Meego

Symbian

Symbian S60

✤ The most popular mobile OS - heavily used by Nokia.

✤ v5.0 is for touch screen devices (Equiv. to Safari 3.1)

✤ v3.0 is the latest for ‘regular’ devices (Equiv. to Safari 2.0)

✤ 2 ‘feature packs’ have been released

✤ The oldest, active, mobile browser that should be supported.

✤ Simulators available on Nokia.com.

Symbian S60

AB

B?

Symbian UIQ

✤ A now-dead implementation of the Symbian OS last released in 2008.

✤ Appears to still be in use, some what, but quickly becoming irrelevant.

✤ Uses Opera Mobile (8.6, 8.65) as its built-in browser.

✤ UIQ is dead and Nokia doesn’t distribute the simulator any more. I found a guy in Russia that has some extra copies (totally legit, heh).

Symbian UIQ

C C

iPhone OS

✤ Apple’s super-popular mobile OS

✤ Apple is very good about pushingupdates, almost all users are on thelatest OS version.

✤ Current with Safari 4

✤ A gotchya: Does not have fixed position support (making it difficult to implement toolbars). Recommend: TouchScroll library.

✤ Simulator is part of the iPhone SDK.

iPhone Simulator

No 2.xsimulator!

B

A

A

Blackberry OS

✤ Blackberry continues to be enormously popular - and growing rapidly.

✤ Ships with a custom browser, will be switching to a WebKit-based one in 6.0.

✤ Anything older than Blackberry 4.6 is really frightening.

✤ Andrew Dupont calls 4.6 the “rubicon”.

✤ 4.6/4.7 have some strange quirks, 5.0 is pretty decent.

✤ Blackberry provides simulator downloads on their dev site.

Blackberry OS

B

B

B

Android

✤ Rapidly growing OS from Google.

✤ Tons of manufacturers are modifying and shipping it.

✤ Expect Android to grow drastically over the next couple years.

✤ Lack of control has yielded extreme version fracturing.

✤ Simulators are an easy download.

Android

AA

A

Windows Mobile

✤ Windows Mobile 6.5 is the current release.

✤ Uses IE 6 as its rendering engine.

✤ Window Mobile 7.0 is coming soon.

✤ Uses IE 7 as its rendering engine. :-( :-( :-( :,(

✤ Window Mobile 6.1 is still relatively popular but is based off of IE 4.0 - this is a non-starter.

✤ Windows Mobile 6.5 simulator is an easy download.

Windows Mobile

7.0? :-(

BF

WebOS

✤ Palm’s (HP’s?) Mobile OS

✤ Has a minimal amount of traction.

✤ Built off of WebKit, easy to support.

✤ 1.4 simulator ships with 1.3 browser (wtf!?)

✤ I’m testing using a physical Palm Pre.

✤ Palm Pre simulator is an easy download.A

Maemo / Meego

✤ Linux-based OS used for tablets (and potentially phones). Used by Nokia.

✤ Not very popular.

✤ Has a browser called ‘MicroB’ that uses Gecko.

✤ Mostly interesting as Fennec is capable of running on it.

✤ Simulator requires Linux to run - haven’t figured it out yet.

C

Platform-Independent Browsers

✤ A number of browsers target multiple platforms. Some work directly with carriers (Opera, Netfront) whereas others provide downloads (Opera, Fennec).

✤ Opera

✤ Fennec

✤ Netfront

✤ Phonegap (App Platform)

Opera Mobile and Opera Mini

✤ Hugely popular mobile browser (especially Mini).

✤ Available on a large number of platforms.

✤ Even ships as the default browser on some.

✤ Opera Mobile and Opera Mini couldn’t be more different.

✤ Mobile is a full-featured browser.

✤ Mini is a glorified bitmap viewer.

✤ Pulls from a proxy, no JS executing on client.

Opera Mobile and Opera Mini

A

C

CSimulator is awesome!

Fennec

✤ Codename for Firefox on mobile devices

✤ Released for Maemo, alpha release out for Android (watch this!)

✤ Equivalent to the latest releases of Firefox

A B

Simulator is awesome!

Netfront

✤ Used on Playstation and N-Gage

✤ Download available for Windows Mobile.

✤ Highly crippled, custom, browser.

✤ Not worth supporting.

✤ Compatibility can be a real pain.

C

Phonegap

✤ Not a browser, a way to develop cross-platform applications.

✤ Uses WebKit as the rendering engine.

✤ Use HTML/CSS/JS to develop deployable apps.

✤ Quite popular, easy to support.

A

Other Browsers

✤ Not worth the time for such small market share.

✤ Ozone (Uses WebKit)

✤ Openwave (Dead)

✤ Bolt (Uses WebKit)

✤ JIL (Uses WebKit, Widget Platform) C

Summary

✤ What do we need to support? Roughly:

✤ Opera 9.5, 10

✤ Internet Explorer 6, 7

✤ Firefox 3.5+

✤ Safari 2, 3, 4

✤ (and Blackberry... weeee...)

✤ Don’t assume it’ll work, download the simulators and test!

What Devices to Buy?

✤ For level:

✤ 1 iPhone 3GS: Latest running 3.x (* Watch for new 4.x iPhone)

✤ 1 iPad: Running 3.2

✤ 1 Nokia N97 Running S60v5

✤ 1 Palm Pre: Running 1.4

✤ 2 Android Phones: HTC Magic (1.5/1.6), Droid Incredible (2.1)(* Watch for new 2.2 Android)

✤ And download the Opera and Fennec Simulators and Phonegap

A

-Grade Costs

✤ $315 - iPhone 3GS

✤ $499 - iPad

✤ $370 - Nokia N97

✤ $399 - Palm Pre

✤ $349 - HTC Magic

✤ $637 - Droid Incredible

✤ Total: $2569

A

What Devices to Buy?

✤ For level (in addition to A level):

✤ 1 1st Gen iPhone: Running 2.x

✤ 1 Nokia N96: Running 3.0v2

✤ 3 Blackberry Devices: Curve 8900 (4.6), 2x Tour 9630 (4.7 and 5.0)

✤ 1 HTC HD2: Running Windows Mobile 6.5

B

-Grade Costs

✤ $315 - 1st Gen iPhone

✤ $360 - Nokia N96

✤ $310 - Curve 8900

✤ $350 x 2 - Tour 9630

✤ $600 - HTC HD2

✤ Total: $2285

✤ -Grade + -Grade: $4854

B

BA

More Information

✤ Coming Soon at:

✤ http://jquerymobile.com/

✤ See also:

✤ http://testswarm.com/ and

✤ http://swarm.jquery.org/

✤ And:

✤ http://quirksmode.org/m/