Agile Tools for Mobile

Agile Tools for Mobile

By Kevin Rohling

ABOUT ME

• Shipped apps on iOS, Android, Windows Phone… even a Blackberry app once. • Founded cisimple (CI for Mobile), sold it to Electric Cloud, now it’s called Ship.io • Startup junkie, currently VP of Product at Emberlight • Likes to hack on hardware things

WHAT IS EMBERLIGHT YOU ASK?

PRE-ORDER at emberlight.co

http://emberlight.co

WHAT DOES AGILE MEAN (TO ME)?

MOVE FAST AND ADAPT QUICKLY


MOVE FAST AND ADAPT QUICKLY

Release&

Collect&Data&Build&

Test&


ALWAYS BE READY TO SHIP

ONE SIZE DOES NOT FIT ALL

Business Drivers Engineering Challenges

Early Startup Initial Traction, Funding Stability, UX

Consulting Firm Client Status Updates, Deadlines Ad Hoc Distribution, Unit Testing

Large Product Team Retention, Growth, Revenue Automation, Communication

PROCESS

MOBILE IS HARDER• Testing, Simulators or Devices?

MOBILE IS HARDER• Testing, Simulators or Devices? • App Stores

MOBILE IS HARDER• Testing, Simulators or Devices? • App Stores • Code Signing

MOBILE IS HARDER• Testing, Simulators or Devices? • App Stores • Code Signing • Distribution

TOOLS TO PROMOTE AGILITY-NESS• Carbon-Based (Human) Testing

• Distribution • Crowdsourcing

• Testing Tools • Unit Testing • Functional Testing

• Continuous Integration (CI)

CARBON-BASED (HUMAN) TESTING

• Fast, Detailed feedback • Avoid disrupting your actual users • Real people do the darn’dest things


• Fast, Detailed feedback • Avoid disrupting your actual users • Real people do the darn’dest things

YOUR ENGINEERS DON’T COUNT.


Challenges: • Distribution • Quality vs. Ranty feedback • Latency


Strategies: • Friends & Family • Curated Group of Testers • Testing Services • In-House Testing Team

$

CARBON-BASED TESTING DISTRIBUTION

CARBON-BASED (HUMAN) TESTING - DISTRIBUTION

Good: Great iOS support, acquired by Apple, iTunesConnect integration Bad: Only supports iOS, mediocre UX

Note: TestFlightApp.com no longer available after 2/26/15

http://TestFlightApp.com


Good: Supports iOS and Android, great API, great UX Bad: iOS UDID nightmare

Good: Alpha/Beta Program, Staged Rollouts, Uses your actual users Bad: Can result in negative app reviews, Android only


CARBON-BASED TESTING CROWD TESTING

CARBON-BASED (HUMAN) TESTING - CROWD

Good: Massive curated audience Bad: Expensive, Very enterprise-y


Good: Starts at $49 Bad: Value of video recording is debatable


TESTING TOOLS

UNIT vs FUNCTIONAL

• Unit Testing - testing small pieces of code • Functional Testing - testing button clicks and UI interaction

UNIT/FUNCTIONAL TESTING

BENEFITS: • Instant Gratification (GREEN is good!) • Repeatable • Can automatically test every commit

UNIT/FUNCTIONAL TESTING

CHALLENGES: • Unit Tests are not users • LOTs of work to write and maintain them • Which framework to use?

• What language/framework do your developers know? • Open Source/Community Support • 3rd Party Framework requirements

UNIT/FUNCTIONAL TESTING FRAMEWORKS

WHICH ONE TO USE?

TESTING TOOLS

POPULAR TESTING FRAMEWORKS (iOS)

Good: Baked into Xcode, Performance Testing Easy Async Tools

Bad: ?

XCTest


Good: Functional (UI) testing, Easy test recorder Bad: Part of Instruments, Uses JavaScript

UI Automation


Good: Open Source, Functional (UI) testing, Focus on usability Bad: Requires integrating 3rd Party Framework

KIF

TESTING TOOLS

POPULAR TESTING FRAMEWORKS (Android)

According to a recent survey by Xamarin: -77% of developers said app quality is “very important or mission critical” -80% of mobile developers test their apps manually -Only 13% of mobile developers use automated testing


Good: Baked into Android/Eclipse/Android Studio Bad: Running tests is slow


Good: Functional (UI) testing Bad: 3rd Party Framework, Running tests is slow


Good: Tests go fast Bad: 3rd Party Framework, Flaky

TESTING TOOLS - CROSS PLATFORM

POPULAR TESTING FRAMEWORKS (Cross-Platform)

Good: Built on Selenium (Web Driver) Bad: Requires a running web server

POPULAR TESTING FRAMEWORKS (Cross-Platform)

Good: Popular Cucumber syntax Bad: Requires a running web server on the device

CONTINUOUS INTEGRATION


AUTOMATICALLY…

Build Test Deploy

EVERY COMMIT.


HOW DOES IT WORK

1. Developer commits code


HOW DOES IT WORK

2. CI Server picks up the change.


HOW DOES IT WORK

3. Code is automatically compiled


HOW DOES IT WORK4. Tests are run


HOW DOES IT WORK

5. Application is deployed


WHY? • Reduce the need for human-based testing • Faster Feedback • Automate EVERYTHING

CI TOOLS/SERVICES FOR MOBILE

Good: Open Source, Lots of plugins Bad: Self-Hosted, DIY solution

Jenkins


Good: Hosted solution, OS X support, Lots of plugins Bad: Tedious setup process

Travis CI


Good: Hosted solution, Designed specifically for mobile, Easy setup Bad: Less flexible than other solutions

Ship.io


Good: Integrated w/ Xcode, Apple-Supported Bad: Self-Hosted, iOS Only

Xcode CI

KEVIN ROHLING @kevinrohling

[email protected]

mailto:[email protected]

Technology

Agile Tools for Mobile