Embed Size (px)
Citation preview
BW2Session6/8/1611:30AM
MobileAppTesting:Planning,Priorities,Execution,andReporting
Presentedby:
JasonArbon
Appdiff.com
Broughttoyouby:
350CorporateWay,Suite400,OrangePark,FL32073888---268---8770··[email protected]://www.techwell.com/
JasonArbonAppdiff.comJasonArbonistheCEOofAppdiff,whichisredefininghowenterprisesdevelop,test,andshipmobileappswithzerocodeandzerosetuprequired.HewasformerlythedirectorofengineeringandproductatApplause.com/uTest.com,whereheledproductstrategytodelivercrowdsourcedtestingviamorethan250,000communitymembersandcreatedtheappstoredataanalyticsservice.JasonpreviouslyheldengineeringleadershiprolesatGoogleandMicrosoft,andcoauthoredHowGoogleTestsSoftwareandAppQuality:SecretsforAgileAppTeams.
Mobile App TestingPlanning, Priorities, Execution, and Reporting
Jason Arbon, CEO of Appdiff
Agile + CI. It is all about Speed.
2
Background
My Background
3
Background
Plug for my books
4
Mobile is Different
5
Fewer tests -> faster and more agile
6
Automation -> critical to get value from CI
Goal: Faster Testing
Monsters: risk
7
Attributes: user feedback
Where to Focus
8
Agile Deployment Competition Devices Real-World
UsersSecurity UX Reviews Metrics
Mobile Quality Monsters
9
Stability Satisfaction Usability Devices Inter-Op
PricingSecurity Elegance Content Privacy
Mobile Quality Attributes
Review attribute frequency
10
Review attribute scores
Where to Focus by App Category
Games
+Content+Satisfaction
-Privacy-Security
11
Users Care
Low Priority
Finance
+Usability+Security
-Elegance-Privacy
12
Low Priority
Users Care
Games
+Privacy+Stability
+Usability+Satisfaction
13
Hard to Get Right
Easy, but nail it!
Finance
+Pricing+Stability
+Usability+Performance
14
Easy, but nail it!
Hard to Get Right
How To Test
15
Now that we know what to test
Beta Testing Crowd Testing
Manual Testing
Beta Management (Applause)
+Fast and Easy Bug Reporting+Auto distribution and Update of CI Builds
Unit Test Automation
18
Embarrassingly EasyFew Do It
High Value in Small Numbers
Often pass even when app is unusable
UI-Based Test Automation
19
Embarrassingly difficult + expensiveVerify basic functionality so humans don’t have to
Verify $ and differentiating features of App
20
Warning: Most Automation Efforts
FAIL
21
But, Everyone Tries Anyway
22
Many Fail More than Once
Automation Frameworks
25
AppiumMany Languages, Open Source,
iOS + Android + Web60% code-reuse cross plat
iOS-Only : XCTest. UI as of Xcode 7Android-Only: Espresso. Fast.
Simulators and Emulators
26
Good enough for 90%Every dev machine has one
Cloud Providersif you have > 100 tests
Parallelism for CI/Agile Speed
27
Google Test Cloud Devices
Real Devices
28
If you like pain and spending moneyAfter you have done everything else
Only if you have to….
Real Devices
29
In-House Device LabsBetter things to spend time and money on
Cloud ProvidersCan be flaky
Expensive, Google Least ExpensiveDifficult Local Repro’s
Testing Focus Allocation for Agile Teams
30
Performance
31
Highest priority for most app teams
SDKsTrack real world performance from inside your app
Install and MaintainData after Users Impacted
Mobile Automation and Performance
Without the pain
33
Appdiff. Completely Automated< 15 Minutes. Thousands of Tests.
http://www.appdiff.com/bscwest
36
![Presentazione FY11 UK.ppt [modalità compatibilità] · Superior Execution IS the Strategy …Priorities #1 Protect capital and liquidity #2 Deliver sustainable profitability in excess](https://img.pdfslide.us/doc/110x75/6024b42c8960720d690aa808/presentazione-fy11-ukppt-modalit-compatibilit-superior-execution-is-the-strategy.jpg)
