Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
MobileAppTestingisnotsomethingspecial
Simon‘Peter’Schrijver
TesT-PRO
@simonsaysnomore
simonsaysnomore.wordpress.com
MycareerinMobile(App)Testing
• Between2006and2014
• TestManager
• Didalotoftestexecutionmyself
• Projects
• IntroductionofnewMVNO
• Upgradeofnetworksystems
• HLR
• IN
• CDRprocessing
• CMS
• Billing
• IntroductionofiPhone(2008)
• DataRoaming
MycareerinMobile(App)Testing
• BetweenJanuaryandOctober2014
• SeniorTelecomApplicationTester
• Projects
• BuildingHLR
• AuC
• HLR
• IN
• Automation
• SOAPUI
MycareerinMobile(App)Testing
• FromOctober2014upuntilnow
• SeniorTesterMobileApp
• Projects
• Jumbo
• ABN-AMRO
AMobileDeviceisspecial
• AsSteveJobssaidduring
theintroductionoftheiPod
“Iwantpeopletocarry
1000songsintheir
pocket”*
• HelateraddedtheInternet,
TouchandPhonetoit
*Source:http://genius.com/Steve-jobs-ipod-introduction-annotated
AMobileDeviceisspecial
• Youcarryalotofthingswithyouononedevice
• Itisapartof
everybody'slive
• Awholenewindustryis
created
AMobileDeviceisspecial
• Becauseitismoving
• Bringssomenewthingstotest
for
– Movement
– Edgeofnetwork
– Portrait/Landscape
– Touch
– Notifications
– Speedofdata
– Batteryofdevice
– Waitingtime
AMobileDeviceisspecial
• Networkinterfaces
– Bluetooth
– NearFieldCommunication
– GlobalPositioningSystem
– Wifi
– Cellularnetworks
• Sensors
FragmentationAndroidin2015
• Uniquedevices
24093(increaseof28%)
• Versions
Kitkat(4.4)runson37.8%of
thosedevices
• 1294Androidmanufacturers
• 1.4billionactiveuserper
monthThefirstdevicewentonsaleinOctober,2008
Source:https://androidhub.intel.com/en/posts/peachpellen/Infographic_Android_in_2015.html
Fragmentationbydevice
Source:http://opensignal.com/reports/2015/08/android-fragmentation/
Fragmentationbybrand
Source:http://opensignal.com/reports/2015/08/android-fragmentation/
WhatisaMobileApp?
Amobileapplication(or
mobileapp)isasoftware
applicationdesignedtorunon
smartphones,tablet
computersandothermobile
devices.Source:Wikipedia
TopConsumerMobileApplications
• MoneyTransfer
• Location-BasedServices
• MobileSearch
• MobileBrowsing
• MobileHealth
• MonitoringMobilePayment
• MobileMusic
• MobileAdvertising
• MobileInstantMessaging
Source:http://www.nielsen.com/us/en/insights/news/2015/tops-of-2015-digital.html
TypeofMobileApp?
Thereareprimarilythreetechniquesforcreating
MobileApps:
• Nativeapplications
• Mobilewebapplications
• Hybridmobileapplications
– Web-to-Appapplications
– Web-and-Appapplications
TypeofMobileApp?
Native
• Installedapplicationscanuseall
mobileortabletdevicefeatures
suchascamera,calendar,and
contacts.
• Supporttherichestuser
experience(forexample,gaming
applications)
• Builtusingeachphone'snative
SDK
Source:http://www.worklight.com
TypeofMobileApp?
Web
• Accessibleovertheinternet
withoutinstallingsoftware
• Usedevicebrowsersto
providenative-looking
applications
• Builtusingstandardweb
technologiessuchasHTML5,
CSSandJavaScript
Source:http://www.worklight.com
TypeofMobileApp?
Hybrid
• Leveragewebtechnologyand
nativedevicefeatures
• Installedapplicationsthatusethe
devicebrowsertodisplayweb-
baseduserinterface
• Abletousenativedevicefeatures
• Builtusingstandardweb
technologies(HTML5,CSS,JS)
andphone'snativeSDK.
Source:http://www.worklight.com
Nativevs.Webvs.Hybrid
Source:http://www.worklight.com
WhatisMobileAppTesting?
• Softwareonamobiledevice
• Interactwithanuser
• Senddataoveraninterface
• Toabackendsystem
• Waitforaresponse
• Andprocessthisresponse
WhatisMobileAppTesting?
• Fragmentation
– Devices
– Brand
– Operatingsystem
• Featuresonthedevice
• Native,Hybrid,Web
• FeaturesoftheApp
MobileAppTesting-Fragmentation
• Devices
• Brands
MobileAppTesting-Fragmentation
• Android– 4.0
– 4.1,4.2,4.3
– 4.4
– 5.0,5.1
– 6.0
• iOS
– 7.0,7.1
– 8.0-8.4
– 9.0–9.3
– 9.4?
– 10.0(sept16)
MobileAppTesting-Fragmentation
Howtoassessfragmentation?
• Definetargetgroupstonarrowyourscope
• Choosebetweensimulators(emulators)andreal
devices(rentaldevices)
• Assesstherisksyouarefacing
• Testsmart
MobileAppTesting-Fragmentation
Testapproach(manual)
• Integration/regressionteston1realAndroidandiOSdevice,
completetestset
• 3-4differentAndroid/iOSdeviceswithreducedtestsetbased
onassessment
• Timeleftover,testwithotherdevices
Startautomationassoonaspossiblewithnativeautomation
tools(EspressoforAndroid,XCTestforiOS)
MobileAppTesting-Featuresonthedevice
• Network
• Camera
• Battery
• Touchscreen
• Voicerecording
• Location
MobileAppTesting-Native,Hybrid,WebNative
• Installationtest
• InterfacewithAPI
• Touch
• DifferentOS
• Interactionwithdevice
attributes,likecamera
MobileAppTesting-Native,Hybrid,WebWeb
• Reachedbynavigatingtoawebsiteby
wayofanURL
• Allcodeexecutedfromaserver,
resultinginnetwork-dependent
performance
• Havetotestdifferentmobilebrowsers.
• Responsivewebdesign,responds
automaticallytowhicheversizedevice
auserison
MobileAppTesting-Native,Hybrid,WebHybrid
• BestfromtheNativeworld
andtheWebworld.
• Accesstodevice-specific
features
MobileAppTesting-Question
Whichoneofthesefourexpressionsdoesnotbelonghere?
• Usability
• Mobile
• Performance
• Security
Mobileismadeupasaspecialismbythecommercialpeople
Guesswhat?Youcanbecomea“CertifiedMobileTester”(via
ASTQB)
MobileAppTesting-FeaturesoftheApp
Makesuretheuserhasanawesomeexperience
Thatmeans,thefunctionalityoftheappisinorder,
thesecuritymeasuresareinplace,theperformance
oftheappiswithinacceptedlimitsandthe
usabilityoftheappisuserfriendly.
These4qualityattributesmustmeettheexpected
requirements
SkillsforaMobileApptester
Amobiletestermustpossess
certainskillswhowillmakehim/
hervaluable.
Improveyourskillsbylearning
fromotherApps.
Watchhowotherpeopleuse
theirApps
• Communication
• Curiosity
• Criticalthinking
• Tenacity
• ConstantLearner
• Creativity
• Customerfocus
• Programmingandtechnical
skills
SkillsforaMobileApptester-Communication
• Listencarefully.
• Don’tinterruptotherpeoplewhile
they’respeaking.
• Speakclearlyandprecisely.
• Don’tgetpersonalwhen
communicatingwithotherpeople.
• Beabletocommunicateondifferent
levels,rangingfromtechnicaltonon-
technicalaudiences.
• Improveyourvocabularybyreading
books,blogs,andnewspapers.
DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-Curiosity
Acurioussoftwaretester
exploresthesoftwaretogetas
muchinformationaspossibleout
ofthesystemtoidentify
potentialproblemsandraise
interestingquestionsaboutthe
product
Source:DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-CriticalThinking
• “Criticalthinkingisthinking
aboutthinkingwiththeaim
ofnotgettingfooled.”–
MichaelBolton
• Gainadeepunderstanding
oftheproduct
• Focusontherighttesting
work
DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-Tenacity
Softwaretestersmustbetenacious
andexplainorraisebugsoverand
overagainuntilthesoftwarequality
standardshavebeenmet.Butbe
carefulwiththisasyoumayendup
beingconsideredanuisance.Here
it’simportanttorelyonyourstrong
communicationskills.
DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-ConstantLearner
• Softwaretestersneedtotake
noteofchangestakingplace
aroundtheminordertoadapt
andlearnnewapproaches,
methodsandtechnologies.
• Tokeeppaceandlearnnew
techniquesandtools,software
testerscanreadblogsorbooks
andattendconferencesand
trainingcourses.
DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-Creativity
• Thecreativityprocessstarts
withgeneratingthetest
ideaandcollectingthetest
data.
• Becreativeinordertouse
mobiledevicesindifferent
waysbypayingattentionto
alltheinterfaces,sensors,
andlocations.
DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-CustomerFocus
• Trytothinklikeacustomerinorder
todeterminewhetherthesoftware
beingtestedmeetscustomerneeds
• Requiresyoutobeaproductand
fieldexpertwithinyourteam
• Talktocustomerstodeterminetheir
needsandproblems
• Usetheknowledgetoeveryphaseof
thesoftwaredevelopmentprocess
DanielKnott-Hands-OnMobileAppTesting
SkillsforaMobileApptester-Programmingskills
• Mobiletesterswithno
programmingskillsneedtotrain
themselvesby– readingabookaboutprogramming
languagesorpatterns
– followingaprogrammingtutorialonthe
Internet
– attendingaprogrammingcourse.
• It’salsopossibletoaska
developertotrainthemobile
testerwithinaprojector
company.
DanielKnott-Hands-OnMobileAppTesting
MobileAppTestCoverageModel
• Heuristic:LONGFUNCUP
• DhanasekarSubramaniam
• Testcoveragemodelfocussedon
testingofMobileApp
• Location
• Orientation
• Network
• Gesture
• Function
• UserScenarios
• Notification
• Communication
• Updates
• Platform
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Location
• Appshouldbesmartenoughto
trackyourcurrentlocationand
providedsuggestions
surroundingyourlocality
• Appasksforuserpermission
beforetrackingthelocation
• Optionstomockyourlocations
fortestingtheapp
• DonottestanAppsittingatyour
desk
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Orientation
• Changetheorientation,app
shouldbeabletoprovidea
consistentuserexperience
• Testallyourscreens,pop-ups,
toastmessages,formsinall
supportedorientations
• Instancewherethefilledinform
datadisappearswhenyou
changetheorientation
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Network
• Mobiledevicessupportsboth
cellularandwi-fi.
• Theycanautomaticallyswitch
betweenanyavailablenetworks.
• Howdoestheappbehavewhen
thedeviceswitchesbetween
networks?
• Testforsignalstrength,network
availabilityandreliability
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Gesture
• Doesyourappsupportsall
standardgestures?
• Isitconsistentacrosstheapp?
• Ifitusesanynewgestures,isit
easyfortheuserstounderstand?
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Function
• Anyfunctionthatdefinesor
distinguishestheproductor
fulfillscorerequirements.
• Testforinteractions,error
handling,startingandclosingof
theapp,fileaccess,navigation,
multimediaandsync.
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP–UserScenarios
• Anyscenarioshouldhaveastory
thatiscredible,motivating,
complexandeasytoevaluate
• Trytothinkaboutdisfavored
usersandhowtheymighttryto
exploittheapp.
• Comparewithcompetitororweb
interfacetogetmorereallife
scenarios
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Notifications
• Notificationsenablean
applicationtoinformits
usersthatithassomething
forthem.
• Testforallavailabletypes
ofvisualnotifications,
soundandvibration.
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Communication
• Theprimarypurposeforusing
mobilephonesisvoicecalls.How
doestheappbehaveafter
interruptionsbyanincoming
call?
• Whatabouttextmessage,Skype
videocallorFaceTime?
• Testforsuchinterruptionsfrom
voicecommunicationandall
necessaryfunctionsofyourapp.
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Updates
• Mobileappgetsfrequent
updates.Notonlyappupdates
butOSupdatesaswell.
• Whatchanges,whichnew
featuresareinthelatestOS
update
• Analyseiftheappneedsany
modificationbecauseoftheOS
updates.
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
LONGFUNCUP-Platform
• Haveagoodunderstandingof
thepopularmobileplatforms
especiallyAppleandAndroid
• Beawareofthehistoryandalso
thelatesttrendsinthemobile
platform.
• Whattoolsareavailablefor
testing
• Whatkindoftest-abilitylayers
areprovidedbytheplatform
https://testingideas.wordpress.com/2014/08/17/mobile-app-test-coverage-model-long-fun-cup/
Summertimebug
ViathebrowserIenteredmybirthday,onthemobile
appIdisplayedmybirthday.Itwasdisplayedwrong
https://simonsaysnomore.wordpress.com/2016/03/08/the-summertime-bug/
Nextyear
Usefulinformation
• http://katrinatester.blogspot.co.nz/2015/08/mobile-testing-pathway.html
• http://adventuresinqa.com/2016/05/03/10-mobile-app-testing-mistakes-to-avoid/
• http://adventuresinqa.com/2015/06/22/top-10-best-practices-for-mobile-app-
testing/
MobileApptestingisnotsomethingspecial
• Mobiledeviceisachannel.Theusergainsaccessvia
anApptothebackendwherealltheinformationis
stored.
• MobileApptestingisbasedonfourquality
attributes.Functionalityisthemostimportantone.
• Whenyouasatesteralreadyhavesometest
experience,youcandoMobileApptesting
I’maTester
• 50yearsold
• Testersince1997
• TestMobileAppssinceNovember2014
• GottheMobileAppstestingjobviamynetwork
• Basedonmyreputationasatester
• Myjobinterview(therewasnone)
• LearntotestMobileAppsjustbymyself
SimonSaysThankYouDoyouhaveanyquestions?
@simonsaysnomore
simonsaysnomore.wordpress.com