The fine art of mobile testing

  • View

  • Download

Embed Size (px)


Why most companies are behind the curve on mobile testing and how you can get ahead


  • 1.The Fine Art of Mobile TestingWhy most companies are behind the curve on mobile testing andhow they can move into the leadA Point of View from the Capgemini Group

2. Business Critical Mobilitywere the norm. There was little or no IT-impact and fewcompanies had a structured approach to quality assuranceMobile has become a game-changer across virtually alland testing online Organizations around the world now drivesignificant value through providing continuous access to Support core business The Web grew in importanceand began supporting companies and organizations coreservices, anytime and anywhere - to millions of customersbusiness processes. The solutions at first supported only aand thousands of employees over an astonishing arraylimited dynamic user interaction on both internal intranetof devices.sites and external sites. Sales and marketing teams wereCompanies, public bodies, and other organizations report that the primary business drivers, with IT support oftenmobile entities, such as apps and responsive web sites, consisting of inefficient and siloed system integration(those that optimize viewing and navigation across a wide solutions. Quality assurance happened occasionally, if at all,range of devices) account for their fastest growth in terms ofand without a clear structure. Lack of experience, methodsnumber of interactions and transactions. Many media and tools were issues that kept quality on a typical it workscompanies report that their services are now accessed moreif were lucky level.frequently through mobile devices than through PCs. Similarly, Drive new business Today, web solutions are not onlytravel companies report that ticket sales through apps andbusiness critical but core enablers of new businesstransactional-based websites are on an astonishingly rapidopportunities. Online experiences are engaging andgrowth curve with sales through mobile apps accounting fortransactional in nature. Community and social mediaan increasing proportion of their revenues every month. elements are now included as integral, natural parts ofIn many respects, mobile solutions are following in the solutions. Most successful companies in all sectors havefootsteps of the Web. It may be helpful for our current already integrated online into their core products andunderstanding of mobile solutions to compare how the twoservices. Through the emerging Responsive Web Designfields have developed.(RWD) trend, the Web is quickly becoming mobile first,In the early days of the Web, companies were experimentingwhich means that new online services created now areand discovering. Quickly, static home pages became highly designed primarily for mobile users first. The entiredynamic platforms for rich media and engaging experiences.organization is engaged in the planning and design of webThe Web brought both core business processes and newsolutions now, including Product Development, Service andbusiness models online. Mobile solutions are taking a similar HR. IT departments have learned to support new solutionspath: after a brief period of experimentation in a new business using highly scalable, cloud-enabled and service-orienteddevelopment channel they are becoming increasinglyarchitectures. Quality assurance measures have maturedbusiness critical.and are supported by a wealth of expertise, methods,experience and tooling. Common quality assuranceLets first take a look at how different aspects of the Web havechallenges still include addressing distributed solutions,developed over time (Figure 1)usability, different types of browsers and security.As you can see, there are three phases through which websolution aspects have passed:The same phases and aspects are as relevant for mobile solutions as for web solutions (Figure 2): Discovery This is when the market is experimenting inand discovering the channel. Static home pages with Discovery For the past few years, the market has beenaddresses to physical office locations, simple marketing discovering and experimenting in the mobile channel.brochures and perhaps an e-mail address for contacts Simple and narrowly scoped apps for branding purposes were initiated most often by marketing departments. The Figure 1: Phase Progression of Web Solution AspectsPhasesDiscovery Support core business Drive new businessFunctionalStatic home pages Limited dynamic support for Engaging online experiences, internal andScope user interactionexternal, highly transactional, communitybuilding, social media integrated, mobilesupported through responsive web designOrganizationMarketing Sales and Marketing Sales, Marketing, Product Development, Solution Service, HR and IT AspectsIT impact N/A Limited siloed system Service-oriented architecture, Cloud-integration supported scalability, mature securityimplementationsQuality N/A Occasional and unstructured, Integrated, structured and methodological,Assurance no toolsstrong toolingQuality N/A Lack of experience, methods Complexities of componentized andAssurance and tools. Labor intensivedistributed solutions, usability, browsers, andIssuesmanual testingsecurity2 The Fine Art of Mobile Testing 3. Mobile Testing the way we see it Figure 2: Phase Progression of Mobile Solution AspectsPhasesDiscovery Support core business Drive new business Functional Disconnected apps Limited dynamic support for Engaging mobile experiences, internal and Scopeuser interactionexternal, highly transactional, communitybuilding, social media integrated, optimized forreach and platform Organization Marketing Sales and Marketing Sales, Marketing, Product Development, Solution Service, HR and IT Aspects IT impactN/A Limited dynamic support for Service-oriented architecture, cloud-user interactionsupported scalability, mature securityimplementations QualityN/A Occasional and unstructured, Integrated, structured and methodological, Assuranceno toolsstrong tooling QualityN/A Lack of experience, methods Complexities of componentized and Assuranceand tools. Labor intensivedistributed solutions, usability, platforms, Issues manual testingscreen sizes, and security impact on IT was minor and few companies applied a require particular attention. They are: structured approach to mobile testing. Platform Fragmentation Support core business Just as with web solutions, The Mobile Devices Physical Characteristics mobile solutions has grown in importance and now User Experience supports companies and organizations core business Performance processes. The simple, disconnected early apps evolved Security into transactional apps, which enable the user to sign in, System Integration manage accounts, and buy (and even sell) products and Managing App Distribution. services. Internal apps for field service, sales support and decision support through various business intelligence Platform Fragmentation solutions were and are being built to accelerate internalThe most pressing and obvious mobile testing challenge is processes and reduce costs. While inefficient, siloedensuring quality across the ever-increasing range of system integration solutions are still common, more oftendevice types. mobile solutions are being designed to reuse existingArguably, there are at least three relevant smartphone and investments in service-oriented architectures that were firsttablet operating system platforms which mobile solutions built to support web solutions. The latest researchneed to support: Googles Android (with the largest market published in the World Quality Report, 2012-20131 indicatesshare), Apples iOS (second largest and with the user base that quality assurance and mobile testing only happenwhich is the most active in downloading, buying and using occasionally and without a clear structure. We find that the apps), and Microsofts Windows 8/Windows Phone. primary reasons are lack of experience, lack of devices,Each platform has its own set of characteristics, from user and lack of tools.experience and user interface to technology frameworks, app Drive new business As mobile solutions mature they are distribution, and execution models: increasingly core enablers of new business opportunities. The most successful apps and responsive webs are Google Android is a popular platform among many engaging and highly transactional in nature. Just as with the smartphone makers, which makes it an especially Web, community and social media integration are natural fragmented platform. Android has several major versions solution elements. Some companies have developed acorresponding to even more Application Programming business-driven mobile strategy that involves all parts of theInterface (API) sets. There can be hundreds of different organization. Mobile testing is on its way to maturity as Android devices in an installed base at any given time testing experts learn new methods and are enabled withand location. more sophisticated devices and tools. Apple iOS runs in iPhones, iPod Touchs, iPads and iPad Minis. There is a strong platform consistency across iOSMobile Testing Challengesdevices, but there are significant differences in deviceJust as mobility in general has followed in the footsteps of the sizes, screen resolutions, camera resolutions, and so on.Web, mobile testing is following along the same path as webThe iOS user base is typically quick to adopt new versionstesting. The two share many drivers but there are some areas of the operating system, which means no more than threeof mobile testing that are distinctly more challenging and versions of the operating system, including the latest, will likely need to be supported to address most target markets.1 World Quality Report 2012-13 contains analysis and commentary based on data collected from 1,553 in-depth interviews with organizations in 25 countries. The report can be downloaded in full at http://www.capgemini. com/wqr2012 3 4. Microsoft Windows 8 and Windows Phone areFigure 3: ROI of test automation is earned