View
221
Download
1
Category
Preview:
DESCRIPTION
Are you in control of all app defects - no matter the device or operating environment? If not, we recommend you download our Test Coverage Index. It's the best (and only) resource that will help you make smart decisions and deliver great mobile apps everytime. That's continuous quality.
Citation preview
Mobile Test Coverage Index
THE
DEVICE SELECTOR™
OPTIMIZE
RESET
30%10-16
50%17-25
80%26-32
BU
1. DEVICE COVERAGE
2. CUSTOMER SPECIFIC REQUIREMENTS
3. DEVICE CHARACTERISTICS
4. MIX CATEGORIES
Geography Targets
OS Mix
Reference Devices Market Leaders Legacy Devices New & Emerging
Hardware Mix Form Factors (Screen Size) Launch Date
BU Requirements Tablets/SmartphonesOS Targets Legacy length of OS/DevicesCarriers
Edition 2 August 2015
FEATURING NEW EUROPEAN DATA
UK Germany
2
IntroductionThe latest Mobile Test Coverage Index, released in August 2015, is the second in a series of reports Perfecto Mobile will publish each quarter to help answer the question: On what devices and operating systems should I be testing my apps and websites? As device proliferation continues, mobile app developers, testers and product managers need an effective strategy for ensuring real world performance and consistent quality for any user on any device or platform. To help mitigate risk, Perfecto Mobile has developed the Continuous Quality Coverage Framework™ (CQCF), an answer for generating coverage requirements based on evolving market share data, industry research and Perfecto Mobile’s proprietary MobileCloud™ data. The CQCF test coverage optimizer is designed to convert best-practice methodologies into an indexing algorithm for reaching 30%, 50% or 80% coverage for specific geographic, technical and project requirements. As part of the CQCF, we have created this quarterly Mobile Test Coverage Index to serve as an industry baseline that companies can customize to their own needs. The report aggregates and analyzes a wide range of public, private and proprietary data – 4,000+ device profiles and 360,000+ Perfecto Mobile user profiles – to calculate test coverage recommendations for the U.S. and EU5 markets, which include Spain, France, Italy, Germany and the UK. What’s new in this edition of the index? We highlight device coverage and usage data for Germany and the UK—two European countries where adoption of newer iOS, Android and Windows Phone devices is high. We have also included a calendar of new and upcoming mobile devices and platforms so you can plan your coverage strategy accordingly.
3
What the Index Helps You AchieveWith a rapidly expanding device matrix, achieving full coverage of the mobile device market is neither practical nor necessary. Instead, businesses must choose devices based on priorities, relevance and strategic objectives for a given project. With that in mind, this report offers device selection for three levels of mobile market coverage —30%, 50% and 80%.
Index MethodologyOur device selections were developed based on extensive market analysis across four key sectors:
1. OS coverage: Assures performance across relevant OS versions from Android, iOS, Windows Phone and BlackBerry.
2. Screen sizes and resolutions: Analyzes how manufacturer and device variations can impact app quality.
3. Hardware coverage: Accounts for discrepancies in hardware capabilities such as CPU, memory, system on chip.
4. Device mix: Reduces risk and ensures broad app appeal by covering emerging technology, popular favorites and reference devices.
By analyzing an extensive dataset containing over 360,000 consumer usage profiles across 4,000+ unique device configurations, we identified a recommended list of smartphones and tablets to test and develop against in order to assure sufficient coverage of the U.S. and EU5 markets.
30% of the market can be covered through an indexed list 10-16 devices. In this first level, we have included the most important devices in the U.S. and Europe based on market adoption, market device leaders, reference devices and device characteristics such as screen size, hardware and more. Additionally, we have assigned leading versions of Android, iOS, Windows Phone and Blackberry to these devices.
50% of the market can be covered with 17-25 devices. In addition to those listed in the 30% coverage index, this second level accounts for an increased number of OS versions, carriers and devices from manufacturers such as Sony, LG, Samsung, Apple, HTC and more.
80% market coverage can be achieved with an index of 26-32 devices. At the third level, an organization gains a high level of confidence for coverage in the specific market(s) being targeted. This index helps organizations identify defects in devices heavily used by end-users in both regions.
30% 50% 80%
MT
3
4
Key Takeaways
• Apple continues to dominate the U.S. market, while Android dominates Europe. Apple’s steady success is backed up by industry market share data as well as Perfecto Mobile’s own cloud data that shows a huge preference for iOS testing in the U.S. This success is likely to continue with the releases of iPhone 6S and 6S Plus this fall.
• Both market share data and Perfecto Mobile cloud data reveal that the Windows Phone platform is much more popular in Europe than the U.S., so it should be included in your index if you have a presence in European markets. Windows Phone accounts for 10% market share in the EU5 region, with a breakout of 7% market share in Germany and 13% in the UK. Share in the U.S., however, is less than 3%.
• The leading smartphone screen resolutions across geographies are 1440 x 2560 and 1080 x 1920 and the leading screen sizes are between 5-5.5”, which accounts for almost 80% of the market.
• It is important to include Sony devices in the index when addressing the EU5 region as the Sony brand has 8% market share. In the U.S., however, Sony holds a share of less than 1%.
• While adoption of Google Nexus devices is slow, they are the first devices to run the latest version of Android. From a test standpoint, this early access makes Nexus an essential part of every index. In contrast, it takes up to six to eight months to get access to the latest Android version on non-Nexus devices.
• While iPhone 6 and iPhone 5S are the two most popular devices in the U.S. and the UK, Samsung Galaxy S5 tops the list in Germany.
• Apple and Samsung are neck-and-neck in the UK with similarly strong market share (28% and 30% respectively).
What to Keep an Eye on in Q3
• iOS 9 and iPhone 6S and 6S Plus will be available this fall. The iOS 9 beta is available now for developers and we recommend to start testing on it right away.
• The next major Android release (Android M) is also releasing this fall. The Android M beta is now available, and we recommend testing your mobile apps on it using relevant Nexus devices.
• Chinese device maker Huawei recently released a new device (Huawei Ascend P8) and is growing quickly. We believe Huawei devices will soon be widely adopted and recommend testing on them as soon as possible.
7%13%
3%30%
Windows Phone devices in GermanyWindows Phone devices in UK
Apple & Samsung share the same strong market share in the UK
Chinese manufacturer Huawei is starting to gain market share (3%) in Europe, mainly in Germany
UK Germany
5
30-80% Coverage Device Name Release Date Screen Size Screen Resolution System On Chip
1 iPhone 6 Sep-2014 4.7’’ 750 x 1334 Apple A8
2 iPhone 5S Sep-2013 4.0’’ 640 x 1136 Apple A7
3 Samsung Galaxy S6 Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
4 Samsung Galaxy S5 Feb-2014 5.1’’ 1080 x 1920 Qualcomm Snapdragon 801
5 iPhone 6 Plus Sep-2014 5.5’’ 1080 x 1920 Apple A8
6 Samsung Galaxy S6 Edge Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
7 Samsung Galaxy Note 4 Sep-2014 5.7’’ 1440 x 2560 Qualcomm Snapdragon 805
8 Apple iPad Mini 3 Oct-2014 7.9’’ 1536 x 2048 Apple A7
9 Apple iPad Air 2 Oct-2014 9.7’’ 1536 x 2048 Apple A8X
10 HTC One M8 Mar-2014 5.0’’ 1080 x 1920 Qualcomm Snapdragon 801
11 Motorola Google Nexus 6 Oct-2014 6.0’’ 1440 x 2560 Qualcomm Snapdragon 805
12 HTC One M9 Mar-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 810
13 LG G4 Apr-2015 5.5’’ 1440 x 2560 Qualcomm Snapdragon 808
14 LG G3 May-2014 5.5’’ 1440 x 2560 Qualcomm Snapdragon 801
15 Samsung Galaxy Tab 4 – 8’’ Apr-2014 8.0’’ 800x 1280 Qualcomm Snapdragon 400
16 LG G Pad10.1 LTE July-2014 10.1’’ 1280 x 800 Qualcomm Snapdragon 400
17 iPhone 5 Sep-2012 4.0’’ 640 x 1136 Apple A6
18 Google (LG) Nexus 5 Oct-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
19 Motorola Moto G (2nd Gen) Sep-2014 5.0’’ 720 x 1280 Qualcomm Snapdragon 400
20 Samsung Galaxy Note Pro 12.2 Jan-2014 12.2’’ 2560 x 1600 Exynos 5 Octa 5420
21 Samsung Galaxy Tab 4 -10.1 Apr-2014 10.1’’ 1280 x 1800 Qualcomm Snapdragon 400
22 Google (HTC) Nexus 9 Oct-2014 8.9’’ 1536 x 2048 Nvidia Tegra K1
23 Apple iPad Air Oct-2013 9.7’’ 1536 x 2048 Apple A7
24 Sony Xperia Z3v Oct-2014 5.2’’ 1080 x 1920 Qualcomm Snapdragon 801
25 Amazon Kindle Fire HDX 8.9 Sep-2013 8.9’’ 2560 x 1600 Qualcomm Snapdragon 800
26 Samsung Galaxy S4 Mar-2013 5.0’’ 1080x 1920 Exynus 5 Octa 5410
27 Motorola Moto X (2nd Gen) Sep-2014 5.2’’ 1080 x 1920 Qualcomm Snapdragon 801
28 LG G Pad 8’’ LTE Aug-2014 8.0’’ 800 x 1280 Qualcomm Snapdragon 400
29 iPhone 4S Oct-2011 3.5’’ 640 x 960 Apple A5
30 HTC Desire 510 Aug-2014 4.7’’ 480 x 854 Qualcomm Snapdragon 400
31 Motorola Droid MAXX July-2013 5.0’’ 720 x 1280 Qualcomm Snapdragon S4 Pro
32 HTC One X Feb-2012 4.7’’ 720 x 1280 Nvidia Tegra 3
*Rankings based on Perfecto Mobile cloud usage and market data
30%
50%
80%
Test Coverage – U.S.
6
Test Coverage – EU5
30-80% Coverage Device Name Release Date Screen Size Screen Resolution System On Chip
1 iPhone 6 Sep-2014 4.7’’ 750 x 1334 Apple A8
2 iPhone 5S Sep-2013 4.0’’ 640 x 1136 Apple A7
3 Samsung Galaxy S6 Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
4 Samsung Galaxy S6 Edge Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
5 iPhone 6 Plus Sep-2014 5.5’’ 1080 x 1920 Apple A8
6 Samsung Galaxy S5 Feb-2014 5.1’’ 1080 x 1920 Qualcomm Snapdragon 801
7 Samsung Galaxy Note 4 Sep-2014 5.7’’ 1440 x 2560 Qualcomm Snapdragon 805
8 Apple iPad Mini 3 Oct-2014 7.9’’ 1536 x 2048 Apple A7
9 Apple iPad Air 2 Oct-2014 9.7’’ 1536 x 2048 Apple A8X
10 HTC One M8 Mar-2014 5.0’’ 1080 x 1920 Qualcomm Snapdragon 801
11 Motorola Google Nexus 6 Oct-2014 6.0’’ 1440 x 2560 Qualcomm Snapdragon 805
12 Samsung Galaxy S4 Mar-2013 5.0’’ 1080x 1920 Exynus 5 Octa 5410
13 LG G4 Apr-2015 5.5’’ 1440 x 2560 Qualcomm Snapdragon 808
14 LG G3 May-2014 5.5’’ 1440 x 2560 Qualcomm Snapdragon 801
15 iPhone 4S Oct-2011 3.5’’ 640 x 960 Apple A5
16 Huawei Ascend P7 May-2014 5.0’’ 1080 x 1920 Hisilicon Kirin 910T
17 Huawei Ascend Y300 Mar-2013 4.0’’ 480 x 800 Qualcomm Snapdragon S4 Play
18 Google (LG) Nexus 5 Oct-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
19 HTC One M9 Mar-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 810
20 Huawei Ascend Mate 2 – 4G Jan-2014 6.1’’ 720 x 1280 Qualcomm Snapdragon 400
21 Sony Xperia Z3 Oct-2014 5.2’’ 1080 x 1920 Qualcomm Snapdragon 801
22 Google (HTC) Nexus 9 Oct-2014 8.9’’ 1536 x 2048 Nvidia Tegra K1
23 Samsung Galaxy Tab 4 8’’ Apr-2014 8.0’’ 720 x 1280 Qualcomm Snapdragon 400
24 Sony Xperia Z1 Sep-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
25 Samsung Galaxy S3 May-2012 4.8’’ 720 x 1280 Exynos 4412 Quad
26 Motorola Moto G (2nd Gen) Sep-2014 5.0’’ 720 x 1280 Qualcomm Snapdragon 400
27 Apple iPad Air Oct-2013 9.7’’ 1536 x 2048 Apple A7
28 Sony Xperia Z3 Compact Sep-2014 4.6’’ 720 x 1280 Qualcomm Snapdragon 801
29 iPhone 5 Sep-2012 4.0’’ 640 x 1136 Apple A6
30 Huawei Ascend P8 Apr-2015 4.6’’ 720 x 1280 HiSilicon Kirin 930/935
31 Samsung Galaxy A7 Feb-2015 5.5’’ 1080 x 1920 Exynos 5 Octa 5430
32 Samsung Galaxy Tab S 10.5’ Jun -2014 10.5’’ 2560 x 1600 Exynos 5 Octa 5420
30%
50%
80%
*Rankings based on Perfecto Mobile cloud usage and market data
7
30-80% Coverage Device Name Release Date Screen Size Screen Resolution System On Chip
1 iPhone 6 Sep-2014 4.7’’ 750 x 1334 Apple A8
2 iPhone 5S Sep-2013 4.0’’ 640 x 1136 Apple A7
3 Samsung Galaxy S6 Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
4 Samsung Galaxy S6 Edge Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
5 iPhone 6 Plus Sep-2014 5.5’’ 1080 x 1920 Apple A8
6 Samsung Galaxy S5 Feb-2014 5.1’’ 1080 x 1920 Qualcomm Snapdragon 801
7 Samsung Galaxy Note 4 Sep-2014 5.7’’ 1440 x 2560 Qualcomm Snapdragon 805
8 Apple iPad Mini 3 Oct-2014 7.9’’ 1536 x 2048 Apple A7
9 Apple iPad Air 2 Oct-2014 9.7’’ 1536 x 2048 Apple A8X
10 HTC One M8 Mar-2014 5.0’’ 1080 x 1920 Qualcomm Snapdragon 801
11 Motorola Google Nexus 6 Oct-2014 6.0’’ 1440 x 2560 Qualcomm Snapdragon 805
12 Samsung Galaxy S4 Mar-2013 5.0’’ 1080x 1920 Exynus 5 Octa 5410
13 LG G4 Apr-2015 5.5’’ 1440 x 2560 Qualcomm Snapdragon 808
14 LG G3 May-2014 5.5’’ 1440 x 2560 Qualcomm Snapdragon 801
15 iPhone 4S Oct-2011 3.5’’ 640 x 960 Apple A5
16 Huawei Ascend P7 May-2014 5.0’’ 1080 x 1920 Hisilicon Kirin 910T
17 Huawei Ascend Y300 Mar-2013 4.0’’ 480 x 800 Qualcomm Snapdragon S4 Play
18 Google (LG) Nexus 5 Oct-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
19 HTC One M9 Mar-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 810
20 Huawei Ascend Mate 2 – 4G Jan-2014 6.1’’ 720 x 1280 Qualcomm Snapdragon 400
21 Sony Xperia Z3 Oct-2014 5.2’’ 1080 x 1920 Qualcomm Snapdragon 801
22 Google (HTC) Nexus 9 Oct-2014 8.9’’ 1536 x 2048 Nvidia Tegra K1
23 Samsung Galaxy Tab 4 8’’ Apr-2014 8.0’’ 720 x 1280 Qualcomm Snapdragon 400
24 Sony Xperia Z1 Sep-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
25 Samsung Galaxy S3 May-2012 4.8’’ 720 x 1280 Exynos 4412 Quad
26 Motorola Moto G (2nd Gen) Sep-2014 5.0’’ 720 x 1280 Qualcomm Snapdragon 400
27 Apple iPad Air Oct-2013 9.7’’ 1536 x 2048 Apple A7
28 Sony Xperia Z3 Compact Sep-2014 4.6’’ 720 x 1280 Qualcomm Snapdragon 801
29 iPhone 5 Sep-2012 4.0’’ 640 x 1136 Apple A6
30 Huawei Ascend P8 Apr-2015 4.6’’ 720 x 1280 HiSilicon Kirin 930/935
31 Samsung Galaxy A7 Feb-2015 5.5’’ 1080 x 1920 Exynos 5 Octa 5430
32 Samsung Galaxy Tab S 10.5’ Jun -2014 10.5’’ 2560 x 1600 Exynos 5 Octa 5420
Test Coverage – Germany
30-80% Coverage Device Name Release Date Screen Size Screen Resolution System On Chip
1 Samsung Galaxy S5 Feb-2014 5.1’’ 1080 x 1920 Qualcomm Snapdragon 801
2 iPhone 5S Sep-2013 4.0’’ 640 x 1136 Apple A7
3 Samsung Galaxy S6 Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
4 iPhone 6 Sep-2014 4.7’’ 750 x 1334 Apple A8
5 iPhone 6 Plus Sep-2014 5.5’’ 1080 x 1920 Apple A8
6 Samsung Galaxy S3 May-2012 4.8’’ 720 x 1280 Exynos 4412 Quad
7 Samsung Galaxy S4 Mar-2013 5.0’’ 1080x 1920 Exynus 5 Octa 5410
8 Apple iPad Mini 3 Oct-2014 7.9’’ 1536 x 2048 Apple A7
9 Apple iPad Air 2 Oct-2014 9.7’’ 1536 x 2048 Apple A8X
10 Nokia Lumia 520 Apr-2013 4.0’’ 480 x 800 Qualcomm MSM8227
11 Sony Xperia Z1 Sep-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
12 HTC One M8 Mar-2014 5.0’’ 1080 x 1920 Qualcomm Snapdragon 801
13 LG G4 Apr-2015 5.5’’ 1440 x 2560 Qualcomm Snapdragon 808
14 Google (LG) Nexus 5 Oct-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
15 Samsung Galaxy Tab 4 – 8’’ Apr-2014 8.0’’ 800x 1280 Qualcomm Snapdragon 400
16 Sony Xperia Z3 Compact Sep-2014 4.6’’ 720 x 1280 Qualcomm Snapdragon 801
17 iPhone 5 Sep-2012 4.0’’ 640 x 1136 Apple A6
18 Samsung Galaxy Tab 10.5 S Jun-2014 10.5’’ 2560 x 1600 Exynos 5 Octa 5420
19 Samsung Galaxy S6 Edge Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
20 LG G3 May-2014 5.5’’ 1440 x 2560 Qualcomm Snapdragon 801
21 Samsung Galaxy Tab 4 -10.1 Apr-2014 10.1’’ 1280 x 1800 Qualcomm Snapdragon 400
22 Google (HTC) Nexus 9 Oct-2014 8.9’’ 1536 x 2048 Nvidia Tegra K1
23 HTC One M9 Mar-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 810
24 Sony Xperia Z Jan-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon S4 Pro
25 Motorola Google Nexus 6 Oct-2014 6.0’’ 1440 x 2560 Qualcomm Snapdragon 805
26 Huawei Ascend Y300 Mar-2013 4.0’’ 480 x 800 Qualcomm Snapdragon S4 Play
27 Motorola Moto G (2nd Gen) Sep-2014 5.0’’ 720 x 1280 Qualcomm Snapdragon 400
28 HTC One M7 Feb-2013 4.7’’ 1080 x 1920 Qualcomm Snapdragon 600
29 iPhone 4S Oct-2011 3.5’’ 640 x 960 Apple A5
30 HTC One Mini July-2013 4.3’’ 720 x 1280 Qualcomm Snapdragon 400
31 LG G Pad 8 May-2014 8.0’’ 800 x 1280 Qualcomm Snapdragon 400
32 Sony Xperia Tablet Z2 Feb-2014 10.1’’ 1200 x 1920 Qualcomm Snapdragon 801
30%
50%
80%
*Rankings based on Perfecto Mobile cloud usage and market data
8
Test Coverage – UK
30-80% Coverage Device Name Release Date Screen Size Screen Resolution System On Chip
1 iPhone 6 Sep-2014 4.7’’ 750 x 1334 Apple A8
2 iPhone 5S Sep-2013 4.0’’ 640 x 1136 Apple A7
3 Samsung Galaxy S6 Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
4 Samsung Galaxy S5 Feb-2014 5.1’’ 1080 x 1920 Qualcomm Snapdragon 801
5 iPhone 6 Plus Sep-2014 5.5’’ 1080 x 1920 Apple A8
6 Sony Xperia Z3 Oct-2014 5.2’’ 1080 x 1920 Qualcomm Snapdragon 810
7 Samsung Galaxy S4 Mar-2013 5.0’’ 1080x 1920 Exynus 5 Octa 5410
8 Apple iPad Mini 3 Oct-2014 7.9’’ 1536 x 2048 Apple A7
9 Apple iPad Air 2 Oct-2014 9.7’’ 1536 x 2048 Apple A8X
10 HTC One M9 Mar-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 810
11 Nokia Lumia 520 Apr-2013 4.0’’ 480 x 800 Qualcomm MSM8227
12 Samsung Galaxy S3 May-2012 4.8’’ 720 x 1280 Exynos 4412 Quad
13 LG G4 Apr-2015 5.5’’ 1440 x 2560 Qualcomm Snapdragon 808
14 Google (HTC) Nexus 9 Oct-2014 8.9’’ 1536 x 2048 Nvidia Tegra K1
15 Samsung Galaxy Tab 4 – 8’’ Apr-2014 8.0’’ 800x 1280 Qualcomm Snapdragon 400
16 Motorola Google Nexus 6 Oct-2014 6.0’’ 1440 x 2560 Qualcomm Snapdragon 805
17 iPhone 5 Sep-2012 4.0’’ 640 x 1136 Apple A6
18 Samsung Galaxy Tab 10.5 S Jun-2014 10.5’’ 2560 x 1600 Exynos 5 Octa 5420
19 Samsung Galaxy S6 Edge Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
20 LG G3 May-2014 5.5’’ 1440 x 2560 Qualcomm Snapdragon 801
21 Nokia Lumia 635 Jun-2014 4.5’’ 480 x 854 Qualcomm Snapdragon 400
22 Google (LG) Nexus 5 Oct-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
23 Samsung Galaxy Note 4 Sep-2014 5.7’’ 1440 x 2560 Qualcomm Snapdragon 805
24 Sony Xperia Z Jan-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon S4 Pro
25 HTC One M8 Mar-2014 5.0’’ 1080 x 1920 Qualcomm Snapdragon 801
26 Samsung Galaxy Tab 4 – 7 Apr-2014 7.0’’ 800 x 1280 Marvell PXA1088
27 Sony Xperia Z2 Tablet Feb-2014 10.1’’ 1200 x 1920 Qualcomm Snapdragon 801
28 HTC One M7 Feb-2013 4.7’’ 1080 x 1920 Qualcomm Snapdragon 600
29 iPhone 4S Oct-2011 3.5’’ 640 x 960 Apple A5
30 Motorola Moto G (2nd Gen) Sep-2014 5.0’’ 720 x 1280 Qualcomm Snapdragon 400
31 Sony Xperia M2 Feb-2014 4.8’’ 540 x 960 Qualcomm Snapdragon 400
32 LG G2 Aug-2013 5.2’’ 1080 x 1920 Qualcomm Snapdragon 800
30%
50%
80%
*Rankings based on Perfecto Mobile cloud usage and market data
9
30-80% Coverage Device Name Release Date Screen Size Screen Resolution System On Chip
1 iPhone 6 Sep-2014 4.7’’ 750 x 1334 Apple A8
2 iPhone 5S Sep-2013 4.0’’ 640 x 1136 Apple A7
3 Samsung Galaxy S6 Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
4 Samsung Galaxy S5 Feb-2014 5.1’’ 1080 x 1920 Qualcomm Snapdragon 801
5 iPhone 6 Plus Sep-2014 5.5’’ 1080 x 1920 Apple A8
6 Sony Xperia Z3 Oct-2014 5.2’’ 1080 x 1920 Qualcomm Snapdragon 810
7 Samsung Galaxy S4 Mar-2013 5.0’’ 1080x 1920 Exynus 5 Octa 5410
8 Apple iPad Mini 3 Oct-2014 7.9’’ 1536 x 2048 Apple A7
9 Apple iPad Air 2 Oct-2014 9.7’’ 1536 x 2048 Apple A8X
10 HTC One M9 Mar-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 810
11 Nokia Lumia 520 Apr-2013 4.0’’ 480 x 800 Qualcomm MSM8227
12 Samsung Galaxy S3 May-2012 4.8’’ 720 x 1280 Exynos 4412 Quad
13 LG G4 Apr-2015 5.5’’ 1440 x 2560 Qualcomm Snapdragon 808
14 Google (HTC) Nexus 9 Oct-2014 8.9’’ 1536 x 2048 Nvidia Tegra K1
15 Samsung Galaxy Tab 4 – 8’’ Apr-2014 8.0’’ 800x 1280 Qualcomm Snapdragon 400
16 Motorola Google Nexus 6 Oct-2014 6.0’’ 1440 x 2560 Qualcomm Snapdragon 805
17 iPhone 5 Sep-2012 4.0’’ 640 x 1136 Apple A6
18 Samsung Galaxy Tab 10.5 S Jun-2014 10.5’’ 2560 x 1600 Exynos 5 Octa 5420
19 Samsung Galaxy S6 Edge Mar-2015 5.1’’ 1440 x 2560 Exynos 7420
20 LG G3 May-2014 5.5’’ 1440 x 2560 Qualcomm Snapdragon 801
21 Nokia Lumia 635 Jun-2014 4.5’’ 480 x 854 Qualcomm Snapdragon 400
22 Google (LG) Nexus 5 Oct-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon 800
23 Samsung Galaxy Note 4 Sep-2014 5.7’’ 1440 x 2560 Qualcomm Snapdragon 805
24 Sony Xperia Z Jan-2013 5.0’’ 1080 x 1920 Qualcomm Snapdragon S4 Pro
25 HTC One M8 Mar-2014 5.0’’ 1080 x 1920 Qualcomm Snapdragon 801
26 Samsung Galaxy Tab 4 – 7 Apr-2014 7.0’’ 800 x 1280 Marvell PXA1088
27 Sony Xperia Z2 Tablet Feb-2014 10.1’’ 1200 x 1920 Qualcomm Snapdragon 801
28 HTC One M7 Feb-2013 4.7’’ 1080 x 1920 Qualcomm Snapdragon 600
29 iPhone 4S Oct-2011 3.5’’ 640 x 960 Apple A5
30 Motorola Moto G (2nd Gen) Sep-2014 5.0’’ 720 x 1280 Qualcomm Snapdragon 400
31 Sony Xperia M2 Feb-2014 4.8’’ 540 x 960 Qualcomm Snapdragon 400
32 LG G2 Aug-2013 5.2’’ 1080 x 1920 Qualcomm Snapdragon 800
Android and iOS – What You Need to KnowAndroid: Android versions KitKat and Jelly Bean are the most widely adopted and should be included in Android test plans. Lollipop, the latest Android version, also remains important because it’s running on exclusive Google Nexus devices. And while Gingerbread is the oldest version, it’s worth doing key regression tests on because it still has 10% market share in Europe. Note that Android M (6.0) beta is available for testing now in the Perfecto Mobile CQ Lab. We highly recommend starting preliminary testing on this platform prior to its general release, scheduled for September.
*Add the listed OS versions in the 30% column, 50% column and 80% column together to achieve the full 80% OS coverage spectrum.
iOS:The iOS family accounts for the majority of the OS market. iOS 8 makes up 84% of this family, but there are earlier iOS 8 versions that should be included in test plans for app functionality, regression and performance. The same goes for iOS 7, which still accounts for 14% of total users. For older versions like iOS 6, we recommend testing only the latest 6.1.6 version for basic sanity testing on legacy iPhone devices. Looking forward, iOS 9 beta is now available to developers in the Perfecto Mobile CQ Lab and should be included in test plan activities. It is scheduled for general release in September.
*Add the listed OS versions in the 30% column, 50% column and 80% column together to achieve the full 80% OS coverage spectrum.
Platform Family 30% 50% 80%
Lollipop 5.1 5.0.2 5.0, 5.0.1
KitKat 4.4.4 4.4.3, 4.3 4.4.2
Jelly Bean 4.2.2 4.1.2 4.1.1
Ice Cream Sandwich 4.0.4 4.0.3 NA
Gingerbread 2.3.7 2.3.6 2.3.3, 2.3.5
Platform Family 30% 50% 80%
iOS 8 8.4 8.3 8.2, 8.1.3
iOS 7 7.1.2, 7.1.1 7.0.4 7.0
iOS 6 NA NA 6.1.6
10
What’s New and On the HorizonTo keep developers aligned with changes in usage trends, Perfecto Mobile refreshes the granular data powering the CQCF analytics on a quarterly basis. Here is a preview of new and upcoming releases, which we will factor into the next report.
New & Emerging Devices
New & Emerging Operating Systems
Device Name Release Date Screen Size Screen Resolution System on a Chip
iPhone 6S/6S Plus Fall-2015 4.7”/5.5” 750x1334/1080x1920
Samsung Galaxy Tab A 9.7’’ May-2015 9.7’’ 768 x 1024 Nvidia Tegra 3
Samsung Galaxy Note 5 Aug-2015 6.0’’ 2560 x 1440 Qualcomm Snapdragon 820
Motorola Moto G (3rd Gen) 2015 Q3-2015 5.0’’ 1080 x 1920 Qualcomm Snapdragon 410
LG G Pro 3 Apr-2015 6.0’’ 2560 x 1440 Qualcomm Snapdragon 820
Sony Xperia C4 May-2015 5.0’’ 1080 x 1920 MediaTek MT6752
Sony Xperia M4 Aqua June- 2015 5.0’’ 720 x 1280 Qualcomm Snapdragon 615
Huawei Ascend P8 Apr-2015 5.2’’ 1080 x 1920 HiSilicon Kirin 930/935
OS Name Version Expected Release Date
iOS 9.0 Beta iOS 9 – September
Android 1) 5.2 2) Android M 6.0 Beta Android M – September
Windows 10 Mobile Insider Preview – Build 10166 Windows 10 Mobile – September
Mob
ile M
arke
t Cal
enda
r 201
5Be
sur
e to
pla
n ah
ead
to g
et m
axim
um te
st c
over
age.
DEV
ICES
APR
ILM
AYJU
NE
JULY
AUG
UST
SEPT
EMBE
RO
CTO
BER
PLAT
FOR
M
iOS
9 Be
ta
iOS
9W
indo
ws
10 fo
r Pho
nes
and
Andr
oid
M
You
can
do b
eta
test
ing
now
in
the
Perfe
cto
Mob
ile C
loud
New
iPho
ne 6
/6 P
lus
& iP
ads
Sam
sung
Not
e 5
Mot
orol
a M
oto
G (3
rd)
Lum
ia 9
50Lu
mia
950
XL
Sony
Xpe
ria Z
4 (Z
3+)
Sony
Xpe
ria
M4
Aqua
Sam
sung
Gal
axy
Tab
A 9.
7’’ T
able
tSo
ny X
peria
Z4 T
able
t
App
le W
atch
Rel
ease
Sam
sung
Gal
axy
S6Sa
msu
ng G
alax
y S6
Edg
e
LG G
4N
okia
Lum
ia 6
40 X
L
ZTE
Blad
e S6
Plu
sH
uaw
ei P
8Le
novo
A60
00 P
lus
Firs
t Sam
sung
dev
ices
toru
n An
droi
d Lo
llipop
off
the
shel
f
ww
w.pe
rfect
omob
ile.c
om/te
stco
vera
ge
12
����� ����� ����� ����� ����� �����
����� ����� ����� ����� ����� ����� �����
39% 3% 17%28% 5%8%
37% 4% 4% 19%18% 10% 8%
����� ����� ����� ����� ����� �����
����� ����� ����� ����� ����� ����� �����
39% 3% 17%28% 5%8%
37% 4% 4% 19%18% 10% 8%
OS and Device Breakdown by RegionU.S.
EU5
Device Makers
Device Makers
Other
Other
39+58+3+yScreen SizesOS
4” - 5”
5’’- 5.5’’
3.5”- 4”
5.5” - 6”58+20+13+9+y58%58%
39%
3%
20%
13%
9%
60+18+16+6+y60%18%
16%
6%18+72+10+yScreen SizesOS
4” - 5”
5’’- 5.5’’
3.5”- 4”
5.5” - 6”
72%
18%10%
*Based on acquired market share data
13
60+21+14+5+yUK
Germany
Device Makers
Device Makers
Other
Other
60+21+14+5+y60%
60%
21%
21%
14%
14%
5%
5%
28+59+13+yScreen SizesOS
4” - 5”
5’’- 5.5’’
3.5”- 4”
5.5” - 6”
59%
28%13%
17+76+7+yScreen SizesOS
4” - 5”
5’’- 5.5’’
3.5”- 4”
5.5” - 6”
76%
17%7%
����� ����� ����� ����� ����� �����
����� ����� ����� ����� ����� �����
30% 6% 16%28% 7%13%
42% 6% 20%17% 7%8%
����� ����� ����� ����� ����� �����
����� ����� ����� ����� ����� �����
30% 6% 16%28% 7%13%
42% 6% 20%17% 7%8%
*Based on acquired market share data
14
Perfecto Mobile uses a proprietary Test Coverage Optimizer to determine the exact number of devices that represent a 30/50/80% coverage rate. First, we aggregate raw data from public and private sources to understand usage profiles at a granular level. However, simply taking the X most popular devices will not ensure adequate coverage in the real world. Hardware/screen size/OS permutations are constantly changing and new promotions and device releases can impact turnover rates. The Perfecto Mobile Test Coverage Optimizer factors data from each of the four sectors to create a precise list to test and develop against for each targeted coverage rate. While the 30% coverage rate of 10-16 devices will provide basic device/OS coverage and some of the mix considerations, we recommend looking at the higher coverage goals as development progresses.
The number of devices in each test bracket may vary from quarter to quarter to reflect changes in mobile preferences. Additionally, the information in this report is based on generic requirements. Customizing test parameters for specific development projects may also cause the numbers to shift slightly. It is important to note that developers are more successful when the target device list is continuously tailored to specific test requirements. The Perfecto Mobile CQCF enters project specific device parameters for geography, screen size, OS, hardware, BU requirements, device mix and more. The CQCF analyzes current market data to output a customized, highly targeted Mobile Test Coverage Index.
30% 50% 80%
accounts for 10-16 unique device models
accounts for 17-25 unique device models
accounts for 26-32 unique device models
How the CQCF Works
Customer Speci�cCon�gurationRequirements
Customer SDLCConsiderations
Market Data & Perfecto Mobile
Cloud Data
Continuous Quality
Coverage Framework
Inputs Recommendations
1
3
2
1
2
3
4
Unique devices and operating systems for U.S. & Europe
Quality lab con�guration tuned for speci�c SDLC Practices
Total device quantity for lab sizing based on team size, locations and environment
speci�c requirements
TEST COVERAGEOPTIMIZER
*Items 2 & 3 are fully supported as part of an on-demand workshop by the Perfecto Mobile professional services team.
perfectomobile.com
© 2015 Perfecto Mobile Ltd.
About Perfecto MobilePerfecto Mobile, the world’s leader in mobile app quality, provides a hybrid cloud-based Continuous Quality Lab that enables mobile app development and testing teams to deliver better apps faster. The Continuous Quality Lab supports testing processes earlier and more often in the development cycle, giving way to faster feedback and improved time to market. Users can access an exhaustive selection of real mobile devices connected to live networks around the world and leverage them for testing and monitoring throughout the mobile application development lifecycle – from development, functional and performance testing to monitoring and support.
www.perfectomobile.com/testcoverageor call 781-205-4111
781-205-4111
Recommended