23
Mobile Testing Center of Excellence Group Perfectomobile Effective Mobile Performance Testing Using Real Devices Enabling Enterprise mobility Extending Software Quality to Mobile

Webinar effective mobile performance testing using real devices

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Webinar effective mobile performance testing using real devices

Mobile Testing Center of Excellence GroupPerfectomobile

Effective Mobile Performance Testing Using

Real Devices

Enabling Enterprise mobility

Extending Software Quality to Mobile

Page 2: Webinar effective mobile performance testing using real devices

PresentersLydia Casillas

Vice Presidents of

Product Sales &

Marketing

Dave Berg

Vice President of

Product Strategy

Amir Rozenberg

Director of Product

Management,

Mobile Monitoring

& Performance

Testing

Uzi Eilon

Pre-Sales Director

Page 3: Webinar effective mobile performance testing using real devices

Webinar Agenda

4

• Q&A

3

• Mobile Performance Considerations pains and solutions- SkyIT

• MobileCloud Performance Live Demo

2

• Mobile Performance Keys to success -Shunra

1

• Intro

Page 4: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.

Mobile App PerformanceKeys to Success

Page 5: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.5

Shunra overviewCompany Overview Representative Customers

� Solution: Network virtualization for software testing to ensure application performance early in the dev / ops lifecycle

� Capture production network conditions

� Test applications on a virtualized network in the test environment

� Analyze and optimize application performance

� Customers: Over 2,500 customers75% of the Fortune 500

� Market Success: � 5 of the Fortune 10 companies� 7 of the top 10 banks in the US� 5 of the top 10 global financial institutions� 6 of the top 10 global retailers� 7 of the top 12 global auto manufacturers� 6 of the top 10 aerospace/defense

companies� 5 of the top 7 global pharmaceuticals� 4 of the top 5 global telecoms

� Value Prop: Average customer ROI < 3 monthsAverage customer saves $1 million per year

Page 6: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.6

State of the testing market

Testing Remains Challenging (World Quality Report, Sept. 2012)� Organizations struggle with the challenges of mobile – only 31% formally

test mobile apps

� Lack of testing tools and knowledge persists – 59% characterize QA teams ability to effectively test as merely “average”; two- thirds do not have the right tools and over one-third lack appropriate tes ting methodologies

Performance Awareness (Computerworld UK, May 2012)� Performance problems are highly prevalent – 74% cite performance

issues with critical business applications

� Performance problems are getting worse – 82% report speed and responsiveness problems in the past year and nearly half report issues occurring at greater frequency

Page 7: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.7

The average application connects to 38 discreet services

Mainframe

Storage

DB

Servers

Web Services

App

Servers

Web

Servers

Load

Balancers

MobileComponents

Major ISP

3rd Party

Cloud Services

Content DeliveryNetworks

Mobile Carriers & ISPs

Browsers &Devices

Customers

Employees

Private or Public

Cloud Services

Application complexity increases failure

Page 8: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.8

In the time it takes to blink an eye, 400 milliseconds to be exact , online and mobile users will have begun to abandon an application.

App Provider Competitor

Abandon the transaction, try

later on computer

43%

Become more likely to prefer a competitor

Log a complaint with customer

service

Go directly toa competitor

16% 14% 12%

Source: Harris Interactive Mobile Transactions Survey for TeaLeaf, March 2011

When customers “struggle” with an app…

Page 9: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.9

“Real-world” virtualization improves the reliability of test environments and results

Requirements for accurate testing

Ser

vice

sTest

PlatformSoftware defects cost the U.S. economy $59.5 billion annually. A full one-third of this cost could be avoided if better software testing was performed.

The Economic Impacts of Inadequate Infrastructure for Software Testing, NIST

Page 10: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.10

Network virtualization for software testing

Comprehensive application performance testing

• Discovery of production network conditions across Mobile, Cloud, WAN and Internet networks

• Testing of applications in an environment with virtualized production network conditions

• Analysis and optimization of performance before deployment and before poor performance affects users

Page 11: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.11

Analysis, remediation and optimization

• Automated optimization/remediation recommendations, proven to improve HTTP application performance by up to 40%

• Automated Performance Scorecard highlights problem transactions

• SLO/SLA validation

• Secure analysis with HTTP/HTTPS decode

• Waterfall diagram for resource analysis (resource size and load time)

Shunra Analytics

Page 12: Webinar effective mobile performance testing using real devices

© Copyright 2012 Shunra Software, LLC. Information contained herein is subject to change without notice.12

7 techniques you can use right now

1. Put stylesheets at the bottom

2. Don’t put scripts in the HEAD

3. Use HTML5 storage

4. Cache your redirects

5. Design for progressive rendering

6. Reference all images in the HTML

7. Increase your server’s keep-alive timeout

Suggested Optimization Tips

Page 13: Webinar effective mobile performance testing using real devices

Mobile Performance

Considerations, Pains and

Solutions

Lydia Casillas

Sky IT Group

Page 14: Webinar effective mobile performance testing using real devices

About Sky I.T. Group

• Established in 1997• Headquartered in NYC, Office in: ATL, Raleigh, Boston,

Baltimore, Ft. Lauderdale & San Jose• 60+ Employees• Focus: Leverage Technology to solve Business Challenges.

People, Process & Technology• Perfecto, HP & Shunra Software Reseller• Perfecto & HP Solution Provider – Professional Services• Perfecto & HP Education Provider

• Specialty Focus:• Managed “Testing-as-a-Service”• Application Testing Center-of-Excellence

• Service Delivery – Throughout the USA

Customers-at-a-Glance

Page 15: Webinar effective mobile performance testing using real devices

Performance Testing

Mobile Web & Native

Applications are a

NECESSITY!!!

– Customers, Customers, Customers

– More users are moving to smartphone & tablets to access business

applications, purchase consumer goods or manage their daily lives,

than ever before.

– User expectations are at a high…no one wants to wait anymore

– Businesses need to keep up with these trends or they will lose

customers faster than acquiring them.

– 57% of online consumers will leave a site after waiting more than 3

seconds. 80% of those people will never return.

Satisfied

customers

tell 3 friends Angry

customers

tell 3,000

Angry

customers

tell 3,000

Why are more companies making Mobile Performance Testing a Higher priority &

initiative?

Page 16: Webinar effective mobile performance testing using real devices

Common Customer Pain Points

in Mobile Performance TestingIn speaking with Financial & Commercial Entertainment Customers…

1. Devices: – Increasing number of devices w/different OS & Screen sizes

• Cost: Apple, Android, Blackberry, Windows Mobile , etc.

– Availability of devices from multiple providers to simulate real network traffic & performance usage, to test against.

• How can you afford to test all these different devices, multiple providers when it changes daily?

2. Users:– 100M+ users belong to the Gigabyte Club (Over 1GB/month).

• Can our infrastructure support that load?

3. Network:– Limited & Variable bandwidth on Mobile devices when data

connectivity is spotty.

– Network Performance Impact with shared mobile and web users. • Are my customers getting the same service experience on their mobile device as

they are on a desktop?

Fea

ture

s

OS & Browsers

Mo

bile

Manufacturers

Wire

less

Carriers

Location

Page 17: Webinar effective mobile performance testing using real devices

Considerations: Mobile Performance Testing

Satisfied

customers

tell 3 friends Angry

customers

tell 3,000

Angry

customers

tell 3,000

Manual vs.

Automation

In-house vs.

Outsourced

Agile vs.

Waterfall

Page 18: Webinar effective mobile performance testing using real devices

Type of Mobile Application &

Service

Mobile Service: Reservation Service Mobile App, Customer profile,

Travel Details, Travel Specials

Mobile Device Support: iOS, Android (recently releases)

Challenges: • Various Android devices to test against

• Simulating user load with Mobile & standard user Traffic

• Analyze Network Traffic at different peak travel times

• No visibility into Mobile App Transactions

Current Technology Gaps: • Supporting a wide Variety of Protocols or Custom Protocols

• Stress Testing – High user count simulations

• Deep Diagnostics to pinpoint cause of performance issues

• Limited Network Visibility

• Performance Analysis

Customer Profile: Travel, Recreation &

Leisure Customer

Page 19: Webinar effective mobile performance testing using real devices

Functional Performance Availability

How do we support these Pain

Points & Gaps

+UFT BSM

Backend Monitoring:

SiteScope/Diagnostics

Mobile Network

Characteristics

Mobile Protocols:

PC

LR

End-to-End Mobile Application Lifecycle Management

PEOPLE + PROCESS + TECHNOLOGY

Page 20: Webinar effective mobile performance testing using real devices

Mobile Performance

Demo

Uzi Eilon

Page 21: Webinar effective mobile performance testing using real devices

Come Meet Us At HP Discover

Page 22: Webinar effective mobile performance testing using real devices

In this white paper, you will be able to gain insights

into the following mobile performance aspects:

Key building blocks for a mobile performance solution

Best practices and process to build a mobile performance

solution

Best practice solution design overview

LINK to download (Registration required):

http://help.perfectomobile.com/article/AA-01594/

Mobile Performance White paper

Page 23: Webinar effective mobile performance testing using real devices

If you have any further questions,

please contact:

Perfecto [email protected]

[email protected]

[email protected]

[email protected]

[email protected]

Thank You!

Join us on LinkedIn

Mobile Testing Center of Excellence