ABCs of Performance Testing Series I€¦ · need to be tested? StormRunner Load Center Type of performance test LoadRunner Customer can always use a mix and match approach and share

© Copyright 2014 Vivit Worldwide © Copyright 2014 Vivit Worldwide

ABCs of Performance Testing Series I August 20, 2014

© Copyright 2014 Vivit Worldwide

Brought to you by


Hosted by

Richard Bishop

Vivit United Kingdom Chapter Leader

Vivit Director of Education


Today’s Speakers

Mark Tomlinson

Practitioner of Performance Testing

West Evergreen Consulting, LLC

Silvia Siqueira

PMP, ITIL Expert, Scrum Master

WW Senior Product Marketing Manager

HP Software


Housekeeping

• This “LIVE” session is being recorded

Recordings are available to all Vivit members

• Session Q&A:

Please type questions in the Questions Pane


Webinar Control Panel

Questions

Toggle View Window between

Full screen/window mode.


NEWS August 20, 2014


News: Free Meetup in NY

Free Meetup in NY

– Learn what is new for Application Performance

lifecycle and DevOps presented by Prod mgmt.

and RnD

– September 9 (Monday) 4 pm to 7 pm.

– Venue: New York Hilton Midtown NY

– Register at: http://vivitworldwide.site-

ym.com/events/event_details.asp?id=483217&group=

– Attendees will be invited to attend the Velocity Ignite

Reception Event -7 pm to 8:30 pm

http://velocityconf.com/velocityny2014/public/content/hotel

http://vivitworldwide.site-ym.com/events/event_details.asp?id=483217&group



http://velocityconf.com/velocityny2014/public/schedule/stype/event

http://velocityconf.com/velocityny2014/public/schedule/stype/event


News: Velocity Conference NY

Conference Velocity NY – Sept 15-17

– HP Platinum Sponsor

– 4 sessions by HP Prod Mgr and RnD

– Venue: New York Hilton Midtown NY

– Take advantage of 25% discount:

• Code HP25

– Register at:

– http://velocityconf.com/velocityny2014


http://velocityconf.com/velocityny2014

http://velocityconf.com/velocityny2014


News: Free Meetup in Anaheim-CA

Free Meetup in Anaheim - CA

– Learn what is new for Mobile and Application

Lifecycle

– October 13 (Monday) 4 pm

– Venue: Disneyland® Hotel

– Register at: http://vivitworldwide.site-

ym.com/events/event_details.asp?id=482076&group=



http://starwest.techwell.com/hotel-travel-1






NEWS: StarWest Conference

Conference StarWest CA – Oct 15-17

– Breaking Software

– HP Platinum Sponsor: 2 sessions by HP Prod Mgr and RnD

– Venue: Disneyland® Hotel

– Exclusive vendor discount code S14VW to save up to $200 off their STARWEST

registration! Register at: http://starwest.techwell.com/




http://starwest.techwell.com/




Never-ending PerfBytes Podcasting

As Summer comes to a close James and Mark will get back into full-time publishing in the Fall we have lots of new ideas. PerfRants, sponsor topics, interviews and guest hosts - and as always Satan will keep bringing us the News of The Damned with the hottest in website and system failures across the globe. Listen now at www.perfbytes.com!

http://www.perfbytes.com/


NEWS: Discover Barcelona Dec 2014

HP Discover Barcelona

Registration and Call for papers is now open:

https://h30550.www3.hp.com/portal/cfp/home.ww

https://h30550.www3.hp.com/portal/cfp/home.ww


NEWS: Additional events

• October 16-18 - WOPR #23 "the twenty-third gathering of the Workshop on Performance

and Reliability“, in Waltham, Massachusetts.

And for those who don't know WOPR is a peer conference and the subject is "Performance

Escalation" ...and I will be serving as the content owner. For more information visit

www.performance-workshop.org

• November 3rd-6th STPCON Fall - being held in the mile-high city of Denver, Colorado - a

really great conference for performance testers and engineers, we have a great line up of

workshops and sessions - learn more at www.stpcon.com

http://www.performance-workshop.org/



http://www.stpcon.com/


ABCs of Performance Testing Series I August 20, 2014


Polling Question #1

What is your knowledge level for Performance testing?

a. None

b. Basic

c. Intermediate

d. Advanced

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17

Important facts about performance

44% of mobile app defects are found by end users

63 percent of testers stated they were not

63% not able to cover the number of test

cases or test across all the different

devices and OS versions

Interface problems at 58 %

Performance at 52%

Functionality at 50%

Device compatibility at 45%


What is Performance?

• Performance Testing – research

• Performance Validation – gatekeeping

• Performance Engineering – design/dev

• Performance Management – maintenance

• Performance Optimization – re-design

• Performance Capacity – trending, forecasting

• Performance Review – managing people…


Performance Requirements*

Performance is an attribute of Quality:

• Usability – is the site usable if it’s too slow?

• Scalability – can it handle more users?

• Reliability – will it stay up?

• Availability – how long will it stay up?

• Manageability – can we monitor it?

• Maintainability – can we keep it up?

(Scott Barber calls them “desirements” more accurately)

Performance testing lifecycle

Step I

Step II Test preparation: creating the scripts and

scenarios

Step III Test execution: running and monitoring

scenarios

Step IV Test analysis and reporting: understanding the results

Monitoring in production Step V

Planning the test

Planning the testing

Business goals and SLAs

Test requirements

Criticality

Application under test (AUT)

User behavior, user locations, and application timeframe usability

Special factors regarding the application under test

Inter-disciplinary Team


Performance Organizations

Performance Testing Practice

Production Performance

(a.k.a. “the Real World”)

Development (very unreal world)


Performance Test Design

Proper performance test design starts by stating a hypothesis about the system and then determining the

parameters for the system conditions to simulate against the system and the specific measurements to

be taken during the simulation to support the calculated analysis required to determine the results of

the test.

(…or just throw some load at it!)


Test Types: Load Test

Objective: validate the system’s ability to handle a sustained load at

a constant specific level (e.g. Run for 2 hours at 1000 trans/sec)

1x Load for 1-2 hours


Test Types: Stress Test

Objective: apply gradually, continuously increasing load to the system until an error state and/or failure occurs (e.g. increase # of virtual users until the system crashes)


Test Types: Scalability Test

Objective: measure system performance at consistent intervals of increased load (1x, 2x and 3x), useful for calculating a trend for performance projections

1x

2x

3x


Business Value of Performance

Money Risk



Every business leader wants to:

• ensure a positive customer experience

• ensure a system can grow and scale

• ensure the system can sustain heavy traffic

• ensure the application works in diverse conditions

• estimate future spending to support the system

• optimize the system to meet new goals



Find ways to convey the meaning of performance results:

• show how performance impacts customers

• always find a way to make your test relevant

• show performance impacts to the bottom line

• find a way to win over your stakeholders

• target multiple audiences with specific results

• expand the sophistication of your test


The graphs from your testing tool can help to show interesting technical anomalies or exceptions – like a logout exception shown above.



Graph shows a scalability test that fails to reach 2x or beyond



$10K/sec

$1K/sec

Add Financial Information to the Chart

2X


Join the Revolution


Business Growth

The cost of doing performance testing

The need to be proactive: App Performance Testing

Trusted Brand

Revenue

High Productivity

Customer Satisfaction

Applications

Faster user response time


Polling Question #2

Are you currently using any performance testing solutions?

a. Yes

b. No


HP Performance testing solution

StormRunner Load

Simple, Smart, Scalable

Cloud Testing

Free public beta

LoadRunner

Easy, Fast, Powerful

Performance and load testing

Free 50 Virtual users

For unlimited time

Performance Center

Global Collaboration and

Sharing

Full LoadRunner Capabilities

plus Center of Excellence

Free 50 Virtual users

For unlimited time

Network Virtualization + Service Virtualization

If you ever dream to have it all together, you have it now!


Selecting the HP Performance testing solution Customer needs for

performance testing

Web and mobile

(test in the cloud)

Simple, Scale, Agile

AUT in the cloud

Delivered by HP

Any application

Easy, Fast, Powerful

Site. Project based

test

Global and Complete

End-to-End

performance testing

On Premise Cloud based, on

demand

Delivered on SaaS

Where the

test will be

executed

What app

need to be

tested?

StormRunner

Load

Type of

performance

test

LoadRunner

Customer can always use a mix and match approach and share scripts between LR/PC and SRL e.g. if

customers wants an on-prem deployment but wants to add capacity for a new mobile team they could use

LR/PC + SRL together

Performance

Center

On premise


Why HP

Trusted by

millions of users

Most comprehensive

features compared

to others

Solutions for ANY

performance testing

need

50 FREE Virtual users

for an unlimited time

http://www8.hp.com/us/en/software-solutions/loadrunner-load-testing/try-now.html?action=downloadsoftware&tcmid=tcm:245-865742&originid=415817&jumpid=ba_ci9kerb358






Download HP LoadRunner now

50 Vusers free for unlimited time

www.hp.com/go/loadrunner

Join us @hploadrunner

Facebook group: HP LoadRunner

(https://www.facebook.com/groups/HPLoadRunner/)

http://www.hp.com/go/loadrunner

https://www.facebook.com/groups/HPLoadRunner/


SUMMARY

• Get started with Performance Objectives (“requirements”)

• Decide which kinds of tests will best fit the goals

• Find a stakeholder in IT and Business for each test type

• Get organized and follow the right process for your work

• Gather your results data into a meaningful conclusion:

– use proper visualizations that convey the message

– your business leaders aren’t performance engineers

– your technical leaders aren’t thinking about money or risk


Polling Question #3

What other topics are you interest for webinars?

a. Best Practices and Requirements

b. Cloud performance testing

c. Transaction simulation, Test execution, metrics analysis

d. Continuous testing, Continuous integration

e. Advanced topics: Websockets, HTML5, Speedy, Center of

Excellence


Contact me…

Mark Tomlinson

Independent Performance Consultant

[email protected]

775.773.8780

mtomlins.blogspot.com / @mtomlins

www.perfbytes.com / @PerfBytes

Questions and Answers?

Thanks


HP Performance testing solutions

Delivered on SaaS

StormRunner

Load LoadRunner

Customer can always use a mix and match approach and share scripts between LR/PC and SRL e.g. if

customers wants an on-prem deployment but wants to add capacity for a new mobile team they could use

LR/PC + SRL together

Performance

Center

On premise Performance

Testing solutions

Profiler solution

to pinpoint root cause Diagnostics

Detect Network behavior Network Virtualization

Virtualize test environment Service Virtualization

Monitor with SiteScope

Documents

ABCs of Performance Testing Series I€¦ · need to be tested? StormRunner Load Center Type of performance test LoadRunner Customer can always use a mix and match approach and share