Structured Load Tests for Web ApplicationsMartin Lugan

QC Engineering Team

Status: 20.06.2013

Seite 2

SpeakerMartin Lugan is a Test Engineer in the Quality Control domain at Haufe-Lexware in Freiburg

He is responsible for topics such as PLS tests, test environment virtualisation and test data anonymisation .

Contact information:

Martin LuganHaufe-Lexware GmbH & Co. KGMunzinger Str. 9 79111 FreiburgTelephone +49 761 898-5209 E-Mail Martin.Lugan@haufe-lexware.comWeb http://www.haufe-lexware.com/

Seite 3

Introduction

Seite 4

Agenda

Why do we carry out structured Performance, Load and Stress tests (PLS)

The Test Types for PLS Tests

Requirements for Structured Performance and Load Tests

Test Planning: Mandatory information and risks

Test Execution

Result Assessment

Test Reports and Validation

Appendix

Seite 5

Why do we carry out structured Performance, Load and Stress tests?

Seite 6

What are the advantages of Load Tests?

Seite 7

Performance Management in Application Lifecycle Management (ALM)

Definition of non-functional requirements

Test types and test environment design

(Performance)

Lab and module test

Performance, Load and Stress test according to demand

Performance (End2End) Monitoring

Seite 8

Requirements for Structured Performance and Load Tests

Seite 9

PLS Test Types: Performance Test / Stress Test

Measurement of the processing speed, respectively measurement of response

times for certain use cases. Goal: Performance optimisation

System response observation in case of overload

Goal: Localising system limits, robustness, security

0

0,5

1

1,5

I teration1

I teration2

I teration3

I teration4

I teration5

Performance Test

User

Seite 10

PLS test types fail-over / scalability load test

System behavior observation in case a system component breaks down

Goal: Robustness, functional testing of the fail-over strategy

System behavior measurement depending on an ever-increasing load

Goal: the system overcomes the expected load, SLA, bottlenecks

0

50

100

150

1h 2h 3h 4h 5h

Fail-Over Test

User

<-Fail ->

Seite 11

PLS test types: Continuous Load Test

System behavior measurement depending ondifferent loads.

Goal: Stability check, resource leaks

Seite 12

Tools 1. Loadrunner 2. NeoloadAdvantages - Connection to HP-ALM

- The broadest selection of protocols and scripting techniques

- Extensibility through plug-ins or function libraries

- A good reporting module

- Simple and intuitive conditions- A relatively simple setup of the test

system- Successful inclusion of Java Applets- Licence manager

Disadvantages - Complicated handling- Time-consuming setup- No licence manager

- Limited extensibility- Limited protocol selection

Tools

Seite 13

Example for the Content of a Test Plan

Seite 14

Script creation and the test environment

Seite 15

Test Execution:

Seite 16

Typical response time wave

•Stability •Elasticity •Overload

Seite 17

Results Assessment - Successful test run

Even for an ever-increasing load generated by an increasing number of virtual users, the response times will only grow marginally.

However, the individual transactions with strongly differing response times are not satisfactory.

Seite 18

Results Assessment - Typical resource problem from the server side

The increasing number of vUsers means longer response times.

Seite 19

Results assessment - further errors

In this test, the load was raised in the first hour.

Afterwards, the load remained the same. After 1:30, we can see a significant drop in the load times.

The cause was an incorrectly configured caching of database accesses.

Seite 20

Results Assessment - further errors

In this test run, the DB connections were not closed by the application (see monitoring graph above - test run starting at 12:30). The result is, that after reaching the configured maximum DB connections, the server had to be restarted twice.

Seite 21

Monitoring

Seite 22

Test Report

Content

Test plan and deviations

The test plan is annexed and the deviations are documented

Deviations in the test execution

All deviations need to be documented

Results All relevant results are displayed and summarised.

Release If the target values defined in the test plan are reached, then the release of a product can be recommended.

Logs and scripts Are attached or archived

Seite 23

Validation In order to check if the tests have delivered realistic and meaningful results, a validation of results should be done.

- e.g. by comparing the resource load on the test and live system.

- Do we encounter load or performance problems in the live system that we did not encounter in the test execution?

<- CPU Load Test system

<- CPU Load Test system on a normal Monday morning

Seite 24

Appendix

References:

The slides 6, 17 were taken from „PLS-Roadshow” – Markus Lobreyer & Pandora Project

Translation into English: Victor Giurgiu

Seite 25

Performance Management Maturity Model (Maturity Levels)Level Description

0 Troubleshooting in production and sporadic tests

1 Performance Engineering for selected applications

2 Standardised Performance Assurance on strategic platforms

3 Known IT Performance Processes

4 Self-optimising Processes