49
© Copyright IBM Corporation 2006 CONFIDENTIAL – Not for Distribution PLANNING LOAD TESTING PLANNING LOAD TESTING LESSON 1 LESSON 1

Testing using load runner performance testing

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Testing using load runner  performance testing

ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution

PLANNING LOAD TESTINGPLANNING LOAD TESTING

LESSON 1LESSON 1

Page 2: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Why Plan?Why Plan?

So that you will be able to:

Build test scenarios that accurately emulate your system

Know the required resources for testing

Define the load test objective

Define success criteria (in measurable terms)

Page 3: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

LoadRunner WorkflowLoadRunner Workflow

Plan Load TestingPlan Load Testing11

Create Scenario3Run Scenario4Analyze Results5

Create Vuser Scripts2

Debug and Run Scenario Again6

Define load testing objectives

Analyze system under test

Plan LoadRunner Implementation

Page 4: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Planning Load Testing Planning Load Testing

In this lesson you will learn how to:

1 - Define load testing objectives

Objectives

2 - Analyze the system under test

3 - Plan LoadRunner implementation

Vuser Vuser

VuserVuser

Controller

Page 5: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

1 - Define Load Testing Objectives1 - Define Load Testing Objectives

Why test objectives?

“Successful projects are those that have a defined test objective… and provide actionable results and conclusion…”

Source: “The Art of Testing Network Systems” Robert W. Buchanan, Jr.

Limit your objectives

“...Projects should have one or, at the most, two testobjectives… More than two objectives dilutes the effortand makes [ testing ] virtually impossible.”

State objectives in measurable terms

“...There is nothing worse than completing [ a test ]and then disagreeing on whether or not the system passed.”

Page 6: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Common Test ObjectivesCommon Test Objectives

Application response timeHow long does it take to complete a task?

Configuration sizingWhich configuration provides the best performance level?

AcceptanceIs the system stable enough to go into production?

RegressionDoes the new version of the software adversely affect response time?

ReliabilityHow stable is the system under a heavy work load?

Capacity planningAt what point does degradation in performance occur?

Bottleneck identificationWhat is the cause of degradation in performance?

Product evaluationWhat is the best server for 100 users?

Page 7: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

State Objective in Measurable TermsState Objective in Measurable Terms

Measurable objectives provide baselines for evaluating results

Examples

Examples

Response timeunder a load of 50 sales representatives, the maximum response time for any transaction will not exceed 1 minute

Reliabilitythe system will run for 72 hours at 50% load(as measured by CPU utilization) without a crash, or any corrupted data

Page 8: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

2 - Analyze The System Under Test2 - Analyze The System Under Test

Identify system componentsschematic diagram of system

Describe configuration of each componenthardware/CPU speed

operating system

memory

Describe user profiletypes of users

common tasks for each user

task distribution during the work day

Page 9: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Identify System ComponentsIdentify System Components

Example

ExampleClients

586/ Windows 98

OCI

Pentium 400MHZ

NT 4.0

Oracle 8.0

Server

Page 10: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Describe ConfigurationDescribe Configuration

Front End ClientNo of clients 120Hardware/Memory 586 PC / 32 MbOperating system Windows 95Development tool Visual Basic

Example

Example

Middleware/App Server (if exists)Server typeCommunication with serverServer operating system

Back End ServerServer type Database- Oracle 8.0Communication with server ODBCServer operating system NT

Other Network ComponentsCommunication device handles 50 calls max.

Provide details about the configuration of each component:

Page 11: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

User ProfileUser Profile

Who are the users?

How many users are there?

What tasks do the users perform?

Business Process

Type and number of users

Task frequency (throughput per hour)

ReservationAgent (170)

Freq. FlyerAgent (50)

Manager (30)

1- Enter Orders

3 - Calc. F.F. Miles

4 - Calc. Sales

2 - Update Orders

100

50

70

8

B.P.Users

Example

Example

25

10

5

How often do users perform their tasks?

Page 12: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Transaction ProfileTransaction Profile

What tasks do the users perform?

2- Enter Orders

4 - Calc. F.F. Miles

5 - Calc. Sales

3 - Update Orders

125

60

75

8

Business ProcessTransactions Per Hour

1- Logon 10

Example

Example

Tasks Frequency

How often do users perform their tasks?

Page 13: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

3 - Plan the LoadRunner Implementation3 - Plan the LoadRunner Implementation

1 - Set up test environment

2 - Define type and number of Vusers to be created

3 - Determine hardware required to run Vusers

Page 14: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Set Up Test EnvironmentSet Up Test Environment

All testing should be done in a test environment

Allocate and configure machines to real-world conditionsone for each type of GUI Vuser

one for the database server

any other middleware server(s)

Allocate machines to run necessary number of DB Vusers

Allocate and configure one machine for Controller

Seed test database with appropriate amounts of data

Verify that business processes function correctly

Page 15: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

What Is a GUI Vuser?What Is a GUI Vuser?

A WinRunner script, performing actions on the client application

WinRunner

Allows you to measure the response time a real user will experience end-to-end

One GUI Vuser per host machine

Host

GUI Vuser

Client application

Page 16: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

What Is a DB Vuser?What Is a DB Vuser?

Multiple DB Virtual UsersMultiple DB Virtual Users ServerServer

API Calls

C program that sends API calls to the server

Allows you to generate load on the server using few machines

Excludes application’s GUI processing time

Many DB Vusers per host machine

C Program

C Program

C Program

C Program

Page 17: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

GUI vs. DB VuserGUI vs. DB Vuser

Database Server

Application Server

Application Server

Application Server

WAN and/or

Internet

Example

Example

GUI VuserGUI Vuser: End-to-end (including GUI)

GUI VuserGUI Vuser: End-to-end (including GUI)

DB VuserDB Vuser: Network and Server (No GUI)

DB VuserDB Vuser: Network and Server (No GUI)

Page 18: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Type and Number of VusersType and Number of Vusers

How many Vusers of each type (DB/GUI) do you need to run?

Guidelines:one or few GUI Vusers for each type of user connection

multiple DB Vusers generate remaining load for that user

100 customer service users(LAN connection) 2 98

50 Tokyo users (dial-in ISDN connection) 1 49

5 background batch processes — 5

20 managers(15 with 486; 5 with 586 machines)

1 (486 PC)1 (586 PC) 18

UserExample

Example

DBGUI

Page 19: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Which Hardware/Configuration?Which Hardware/Configuration?

Based on previous steps, which hardware and software configuration is needed for testing?

How many machines (PC/UNIX)?

How much memory?

Processor speed?

How much disk space?

How much swap?

Page 20: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Hardware/Configuration GuidelinesHardware/Configuration Guidelines

General:

One machine for Controller

One machine per GUI Vuser

GUI Vuser machine configuration should be similar to ‘real-world’ machine

Test server machine should be configured to real world

DB Vuser machines:

Can use a combination of NT and UNIX

Machine configuration depends on number of Vusers and activity of user

Exact configuration requirements in Appendix A

Page 21: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Which Hardware/Configuration?Which Hardware/Configuration?

Controller

NT machine Pentium 200 MHz 64 MB memory

Database Server

1 GUI Vuser(Accounting)

• NT machine • Pentium 200 MHz• 32 MB memory

29 DB Vuser(Accounting)

• NT machine•Pentium 166

MHz• 64 MB memory

UNIX machine 256 MB memory

Example

Example

450 DB Vusers(Bank tellers)

Page 22: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Summary Summary

Define load testing objectiveslimit your objectives

state objectives in measurable terms

Analyze the system under testsystem components

configuration of each component

user profile (users, tasks, distribution)

Plan LoadRunner implementationset up test environment

decide type and number of Vusers to be created

hardware required to run Vusers

Page 23: Testing using load runner  performance testing

CONFIDENTIAL 23 © Copyright IBM Corporation 2006

Lab ExerciseLab Exercise

Page 24: Testing using load runner  performance testing

ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution

LOADRUNNER SCENARIOS

LESSON 2

LR/NTIntro6.0/02Copyright 2000Mercury Interactive Corp.

Page 25: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

LoadRunner Workflow

Plan Load TestingPlan Load Testing11

Create ScenarioCreate Scenario33Run ScenarioRun Scenario44Analyze ResultsAnalyze Results55

Create Vuser ScriptsCreate Vuser Scripts22

Debug and Run Scenario AgainDebug and Run Scenario Again66

• Define and connect hosts

• Define Vusers and scripts

• Group and schedule Vusers

• Set rendezvous policy

Page 26: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Lesson Objectives

In this lesson you will learn: LoadRunner elements How to set up a scenario How to run a scenario

Page 27: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

LoadRunner Environment

LoadRunnerController

Vusers

Host 1 Host 2

VusersDatabaseServer

Page 28: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

The LoadRunner Controller

Runs scenarios Controls and communicates with each Vuser via the Agent Displays output messages Displays execution status Collects performance data

Page 29: Testing using load runner  performance testing

CONFIDENTIAL 29 © Copyright IBM Corporation 2006

Controller Main Window

Initialize /Run Vuser

Vusergroups

Vuser details

Sub-window tabs

Page 30: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Controller Sub-Window Tabs

Hosts window- all available hosts and their connection statuses

Scripts window- paths to all available Vuser scripts

Vusers window- create and select Vusers and groups

Online window- view real-time system resources during scenario execution

Page 31: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

How Does Controller Communicate with Vusers ?

Start the Remote Command Launcher on each host machine

Controller

Machine A

Machine B

RemoteCommandLauncher

• RCL starts the Agent and initializes the Vusers

INITIALIZE

Agent

Vusers

• Controller Runs Vusers via the Agent

RUN

Page 32: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

The Scenario

An environment that describes all the conditions under which you want to test your system

virtualusers

loadconditions

hostmachines

resultslocation

testscripts

Page 33: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Setting Up a Scenario

1 - Define Hosts

2 - Connect Hosts

3 - Define Vuser scripts

4 - Add Vusers

5 - Set rendezvous policy (optional)

6 - Set up Vuser Scheduling (optional)

Page 34: Testing using load runner  performance testing

CONFIDENTIAL 34 © Copyright IBM Corporation 2006

Machine name orIP address

Host Status, Ready/Active

1 - Define Hosts

Results handling

information

UNIX information

for DB Vusers

Initialization Quota

Limits per host for each Vuser

type

WinRunner information

for GUI Vusers

Page 35: Testing using load runner  performance testing

CONFIDENTIAL 35 © Copyright IBM Corporation 2006

2 - Connect Hosts

Connect /disconnect host

Connection statusHost name

Page 36: Testing using load runner  performance testing

CONFIDENTIAL 36 © Copyright IBM Corporation 2006

3 - Define Vuser Scripts

A scenario typically contains multiple instances of each script

Vusers assignedto this script

Transactions and Rendezvous in script

Path to script

Vuser type depends on script type

Page 37: Testing using load runner  performance testing

CONFIDENTIAL 37 © Copyright IBM Corporation 2006

4 - Add Vusers

Quantity Hostname

Script name

Click the Add Vusers toolbar button

Page 38: Testing using load runner  performance testing

CONFIDENTIAL 38 © Copyright IBM Corporation 2006

4 - Add Vusers (cont.)

NOTE: Run-Time Settings are discussed in the VuGen course

Host machine

Script name

1. Select an existing Vuser from the grid2. Select Vuser > Details from the menu

Page 39: Testing using load runner  performance testing

CONFIDENTIAL 39 © Copyright IBM Corporation 2006

5 - Define Rendezvous Policy(optional)

Enable/disablerendezvous

Specify when torelease Vusers

Specify maximumtime Vusers wait

to be released

Release Vusersthat arrived

Page 40: Testing using load runner  performance testing

CONFIDENTIAL 40 © Copyright IBM Corporation 2006

• LoadRunner Program group – select SchedulerDouble-click to schedule a new group

Properties button to select details

Specify duration settingsSpecify Incremental Initialization PaceSpecify Incremental Run PaceSpecify Termination Pace – only available when a group duration has been set

Visual Group Settings

6 - Scheduling Vusers(optional)

Page 41: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Scenario Wizard

Use theScenario Wizard to simplifyscenario creation

Page 42: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

LoadRunner Workflow

Plan Load TestingPlan Load Testing11

Create ScenarioCreate Scenario33Run ScenarioRun Scenario44Analyze ResultsAnalyze Results55

Create Vuser ScriptsCreate Vuser Scripts22

Debug and Run Scenario AgainDebug and Run Scenario Again66

• Check Remote Command Launcher

• Specify results directory

• Initialize Vusers

• Run Vusers

Page 43: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Running a Scenario

1 - Make sure Remote Command Launcher is active on all hosts

2 - Specify the directory for storing execution results- By default, the runtime files are stored locally on each host. After the scenario, the

results are collated and processed on the Controller machine.- Alternatively, instruct LoadRunner to use a shared network drive- Use Mercury’s TestDirector to store results to a project

3 - Initialize Vusers

4 - Run Vusers

Page 44: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

The Vuser Statuses

Initializing Ready Running

Rendezvous

Exiting

Done

Page 45: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Tips for Smooth Scenario Execution

Make sure Remote Command Launcher is active on each host. Verify successful execution of each Vuser script before running with

LoadRunner. Make sure all scripts have the proper path and run-time settings. Disconnect from the host and refresh after editing a script, to ensure that

the new version of the script is transferred to the host.

Page 46: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Managing the ScenarioGood Practices:

Run the scenario gradually for easy problem solving:- individual Vusers- a group of Vusers- entire scenario

Initialize Vusers before running- initializing process is resource intensive- ensures that all Vusers start from same point

Page 47: Testing using load runner  performance testing

CONFIDENTIAL © Copyright IBM Corporation 2006

Summary

Set up a scenario1 - define Hosts

2 - connect Hosts

3 - define Vuser scripts

4 - add Vusers

5 - set rendezvous policy (optional)

6 - schedule Vusers (optional)

Run a scenario1 - check Remote Command Launcher

2 - specify results directory

3 - initialize Vusers

4 - run Vusers

Page 48: Testing using load runner  performance testing

ibm.com/bcs © Copyright IBM Corporation 2006CONFIDENTIAL – Not for Distribution

SCENARIOS

DEMO

Page 49: Testing using load runner  performance testing

CONFIDENTIAL 49 © Copyright IBM Corporation 2006

Lab Exercise