Upload
judith-cougill
View
221
Download
5
Tags:
Embed Size (px)
Citation preview
Web Performance, Load, and Stress Tests
Mainframe and Open System Network
Natural Conference Boston
Dieter W. Storr
Los Angeles Times
May 2004 Dieter W. Storr -- [email protected]
2
Performance, Load, and Stress TestsContents
What are performance, load, and stress tests?
Why and when should you test?
What response time do you expect?
What parts play a role in the network?
Network basics
Logical and physical transactions
May 2004 Dieter W. Storr -- [email protected]
3
Performance, Load, and Stress TestsContents
What tools can you use to test these parts?
When should you start performance tests?
Web site performance, analysis and tuning
ChecklistLinks
May 2004 Dieter W. Storr -- [email protected]
4
What are Performance Tests?
Test each part of the system, such as web server, web applications, databases, etc.
Tests can run usually without requesting page images in order to concentrate the testing on the script and code itself
?
?
?
?
May 2004 Dieter W. Storr -- [email protected]
5
What are Load Tests?
Load tests are performed by testing the website using the best estimate of the traffic your website needs to support.
Define the maximum time it should take for a page to load -- from a usability and customer experience standpoint, not from a technical one.
?
?
?
?
May 2004 Dieter W. Storr -- [email protected]
6
What are Stress Tests?
To simulate “brute force” attacks that apply excessive load to system, such as web server, web applications, databases, etc.
To estimate the maximum load that your system can support
To learn the traffic thresholds of your system
How it will respond after exceeding its threshold
?
?
?
?
May 2004 Dieter W. Storr -- [email protected]
7
What are Stress Tests?
Ramp test A variation of stress testThe number of users is increased
over the life of the test
?
?
?
?
May 2004 Dieter W. Storr -- [email protected]
8
Why Performance Tests?
Determine that all functions do what they meant to do (function test)
Ensure that tasks and objectives are easily to be accomplished (usability test)
Guarantee a result from the website in an acceptable time (performance test)
May 2004 Dieter W. Storr -- [email protected]
9
Why Performance Tests?
Optimize each part of the system for increased web traffic
Ensure that the end-user’s mouse click is not met with silence
May 2004 Dieter W. Storr -- [email protected]
10
Why Load Tests?
To compare the maximum time it should take for a page to load with the effective used time.
Calculate the impact of exceeding that maximum time. Will you lose sales or prospective customers?
Calculate the anticipated load and load pattern for the website.
Determine to do some work on the application or webserver.
May 2004 Dieter W. Storr -- [email protected]
11
Why Stress Tests?
Determine the maximum load
Test excessive system load
Determine the traffic thresholds of your system
Experience the reaction of the system after exceeding its threshold
Ramp tests can help to determine what maximum load a server can handle while providing optimal access to web resources
May 2004 Dieter W. Storr -- [email protected]
12
When Performance Tests?
You cannot start early enough!Start before a single line of code is written
Network Load balancer Application Database Web servers
During development, all software engineers and QA team should have access to performance test tools
May 2004 Dieter W. Storr -- [email protected]
13
When Load Tests?
As soon as web pages are working
QA team should conduct the first load tests
Performance testing should be part of the regular testing routine each day for each build of the software
May 2004 Dieter W. Storr -- [email protected]
14
What Response Time Do You Expect?
Does your company set limits?KEEP in mind:Many websites today have a serious
business mission => to take moneyReason: Advertisement opportunities,
selling retail products, etc.Websites and applications need to be
up and running at all times
May 2004 Dieter W. Storr -- [email protected]
15
What Response Time Do You Expect?ResponseTime
User's View
< 0.1 s User feels that the system is reactinginstantaneously
< 1.0 s The user experience is notcompromised. Although the user isunhappy with the wait, they are stillfocused on the current web page
<10 s As wait times get close to 10s, studieshave shown that the likelihood of userdistraction increases greatly
> 10 s User is most likely distracted from thecurrent website and loses interest
Source: Jakob Nielsen, Designing Web Usability, New Riders Publishing, 2000
May 2004 Dieter W. Storr -- [email protected]
16
What Response Time Do You Expect?
95% of all web requests should be processed in less than 10 seconds!
BE AWARE:If performance slows even a little, fickle web users are likely to jump quickly to a competitor’s site
May 2004 Dieter W. Storr -- [email protected]
17
What Parts Play a Role in The Network?
ADABAS
NATURAL
CGI
Web Server
Web Browser
Basic Connections
May 2004 Dieter W. Storr -- [email protected]
18
What Parts Play a Role in The Network?
HubsSwitchesRoutersFirewallsLoad BalancerApplication ServersDatabase ServersWeb Servers
May 2004 Dieter W. Storr -- [email protected]
19
Network Basics
Hub / Switch / Bridge / Router / Firewall
In data communications, a hub is a place of convergence where data arrives from one or more directions and is forwarded out in one or more other directions. A hub usually includes a switch of some kind.
See USB 7-Port HubTo connect severalUSB devices
May 2004 Dieter W. Storr -- [email protected]
20
Network Basics
Hub / Switch / Bridge / Router / Firewall
In telecommunications, a switch is a network device that selects a path or circuit for sending a unit of data to its next destination.
See 8-Port Network Switch
May 2004 Dieter W. Storr -- [email protected]
21
Network Basics
Hub / Switch / Bridge / Router / Firewall
A switch may also include the function of the router, a device or program that can determine the route and specifically what adjacent network point the data should be sent to.
In general, a switch is a simpler and faster mechanism than a router, which requires knowledge about the network and how to determine the route
May 2004 Dieter W. Storr -- [email protected]
22
Network Basics
Hub / Switch / Bridge / Router / Firewall
In telecommunication networks, a bridge is a product that connects a local area network (LAN) to another local area network that uses the same protocol (for example, Ethernet or token ring).
May 2004 Dieter W. Storr -- [email protected]
23
Network Basics
Hub / Switch / Bridge / Router / Firewall
On the Internet, a router is a device or, in some cases, software in a computer, that determines the next network point to which a packet should be forwarded toward its destination.
See 4-Port Cable/DSL Router
May 2004 Dieter W. Storr -- [email protected]
24
Network Basics
Hub / Switch / Bridge / Router / FirewallA system designed to prevent unauthorized
access to or from a private network. Firewalls can be implemented in both hardware and software, or a combination of both.
PIX Firewall from CISCOPrivate Internet Exchange Firewall
Gauntlet Firewall from NAINetwork Associates Inc.
May 2004 Dieter W. Storr -- [email protected]
26
Web Server
Firewall
Appl. Server
InternetInternet
Screen Scraper
Firewall
Com-pleteNATURAL ADABAS
CICSCOBOL VSAM
Firewall
SSL
OracleFirstLogic
DBs
Web Server
Appl. Server
Screen Scraper
CISCO Router
e.g. Apache
e.g. WebLogic BEA
e.g. Attachmate
Main-frame
Open
System
May 2004 Dieter W. Storr -- [email protected]
27
Logical and Physical Transactions?
ADABAS
NATURAL
Appl. Server
Web Server
Web Browser
May 2004 Dieter W. Storr -- [email protected]
28
Response Time / Click Time Tests
ADABAS
NATURAL
Appl. Server
Web Server
Web BrowserFunction:
Display my account
May 2004 Dieter W. Storr -- [email protected]
29
What Tools Can You Use to Test?
Web Server Stress ToolUser simulationLogging periodSelect URLsURL recorderData mergingBrowser simulationReviewing summary and detailed log resultsAnalyzing graphical results
Web Browser
May 2004 Dieter W. Storr -- [email protected]
30
CSS Stress Test 5 Users 10 Clicks25 Sep 2003, 11:52-11:53
May 2004 Dieter W. Storr -- [email protected]
31
CSS Stress Test 5 Users 10 Clicks14 Oct 2003, 14:05
May 2004 Dieter W. Storr -- [email protected]
32
09/25/2003 11:52:02 AM --- WEBSERVER STRESS TOOL
User #1 Click #1 started, used URL #1 (CSS-Prod-1) POST-DATA= "[email protected]&password=<snip>
User #1 Click #1: Request 1 HTTP State: Initialized (0 ms)
User #1 Click #1: Request 1 HTTP State: DNS Lookup begin (7 ms)
User #1 Click #1: Request 1 HTTP State: DNS Lookup done (17 ms)
User #1 Click #1: Request 1 HTTP State: Connected (504 ms)
User #1 Click #1: Request 1 HTTP State: Waiting for header (512 ms)
User #1 Click #1: Request 1 HTTP State: Receiving Header (5937 ms) (=Time to First Byte)
User #1 Click #1: Request 1 HTTP State: Waiting for body (5947 ms)
User #1 Click #1: Request 1 HTTP State: Done (5999 ms)
User #1 Click #1: CLICK-Request 1: Time=6008 ms, TFB=5937ms, Bytes=16038, HTTP-StatusCode=200 (OK)
User #1 Click #1: !Success! All requests of click done (6046 ms)
May 2004 Dieter W. Storr -- [email protected]
33
What Tools Can You Use to Test?
APAS/INSIGHTADABAS
DWSCWUSE: SUMMARIZE (LS=150) COUNT (HD='COMMANDS') PERCENT (HD='% OF TOTAL' PF=G10) SUM(TOTAL-IO) (HD='TOTAL IO') PCT(TOTAL-IO) (HD='% OF TOTAL' PF=G10) MEAN(TOTAL-IO) (HD='AVG IO') MAX(TOTAL-IO) (HD='MAX IO') MEAN(DURATION) (HD='AVG DUR') MAX(DURATION) (HD='MAX DUR') MEAN(ENQ-TIME) (HD='AVG ENQ') MAX(ENQ-TIME) (HD='MAX ENQ') BY USER-ID WHERE USER-ID(2) = 'CW' INSIGHT-LINES = 30 PRINT TO STRESS01;
May 2004 Dieter W. Storr -- [email protected]
34
USER ADABAS ID COMMANDS % OF TOTAL -------- ---------- ----------- CW0016 269 |====== CW0017 269 |====== CW0018 269 |====== CW0019 369 |========= CW0020 369 |========= CW0021 369 |========= CW0022 369 |========= CW0023 369 |========= CW0024 269 |====== CW0025 269 |====== ---------- ----------- ======== 3,190 |========>
AVG DUR SEC MAX DUR SEC------------ --------- 0.00033 0.006 0.00052 0.008 0.00036 0.005 0.00058 0.014 0.00041 0.011 0.00043 0.008 0.00045 0.008 0.00038 0.007 0.00034 0.006 0.00049 0.009 ------------ --------- 0.014
ADABAS on Enterprise ServerCSS Stress Test
5 Users 10 Clicks25 Sep 2003, 11:52-11:53
May 2004 Dieter W. Storr -- [email protected]
35
What Tools Can You Use to Test?
APAS/INSIGHT ADABAS
0090 DWSCWOBX: EXTRACT (LS=150) 0100 NAT-LOAD-COUNT (HD='OBJ-LOAD') 0110 DATE TIME 0120 NAT-PROGRAM CMD 0130 USER-ID 0140 GLOBAL-ID 0150 WHERE USER-ID(2) = 'CW' AND 0160 (NAT-PROGRAM = 'IWS00P00' OR 0170 NAT-PROGRAM = 'IWD10P10') 0180 INSIGHT-LINES = 200 0190 PRINT TO STRESS01 0200 ;
May 2004 Dieter W. Storr -- [email protected]
36
NATURAL PROGRAM OBJ-LOAD -------- -------- IWD10P10 50 -------- 50
ADABAS on Enterprise ServerCSS Stress Test
5 Users 10 Clicks25 Sep 2003, 11:52-11:53
May 2004 Dieter W. Storr -- [email protected]
37
Response Time / Click Time Tests
ADABAS
NATURAL
Appl. Server
Web Server
Web BrowserFunction:
Display my account
200ms - 20s
0.00033s - 0.014s
May 2004 Dieter W. Storr -- [email protected]
38
What Tools Can You Use to Test?
STRESS -- self-writtenADABAS
1 Start process with heavy CPU workload 2 Start process with many ADABAS Calls 3 Start process with heavy CPU workload and many ADABAS calls 4 Start process with many ADABAS calls and excessive terminal I/O's 5 Start asynchronous processes (ADABAS & CPU)
May 2004 Dieter W. Storr -- [email protected]
39
May 2004 Dieter W. Storr -- [email protected]
40
May 2004 Dieter W. Storr -- [email protected]
41
May 2004 Dieter W. Storr -- [email protected]
42
May 2004 Dieter W. Storr -- [email protected]
43
May 2004 Dieter W. Storr -- [email protected]
44
What Tools Can You Use to Test?
Com-pleteUCTRL NETSPYTMON/MVS
TP Monitor
May 2004 Dieter W. Storr -- [email protected]
45
UCTRL17:49:14 TID 6 COMTEST USER ZINDWS 03/16/04 --- COM-PLETE CONTROL FACILITY --- UCT1
Function FC Function FC Function FC-------------------- -- -------------------- -- -------------------- --ADABAS interface ... AI Program lookaside .. IP Task groups ........ PGADABAS statistics .. AS Program in-stor dir. OP Task list .......... PLBuffer activities .. BA Resident programs .. RP Thread groups ...... TGCommon storage ..... CS Find program ....... FP Thread subgroups ... SGDB2 thread status .. D2 Queue overview ..... QO Thread list ........ TLFBPM statistics .... FB Roll activities .... RA User activities .... UAMessage log ........ CM Server overview .... SO User status......... US Terminal overview .. TO VSAM statistics .... VS Terminal status .... TS Print statistics ... PR
Select function .... Operand ............
Operator command:
Enter-PF1---PF2---PF3---PF4---PF5---PF6---PF7---PF8---PF9---PF10--PF11--PF12--- End
May 2004 Dieter W. Storr -- [email protected]
46
NETSPY Time Host Avg-Resp User Avg-Resp Worst Host_Resp No. Session No. Input No. Output
13:30 0.1 0.1 10.9 14 3777 399613:45 0.0 0.0 0.9 4 749 78014:00 0.1 0.1 5.7 25 10342 1102914:30 0.1 0.1 4.1 2 2094 2114
Netspy Response Time
0.0
2.0
4.0
6.0
8.0
10.0
12.0
13:30 13:45 14:00 14:30
Test Time
Re
spo
nse
Tim
e i
n S
eco
nd
s
Host Avg-Resp
User Avg-Resp
Worst Host_Resp
May 2004 Dieter W. Storr -- [email protected]
47
TP Monitor Com-plete Consumption
CPU TOTAL CPU % Paging Rate IO Rate19.31 1.60 0.00 26.50
COMTEST EXECUTION DETAIL
0.00
5.00
10.00
15.00
20.00
25.00
30.00
CPU TOTAL CPU % Paging Rate IO Rate
CATEGORY
VA
LU
E
Series1
May 2004 Dieter W. Storr -- [email protected]
48
TMON
Alerts Average response time CPU usage I/O usage Number of ended transactions rate
May 2004 Dieter W. Storr -- [email protected]
49
What Tools Can You Use to Test?
Attachmate
Run Time ApplicationManagement & Control Services (MSC) --
monitoring pools and sessionsJava console running in foreground
Screen Scraper
May 2004 Dieter W. Storr -- [email protected]
50
What Tools Can You Use to Test?
Attachmate Run Time Application
Number of sessions to host (mainframe)
Maximum sessionsNumber of failuresQueue size
Screen Scraper
May 2004 Dieter W. Storr -- [email protected]
51
What Tools Can You Use to Test?
Attachmate ServersNT utilities Sampling at 10-second
intervals
Screen Scraper
May 2004 Dieter W. Storr -- [email protected]
52
What Tools Can You Use to Test?
Attachmate Servers
Analyze results:CPU utilizationMemory utilizationRun queuesSwap space utilizationDisk I/O - most busy which disk hit the hardest I/O wait > 10%Page file - swaps
Screen Scraper
May 2004 Dieter W. Storr -- [email protected]
53
What Tools Can You Use to Test?
Cisco 2900 SwitchManual testing/sampling
Analyze resultsLoad on network componentsPacket throughput
Content Server Switch
May 2004 Dieter W. Storr -- [email protected]
54
What Tools Can You Use to Test?
Cisco 4000 Router (10BaseT)Manual testing/sampling
Analyze resultsLoad on network components
http://www.scd.ucar.edu/nets/devices/routers/ios_analysis.html
Packet throughputRouter Configuration:
http://www.swcp.com/~jgentry/topo/cisco.htm
Router
May 2004 Dieter W. Storr -- [email protected]
55
Dedicated line broadband speedsand prices
T-1 - 1.544 megabits per second (24 DS0 lines)Ave. cost $400.-$650./mo.
T-3 - 43.232 megabits per second (28 T1s)Ave. cost $6,000.-$16,000./mo.
OC-3 - 155 megabits per second (100 T1s)Ave. cost $20,000.-$45,000./mo.
OC-12 - 622 megabits per second (4 OC3s) no price
OC-48 - 2.5 gigabits per seconds (4 OC12s) no price
OC-192 - 9.6 gigabits per second (4 OC48s) no priceSource: http://www.infobahn.com/research-information.htmprices updated: 16 March 2004
May 2004 Dieter W. Storr -- [email protected]
56
What Tools Can You Use to Test?
OracleOracle utilitiesApplication-specific log filesPerformance Monitoring and Tuning Tools
• TKProf• UTLBSTAT.SQL and UTLESTAT.SQL - Begin and end
stats monitoring
• Statspack• Oracle Enterprise Manager - Tuning Pack
RDBMS
May 2004 Dieter W. Storr -- [email protected]
57
What Tools Can You Use to Test?
OraclePerformance Monitoring and
Tuning Tools (cont”)• DBXRAY: A real-time graphical snapshot of the health of your
Oracle database- anytime, anywhere.• Patrol: Intelligent and non-intrusive for proactive database
monitoring, including Oracle Parallel Server and Oracle Symmetric Replication environments.
• SQL-Explorer for Oracle The most advanced SQL tuning tool available. Identify, model and tune problematic SQL
statements to boost application performance.
RDBMS
May 2004 Dieter W. Storr -- [email protected]
58
What Tools Can You Use to Test?
Oracle
Analyze results:Number of Oracle processesBuffer hit cacheBlock I/O rateNumber of cursors createdNumber of connectionsLog information regarding number of
registrations, starts, etc.
RDBMS
May 2004 Dieter W. Storr -- [email protected]
59
What Tools Can You Use to Test?
SSL Accelerator• Secure Socket Layer (protocol between
HTTP and TCP) -- appliance or card
Manual testing/sampling
Analyze results:Load on network componentsPacket throughput
Optimizing SSL processing for Web http://www.nwfusion.com/news/tech/2000/1023tech.html?nf
SSL
May 2004 Dieter W. Storr -- [email protected]
60
What Tools Can You Use to Test?
PIXManual testing/sampling
Analyze results:Load on network componentsPacket throughput
Firewall
May 2004 Dieter W. Storr -- [email protected]
61
What Tools Can You Use to Test?
Gauntlet (several)Manual testing/sampling
Analyze results:MemoryDisk I/OFull system log
Firewall
May 2004 Dieter W. Storr -- [email protected]
62
What Tools Can You Use to Test?
Sun OSSolaris utilitiesSampling at 10-second intervals
Web Server
May 2004 Dieter W. Storr -- [email protected]
63
What Tools Can You Use to Test?
Sun OS
Analyze results:CPU utilizationMemory utilizationRun queuesSwap space utilizationDisk I/O -- most busy which disk hit the hardest I/O wait > 10%Page file -- swaps
Web Server
May 2004 Dieter W. Storr -- [email protected]
64
What Tools Can You Use to Test?
Apache ApplicationLog Analyzer
Analyze results:Number of sessionsBytes per sessionSource addressDestination address
Web Server
May 2004 Dieter W. Storr -- [email protected]
65
What Tools Can You Use to Test?
Sun OSSolaris utilitiesSampling at 10-second
intervals
Application Server
May 2004 Dieter W. Storr -- [email protected]
66
What Tools Can You Use to Test?
Sun OS
Analyze results:CPU utilizationMemory utilizationRun queuesSwap space utilizationDisk I/O -- most busy which disk hit the
hardest I/O wait > 10%Page file -- swaps
Application Server
May 2004 Dieter W. Storr -- [email protected]
67
What Tools Can You Use to Test?
BEA WebLogic ApplicationWebLogic administrative consoleSpotlight
Application Server
May 2004 Dieter W. Storr -- [email protected]
68
What Tools Can You Use to Test?
BEA WebLogic Application
Analyze results:Number of active queuesNumber of connectionsNumber of active socketsTypes of accessTime of access
Application Server
May 2004 Dieter W. Storr -- [email protected]
69
Web Site PerformanceAnalysis and Tuning
In-House or Web Caching Near the network edge, e.g. proxy cache or
browser: static HTML, JavaScript, and image files Cache-control header “no-cache” “re-fetch”
“expiry date”
Compression HTML is compressed before being sent through
the network.
May 2004 Dieter W. Storr -- [email protected]
70
Web Site PerformanceAnalysis and Tuning
SSL Acceleration (box or card) Consider the number of transactions per second
(TPS). TPS will be more important on SSL Acceleration appliances than cards.
An SSL Acceleration hardware appliance (box) can deal with transactions for all your web servers and scale quickly.
Network Load Balancing Round Robin Lowest response time Least connections Proximity
May 2004 Dieter W. Storr -- [email protected]
71
Checklist -- Getting Started?Determine your environment
• Mainframe, Open System, Middleware
Evaluate Stress Test Software• Write your own or buy one
Understand the different components of a web site download
• DNS lookup• Redirect• First byte download• Base page download• Content download
May 2004 Dieter W. Storr -- [email protected]
72
Checklist -- Getting Started?What kind of content?
• Dynamic or static• .html or image
Number of serversWhere does traffic comes from
• Internet, Intranet, or Extranet
May 2004 Dieter W. Storr -- [email protected]
73
Create a Stress Test Plan
Test Organization• Overview• Participants
• Technical team• Coordination team• Application team
• Objectives• Infrastructure• Application
• Assumptions / pretest conditions• Communication plan
May 2004 Dieter W. Storr -- [email protected]
74
Create a Stress Test Plan
Environment / Test Components• Overview• Overall success criteria• Transactions to be tested
• Logical and physical• Application functions
• Data indentification• Script development• High-level system diagram• Components
• Responsibility• Metrics• Tool / method
May 2004 Dieter W. Storr -- [email protected]
75
Create a Stress Test Plan
Sequence of Events• Summary• Process• Schedule of events
May 2004 Dieter W. Storr -- [email protected]
77
Links to Interesting Websites
http://www.edwardtsai.com/web-site-performance.html
http://www.nwfusion.com/news/tech/2000/1023tech.html?nf
http://www.scd.ucar.edu/nets/devices/routers/ios_analysis.html
http://www.swcp.com/~jgentry/topo/cisco.htm