Upload
others
View
10
Download
0
Embed Size (px)
Citation preview
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Introduction to mobile performance testing Igor Markov, Shlomi Zalma, Shane Evans, Wilson Mar June 11 & 13, 2013
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
HP LoadRunner components
Virtualize the Network 4
Stay Connected
Analysis, Diagnostics and Reporting
Database Internet/WAN Application Server(s) Client(s) End User
Test Scenario
Measure Impact
6
Generate Load 3 Author Scripts 1
Plan and Execute Tests 2 HP Live Network 7
5
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4
Incremental integrated development workflow
Deploy app
Run test cases
Provision devices
Execute build
Application Security Assessment
Test results
Check-in & Code Scan Fortify Source Code Analysis
UFT / LoadRunner
WebInspect
ALM Lab Management
Cloud Delivery Automation
Jenkins
HP integrated suite
LR Analysis
Executive Scorecards
Analytics Portal
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
Workflow for performance engineering
1. Plan 2. Provision 3. Verify 4. Script 5. Scenarios
Workload Templates Data Scenarios Attributes Cost vs.
Benefit
Domains Servers Ports IP Addresses SSL certs. Licenses
Batch Jobs Network
Variability Report
Samples
Transactions Resource
Usage Variations Retry and
Recovery
Report Templates
Reruns Baselines Comparisons Archival
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
Create a new script, mobile protocols
HP LoadRunner VuGen
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Author mobile TruClient scripts
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
Recording methods
TruClient Ajax
HTTP/HTML
HP LoadRunner VuGen
App web server
User Agent >
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Create a new script, mobile TruClient protocol
HP LoadRunner VuGen
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
TruClient recording
HP LoadRunner VuGen 11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
Devices with User Agent strings
Count: 27
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12
Sample User Agent for Samsung Galaxy S3
User Agent string example
Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
720 x 1280
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
Invoke Firefox in VuGen for TruClient
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML mobile recording
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Create a new script, HTML recording protocol
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17
HTTP/HTML
TruClient
HTTP/HTML recording methods wizard
Proxy Recording
Record Emulator
Analyze Traffic
Record & Analyze Traffic
TruClient Ajax HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Record emulator recording method
Android
Device Emulator
HTTP/HTML
Record Emulator
APK
APK
TruClient Ajax HP LoadRunner VuGen11.5x
App web server
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
Record emulator requirements
Java Development Kit http://www.oracle.com/technetwork/java/javase/downloads/index.html
Android SDK with ADT http://developer.android.com/sdk/index.html
HTTP/HTML
Record Emulator
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Record emulator
HP LoadRunner VuGen11.52
HTTP/HTML
Record Emulator
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
Record emulator options
HP LoadRunner VuGen11.52
HTTP/HTML
Record Emulator
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22
Proxy recording
Android
LoadRunner Proxy
iOS
Port 8080
HTTP/HTML
Proxy Recording
Port 80/443
APK
HP LoadRunner VuGen11.5x
App web server
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23
Proxy recording
HP LoadRunner VuGen11.52
HTTP/HTML
Proxy Recording
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24
Record and analyze traffic
Android
Wireshark
iOS
HTTP/HTML
Record & Analyze Traffic
Ad hoc network
APK .pcap
HP LoadRunner VuGen11.5x
App web server
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25
Record and analyze traffic
HP LoadRunner VuGen11.52
HTTP/HTML
Record & Analyze Traffic
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26
Record and analyze traffic
HP LoadRunner VuGen11.52
HTTP/HTML
Record & Analyze Traffic
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27
Analyze traffic
Mobile Recorder
Android
.lrpcap
HTTP/HTML
Analyze Traffic
Android 4.1+ (rooted)
App web server
APK
HP LoadRunner VuGen11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28
Analyze traffic
HP LoadRunner VuGen11.52
HTTP/HTML
Analyze Traffic
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29
Analyze traffic—options
HP LoadRunner VuGen11.52
HTTP/HTML
Analyze Traffic
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Scenario runs
https://connections.houston.hp.com/docs/DOC-15086
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32
Concurrency and rendezvous
[P10]
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33
Wi-Fi vs. 3G response time
[P01] HP Network Virtualization by Shunra
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34
3G vs. 4G latency spikes
HP Network Virtualization by Shunra
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35
http://www.youtube.com/watch?v=7CGCief0EaU&hd=1&t=19m35s
Bandwidth Utilization In (Down)
Bandwidth Utilization Out (Up)
Response Time
Login with more VUsers
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 36
Ramp up and down
Resources
• Processes, Threads
• Memory (leaks?)
• CPU (saturation?)
• Disk I/O (thrashing?)
• Network I/O
• Transaction Throughput
• Etc.
[P04]
Response time
Resources per user
Clean-up Resources over time
Back-end Resources
HP LoadRunner Analysis 11.52
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37
Capacity in 3 dimensions
[P04]
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 38
Scalability
[P01] http://wwwperftest.com
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 39
Isolate end-user performance by carrier
HP Business Service Management
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 40
Isolate end-user performance by device
HP Business Service Management
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you