Upload
rtts
View
2.645
Download
0
Embed Size (px)
Citation preview
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Performance Testingfor Mobile Applications
Moderator
Laura PoggiMarketing Manager
Matthew AdcockDivision Manager, Performance & Scalability
Presenter
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
The Software Quality Experts
1996 Founded
Locations New York | Philadelphia | Atlanta | Phoenix
400 + Projects
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
What we offer
Extensive knowledge of the leading test tools
Unmatched experts for both strategy and implementation
Innovative services for all project sizes and lengths
1
2
3
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Objectives• Review the current mobile load testing ecosystem & challenges.• Provide paths for interim load testing solutions for mobile
applications.• Limit additional software costs related to mobile load testing
requirements.• Enable collaboration amongst peers.
"Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime."
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Load Testing in Our ContextHTTP
HTTP
Resource Monitoring
Test Agent
Virtual Users
Test CoordinatorTest Agent
Test Results
Virtual Users
Web Application
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
What’s Happening?• Mobile usage is taking off. Consumers have high expectations on
usability and performance.• Consumers are using tablets and smart phones; i.e. there are many
platforms to support.• The technology is evolving rapidly; i.e. HTML5 vs. Native Apps.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
There are Billions of Installs
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
What’s the Future of Mobile?The future is complex:
• HTML5• Single codebase running across many platforms using a web browser.• Inherent maintenance benefits & upgrade pathways.• Possibly shorter time to market.
• Native Apps• Multiple codebases each targeting a specific platform.• Monetary incentives for software developers.• Increased brand management.• Sophisticated user interface.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Testing is Always Catching Up
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Mobile Load Testing Future?
Business Challenges+ Technical Challenges Bumpy Ride
• Mobile load testing is nascent; i.e. we’re in an early phase.
• There’s a myriad of choices and/or vendors cropping up weekly.
• Technology is changing rapidly.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Business Challenges• Load test tool licensing constraints and/or budget limitations.• Load test tool procurement lengthy and/or cumbersome.• Accelerating time-to-market; i.e. “hurry-up and test”.• A plethora of target devices & operating systems to support.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Technical Challenges• Mobile architecture not supported out the box.• Limited features of load testing tool; i.e. not keeping pace.• Creating scripts may be unorthodox and/or unclear.• Emulating large numbers of concurrent users may be required.• Incorporating network conditions becomes important.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Some Existing Tools
Commercial w/ Traditional Install Hosted/SAAS Open Source
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Short-Term Solutions• Scrutinize the mobile architecture; know the pain points and
challenges.• Be creative and use what you have to limit costs related to tool
transition, vendor selection, and/or constraints within your organization.
• Leverage open source technology.• Take advantage of free utilities and web-based services.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Our FocusPrimary Focus
− Creating automated test scripts for mobile applications.Honorable Mentions
− Emulating network conditions of mobile devices.− Scaling up load generation to support increasing consumption of on-demand
services.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Test Automation Challenges• Many load test tools run on desktops NOT mobile devices.• Mobile network traffic occurs on a Wi-Fi or Cellular network.
Desktop Web Application
LAN LAN
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Test Automation Solutions1. Spoof mobile device from a desktop web
browser.2. Proxy recording using actual mobile
device.3. Packet captures (i.e. PCAP) using actual
mobile device.
Script Capture
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Spoofing Mobile DevicesUse Case: Applicable if the target application runs within a mobile web browser. This can’t be used for native apps.
Strategy• Most web browsers can be modified to appear to be running on a mobile
device.• With the web browser configured to imitate a mobile device, script recording
can occur from the desktop where the load testing tool is installed.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Spoofing Relies on Metadata• HTTP clients must use standardized metadata in the header of a HTTP
request to indicate the client’s OS, browser type and version, along with extensions.
• The “User-Agent” field is the key to this solution.• Many web applications use this metadata to redirect customers to
mobile-specific URLs and/or mobile-specific content.
Desktop Web Application
HTML, CSS, JPEG JSON, XML
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
User-Agent Metadata
GET http://www.gasbuddy.com/ HTTP/1.1Host: www.gasbuddy.comUser-Agent: Mozilla/5.0 (Windows NT 5.1; rv:18.0) Gecko/20100101 Firefox/18.0Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Language: en-US,en;q=0.5Accept-Encoding: gzip, deflateConnection: keep-alive
Mozilla/[version] ([system and browser information of client]) [browser platform] ([browser platform details]) [extensions]
Example:
The User-Agent is composed of the following tokens:
Desktop
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
User-Agent for Windows XPGET http://www.gasbuddy.com/ HTTP/1.1Host: www.gasbuddy.comUser-Agent: Mozilla/5.0 (Windows NT 5.1; rv:18.0) Gecko/20100101 Firefox/18.0Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Language: en-US,en;q=0.5Accept-Encoding: gzip, deflateConnection: keep-alive
Desktop
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
User-Agent Glossary
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
User-Agent for Google NexusGET http://gasbuddy.com/ HTTP/1.1Host: gasbuddy.comConnection: keep-aliveUser-Agent: Mozilla/5.0 (Linux; Android 4.2.2; Nexus 7 Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.166 Safari/535.19Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8Accept-Encoding: gzip,deflate,sdchAccept-Language: en-US,en;q=0.8Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
User-Agent Makes a Difference
Desktop
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Modifying the Web BrowserCapabilities differ by web browser and version:
• Firefox has configuration web page.• Earlier versions of Internet Explorer required Add-ons; latter versions have
built-in features using its Developer Tools menu item.• Google Chrome has Developer Tools; earlier version required command-line.• Safari as a Develop menu bar that can toggle the User-Agent.
Detailed Instructions:http://www.howtogeek.com/113439/how-to-change-your-browsers-user-agent-without-installing-any-extensions/
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Make Firefox an iPad
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Firefox Meets iPad• Customer goes to the GasBuddy home page at
http://www.gasbuddy.com/.• The GasBuddy application detects that the
browser session emanates from a mobile device and redirects the web browser to the mobile version of the web site.
• The mobile functionality is now accessible within a desktop browser, such as location aware browsing.
Begin test script capture from the load testing tool…
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
User-Agent Spoofing CaveatSome desktop web browsers may not behave the same as a mobile web browser…
− Application may require device-specific information, such as GPS.
− UI may want to use location aware browsing
There are other approaches if this is the case…
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Proxy RecordingUse Case: Applicable for browser-based applications on mobile devices and native apps. The mobile device and load testing tool desktop must use Wi-Fi.Strategy
• “Point” the client’s user interface to an intermediate proxy running on your desktop.
• Some load testing tools can capture HTTP traffic directly by its recording mechanism.
• Ancillary utilities can capture the HTTP traffic, if the load testing tool is not capable to so.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Proxy Recording Steps
Desktop
Web Application
LAN
1. Set the Wi-Fi settings on the mobile device to use a HTTP Proxy.
2. Launch the load testing tool recording mechanism or launch the web proxy utility on your desktop.
3. Navigate through the mobile application; either via a web browser or native app.
4. After recording work with the automated test script or transform the captured conversation into the format for the load testing tool.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Enabling a Proxy for a Device• Proxy settings are found within the Wi-Fi options of the SSID.• Access to the settings is dependent upon the device.• Enter the Wi-Fi IP address and port of the web proxy or test tool.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
JMeter Proxy Recording• JMeter uses a web proxy for script capture.• The proxy runs on the desktop on user-defined port number.• The mobile device needs to “point” to this desktop’s IP address and port
number to be captured by JMeter.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
JMeter Proxy Recording
DesktopWeb
Application
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Example: JMeter Test Plan
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Fiddler Proxy RecordingWhat happens if the load testing tools don’t support proxy recording?
• Use an ancillary tool (that’s free) to capture the HTTP conversation.• Manually transform the HTTP conversation into an automated test script.• Programmatically convert the HTTP conversation into an automated test
script.
http://www.fiddler2.com/fiddler2/
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Fiddler Setup
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Fiddler Proxy Recording
DesktopWeb
Application
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Fiddler Session• Captures all HTTP requests.• The raw details of each HTTP
request are stored.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Fiddler Session to Script• Manually edit load test tool script with Fiddler Session data. Or• Use Fiddler Plug-ins to automate the transformation process.
Support for Visual Studio Web Tests already exists. Custom formats can be created using any .NET language.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Packet Capture (PCAP)Refers to the act of capturing network traffic, as well as a well-defined structure for saving and manipulating the data.
• Libraries exist for both UNIX-like systems (libcap) and Windows systems (WinPcap); i.e. network capture is not tied to your Windows desktop and can even be performed on the mobile device.
• Network captures can be enabled via tools, such as tcpdump, WireShark, and Microsoft Network Monitor (all are free!).
• There is an API built in C. There are other programming languages that have a wrapper API built; i.e. network captures can be programmatically manipulated for our scripting purposes.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Packet Capture (PCAP)Use Case: Applicable for browser-based applications on mobile devices and native apps. You only need access to the PCAP traces; i.e. the mobile device and your desktop do not need to be coupled.
Strategies• Setup a Wi-Fi hotspot on a desktop computer with a packet capture
mechanism. Point the mobile device to the hotspot. You can also use the Wi-Fi proxy strategy that was discussed previously.
• Enable PCAP capture on mobile device. This will require the mobile device to be rooted.
• Plug packet capture device into network switch that will eventually “see” the mobile traffic.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
PCAP Using Wireshark
DesktopWeb
Application
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Using Wireshark• Ability to open existing captures,
filter conversations, and view packet details.
• Ability to launch new packet captures.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Starting a Wireshark Capture• You need to pick the network
interface to capture packets off of.
• We need to make sure that the wireless network interface is used, if we are using the Wi-Fi hotspot or web proxy setup.
• Start and Stop the capture when appropriate.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
A Wireshark Capture
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Filtering a Wireshark Capture
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
What’s in Wireshark Capture?
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
There is PCAP in the “Cloud”
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
What Can We Do?• Wirehark provides useful views
into the HTTP packets.• This information can be manually
transformed into an automated test script using any load testing tool.
• The transformation can be automated, if you are comfortable with programming.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
PCAP ProgrammingPCAP traces can be programmatically parsed. We can use this functionality to automate the test script transformation process.
• Pcap.NET: http://pcapdotnet.codeplex.com/• jNetPcap: http://jnetpcap.com/• dpkt: http://code.google.com/p/dpkt/• node-pcap-parser:
https://github.com/nearinfinity/node-pcap-parser/blob/master/test/pcap-parser-test.js
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
PCAP Programming Examples
http://jnetpcap.com/tutorial/usage http://jon.oberheide.org/blog/2008/10/15/dpkt-tutorial-2-parsing-a-pcap-file/
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Test Automation Re-Cap1. Spoof mobile device from a desktop web
browser. Relies on modifying your browser’s User-Agent metadata.
2. Proxy recording using actual mobile device. Record directly into test tool or use web proxy debugger.
3. Packet captures (i.e. PCAP) using actual mobile device. Use packet capture to get HTTP contents and/or program a transformation solution.
Script Capture
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Emulating Network Conditions• Networks can influence the end-
user experience, as well as the scalability of the mobile application.
• Typical testing lab environments do not introduce network variability into the performance equation.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Emulating Network ConditionsCharacteristics to consider:
− Latency− Bandwidth− Packet Loss− Jitter
Challenges:• Not a lot of “accessible” options; i.e.
open source and/or turnkey.• Mostly commercial vendors.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Emulating Network Conditions
Network Emulator for Windows Toolkit
(NEWT)
Commercial Solutions Open Source Solutions
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Scaling Up Load GenerationTypical Requirements:
− Large numbers of users requires hardware to emulate.− Hardware may only be needed on-demand.− Requirement to geographically distribute virtual users.
Solutions:• Use cloud services to build out infrastructure.• Leverage self-service solutions w/ pre-configured infrastructure.• Nothing is “free” here.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Scaling Up Load Generation
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Moving Forward• We’re accessible for further discussions.• Potential for future webinars to continue our discussion.• Foster collaboration. We’ve created a LinkedIn User Group in order to
expand upon this discussion.
© 2013 Real-Time Technology Solutions, Inc.360 Lexington Ave. FL 9, New York, NY 10017www.rttsweb.com | (212) 240-9050
Thank you! Questions?
Linkedin GroupPerformance Testing- Mobile Applications
Matthew [email protected]
Websitewww.rttsweb.com
Call us(646) 329-9309