QUALITY OF SERVICE FOR CLOUD-BASED MOBILE APPS: Aruba Networks AP-135 and Cisco AP3602i Conducted at the Aruba Proof-of-Concept Lab November 2012

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 2

Statement of test result confidence • Aruba makes every attempt to optimize all vendors for performance and follow best

practices for configuration as published by the vendor.

• Aruba makes every attempt to make a fair and apples-to-apples comparison, including AP mounting position, client location, transmit power, channel and the latest shipping firmware.

• Aruba ensures the test bed environment to be free of any interference sources. Also, the 802.11 Wi-Fi channels configured are ensured to be consistent when testing 2.4 GHz and 5 GHz bands for all vendors.

• Aruba believes the test results are both repeatable and reproducible in similar testing environments.

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 3

Table of Contents Statement of test result confidence ........................................................................................ 2  Table of Contents ...................................................................................................................... 3  Executive summary .................................................................................................................. 4  

Test bed setup .................................................................................................................................... 6  Scenario 1: Mobile app performance at high-density ........................................................................ 7  Scenario 2: Mobile app performance with higher QoS ..................................................................... 8  

About Aruba Networks, Inc. ................................................................................................... 10  Appendix .................................................................................................................................. 16

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 4

Executive summary Today’s enterprise access networks are witnessing three key trends – BYOD, cloud based applications and ubiquitous Wi-Fi. Newer generations of smartphones and tablets are rapidly introduced in the market with greater processing power. Users acquire these devices and bring them to work while also bringing in a mixture of applications with them. Some of these are corporate applications, while some applications are personal. With every application having its own set of requirements for latency and bandwidth, it’s a daunting task for IT to ensure quality of service. This report benchmarks specific use cases where a corporate application is protected in a congested network by giving it special preferential treatment. The first test scenario measures the amount of time it takes a single user to download a large file from the cloud while increasing load on the access point to 15 devices. The second scenario takes it a step further where one user is downloading a file from a corporate cloud storage server while the other users are using personal storage applications. Here the goal is to show how application performance can be guaranteed in spite of network congestion. As part of the benchmarking process, the tests were run and compared with a Cisco WLAN solution. The key highlights from the testing are as follows. Table 1: Test results summary

Test Scenario Description Aruba Cisco Aruba Advantage

Corporate cloud application with background traffic

Amount of time to download a large file with increasing

load (up to 15 devices) and background traffic (seconds) 16s 116s 7x faster

Corporate cloud application

with preferred access

Amount of time to download a large file with preferential

treatment for corporate application and best effort for

personal applications (seconds) 12s 129s

11x faster

These test results indicate a significant variation in performance between Aruba and Cisco Wi-Fi products. Customers are advised to exercise their own judgment before making a vendor decision for their Wi-Fi network. The rest of the document provides comprehensive details of the test cases, test bed setup, observations and results collected.

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 5

Test environment Table 2: Devices under test

Vendor Device Quantity Firmware version

Aruba

AP-135 1 ArubaOS 6.1.3.4

3600 Mobility Controller 1 ArubaOS 6.1.3.4

Cisco

AP-3602i 1 7.2.110

5508 WLAN Controller 1 7.2.110

The following table shows the detailed information on various network components that were part of the infrastructure used for performance tests. Table 3: Test equipment Item Component Specifications Details

1

Device OS/Type Make and Model Quantity OS Version

Laptops

MacBook Pro 3 10.7.3

Dell Latitude 2 Windows 7

Tablets iPad 5 5.1.1

Smartphones iPhone 5 5.0.1

2 Cloud storage apps

www.box.com

www.dropbox.com

www.amazon.com/clouddrive

docs.google.com

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 6

Test bed setup Figure 1 below shows the logical network topology of the test bed. All network components were connected at Layer 2 using the Aruba S3500 Mobility Access Switch. The switch performed DHCP services providing IP addresses to all the devices. Aruba’s ClearPass Policy Manager provided RADIUS authentication services for the Wi-Fi clients.

Figure 1: Test bed setup

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 7

Note: A dedicated high-speed WAN link was used for all test cases that involved downloading a file from the Internet. Periodic speed tests were run to make sure that WAN link was not the bottleneck for any of the testing. A sample screenshot from speedtest.net is captured below.

Test scenarios Based on customer demand, the following real-world scenarios were benchmarked.

Scenario 1: Mobile app performance at high-density This test measures the amount of time it takes a single user to download a large file from the cloud while increasing the total number of active mobile devices on the access point from five to fifteen. This test captures the effects of congestion on a business critical application like Box by simulating real world background traffic using Ixia Chariot traffic generator. The file size used for download is 33MB and the device used for download is a 450 Mbps capable laptop. Tests were conducted in the 2.4-GHz and 5.0-GHz bands. The tests in 2.4 GHz had 15 devices (five laptops, five smartphones and five tablets) while the 5.0-GHz test was restricted to 10 devices (five laptops and five tablets) since the smartphones do not support 5.0 GHz. As you see from the charts, when there is no background traffic and just one client (baseline) both Aruba and Cisco perform the same. As contention increases with more clients, the performance for Aruba WLAN stays consistent and predictable while it takes longer and longer to download the same file on the Cisco WLAN, with Aruba performing up to seven-times faster.

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 8

Scenario 2: Mobile app performance with higher QoS The second scenario takes it a step further where all users are downloading the same file from different cloud storage services; one of them being the corporate Box service. The personal cloud storage applications used are Dropbox, Amazon Cloud and Google Docs. Here the Aruba wireless LAN is configured to assign higher quality of service to Box while the rest of the mobile apps are assigned best-effort quality. The test was repeated with both 2.4 GHz and 5.0 GHz. The time taken to download the file was measured on both Aruba and Cisco wireless LANs.

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 9

Table 4: Corporate mobile app (Box) performance

2.4 GHz Box Dropbox Amazon Drive Google Docs

Aruba 11.7 136.3 113 98.7

Cisco 129 95.3 99 83

5 GHz Box Dropbox Amazon Drive Google Docs

Aruba 28.0 217.0 280.0 281.3

Cisco 107.3 107.3 118.3 116.3

In a Cisco WLAN, there is no ability to provide quality of service per each of the cloud based applications, severely impacting the performance of the Box application. The same file was downloaded up to 11 times faster on Aruba Wi-Fi. What do the tests reveal? In today’s congested wireless network, with hundreds of devices running both corporate and personal apps, it’s critical for the network infrastructure to be able to distinguish between different mobile apps and provide preferential treatment to the ones that are most important. Without preferential access, IT organizations run the risk of lost productivity and dissatisfied end users. With increasing adoption of mobile devices, we have more demanding data applications – they are no longer on-demand thanks to cloud based storage apps such as iCloud, Dropbox, Box, Amazon Cloud, Google Docs among others. Some of these applications automatically download and upload data, even when the mobile device is in your pocket or in your briefcase. And in addition, they consume as much bandwidth as they can from the air since by default they are designed to use as much bandwidth as possible. Due to presence of such applications, many of the Wi-Fi deployments today report as much upload traffic as download traffic. Given that Wi-Fi bandwidth is shared among upload and download traffic, this creates additional stress on the wireless networks as the mobile device count continues to increase. Aruba Wi-Fi provides IT managers the capability to prioritize mission critical mobile apps different from the rest thereby ensuring availability and predictable performance over the air.

QoS for Cloud-Based Mobile Apps Benchmark Testing: Aruba AP-135 and Cisco AP3602i

Aruba Networks, Inc. 10

About Aruba Networks, Inc. Aruba Networks is a leading provider of next-generation network access solutions for the mobile enterprise. The company’s Mobile Virtual Enterprise (MOVE) architecture unifies wired and wireless network infrastructures into one seamless access solution for corporate headquarters, mobile business professionals, remote workers and guests. This unified approach to access networks enables IT organizations and users to securely address the Bring Your Own Device (BYOD) phenomenon, dramatically improving productivity and lowering capital and operational costs. Listed on the NASDAQ and Russell 2000® Index, Aruba is based in Sunnyvale, California, and has operations throughout the Americas, Europe, Middle East, Africa and Asia Pacific regions. To learn more, visit Aruba at http://www.arubanetworks.com. For real-time news updates follow Aruba on Twitter and Facebook.

© 2012 Aruba Networks, Inc. Aruba Networks’ trademarks include AirWave®, Aruba Networks®, Aruba Wireless Networks®, the registered Aruba the Mobile Edge Company logo, Aruba Mobility Management System®, Mobile Edge Architecture®, People Move. Networks Must Follow®, RFProtect®, and Green Island®. All rights reserved. All other trademarks are the property of their respective owners. TR_CloudApps_111412