View
111
Download
1
Category
Preview:
Citation preview
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Network Virtualization (NV) Enablement – How to POCMatthew Brady – EMEA NV Pre-Sales15th May 2014
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
POC OverviewScoping (Discovery)ObjectivesPre-RequisitesHow to Select Network ScenariosWhat to CoverHow to TestHow to ReportFollow Up
Agenda
Content Intended as Aide Memoire/Checklist- Won’t Cover All Points- Questions/Feedback Throughout!
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Overview
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4
Proof of Concept
“A proof of concept (POC) … is a realization of a certain method or idea to demonstrate its feasibility, or a demonstration in principle, whose purpose is to verify that some concept or theory has the potential of being used. A proof of concept is usually small and may or may not be complete”
- Wikipedia
What is a POC?• Not a Trial (Self-Guided)• Not a Pilot (Actual Project Deliverables)• Limited in Scope and Length
• Low number of vusers • 2-3 days total time• Single Application• Limited Journeys
• Usually AssistedGive-Get?• Sponsor Required• Conditional Commitment/Access to Power• Final Results Presented to Power
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Scoping
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6
Overall• What Is The Name And Description Of The Application To Be Covered By The
POC?• Who Will Be Involved?• Where Will It Be Run, Using What Resources?
Architecture/Technology• What Is The Outline Architecture Of The Application (Overview Only Is
Required)?• Which Client Platforms Will Be Used (Windows/Android/Ios)? Which Client
Types Should Be Included In The POC Test Activities?• How Will It Be Delivered To Users (Web Browser, Client Application, Mobile
App Etc.) And Using Which Technologies (HTML5/JS/.Net/Java Etc.)?
Scoping Questions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7
Geography/Connectivity• Where Will It Be Hosted When In Production?• Who Will Be The End-users? Where Will They Be Located? How Will They Be
Connected?• Is The Client Communication Encrypted (Https)?• If Mobile Users, Which Operators And Connection Types (2.5g, 3g, 3.5g, 4g)?• If Web Users, Which Connection Types (Dialup, Dsl, Cable, Satellite, Other)?
Types Of Testing/Tools• Single-user Functional And/Or Multi-user Non-functional• Manual, UFT, LR/PC, Sprinter And/Or SV• Desktop, Web And/Or On-device
Scoping Questions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8
Scripting/Automation• Do LR Scripts Exist For The Application Already? If So, How Many Are There?
What Protocols Do They Use? Which Ones Will Be Selected For The POC?• If Lr Scripts Do Not Exist, When Will They Be Available? Which Protocols Will
They Use?• Which Lr Version Is Currently Installed? Will The POC Use This Environment
Or Will A New Environment Be Setup For The POC? What OS Is Used For LR Controllers/Load Generators?
Objectives• What Is The Test Objective?
• 5000 Concurrent Users, 100k Transaction Per Hour, Functional Tests?• How Will Test Success Be Determined (E.g. Response Times, Server CPU
Etc.)?• What Slas Or Other Requirements Exist?
Scoping Questions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Objectives
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10
NV With LR/PC• Record/Sample Remote Network Conditions• Configure And Apply Network Conditions In Tests, Including Additional
Reporting Capabilities• Analyse The Impact Of Remote Users On System Load• Analyse And Optimize The Application Flows For Performance
Success Criteria
Client Objectives – Multi User Testing
Criterion Status after POC
Install, license and configure HP NV software components CompleteDemonstrate HP Network Catcher to profile/sample mobile network conditions
Complete
Execute automated tests using HP NV for HP to apply network conditions
Complete
Demonstrate additional HP NV reporting and metrics CompleteDemonstrate using HP NV Analytics to optimize and improve performance of application flows
Complete
Provide comparison report and analyse comparative throughput of network virtualization in tests
Complete
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11
NV For Desktop/Mobile• Install, License And Deploy Of HP NV For Mobile• Record/Sample Mobile Network Conditions For UK Consumers• Apply Network Conditions, Time Transactions And Capture Traffic Flowing From
On-device Tests• Analyse And Optimize The Application Flows For Performance Including HTTPS
Decoding• Demonstrate Using API/Command Line To Control Network Conditions For
AutomationSuccess Criteria
Client Objectives – Single User Testing
Criterion Status after POC
Install, license and configure HP NV software components CompleteDemonstrate HP Network Catcher to profile/sample mobile network conditions
Complete
Execute manual and automated on-device tests using HP NV for Mobile to apply network conditions
Complete
Demonstrate additional HP NV reporting and metrics CompleteDemonstrate using HP NV Analytics to optimize and improve performance of application flows
Complete
Provide comparison report, and analyse comparative throughput of network virtualization in tests
Complete
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12
1. Ensure Successful POC• Keep it Simple, Well-defined Scope• Ensure S/W Pre-installed• Check & Verify Objectives, Prerequisites• Limit Time/Effort but Provide Maximum Results2. Demonstrate Maximum Value/Impact• Work Towards the Final Report – Not Client-Defined Schedule• Carefully Select Network Test Scenarios – Include Worst Cases• Opt For Poorer Network Conditions3. Progress Sales Process• Agree Success Criteria Before POC• Demonstrate Compliance to Success Criteria• Provide Detailed Report With Executive Summary• Present Results to Executive Team
HP Objectives?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Prerequisites
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14
Environment• Identify Testing Servers
• Controller(s), Load-Gens• Scripting Workstation• Testing Workstation
• Check Admin Rights• Check Server Specification/OS Version etc.
Resources• When can the Application be tested? Will it remain unchanged through the
POC?• Required Additional Software (Java, Silverlight)• Internet Connectivity (for NC Global Library etc.)• Software Download(s)• Demo Licenses & Extensions
Prerequisites
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15
LoadGen
All Protocols Except CitrixNV for LR/PC
Network NV-M
LoadGenNV-M
LoadGenNV-M
NV-M(Controller Compone
nt)
NV-M(PC Server Compone
nt)
* 1 day minimum checkout period
LM??
License
NV Analytics
NV Predictor
(+MS Office)
Controller PC Server
Internet
OR
OR
OR
EITHERNC Global DB
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16
Desktop & Web Apps, Manual or Automated Testing (UFT, Sprinter etc.)
NV for Mobile (Desktop Mode)
Network
App Under TestNV-M
Licenses
LM?
NV Analytics
EITHER
OR
NV TM
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17
On-Device Native, Hybrid & Web AppsNV for Mobile (Mobile Mode)
Network
Proxy
New WifiHotspot
On-Device WebInterface
Desktop WebConsole
NV-M
* 1 day minimum checkout period
NV Analytics
SSL Key
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Select NetworkTest Scenarios
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Let’s Begin with a Bit of Theory
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The Network Impact
20
TokyoLondonNew York
New YorkData Center
WAN
Consider 3 Users - 1 in NY, 1 in London And 1 in Tokyo• All Users Access the Same Application in the NY Data Center• Will They All Experience the Application in the Same Way?• If Not, Why Not?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21
Network Impairments
21
Conditions which inhibit the flow of data across a network• Key Impairments are:
• Network Delay (Latency)• Jitter (Delay variation, inter-packet gap variation)• Packet Loss (inverse of delivery rate)• Bandwidth Constraints• Bandwidth Utilization
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22
Mobile Network Impairments
22
Mobile Networks Experience Extreme Conditions of the Same Net Impairments• More Network Delay (Last Mile + Backhaul, RRC latency)• More Jitter (Delay variation, inter-packet gap variation)• More Packet Loss (inverse of delivery rate)• More Bandwidth Constraints• More Bandwidth Utilization
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23
Why is There Delay on a Terrestrial Network?
23
A Bt (msec) = d / (2/3)c
Signald
Packet PacketPacket Packet PacketBUSY
Lost Packet
Network Delay• Propagation Delay• Processing / Insertion / Serialization Delay• Queueing Delay
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24
New York
LondonFrankfurt
Tokyo
Sydney
Signal Speed - Geography
24• Speed of Light = 299,792.458 km/s
New York London Frankfu
rt Tokyo Sydney
New York 5,585km 6,219km 10,871k
m15,998k
m
London 639km 9,585km 16,991km
Frankfurt 9,357km 16,480k
mTokyo 7,793kmNew
YorkLondon Frankfu
rtTokyo Sydney
New York 28ms 31ms 54ms 80ms
London 3ms 48ms 85msFrankfurt 47ms 82ms
Tokyo 39msT = D / (2/3)c
Distance
TheoreticalMinimumSignal Time(One Way)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25
25
Combination of:• Server Connect Latency (First Mile)• Back-Haul/Propagation Latency• Client Connect Latency (Last Mile)
Total Latency
First Mile
Last MilePropagation Latency
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.26
Internet Consumer (Last Mile) Latency
26
ConnectionType RTT
Fiber 17msCable 28msDSL 44ms
http://www.fcc.gov/measuring-broadband-america
Broadband Latency (FCC)• “Fiber-to-the-home services provided 17 ms round-trip latency on average,
while cable-based services averaged 28 ms, and DSL-based services averaged 44 ms.”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.27
Mobile Latency
MobileCell Tower
Mobile Last Mile
50ms to 800msUp to 50% Packet LossBW 56Kbps to 20Mbps
OriginServer
MobileDevice
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.28
Mobile Latency – 1st Data point
28
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
Ofcom study, UK Mobile Networks, May 2012Static Tests:• “Latency… was an average of 117 milliseconds. Nearly two thirds of tests took more than
100 milliseconds”User Panel:• “Average latency was 192 milliseconds and around a third of tests delivered latency of
higher than 200 milliseconds”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.29
Mobile Latency – 2nd Data point
29
Cisco Cloud Index, 2010-2015“Global average fixed latency is 125ms and average mobile latency is 290ms… Global mobile consumer average latency is 307.3ms”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.30
Latency – Summary of Averages
30
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.31
Packet Loss
31
Random Early DetectionAverage queue monitored size & packets dropped based on statistical probabilities. If buffer is almost empty, all incoming packets are accepted. As queue grows, probability of dropping incoming packet grows. When buffer is full, probability reaches 1 and all incoming packets are dropped.
Packets get dropped on an end to end link due to the following:• Network congestion causes queues to saturate• Network congestion triggers RED• Interference on wireless networks• Hardware errors• Firewall Policies
Packet loss is the inverse of delivery rate100% - packet loss rate = delivery rate
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.32
Packet Loss
32
http://pam2012.ftw.at/papers/PAM2012paper6.pdf
Packet Loss, Akamai/MIT Research• Tested 149 Million connections in 51 countries
• Compared PL for mobile networks from non-mobile networks• Most connections have no packet loss (Median PL is 0%).
• 75th percentile usually zero too• Average connection packet loss: 2.5%
• Fixed network average is 2.2%• Implies connections are usually fine, but sometimes terrible
• The average PL for the non-zero half is 5%• Suggests “small” connections have higher Packet Loss
• HTTP Connections are often small
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.33
Jitter
33
Jitter
No Jitter
Jitter-Sensitive Applications:• Video• VoIP
Jitter = Variability over time of latency across a network• PDV – Packet Delay VariationConstant Latency = Zero Jitter
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.34
Mobile Bandwidth in Theory
34
Protocol Max Downlink Max UplinkGPRS 56-115 KbpsCDMA200 (1X) 153.6 KbpsEdge 236.8 Kbps3G Bar 2 Mbps stationary
client384 Kbps moving client
EV-DO 2.5-4.9 Mbps 0.15-1.8 MbpsEDGE Evolution 1.6 Mbps 0.5 MbpsHSDPA 2 Mbps 200 KbpsHSPA+ 14.4 Mbps4G Bar 1 Gbps stationary
client100 Mbps moving client
HSPA+ Release 11 21-672 Mbps 5.8-168 MbpsLTE 100-300 Mbps 50-75 MbpsWiMax 37-365 Mbps 17-376 Mbps
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.35
Mobile Bandwidth
35
You only get those speeds it if you’re this guy…
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.36
Mobile Bandwidth
36
Source (PCWorld): http://bit.ly/JX7MQw
PCWorld/Novarum3G• Download: 0.59 – 3.84 Mbps• Upload: 0.56 – 1.44 Mbps
4G• Download: 2.81 – 9.12 Mbps• Upload: 0.97 – 5.86 Mbps
Date: Apr, 2013
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.37
Download BW Impact on Load Time
DSL
Cable FIBER
Poor Cell Conn
Avg Cell Conn
Load Time (ms)
Download BW (Kbps)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.38
Upload BW Impact on Load Time
DSL
Cable FIBER
Poor Cell Conn Avg Cell Conn
Load Time (ms)
Upload BW (Kbps)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.39
Latency Impact
DSL
CableFIBER
Poor Cell Conn
Avg Cell Conn
Load Time (ms)
Latency (ms)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.40
Network Profiles – Conclusions for Performance
Upload BW doesn’t matter much until <200 kbpsDownload BW Matters at Low Values <1Mbps
• Unless Transfer Volume (Data) is LargeLatency matters the mostPacket Loss is Important but causes variation in results
• Only use in Worst Network Conditions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Select Network Test Scenarios
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.42
1. Determine the Full Range of Client Connection Types, for example:
• Remote Office Tokyo, London• Consumer Web UK, Germany• Consumer Mobile 2.5G India, UK, Germany• Consumer Mobile 3G UK, Germany, Japan• Consumer Mobile 4G UK, Japan
2. Select Test Range of Network Conditions (Not All):• Remote Office London• Remote Office Tokyo• Consumer Web Europe Typical • Consumer Mobile 4G UK Best • Consumer Mobile 3G Europe Typical• Consumer Mobile 2.5G India Worst
Selecting the Network Values for a POC
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.43
1. Work Out One-way Propagation Delay Server To Clients• Calculate KM Distance/200,000
2. Attempt To Validate Using Ping Tests• Or Use Other Tools – Try To Exclude The Last Mile
3. Demonstrate Mechanisms To Capture Network Conditions …• NC Server/Agents• NC App• NC Global Library• Other Tools/DataBut Do Not Use The Results In The POC
Selecting the Network Values for a POC
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.44
5. Select Test Range of Network Conditions (Not All):• Remote Office London• Remote Office Tokyo• Consumer Web Europe Typical • Consumer Mobile 4G UK Best • Consumer Mobile 3G Europe Typical• Consumer Mobile 2.5G India Worst
6. … Which Are:• Representative • Predictable• In Sequence
Selecting the Network Values for a POC (cont.)
Use:• Manual, Static Definitions• Stepped Latency & Bandwidth (Equally Spread?)• Static (Zero?) Packet Loss (Except Final Case)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.45
Example Network Test Profiles• Baseline 0ms 100Mbps/100Mbps 0% PL• Remote Office London 20ms 10Mbps/10Mbps 0% PL• Consumer Web Europe Typical 50ms 5Mbps/2Mbps 0% PL
• Consumer Mobile 4G UK Best 80ms 10Mbps/2Mbps 0% PL• Remote Office Tokyo 100ms 10Mbps/10Mbps 0% PL• Consumer Mobile 3G Europe Typical 130ms 2Mbps/512kbps 0% PL• Consumer Mobile 2.5G India Worst 250ms 512kbps/128kbps 1% PL
Notes:• Always One Way Latency in NV• Baseline Should Have BW Defined to Ensure Stability Between Tests• Network Profiles Should be Common Between Test Types (Functional/Non-
functional)
Selecting the Network Values for a POC (cont.)
(Too Many?)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What to Cover
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.47
CapabilitiesNetwork Discovery
• (NC Server/Agents – Avoid)?• NC Global DB• NC Mobile App• Other Tools
Single User Test - Desktop• Define Network Conditions• Proxy (SSL/HTTPS)?• Transaction Manager• (Integration with UFT Etc. – Avoid?)
Single User Test - Mobile• Define Network Conditions• Proxy (SSL/HTTPS)• Web Console/Mobile Web
Multi User Test• Define Network Conditions• Runtime Metrics• LR Analysis• NV Predictor• (Packet Capture – NV Analytics –
Avoid?)• (API Transaction Marking – Avoid?)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Test
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.49
Check SSL/HTTPS• Use Proxy (Always Needed for Mobile)• Verify Decryption Works (Doesn’t Always)Assess Test Steps• Analyse User Journeys using NV Transaction Manager• Use NV Analytics to Focus on Network-Impacted Steps• Create Modified (Focussed) TM Steps for Network TestingExecute Tests• Baseline• Network Models
Test Process – Single User
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.50
Remember• Save .Shunra files AND TM (Timing) Exports
• Unless using Mobile Console• Good Naming Conventions!Gotchas …• Check the Adapter• Connect TM!• TM Marking (Before 1st/After Last Packet)• TM Times vs NV Analytics Times
Test Process – Single User (cont.)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.51
Check SSL/HTTPS• If Yes – Avoid Packet Capture During Test• Use Manual Test?Check Script Timings• Script Think Time• Paced vs Real-World?• RampDesign Test Scenario• Add/Define Groups to Align with NV Virtual LocationsExecution Sequence:1. Full Test w/o NV2. Low-Volume Test with NV3. Full Test with NV
Test Process – Multi User (LR/PC)
Will Network Make Any Difference to Capacity?• 0.1s Baseline Execution Time – 25s Think Time• 0.2s Baseline Execution Time – 300s Pacing
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.52
Remember• Save Scenarios• Save Results & Analysis Files• Document Network Profiles (Export?)Gotchas• “Local User” – Required For LR/PC Analysis• Blank Virtual Location – For NV Predictor• BW per Virtual Location vs Per Vuser• Packet Capture• Transaction Marking
Test Process – Multi User (LR/PC – cont.)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Report
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.54
Example Docs
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Follow Up
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.56
Presentation
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Q&A
Recommended