NET2415BU Utilizing NSX load balancing for scalability, or ... · 3 NSX load balancing for availability ... Evolution of Server Load Balancing Form Factors 1996 OS-based ... and allow

Luke Hoffer – Sr. Systems Engineer, VMwareRush Maniar – Sr. Product Manger, VMwareLuke Sipple – IT Engineering Manager, Self Esteem Brands LLC

NET2415BU

#VMworld #NET2415BU

Utilizing NSX load balancing for scalability, reliability, and security: Overview, best practices, and customer case study

VMworld 2017 Content: Not fo

r publication or distri

bution

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

2#NET2415BU CONFIDENTIAL



bution

Agenda

1 Load balancing overview

2 NSX load balancing for scalability

3 NSX load balancing for availability

4 NSX load balancing as a security layer

5 Customer case study: SE Brands, LLC




bution

IT Challenges

AVAILABILITY

SECURITY

SCALIBILITY




bution

Increasing Demands Being Placed on IT Today




bution

Evolution of Server Load Balancing Form Factors

1996

OS-based Server Load Balancing

1997

Physical Server Load Balancer

(SLB)

2006

Virtual Contexts

2010

Virtual Appliances

2013

NSX Logical Load Balancing

SLB integrated into Network

Platform




bution

Standby

SLB

Active

SLB

Client Server

Client 1 Server 3

Client 2 Server 4

… …

Persistence Table

VIP

Server 1

Server 2

Server 3

Server Pool A

Server 4

Server 5

Server

Pool B

Client 1

Client 2

What Is SLB?

7



bution

WAN

Application 1

Application 3

Application 2

NSX SLB NSX SLB

NSX SLB

Per-application SLB (vs. per environment)

Scale-out model (vs. scale-up)

Low costs and/or usage-based pricing

Ops intelligence into application

Automation

The Emerging SLB Deployments (Hardware vs Software LB)




bution

What Does NSX LB Do?

web-01 web-02 app-01 db-01app-02

Edge Load Balancer • Server Load Balancing

– High Speed L4 Load Balancing

– L7 Load Balancing and Manipulation

• ADC Features

– SSL Offload

– TCP Multiplexing

• Automation

– Inbuilt Programming Support

– vRA and Openstack Integrated

• Integration with 3rd party Load Balancer

NSX Edge




bution

NSX Load Balancer Customers Momentum

160+




bution

Service Providers

Global Financials

Retail

Healthcare

Integrators

Media & Communications

Transportation

Government

Education

NSX Load Balancer Customers Momentum

11



bution

Simple licensing model

Better capacity planning

Automation

Speed up of delivery

It’s part of NSX Platform

Reducing capex cost

NSX LB

Benefits

Why Customers Are Deploying NSX Logical Load Balancer?




bution

NSX LB Vision: Driving NSX Everywhere

13

Service Provider /Partners Public clouds

New app frameworks

Mobile devices

Virtual Desktop (VDI)

On-premises data center

Internet of things

Availability Scalability

Providing scalability of applications

Security

Inherently Secure Infrastructure

Delivering high availability for applications

#NET2415BU CONFIDENTIAL



bution

NSX LB Main Ingredient: Edge Services Gateway

14

VPN

Form Factor vCPU RAM GB VIPs per ESG Pools per ESG Servers per Pool

Compact 1 0.5 64 64 320

Large 2 1 64 64 320

Quad-Large 4 1 64 64 320

X-Large 6 8 1024 1024 3072




bution

Inline (Transparent) Deployment Topology

Client Address172.30.40.7

Virtual Server Address192.168.20.20

VM 1 Address192.168.1.1

VM 2 Address192.168.1.2

VM 3 Address192.168.1.3




bution

One-Armed (Proxy) Deployment Topology

16

Client Address172.30.40.7

VM 1 Address192.168.1.1

VM 2 Address192.168.1.2

VM 3 Address192.168.1.3

Virtual Server Address192.168.1.20




bution

Agenda









bution

Requirement 1: Distribute Connections to Multiple Servers

18

Overload!!!

Re

sp

on

se

Tim

e

Concurrent Connections

!!!




bution

Solution: Configure Algorithm on Server Pool

19

• ROUND-ROBIN

– Each server is used in turn according to the assigned weight.

• LEASTCONN

– All traffic from a specific client IP address routed to the same server.

• Hashing

– IP-HASH: Selects server based on hash of source IP.

– URI/URL: Hash of URI/URL respectively as defined in algorithm parameters.

– HTTPHEADER: Hash of HTTP header item specified in algorithm parameters.




bution

Requirement 2: Session Persistence

20

Client A

VM 1

VM 2

VM 3

Client B

Client C

Persistence Table

Client A VM 1

Client B VM 2

Client C VM 3




bution

Solution: Define Persistence Method in Application Profile

21

• Cookie Based– Insert: New cookie is added to client

session in addition to those sent by the server.

– Prefix: Edge adds its cookie info to the cookie sent by the server (and removes for return traffic prior to sending to the server). Use when client can’t support multiple cookies.

– App Session: Edge looks for session ID in the URL. i.e. http://fakesite.org/admin;xsessionid=123456abcd where “xsessionid” is the session ID.

• Source IP– All traffic from a specific client IP

address routed to the same server.




bution

http://fakesite.org/admin;jsessionid=123456abcd

Agenda









bution

Requirement: Application Monitoring/Availability

23

VM 1

VM 2

VM 3




bution

Solution: Define/Assign Service Monitors to Pool

24

• Service Monitors available

– ICMP: Pings pool members and marks down if no ICMP echo after specified number of retries.

– TCP/UDP: Attempts to open a connection on the specified socket with option to send/receive configured string and/or TCP extensions once connection is established.

– HTTP/S: Sends HTTP/S get requests to pool members and looks for 200 OK response. Can optionally specify URL, alternate HTTP methods, and/or expected response codes other than 200 OK.




bution

Agenda






6 Product roadmap update with VMware Product Management




bution

Requirement: Stop Some of the Bad Stuff that My Firewall Can’t!!!

26

VM 1

VM 2

VM 3

Slowloris, Brute Force, etc.

Reconnaissance

Apps/Infrastructure unable to support TLS




bution

SSL Offloading Part 1: Install/Assign Certificate

27

• Install Certificate on ESG

– Manage > Settings > Certificates

– to add existing certificate or “Actions” to generate CSR and import.

• Assign certificate to app profile

– Optionally specify cipher and/or client authentication




bution

SSL Offloading Part 2: Create Virtual Server

28

• Virtual Server creation

– Select Previously created app profile

– Select “HTTPS” protocol

– Specify standard HTTP pool

TLS Encrypted Plain old HTTP




bution

Server Masking: Application Rule Applied to Virtual Server

29

• Create application rule

– Select Previously created application rule from “advanced” tab.

• Virtual Server creation

– Select Previously created application rule from “advanced” tab.

Server: SuperSecret




bution

Filter on UserAgent: Application Rule Applied to Virtual Server




bution

Other Security Application Rule Examples

31

Rule Description

tcp-request content reject if !HTTP

Deny requests on the virtual server that are not

HTTP (i.e. other protocols attempting to use port 80,

etc.)

acl Brute_Force fe_sess_rate ge 5

tcp-request connection reject if Brute_Force

Deny requests from clients sending more than 5

connections per second to the Virtual Server

acl Naughty_IP src 192.168.0.1

tcp-request connection reject if Naughty_IPDeny requests from 192.168.0.1




bution

Agenda



3 NSX load balancing for reliability






bution

Customer case studyLuke Sipple – IT Engineering Manager



bution

From old school to SDDC in 18 months

• “We are in a bad place. Let’s re-architect the entire infrastructure. Ready, Go!”

• Drivers for the project:

– DDoS misfires

– Full rack

– No control and slow response to change requests

– Weekend trips to the data center

– Physical SQL servers

• Goals for the project:

– OpEx

– No physical hands-on

– Scalability

– Enhanced security!

CONFIDENTIAL 34



bution

Our environment

• Started on NSX 6.2 in mid-2015 in a Rackspace greenfield environment

• Went full production on 6.2.4 in October 2016

• Now on NSX 6.3.1 in two production environments and one development environment

--

• 93 production VMs, not counting NSX

• Our API pool: Average of 10K requests per minute – 22K peak

• Our ClubHub pool: Average of 12.5K RPM – 17.5K peak

• SQL cluster handles 40 million statements per day with 158 million transactions per day

CONFIDENTIAL 35



bution

Micro-segmentation and ESGs

• 127 rules – where was vRNI when I needed it?!• Separate edge HA pair for each major service

CONFIDENTIAL 36



bution

One-armed design

Servers are still directly accessible where needed (testing purposes), but public NAT and internal DNS all point at the load-balancer to handle inbound traffic.

CONFIDENTIAL 37



bution

Minimal overhead

Traffic average ~200 Mbps and 1100 concurrent connectionsNSX load-balancer VM uses 700 MHz of CPU, 540 MB of RAM, & 2GB of disk space

CONFIDENTIAL 38



bution

Issues encountered along the way

• vmxnet3 performance bug

– Had to manually edit ring buffers and edit some offload settings to regain performance

– VMware resolved this in an ESXi hotfix shortly after

• 6.2.3 DFW global address set issue

– We were one of the first few companies to report this

– VMware had to pull the version and release hotfix

– Thankfully we were not in full production yet

• HA edge services gateways in split-brain

– Happened on a few occasions where traffic would not pass. Quick fix is to reboot the active member and allow the failover.

– VMware now recommends a dedicated HA link. We have not had this issue for a couple versions.

• ARP failures

– VMs on a specific host will not hold ARP. Rebooting the hypervisor resolves the issue.

CONFIDENTIAL 39



bution

API

• Allow non-NSX admins to monitor health• Patch and test safely• Build into automated deployments

CONFIDENTIAL 40



bution

TL;DR

• We were early adopters and accepted the risk… learned a ton along the way

• NSX 6.3 has been very stable

– Our Rackspace data center is currently at 100% uptime for 2017 (knock on wood)

• VMware is pouring resources into the product

• If you don’t have any outlandish layer-7 rule needs, NSX load-balancing can most likely fit your needs

CONFIDENTIAL 41



bution

Join VMUG for exclusive access to NSX

vmug.com/VMUG-Join/VMUG-Advantage

Connect with your peers

communities.vmware.com

Find NSX Resources

vmware.com/products/nsx

Network Virtualization Blog

blogs.vmware.com/networkvirtualization

Where to get started

Dozens of Unique NSX Sessions

Spotlights, breakouts, quick talks & group discussions

Visit the VMware Booth

Product overview, use-case demos

Visit Technical Partner Booths

Integration demos – Infrastructure, security, operations,

visibility, and more

Meet the Experts

Join our Experts in an intimate roundtable discussion

Free Hands-on Labs

Test drive NSX yourself with expert-led or self-paces

hands-on labs

labs.hol.vmware.com

Training and Certification

Several paths to professional certifications. Learn

more at the Education & Certification Lounge.

vmware.com/go/nsxtraining

Engage and Learn Experience

Try Take




bution

https://www.vmug.com/VMUG-Join/VMUG-Advantage

https://communities.vmware.com/community/vmtn/nsx

http://vmware.com/products/nsx

http://blogs.vmware.com/networkvirtualization

http://labs.hol.vmware.com/

http://www.vmware.com/go/nsxtraining



bution



bution

Documents

NET2415BU Utilizing NSX load balancing for scalability, or ... · 3 NSX load balancing for availability ... Evolution of Server Load Balancing Form Factors 1996 OS-based ... and allow