Upload
john-johnson
View
136
Download
5
Tags:
Embed Size (px)
Citation preview
© British Telecommunications plc
VDC DesignACE Training
November 2008
Andrew Holding
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
Scope
•The scope of this training is to ensure that network designers understand the ACE topology and the configuration options used within the VDC design•This is a high-level training to explain basic features and ACE behaviour•It is assumed that attendees have basic load-balancing knowledge
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
What is the ACE Module?
Application Control Engine
Layer 3-7 content-aware, virtualised, application load-balancer with SSL termination & initiation and security
© British Telecommunications plc
What is the ACE Module?
CSS 11506
CSM
Appliances
Cat6K Modules
ACE Module
CSS 11503
CSS 11501
ACE Appliance
© British Telecommunications plc
The Evolution of L4 to L7 Services
Infrastructure simplification with L4–7 Services integration Converged policy creation, management, and troubleshooting Reduced latency (single TCP termination for all functions)
Previous
IntegratedLayer 4
andLayer 7Rules
Now – with Application Control Engine
© British Telecommunications plc
ACE Hardware Architecture
SwitchFabric
Interface
SwitchFabric
Interface
SupConnect
SupConnect
16G
100M DaughterCard 1
DaughterCard 1
DaughterCard 2
DaughterCard 2
8G
8G
SSLCryptoSSL
Crypto
10G
NP1NP1 NP2NP2
10G10G
ControlPlane
SAN OS
ControlPlane
SAN OS
2G
CDESwitch60Gbps
CDESwitch60Gbps
© British Telecommunications plc
ACE Performance/Features
•Max of 4 ACE’s per chassis (64Gbps)
•4Gbps, 8Gbps, 16Gbps single link to Backplane
•4Million Concurrent connections
•~350K L4 connections per second
•Onboard SSL Offload (1K to 15K tps throughput)
•Virtualisation (250 Contexts)
•TCP Reuse
•DDoS protection
•etc
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE software versions
Version number for ACE: 3.0(0)A1(6.3b)
Based on SanOS release 3.0(0)BU identifier is “A”ACE software version 1.6(3b)
SanOS info has now (A2.x) been dropped for simplification;
“show ver” :-Software loader: Version 12.2[118] system: Version A2(1.1) [build 3.0(0)A2(1.1) adbuild_00:25:02-2008/06/05_/auto/adbu-rel3/rel_a2_1_1_throttle/REL_3_0_0_A2_1_1] system image file: [LCP] disk0:c6ace-t1k9-mz.A2_1_1.bin installed license: ACE-08G-LIC ACE-VIRT-050 ACE-SEC-LIC-K9 ACE-SSL-05K-K9
Note: ACE Module and ACE Appliance use different software images
© British Telecommunications plc
ACE software versions (cont’d)
BNCMNSSW01>show mod Mod Ports Card Type Model Serial No.--- ----- -------------------------------------- ------------------ ----------- 1 1 Application Control Engine Module ACE10-6500-K9 SAD1021076N 2 6 Firewall Module WS-SVC-FWM-1 SAD100202V9 3 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL1005CBZP 4 1 SSL Module WS-SVC-SSL-1 SAD094307LT 5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL1004BPJU 6 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD1006061M 7 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD100301YX 8 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAL1005C12A 9 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAD100204FK
Mod MAC addresses Hw Fw Sw Status--- ---------------------------------- ------ ------------ ------------ ------- 1 0030.f275.b454 to 0030.f275.b45b 1.1 8.7(0.22)ACE A2(1.1) Ok 2 0013.c39f.63f8 to 0013.c39f.63ff 4.0 7.2(1) 3.2(4) Ok 3 0016.c810.3284 to 0016.c810.32b3 2.3 12.2(14r)S5 12.2(18)SXF1 Ok 4 0030.f274.f702 to 0030.f274.f709 4.0 7.2(1) 2.1(9) Ok 5 0013.c43a.8cb0 to 0013.c43a.8cb3 4.5 8.1(3) 12.2(18)SXF1 Ok 6 0013.c39f.cce0 to 0013.c39f.cce7 1.9 4.2(3a) Ok 7 0013.c39f.8530 to 0013.c39f.8537 1.9 4.2(3a) Ok 8 0016.c75a.a700 to 0016.c75a.a703 2.2 12.2(14r)S5 12.2(18)SXF1 Ok 9 0015.62e1.aee8 to 0015.62e1.aeeb 2.2 12.2(14r)S5 12.2(18)SXF1 Ok
© British Telecommunications plc
ACE licensing
Base = 5 contexts (plus Admin), 1000 SSL tps, 4Gbps
Contexts = 50, 100 or 250Throughput = 8 or 16GbpsSSL = 5,000, 10,000, 15,000 tps
bncmnace02/Admin# show license status
Licensed Feature Count------------------------------ -----SSL transactions per second 5000Virtualized contexts 50Module bandwidth in Gbps 8
bncmnace02/Admin# show ver….Software loader: Version 12.2[118] system: Version A2(1.1) [build 3.0(0)A2(1.1) adbuild_00:25:02-2008/06/05_/a
uto/adbu-rel3/rel_a2_1_1_throttle/REL_3_0_0_A2_1_1] system image file: [LCP] disk0:c6ace-t1k9-mz.A2_1_1.bin installed license: ACE-08G-LIC ACE-VIRT-050 ACE-SEC-LIC-K9 ACE-SSL-05K-K9…
© British Telecommunications plc
ACE Virtualisation
One Physical DeviceMultiple Virtual Systems
(Dedicated Control and Data Path)
• Traditional Device:– Single configuration file
– Single routing table
– Limited RBAC (Role Based Access Control)
– Limited resource allocation
25% 25% 20%15%15%100%
• Cisco Application ServicesVirtualisation:– Distinct configuration files– Separate routing tables– RBAC with contexts, roles, domains– Management and data resource control– Independent application rule sets– Global administration and monitoring
© British Telecommunications plc
ACE Multiple Contexts
Physical Device
Context 1Admin
ContextContext
Definition,Resource
Allocation,FT Config
Managementstation
Context 2 Context 3
AAA
Admin Context + 250 Contexts (Licensed: five contexts in base code)
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Deployment
Web Client 6500 with
ACE
Web Server
Physical View
© British Telecommunications plc
ACE VLANs
Web Client
ACE
Web Server
Catalyst 6500
Client-side VLAN
Server-side VLAN
Logical View
© British Telecommunications plc
Bridged (Layer 2) Mode
Server Default Gateway:Upstream Router
ACE Bridging
Subnet A
VLAN 10 VLAN 20
© British Telecommunications plc
Routed (Layer 3) Mode
Server Default Gateway:ACE IP
ACE Routing
Subnet AVLAN 10
Subnet BVLAN 20
© British Telecommunications plc
One-Armed Mode
Server Default Gateway:Upstream Router
Subnet BVLAN 20
Subnet CVLAN 30
Subnet AVLAN 10
ACE not in path – PBR or SNAT required for return traffic
© British Telecommunications plc
Routed, Bridged or One-Armed Mode?
All of these “modes” can be mixed within, and between, contexts - the same context can have bridged interfaces, routed interfaces and one-armed interfaces
Advantages of bridged vs routed are;+ Routing protocols can be exchanged through the ACE+ Multicast packets can be passed through the ACE
Disadvantage of bridged vs routed;– Potential for bridge-loop if both ACEs go active-active (RPVST+
used to minimise impact. Note: MST not supported)– If SNAT required, then traffic must be “load-balanced”
One-armed (ACE is not inline for load-balanced traffic)+ Removes potential bottleneck- PBR or SNAT required
© British Telecommunications plc
VDC ACE Topology
ACE has a static default route with a next-hop of the FW1 VRF, and server-subnet routes with a next-hop of the Cust VRF
VLAN%cust1-ace1-ss-vlan%
%cust1-fw1-vrf-name%
%cust1-ace1-vrf-name%
AC
E B
lock
VLAN%cust1-ace1-ns-vlan%
VLAN7
Fir
ewal
l Blo
ck
EIGRP%ace-blade1-hostname%-001/002
VLAN 501
VLAN 601
Subnet A
© British Telecommunications plc
ACE Interface Configuration
interface vlan 231 description Client vlan ip address 172.16.31.5 255.255.255.0 no shutdown
– Routed interfaces:
interface vlan 231 bridge-group 3 no shutdowninterface vlan 232 bridge-group 3 no shutdown
interface bvi 3 description Server Access vlan ip address 172.16.31.5 255.255.255.0 no shutdown
– Bridged interfaces:
© British Telecommunications plc
Which slot is the ACE in?
Cat6k>show modMod Ports Card Type Model Serial No.--- ----- -------------------------------------- ------------------ ----------- 1 1 Application Control Engine Module ACE10-6500-K9 SAD1021076N 2 6 Firewall Module WS-SVC-FWM-1 SAD100202V9 3 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL1005CBZP 4 1 SSL Module WS-SVC-SSL-1 SAD094307LT 5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL1004BPJU 6 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD1006061M 7 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD100301YX 8 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAL1005C12A 9 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAD100204FK
Mod MAC addresses Hw Fw Sw Status--- ---------------------------------- ------ ------------ ------------ ------- 1 0030.f275.b454 to 0030.f275.b45b 1.1 8.7(0.22)ACE A2(1.1) Ok 2 0013.c39f.63f8 to 0013.c39f.63ff 4.0 7.2(1) 3.2(4) Ok 3 0016.c810.3284 to 0016.c810.32b3 2.3 12.2(14r)S5 12.2(18)SXF1 Ok 4 0030.f274.f702 to 0030.f274.f709 4.0 7.2(1) 2.1(9) Ok 5 0013.c43a.8cb0 to 0013.c43a.8cb3 4.5 8.1(3) 12.2(18)SXF1 Ok 6 0013.c39f.cce0 to 0013.c39f.cce7 1.9 4.2(3a) Ok 7 0013.c39f.8530 to 0013.c39f.8537 1.9 4.2(3a) Ok 8 0016.c75a.a700 to 0016.c75a.a703 2.2 12.2(14r)S5 12.2(18)SXF1 Ok 9 0015.62e1.aee8 to 0015.62e1.aeeb 2.2 12.2(14r)S5 12.2(18)SXF1 Ok
© British Telecommunications plc
Configuring ACE VLANs
– Create the necessary VLANs on the Cat6k.
– Group the VLANs into service line card VLAN groups.
– Assign the VLAN groups to individual ACE modules.
vlan 7,2001-2003, 3502,3504
svclc multiple-vlan-interfacessvclc module 1 vlan-group 1svclc vlan-group 1 7,2001,2002
© British Telecommunications plc
Configuring ACE VLANs (cont’d)
– Create the necessary VLANs on the Cat6k.
– Group the VLANs into service line card VLAN groups.
– Assign the VLAN groups to individual ACE modules.
vlan 7,2001-2003, 3502,3504
svclc multiple-vlan-interfacessvclc module 1 vlan-group 1svclc vlan-group 1 7,2001,2002
© British Telecommunications plc
Configuring ACE VLANs (cont’d)
– Create the necessary VLANs on the Cat6k.
– Group the VLANs into service line card VLAN groups.
– Assign the VLAN groups to individual ACE modules.
vlan 7,2001-2003, 3502,3504
svclc multiple-vlan-interfacessvclc module 1 vlan-group 1svclc vlan-group 1 7,2001,2002
© British Telecommunications plc
Verify Cat6k Setup
Cat6k>show svclc vlan-groupDisplay vlan-groups created by both ACE module and FWSM commands
Group Created by vlans----- ---------- ----- 1 ACE 7, 2001-2002 2 FWSM 201-206,401-406,999-1000 3 ACE 2003
Cat6k>show svclc moduleModule Vlan-groups------ ----------- 01 1,3
Cat6k>show firewall moduleModule Vlan-groups------ ----------- 02 2,3
v2003 v2001v401
Group 3 Group 1Group 2
© British Telecommunications plc
Accessing the ACE
Cat6k#session slot 1 processor 0
Connect to the ACE from IOS:
Processor “0” = Control Plane CPU for configuration
Processor “1” = NP1
Processor “2” = NP2
© British Telecommunications plc
Creating ACE Contexts
bncmnace02/Admin# show vlanVlans configured on SUP for this module vlan7 vlan2001-2003
bncmnace02/Admin#configEnter configuration commands, one per line. End with CNTL/Z.bncmnace02/Admin(config)# context developmentbncmnace02/Admin(config-context)# allocate-interface vlan 7bncmnace02/Admin(config-context)# allocate-interface vlan 2001-2003bncmnace02/Admin(config-context)# exitbncmnace02/Admin(config)# exit
1. Create Context from within Admin context
2. Allocate Interfaces
© British Telecommunications plc
Verifying ACE Setup
ACE-Module/Admin# show context development
Name: development , Id: 117Description:Resource-class: defaultVlans: Vlan7, Vlan2001-2003
ACE-Module/Admin# show runGenerating configuration....
context development allocate-interface vlan 7 allocate-interface vlan 2001-2003
© British Telecommunications plc
Accessing ACE Contexts
bncmnace02/Admin# changeto developmentbncmnace02/development#
[Prompt shows ACE hostname and current context]
Access new context
From Admin Context
… or can Telnet/SSH direct to management interface of the relevant context (once it has been created)
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Resource Management
Rates Memory
Per Context Control:
Resource levels for each context Support for oversubscription
– Bandwidth
– Data connections per sec.
– Management connections per sec.
– SSL bandwidth
– Syslogs per sec.
– Access lists
– Regular expressions
– Data connections
– Management connections
– SSL connections
– Xlates
– Sticky entries
© British Telecommunications plc
ACE Resource Management
Minimum Guarantee
Maximum Unlimited
Minimum Guarantee
Maximum Equal To Minimum
© British Telecommunications plc
ACE Resource Management
ACE-Module/Admin(config)# resource-class goldACE-Module/Admin(config-resource)# limit-resource all minimum 10% maximum unlimited
ACE-Module/Admin(config)# context development ACE-Module/Admin(config-context)# member gold
Context 1 MinimumContext 2 MinimumContext 3 MinimumContext 4 Minimum
Total ACE
resources
Oversubscribed Global Pool (unreserved resources)
© British Telecommunications plc
ACE Resource Management
ACE-Module/Admin# show resource allocation-----------------------------------------------------------Parameter Min Max Class-----------------------------------------------------------acl-memory 0.00% 100.00% default 20.00% 200.00% gold syslog buffer 0.00% 100.00% default 20.00% 200.00% gold ...
“default” resource class = 0% minimum, unlimited maximum
“gold” resource class = 10% minimum, unlimited maximum
Looking at the above figures, the gold class is applied to 2 contexts, meaning there is a 200% oversubscription
By default a context is a member of the “default” resource group
© British Telecommunications plc
ACE Resource Management – gotcha’s
• Only allocate the minimum resources required/estimated initially (its hard to recoup resources later), and ensure you have a “reserve”
• Unlike other resources, sticky resources are not allocated by using the “all” keyword. Sticky resources must be allocated individually if required
resource-class gold limit-resource all minimum 20.00 maximum equal-to-min
limit-resource sticky minimum 20.00 maximum equal-to-min
• Bandwidth value is shown in “Bytes” (not Bits)bncmnace02/Admin# show resource usage Allocation Resource Current Peak Min Max Denied--------------------------------------------------------------------------
Context: development <snip> throughput 316 6125 0 500000000 0 <snip>
500,000,000Bps = 4Gbps
© British Telecommunications plc
ACE Resources and Licence Upgrades
• ACE licence can be upgraded from 4-8-16Gbps, and SSL 1K, 5K and 15K SSL tps
• These ACE resources can be limited however a percentage figure is used, not an absolute amount
• This means the amount of resources allocated will vary depending upon the current licence
– 20% of the 4Gbps licence is 800Mbps, whereas 20% of the 8Gbps = 1.6Gbps– 10% of 1000 SSL tps = 100tps, whereas 10% of 5000 SSL tps = 500 tps
• When upgrading an ACE licence, the percentage figure in the resource-class does not change, therefore you must change the percentage allocated if you want the same amount of resources to be allocated to members of that resource-class after the upgrade
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Redundancy
• Two ACEs form a Redundancy pair• Single FT VLAN required between ACEs (not one per context)• Redundant ACEs can be in the same, or different, Catalyst 6500 Chassis
• Each pair of contexts (on two distinct ACE modules) form a redundancy group, one being active and the other standby
• Both ACE modules can be active at the same time, processing traffic for different contexts, and backing-up each other (stateful redundancy)
Example:2 ACE modules4 FT groups4 Virtual Contexts (A,B,C,D)
ACE-1
ACE-2
FT VLAN
AActive
A’Standby
FTgroup 1
BActive
B’Standby
FTgroup 2
CActive
C’Standby
FTgroup 3
DActive
D’Standby
FTgroup 4
© British Telecommunications plc
ACE Redundancy
• Fault-Tolerant (FT) VLAN (/30) carries FT packets, heart beats, config-sync packets, state replication packets
• Configuration synchronisation (bulk and incremental) & state replication is enabled by default
• SSL files (keys and cert’s) are not replicated
• Much like HSRP, each Context is assigned a priority, and the highest priority will become master (if pre-emption enabled)
• Normally recommend pre-emption is only used for operations (failing back to a recovered ACE)
• Possible to oversubscribe resources on both ACEs (active/active), however, a failure of one of the ACEs (or path to the ACE) will reduce capacity by half
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
Key ACE terms
• North (Client) side & South (Server) side VLANs • Real Server – load-balanced servers• Serverfarm – a group of Real servers• Probe – keepalive to Real Servers• Predictor – load-balancing algorithm (e.g. round-
robin, least-connections etc)• VIP – Virtual IP Address. Typically NATed to the
address of the real-servers. Has no dependence on connected subnets.
• Route-Health Injection (RHI). ACE Module* can advertise the reachability of the VIP to the MSFC
* RHI not supported on ACE Appliance
© British Telecommunications plc
ACE Key Terms
Server/South side VLAN
Client/North side VLAN
Real Server
Server Farm
RHI if VIP is Active
© British Telecommunications plc
ACE Interface Configuration
• Think of the ACE as a Firewall– By default, traffic is not allowed “through” or “to” the ACE
• Access-list type “management” is required for traffic “to” the ACE
• IP access-list is required for traffic “through” the ACE
• N.B. Access-list type “ethertype” required in order to allow STP BPDUs (when ACE is in Bridged mode)
© British Telecommunications plc
ACE Interface Configurationaccess-list nonip ethertype permit bpdu
access-list permit-all line 10 extended permit ip any any
interface vlan 2001
description Client_VLAN
bridge-group 1
access-group input nonip
access-group input permit-all
no shutdown
interface vlan 2002
description Server_VLAN
bridge-group 1
access-group input nonip
access-group input permit-all
no shutdown
interface bvi 1
ip address 10.1.1.4 255.255.255.0
no shutdown
© British Telecommunications plc
ACE routes
• Routes are not shared between contexts• Each load-balancing context requires route(s) to servers AND a
route back to the client, before forwarding traffic• Admin context will typically only need management routes
• Within VDC each Context requires;– the default route will have a next-hop of the North-side VRF
HSRP address– Route to server subnets with next-hop of South-side HSRP
address– Management route(s)
ip route 0.0.0.0 0.0.0.0 10.80.199.109 default routeip route 10.80.202.0 255.255.255.192 10.80.199.94 route to rserversip route 10.80.196.0 255.255.254.0 10.80.193.3 management routeip route 147.149.163.128 255.255.255.128 10.80.193.3 management route
© British Telecommunications plc
ACE Real Server Health Monitoring
- “Out-of-band” monitoring (Probes/Keepalives)- Probes can be used to
- Detect the loss of a real server- Monitor a gateway or other remote device for failover
purposes
- Optional port and ip-address probe configuration
- Multiple different native probe types including TCL
support
- Typically recommend a frequent simple probe (e.g. ping
every 5 seconds) combined with a less-frequent more
complicated probe (e.g. HTTP GET every 30 seconds). If
either probe fails, the server will be declared down
© British Telecommunications plc
Rservers, ServerFarms, Predictors and Probes
probe icmp ping interval 5 passdetect interval 120 receive 5
probe tcp tcpprobe port 80 interval 30 open 5
probe http httpprobe port 81 interval 30 passdetect interval 300 request method get url /index.shtm expect status 200 299 open 5
rserver host server1 ip address 10.1.4.101 probe ping inservicerserver host server2 ip address 10.1.4.102 probe ping inservice
serverfarm host farm1 predictor leastconns probe tcpprobe rserver server1 inservice rserver server2 inservice
serverfarm host farm2 probe httpprobe rserver server1 81 inservice rserver server2 81 inservice
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
TCP Connection
ServerClient
SYNSYN_ACK
ACK
DataACK
Data
More Data
ACK
FIN
ACK
ACK
FIN
Initialize
Use
Close
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Load-Balancing Configuration
1. Create L3/L4 class map (define match criteria)2. Create load-balancing policy map (define actions
to perform)3. Create a multi-match policy map to tie the L3/L4
class-maps and policy maps together4. Activate the classification-action rules on either
an interface or “globally”
class-map C1 match <criteria>
class-map C1 match <criteria>
policy-map type loadbalance P1 <action>policy-map type loadbalance P1 <action>
interface vlanX service-policy input MMP1interface vlanX service-policy input MMP1
policy-map multi-match MMP1 match C1
policy P1 match C2
policy P2
policy-map multi-match MMP1 match C1
policy P1 match C2
policy P2
© British Telecommunications plc
ACE Load-Balancing Configuration
L3/L4 Class-map defaults to “match-all”, which means only one VIP address is allowedbncmnace02/dev(config)# class-map fredbncmnace02/dev(config-cmap)# match virtual-address 1.1.1.1 tcp eq 80bncmnace02/dev(config-cmap)# match virtual-address 1.1.1.1 tcp eq 443Error: Only one match virtual-address is allowed in a match-all class-map and it cannot mix with any other match typebncmnace02/dev(config-cmap)#
“match-any” L3/L4 Class-map allows multiple VIP’s
class-map match-any fred 2 match virtual-address 1.1.1.1 tcp eq www 3 match virtual-address 1.1.1.1 tcp eq https
© British Telecommunications plc
Layer 3 & Layer 4 Load-balancing
• L3 & L4 information is present in the first packet of the flow:
Source IP address
Destination IP address
IP Protocol
Protocol ports
• Load-balancing can be made on first packet of a flow
© British Telecommunications plc
Layer 3/4 Flow Setup
SYN
SYN
Identifies VIP (matches class-map)Selects Server FarmMakes Load Balancing Decision
© British Telecommunications plc
Layer 3/4 Flow Setup
SYN
SYN_ACK
ACK
Data
SYN
Identifies VIP (matches class-map)Selects Server FarmMakes Load Balancing Decision
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
Layer 7* Flow Setup
• L7* load-balancing:
URL parsingCookie parsingGeneric HTTP header parsingSSL IDetc
* Layer 5 and above (SSL is Layer 5)
• Requires TCP termination and buffering of multiple packets before a LB decision can be made (this is why L7 load-balancing can never be as fast as L4 load-balancing)
© British Telecommunications plc
Sniffer Trace of HTTP Connection
“Interesting” information only arrives in the 4th packet
GET /css/cavendish/template.css
© British Telecommunications plc
Layer 7 Flow Setup (1/3)
SYN
SYN_ACKChooses seq# and replies w/ SYN_ACK
© British Telecommunications plc
Layer 7 Flow Setup (1/3)
SYN
SYN_ACKChooses seq# and replies w/ SYN_ACK
ACKs data received from clientACK
ACK
Data (e.g. HTTP GET /)Starts buffering client packets
© British Telecommunications plc
Layer 7 Flow Setup (2/3)
SYN
Buffers all packets …… until it has enough data for policy matchingElects serverfarm, makes balancing decisionSends previously buffered SYN to real server
© British Telecommunications plc
Layer 7 Flow Setup (2/3)
SYN
Buffers all packets …… until it has enough data for policy matchingElects serverfarm, makes balancing decisionSends previously buffered SYN to real server
ACK
SYN_ACK
Does not forward SYN_ACK
© British Telecommunications plc
Layer 7 Flow Setup (3/3)
Data
Empties buffer and sends data to server
ACK
Does not forward ACKStarts splicing the flows
© British Telecommunications plc
Layer 7 Flow Setup (3/3)
Data (e.g. HTTP GET /)
Empties buffer and sends data to server
ACK
Does not forward ACKStarts splicing the flows
Data
ACK
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE test topology.100
.101 .102
VLAN 2004 10.1.2.0/24
11Port8110Port81
3/21
3/22 3/23
Sets Cookie serverid=server1
Sets Cookie serverid=server2
VLAN 2006 10.1.4.0/24
54:be
0d:17 ef:5e
VLAN 2001 10.1.1.0/24
VLAN 2002 10.1.1.0/24
Context “landing” b4:55
.4
.1
.5
.6
.1
bt-fwsm-ace
bt-customer
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Layer 3 Policy
•Destination IP address of incoming packet must match the VIP address(es) in the class-map•Any protocol•Any port
© British Telecommunications plc
ACE Layer 3 Policyrserver host server1
ip address 10.1.4.101inservice
rserver host server2ip address 10.1.4.102inservice
serverfarm host farm1 rserver server1 inservice rserver server2 inservice
class-map match-all classmap1 2 match virtual-address 10.1.1.100 any
policy-map type loadbalance first-match policy1 class class-default serverfarm farm1
policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1
interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdowninterface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
ip route 0.0.0.0 0.0.0.0 10.1.1.5ip route 10.1.4.0 255.255.255.0 10.1.1.6
© British Telecommunications plc
ACE Layer 4 Policy
•Destination IP address of incoming packet must match the VIP address(es)•Protocol(s) must match•Port(s) must match
© British Telecommunications plc
ACE Layer 4 Policyserverfarm host farm1 predictor leastconns probe tcpprobe rserver server1 81 inservice rserver server2 81 inservice
class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www
policy-map type loadbalance first-match policy1 class class-default serverfarm farm1
policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active
interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdowninterface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
ip route 0.0.0.0 0.0.0.0 10.1.1.5ip route 10.1.4.0 255.255.255.0 10.1.1.6
© British Telecommunications plc
ACE Layer 7 Policy
•Destination IP address of incoming packet must match the VIP address(es)•Protocol(s) must match•Port(s) must match•Layer 5-7 information (e.g. HTTP URL, Cookie, Header, SSL session ID etc) must match
Note: Regular expression matching is case-sensitive by default
© British Telecommunications plc
ACE Layer 7 Policy
Typically used;• when traffic differentiation is required (e.g. *.jpg sent to farm of Cache Engines, everything else sent to the Web servers)• when traffic manipulation is required (e.g. Cookie insert, HTTP Header insert)
Performance is less than L3/L4 due to;• Delayed Binding• (Layer 7 ME required (depends on persistent rebalance))
© British Telecommunications plc
ACE Layer 7 Policy
Layer 7 Class-maps & Policy-maps can be used to;
•Match on HTTP URL•Match on HTTP headers (cookie, language, host, browser, etc)•Match on string within HTTP payload (not header)•Insert/Delete/Modify HTTP headers (e.g. Insert ClientIP, rewrite URL etc)•Match RADIUS, RDP, RTSP and SIP fields•Generic TCP/UDP data parsing•Match on Source-IP address•Set IP QoS (DSCP) values•TCP Connection re-use
Layer 7 class-maps can use a match-all, match-any, or use nested class-maps (match A or B or [C & D])
© British Telecommunications plc
ACE Layer 7 Policyserverfarm host farm1 serverfarm host caches rserver server1 transparent inservice rserver cache1 rserver server2 inservice inservice rserver cache2
inservice
class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www
class-map type http loadbalance match-any checkforstatic 2 match http url .*\.jpg 3 match http url .*\.pdf
policy-map type loadbalance first-match policy1 class checkforstatic serverfarm caches class class-default serverfarm farm1
policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1
interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdowninterface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
ip route 0.0.0.0 0.0.0.0 10.1.1.5ip route 10.1.4.0 255.255.255.0 10.1.1.6
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Route Health Injection
•ACE can advertise the reachability of a VIP to the MSFC. If the VIP goes down, the route is withdrawn.•Appears as a /32 static route, with the next-hop of the ACE•Allows the MSFC to redistribute the route and advertise using routing protocol•VRF-aware•Default AD = 77
BNCMNSSW01#show ip route vrf bt-fwsm-aceRouting Table: bt-fwsm-ace<Snip>
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masksC 10.1.2.0/24 is directly connected, Vlan2004C 10.1.1.0/24 is directly connected, Vlan2001D 10.1.4.0/24 [90/3072] via 10.1.1.6, 5d23h, Vlan2001S 10.1.1.100/32 [77/0] via 10.1.1.4, Vlan2001B* 0.0.0.0/0 [20/0] via 10.1.2.0 (bt-sc1-fusion), 7w0d
© British Telecommunications plc
VDC ACE RHI
ACE RHI injects active VIPs into Firewall Block VRF
VRF redistributes static routes into EIGRP and advertises northwards
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
Persistence Rebalance
• HTTP 1.0 requires a separate TCP connection for each HTTP request
• HTTP 1.1 supports persistent TCP connections, allowing pipelining of multiple HTTP requests within the same TCP connection
• Processing Layer 7 (within ACE HTTP ME) information is more
resource intensive than simply checking Layer 4 information
• By default, once the ACE has made a Layer 7 (check URL, Language etc) decision on the first packet of a flow, (which farm/server), all subsequent traffic will be sent to that server (“fast-switched”)
• “Persistence rebalance” disables this feature “Persistence” refers to a persistent TCP connection (multiple
pipelined HTTP requests) “Rebalance” refers to whether traffic should be re-balanced to
another serverfarm
© British Telecommunications plc
Persistent Rebalance (cont’d)
• Only required if need to check (or manipulate) every HTTP packet within the same (persistent) TCP connection e.g.;– URL *.jpg & *.gif send to cache engines– HTTP Header “Language=French” send to French farm– HTTP Header Insert – insert information into EVERY HTTP
packet (rather than only the first one)
• Persistence rebalance is disabled by default on ACE (enabled by default on CSM)
• HTTP parameter-map required to modify behaviour
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Connection Handling
• ACE can handle a maximum of 4 million concurrent connections
• Will continually monitor all connections to check whether the connection has closed, and resources can be freed and made available for new connections
• TCP is normally simple – watch for FIN or RST• Impossible to tell for UDP, or “broken” TCP
connections
© British Telecommunications plc
ACE Connection Handling
ACE idle timers• TCP default = 1 hour• UDP default = 2 minutes• ICMP default = 2 seconds
DNS, RADIUS etc LB may need to reduce the timeout so the connection entry does not stay up unnecessarily
With default timers 33K DNS requests per second will utilise 100% of connections (within 2 minutes)
Use connection “parameter map” to change the settingValue = 0 to 4294967294 seconds (136 years )Set timeout to zero to disable the timeout (connection will stay up
for ever)
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE Stickiness
• Required when you need multiple sessions (concurrent or subsequent) from the same user to be sent to the same backend server.
• Many applications work by the Client initiating multiple connections e.g. HTTP sessions
• Without sticky, if ACE load-balances on round-robin, least-connections etc, then connections from the same client are likely to be sent to different servers
• If no sticky entry exists (e.g. first time a client connects), then the Predictor configured on the serverfarm is used to select which server to send the traffic to. At this point, a sticky table entry is created, and can then be used for subsequent connections (until the entry times out)
© British Telecommunications plc
ACE Stickiness
ACE can stick on the following information;
It is important to understand the application and the client profile before deciding which method to use
N.B. Sticky resources are not allocated to a context by default (not included in the “resource all” designation), and need to be specifically assigned
* Requires ACE A2.x
RADIUS attributes*
RTSP Header*SIP Header*SSL Session ID*
Source/Dest IP addressLayer 4 Payload*HTTP Content*HTTP CookieHTTP Header
© British Telecommunications plc
Source IP Stickiness
• Advantages– Simple to configure and troubleshoot
• Disadvantages– Proxy Servers in the path can present a single source IP
address (SNAT) for many clients. Result is all users are sent to the same rserver
– Mega Proxies can change the SNAT IP address mid-session
© British Telecommunications plc
ACE test topology.100
.101 .102
VLAN 2004 10.1.2.0/24
11Port8110Port81
3/21
3/22 3/23
Sets Cookie serverid=server1
Sets Cookie serverid=server2
VLAN 2006 10.1.4.0/24
54:be
0d:17 ef:5e
VLAN 2001 10.1.1.0/24
VLAN 2002 10.1.1.0/24
Context “landing” b4:55
.4
.1
.5
.6
.1
bt-fwsm-ace
bt-customer
© British Telecommunications plc
ACE Source-IP Stickiness
serverfarm host farm1 rserver server1 inservice rserver server2 inservice
sticky ip-netmask 255.255.255.0 address both group1 timeout 60 replicate sticky serverfarm farm1
class-map match-all classmap1 2 match virtual-address 10.1.1.100 any
policy-map type loadbalance first-match policy1 class class-default sticky-serverfarm group1
policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
© British Telecommunications plc
Cookie Stickiness
• Cookie can be – set by the rserver (which is learned by ACE)– set by the ACE (Cookie-insert)
• Cookie can be server-specific (sticky-serverfarm), or per-serverfarm (HTTP class-map)
• Advantages– Combats Proxy issues relating to source-IP stickiness
• Disadvantages– Only supported with HTTP– Client browser must support cookies
© British Telecommunications plc
ACE test topology.100
.101 .102
VLAN 2004 10.1.2.0/24
11Port8110Port81
3/21
3/22 3/23
Sets Cookie serverid=server1
Sets Cookie serverid=server2
VLAN 2006 10.1.4.0/24
54:be
0d:17 ef:5e
VLAN 2001 10.1.1.0/24
VLAN 2002 10.1.1.0/24
Context “landing” b4:55
.4
.1
.5
.6
.1
bt-fwsm-ace
bt-customer
© British Telecommunications plc
ACE Cookie Match
serverfarm host farm1
rserver server1
inservice
rserver server2
inservice
sticky http-cookie serverid cook_group
serverfarm farm1
class-map match-all classmap1
2 match virtual-address 10.1.1.100 tcp eq www
policy-map type loadbalance first-match policy1
class class-default
sticky-serverfarm cook_group
policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
Cookie name set by server
© British Telecommunications plc
Agenda
•Scope•ACE Overview
– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology
•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing
•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT
•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A
© British Telecommunications plc
ACE SNAT
• Source-NAT can be required for Client to Server, Server to Client, Server to Server
• NAT can be performed using either a pool of addresses, or statically with a one-to-one mapping (use where predictable IP is required)
• Within the policy-map you must configure which NAT pool number and which egress interface is to be used
Caveats;The ACE will *not* NAT bridged traffic. It must hit a load-balancing policy in order for SNAT to be implemented
SNAT to the VIP address requires ACE 2.x software
© British Telecommunications plc
ACE SNAT
Dest IP = InternetSource IP = Web
Dest IP = InternetSource IP = ACE NAT
1
2
ACE requires LB policy in order to
“catch” traffic to NAT
e.g. RFC1918-addressed server requires connectivity to the Internet
© British Telecommunications plc
ACE SNATrserver host gwnorth ip address 10.1.1.1 inservice
serverfarm host gateway_north_farm transparent rserver gwnorth inservice
class-map match-all SNAT-CLASS 2 match virtual-address 0.0.0.0 0.0.0.0 any
policy-map type loadbalance first-match SNAT-POL class class-default serverfarm gateway_north_farm
policy-map multi-match SLB-SNAT class SNAT-CLASS loadbalance vip inservice loadbalance policy SNAT-POL nat dynamic 1 vlan 2001
interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all nat-pool 1 10.1.1.250 10.1.1.251 netmask 255.255.255.0 pat no shutdown
interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input SLB-NAT no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
© British Telecommunications plc
ACE Server to Server SNAT
Some applications require server to server load-balancing. For example, load-balanced Web server to Application server traffic
Some topologies (e.g. VDC) require extra configuration in order to ensure server-to-server load-balancing occurs correctly
© British Telecommunications plc
Server to Server without SNAT (1/2)
1. Web Server initiates traffic to Application VIP
2. ACE load-balances traffic to Application server B
By default source-IP is maintained
Application Application
Dest IP = VIPSource IP = Web
Dest IP = App BSource IP = Web1 2
A B
© British Telecommunications plc
Server to Server without SNAT (2/2)
3. App Server replies to Web IP
4. MSFC routes to directly-connected subnet
5. Web Server sends TCP RST since the source IP (and SEQ info) does not match any open sessions
Application Application
Dest IP = WebSource IP = App B
Dest IP = WebSource IP = App B
43
Server sends TCP RST5
A B
© British Telecommunications plc
Server to Server with SNAT (1/2)
1. Web Server initiates traffic to Application VIP
2. ACE load-balances traffic to Application server B
•ACE configured to change source IP to a SNAT IP
Application Application
Dest IP = VIPSource IP = Web
Dest IP = App BSource IP = ACE SNAT1 2
A B
© British Telecommunications plc
Server to Server with SNAT (2/2)
3. App Server replies to ACE SNAT IP
•MSFC routes to ACE
4. ACE changes he Source and Destination IP back to the VIP and Web, and traffic routed correctly
Application Application
Dest IP = WebSource IP = VIP
Dest IP = ACE SNAT
Source IP = App B4
3
A B
© British Telecommunications plc
ACE Server to Server LBserverfarm host farm1 predictor leastconns rserver server1 inservice rserver server2 inservice
class-map match-all classmap1 2 match virtual-address 10.1.1.100 any
policy-map type loadbalance first-match policy1 class class-default serverfarm farm1policy-map multi-match mmp_ss1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active nat dynamic 1 vlan 2002
interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all nat-pool 1 10.1.1.254 10.1.1.254 netmask 255.255.255.255 pat service-policy input mmp_ss1 no shutdown
interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown
© British Telecommunications plc
ACE SSL
• What is SSL• Why terminate SSL on ACE• SSL Termination• Certificate Chains
© British Telecommunications plc
What is “SSL”
• Secure Sockets Layer• Layer 5 protocol – above TCP and below Applications,
such as HTTP, FTP etc
© British Telecommunications plc
ACE SSL components
• SSL Server Certificate• SSL key pair (private key and public key)• Optional – Certificate Chain
© British Telecommunications plc
Without SSL Accelerators
• Server terminates SSL session• Certificates and keys are held on the server• Load-balancer can only act at Layers 3-5, since the layers above
are encrypted (cannot see URL or Cookie)
Client ServerSLB
HTTPS HTTPS
© British Telecommunications plc
Benefits of SSL Accelerator
• Manageability - One cert vs Many (cost, operations effort)
• Troubleshooting – can “sniff” HTTP layer• Stickiness – can see HTTP Cookies• Performance/Scalability
Client ServerSLB
HTTPS HTTP
© British Telecommunications plc
SSL Certificates
Public Key
Private Key
CertificateSigning Request
Public KEYCommon nameDomain name
LocationE-mail
CertificateSigning Request
Public KEYCommon nameDomain name
LocationE-mail
ValidationProcess
Server Public Key
Server Public Key
Application Application
Company Docs
KEY Pair
Server Private Key
Certificate Authority
SSL Server
CertificateCertificate
© British Telecommunications plc
SSL Fundamentals: Key Exchange Packet Flow Overview
Server Public Key
RandomNumber
Generator
RSA Encrypt
“Shared” Secret Key
“Shared” SecretEncrypt & Decrypt
Client Browser
Server Public Key
RSA Encrypt
“Shared” Secret Key
“Shared” SecretEncrypt & Decrypt
Server
Private Key
Data
Data Data
Public Key
Client “Hello”
Server “Hello”
SAasdfkjw1340+jakjb//alkjt
SAasdfkjw1340+jakjb//alkjt
“Data Exchange”
SAasdfkjw1340+jakjb//alkjt
“Key Exchange”
Data
Certificate
© British Telecommunications plc
ACE SSL Termination
ACE SSL configuration is MUCH simpler (single termination point) than CSM/SSLM
The ACE requires the following in order to terminate SSL connections
ACE/context(config) # show crypto filesFilename File File Expor Key/ Size Type table Cert----------------------------------------------------mycert.pem 1275 PEM No CERTmykey.pem 283 PEM Yes KEY
•SSL Server Key-pair (Private and Public Key)•SSL Server Certificate•Optionally – SSL Certificate Authority Certificate Chain
© British Telecommunications plc
SSL Termination
ACE
EncryptedUnencrypted
parameter-map type ssl sslparam
cipher RSA_WITH_3DES_EDE_CBC_SHA
cipher RSA_WITH_AES_128_CBC_SHA
cipher RSA_WITH_AES_256_CBC_SHA
!
ssl-proxy service sslproxy
key mykey.pem
cert mycert.pem
ssl advanced-options sslparam
!
serverfarm host farm1
rserver server1 81
inservice
rserver server2 81
inservice
class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq https !policy-map type loadbalance first-match policy1 class class-default serverfarm farm1!policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active ssl-proxy server sslproxy
© British Telecommunications plc
SSL Certificate Chains
• Optional• Typically required when the Certificate Authority that has signed
the Server Certificate is not trusted by the Client• ACE will send the complete certificate chain, and the client will
check each certificate in turn to see if it trusts the signer (CA)
crypto chaingroup InternalCAcerts cert rootCA.pem cert ouCA.pem cert deptCA.pem
ssl-proxy service secure_access key mykey.pem cert mycert.pem chaingroup InternalCAcerts
© British Telecommunications plc
ACE Configuration spreadsheetNew Context
Layer 3 LB
Layer 4 LB
Layer 7 LB
L4 SSL L7 SSL SNAT
Resource-Class x
Context Name x
FT Group x
BVI x x x x x x x
Routing x
Parameter-map x x x
Crypto chaingroup x x
Ssl proxy service x x
Probe x x x x x x
Rserver x x x x x x
Server farm x x x x x x
Class-map Match-all virtual-address (L3/4)
x x x x x x
http loadbalance (L7) x x
Policy-map Type loadbalance x x x x x x
multimatch x x x x x x
Access-list x
WorksheetRequirement
For stickiness, apply a sticky-serverfarm to the LB policy-map, and apply the serverfarm to the sticky-group
© British Telecommunications plc
ACE Documentation
• Cisco ACE Documentation http://www.cisco.com/en/US/partner/products/ps6906/tsd_products_support_model_home.html
• ACE Design Guidelines coming soon..
• How to use the ACE Packet Capture featurehttp://livelink.intra.bt.com/livelink/livelink.exe?func=ll&objId=70435818&objAction=browse&sort=name&viewType=1
© British Telecommunications plc
Questions?