Extending Enterprise Network into Public Cloud with Ciscod2zmdbbm9feqrf.cloudfront.net/2017/usa/pdf/BRKARC-2749.pdf · Extending Enterprise Network into Public Cloud with Cisco CSR1000v

Extending Enterprise Network into Public Cloud with Cisco CSR1000v

Fan Yang, Technical Marketing Engineer

Tony Banuelos, Product Manager

BRKARC-2749

© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public

Cisco Spark

Questions? Use Cisco Spark to chat with the speaker after the session

1. Find this session in the Cisco Live Mobile App

2. Click “Join the Discussion”

3. Install Spark or go directly to the space

4. Enter messages/questions in the space

How

Cisco Spark spaces will be available until July 3, 2017.

cs.co/ciscolivebot#BRKARC-2749


Your Speaker

Tony Banuelos

[email protected]

Product ManagerProduct Manager at Cisco and at the company for17 years working across different technologieslike VoIP, UC Interoperability, SONET, Cisco VXIand public cloud solution.

Fan Yang

[email protected]

Technical Marketing Engineer5 years in Cisco

Youtube Channel: http://cs.co/csr1000v

BRKARC-2749 4

mailto:[email protected]


http://cs.co/csr1000v


Related Cisco Live Las Vegas 2017 Sessions

Building Hybrid Clouds in Amazon Web

Services with the CSR 1000v

BRKSEC-3007

BRKSEC-2064

BRKARC-2023

Advanced Cisco IOS Security

BRKSDN-2411

NFV Performance - Challenges and Solutions

NGFWv and ASAv in Public Cloud (AWS and

Azure)

[LAB] LTRVIR-2100Deploying Cisco Cloud Services

Router CSR 1000V on AWS and Azure

BRKARC-2749 5

• Introduction of Cisco CSR1000V in Public Cloud

• CSR Use Cases on Public Cloud

• Transit VPC solution

• Licensing and Resources

Agenda

Introduction of Cisco CSR1000V in Public Cloud


What is Public Cloud?

• On-demand extensible network and compute resources

• Supports IaaS model, allowing users to create virtual machines, storage, networking, security, and other services

• Supports open API to automate deployment of application services

• Amazon AWS and Microsoft Azure are leaders in public cloud

BRKARC-2749 8


Enterprises are Moving Applications to CloudNumerous Challenges to Adopt

• Enterprise adoption of cloud continues to grow

• Security is still top of the list concern

• 70% of enterprise cloud solutions are hybrid approach where both private and public clouds are used

• Multi-Cloud becomes strategy for enterprise customers

BRKARC-2749 9


Cloud Adoption NumbersData is collected from 1000 cloud customers across different business segments

Source: RightScale 2017 State of the Cloud

• In 2016 Private Cloud Adoption fell to 72% from 77% the previous year, which impacted hybrid cloud which fell to 67% from 71%

• 95 percent of organizations surveyed are running

applications or experimenting with infrastructure-as-a-

service (Public Cloud)

• 85 percent of enterprises have a multi-cloud strategy, up

from 82 percent in 2016

• Most customers run their application in the cloud, with

41% running apps in public cloud and 38% in private

cloud

BRKARC-2749 10


• CSR is offered on Amazon AWS and Microsoft Azure

• CSR1000V pricing based on technology package, throughput, license term PLUS platform cost

• How do I choose the platform for CSR on AWS or Azure?

How do I Size Cisco CSR 1000V?

Notice: Actual cost will depend on negotiated terms and discounts

BRKARC-2749 11


Cisco CSR 1000V Cloud Platform Options

Size CEF(Mbps) IPSEC(Mbps)

T2.medium 390 300

M3.Medium 300 250

C4.large 575 550

C4.xlarge 860 860

C3.2xlarge 1330 1000

C4.2xlarge 2300 2200

C4.4xlarge 4600 4100

C4.8xlarge 5100 4700

Size CEF(Mbps) IPSEC(Mbps)

D2_v2 1400 680

DS2_v2 1400 800

D3_v2 2000 1200

DS3_v2 2000 1500

D4_v2 2000 1400

DS4_v2 2100 1800

CSR on AWS CSR on Azure

BRKARC-2749 12


• Cisco CSR1000V is supported on EC2 Instance Types: C3, C4, M3, T2 (R4 coming soon)

• Cost of CSR VM hosting depends on instance type model, size, term and region

• AWS offers pay-as-you-go (hourly) and pay-upfront (1Y or 3y term) consumption models

• Instance type size determines achievable CSR1000V performance

• Use AWS “Simple Monthly Calculator” to calculate cost http://calculator.s3.amazonaws.com/index.html

• Next slide shows an example on calculating AWS costs

Cisco CSR1000V on AWS Cloud Platform

BRKARC-2749 13

http://calculator.s3.amazonaws.com/index.html


• Cisco CSR1000V is supported on VM Types: D-series, Dv2-series and DSv2-series

• Cost of CSR VM hosting depends on instance type model, size, term and region

• Azure offers month-to-month consumption model

• VM type size determines achievable CSR1000V performance

• Use Azure “Simple Monthly Calculator” to calculate cost https://azure.microsoft.com/en-us/pricing/calculator/

• Next slide shows an example on calculating Azure costs

Cisco CSR1000V on Azure Cloud Platform

BRKARC-2749 15

https://azure.microsoft.com/en-us/pricing/calculator/


• Azure cost calculatorCSR1000V on Azure Cloud Platform

BRKARC-2749 16


Cisco Cloud Services Router (CSR) 1000VCisco IOS XE Software in a Virtual Appliance Form-Factor

Enterprise-class Networking with Rapid Deployment and Flexibility

*Only Available on Amazon AWS.

Server

Hypervisor

Virtual Switch

OS

App

OS

App

CSR 1000V

Software

• Familiar IOS XE software with ASR1000 and ISR4000

Infrastructure Agnostic

• Runs on x86 platforms

• Supported Hypervisors: VMware ESXi, Linux KVM, Citrix Xen, Microsoft Hyper-V, Cisco NFVIS and CSP2100

• Supported Cloud Platforms: Amazon AWS, Microsoft Azure

Performance Elasticity

• Available licenses range from 10 Mbps to 10 Gbps

• CPU footprint ranges from 1vCPU to 8vCPU

License Options

• Term based 1 year, 3 year or 5 year

• Smart License enabled

Programmability

• NetConf/Yang, RESTConf, Guest Shell and SSH/Telnet

BRKARC-2749 17


IOS-XE Coverage for All Deployment Types

Enterprise Data Center or

Branch

ISR 4400 ASR 1000Hypervisor

CSR 1000v

Cloud Platform

CSR 1000v

BRKARC-2749 18


The Benefits of Bringing IOS XE into Public Clouds

Extends Existing Routing

Topology

Integrates With Existing VPN Topology (Eg.

DMVPN)

Shares Existing Zone Based

Firewall Policies

Network Logging to

Existing Tools

Identifies Cloud Performance

Problems

IOS XE Supportable by Existing IT Staff

Existing Monitoring Tools

Existing Troubleshooting

Steps

BRKARC-2749 19

Public Cloud 101


Region and Availability Zone Concepts

• VM (Virtual Machines) is hosted in multiple data centers across the world. A region is a separate geographic area

• VM instances have to be launched into a specific region. Locating instances close to end users can reduce latency

• Region is consisted by multiple AZs (Availability Zone). Each AZ is isolated, but AZs in a region are connected through low latency and high bandwidth links.

BRKARC-2749 21


Virtual Private Cloud (VPC) Concepts

• VPC is isolated from other’s environment.

• VPCs’ IP ranges (RFC 1918) can overlap.

• IGW (Internet Gateway) provides external

access.

• Granular subnets can be created in VPC.

• Route Table can be associated to subnets

• UDR (User Defined Route) can be added

to route table

• Security Options:

• - Network ACLs protect subnets

• - Security Groups protect instances

• EIP to EIP communication is going through

Cloud Provider’s backbone

Route Table

Internet Gateway

VPC James Bond

CIDR 10.2.0.0/16Subnet A

10.2.1.0/24

Subnet B

10.2.2.0/24

VPC

WebApp1 Instance

IP: 10.2.1.25

Elastic IP Mappings

54.32.54.32 – 10.2.1.25

Internet

BRKARC-2749 22


No Link Local Broadcast in the VPC

• No Link local multicast or broadcast

• Affected Services Include:

• IGPs

• HSRP/VRRP

• BFD

• Proxy ARP, Gratuitous ARP > LISP-VM Mobility

• GRE as work-around for some services, some cloud

10.2.1.12

10.2.1.11

10.2.1.10

VPC

BRKARC-2749 23


Multiple Ways to Insert CSR 1000V as GatewayTwo Armed Mode:

• CSR has one interface in each network.

• Two options to change gateway

1. Change application VM’s default gateway to CSR IP

2. Change application subnet’s route table pointing to CSR as gateway. (Recommended, more flexible and scalable)

• Limitation on # of interfaces for CSR imposed by different cloud providers.

g1 g2

IGW

172.24.2.0/24

g1

IGW Route

Table

VPC

VPC

Public subnet Private subnet

One Armed Mode:• CSR has single interface and a default

gateway pointed towards Internet Gateway.

• Other subnets have route added to their route

table, pointing to the CSR as gateway.

• Instances in other subnets don’t need their

default gateway manually changed.

• Number of subnets is not limited by number of

interfaces

BRKARC-2749 24

CSR1000v Use Cases


CSR 1000V use cases for all public clouds• Extend Enterprise Routing Architecture to Cloud

• Common routing fabric securely extended to cloud• DMVPN, FlexVPN, GETVPN*• Support up to 1000 tunnels

• Remote Worker VPN Access

• FlexVPN IPSEC or SSLVPN via AnyConnect

• Flexible AAA server options for authentication

• Launch applications in regions near your users

• Across Region/Cloud Provider Interconnection• Distribute applications globally

• Accessibility across on-prem and cloud locations

• Overcomes VPN tunnel limitation on AWS and Azure

• Extend on-prem routing architecture into Public Cloud

• Monitor/Analyze/Shape traffic in Public Cloud• Security(vFW, VRF, AVC, Snort IPS/URL Filtering)• Assurance(IP SLA, BFD, QoS)• Scale to hundreds of VPC across regions/accounts (Transit VPC)• Monitoring and troubleshooting with known common tools

virtual private cloud

Cloud, US East

corporate office/branch


Cloud, US West

*GETVPN supported on DX/ER only (no NAT)

VPC

VPC

BRKARC-2749 26


CSR 1000V Routing High Availability on Cloud

• No virtual IP as with HSRP, since Cloud Provider doesn’t allow multicast or broadcast.

• BFD over GRE tunnel(AWS), IPSEC or VXLAN-GPE (Azure*) is enabled between two CSRs to detect failure

• Failure detection is automatic.

• Route Tables for app subnets are re-pointed to surviving CSR.

• CSR itself calls Cloud Provider’s REST API to shift Route Table routes.

CSR Subnet

App Subnet A

App Subnet B

Before HA Failover / After HA Failover

Cloud REST API

BFD

VPC

*Azure drop GRE packets

IGW

BRKARC-2749 27


Traffic Flow During Failover

BFD

CSR-A

CSR-B

InternetBFD

CSR-A

CSR-B

Internet

BFD

CSR-A

CSR-B

Internet

Cloud REST API

BFD

CSR-A

CSR-B

Internet

*Asymmetric routing may exist

VPC

VPC

VPC

VPC

IGW

BRKARC-2749 28


Two deployment models

VPC

Application VPC Gateway

• CSR deployed in application VPC

• Provide IPSEC gateway for entire VPC

• Need high availability

Transit Hub Router

• CSR deployed in dedicated Transit Hub, not in application VPC

• High speed traffic routing for spoke VPC

• High availability is built-in natively

Transit Hub

AZ1 AZ2

Application VPC

VPC

BRKARC-2749 29


CSR1000v Performance in AWS and Azure

• Max 10 NICs

• Support on HVM instance types including T2, M3, C3, C4

• Performance go up to 5Gbps L3 Routing and 4.5Gbps IPSEC

• 2, 4, 8 NICs deployment template

• Support on D2_V2, DS2_V2, D3_V2, DS3_V2, D4_V2, DS4_V2 instances

• Performance with 2Gbps L3 Routing and 1.8Gbps IPSEC

BRKARC-2749 30


Technical comparison between AWS and Azure for CSR 1000v

Feature AWS Azure

IPSEC Throughput 4.5 Gbps 1.8 Gbps

Number of vNIC supported today 10 2/4/8

High Availability (Routing) Supported Supported

Multiple IP addresses on vNIC Supported Supported

Allow Overlapping IP addresses Yes Yes

GRE Tunnel support in VPC/VNet Supported Not supported

L2 Broadcast and Multicast Not supported Not supported

Add or remove interfaces on

running CSR 1000V VMYes No(need to stop instance)

BRKARC-2749 31

VPC Connection Options


Cloud VPC to On-Premise Connection

VPC

Customer Network

New York

Customer Network

San Jose

Internet

Co-location

WAN

* AWS DX (Direct Connect) and Azure ER (Express Route)

Connection Option Use Cases Limitations

VPN IPSEC VPN connections for VPC to

VPC across regions

• Throughput limited by VGW or VPN

instance

• Point to Point

VGW (Virtual Private Gateway)

Dedicated Circuit* Consistent 1G/10G connection to Cloud

Provider Co-Location

• High Cost

• Relationship required for 3rd party

BRKARC-2749 33


VPC to VPC Connection

VPCVPC

Peering

VPC

Co-lo Co-lo

WANus-west us-east

Connection Option Use Cases Limitations

VPC Peering High bandwidth VPC to VPC connection • No across region peering

• Point to Point

Dev QA Prod

VGW (Virtual Private Gateway)

VGW to VGW connection is only supported on Azure today

VPN* IPSEC VPN connections for VPC to

VPC across regions

• Throughput limited by VGW or VPN

instance

• Point to Point

Dedicated Circuit* Consistent 1G/10G connection to Cloud

Provider Co-Location

• High Cost

• Relationship required for 3rd party

BRKARC-2749 34


VPC Peering

• High Bandwidth VPC to VPC Interconnection

• Share Private IP CIDR blocks between the VPCs

• Point to Point

• No Across Region Peering

• No Transit PeeringVPCVPC

Peering

us-west

Dev QA

BRKARC-2749 35


Dedicated Circuit (Direct Connect) Overview

• Dedicated connection between the enterprise and AWS

• Provides (1) private access to VPCs and (2) public access to AWS services (S3, etc)

• Sub-interface on corporate DC router for each service

• BGP peering for route exchange for each service

• 1G and 10G dedicated connections; sub-1G connections available via partners

• Multiple connections for redundancy

• No Native EncryptionVirtual Private Cloud

Virtual Private

Gateway (VGW)

Cisco

ISR/ASR

Direct Connect

CircuitCorporate DC

BRKARC-2749 36


A Closer Look At VGW (Virtual Private Gateway)

• VGW is an easy to use VPN service provided by AWS.

• It supports IPSEC VPN with pre-shared key (no certificate based).

• It supports static route and BGP routing (no route-map and fixed BGP AS number)

• VGW uses two end-points for high availability

• CGW (Customer Gateway) is needed to establish a IPSEC VPN.

• IPSEC can’t be established between two VGWs

• VGW is also used in DX (Direct Connect)

• Static route and BGP routing

• No encryption

BRKARC-2749 37


Comparison: CSR 1000v, VGW and VPC Peering

CSR 1000v

• Hub Spoke network design

• Active/Active for Tunnels

• Across regions and accounts

• Site-to-Site/DMVPN network

• Full Transit Routing functions

• Full Traffic Control (QoS) and

visibility

• Provide HA Redundancy

Features

Price

Performance

• Op to 5Gbps CEF and

4.5Gbps IPSEC

• Two CSRs doubles to 10Gbps

• 400K BGP routes

• Hourly and Annual

• BYOL(Bring Your Own

License)

• Data Transfer*

VGW

• Full mesh network design

• Active/Standby for Tunnels

• Across regions and accounts

• Only Site-to-Site IPSEC

• Basic BGP

• No Traffic Control and visibility

• Provide HA (Two Tunnels per

VPC)

• Max 500Mbps on AWS (up to

1Gbps by contact support)

• 200Mbps on Azure

• 100 routes

• Hourly (per VPN connection)

• Data Transfer*

VPC Peering

• Full mesh network design

• Same region

• No Transit Routing

• No Traffic Control

• Max 50 peers on AWS(up to

125 by contact support)

• Max 10 peers on Azure(up to

40 by contact support)

• Same bandwidth between

instances in same VPC

• Data Transfer*

*same cost for Data Transfer across three solutions, 0.02$/GB bi-directional

Enterprise

Grade

Simple

VPC Conn

BRKARC-2749 38

Transit VPC with CSR1000v


Public Cloud Transit Routing Challenge

• No transit routing capability

• Don’t support across region peering

A-B Peering

B-C Peering

Transit Routing NOT supported

A-to-C-thru-B

Full mesh

Private DC

…

Backhaul2

See next slide

VPC-A

VPC-C

VPC-B

BRKARC-2749 40


Transit Hub Point

• Network transit hub connecting multiple, geographically disperse networks

• High speed routing point in a centralized location

Source: http://www.srfconsulting.com/news/projects/smith-avenue-transit-hub/BRKARC-2749 41


Transit VPC Design

• Dedicated VPC: Simplifies routing by not combining with other shared services.

• CSR1000v Virtual Network Appliances: Provide dynamic routing and VPN network tunnels

• Redundancy: Dynamic routing combined with multi-AZ deployment creates a robust network infrastructure.

• VGW: VPC virtual gateways provide highly available connections to transit VPC virtual network appliances.

BA C

…...

Direct Connect

Express Route

Internet

Private DC

Transit VPC

Spoke VPC

Other

Provider

Networks

CSR1 CSR2AZ1 AZ2

Across regions, accounts/subscriptions

ASR

Automated solution is available on AWS.

Customer can build same solution without automation on Azure.

VPCVPCVPC

VPC

VGW

IGW

BRKARC-2749 42


Traffic Segregation

• Traffic segregation is built-in natively

• Each Spoke VPC is represented as a different VRF in CSR

• Routing is controlled through RT (Route Target)

• Different VPCs can communicate by export/import same RT

• Follow same mechanism to create customized VRF like on-premise VRF

CSR1

MP-BGP

On-Premise VRF

CSR2

VPC-A VPC-B VPC-C

Private DC

VPC-C VRFVPC-B VRFVPC-A VRF

BRKARC-2749 43


High Availability in Transit VPC

• Spoke VGW has two tunnels with both CSRs.

• Spoke VGW doesn’t support load balance across two tunnels. It’s using active standby.

• It’s possible different VGW uses different CSR as active.

• Both CSRs are forwarding traffic independently at same time.

• In case of CSR fail, the other CSR will take over all traffic.

BA C

…...

Transit VPC

Spoke VPC

CSR1 CSR2

VPCVPCVPC

VPC

Active Tunnel

Standby Tunnel

VGW

IGW

BRKARC-2749 44


Connect to DX (Direct Connect) – “Detached” VGW

VPC

AWS Direct

Connect

Private DC

ASR

“Detached” VGW

• Create a “Detached” VGW which is not attached to any VPC.

• DX connection is terminated on “Detached” VGW

• ASR doesn’t learn CIDR of Transit VPC

• Routes will be exchanged through VGW like a middle hop

• Specify same tag on VGW and tunnels will be automatically provisioned like another spoke

• Throughput will be restrained by VGW doing IPSEC encryption (Current 1Gbps)

IPSEC

Encrypted

Non-Encrypted

BGP1

BGP2

CSR2CSR1

Transit VPC

BRKARC-2749 45


Connect to DX (Direct Connect) – “Attached” VGW

Transit VPC

CSR1 CSR2

VPC

AWS Direct

Connect

Private DC

ASR

• Create a VGW for DX and attach it to Transit VPC

• DX connection is terminated on “Detached” VGW

• ASR learns CIDR of Transit VPC

• CSR builds BGP peering with ASR directly

• Manual configuration needed, can’t leverage previous Lambda scripts

• Throughput goes up to 10Gbps with 2xCSR

Tunnel

BGP2

BGP1

VGW

BRKARC-2749 46


Multi Region Deployment

Private DC 1

Transit VPC

Private DC 2

Transit VPC

Tunnel

Tunnel

us-eastus-west

DX/ER

Internet

DX/ER

Internet

Keep localized traffic in same regionASRASR

VPC VPC

VGW

IGW

CSR1

CSR2 CSR3

CSR4

BRKARC-2749 47


Scale Out

Private DC

Transit VPC

DX/ER

Internet

ASR

VPC

VGW

IGW

CSR1 CSR2 CSR3 CSR4

…...

• Add another pair of CSRs to scale out

• Remote end (VGW) has multiple tunnels and do L3 ECMP (Equal Cost Multiple Path)

• Elasticity as you go: monitor CSR real-time throughput and spin up new CSRs on demand.

BRKARC-2749 48


Transit VPC Architecture and Components on AWS

• Transit VPC: VPC deployed with two Cisco CSR

instances in separate AZs

• S3 bucket: Storage location for transit VPC config

files

• KMS (Key Management Service): All data in the

S3 bucket is encrypted using a solution-specific

AWS KMS managed customer master key (CMK).

• VGW Tags: Customer-specified opt-in tags to

automatically join a spoke VPC to the transit

network

• VGW Poller (Lambda function):

• Identifies and configures VGWs to connect to

the transit network (checks all regions every

minute)

• Writes new VPN connection details to an S3

bucket

• Cisco Configurator (Lambda function):

• Pushes VPN configuration to CSR instances

when config files are saved to S3

Spoke VPC

A

Spoke VPC

B

Spoke VPC

‘n’

Corporate Data Center

…

Other

Provider

Networks

Amazon S3 bucket

Cisco Configurator

VGW Poller

Transit VPC

AZ 2AZ 1AWS KMS

BRKARC-2749 49


Transit VPC Security Configuration

• Transit VPC:

• No inbound traffic – all VPN connections originate from CSRs

• CSR Hardening:

• SSH restricted to Cisco Configurator function security group

• SSH public key auth only (password auth disabled)

• Enables EC2 Auto Recovery for CSR instances

• Cisco Configurator:

• Runs inside VPC

• Uses automation-specific, unique SSH keys for auth

• S3 bucket:

• AES-256 SSE for all files

• Bucket policy controls which additional accounts may join the transit network

BRKARC-2749 50


Transit VPC workflowAdding new transit VPC spoke

CSR 1 CSR 2

AZ 1 AZ 2

Transit VPC

VGW Poller Cisco Configurator

A

C

Amazon

S3

bucket

B

1

2 3

4

5

BRKARC-2749 51


Transit VPC workflow (cont.) VGW Poller Logic

VGW Poller

A

C

Amazon

S3

bucket

B

1

2

yes

Does the VGW have the appropriate tag?

Is there an existing VPN connection?

Create Customer Gateways (if required)

for the IPs of the CSR instances

Create a VPN connection to the

Customer Gateway

Download the VPN configuration file in

XML and push it to Amazon S3

VGW Poller logic

No

BRKARC-2749 52


Transit VPC workflow (cont.)Adding new transit VPC spoke

CSR 1 CSR 2

AZ 1 AZ 2

Transit VPC

Cisco Configurator

Amazon

S3

bucket

3

4

SSH into the CSR instances

Apply the Cisco config onto the CSR

instances

Copy the XML VPN configuration file and

SSH keys from the Amazon S3 bucket

Cisco Configurator logic

From the XML file, extract VPN, BGP,

and interface parameters. Create a

Cisco config using these values.

BRKARC-2749 53


Transit VPC Best Practice (1)

• Is CSR dropping packets? Make sure CSR is running at licensed throughput

CSR-BYOL#show license all

License Store: Primary License Storage

StoreIndex: 0 Feature: ax_2500M Version: 1.0

License Type: Permanent

Start Date: N/A, End Date: May 15 2017

License State: Active, In Use

License Count: Non-Counted

License Priority: Medium

CSR-BYOL#show platform hardware throughput level

The current throughput level is 2500000 kb/s

BYOL (Bring Your Own License) Hourly

CSR-hourly#show license all

License Store: Primary License Storage

CSR-hourly#show platform hardware throughput level

The current throughput level is 200000000 kb/s

BR1-16.3.3#show platform hardware qfp active statistics drop

-------------------------------------------------------------------------

Global Drop Stats Packets Octets

-------------------------------------------------------------------------

Ipv4NoAdj 56 12876

Check Packet drop

BRKARC-2749 54



• I observe tunnel status on VGW is down on AWS console.

• Check tunnel status on CSR. VGW status might be a little bit delayed.

• If tunnel on CSR is down or no tunnel info, check if CSR has correct configurations pushed.

• If CSR has configurations, tunnels should be up typically.

• If CSR doesn’t have correct configurations. It means Lambda function has at least one of following problems.

1. VGW Poller can’t poll tag or wrong tag specified on VGW

2. Cisco Configurator can’t push configurations to CSR

• Check Cloud Watch logs to identify root cause for Lambda

Note: CSR security group doesn’t need inbound rule of UDP 500/4500 since IPSEC session is

initialized from CSR to VGW. Security group doesn’t restrict any outbound traffic.

BRKARC-2749 55



• I want to choose active CSR for spoke VPC.

• This is used to enable state full features, like ZBFW and etc.

• By default two CSRs are forwarding traffic at same time.

• Spoke VGW randomly picks one CSR as active, the other CSR as standby.

• You can use “preferred tag” and set specific CSR as active and standby.

Transit VPC

CSR1 CSR2

VPC

Preferred tag=CSR1Active Tunnel

Standby Tunnel

BGP as-path prepend

VGW

BRKARC-2749 56



• How to do maintenance on CSR? For example, version upgrade.

• CSR supports inline upgrade in 16.5.1b version and onwards. It will be the same process as upgrading a physical IOS-XE router (Upload bin and change boot).

• Two CSRs are working as active active.

• Let one CSR stop forwarding traffic gracefully by shutdown tunnels on CSR.

• All traffic will be forwarded to the other CSR.

• Upgrade the CSR to correct version and bring up tunnels.

• Traffic will be load balanced across two CSRs.

• Redo same steps on the other CSR.

BRKARC-2749 57



• How do I manage CSR through private IP, rather than EIP.

• Customer wants to manage CSR through private IP since most NMS (Network Management System) or Network Engineers sits in on premise network.

• For security concern, security group on CSR is only open to internal IPs.

• Create a “MGMT” VRF and tie to a Loopback interface

• Redistribute this loopback interface into BGP domainip vrf mgmt

rd 64512:2

route-target export 64512:0

route-target import 64512:0

interface Loopback0

ip vrf forwarding mgmt

ip address 1.1.1.1 255.255.255.255

router bgp 64512

address-family ipv4 vrf mgmt

redistribute connectedBRKARC-2749 58


Transit VPC SizingSizes include*:

• 2 x 500 Mbps (c4.large)

• 2 x 1 Gbps (c4.xlarge)

• 2 x 2.5 Gbps (c4.2xlarge)

• 2 x 4.5 Gbps (c4.4xlarge)

• 2 x 5 Gbps (c4.8xlarge)

Need SEC technology pack (BGP routing, IPSEC, VRF-Lite)

Number of connections:

• 100 out-of-the-box (VGW limits)

• 1000s with customized route summarization

*Additional virtual appliances can be added to increase aggregate bandwidth and to create additional network paths using BGP multi-path

…

BRKARC-2749 59

Transit VPC Variations

What if I want to push more throughput to spoke and have traffic visibility?


DMVPN Transit VPC• High Throughput: spoke VPC scales

up to 4.5Gbps, 400K routes on CSR, while 1Gbps, 100 routes on VGW

• Inter VPC Traffic: spoke to spoke communication directly which saves Transit CSR throughput

• Redundancy: two CSRs in spoke VPC acts as high availability pair to provide redundancy

• Application Visibility: provide application level visibility in spoke with NBAR capability on CSR

• Advanced Security: push security policy to edge. Provide ZBFW, IPS and URL filtering

BA C

…...

Direct Connect

Internet

Private DC

Transit VPC

Other

Provider

Networks

ASR

VPCVPCVPC

VPC

Variation #1

IGW

DMVPN

Spoke VPC

CSR1 CSR2

BRKARC-2749 62

What if I want to enable security policy and DIA (Direct Internet Access) ?


Central versus DIA (Direct Internet Access)VM software/OS update and etc.

Central Internet Access

• Leverage existing enterprise internet connection and security perimeter

• All traffic traverses the VPN Tunnel

DIA (Direct Internet Access)

• Optimal access to cloud based resources

• Offload Internet traffic from DX or ER

• Doesn’t lose central security enforcement

VPC-A VPC-B VPC-C

Internet

Private DC

VPC-A VPC-B VPC-C

Internet

Private DC

Transit VPCTransit VPC

Security

Security

BRKARC-2749 64


Integrated Security Features on CSR

ACL VRFZone Based

Firewall

Snort IPSWeb Root

URL Filtering

Umbrella

IPSEC Trust Sec AAA

Support Coming

Variation #2

Transit HubVPC

Integrated Security

• Low TCO by enabling security services

• Built-in high availability with routing

• Single device to manage routing and security

CSR1 CSR2

BRKARC-2749 65


Transit VPC

Internet

• Routing: CSR redirects Internet traffic to NGFWv

• Security: NGFWv as standalone IPS VM provides full IPS

features and easily managed through FMCv

• NAT: NGFWv acts as NAT device. NAT/PAT supported

• Automation: One click Launch by using template and scripts

Variation #3.1

Secured DMZby extending Transit VPC

BA C

…...

Spoke VPC

VPCVPCVPC

VPC

NGFWv (Next Generation FireWall Virtual)

FMCv (Firepower Management Center Virtual)

VGW

IGW

CISCO

VERIFIED

https://www.youtube.com/playlist?list=PLCiTBLSYkcoRREnds3OK8W19seZs5n-Vg

Deployment Video

NGFWv

CSR1 CSR2

BRKARC-2749 66

https://www.youtube.com/playlist?list=PLCiTBLSYkcoRREnds3OK8W19seZs5n-Vg


Transit VPC

Internet

Variation #3.2

Deploy IDS In Passive Mode

VPC

NGFWv (Next Generation FireWall Virtual)

FMCv (Firepower Management Center Virtual)

VGW

IGW

CISCO

VERIFIED

ERSPAN

VPC VPC

• IDS (NGFWv) deployed in Passive Mode

• CSR1000v sends traffic through ERSPAN session

• NGFWv inspects traffic over ERSPAN session passively

• Spoke to spoke traffic is agnostic to IDS device

* ERSPAN= Encapsulated Remote Switch Port Analyzer PortNGFWv

CSR1 CSR2

BRKARC-2749 67


Dedicated Security VPC

Security

VPCA BVPC

VPC

VPC

Private DC

Transit VPC

Internet

FW

IPS

Variation #4

• Separate security services into dedicated VPC

• Network team manages Transit VPC

• Security team manages Security VPC

• No end-to-end automation, manual configuration needed

• Additional Internet traffic cost going to Security VPC.

• Additional hop for latency.

0.0.0.0/0

VPC

VGW

IGW

BRKARC-2749 68


Summary on 5 Variations

Variations

/Features

Hub-Spoke Spoke-Spoke Spoke

Throughput

IOS-XE Features

at Spoke

Cost

#0 Transit VPC Solution 1Gbps Lower

#1 DMVPN Transit VPC 5Gbps Higher

Variations

/Features

L4 FW L7 FW IPS/IDS Routing

Security

Separation

VPC

Domain

Separation

Traffic Latency Cost

#2 Integrated Security Lower Lower

#3 Secured DMZ Medium Higher

#4 Dedicated Security

VPC

Higher Higher

BRKARC-2749 69


Summary on 5 Variations

Variations Pros Cons

#0 Transit VPC Solution • Lower TCO by using VGW on spoke

• Centralized routing domain and security enforcement

• Highly automated

• VGW’s throughput and routes limited at spoke

• No traffic control and visibility at spoke

• Capacity limited by two CSRs’ throughput

#1 DMVPN Transit VPC • Higher throughput at spoke

• Spoke to spoke connection, not limited by transit

CSRs’ throughput

• Full enterprise features including traffic control and

visibility at spoke

• Security policy pushed to edge spoke

• Highly automated

• Higher TCO by using CSR on spoke (price close to

VGW if using for 5 years)

#2 Integrated Security • Lower TCO by leveraging existed features on CSR

• L4 firewall, IPS and URL filtering

• Central security enforcement

• Native high availability on CSR

• Throughput impact based on security features enabled

• No L7 firewall and full IPS functions

#3 Secured DMZ • Advanced security features offered by 3rd party VNF

• Separate VNFs for routing and security

• Shared VPC for routing and security

• Higher TCO by adding 3rd party VNF (FW, IPS or IDS)

• High availability depends on 3rd party VNF

• Throughput limited by 3rd party VNF

#4 Dedicated Security VPC • Advanced security features offered by 3rd party VNF

• Separate VNFs for routing and security

• Separate VPC for routing and security

• Higher TCO by adding 3rd party VNF (FW, IPS or IDS)

• High availability depends on 3rd party VNF

• Throughput limited by 3rd party VNF

• One more VPC to manage and additional traffic cost

BRKARC-2749 70


CSR Subnet

App Subnet A

App Subnet Bvirtual private cloud

Branch 1

Cloud Data Center

Internet

BR1

BR2

MC

APIC-EM

us-westus-east


Branch 2

us-west

csr1000v

csr1000v

csr1000v

csr1000v

csr1000v

Physical branch

MPLS/DX

CSR 1000V IWAN on Amazon AWS

BRKARC-2749 71

CSR Programmability



Cloud, US

West

Cloud Center

Deploy Infra

Public Cloud

APIC EM

SD-WAN

SP Infrastructure

NSO

Function

Pack

Ansible

Devops

Lambda

Guest

Shell

Cloud

Formation

CSR1000v Automation

BRKARC-2749 73


CSR1000v Web GUI

BRKARC-2749 74


Application Visibility on CSR1000v

BRKARC-2749 75


• Guest Shell runs in a LXC container

• It gives you native Linux Shell (Command)

access to run customized scripts

• Access to IOS-XE CLI, boot flash

• Python is the language we support today

• You can install AWS CLI and SDK to

automate day-to-day jobs through scripts

• EEM can be leveraged to create Crontab

tasks calling Guest Shell scripts

• Cisco Devnet Lab http://cs.co/90088m320

Guest Shell

Network OS

Guest Shell

Open Application Container

API

Linuxapplications

BRKARC-2749 76

http://cs.co/90088m320


Enable Guest Shell

• Guest shell uses VPG as source interface and connect to outside through NAT

IOS

VPG

Guest Shell Container

eth0G1

CSR 1000v

interface VirtualPortGroup0

ip address 192.168.35.1 255.255.255.0

ip nat inside

interface GigabitEthernet1

ip address dhcp

ip nat outside

guestshell enable

virtualPortGroup 0 guest-ip

192.168.35.2 name-server 8.8.8.8192.168.35.1 192.168.35.2

ip nat inside source list GS_NAT_ACL interface GigabitEthernet1 overload

ip access-list standard GS_NAT_ACL

permit 192.168.0.0 0.0.255.255

BRKARC-2749 77


Enter Guest Shell

sudo -E pip install awscli

sudo -E pip install boto3

aws configure

or configure ~/.aws/config and ~/.aws/credentials

ip-10-0-0-21#guestshell

[guestshell@guestshell ~]$ pwd

/home/guestshell

[guestshell@guestshell ~]$ ls

scripts

[guestshell@guestshell ~]$ uname -a

Linux guestshell 4.4.51 #1 SMP Wed Mar 22 07:08:50 PDT

2017 x86_64 x86_64 x86_64 GNU/Linux

Same Linux Shell Access Install AWS CLI and Python SDK

BRKARC-2749 78


Use Case #1: Monitor CSR Real-Time Throughput by AWS Cloud Watch• Python script in Guest Shell

• Gather CSR throughput by “show platform hardware qfp active datapath utilization”

• Send key metric to AWS Cloud Watch through AWS python SDK boto3

• EEM(Embedded Event Manager) script

• Trigger python script based on regular time interval

• Visualize throughput on Cloud Watch

event manager applet get-throughput

event timer watchdog time 15

action 0.0 cli command "enable"

action 1.0 cli command "guestshell run

/home/guestshell/get-sys-throughput-fyang2.py"

action 10.0 syslog msg "guestshell-get-throughput

executed!"

BRKARC-2749 79


Use Cases #2: Network Services Zone Failover

VPC

FW FW

EIP

CSR1AZ1 AZ2

Cloud REST API

CSR2

Firewall and IPS EIP failover

• Virtual network functions (router, firewall, IPS and etc) deployed across multiple AZs for redundancy

• In case of AZ failure, all networking functions need to failover to a different AZ

• Hard to push all vendors to have same failover mechanism

• Write your own Python scripts to do seamless failover

IPS IPS

BRKARC-2749 80

Guest Shell Demo

Licensing


CSR 1000v Licensing Structure Example:

Technology Package(See next slide for details)

Throughput License Type

Pick one option from each column…

IPBase

250 Mbps

1-Year

IPBase

10 Mbps

50 Mbps

100 Mbps

250 Mbps

500 Mbps

1 Gbps

2.5 Gbps

5 Gbps

Subscription

(1-year, 3-year or 5-year)

Utility Based

10 Gbps

SEC

AppX

AX

Note: CSR add-on license options not shown above

BRKARC-2749 83


Technology Package IOS-XE Features

IP Base

Basic Networking: BGP, OSPF, EIGRP, RIP, ISIS, IPv6, GRE, VRF-LITE, NTP, QoS, PBR, BFD

Multicast: IGMP, PIM

High Availability: HSRP, VRRP, GLBP

Addressing: 802.1Q VLAN, EVC, NAT, DHCP, DNS

Basic Security: ACL, AAA, RADIUS, TACACS+

Management: IOS-XE CLI, SSH, Flexible NetFlow, SNMP, EEM, NETCONF

SECIP Base Plus…

Advanced Security: Zone Based Firewall, IPSec VPN, EZVPN, DMVPN, FlexVPN, SSLVPN, GETVPN

High Availability: Box-to-box HA for FW and NAT

AppX

IP Base Plus…

Advanced Networking: L2TPv3, MPLS, VRF, VXLAN (Except L3 VXLAN-GPE)

Application Experience: WCCPv2, AppNAV, NBAR2, AVC, IP SLA

Hybrid Cloud Connectivity: LISP, OTV, VPLS, EoMPLS

AX ALL FEATURES

CSR 1000v Technology Package Features

Feature in Red will not work in AWS/Azure – limitation of public cloud infrastructure(lack of L2 support, Multicast not support)BRKARC-2749 84


Flexible Licensing Options on Public Cloud

Cloud Provider Subscription Model BYOL (Bring Your Own License)

1-year, 3-year and 5-yearHourly Annual

AWS

Azure

Purchase

Model

Management

Model

License

Model

UDI Independent 1-click Re-host License Utilization

PAK

Smart

Licensing

No TAC TAC TAC

TAC

Non-convertibleconvertible

BRKARC-2749 85


License Behavior Change

Boot Up

1Mbps after 16.5

100Kbps before 16.5

Licensed

Running at

Licensed throughput

Expiration

Date90days

Sending expiration

Warning Syslog

90days Grace Period

Loss connectivity to SL Server

Keep running at

previous throughput

1 year

Keep running at previous

Throughput

CLI will be blocked

Throttle to

1Mbps or 100Kbps

SL ID_TOKEN

Expires 1 Year

Throughput

Licensed Period + 1 Year

CLI will be blocked

BRKARC-2749 86

Additional Resources


Joint Webinar with Under Armour and Adobe

• Webinar recording on Youtube:

• https://www.youtube.com/watch?v=aLk8ExZ14v8

• Webinar deck on Slideshare:

• http://www.slideshare.net/AmazonWebServices/cisco-csr-1000v-securely-extend-

your-apps-to-the-cloud

BRKARC-2749 88

https://www.youtube.com/watch?v=aLk8ExZ14v8

http://www.slideshare.net/AmazonWebServices/cisco-csr-1000v-securely-extend-your-apps-to-the-cloud


Book: Virtual Routing in the Cloud

Available now at http://cisco.safaribooksonline.com

Virtual Routing in the Cloud, First Edition

By: Arvind Durai, Stephen Lynn, Amit Srivastava

Publisher: Cisco Press

Pub. Date: April 22, 2016

Print ISBN: 978-0-13-413567-0

BRKARC-2749 89

http://cisco.safaribooksonline.com/


Cisco CSR1000V Miercom report: http://miercom.com/pdf/reports/20161111.pdf

• CSR1000V on private cloud platforms delivers up to 20Gbps on a single x86 server, across 3 CSRs

• CSR1000V on Amazon AWS delivers up to 5Gbps of encrypted traffic running on Instance type C4.8xlarge

• Miercom tested different combinations of features enabled to determine real world performance (IPV4 Forwarding, QoS, NBAR, Firewall, IPSEC)

Miercom Performance testing of CSR1000V

Miercom is a world leading independent testing and consultant provider. It

provides unbiased hands-on testing, research and certification services.

BRKARC-2749 90

http://miercom.com/pdf/reports/20161111.pdf


Additional Resources

Free CSR Test Drive Program on AWS https://www.ciscotestdrive.com

Public Documentation:

• 20+ Demo Videos on CSR 1000V Youtube Channel https://www.youtube.com/playlist?list=PLCiTBLSYkcoTUS6b4MFthdvhDrseo6MeN

• CSR 1000V Configuration Guide for AWS http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/aws/b_csraws.html

• CSR 1000V Configuration Guide for Azure http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/azu/b_csr1000config-azure.html

AWS Mailer ([email protected])

Azure Mailer ([email protected])

BRKARC-2749 91

https://www.ciscotestdrive.com/

https://www.youtube.com/playlist?list=PLCiTBLSYkcoTUS6b4MFthdvhDrseo6MeN

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/aws/b_csraws.html

http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/azu/b_csr1000config-azure.html



Key Takeaways


Summary: CSR 1000V is built for the cloud

• CSR 1000V runs industry-leading Cisco IOS-XE software.

• CSR 1000V supports comprehensive networking features to best suit enterprise needs in cloud journey.

• CSR 1000V abstracts different public cloud networking capability and gives customer an unified view of management.

BRKARC-2749 93


• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 gift card.

• Complete your session surveys through the Cisco Live mobile app or on www.CiscoLive.com/us.

Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on demand after the event at www.CiscoLive.com/Online.

http://www.ciscolive.com/us

http://ciscolive.com/Online

http://ciscolive.com/Online

http://www.ciscolive.com/Online


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Related sessions

BRKARC-2749 95

Thank you

Documents

Extending Enterprise Network into Public Cloud with Ciscod2zmdbbm9feqrf.cloudfront.net/2017/usa/pdf/BRKARC-2749.pdf · Extending Enterprise Network into Public Cloud with Cisco CSR1000v