MPLS AWARE IP SERVICES · 2018. 1. 9. · © 2004 Cisco Systems, Inc. All rights reserved. Cisco Internal Use Only 2 MPLS Aware IP Services, 09/04 © 2004 Cisco Systems, Inc. All

1MPLS Aware IP Services, 09/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Internal Use Only

MPLS AWARE IP SERVICES

Andy ChienConsulting System [email protected]

222© 2004 Cisco Systems, Inc. All rights reserved. Cisco Internal Use Only

MPLS Aware IP Services, 09/04 222© 2004 Cisco Systems, Inc. All rights reserved.

SP Managed Services OverviewSP Managed Services Overview

RST-10619776_05_2004_c1


MPLS Aware IP Services, 09/04

SP Managed Service OfferingsR

even

ue

ManagedServices

The Key Is Moving Up the Value Chain by Providing New Services

Co-locationCo-locationManaged Hosting Services

Managed Hosting Services

ManagedApplication

Services

L2/L3 Connectivity

Data Center Space

L2/L3 Connectivity

For VPNsBasic

HostingManaged Security

Managed Network Services

Platform Services

E-Comm App Mgmt

Business Logic

Customer Relation

“MPLS VPNs can offer an entry for selling managed IP services. The clever Service Providers will base their business (and long-term profitability) on value-added services, not exclusively on access.”

Gartner Group, May 17, 2001

“MPLS VPNs can offer an entry for selling managed IP services. The clever Service Providers will base their business (and long-term profitability) on value-added services, not exclusively on access.”

Gartner Group, May 17, 2001



Managed IPT/Video/Desktop

Managed LAN

Managed CPE

Extending the scope of SP ServicesComplementing Connectivity with Value Added Services

ManagedConnectivityNetwork based

SharedServices



MPLS Services OverviewMPLS Services Overview

RST-10619776_05_2004_c1



MPLS Fundamental:Virtualization + A hierarchy of Labels

MPLSMPLSCoreCore

VPN A

VPN B

VPN C

VPN A

VPN B

VPN C

Corelabel

VPN label IP data

VPN label IP data VPN

label IP data

IP data IP data

MPMP--iBGPiBGPoror

LDPLDP



MPLS services on « Hierarchical Network »

MPLSMPLS

INTERNET

Regional Site

LL

INTERNET

IPSec

Core is hidden from EdgeSecurity / Availability

Virtualisation

CEPE

P



MPLS services on « any type of links »

MPLSMPLS

PSTNISDNBranch

Home

Travel

ADSL/Cable

Branch Home

INTERNET

SharedServices

Regional Site

LL

Frame-RelayATM

Ethernet

Remote Sites

INTERNETBranch

Home

TravelIPSec Central

Site

TDMMUX

(Fiber / WDM / POS / Ethernet/ ATM / FR / PPP, Tunnel)



IP-VPN services in « any to any routing »

PSTNISDNBranch

Home

Travel

ADSL/Cable

Branch Home

INTERNET

SharedServices

Regional Site

LL

Frame-RelayATM

Ethernet

Remote Sites

INTERNETBranch

Home

TravelIPSec Central

Site

TDMMUX

MPLSMPLSPrivatePrivate

AnyAny--toto--AnyAnycommunicationcommunication



MPLS services using Label-swaping

MPLSMPLS

PSTNISDNBranch

Home

Travel

ADSL/Cable

Branch Home

INTERNET

SharedServices

Regional Site

LL

Frame-RelayATM

Ethernet

Remote Sites

INTERNETBranch

Home

TravelIPSec Central

Site

TDMMUX

IP aware transportOSPF / IS-IS controlled Transport

Meshed Transport network



MPLS IP-VPN (Virtual Private Network)

MPLSMPLSIPIP--VPNVPN

PSTNISDNBranch

Home

Travel

ADSL/Cable

Branch Home

INTERNET

SharedServices

Regional Site

LL

Frame-RelayATM

Ethernet

Remote Sites

INTERNETBranch

Home

TravelIPSec Central

Site

TDMMUX

Intranet1

ExtranetHosting

Intranet2

InternetMulti-customers on a common IP backbone

PEs auto-discover others via BGP Isolation of core transport versus edge

No more complex OSPF Network for customerAttachement to core is any type



MPLS IP-VPN (VRF-Lite)

MPLSMPLSIPIP--VPNVPN

PSTNISDNBranch

Home

Travel

ADSL/Cable

Branch Home

INTERNET

SharedServices

Regional Site

LL

Frame-RelayATM

Ethernet

Remote Sites

INTERNETBranch

Home

TravelIPSec Central

Site

TDMMUX

Intranet1

ExtranetHosting

Intranet2

Internet

Multi-VPN on a siteVirtual Routing

Applicable on small sitesUsefull in MAN



IP- QoS to application

Central Site

Remote Sites

Remote Sites

Regional Sites

End-to-End SLA mesurement

MPLSMPLSIPIP--VPNVPNL2 VPNL2 VPN

QoSQoS

Hierarchical DiffServ Domain / additional TE for core

End to End QoSApplication level QoS

Per class modelService Level Agreement

QoS transparency



MPLS L2-VPN (L2 transport over MPLS)

Frame-RelayATM

Remote Sites

Frame-RelayATM

Remote Sites

Frame-RelayATM


Regional Site

Ethernet

Central Site

Ethernet

Ethernet

RegionalSite

Central Site

Ethernet

Regional Sites

Optimize existing VC offerHigh-speed Ethernet Leased-line offer

Large site interconnection MANHigh-speed / low-cost IP-VPN aggregation

Complement IP-VPN no implication into customer routing




QOS /QOS / TETE

Traffic-Engineering (Network optimisation)

Central Site

Remote Sites

Remote Sites

Central Site

Ethernet

Regional Sites

Load repartitionFlow path separation (Real time / Critical / BE)

Bandwidth brokerageSub-50ms back-up even in meshed network



MPLS VRF Aware Services –VRF-LiteMPLS VRF Aware Services –VRF-Lite

RST-10619776_05_2004_c1



VRF-Lite - Extending MPLS-VPN

Clients HQ

PERouter

MPLSNetwork

Customer CE or Wholesale

Provider

SubInterfaceLink *

SubInterface Link – Any Interface type that supports Sub Interfaces, FE-Vlan, Frame Relay, ATM VC’s

PERouter

Clients



VRF-Lite - a standalone Virtual-router !

No MPLS, nor MP-iBGP on CENo MPLS, No MPLS, nornor MPMP--iBGPiBGP on CEon CE

Local Inter-VRF routing is supported

PE

VPN Site

(sub)interface associated with another VRF

(sub)interface associated with VRF

CE

VLAN 2VLAN 2

VLAN 1VLAN 1MPLS VPNMPLS VPN



VRF-Lite Architecture

--

CE1

CE

P

P P

VRF X

VRF B

VRF B

VRF A

Customer A Site A1149.27.2.0

Customer B Site B1149.27.2.0

Customer B Site B2149.27.1.0

Customer A Site A2149.27.1.0

VRF A

VRF YVRF D

VRF C

Site Network

Each customer network uses an independent IGP.

Customer Edge

Maintains one VRF per VPNIngress interface used to determine appropriate

VRF

Provider Edge

Maintains one VRF per attached VPNIngress interface used to determine appropriate VRF

MPLS/VPN Network

VRF X

VRF Y

VRF C

VRF D

•Site A1 communicates with Site A2

•Site B1 communicates with SiteB2

•VRF X on CE1 is connected to VRF A on PE1

•VRF Y on CE1 is connected to VRF B on PE1

•VRF C on CE2 is connected to VRF B on PE2

•VRF D on CE2 is connected to VRF A on PE2

•PE1

CE2

PE2



VPN-ASite A1 VPN-A

Site A2149.27.2.0/24

CE-1

PE-1

PE-2

CE-2

VPN-A FIB149.27.2.0/24,

Label Stack {41 28}

P-1

P-1 LFIB149.27.2.0/24 In label {41}

Out label {implicit-null}

149.27.2.2728

PE-1 LFIB149.27.2.0/24 (V)

In label {28} Out label {Untagged}

149.27.2.27

149.27.1.0/24

VPN-BSite B1 VPN-B

Site B2149.27.2.0/24

CE-1

PE-1

PE-2

CE-2

VPN-A FIB149.27.2.0/24,

Label Stack {41 29}

P-1

P-1 LFIB149.27.2.0/24 In label {41}

Out label {implicit-null}

149.27.2.2729

PE-1 LFIB149.27.2.0/24 (V)

In label {29} Out label {Untagged}

149.27.2.27

149.27.1.0/24

149.27.2.27

149.27.2.272841

149.27.2.27

149.27.2.272941

149.27.2.27

149.27.2.27

VRF Y

VRF Y

VRF X

VRF X

VRF D149.27.2.27

VRF D

VRF C VRF C149.27.2.27

VRF B

VRF A

VRF A

VRF B

Data Forwarding in MPLS-VPN with VRF-Lite CE



VRF-Lite CE Architecture: Replaces Separate CE Routers

CE router

PE routerMPLS network

Site 1

Engineering

HR

Finance

CE router

CE router



VRF-Lite CE Architecture: Operational Model

CE - VRF

Client 510.1/24

PE

Client 1

10.1/24Client 2

11.1/24

MPLSNetwork

Client 3

12.1/24

Client 4

13.1/24

One E1 line with MultiplePoint-to-Point Sub-Interfaces

CE-VRF

1. CE-VRF learns Client 1’s VPN Green routes from a sub-interface of the Fast Ethernet interface directly attached to CE-VRF. CE-VRF then installs these routes into VRF Green

PE2. PE 1 learns Client 1’s VPN Green routes from

the CE-VRF and installs them into VRF Green.

Local VPN Blue routes from Client 4 are not associated with VPN Green and are not imported into VRF Green



Application 1: Internet Services and VPN Services Using A Single CE

MPLS Network

VPN- PE2

11.0.0.0/24

VPN- PE3

VPN- PE110.0.0.0/24

VRF REDRD 64512:1RT export 64512:1RT import 64512:1


Central Site

RegionalSite2

RegionalSite1


Internet

Internet - PE2

InternetGateway

Firewall

CE3

CE2

CE1

VRF InternetRD 65000:1

Data forwarding Path from Regional Sites to InternetData forwarding Path from Regional Sites to Internet

Default Route injected into VPNDefault Route injected into VPN

Frame Relay LinkFrame Relay Link

VRF-Lite CEVRF-Lite CE

12

3

4

7

6

5

8 9

12



Application 2: Wholesale Model



Application 3: Integrate Server Farm with Virtual Firewall Services

10.20.1.0/24

Red VPNRed VPN

PEPE--LeftLeft PEPE--RightRight

VRFVRF VRFVRFRed VPNRed VPN

10.20.1.0/24

10.20.4.0/24

Red VPNRed VPN

Red VPNRed VPN

10.20.4.0/24

VRFVRF

VRFVRF

VRFVRF

VRFVRF VRFVRF

VFWVFW VFWVFWFWSMFWSM

Cat6KCat6K

VRFVRF--LiteLite VRFVRF--LiteLite

10.20.3.0/24Red VPNRed VPN

Red VPNRed VPN

10.20.3.0/24

10.20.2.0/24

Red VPNRed VPN

Red VPNRed VPN10.20.2.0/24

Server Farm

Server Farm

FWSM 2.1 Virtual FirewallContexts: Multiple logical FirewallsEach context has its own policies (NAT, ACL, fixups, etc.)FWSM only understands IPv4 - don’t insert between PE to P or P to P routers



MPLS VRF Aware Services –Half-Duplex VRFMPLS VRF Aware Services –Half-Duplex VRF

RST-10619776_05_2004_c1



• Problem PE requires multiple VRF tables for multiple VRFs to push spoke traffic via hub

If the spokes are in the same VRF, traffic will be switched locally and will not go via the hub site

• SolutionHDVs allows all the spoke site routes in one VRF

• BenefitScalability for RA to MPLS connections

Reduces memory requirements by using just two VRF tables

Simplifies provisioning, management, and troubleshooting by reducing the number of Route Target and Route Distinguisher configuration

Why Half-Duplex VRF ?



PE MPLSCORE ISP

ISPHUB

VPNport

VPN port

VPNport

A

B

• Dedicated (separate) VRF per spoke is needed to push all traffic through upstream ISP Hub

Spoke AVRF

Spoke BVRF

CEHUBSite PE

SpokeSite PE

Hub & Spoke Connectivity Without HDV Requires Dedicated VRF Tables Per Spoke

Wholesale Provider



PE MPLSCORE ISPCE

ServiceLoopback

HUB

VPNport

VPNport

VPN port

A

B

• If two subscribers of the same service terminate on the same PE-router, then traffic between them can be switched locally at the PE-router (as shown), which is undesirable

• All inter-subscriber traffic needs to follow the default route via the Home Gateway (located at upstream ISP).

Single VRF table

HubSite PE

SpokeSite PE

Hub & Spoke Connectivity Without HDV Using A Single VRF



• Upstream VRFUsed to forward packets from Spokes to Hub

Contains a static default route

• Downstream VRFUse to forward packets from Hub to Spoke

Contains a /32 route to a subscriber (installed from PPP)

Terminology



PE MPLSCORE ISPCE

HUB

VPNport

VPNport

VPN port

A

B

• If two subscribers of the same service terminate on the same PE-router, traffic between them is not switched locally

• All inter-subscriber traffic follows the default route via the Home Gateway (located at upstream ISP)

Single VRF table

HUBSite PE

SpokeSite PE

Hub & Spoke Connectivity With HDVUsing A Single VRF



1. HDVs are used in only one direction by incoming traffic Ex: upstream toward the MPLS VPN backbone or downstream toward the attached subscriber

2. PPP client dial, and is authenticated, authorized, and assigned an IP address.

3. Peer route is installed in the downstream VRF tableOne single downstream VRF for all spokes in the single VRF

4. To forward the traffic among spokes (users), upstream VRF is consulted at the Spoke PE and traffic is forwarded from a Hub PE to Hub CE

Return path: downstream VRF is consulted on the Hub PE before forwarding traffic to appropriate spoke PE and to the spoke (user)

5. Source address look up occurs in the downstream VRF, if unicast RPF check is configured on the interface on which HDV is enabled

Half Duplex VRF Functionality



1. PPP user initiates a session with PPP session using a name [email protected] and password

2. LAC/PE-router sends username information to the WholesaleServiceProvider Radius Server3. ISP-A (service name) is used to index into a profile that contains information on the IP

address of the Radius server of the ISP-A4. [email protected] and password is then forwarded from the Wholesale Provider

Radius server (which acts as a "proxy-radius"), towards the ISP Radius server5. ISP-A Radius server authenticates and assigns IP address6. ISP-A Radius server sends "Access-Accept" to Wholesale Service Provider Radius Server7. The wholesale Service Provider Radius server adds authorization information to the

Access-Accept, (based on the domain or servicename)and the VRF to be used by Subscriber-A, and forwards it to PE-WholesaleProvider-LAC router

8. PE-WholesaleProvider-LAC router creates temporary Virtual-Access interface (with associated /32 IP address) and places it into the appropriate VRF

PE-WholeSaleProvider-LAC PE-ISP

PPP UserSubscriber-A

Wholesale Service Provider AAA Server

ISP-AAAA Server

MPLS Core

Subscriber Connection Process



!

interface <> ip vrf forwarding <vrf-name1> [downstream <vrf-

name2>]

!

vrf-name1: First VRF that the interface is associated with.

vrf-name2: This is the downstream VRF. PPP peer route and per-user routes from AAA server are installed in this VRF.

Configuration Command



• Reverse Path Forwarding (RPF)Used by Service Provider determine the source IP address of an incoming IP packet and ascertain whether it entered the router via the correct inbound interface

• ConcernHDV populates a different VRF than the one used for “upstream”forwarding

• SolutionExtend the RPF mechanism so the “downstream” VRF is checked

• To enable RPF extension, configure:ip verify unicast reverse-path <downstream vrfname>

Reverse Path Forwarding Check



MPLS VRF Aware Services –VRF NAT for Shared ServicesMPLS VRF Aware Services –VRF NAT for Shared Services

RST-10619776_05_2004_c1



MPLS—VPN Network

VPN “B”

VPN “B”VPN VPN ““AA””

VPN VPN ““AA””

MPLS/VPN: Before Managed Shared Services

• Services need to be replicated per VPNPoor efficiencyHigh Traffic LoadManagement nightmare

Services for VPN AServices for VPN A Services for VPN B

ERP

InternetGateway

VideoServer

HostedContent

H.323Gatekeeper

ERP

InternetGateway

VideoServer

HostedContent

H.323Gatekeeper



MPLS/VPN: Supporting Shared Services

Internet

Shared Services for All VPNs

InternetGateway

VoIPGateway

ERP VideoServer

HostedContent

• IP services move into Service Provider network and become sharable

Increases enterprise outsourcing flexibilityCreates new Service Provider revenue opportunities

Internet Connectivity Options

Cisco MPLS—VPN Network

VPN “B”VPN VPN ““AA””

VPN “B”

VPN VPN ““AA””

PSTN



Shared Services

OUTSIDE12.10.X.0

NAT PE

NAT & MPLS VPN for Shared Services

Internet

CE-A210.88.2.0 10.88.3.0

10.88.2.0

CE-B2

VRF-A VRF-B

MPLS-VPN

INSIDETAG INTERFACE

CE-B2VRF-B

VRF-B

VRF INSIDE OUTSIDE

B 10.88.3.1 172.0.1.2

A 10.88.1.1 172.0.0.1B 10.88.1.1 172.0.1.1

CE-A110.88.1.0

CE-B110.88.1.0

CE-B3

VRF-BVRF-A



Implementation with multiple NAT pools

B

Ethernet 0

outside if

MPLS Backbone

NATip nat pool pool1 172.0.0.1 172.0.0.254 mask 255.255.255.0ip nat pool pool2 172.0.1.1 172.0.1.254 mask 255.255.255.0ip nat inside source list 1 pool pool1 vrf Aip nat inside source list 1 pool pool2 vrf B

Routingip route vrf A 172.0.3.0 255.255.255.0 172.0.3.1 globalip route vrf B 172.0.3.0 255.255.255.0 172.0.3.1 global

Interfaceinterface ethernet0ip nat outsideinterface serial1ip nat insideinterface serial2ip nat inside

Serial 1

Inside

NAT PE

A



MPLS VRF Aware Services –HSRP for IP Edge Redundancy MPLS VRF Aware Services –HSRP for IP Edge Redundancy

RST-10619776_05_2004_c1



IP Redundancy for the Provider Edge

MPLS-VPN

CE-A1 CE-A2CE-B1

CE-B2

CE-B3

VRF-B

VRF-A VRF-B

VRF-BVRF-A

NAT PENAT PE

VRF-B

VRF-B

VRF-A

VRF-A

HSRP/GLBP/VRRP

Internet

Shared Services

10.2.1.010.2.1.0 10.2.2.0 10.2.3.0

10.2.2.0

10.2.3.0

10.2.4.0



IP Redundancy – HSRP Example

VRF-A VRF-B VRF-A VRF-B

VRF-A vIP: 10.2.0.20 VRF-A vIP: 10.2.1.20

GW: 10.2.0.20 GW: 10.2.1.20

e0 e0PE1 PE2



MPLS VRF Aware Services –DHCP/DHCP Relay MPLS VRF Aware Services –DHCP/DHCP Relay

RST-10619776_05_2004_c1



Why DHCP Relay for MPLS VPNs?

• Assign IP Addresses from shared DHCP service • Addresses are assigned per subnet, per VRF • The DHCP Server requires VPN information be included in

DHCP requests• DHCP Relay uses the VPN identifier sub option• The VPN identifier (sub option) also allows any DHCP reply to

be properly forwarded back to the relay agent• VRR/VPNID support in V5.5 CNR



InternetMPLS-VPNDHCP Relay

Agent

DHCP-Relay for MPLS-VPNs

CE-A1

CE-A2

CE-B1 CE-B2 CE-B3

VRF-BVRF-A VRF-B VRF-B

VRF-A

10.88.1.010.88.1.0 10.88.2.0

10.88.2.0

10.88.3.0

VRF-B

VRF-B

VRF-A

VRF-A

Corporate DHCP Server

DHCP?

DHCP 10.88.8.1DHCP 10.88.8.1

10.88.8.1

DHCP+ VRF-A

VRF-A

• End station makes DHCP Request• DHCP Relay Agent notes VPN info and

forwards request to correct server• Server assigns address and replies



Internet

10.88.8.1

DHCP-Relay for MPLS-VPNs - Shared

MPLS-VPN

CE-A1 CE-A2CE-B1

CE-B2

CE-B3

VRF-B

VRF-A VRF-B

VRF-BVRF-A

NAT PENAT PE

10.88.1.010.88.1.0 10.88.2.0

10.88.2.0

10.88.3.0

VRF-B

VRF-B

VRF-A

SP SharedDHCP Server

DHCP?

VRF-ADHCP+DHCP 10.88.8.1

10.88.8.1

DHCP RelayAgent

VRF-A

VRF-B

10.88.8.110.88.8.1

VRF-A

• End station makes DHCP Request• DHCP Relay Agent adds VPN info• Server assigns address based on option 82 data and replies



MPLS VRF Aware Services –ODAP MPLS VRF Aware Services –ODAP

RST-10619776_05_2004_c1



On Demand Address Pools (ODAP)

• ODAP Manager allows pools of IP addresses to be dynamically increased or reduced in size depending on the address utilization level

• ODAP supports address assignment using the DHCP for customers using private addresses

• Each ODAP is configured and associated with a particular MPLS VPN

• Works with Cisco Network Registrar (CNR) 5.5 (DHCP) and/or Access Registrar 1.7 (RADIUS)



Why ODAP for MPLS VPNs?

• Automate assignment of IP Addresses from shared DHCP server or RADIUS server

Upon configuration, pool manager requests initial subnet from server

Addresses are assigned per subnet, per VRFPool manager monitors utilization of pool and expands as necessary

• DHCP Option 82 sub options used to communicate necessary VPN information

• The VPN identifier also allows replies to be properly forwarded back to the relay agent



MPLS VPN ODAP Details

• Support for DHCP clients and PPP sessions on per interface basisODAP Manager feature allows the DHCP server to distinguish between a

normal DHCP address request and a request from a PPP clientUseful for router auto-install and Layer 2 attached networks

• ConfigurationSet initial pool sizeHigh/Low utilization mark (% of pool)Expansion/Contraction increment

• Monitor function expands and contracts address pool as needed• Appropriate routes added to VRF tables in PE routers as needed

ip dhcp pool green_poolvrf Greenutilization mark high 60utilization mark low 40origin dhcp subnet size initial /24 autogrow /24



• PE router is configured for ODAP• PE router is configured for ODAP• ODAP requests initial pool for VRF-A from server• PE router is configured for ODAP• ODAP requests initial pool for VRF-A from server• CE router is installed and PPP link established to PE router

ODAP for MPLS-VPNs:Provisioning and Startup

MPLS-VPN

DHCP?

IOS DHCPServer

PEPE

• PE router is configured for ODAP• ODAP requests initial pool for VRF-A from server• CE router is installed and PPP link established to PE router• CE router uses DHCP Proxy to obtain addresses for downstreamdevices

VRF-A

VRF-B

CE-A110.88.1.0

CE-B110.88.1.0

use 10.88.1.0/25

ODAP

DHCP+ VRF-A

DHCP (CNR r5.5) or RADIUS

Server

10.88.1.0/2510.88.1.128/25

DHCP 10.88.1.114DHCP 10.88.1.114



• ODAP requests initial pool for VRF-A from server• ODAP requests initial pool for VRF-A from server• End station makes DHCP Request• ODAP requests initial pool for VRF-A from server• End station makes DHCP Request• DHCP Server fulfills request from pool – reaches 90%

• ODAP requests initial pool for VRF-A from server• End station makes DHCP Request• DHCP Server fulfills request from pool – reaches 90%• ODAP Pool Manager requests expansion

• ODAP requests initial pool for VRF-A from server• End station makes DHCP Request• DHCP Server fulfills request from pool – reaches 90%• ODAP Pool Manager requests expansion• Server allocates another subnet and replies

ODAP for MPLS VPNs: Address Pool Management

MPLS-VPN

NAT PE

IOS DHCPServer

• ODAP requests initial pool for VRF-A from server• End station makes DHCP Request• DHCP Server fulfills request from pool – reaches 90%• ODAP Pool Manager requests expansion• Server allocates another subnet and replies• PE adds subnet routing information to VRF

10.88.1.0/2510.88.1.128/25OK, use 10.88.1.128/25OK, use 10.88.1.128/25

PEPEVRF-A

VRF-B

CE-A110.88.1.0

CE-B110.88.1.0

DHCP (CNR r5.5) or RADIUS

Server

Give me a subnet for VRF-A

DHCP?

DHCP 10.88.1.114DHCP 10.88.1.114

DHCP+ VRF-A



MPLS VRF Aware Services –VRF Select MPLS VRF Aware Services –VRF Select

RST-10619776_05_2004_c1



Why VPF Select?

• Allows access providers to map DSL/Cable customers to any ISP that provides VPN capabilities

• Allows remote users to connect to VPNs, irrespective of access provider.



How VPN Select Works

• De-couple the association between VRF and an interface and populate a source IP address table used to select VRF

• VRF selection is performed at the ingress interface on the PE router

• Use a two-table lookup mechanism at the ingress interface of the PE router. Perform

1. ‘Criteria Selection’ table look up to select a VRF table

2. Look up the destination IP address of the packet on the selected VRF table to determine the output int. & adjacency



VRF Select – Deployment Scenario

20.1.1.1

30.1.1.1

40.1.1.1

ISP1 owns 20.x.x.x network



PE

PE

CE

CE

CE

Broadband access network

MPLSMPLSVPNVPN

VPN1

VPN2

VPN3

• VRF Select decouples the interface with a VRF• The VRF Selection will be based on the source

address of the incoming traffic

VPN1vrf

VPN2vrf

VPN3vrf

PE



Equal Access Network - Single VLAN withPBR Architecture

PC

TV

PC

TV

Customer 1

802.1Q

ISP #1

ISP #2

MPLSNetwork

PE Router

Aggregation andPE Router

AccessSwitch

MPLS VPNsTraffic To/From

Subnet 1

Traffic To/FromSubnet 2

Traffic From/ToSubnet 3

IP Addr FromSubnet 3

Voice Services

Infrastructure

Traffic From/To

Subnet 5

PBRBased onSource IP(VRF Select)

RGWPCPC

TVTV

PCPC

TVTV

Customer 1

Customer 2

802.1QTrunk

ISP #1

ISP #2ISP #2

MPLSNetwork Access

(U-PE)

(N-PE)

RGW

STB

STB








MPLS VRF Aware Services –IPSecMPLS VRF Aware Services –IPSec

RST-10619776_05_2004_c1



Why VRF-Aware IPSec?

• Enterprises are looking to expand their IPSec VPNs to geographically separate locations for internal or outsourced services

• Reduces two box solution to one box solution

• Provide additional security to MPLS VPN traffic1. Protect critical data

2. Selected VPN sites that might be crossing multiple Service Providers

3. Support off-net remote access over the Internet

-Site to site

-Broadband user connections

-Dial-In, mobile user connections



Corp ASite 1

Corp ASite 2

Corp ASite 3

Corp ASite 4

Corp ASite 5

IPSec 3DES/AES Encrypted Tunnels

PE

PE

PE

IPSec and

MPLS PE

Corp BSite 1

Corp BSite 1

Corp BSite 2

Corp BSite 2

PE

Internet

PE

IPSec Off-Net Service for Multiple MPLS VPNs

Company Confidential

MPLSNetwork



Cisco IOSMPLS

PE

Leased Line/Frame Relay/ATM/

DSL Dedicated Access

InternetCable/DSL/ISDN ISP

Local or Direct-

Dial ISP MPLSMPLS

Cisco VPN Client Software Is Tunnel Cisco VPN Client Software Is Tunnel Source for Access VPNs and BranchSource for Access VPNs and Branch--Office; Router Originates SiteOffice; Router Originates Site--toto--Site Site

Tunnel with VPN ConcentratorTunnel with VPN Concentrator

Cisco Router Terminates IPSec Tunnels and Maps Sessions into

MPLS VPNs

21223*228IPSec SessionIPSec SessionIP IPMPLS VPNs VLANsVLANs

Remote Users/ Telecommuters

MPLS CoreCorporate

IntranetBranchOffice

Access/Peering PoPs

MPLS VPNsMPLS VPNs

VLANsVLANsBi-Directional IPSec SessionBi-Directional IPSec Session

Cisco IOS IPSec + MPLS PE Single box Solution



VRF-Aware IPSec Key Elements

• VRF instance• MPLS distribution• Key rings:

Are requiredThey store keys belonging to different VRFsIKE exchange is authenticated if the peer key is present in the keyring belonging to the FVRF of the IKE SA

• Front door VRFLocal endpoint (or outer IKE source/destination) of the IPSec tunnel belongs to the FVRF

• Inside VRFThe source and destination addresses of the inside packet belongto the IVRF



VRF-Aware IPSec Packet Flow

Packet Flow From an IPSec Tunnel1. An IPSec-encapsulated packet arrives at the PE router from

the remote IPSec endpoint

2. IPSec performs the Security Association (SA) lookup for the Security Parameter Index (SPI), destination, and protocol

3. The packet is decapsulated using the SA and is associated with IVRF

4. The packet is further forwarded using the VRF routing table

Corp ASite 1

Corp ASite 2MPLS Core

PE PE

FVRF




Corp ASite 1

Corp ASite 2MPLS Core

PE PE

VRF-Aware IPSec Packet Flow

Packet Flow INTO an IPSec Tunnel1. A VPN packet arrives from the Service Provider MPLS

backbone network to the PE and is routed through an interface facing the Internet

2. The packet is matched against the Security Policy Database (SPD), and the packet is IPSec-encapsulated; the SPD includes the IVRF and the access control list (ACL)

3. The IPSec-encapsulated packet is then forwarded using the VRF routing table

IVRF




MPLS VRF Aware Services –IOS Firewall MPLS VRF Aware Services –IOS Firewall

RST-10619776_05_2004_c1



Why VRF-Aware Cisco IOS Firewall?

• Virtualizes Cisco IOS FW components

• Offers single box solution reducing CAPEX/OPEXSP can offer per VPN customized FW services in addition to VPNs

Includes support for all the options as in non-VPN Cisco IOS FW

Distributed or non-distributed models are supported

• Allows SP to offer managed FW services to protect customer intranet, extranet, VPNs, shared services segment



Shared Service

MPLS Cloud

PE3

Site A

CE

Site A

CE

Site B CE

PE2PE1

VPN Firewall (VPN1-FW, VPN2-FW)

Shared Service Firewall (SS- FW)

VPN FW Protects VPN

SS FW Protects SS

VRF-Aware Cisco IOS FirewallDistributed Model



Shared Service

MPLS Cloud

PE3

Site A

CE

Site A

CE

Site BCE

PE2PE1

VPN Firewall (VPN1-FW, VPN2-FW)

Shared Service Firewall (SS-FW)

SS FW Protects SSc

VPN FW Protects VPN

VRF-Aware Cisco IOS FirewallHub-and-Spoke Model



VRF-Aware Cisco IOS FirewallConfiguration

1. Define firewall rules for VPN and shared services… etc.

ip inspect name <policy> vrf <vrf name>ip inspect name bank-vpn-fw vrf bank

2. Apply this rule to in/out on a VRF interface

interface Ethernet0/1.10description VPN Site Bank(CE) to PE1ip inspect bank-vpn-fw in



MPLS VRF Aware Services –Ping and TracerouteMPLS VRF Aware Services –Ping and Traceroute

RST-10619776_05_2004_c1



Ping for MPLS VPNs

•VRF aware ping to verifyConnectivity between PE routersConnectivity between CE-PE routersTraffic stays within the corresponding VPN

Pinging CE’s loopback address

PE2# ping vrf red 222.2.2.1

Type escape sequence to abort.Sending 5, 100-byte ICMP Echos to 222.2.2.1, timeout is 2 seconds! ! ! ! !Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

Note: Use IP ping for CE-CE ping tests.



Traceroute for MPLS VPNs

•To verifyTransport addressesPath traversed for a VPNTraffic stays within the corresponding VPN pathTraceroute from a PE to a CE(loopback address)

PE2# traceroute vrf red 222.2.2.1

Type escape sequence to abort.Tracing the route to 222.2.2.1

1. 111.0.1.17 4 msec 0 msec 4 msec2. 111.0.1.101 0 msec 0 msec 0 msec3. 111.0.1.102 0 msec 0 msec 0 msec

* Note: Backbone routers must be configured to propagate & generate IP TTL.



Traceroute CE-CE

CE RouterLoop 10.1.1.1Serial 100.200.4.2

CE RouterLoop 3.1.1.1

CE RouterLoop 3.2.2.2Serial 100.200.5.2

PPE

PE

MPLS CORE

CE-Router> tracerotue 3.1.1.1

Type escape sequence to abort.Tracing the route to 3.1.1.11. 100.200.5.1 0 msec 0 msec 0 msec2. 100.200.2.2 4 msec 4 msec 4 msec

Serial100.200.5.1

100.200.2.2

100.200.2.1

Notice how CE-CE traceroute works in MPLS VPN environment

VRF A

VRF A



MPLS VRF Aware Services –Others MPLS VRF Aware Services –Others

RST-10619776_05_2004_c1



VRF-Aware Cisco IOS FirewallConfiguration

1. VRF Aware DNS

2. VRF Aware SNMP

3. VRF Aware Syslog

4. VRF Aware AAA

5. VRF Aware SAA

6. VRF Aware TACAS+

7. VRF Aware RADIUS



MPLS Architectural Security Current Status and Standards UpdateAndy ChienConsulting System [email protected]



MPLS Architectural Security Attributes

797979© 2003 Cisco Systems, Inc. All rights reserved.Presentation_ID



Inherent Security Attributes

• Addressing and Routing separation

• Resistance to Label Spoofing



Addressing and Routing Separation

• Use of different virtual routing/forwarding (VRF) instances on the PE for each customer or group of customer sites connected to the PE

• VRF context aware for learned routes

• Multiprotocol BGP is NOT VPN aware its primary function is to distribute customer routes between PE routers



The Principle: A “Virtual Router”

!ip vrf Customer_Ard 100:110route-target export 100:1000route-target import 100:1000

!interface Serial0/1ip vrf forwarding Customer_A

!

Virtual Routing and Forwarding Instance Route Distinguisher:

Makes VPN routes unique

Export this VRF with community 100:1000

Import routes from other VRFs with

community 100:1000

Assign Interface to “Virtual Router”



Address Space Separation

Route Distinguisher IPv4 Address

VPN IPv4 Address

64 bits 32 bits

Within the MPLS core all addresses are unique due to the Route Distinguisher



Routing Separation Fundamentals

• Each (sub-) interface is assigned to a VRF

• Each VRF has a RD (route distinguisher)

• Routing instance: within one RD -> within one VRF

-> Routing Separation



Visible Address Space

Hiding of the MPLS Core Structure

• VRF contains MPLS IPv4 addresses• Only peering Interface (on PE) exposed (-> CE)!

-> ACL or unnumbered• No mpls ip propagate-ttl forwarded on PE (mitigate traceroute

results)

PEMPLS core

IP(PE; l0) P

CE2IP(CE2) IP(PE; fa1) VRF CE2

CE1IP(CE1) IP(PE; fa0) VRF CE1

P

P P



Resistance to Label Spoofing

• Label spoofing is the ability of the upstream router to replace or insert a label into a packet that was not originally allocated by the downstream router

• PE router expects IP packet from CE

• Labelled packets will be dropped

• Thus no spoofing possible

• Cisco router does not accept labelled packets on an interface that is NOT enabled for label switching

• CE router can spoof source or destination address before packet arrives at the PE, but this would only affect the customer’s own VPN (address separation attribute)

• Customer would be spoofing self



Word on Static Labels

• Available 12.0 (23)S onward

• Permits static bindings to be configured between labels and IPv4 prefixes

• Allows provisioning of static cross-connects in the mid-point of a label switched path (LSP)

• Cisco IOS does not permit label for a prefix to be modified by using static commands if an LDP peer has previously provided a label

• CsC discussed further in this presentation



Finally: What is NOT Separated

• One CPU, one memory, handling many VRFs, many routing processes

• If one VRF uses all CPU/memory resources, other VRFs will be affected

Separation under DoS? Not really!

Separation against Intrusions? Yes, that works!!!



True Virtualisation?

• “True” solution: Full virtualisationEvery process (VRF) get assignments of CPU/memory

CPU controlled

• Current “Workarounds” (not that bad actually!):Max-route limit, routing security (MD5), general security (e.g. no SNMP allowed)



Comparison to ATM/FR Networks




Layer 2 Comparison Context

• VPNs delivered via Layer 2 point-to-point connections such as ATM, Frame Relay

• Address and routing separation in MPLS-VPN architecture is equivalent to Layer 2 models

• MPLS-VPN service provider core network is invisible to a customer network, as is a customer network to the core network

• An MPLS-VPN network is resistant to DoS attacks as a Layer 2 network



Non-IP networks: Not 100% secure!!Example: Telephone Network

“I had access to most, if not all, of the switches in Las Vegas,” testified Mitnick, at a hearing of Nevada's Public Utilities Commission (PUC). “I had the same privileges as a Northern Telecom technician.”Source: http://online.securityfocus.com/news/497



Non-IP networks: Not 100% secure!!Example: ATM Switch

“a single 'land' packet sent to the telnet port (23) of either the inband or out-of-band interface will cause the device to stop responding to ip traffic. Over the course of 6-1/2 minutes, all CPU will be consumed and device reboots.”Source: Bugtraq, 15 June 2002: “Fore/Marconi ATM Switch 'land' vulnerability”, by [email protected];



Comparison with ATM / FR

ATM/FR MPLSAddress space separation yes yes Routing separation yes yes Resistance to attacks yes yes Resistance to Label Spoofing

yes yes

Direct CE-CE Authentication (layer 3)

yes with IPsec



Direct CE-CE Authentication

• On ATM/FR: You can “see” the other CE It is layer 2, so for example CDP works.

• On MPLS: You are peering with a “cloud”No direct visibility of other CEs

• This is a Feature, not a Bug!Key advantage of MPLS: No n2 problem of direct CE-CE peerings!For Security, need to be cognizant of issues we will discuss throughout this workshop



Problem without CE-CE Authentication

• If SP accidentally puts a new CE into a wrong VPN, security of this VPN is compromised!!

Just requires wrong VRF info for an interface!!

Easy mistake to make!

The intruded VPN will not notice that!!! (the CE will)

• In Practice: Need to configure additional security: E.g. routing MD5

Mostly done by provisioning tools Less error-prone

If accidental, it is unlikely that CE has bad intention

If malicious, … bad luck for VPN!!! (Would need IPsec)



Summary: Architectural Security

• MPLS can be equally secured as ATM/FR (this is pretty much industry wide acknowledged)

• If MPLS is misconfigured security problemsCustomer needs to trust SP

But: The same applies to ATM/FR really…

• Cisco believes MPLS is good for Enterprise VPNsWe are using it in EMEA, without IPsec on top!

Good show-case!!



Inter-As and CsC Considerations




From RFC2547bis: Data Plane Protection

• Inter-AS should only be provisioned over secure, private peerings

• Specifically NOT: Internet Exchange Points (anyone could send labelled packets!! No filtering possible!!)

1. a backbone router does not accept labeled packets over a particular data link, unless it is known that that data link attaches only to trusted systems, or unless it is known that such packets will leave the backbone before the IP header or any labels lower in the stack will be inspected, and …



From RFC2547bis: Control Plane Protection

2. labeled VPN-IPv4 routes are not accepted from untrusted or unreliable routing peers,

• Accept routes with labels only from trusted peers

• Plus usual BGP filtering (see ISP Essentials*)



mbehring

Inter-AS: Case 10.a)VRF-VRF back-to-back

• Control plane: No signalling, no labels

• Data plane: IPv4 only, no labels accepted

• Security: as in 2547

• Customer must trust both SPs

Cust. Cust.AS 1 AS 2CE CE

PE ASBR PEASBR

IP dataLSP LSP



Security of Inter-AS 10.a)

• Static mappingSP1 does not “see” SP2’s network

And does not run routing with SP2, except within the VPNs.

Quite secure

• Potential issues: SP 1 can connect VPN connection wrongly(like in ATM/FR)



mbehring

Inter-AS: Case 10.b)ASBR exchange labelled VPNv4 routes

• Control plane: MP-BGP, labels

• Data plane: Packets with one label

• AS1 can insert traffic into any shared VPN of AS2



PE ASBR PEASBR

VPN label IP data

MP-BGP+labels

LSP LSP



Security of Inter-AS 10.b)

• ASBR1 does signalling with ASBR2MP-BGP: has to be secured, dampening etcOtherwise no visibility of the other AS (ASBR1 – ASBR2 is the only interface between the SPs.)

• Potential Issues:SP1 can bring wrong CEs into any shared VPNSP1 can send packets into any shared VPN (not into VPNs that are not shared, since label is checked);

SP can make any shared VPN insecure



mbehring

Inter-AS: Case 10.c)ASBRs exchange PE loopbacks

• Control plane: ASBR: just PE loopback + labels; PE/RR: VPNv4 routes + labels

• Data plane: PE label + VPN label

• AS1 can insert traffic into VPNs in AS2



PE ASBR PEASBR

LSP

PE loopb+labels

VPN IP dataPE label

VPNv4 routes + labels



Security of Inter-AS 10.c)

• ASBR-ASBR signalling (BGP)RR-RR signalling (MP-BGP)

Much more “open” than 10.a) and 10.b)

LSPs between PEs, BGP between RR, ASBR

• Potential Issues:SP1 can bring a CE into any VPN on “shared” PEs

SP1 can intrude into any VPN on “shared” PEs

• Very open architectureprobably only applicable for ASes controlled by the same SP.



Inter-AS Summary and Recommendation

• Three different models for Inter-ASDifferent security properties

Most secure: Static VRF connections (10.a), but least scalable

• Basically the SPs have to trust each otherHard / impossible to secure against other SP in this model

• Okay if all ASes in control of one SP

• Current Recommendation: Use 10.a)



Inter-AS Recommendation

• Start with 10.a) (static VPN connections)Not many Inter-AS customers yet anyway Easy start

• Maybe at some point (when many Inter-AS customers), move to 10.b) (ease of provisioning)

• 10.c) felt by most SPs as too open. Current recommendation: Only when both ASes under one common control



Carrier’s Carrier

• Same principles as in normal MPLS

• Customer trusts carrier who trusts carrier

Carrier’sCarrierCust. Cust.Carrier Carrier

CE CEPE

PE

PE

PEPE PE

IP

label

label

data

IP data

label IP data

label IP data

IP data



Carrier’s Carrier: The Interface

• Control Plane:PE1 assigns label to PE2

• Data Plane: PE1 only accepts packets with this label on this i/f

PE1 controls data plane

No label spoofing possible

Carrier’sCarrierCarrier

PE2 PE1



Carrier’s Carrier: Summary

• Can be secured wellCarrier has VPN on Carrier’s Carrier MPLS cloud

Carrier cannot intrude into other VPNs.

Carrier can mess up his own VPN (VPNs he offers to his customers)

• End customer must trust both SPs.


MPLS Aware IP Services, 09/04 112112112MPLS Aware IP Services, 09/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Internal Use Only


QoS Service Model & Case Study

Andy ChienConsulting System [email protected]



• 4 QoS levels with differentiated SLAs will be offered :Real Time (Highest) CoS: Used primarily for voice and video traffic

Interactive CoS: Used for time sensitive business data

Business CoS: Used for high priority business data

Default (Lowest) CoS: Used for all other applications

• However, billing is based on the aggregated Service Contract Rate plus the premium charge for Real Time class/Interactive/Business. The PE router is also required to keep track the customer’s per-CoS statistics.

• The SLA specified below is between PEs in different POPs. However, PE ingress/egress port will be excluded

N/AN/AN/AN/AN/ADefault

N/A<40ms99.90%99.99%99.95%Business

<12ms<40ms99.90%99.99%99.95%Interactive

<8ms<40ms99.95%99.99%99.95%Real Time

JitterLatencyPacketDelivery

NetworkAvailability

ServiceAvailability

Class Name

QoS Levels & SLAs



0%20%40%60%100%Interactive

100%60%30%20%0% Real Time

0%20%20%20%0%Business

Voice and Video

only

(QoS SP 5)

Voice and Video

(QoS SP4)

Business Critical

(QoS SP 3)

Business General

(QoS_SP2)

General

Data

(QoS_SP 1)

Service CategoriesIP QoS Class

• 5 Service Categories are proposed by marketing.

• Different QoS Service Profiles may be subscribed in different locations of the customer to meet the customer’s specific needs.

• Recommend SP 3 for Big Enterprise

Need to reserve the minimum BW for routing and data.Default can use unallocated Bandwidth

QoS Service Profile



• QoS product will support 4 classes of service, (RealTime, Interactive, Business and Default) each with it’s own SLA

• Customers shall mark their traffic according to SP defined 4 QoS levels

• The following parameters need to be specified in the service contract:Type of access technology (ATM, FR, PPP or Ethernet)

Access line rate at physical layer

Aggregated Service Contract Rate at Layer 2

The Service Contract Rate will be symmetric in incoming and outgoing direction from a VPN site.

Since the customer with lease line is allowed to send traffic at the line rate, no separate Service Contract Rate needs to be specified in the service contract.

The same service profile applies to both directions.

QoS Service Model



• The customer CE router shall perform per-interface based L2 shaping to ensure that the traffic sent to SP conforms to the Service Contract Rate.

• L2 policing will be performed at ingress of L2 network based on the Service Contract Rate. Non-conforming traffic will be dropped. The PE router will perform L2 shaping at egress to ensure that the traffic going to the customer site conforms to the service contract.

• L3 Traffic policing will be applied to RealTime/Interactive/Business traffic coming to and leaving from the MPLS network, and the non-conforming traffic will be dropped.

Policing will be done at ingress and egress of PE router based on RealTime/Interactive/Business service rate.

L2 overhead will be subtracted when L2 service rate is converted to L3. There is no perfect formula for bandwidth conversion between L2 and L3. Our recommendation is that we should always leave some extra room for this conversion

QoS Service Model



• Customer CoS transparency will be preserved across SP network using MPLS Short Pipe mode.

At edge, customer DifferServ marking will be mapped to MPLS EXP bits. The QoS functions in the SP network will be honored based on EXP bits.

• The customized mappings can be also supported in the future to accommodate the customer’s specific needs.

For example, if a customer wants to achieve the Real Time class SLA for their Interactive/Business and Default class, and if they don’t want to change their existing marking scheme, a customized mapping needs to developed to map all traffic to the Real Time Class.

QoS Service Model



• For each VPN site, the customer needs to specify the following parameters in the VPN service order.

Access Line Rate (X): It refers to the

physical layer.

Aggregated Service Contract Rate (Y):L2 Service Contract Rate.

Real Time Class Rate (Z): 20% * L2 Service Contract Rate.

For Lease Line, a customer can send traffic at line rate (Service Contract Rate = Access Line Rate)

• The L2 overhead will be subtracted when the above L2 traffic parameters are converted to L3 parameters

AccessCircuit or LL

rate = x

Real Time Class

AggregateShapedrate = y

Interactive/Business

Real Timerate = z

QoS Service Contract



Use any-to-any SLA to replace traditional L2 Point-to-Point SLAReal Time traffic shall be limited to the Service Contract for Real Time Class at

ingress and egress.For the service profile allowing 100% Real time, at least 5% bandwidth shall be

allocated for customer control and critical business data.

MPLS VPN Core

VPN_A site 1

VPN_A site 2

ISR= Ingress Service RateESR= Egress Service Rate

Aggregated ISR = 5 M bps

ISR for Real Time = 2 M bps

Aggregated ESR = 5 Mbps

ESR for Real Time = 2 Mbps VPN_A site 3

VPN_A site 4

QoS Services : Any-to-Any SLA



• Flexible, application-driven SLAs require sophisticated QoS at edge of the network.

• With over-provisioning at the core network today, only very simple QoS scheme may be required in the core.

• When the over-provisioning model is evolving to the right provisioning model to reduce the transport cost by increasing core link utilization, QoS complexity may need to be increased to achieve end-to-end QoS for the customer applications.

Core

Nee

d Fo

r QoS

Com

plex

ity

Edge Edge

Now Future

QoS Requirements for Network Edge and Core



• A decoupled IP DifferServ model is proposed to achieve a clear DifferServ Domain and QoS Policies separation between customers, network edge and network core.

• At MPLS core, a consistent MPLS QoS scheme can be supported, based on MPLS EXP bits.

• Customer QoS policies could be preserved transparently across the provider’s network.

PE

VPN-A VPN-A

Network Edge IP DiffServ Domain

CE PE CE

Customer IP DiffServ Domain

ConvergedMPLS Core

MPLS Core DifferServ Domain

QoS Services over the Common MPLS Infrastructure



• A consolidated MPLS QoS marking framework is proposed to support all SP QoS products.

Up to 4 SP MPLS QoS levels can be supported inside the core, independent of QoS levels supported at edge.

MPLS Class 1 (Real Time)

MPLS Class 2 (Premium Data)

MPLS Class 3 (Normal Data)

MPLS Class 4 (Default)

• MPLS label reserves 3 EXP bits for packet marking.

EXP 6,7 for IP routing protocols and management data.

EXP 0-5 are used to mark the customer traffic.

• With this common MPLS QoS Marking Scheme, The edge QoS class that is specific to one SP service

can be mapped to 4 common MPLS QoS Classes in the Core.

Up to 4 queue levels could be supported using this marking scheme in the core.

000

010

001 (For exceeding traffic)

100

011 (For exceeding traffic)

101

110 (Reserved for SP

111 (Reserved for SP internal)

MPLS EXP

MPLS QoS Class 4 (Default)

MPLS QoS Class 3 (Normal Data)

MPLS QoS Class 2 (Premium Data)

MPLS QoS Class 1

(Real Time)

To be defined in the future

To be defined in the future

Core MPLS QoS Class

IPP =0

(Customer Default)

IPP = 2

(Customer Business)

IPP = 7,6,4 (Customer Interactive)

IPP = 5 (Customer Real Time)

SP Internal Control Data

SP Internal Control Data

Edge QoS + RP/NM

MPLS QoS Marking Framework



Service Product 1:•Apply QoS Service Policies for Product 1•Supports 5 QoS Classes

Within MPLS Core:(for Multi-Services)

•Up to 4 queue levels can be implemented based on 4 MPLS levels




Egress PE (for Service Product 1)

•Remove MPLS Label•Classify Customer Packets to 5 QoS Service Classes based on IPP/DSCP

C1

C2

C3

C4

C5

C1

C2

C3

M1

M2

M3

M4

Egress PE

Ingress PE(for Service Product 1):

•Classify Customer Packets to 5 QoS Service Classes based on IPP/DSCP•Map 5 Service Classes to 4 Core MPLS Classes

C1

C2

C3

C4

C5

C1

C2

C3

M1

M2

M3

M4

Ingress PE

Consolidated IP/MPLS QoS Architecture to Support Multi-Services



• MPLS Short Pipe mode is recommended to tunnel a customer’s DifferServ marking.

• Inside the MPLS network, PHB will be honored based on MPLS EXP bits.

• At PE egress, PHB will be honored based on a customer’s IPP/DSCP markings.

P1

PE2PE1VPN A CE-A1

VPN ACE-A2P3

P2

MPLS+DiffServDomain

DiffServ over IPDomain

DiffServ over IPDomain

IPP orDSCP

EXP

IPP orDSCP

EXP

IPP orDSCP

IPP orDSCP

Egressscheduling isbased on IPP

Egressscheduling is

based on EXP notIPP or DSCP

CustomerIPP or DSCPnot changed

IPP - IP Precedence ValueDSCP - DiffServ Code PointEXP - MPLS Experimental bit

Customer-marked IPP

or DSCP

MPLS Short Pipe Mode is Recommended

SPSP



VPN_A site 2

SP Core

VPN_A site 1

Customer Traffic Flow

PE Outbound (ToCE)

PE BB BB PE

PE Inbound (FrCE)

PE Outbound (ToBB)

Ingress Edge Egress Edge

BB Outbound(ToPE)

PE Inbound (FrBB)

QoS Reference Model

BB Outbound(ToBB)



• It is recommended to enable QoS in SP core network when the link utilization is over 50%

Since the RealTime traffic will be impacted when link load over 50%

• Real Time Traffic can be sold on PE with 25% of uplink bandwidth if failure.

• Interactive Traffic can be sold on PE with 45% of uplink bandwidth if failure and Max Real Time Traffic.

• Business Traffic can be sold on PE with 20% of uplink bandwidth if failure and Max Real Time Traffic.

• Default Traffic can be sold on PE with unlimited bandwidth.

• ToFab queuing is recommended at PE inbound from both CE and P direction

Because GSR ToFab queue buffers at ingress LC are dynamically shared between destination slots, to prevent buffer exhaustion and packet drop triggered by security attack such as DOS, it is recommended to allocate the maximum buffers for each destination slot by setting the maximum queue length or configuring RED on ToFab queues.

• It is not recommended to enable ToFab QoS for P Inbound (from PE). The reasons are:The fabric congestion rarely happen from a lower speed line card to higher speed line cards.

QoS Recommendations



An Overview of Recommended QoS

PE PE

PE

CE

CE

L2 TransportNetwork

L2 Transport NetworkSP

MPLS Backbone

Lease Line

CE OutboundCE Outbound•Traffic shaping based on L2 Service Contract Rate•Make sure that Real Time traffic conforms to the service contract •IP traffic marking using DSCP or IP Precedence

PE InboundPE Inbound (FrCE)•Classification and Mapping between IPP, CoSand MPLS EXP•L3 Policing for Real Time Class•ToFrb Queuing /scheduling/dropping with MDRR/RED•MPLS Short Pipe to preserve customer CoStransparency

PE Outbound (PE Outbound (ToCEToCE) ) •Classification based on IPP•L3 Policing based on Real Time class rate•Aggregated shaping based on L2 Service Contract Rate •Per Port/VC/VLAN based MDRR/RED•MPLS Short Pipe to preserve customer CoS transparency

L2 QoS at ingress (pointL2 QoS at ingress (point--toto--point )point )•Traffic policing at ingress based on L2 service contract•L2 queuing/ scheduling/dropping

PP

BB Outbound (BB Outbound (ToPEToPE))•Queuing /scheduling/dropping with MDRR/RED for GE link between PE and BB

PE Outbound (PE Outbound (ToBBToBB))•Queuing /scheduling/dropping with MDRR/RED for GE link between PE and BB

PE Inbound (FrBB)•ToFab Queuing /scheduling/dropping with MDRR/RED

Customer Traffic Flow

L2 QoS at Egress L2 QoS at Egress (point(point--toto--point )point )•Traffic policing based on L2 service contract•L2 queuing/ scheduling/dropping

CE



No policing **

Police at 60%

Drop non-conforming

Police at 30%

Drop non-conforming

Police at 20%

Drop non-conforming

Drop

All Real Time traffic

(IPP=5,7)

Ingress Policing for Real Time

SP Standard Classification & Police

(based on IPP)


(based on IPP)


(based on IPP)


(based on IPP)

SP Standard Classification & Police*

(based on IPP)

Ingres/Egress

QoS Classification

No policing

Police at 20%

Drop non-confirming

Police at 60%

Drop non-confirming

Police at 40%

Drop non-confirming

Police at 20%

Drop non-confirming

Police at 60%

Drop non-confirming

Police at 20%

Drop non-confirming

No Policing

Ingress /Egress Policing for other classes

Police at 95% ***

Drop non-conforming

Standard Mapping (SP QoS to Core MPLS QoS)

Voice and Video only (up to 100% Real Time)

Police at 60%

Drop non-conforming


Voice and Video

(60% Real Time

20% Interactive 20% Business)

Police at 30%

Drop non-conforming


Business

Critical

(30% Real Time

40% Interactive

20% Business)

Police at 20%

Drop non-conforming

Standard Mapping (SP Edge QoS to Core MPLS QoS)

Business General

(20% Real Time

60% Interactive 20% Business)

Drop

All Real Time traffic

(IPP=7,5)

Standard Mapping (SP Edge QoS to Core MPLS QoS)

General

Data

(100% Interactive)

Egress Policing for Real Time

Ingress DifferServ Mapping

(IP CoS-to-EXP)

Service Profile

QoS Summary for 5 Service Profiles



IP Packet Flow for PE Inbound (From CE)

Real Time

Interactive

Business

CoS Based Policing

CHT-IP CoS-to-EXP Mapping

Default

Rate Queue 1

High Priority

Rate Queue 2

Default

MDRR

ToFab Queues

PacketClassification

IP Interfacefor VPN access

Rate Queue 1

High Priority

Rate Queue 2

Default

MDRR

ToFab Queues

RED

FabricDestination Card 1

Destination Card 16

RED

Ingress Edge:

• Up to 4 core MPLS Classes can be implemented for ToFab queues.

• It is recommended to map SP Real Time to Core MPLS cCass 1, Interactive to MPLS Class 2, Business to MPLS Class 3 and Default to MPLS Class 4

• It is recommended to police exceeding traffic to MPLS Class 2 and MPLS Class 3 with WRED enabled on the Queue.

MPLS Class 1

MPLS Class 2

MPLS Class 3

MPLS Class 4



QoS Recommendations for PE Inbound

• Packet Classification: Customer packets coming from the access line are first classified to 4 NVPN QoS classes. It is done based on the customer’s DSCP or IP Precedence marking.

• CoS-based Policing: The rate limit (policing) should be enforced for Real Time class to ensure that the Real Time traffic sending from a customer VPN site conforms the service contract. Exceeding traffic will be dropped.

• IPP-to-EXP mapping: IPP-to-EXP mapping should be performed at ingress. MPLS Pipe mode is recommended to preserve the customer’s DSCP/IPP marking.

• ToFab Queue MDRR/RED is recommended on ToFrb queues for packet scheduling. ToFab queues are per destination card based aggregated queues. Strict priority-like queuing is recommended to meet the differentiated SLA targets for 4 MPLS QoS classes.



Example : SLA for Service Profile 3

• Real TimeRate = 30%, Burst = 10ms of Linerate (IP kbps), Excess Dropped

Confirm Traffic: Latency <= 15ms, Drop Rate = 0%

• InteractiveRate = 40%, Burst = 30ms at this rate, Excess Marked out

Latency (In Packets) <= 30msec



Example : SLA for Service Profile 3

• Business

Rate = 20%, Burst = 80ms at this rate, Excess Marked out

Latency (In Packets) <= 80msec

• Deafult

non-guarantee bandwidth, use available bandwidth



Example : PE inbound from CE (QoS Service Profile 3)

class-map match-any RealTime-IPmatch ip precedence 5!class-map match-any Interactive-IPmatch ip precedence 4!class-map match-any Business-IPmatch ip precedence 2!class-map match-any SP-IPmatch ip precedence 6match ip precedence 7

!



Example : PE inbound from CE (QoS Service Profile 3)!policy-map ingress-oc3class RealTime-IPset mpls exp 5police 46464000 193750

confirm-action transmit exceed-action drop!class Interactive-IPset mpls exp 4police 61952000 581250

conform-action transmit exceed-action set-mpls-exp-transmit 3!

class Business-IPset mpls exp 2police 30976000 1550000

conform-action transmit exceed-action set-mpls-exp-transmit 1!

class SP-IPset mpls exp 4!class-defaultset mpls exp 0!



Example : PE inbound from CE (QoS Service Profile 3)slot-table-cos E3-ToFabdestination-slot all E3rx-cos-slot all E3-ToFab

cos-queue-group E3precedence 0 queue 0precedence 1 queue 1precedence 2 queue 1precedence 3 queue 2precedence 4 queue 2precedence 5 queue low-latencyprecedence 0 random-detect-label 1precedence 1 random-detect-label 0precedence 2 random-detect-label 1precedence 3 random-detect-label 0precedence 4 random-detect-label 2

random-detect-label 0 1059 2083 1random-detect-label 1 2119 4167 1random-detect-label 2 4237 8333 1queue 0 19queue 1 46queue 2 100queue low-latency strict



Ingress Edge:

MPLS Class 3 (Business)

ClassificationBased On EXP

MPLS Class 4 (Normal)

Rate Queue 1

High Priority

Rate Queue 2

Default

Port based MDRR

FrFab Queues

Fabric

IP Interface

REDMPLS Class 2 (Premium)


• PE link to BB: LLQ and DRR queue. It is recommended to map MPLS QoS Class 1 to LLQ and map MPLS QoS Class 2, 3, and 4 to the DRR queue.

IP Packet Flow for PE Outbound (To P)



Business

Default

Rate Queue 1

High Priority

Rate Queue 2

Default

Per Interface based MDRR

FrFab Queues Shaping

IP Interface

Real Time

Interactive

Priority Police 50%

Bandwidth Remaining 60%



RED

Bandwidth is Equivalent to Weight.

2-Prameter Modified Deficit Round Robin



Business

Deafult

Rate Queue 1

High Priority

Rate Queue 2

Default

Per Interface based MDRR


IP Interface

Real Time

Interactive

PriorityPolice 50%

Min BW: 30%Remaining 60%

Min BW: 20%Remaining 30%

Remaining 10%

RED

• 3-Priority MDRR with Minimum BW Guarantee • 1st Priority: serves High Priority Queue• 2nd Priority: serves the queue with minimum bandwidth guarantee• 3rd Priority: MDRR based on remaining bandwidth (weight)

3-Prameter MDRR with Minimum BW Guarantee



QoS Recommendations for PE Outbound

• Packet Classification: Customer Packets coming from the switch fabric are first classified to 4 SP defined CoS classes. It is done based on MPLS EXP bits.

• MDRR/RED: Port based MDRR/RED is recommended to provide differentiated IP CoSs towards the core-facing trunk. Strict priority-like queuing is recommended to meet the differentiated SLA targets. RED is recommended for MPLS Class 2, class 3 and class 4 to optimize TCP performance.

• Trunk Overbooking: Trunk over-engineering shall be supported for the core-facing trunk. 2.5:1 for trunk overbooking ratio is recommended to allow more efficient use of the network resources.



Example : PE outbound to P (QoS Service Profile 3)

!class-map match-any RealTime-EXPmatch mpls experimental 5class-map match-any Premium-EXPmatch mpls experimental 4match mpls experimental 3class-map match-any Normal-EXPmatch mpls experimental 2match mpls experimental 1



Example : PE outbound to P (QoS Service Profile 3)policy-map uplink-oc48class RealTime-EXPpriority police percent 50!class Premium-IPbandwidth percent 30bandwidth remaining percent 60random detectrandom-detect precedence 3 1059 2083 1random-detect precedence 4 4237 8333 1!

class Normal-IPbandwidth percent 20bandwidth remaining percent 30random detectrandom-detect precedence 1 1059 2083 1random-detect precedence 2 2119 4167 1!

class-defaultbandwidth remaining percent 10random detectrandom-detect precedence 0 1059 2083 1



P to P :

MPLS Class 3 (Business)



Rate Queue 1

High Priority

Rate Queue 2

Default

Port based MDRR

FrFab Queues

Fabric

IP Interface

REDMPLS Class 2 (Premium)


• P link to P : LLQ and DRR queue. It is recommended to map MPLS QoS Class 1 to LLQ and map MPLS QoS Class 2, 3, and 4 to the DRR queue.

IP Packet Flow for P Outbound (To P)



Example : P outbound to P (QoS Service Profile 3)




Example : P outbound to P (QoS Service Profile 3)slot-table-cos E6-ToFabdestination-slot all E6rx-cos-slot all E6-ToFab





Example : P outbound to P (QoS Service Profile 3)policy-map core-oc192class RealTime-EXPpriority police percent 50!class Premium-IPbandwidth percent 30bandwidth remaining percent 60random detectrandom-detect precedence 3 4237 8333 1random-detect precedence 4 16949 33333 1!





QoS Recommendations for P Outbound (To PE)

• Packet flow and architecture recommendations are the same as PE Outbound (To P).

Egress Edge:



Example : P outbound to PE (QoS Service Profile 3)




Example : P outbound to PE (QoS Service Profile 3)policy-map downlink-oc48class RealTime-EXPpriority police percent 50!class Premium-IPbandwidth percent 30bandwidth remaining percent 60random detectrandom-detect precedence 4 4237 8333 1random-detect precedence 3 1059 2083 1!





Example : P outbound to PE (QoS Service Profile 3)slot-table-cos E6-ToFabdestination-slot all E6rx-cos-slot all E6-ToFab





IP Packet Flow for PE Inbound (From P)

IP Interface

Egress Edge:

MPLS Class 1(Real Time)

MPLS Class 2(Premium)

Classification Based On EXP


Fabric

Rate Queue 1

High Priority

Rate Queue 2

Default

MDRR

ToFab Queues

Rate Queue 1

High Priority

Rate Queue 2

Default

MDRR

ToFab Queues

RED

Destination Card 1

Destination Card 16

RED

MPLS Class 3(Business)



QoS Recommendations for PE Inbound (From P)

• Packet Classification: Customer Packets coming from the Interface are first classified to 4 SP defined CoS classes. It is done based on MPLS EXP bits.

• MDRR/RED is recommended on ToFrb queues for packet scheduling. ToFab queues are per destination card based aggregated queues. Strict priority-like queuing is recommended to meet the differentiated SLA targets for 4 CoS classes.



Example : PE inbound from P (QoS Service Profile 3)




Example : PE inbound from P (QoS Service Profile 3)slot-table-cos E3-ToFabdestination-slot all E3rx-cos-slot 1 E3-ToFab

cos-queue-group E3precedence 0 queue 0precedence 1 queue 1precedence 2 queue 1precedence 3 queue 2precedence 4 queue 2precedence 6 queue 2precedence 5 queue low-latencyprecedence 0 random-detect-label 1precedence 1 random-detect-label 0precedence 2 random-detect-label 1precedence 3 random-detect-label 0precedence 4 random-detect-label 2precedence 6 random-detect-label 2




IP Packet Flow for PE Outbound (To CE)

Business


Default

Rate Queue 1

High Priority

Rate Queue 2

Rate Queue 3

Per Port/VC/VLAN MDRR


Fabric

Real Time

Interactive

CoS Based Policing

IP Interface

Egress Edge:



QoS Recommendations for PE Outbound (To CE)

• Packet Classification: Customer Packets coming from Fabric are first classified to 4 SP defined CoS classes. It is done based on IPP/DSCP.

• CoS-based Policing: Policing should be enforced for Real Time class to ensure that the Real Time traffic sending to a customer VPN site not exceeding the Service Contract rate, and exceeding traffic will be dropped. For Video and Voice only service profile, it is recommended to enforce rate limiting at 95% for real time to reserve at least 5% bandwidth for customer control.

• Per sub-interface based traffic shaping: Shaping is recommended to ensure the aggregated customer traffic not exceed the L2 Service Contract Rate per access interface (VC or VLAN). For lease line, no shaping is required at edge.

• MDRR/RED: Port/VC/VLAN based MDRR/RED is recommended to provide differentiated IP CoSs towards the customer access sub-interface. Strict priority-like queuing with minimum bandwidth guarantee for each non-priority queue is recommended to meet the differentiated SLA targets. RED is recommended for Interactive Data, Business and Defaultl class to optimize TCP performance.



Example : PE outbound to CE (QoS Service Profile 3)

class-map match-any RealTime-IPmatch ip precedence 5!class-map match-any Interactive-IPmatch ip precedence 4match ip precedence 3!class-map match-any Business-IPmatch ip precedence 2match ip precedence 1!class-map match-any SP-IPmatch ip precedence 6match ip precedence 7

!



Example : PE outbound to CE (QoS Service Profile 3)

policy-map egress-oc3class RealTime-IPprioritypolice 46464000 193750

confirm-action transmit exceed-action drop!class Interactive-IPbandwidth percent 40police 61952000 581250

conform-action transmit exceed-action droprandom detectrandom-detect precedence 3 1059 4167 1random-detect precedence 4 4237 8333 1!class Business-IPbandwidth percent 20police 30976000 1550000

conform-action transmit exceed-action droprandom detectrandom-detect precedence 2 2119 4167 1random-detect precedence 1 1059 4167 1



Example : PE outbound to CE (QoS Service Profile 3)!class SP-IPbandwidth percent 10bandwidth remaining percent 10random detectrandom-detect precedence 6 4237 8333 1

random-detect precedence 7 4237 8333 1

!class-defaultBandwidth remaining percent 90random detect

random-detect precedence 0 1059 2083 1


MPLS Aware IP Services, 09/04 160160160© 2004, Cisco Systems, Inc. All rights reserved.

Documents

MPLS AWARE IP SERVICES · 2018. 1. 9. · © 2004 Cisco Systems, Inc. All rights reserved. Cisco Internal Use Only 2 MPLS Aware IP Services, 09/04 © 2004 Cisco Systems, Inc. All