MPLS VPSs in IP Tunnel Environments

8/4/2019 MPLS VPSs in IP Tunnel Environments

1/29

White Pape

2009 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information. Page 1 of 2

Deploying and Configuring MPLSVirtual Private Networks In IP

Tunnel Environments

Russell Kelly

[email protected]

Craig Hill

[email protected]

Patrick Naurayan

[email protected]


2/29

White Pape


TableofContents

Introduction................................................................................................................. 3

MPLS Over GRE Deployment Examples ......................................................... 4

MPLS L2VPN Deployment EoMPLS over GRE .......................................... 6

Tunnel Scenarios ...................................................................................................... 8

Forwarding Plane Label Stack information.................................................... 9

Detailed Packet and Label Path Information ...............................................11CompleteGREencapsulatedIPpacketisasshownbelow................................. 11

L3VPNControlPlaneInformation ............................................................................. 11

L3VPNForwardingPlaneInformation .................................................................... 12

L2VPN(EoMPLS)ForwardingandControlPlaneInformation........................12

Fragmentation in MPLS over GRE Networks ..............................................13

Resolve IP Fragmentation, MTU, MSS, and PMTUD - Issues with GRE........................................................................................................................................14

Example Configuration of an L3 MPLS VPN over GRE PE Router .....15

Inter-AS over GRE Hub and Spoke Scaling MPLS over GRE Designs........................................................................................................................................16VPNv4routesdistributionbetweenASBRs(OptionB) ....................................... 16

ApplicationofInter-ASOptionBtoaVPNHubandSpokeDesigns ................ 17

Encryption in MPLS over GRE Networks ......................................................18QoS in MPLS over GRE Networks ...................................................................20

Design Notes for MPLS over GRE ...................................................................25

LxVPNoGRE Case Study......................................................................................26Caseoverview.................................................................................................................... 26

The L2VPN Case: .......................................................................................................................27

The L3VPN Case: .......................................................................................................................27CaseStudyRouterConfigurations .............................................................................. 28


3/29

White Pape


Introduction

Service providers (SPs), and Enterprises alike are migrating from existing ATM, Frame Relay (FR),

and Time Division Multiplex (TDM) infrastructures to an IP-based backbone. Current IP backbones

can no longer be designed just to transport IP packets. Instead, Next Generation (NG) Internet

Protocol (IP) backbones must be capable of providing multiple IP services over a single physical

infrastructure, using techniques such as differentiated quality of service (QoS) and secure transport

layer. In addition, NG IP backbones should provide Layer 2/3 VPNs, IP multicast, IPv6, and

granular traffic-engineering capabilities.

Ultimately, these IP backbones should be scalable and flexible enough to support the mission-

critical, time-sensitive applications that all modern networks require and to meet new demands for

applications, services, and bandwidth. Multiprotocol Label Switching (MPLS), when used on an IP

backbone, provides the mechanism to offer rich IP services and transport capabilities to the routing

infrastructure.

Additionally providing the capabilities to offer MPLS based VPN s over a non-MPLS capable IP core

offers an extremely flexible, cost efficient virtualized WAN design that is simple to configure, whilst

at the same time maintaining the support for core infrastructure services such as security and QoS

An example of a converged tunneled MPLS VPN architecture providing L2 & L3 VPN Services is

detailed in the diagram below. This deployment example is well suited to a high bandwidth

deployment running tunneled MPLS between regional locations, where the number of tunnels is

relatively few. However the throughout required for each tunnel may be in the 1 10Gbps range.

Figure 1. Converged L2 & L3 MPLS VPN over GRE Deployment Example

This white paper examines the advanced Virtual Private Network (VPN) capabilities in next

generation application aware WAN designs specifically focusing on MPLS VPN over an IP-only

core; that being deployment of MPLS VPN over IP Tunnels (GRE) and will examine the benefits,

deployment options, configurations, as well as the associated technologies such as IPSec, QOS

and Fragmentation.


4/29

White Pape


MPLS Over GRE Deployment Examples

The implementation assumes that either the Enterprise or the Service provider has procured a

Layer 3-based service such a Layer 3 VPNs from a provider interconnecting the MPLS MAN, or inthe case of many customers, to interconnect MPLS MAN across an inconsistent IP transport with

different MTUs, encryption, and tunneling capabilities where the a viable option is to encapsulate

in GRE. The MAN may have multiple connections between them to provide load balancing and/or

redundancy.

Below are two common examples of MPLS over GRE topologies; site to site and hub and spoke.

As shown in Figure 2, the WAN edge router used for interconnecting MAN plays the role of a P

device even though it is a CE for the SP VPN service. It is expected to label switch packets

between the MAN1 and MAN2 across the SP network. Note: The GRE encapsulating or de-

encapsulating router can be either a P or PE router.

Figure 2. Site to Site Tunneled MPLS VPN Deployment Example

A point-to-point GRE tunnel is set up between each edge router pair (if full mesh is desired). From a

control plane perspective, the following protocols should be run within the GRE tunnels:

IGP such as EIGRP or OSPF for MPLS device reach ability (P/PE/RR)

LDP for label distribution

MP-iBGP for VPN route/label distribution


5/29

White Pape


Figure 3. Hub and Spoke Tunneled MPLS VPN Deployment Example

Multiple point-to-point GRE tunnels on the hub or mGRE (if using NHRP/2547oDMVPN). From a

control plane perspective, the following protocols should be run within the (m)GRE tunnel(s):

Routing protocol of the provider to learn the Branch and the Head-end s physical interface

addresses (tunnel source address). Static routes could also be used if these are easily

summarized.

GRE tunnel between the branch PE and the head-end P router.

IGP running in the enterprise global space over the GRE tunnel to learn remote PEs and

RRs loopback address.

LDP session over the GRE tunnel with label allocation/advertisement for the GRE tunnel

address by the branch router.

MP-iBGP session with RR, where the branch router s BGP source address is the tunnel

interface addressthis forces the BGP next-hop lookup for the VPN route to be

associated with the tunnel interface.

Many more details on these and other deployment examples along with configurations can be

found at the following location:

http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/ngwane.pdf


6/29

White Pape


MPLS L2VPN Deployment EoMPLS over GRE

EoMPLS technology is currently the solution which best answers the problem of Layer 2 extension

over long distances. However, it has traditionally required an enterprise to migrate the core toMPLS switching, which is often a burden when the core is not dedicated to L2VPN extension.

Figure 4. EoMPLS over GRE Tunnel For L2VPN Transport Over an IP Core

MPLS requires specific expertise for deployment; maintenance and migration of the existing IP core

to a new MPLS core can be complex. To ease the adoption of Layer 2 extension, the solution is to

encapsulate the EoMPLS traffic over a GRE tunnel. This allows for the transport of all Layer 2 flows

over the existing IP core, eliminating the need for a complex migration.

This solution creates a high performance hardware switched GRE tunnel that encapsulates the

EoMPLS frames within. This allows IP tunneling of L2 MPLS VPN frames at Gigabit per second

rates, in the case of the ASR1000, up to 20Gbps, or the to the bandwidth of the forwarding engine,

also known as the ESP, installed in the platform.

The L2VPN over IP design is identical to the deployment over MPLS: EoMPLS "port modexconnect" being the default option for point-to-point connection.

In the following EoMPLSoGRE design, the GRE connection is established between the two Data-

center core routers, and then the MPLS LSP is established over this tunnel. This provides an

extremely flexible datacenter inter-connect up to 10Gbps bi-directional forwarding (with the

ASR1000 ESP20), whereby distributed and smaller datacenters can be L2 integrated over vanilla

IP networks.


7/29

White Pape


Figure 5. IP Tunneled L2 VPN For Datacenter Interconnect

Additionally, with platforms such as the ASR1000, IPSec can be used to encrypt the GRE tunnels.

This further expands the use-cases for this deployment in that one can now tunnel these L2

transports securely over un-trusted IP backbones or even the Internet


8/29

White Pape


Tunnel Scenarios

There are three scenarios described below where L2VPN and L3VPN over GRE are typicallydeployed by customers on PE or P routers.

As shown in Figure 6 the customer has not transitioned any part of their Core to MPLS but would

like to offer EoMPLS and Basic MPLS-VPN services. Hence GRE tunneling of the MPLS labeled

traffic is done between PEs. This is the most common scenario seen in various customers

networks.

Figure 6. PE PE GRE Tunnels

The second scenario shown in Figure 7 is one where MPLS has been enabled between PE and P

routers but the network core may have non-MPLS aware routers or IP encryption boxes. In this

scenario GRE tunneling of the MPLS labeled packets is done between P routers.

Figure 7. P P GRE Tunnel

Figure 8 demonstrates a network where the PP Nodes are MPLS aware while GRE tunneling is

done between a PE P non-MPLS network segments.


9/29

White Pape


Figure 8. PE P GRE tunnels

Forwarding Plane Label Stack information

The following section outlines the label stack and forwarding operation in the three scenariosoutlined previously without encryption. Figures 4 & 5 detail the topology in a Branch to Hub

configuration. However the label imposition/swapping is the same whether the topology is

branch/MAN or MAN/MAN. The important consideration is the header differences and operations.

The configuration is the same in all cases with respect to IOS CLI.

Figure 9. MPLS over GREForwarding Plane for P-P Router Tunnel

As shown in Figure 9, the P router receives a labeled packet for the MPLS enabled MAN (LDP1). It

label swaps with the appropriate LDP label (LDP2) advertised by the P for the destination next-hop

address (Tunnel destination address). It then encapsulates the labeled packet in a GRE tunnel with

the hub P as the destination before sending it to the provider. Since in this example SP is providing

Layer 3 VPN service, it further pre-pends its own VPN and LDP labels for transport within its

network. The hub P receives a GRE encapsulated labeled packet. It de-encapsulated the tunnel

headers before label switching it out to the appropriate outgoing interface in the MPLS MAN for the

packet to reach the eventual PE destination.


10/29

White Pape


Figure 10. MPLS over GREForwarding Plane for P-PE Router Tunnel

As shown in Figure 10, the branch router attaches the appropriate VPN label for the destination

along with the LDP label advertised by the hub P for the destination next-hop address. It then

encapsulates the labeled packet in a GRE tunnel with the hub P as the destination before sending it

to the provider. Since in this example SP is providing Layer 3 VPN service, it further pre-pends its

own VPN and LDP labels for transport within its network. The hub P receives a GRE encapsulated

labeled packet. It de-encapsulated the tunnel headers before label switching it out to the

appropriate outgoing interface in the MPLS MAN for the packet to reach the eventual PE

destination.

Figure 11. MPLS over GREForwarding Plane for PE-PE Router Tunnel

As shown in Figure 11, the routers attach the appropriate VPN label for the destination advertised

by the PE router. It then encapsulates the labeled packet in a GRE tunnel with the hub PE as the

destination before sending it to the provider. Since in this example SP is providing Layer 3 VPN

service, it further pre-pends its own VPN and LDP labels for transport within its network. The hub

PE receives a GRE encapsulated labeled packet. It de-encapsulated the tunnel headers before

forwarding it out to the appropriate outgoing interface based on the VPN label information and the

VRF routing table.

As can be seen from the headers, this adds a large amount of overhead to the MTU. The two SP

headers are for the SP provided VPN and in the context of this MPLSoGRE testing are not

considered for the SP Core the customers traffic appears as vanilla IP traffic (sourced from the

Tunnel source IP interface, IPv4 loopback or other IPv4 interface advertised within the IP Core)


11/29

White Pape


Detailed Packet and Label Path Information

Complete GRE encapsulated IP packet is as shown below

The figure below expands out the packet format for an MPLS VPN Labeled packet tunneled overGRE between two PE routers thats are connected through the GRE tunnel. One can see from the

diagram that the VPN label is appended to the original packet, not however the IGP LDP label as

well because the PEs are effectively directly connected (no P core). Additionally the GRE header

and new IP header are appended for the GRE transport.

Figure 12. Detail of the Encapsulated VPN Traffic in a GRE Tunnel

L3 VPN Control Plane Information

The following three diagrams detail the control and forwarding paths for L2 and L3 VPN traffic over

GRE

Figure 13. Detail of the Control Plane Communication and Messaging For L3VPN Over GRE


12/29

White Pape


L3 VPN Forwarding Plane Information

Figure 14. Detail of the Forwarding Plane Communication For L3VPN Over GRE

L2 VPN (EoMPLS) Forwarding and Control Plane Information

Figure 15. Detail of the Forwarding and Control Plane Messaging for L2VPN Over GRE


13/29

White Pape


Fragmentation in MPLS over GRE Networks

In general, the rule for dealing with fragmentation in MPLS over GRE environments is the same as

in the pure IP traffic over GRE. The most important consideration with MTUs when MPLS is being

used is that one can now override the MPLS MTU on a GRE tunnel interfaces separate to the

physical interface MTU and the ip mtu. The interface command mpls MTU can be raised up to a

max value max (65536). Raising this MPLS MTU value has the same affect for MPLS traffic as

raising the ip mtu on tunnel interfaces does for ip traffic, in that now when an MPLS encapsulated

packet arrives on a tunnel interface and it has the df-bit set it will not be dropped as long as the

MPLS MTU > MTU of the labeled packet. Note here that raising the ip mtu on the tunnel interface

has no effect on the MPLS mtu MPLS MTU is either raised/lowered on the interface using the

MPLS MTU command or it is derived from the physical interface MTU.

The ability to modify the MPLS MTU separately is particularly useful on a P-P or a P-PE router

topology if MPLS packets arrive with a df-bit set and MPLS MTU > MTU on the P routers interface.

This is an issue because there is no way to clear the df-bit on a labeled packet, as a df-bit can only

be cleared on an IP packet. Additionally it may not be an option in some tunneling environments toraise the physical interfaces MTU in the core, where the provider dictates the core MTU (the

Internet being a perfect example). In this case one raises the MPLS MTU on the tunnel separately

to the ip mtu; this causes the labeled packets to be encapsulated, but once they are encapsulated

in GRE they will be fragmented, again forcing the receiving router to reassemble the MPLS packet

before switching on through the labeled core. This is normally not an issue in provider edge (PE-

PE) tunneled environments as Policy Based Routing (PBR) can be used on ingress to clear the df-

bit on the IP traffic. However there might still be cases where end hosts cannot handle fragmented

packets therefore even in PE-PE environments one must retain not fragment the MPLS packet and

retain the original df-bit, therefore raising the MPLS MTU is required here too.

It is best practice for all routers in the MPLS domain to honor the same MTU, as not doing so forces

the receiving router to reassemble fragments. The same best practice holds for GRE tunneled

environments, whereby receiving routers are forced to reassemble GRE packets if fragmentation

post-encapsulation is occurring. The best approach of course is to control the ip mtu of the sending

clients pre-MPLS and GRE encapsulation, as there are performance ramifications with having to

reassemble on the router. The main issue is that the router has to account for all streams to all

hosts and keep track of all fragments before reassembling and forwarding on to the multitude of

end hosts.

On most routing platforms the reassembly of packets is not done in hardware, instead it is sent to

the control plane for reassembly. The main reason for this design is simple: in the past the

hardware forwarding engines did not have the intelligence or memory management capabilities to

track and reassemble packets. The major downside of doing reassembly in the control plane wasthe lack of performance. The Cisco ASR1000 series, with the Quantum Flow Processor (QFP) now

has the ability to reassemble GRE in hardware, giving an order of magnitude greater performance

than any other current routing platform for even fragmented MPLS over GRE packets.

Over all one needs to consider the VPN backbone as a whole, find the low water mark - read

lowest MTU in the backbone - and design/account for it. Now all hosts who come in with an MTU

larger than the IP MTU of the tunnel will get an icmp cant fragment error message back so that it

can lower its MTU accordingly. Then, even if they still set the DF bit, once they are sending at

1400B, this bit can at least be cleared for the IPSec packet allowing IPSec fragmentation in the

core and reassembly at the remote/receiving router.


14/29

White Pape


Resolve IP Fragmentation, MTU, MSS, and PMTUD - Issues with GRE

In general it is best practice to avoid fragmentation across a GRE tunnel (or any tunnel for that

matter) to avoid having to fragment and especially reassembling any packet on a routing platform,whether this is reassembling GRE, IPSec or IP in IP packets.

There are numerous methods to avoid fragmentation including setting the end hosts MTU to a

value low enough to account for the VPN overhead be it GRE, IPSec a combination or another

encapsulation protocol.

To manage TCP traffic one can employ tcp adjust MSS, as well as setting the IP MTU on the tunnel

interface. To account for non-TCP traffic in an IP environment policy based routing can be used to

clear any DF bit set by an application (this is by default for PMTUD hosts)

These methods and a further explanation are covered in the links below:

http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml

http://www.cisco.com/en/US/tech/tk827/tk369/technologies_tech_note09186a0080093f1f.shtml


15/29

White Pape


Example Configuration of an L3 MPLS VPN over GRE PE Router

In this example configuration EIGRP is being used to advertise GRE endpoints and the loopback

interfaces for the LDP peering. This is the most common implementation; however in the PE-PE

use case one does not necessarily need the IGP over the tunnel to advertise the loopbacks, or

tunnel endpoints, and the external IGP is used to advertise the tunnel endpoints and additionally

the MP-BGP peering addresses over the tunnel. If the IGP can be excluded, there is essentially

only BGP and LDP running over the GRE tunnel this will allow for greater control plane scale as

the potential scaling issues inherent in an IGP peering in the hub and spoke topologies is removed.

Figure 16. L3 VPN Over GRE Configuration

This can be further enhanced to only run BGP peering between the hub and multiple spokes by

running Inter-AS over GRE whereby one is simply using labeled BGP to pass the VPN

information to the spoke. This is particularly useful if one wants to scale the hub and spoke

(remote sites) and adhere to a hub->spoke topology. This is covered in the following Inter-AS over

GRE section.


16/29

White Pape


Inter-AS over GRE Hub and Spoke Scaling MPLS over GRE Designs

One additional method of scaling an MPLS over IP topology is to utilize Inter-AS VPN over GRE to

pass the MPLS VPN information between the hub location and the numerous remote sites. In this

scenario both the hub and all of the spokes act as PE and ASBR routers and the topology is purehub and spoke.

The major benefit of using Inter-AS over GRE in this manner is that now the control plane only has

to manage N x eBGP sessions, as opposed to a BGP, LDP and an IGP session to every remote

site. This makes the solution very scalable and well suited to tunnelled VPN hub/spoke

deployments as a single eBGP peer over GRE will carry all customer VPNv4 routes across the AS

boundary. As in this case an eBGP peering session is all that is required between the hub (core

PE in one AS) and numerous site routers (remote PE routers in a different AS). Additionally any

QoS and encryption requirements work just as they would in the traditional MPLSoGRE solution.

VPNv4 routes distribution between ASBRs (Option B)

Traditionally the MPLS VPN InterAS feature provides a method of interconnecting VPNs between

different MPLS VPN service providers. This allows sites of a customer to exist on several carrier

networks (Autonomous Systems) and have seamless VPN connectivity between these sites.

Figure 8 below illustrates the operation of Inter-AS, where two ASBRs share VPNv4 routes and

VPN labels to provide Inter-AS MPLS VPN reach ability

Figure 17. Example of Inter-AS Option B Operation

In option B, the AS border routers (ASBR) peer with each other using an eBGP session. The ASBR

also performs the function of a PE router and therefore peers with every other PE router in their AS.

The ASBR does not hold any VRFs but instead will hold all or a subset (those that need to be

passed to the other AS) of the VPNv4 routes from every other PE router. The VPNv4 routes are

kept unique in the ASBR by use of the route-distinguisher.

The ASBR can control which VPNv4 routes it accepts through the use of route-targets. The ABSR

then exchanges the VPNv4 routes, plus the associated VPN label with the other ASBR using


17/29

White Pape


eBGP. This procedure requires more co-ordination between carriers such as eBGP peering and the

route-targets that will be accepted.

Application of Inter-AS Option B to a VPN Hub and Spoke Designs

This control and data forwarding path detailed previously can also be used in hub and spoketopology where the hub is in one AS and all the spokes are in another AS. The hub peers with all

spokes, but the spokes only peer with the hub router.

The solution additionally assumes the backbone network does not carry MPLS customer traffic and

hence in this case will only provide native IP connectivity from aggregating ASR 1000 to the remote

routers. In order to provide the MPLS functionality for the overlay network, GRE tunnels run as a

transport mechanism over the IP backbone. Figure 11 below illustrates the high level WAN

connectivity and how the GRE tunnels will be configured between Aggregation and sites for primary

and backup routing.

Once the GRE endpoints are reachable and the GRE tunnels are established, another EBGP

session will be set up from ASR hub router to all spokes. The configuration of the tunnels is shown

below. To enable sending and receiving of MPLS packets over these GRE tunnels when using

BGP to advertise the MPLS VPNs, simply configure mpls BGP forwarding on the tunnel interface.

interface Tunnel1description Primary tunnel to ASRip address 10.0.0.1 255.255.255.252mpls bgp forwardingqos pre-classifytunnel source GigabitEthernet0/0tunnel destination 192.168.1.1

Table 1. Remote Site GRE Tunnel Configuration

interface Tunnel1description Primary tunnelip address 10.0.0.2 255.255.255.252mpls bgp forwardingqos pre-classifytunnel source Loopback0tunnel destination 192.168.0.1

Table 2. Aggregation GRE Tunnel Configuration


18/29

White Pape


Encryption in MPLS over GRE Networks

Some integrated services platforms can additionally offer encryption of these IP tunnels to enable

tunneling over even untrusted IP backbones or the Internet. This is one of the key advantages

with the ASR 1000 and the integrated services is that the GRE tunnels can be encrypted at gigabit

data rates by simply applying a crypto map to the egress interface, or more commonly tunnel

protection directly to the tunnel interface.

An example in the configuration is provided in the section below.

mpls ldp router-id Loopback0 force!crypto isakmp policy 1encr 3desauthentication pre-sharegroup 2

!crypto isakmp key cisco123 address 192.168.0.2crypto ipsec transform-set 3DES esp-3des esp-md5-hmacmode transport

!crypto map ASR 1 ipsec-isakmpset transform-set 3des

!!interface Tunnel1ip address 10.10.0.1 255.255.255.0mpls iptunnel source 10.0.0.1tunnel destination 10.1.0.2tunnel protection ipsec profile ASR

!interface Loopback0ip address 2.2.2.2 255.255.255.255

Table 3. Example Configuration For MPLSoGREoIPSec

As can be seen, to enable encryption on the IP tunnel all that needs to be configured is a transform

set and IPSec profile and for this profile to be applied to the IP Tunnel. This will then ensure all the

traffic traversing the IP Core is protected, including the label information.

Cisco routers also provide the capabilities to ensure that fragmentation can be avoided in the core

with PMTU discovery; this is especially relevant when there is IPSec involved, as this can add an

additional 70+ bytes to the IP Header. The ability therefore to enabled IPSec pre or post

encryption to allow for MPLS packets with df-bits set or to allow for legacy applications to not have

to reassemble even with this degree of encapsulation is very valuable in a core routing platform.

A full configuration for an L3VPN provided over an encrypted (protected) GRE tunnel is detailed in

Table 4 below


19/29

White Pape


Table 4. Configuration For L3VPN over GRE With IPSec

ip vrf vpn1rd 100:1route-target export 100:1route-target import 100:1

!crypto isakmp policy 1

encr 3desauthentication pre-sharegroup 2

!crypto isakmp key cisco123 address 192.168.0.2crypto ipsec transform-set 3DES esp-3des esp-md5-hmacmode transport

!crypto map ASR 1 ipsec-isakmpset transform-set 3des

!interface Tunnel1ip address 10.1.0.1 255.255.255.0mpls iptunnel source 192.168.0.1tunnel destination 192.168.0.2tunnel protection ipsec profile ASR

!interface Loopback0ip address 2.2.2.2 255.255.255.255

!mpls ldp router-id Loopback0 force!interface GigabitEthernet0/2/4no ip addressnegotiation autono cdp enable

!interface GigabitEthernet0/2/4.1encapsulation dot1Q 21ip vrf forwarding vpn1ip address 10.0.0.1 255.255.255.0

!!

interface GigabitEthernet0/2/7.1encapsulation dot1Q 20ip address 192.168.0.1 255.255.255.0

!router ospf 100log-adjacency-changesnetwork 2.2.2.2 0.0.0.0 area 0network 10.1.0.1 0.0.0.0 area 0

!router bgp 100no synchronizationbgp log-neighbor-changesneighbor 10.1.0.2 remote-as 100no auto-summary

!address-family vpnv4neighbor 10.1.0.2 activateneighbor 10.1.0.2 send-community extended!address-family ipv4 vrf vpn1no synchronizationneighbor 10.0.0.2 remote-as 20neighbor 10.0.0.2 activateexit-address-family


20/29

White Pape


QoS in MPLS over GRE Networks

In environments where the MPLS is being tunneled over GRE, the customer can use the

automatically reflected TOS header in the GRE packet. The diagram below illustrates the

TOS/PREC reflection that can be propagated to the IPSec header if cryptography is enabled or to

the GRE IP header. The MPLS EXP value (Prec) is set as derived from the initial IP TOS setting.

When the packet is decrypted or de-encapsulated at the remote tunnel, the MPLS EXP value will

be set on the MPLS packet and can be utilized accordingly.

Figure 18. TOS Reflection In MPLSoGRE and MPLSoGREoIPSec Configurations

A service can also be applied to the egress physical interface to explicitly set the ip precedence (as

below)

As an extension on the above schema, there is the option on the ingress IP traffic linkage to set

both a qos-group and marking of the IP traffic, such that both can be matched on in the child egress

service policy to identify traffic types per vrf. Additionally the tunnel itself can be shaped by

matching each tunnel in an ACL that matches GRE source and destination IP Addresses. Up to

255 tunnels can be shaped per physical (or logical vlan sub-interface) in IOS XE release 2.3.0.

This QoS design is an adaptation of the DMVPN QoS model detailed in the QoS and VPN

Deployment Guide version 1 white paper.

class-map match-all exp2match mpls experimental topmost 2

!policy-map exp2

class exp2set ip precedence 2

!

interface Tunnel1ip address 192.168.0.1 255.255.255.0

qos pre-classifympls iptunnel source 10.0.0.1

tunnel destination 10.1.0.1!

int gi0/1/1

service policy exp2 out


21/29

White Pape


Figure 19. Example QoS Policy for and MPLSoGRE or MPLSoGREoIPSec Topology

This exact configuration can be scaled to such that on any single interface or sub-interface up to

255 tunnels can be matched at the parent level and the MPLS VPN traffic within this site can be

allocated bandwidth or prioritized accordingly. The configuration in Table 5 outlines this

configuration

For the EoMPLSoGRE case, one can employ a similar schema, setting QoS group, setting the

MPLS EXP bits and/or policing on in egress and then using this qos-group and EXP to allocate

bandwidth on a per pseudo wire basis within the shaped tunnel. See the configuration below

(Table 6) as an example.


22/29

White Pape


Table 5. QoS Configuration for Figure 16 For a Single Tunnel

class-map match-all vrf1-highmatch qos-group 40match mpls experimental topmost 5class-map match-all vrf1-mediummatch qos-group 40match mpls experimental topmost 4

class-map match-all vrf1-lowmatch mpls experimental topmost 0match qos-group 40

class-map match-all vrf2-highmatch qos-group 30match mpls experimental topmost 5class-map match-all vrf2-mediummatch qos-group 30match mpls experimental topmost 4class-map match-all vrf2-lowmatch mpls experimental topmost 0match qos-group 30

class-map match all Site1match access-group name Site1

class-map match-any controlmatch ip precedence 6 7match mpls experimental topmost 6 7

!policy-map childclass vrf1-high

priority level 1police 1000000

class vrf2-highpriority level 2police 500000

class vrf1-mediumbandwidth remaining ratio 8

class vrf2-mediumbandwidth remaining ratio 15

class control

bandwidth remaining ratio 1class vrf1-low

bandwidth remaining ratio 2class vrf2-low

bandwidth remaining ratio 4

policy-map Parentclass Site1

shape average 5120000service-policy child

Int Gi0/0/0 The egress physical interfaceService policy output Parent

ip access-list extended Site1permit gre host host


23/29

White Pape


Ingress:=====

class-map match-any BestEffort-EoMPLSmatch cos 0 7class-map match-any Business-EoMPLS

match cos 1 2class-map match-any Multimedia-EoMPLSmatch cos 3 4class-map match-any Realtime-EoMPLSmatch cos 5

policy-map Ingress-EoMPLSclass Realtime-EoMPLSpolice cir 128000 bc 8000 be 8000 conform-action set-mpls-exp-transmit 5exceed-action dropclass Multimedia-EoMPLSpolice cir 128000 bc 8000 be 8000 conform-action set-dscp-transmit 3exceed-action dropclass Business-EoMPLSpolice cir 128000 bc 8000 be 8000 conform-action set-qos-transmit 2exceed-action dropclass BestEffort-EoMPLS

set qos-group 1

Egress:=====

class-map match-any BestEffort-EoMPLS-Egressmatch qos-group 1class-map match-any Business-EoMPLS-Egressmatch qos-group 2class-map match-any Multimedia-EoMPLS-Egressmatch dscp 3class-map match-any Realtime-EoMPLS-Egressmatch ip prec 5

class-map match all GRE_DCI_Tunnel1match access-group name GRE_DCI_Tunnel1

ip access-list extended GRE_DCI_Tunnel1permit gre host host

policy-map Egress-EoMPLS-Childclass Realtime-EoMPLS-Egressset mpls exp 5priority level 1class Multimedia-EoMPLS-Egressset mple exp 3priority level 2class Business-EoMPLS-Egressset mpls exp 2class BestEffort-EoMPLS-Egressset mple exp 2

policy-map Egress-EoMPLS-Parentpolicy-map parent

class GRE_DCI_Tunnel1shape average 600000000service-policy Egress-EoMPLS-Child

Table 6. QoS Configuration Example for EoMPLSoGRE


24/29

White Pape


It is important to note that when the GRE tunnel is encrypted, this same QoS policy will work as the

ability to look at the inner MPLS EXP exists whether the outer header is GRE or IPSec.

For further detail on these and other QoS features available on the ASR 1000 follow the link to the

paper in the URL below

http://www.cisco.com/en/US/prod/collateral/routers/ps9343/solution_overview_c22-449961_ps9343_Product_Solution_Overview.html


25/29

White Pape


Design Notes for MPLS over GRE

The solution of running MPLS VPNs over a GRE and protected GRE infrastructure has obvious

cost saving and flexible network design advantages. There are some important points/restrictions

to take note of these mostly pertain to high availability designs.

There is currently no support for IP Tunnel HA in IOS, therefore during an RP switchover

all tunnels will go down and have to be re-initialized; this subsequently causes all IPSec

tunnels to have to be re-established as well as IGP and LDP adjacencies.

There is no BFD support on tunnel interfaces to design in fast peer down detection in the

IGP core in IOS currently

The feature IGP-LDP sync is not supported on tunnel interfaces.

There is no support for keep-alive when using the tunnel protection (see link):

http://www.cisco.com/en/US/tech/tk827/tk369/technologies_tech_note09186a008

048cffc.shtml#t7


26/29

White Pape


LxVPNoGRE Case Study

Case overview

The information below will demonstrate a case study of EoMPLSoGRE and L3VPNoGRE scenarios

running simultaneously on Cisco PEs using the existing IOS implementation. (This was tested with

a SIP-400 on a 7600 and on an ASR1000).

The topology shown in Figure 18 is used for our case study.

Figure 20. Case Study Set-up

In the above scenario, the following configuration rules are used:

Create a GRE tunnel between each PE router.

PEs core-facing interface address is used as GRE tunnel source

The tunnel end-points IP addresses should be reachable via the core-facing interface.

PEs use static routing via the core-facing interface for GRE tunnel destination LDP is enabled on the GRE tunnel but not on any interfaces in the IP Core

Static route pointing to remote PE LDP-ID via GRE tunnel is used

Tunnel keep-alive is enabled (as no IPSec in this scenario)

MTU of core-facing interface is set to allow for forwarding of jumbo frames

The tunnel IP addresses should be reachable via the core facing GE interface.

The attachment circuit configuration for EoMPLS Port and VLAN modes use MPLS as the

encapsulation protocol.

L3VPN VRFs are running eBGP between PE CE

No QoS is being done on the VRFs or Attachment circuits


27/29

White Pape


The L2VPN Case:

The VC label binding for attachment circuits will be distributed by PEs via a targeted LDP session

across the GRE tunnel. Hence, since the each PE routers are penultimate to each other over theGRE tunnel their label binding for each other LDP-ID will be implicit-null. The next-hop PE of each

EoMPLS pseudo wire will be learned via the GRE tunnel as shown later in the verification

procedures. All EoMPLS traffic will be forwarded via the GRE tunnel.

However, it is expected that some customers may chose to map specific pseudo wires or pw-class

to unique GRE tunnels.

The L3VPN Case:

The VPNv4 prefixes, labels and next-hops are learned by remote PEs through MP-iBGP and arenot known to the non-MPLS Core: This is achieved by defining a static route to BGP next-hop PE

through a GRE Tunnel across the non-MPLS network. When routes are learned from the remote

PE they will have a next-hop of the GRE Tunnel. Thus, all traffic across the IP Core will be sent

using the GRE Tunnel.


28/29

White Pape


Case Study Router Configurations

PE1 Configuration

vrf definition VPN1rd 100:1address-family ipv4route-target both 100:1exit-address-family!

mpls label protocol ldpmpls ldp neighbor 10.10.10.11 targeted

mpls ldp router-id Loopback0 force!interface Tunnel0ip address 10.9.1.1 255.255.255.0

mpls label protocol ldpmpls ipkeepalive 10 3tunnel source TenGigabitEthernet2/1/0

tunnel destination 10.1.3.2!interface Loopback0ip address 10.10.10.10 255.255.255.255!

interface TenGigabitEthernet2/1/0mtu 9216ip address 10.2.1.1 255.255.255.0

!interface TenGigabitEthernet9/1no ip address

!interface TenGigabitEthernet9/1.11vrf forwarding VPN1encapsulation dot1Q 300ip address 192.168.1.1 255.255.255.0

!interface TenGigabitEthernet9/2mtu 9216no ip address

xconnect 10.10.10.11 200 encapsulation mpls!router bgp 65000bgp log-neighbor-changesneighbor 10.10.10.11 remote-as 65000neighbor 10.10.10.11 update-source Loopback0

neighbor 192.168.1.2 remote-as 100!

address-family vpnv4neighbor 10.10.10.11 activateneighbor 10.10.10.11 send-community extended!address-family ipv4 vrf VPN1

no synchronizationneighbor 192.168.1.2 remote-as 100neighbor 192.168.1.2 activateneighbor 192.168.1.2 send-community extended

!ip route 10.10.10.11 255.255.255.255 Tunnel0ip route 10.1.3.0 255.255.255.0 10.2.1.2


29/29

White Pape

PE2 Configuration

vrf definition VPN1rd 100:1address-family ipv4

route-target both 100:1exit-address-family!

mpls ldp neighbor 10.10.10.10 targetedmpls label protocol ldpmpls ldp router-id Loopback0 force

!interface Tunnel0ip address 10.9.1.2 255.255.255.0mpls label protocol ldp

mpls ipkeepalive 10 3tunnel source TenGigabitEthernet3/3/0

tunnel destination 10.1.1.1!interface Loopback0ip address 10.10.10.11 255.255.255.255

!interface TenGigabitEthernet2/1mtu 9216no ip addressxconnect 10.10.10.10 200 encapsulation mpls

!interface TenGigabitEthernet2/3mtu 9216no ip address

!interface TenGigabitEthernet2/3.11vrf forwarding VPN1encapsulation dot1Q 300ip address 192.168.2.1 255.255.255.0

!interface TenGigabitEthernet3/3/0mtu 9216

ip address 10.3.1.2 255.255.255.0!router bgp 65000bgp log-neighbor-changesneighbor 10.10.10.10 remote-as 65000neighbor 10.10.10.10 update-source Loopback0neighbor 192.168.2.2 remote-as 200!address-family vpnv4neighbor 10.10.10.10 activateneighbor 10.10.10.10 send-community extendedexit-address-family!address-family ipv4 vrf VPN1no synchronizationneighbor 192.168.2.2 remote-as 200neighbor 192.168.2.2 activate

neighbor 192.168.2.2 send-community extendedexit-address-family

ip route 10.10.10.10 255.255.255.255 Tunnel0

ip route 10.1.1.0 255.255.255.0 10.3.1.1

Documents

MPLS VPSs in IP Tunnel Environments