Cisco Service Provider Architecture and Strategy

Evolved Programmable Network

Jaroslaw Grabowski

Consultant System Engineer

May 2015

Cisco Service Provider Architecture and Strategy

© 2015 Cisco and/or its affiliates. All rights reserved. 2

Executive notes

Strategic directions

Strategic technologies

Technical High Level view

End-to-End Architecture and products positioning

Simplification and Layers reduction

Programmability, Virtualization and Orchestration

Q&A

Service Provider Networks. Wireline operations Agenda


Grade of Packet Networks

Home Grade Defined by users’ preferences and budget

Enterprise Grade Defined by needs Used by single entity/corporation

Carrier Grade Shared by multiple of entities. High Scale Used for mission critical applications. Source of income for Service Providers. 99.999% availability (5 min unavailable per year), etc..

Today Subject

4 © 2015 Cisco and/or its affiliates. All rights reserved.

Executive Notes. Strategic directions


Business Motivations High level observations

Service Providers business:

Users consume more bandwidth

Users are not willing to pay more in competitive environment

Traffic & Services

Revenue

Costs

Time


Business Motivations High level directions

Time

VAS Opportunity

Directions:

Work on Value Added Services (VAS) and new Monetization strategies

Move bits cheaper. Optimize Total Cost of Ownership (TCO)

Traffic & Services

TCO optimization


Business Motivations High level directions

TCO optimization

Time

VAS Opportunity

Directions:

Work on Value Added Services (VAS) and new Monetization strategies

Move bits cheaper. Optimize Total Cost of Ownership (TCO)

Hardware Efficiency

Network Operations Simplification

Programmability & Orchestration & Automation

Validated Design recommendations

Traffic & Services

Main Drivers


TCO (Total Cost of Ownership)

Cost efficiency

Hardware, Software, Installation and integration of hardware and software, Warranties and licenses, License tracking – compliance, Migration expenses, Risks: susceptibility to vulnerabilities, availability of upgrades, patches and future licensing policies, etc. Operation expenses: Infrastructure (floor space), Electricity (for related equipment, cooling, backup power), Testing costs, Downtime, outage and failure expenses, Diminished performance (i.e. users having to wait, diminished money-making ability), Security (including breaches, loss of reputation, recovery and prevention), Backup and recovery process, Technology training, Audit (internal and external), Insurance, Information technology personnel, Corporate management time. Long term expenses: Replacement, Future upgrade or scalability expenses, Decommissioning

OPEX 80%

5 Years

0

20

40

60

80

100

1 2 3 4 5

CAPEX


Executive Notes. Cisco development directions


Evolved and Programmable Simplicity and Performance


Evolved and Programmable Simplicity and Performance

Easy Interface


Evolved and Programmable Networks Simplicity and Performance


NCS NCS

APIs

APIs

EDGE CORE

Access

VM VM

Edge

Core

VM

Access

Evolved Services Platform

VM / Storage Control

Service Catalog Service Orchestration Apps

VM

Applications and Services

CDN

Easy Interface (API)


Evolved and Programmable Hiding complexity - chain


NCS NCS

APIs

APIs

EDGE CORE

Access

VM VM

Edge

Core

VM

Access




VM


CDN

Programmable environment delivering services

Management and orchestration

Applications


Evolved and Programmable Hiding complexity - chain


NCS NCS

APIs

APIs

EDGE CORE

Access

VM VM

Edge

Core

VM

Access




VM


CDN

Customers (or Everything in IoE) using Application and Services

Service Provider focusing on Value Added Services creation and delivery.

Self organizing network delivers network functions

Cisco to develop


Evolved and Programmable Networks


NCS NCS

APIs

APIs

EDGE CORE

Access

VM VM

Edge

Core

VM

Access




VM


CDN

Network Simplification

Network Programmability

Subject for discussion in this session.


Cisco Design Guides.

Cisco

UMMT v1.0

Cisco

UMMT v2.0

Cisco

UMMT v3.0

Cisco

FMC v1.0

Cisco UMMT

Unified MPLS for Mobile Transport

Fixed and Mobile Convergence

EPN and assumptions for next are defined

Dec2011 Sep2012 Apr2012

Feb2013

Cisco

FMC v2.0

Sep2013

Cisco

EPN v3.0

Apr2014

Migration from SDH to packet networks

Adding wireline Corporate/Residential

Evolve Programmable Networks

Adding (SDN type) programmability and orchestration

Cisco

EPN v4.0

Oct2014


Packet networks Architecture


Cisco EPN (Evolve Programmable Networks) All services from single network

Residential Services

Internet, Voice (Basic Package)

Video, Mobility (3Play, 4Play)

Corporate Services

L0VPN (Optical Lambda)

L1VPN (E-Line, TDM circuit)

L2VPN (E-LAN)

L3VPN (Routing and Internet)

L3VPN+ (IPsec Secured VPNs)

Transport Services

Wholesale

Mobile and IP RAN

DC Interconnect


Cisco EPN (Evolve Programmable Networks) Single network for all Services

Multiple locations (POP – Point of Presence)

Multiple device types and vendors

Multiple services

Multiple government requirements

Proven design principles

Reduce complexity!! Make problems smaller!! Divide an Conquer

Crate layers and building blocks

Unification over POPs

Shortcuts over layers are good for short time in long term causes complexity and chaos


EPN Architecture High Level View

Core

Aggregation

Access

Edge

Edge

Network layers hierarchy:

Subscribers/Customers

Access Layer

Aggregation

Edge

Core

Optical Transport

Functional blocks

Network Management Centre

Data Center and Content/Applications

Interconnection & peering


Access Layer

Adaptation to specific media (Fiber Optic, Metallic, wireless)

Unifying on Ethernet

L2 – transparent, L3 for advanced services

Massive deployment

Simple & Low CAPEX

Aggregation Layer

Aggregating multiple Access Nodes to high speed links

Carrying multiple services with MPLS separation

‘Real’ routers but very often carrying L2 flows over emulated VPLS or H-VPLS

EPN Architecture Access and Aggregation

Core

Aggregation

Edge

Edge


Edge layer

SEN – Service Edge Node

Residential SEN – BNG/BRAS

Business SEN – MSE (Multiservice Edge)

Video SEN

RNC - mobile

Core layer

Fast and reliable data forwarding and routing

Transport (DWDM) layer

Dense, long distance and reliable data transport

EPN Architecture Core, Edge, Transport

Core

Aggregation

Access

Edge

Edge


EPN Architecture Hierarchy

Core

Aggregation

Access

Edge

Edge

Hierarchical design is proven architecture for simplification of IP/MPLS networks

Hierarchy: each layer has specific role

Modular topology - building blocks

Unification for PoP and Regions

Easy to grow, understand, and troubleshoot. Adding new nodes does not destabilize the network

Creates small fault domains - clear demarcations and isolation

Promotes load balancing and redundancy

Promotes deterministic traffic patterns

Incorporates balance of both Layer 2 and Layer 3 technology, leveraging the strength of both


EPN Architecture Summary

Core

Video Broadcast VoIP VoD iFrame Cache

Managed Business Services

(Storage, VoIP, Security)

Authentication And Billing

Broadband Policy Manager

Aggregation Access

Agg DSL

Cable

FTTX

Ethernet

Business MSE

Edge

BRAS

DPI

Residential

STB

Mobile

Corporate

Business

Corporate

Core DWDM SDH, TDM Regional DWDM EoDWDM,

Xponder 10G, 40G, 100G IPoDWDM

ROADM, WXC,

Tunable


Cisco Products Positioning


SP Products Positioning. Wireline focus Access and Aggregation

Aggregation Access

Core Edge

100GE

100GE

PRIME Management portfolio

ASR 901 ASR 901S

ME3600X

ASR9000v

ME3600X-24CX

ME4600 FTTx system Eth. Point-Point GPON

SP WiFi ISRs

GPON

ME1200 NIT

xDSL MSAN

ASR920

UBR Cable & HFC

ME4600 Dense Access 100K subscribers from single Rack

2x100GE uplink NG-PON2

ASR 920 Range of new chassis Pay as you growth model


SP Products Positioning Access and Aggregation

Aggregation Access

Core Edge

100GE

100GE


ASR 901 ASR 901S

ME3600X

ASR9000v

ME3600X-24CX

ME4600 FTTx system Eth. Point-Point GPON

ME3800X

ASR9001

ASR903

ASR9006

ASR9010

SP WiFi

ASR902

ISRs

GPON

ME1200 NIT

xDSL MSAN

ASR920

UBR Cable & HFC

400G switching 100GE

8x100GE LC


Small Access Nodes connectivity:

1GE: 5-20

Medium Access Nodes connectivity:

1GE: 10-40

10GE: 2-4

Big (with Redundancy)

Access Nodes connectivity:

1GE: 20-100 +

10GE: 4-20 +

Ethernet

only

ME3600x, ASR901, ASR920 ASR9001, ME3800x, ASR9000v

ASR9000

Mixed

Ethernet

+ TDM

CES

ASR902, ME3800x-24cx

ASR901

ASR9000

ASR 903, ME3800x-24cx

ASR9000

ALL: IP/MPLS, 10GE MPLS uplink, Synchronous Ethernet, power redundancy Extended : Control/Switching redundancy, CES (Circuit Emulation), 100GE

Ethernet transport Access & Aggregation IP/MPLS controlled


ASR9010

ASR9922

ASR 1000

ASR9001

ASR9006

ASR9912

SP Products Positioning Core, Edge, Optical Transport

Aggregation Access Core Edge

100GE

100GE


20Gbps/Slot

Modular switching

Modular switching

New generation 400GE NPU


ASR9010

ASR 9000

ASR9922

ASR 1000

ASR9001

ASR9006

ASR9912 CRS1, CRS-3, CRS-X

NCS 6000

SP Products Positioning Core, Edge, Optical Transport

Aggregation Access Core Edge

100GE

100GE


NCS 2000

NCS 4000

20Gbps/Slot 128Tbps/System

NCS 6000 IP and Optical integration

2T per slot (2014)


SP Products Positioning Data Center, Cloud, Content distribution

Aggregation Access

Core Edge

100GE

100GE

Management and Orchestration

UCS Unified

Computing

System

x86 servers

Nexus

2000

5000

7000

9000 ACI

Application

Centric

Infrastructure

Videoscape

Content

Distribution

Systems

DC orchestration

NfV Network

Function

Virtualization

Network @

x86 servers

Routers

Switches

Firewalls

etc.


Typical Wireline Architecture

3rd party Access Networks

Rack Centre Cisco

Nexus

Cisco

UCS

10GE Rings

100GE

Multiplanar Core

PE-node

ASR9006

P-node

ASR9010

P-node

ASR9010

P-node

ASR9010

P-node

ASR9920

ME4600

ASR901

ASR920

ASR903

Dense Access

Sparse Access


SP Products Selection. Wireline operations Network Management & Orchestration

Aggregation Access

Core Edge

100GE

100GE

Prime

Central

Prime Performance Manager

Prime Provisioning

Prime Optical Prime Network

OSS/BSS

EMS & NMS


SP Products Selection. Wireline operations Network Management & Orchestration

Aggregation Access

Core Edge

100GE

100GE

WAN Automation Engine

ESP. Evolved Service Platform

Prime

Central

Prime Performance Manager

Prime Provisioning

Prime Optical Prime Network

SDN component

OSS/BSS

EMS & NMS

Elastic Service Controller NSO ……


Cisco Prime Network

Network and Service Management • MPLS, CE, IPRAN/MToP support

• Service discovery, network & service maps

• Service fault management & troubleshooting

• Graphical fault visualization

• Complete CE and MToP service activation

• Activation “Point & Click” GUI or via NB API

• Topology-based root cause

• Service impact analysis

• Graphical workflow builder

Foundation Abstract VNE model and mediation

layer

Distributed scale, carrier class, HA

Telnet, web service and SNMP APIs

SDK and developer support

Sun/Solaris server; Windows client

Customizable, configurable

NB Event, Alarm &Ticket

notifications

Solution integrations with

provisioning, inventory and

performance systems

Element Management NE and topology auto-discovery

NE Physical & Logical Inventory

Network Topology

Event, alarm and user-TCA management

Configuration support (script builder)

200+ built-in configuration scripts

Open toolkit for extensions

NE configuration archiving

NE Image management

Security: authentication, RBAC

50+ device families, 300+ NE-types


Network Simplification


Biggest investment Physical infrastructure

Out of control factors

Customers locations

Costly and time consuming to build

Infrastructure

Fiber optic ducts/lines

Access lines (FO, Copper, Cable, etc)

Main PoPs with appropriate infrastructure


Creating services and fulfilling requirements

Business Services Provisioning

Residential Services Provisioning

Access Network

Aggregation Network Spanning Tree, QinQ management,

Edge & Core Network MPLS

IP RAN


Where complexity come from? Too many things to control..

Business Services Provisioning

Residential Services Provisioning

Access Network

Aggregation Network Spanning Tree, QinQ management,

Edge & Core Network MPLS

IP RAN

Multiple technologies

Multiple layers

Multiple touch-points for service provisioning

Limited End-End service visibility

Manual interventions for protection, QoS, etc ..



What about MPLS? History in the nutshell

Aggregation Edge Multiservice Core

Access CPE Data

Center

Intelligent Edge

Multiservice Core

CPE

Large Scale Aggregation

Intelligent Edge

Multiservice Core

CPE

Access Aggregation Data Center

Data Center Access

ATM/FR/SDH

High performance IP

VPNs, Fast Convergence, Traffic Engineering

Large scale , Manageability

L2 L2

L2

Ethernet QinQ

MPLS still evolving

Aggregation

L2 Ethernet QinQ

QinQ – cumbersome to manage and provision (VLANs…)

QinQ – Not scalable (4000 serv.)

L2 protection Cumbersome to manage

L2 aggregation appears to be cheaper but…..

L2 protection Slow and not predictable


No scalability issues

Local VLAN significance

50ms protection

and more to follow…


We do can keep existing investments and evolve…

Aggregation Edge Multiservice Core

Access CPE Data

Center

Intelligent Edge

Multiservice Core

CPE


Intelligent Edge

Multiservice Core

CPE

Access Aggregation Data Center


Intelligent Edge

Multiservice Core

CPE

Data Center Access

Efficient Access

ATM/FR/SDH

Virtual Data Center

Dynamic Optical Transport

High performance IP

VPNs, Fast Convergence, Traffic Engineering

Large scale , Manageability

Simplification, Service Virtualization

L2 L2

L2

L3

L2

Seamless MPLS Transport

nLight 1

2

3

Ethernet QinQ


Biggest investment

Out of control factors

Customers locations

Costly and time consuming to build Infrastructure

Fiber optic ducts/lines

Access lines (FO, Copper, Cable, etc)

Main PoPs with appropriate infrastructure


Vertical split is more efficient Hiding Complexity step no 1.

Network Services

Transport

Configure once when node is added, replicated template or auto-configuration

Decouple Service Definition and Transport

Configured per service


Minimum touch points.

Service Layer

Configured per service

Minimum touch points

Transport

Services

Protection, QoS, synchronization, separation, automation, OAM

interface Ethernet ...

xconnect Target_node Service_ID

encapsulation mpls


Typical Wireline Architecture

3rd party Access Networks

Rack Centre Cisco

Nexus

Cisco

UCS

10GE Rings

100GE

Multiplanar Core

PE-node

ASR9006

P-node

ASR9010

P-node

ASR9010

P-node

ASR9010

P-node

ASR9920

ME4600

ASR901

ASR920

ASR903

Ethernet E-Line

(any VLAN)

Or TDM Circuit

for E1 ports

Ethernet

E-Lin

Or TDM

Circuit


MPLS Transport is hiding all the complexity

Transport Layer Ethernet/ MPLS/ IP

Configured only once per node!! Generic

templates

Protection. Sub second recovery across whole network (30-200ms). IP/MPLS tools

(BGP PIC, IP FRR, MoFRR, FC..)

Synchronization. SyncE, 1588v2.

Traffic Engineering. IP IGP, MPLS TE, MPLS

Segment routing

OAM Transport Level. IP/MPLS tools and Fault

Management.

Service Separation. IP/MPLS

QoS

Autonomic Networking. Zero-touch network

elements insertion

Network Services

Transport


High scale MPLS networks requires hierarchy

Cisco proposition

Automatic and hierarchical label distribution over BGP

Transport

How to build MPLS transport network in EPN design documents

Future ultimate IP/MPLS networks on Segment Routing concept


Autonomic Networking

Automatic IP and Infrastructure Configuration download

Transport

NOC

Access-Aggregation Network

Services L2VPN • Sub-int • VLAN operations • QoS • MAC Security • VPLS, PW

Infrastructure template: • Interface IP • Loopback IP • LDP • RSVP • ISIS

Services L3VPN • Sub-int VLAN • ACL • QoS • VRF • PE-CE Routing


Extended SDN concept


Segment Routing Simplifying MPLS operations


“Classical” IP/MPLS in action LDP + IP routing

N1

N2

N3

N4

N5

N6

N7

N9

Via IGP (ISIS, OSPF). Each Router is building IP Routing Topology

Via LDP. Each Router is advertising its IP prefixes to label binding

Labels are used to program the path

Routing selects shortest path


“Classical” IP/MPLS in action LDP + IP routing

N1

N2

N3

N4

N5

N6

N7

N9

Each Router is building IP Routing Topology (ISIS, OSPF)

Each Router is advertising its IP prefixes to LDP binding

Routing selects shortest path

Labels are used to program the path

209 N9

209 N9

409 N9

409 N9

609 N9

609 N9 N9

N9

All Labels locally significant

N9 Packet/Frame/MPLS(VPN) etc.


Segment Routing. Simplification 1) Routing distribute labels. 2) Unique label per node.

N1

N2

N3

N4

N5

N6

N7

N9

Node N9 has label 909

Each Node has unique label assigned as node ID

Each Router is building IP Routing Topology AND distribute label to IP prefix binding.

Simple extension to ISIS, OSPF

909

606

707

404

505 303

202

101

Topology +labels

Topology +labels

Topology +labels Topology

+labels


Segment Routing. Simplification The same MPLS forwading

N1

N2

N3

N4

N5

N6

N7

N9

909 N9

909 N9

909 N9

909 N9

909 N9

909 N9 N9

N9

Node N9 has label 909

Node Segment

Segment Routing is using the same forwarding paradigm like ‘classical’ LDP based IP/MPLS

The same Label is maintained through specific segment

No changes on services layer. The same PseudoWire, L3/L2 VPN infrastructure is used.

Easiness of ECMP implementation

909

606

707

404

505 303

202

101


Traffic Engineering The biggest change

N1

N2

N3

N4

N5

N6

N7

N9

Congested Link

Typical use-case:

Avoiding Congested lines.

All routers are selecting shortest paths to the destination

Some links might be congested – causing traffic outages

Traffic Engineering is needed to steer traffic over ‘longer’ but less congested links

Shortest Path


Traffic Engineering RSVP-TE

N1

N2

N3

N4

N5

N6

N7

N9

Tunnel is needed

Congested Link

HeadEnd

RSVP-TE signaling protocol setups tunnel

HeadEnd sending downstream through RSVP-TE (PATH) requests

TailEnd confirms through RSVP-TE (RESV) message and tunnel is setuped.

All Mid-Point nodes keep soft state of the tunnel in the memory.

TailEnd

“Classical” RSVP-TE


Segment Routing Programmability Source Routing

N1

N2

N3

N4

N5

N6

N7

N9

909 N9

Adjacency Segment

HeadEnd “programming” path in the label stack

Nodes advertised “Adjacency Label” per link. E.g node N4 is advertising its link towards N5 as label 425

None of Mid-Points needs to keep the state of the tunnel. State is kept only in HeadEnd.

HeadEnd

425 404

909

606

707

404

505 303

202

101 425

Go to node N4

Take link N4-N5 (advertised as label 425)

Go to node N9


Segment Routing Programmability Source Routing

N1

N2

N3

N4

N5

N6

N7

N9

909 N9

909 N9

N9

Adjacency Segment

HeadEnd “programming” path in the label stack

Nodes advertised “Adjacency Label” per link. E.g node N4 is advertising its link towards N5 as label 425

None of Mid-Points needs to keep the state of the tunnel

HeadEnd

425 404

909 N9

425 909 N9

425

425

POP

909

606

707

404

505 303

202

101 425

Go to N4

Take link to N5

Go to N9


Real case example CoS based TE

• Tokyo to Brussels

– data: via US: cheap capacity

– voip: via Russia: low latency

• CoS-based TE with SR

– IGP metric set such as

> Tokyo to Russia: via Russia

> Tokyo to Brussels: via US

> Russia to Brussels: via Europe

• Tokyo CoS-based policy

– Data and Brussels: push the node segment to Brussels

– VoIP and Brussels: push the anycast node to Russia, push Brussels

Node segment to Brussels

Node segment to Russia


Scalability

N: # of nodes in the network

A: # of adjacencies per node

An SR (Segment Routing) core router scales much than with RSVP-TE

The state is not in the router but in the packet

N+A vs N^2


IETF • Simple ISIS/OSPF extension

• Considerable support from vendors

• Consensus reached...


Segment Routing Programmability Application control – full picture

N1

N2

N3

N4

N5

N6

N7

N9

Segment routing offers simplified programmability

Paths computation could be performed by centralized logic. SDN (Software Define Networks) approach.

Paths used for link/node protections

Paths for advanced Traffic Engineering

909

606

707

404

505 303

202

101

EPN Evolved Programmable Network Layer

ESP Evolved Services Platform Layer

Applications

Visualization/ Analytics

Bandwidth Orchestrator

Collector Programming

API


WAN Automation Engine Understand and control your network


SP Network in reality - it is not a cloud! Common question

How much bandwidth my services consume?

How traffic flow through specific links?

What will happen if something goes down?

How to expand the network most efficiently?

How to steer the traffic to increase the value of the network? (reduce congestion, re-use bandwidth, assure protection etc..)


SP Network in reality - it is not a cloud! Solutions

How much bandwidth my services consume?

How traffic flow through specific links?

What will happen if something goes down?

How to expand the network most efficiently?

How to steer the traffic to increase the value of the network? (reduce congestion, re-use bandwidth, assure protection and latency etc..)


• Use real statistics

• Simulate real routing protocols behavior

• Bi-directional

• Green is good

66

Capacity Visualization


• Use the Create Growth Plans tool using the demand growth percentage to see where to add capacity and when you will need it

67

Capacity Planning


Failure Impact Analysis Need to understand where traffic will go and what the impact will be if something changes on the network (planned or unplanned)

• Use MATE Design to visualize the network utilization

• Show the demands table

• Identify how traffic traverses the network

• In the Demands table, select the demand from London to Budapest

• Simulate failures or maintenance plans and examine where traffic will go…

• On the node Berlin, right click and select Fail

• …And what the impact will be

• Click an empty part of the plot to de-select the demand

• Recover the failure 68


References

PTT ISP Mobile MSO Enterprise &

Government


WAE – snapshot from inside

Multivendor Network Devices

WAVE Platform

ALU Juniper Cisco Huawei

Predictive Model

Visualization and Analytics

Demand/Path Placement Engine

Programming Modules Collection Modules

I2RS OpenFlow OnePK PCEP NMS/EMS NetFlow CLI SNMP BGP-LS Configlets

Collection Drivers Programming Drivers


WAE – snapshot from inside

Multivendor Network Devices

WAVE Applications

WAVE Platform

ALU Juniper Cisco Huawei

Cisco Applications

Other 3rd Party Applications

PRIME Base Client

App

REST APIs

MATE Design

MATE Live

Predictive Model

Visualization and Analytics

Demand/Path Placement Engine

Programming Modules Collection Modules

I2RS OpenFlow OnePK PCEP NMS/EMS NetFlow CLI SNMP BGP-LS Configlets

Collection Drivers Programming Drivers


Thank You