ARCHSTONE Advanced Resource Computation for Hybrid Service and TOpology NEtworks Project Review Meeting LBNL, Berkeley, CA October 5 th, 2010

ARCHSTONEARCHSTONE

Advanced Resource Advanced Resource Computation for Hybrid Service Computation for Hybrid Service

and TOpology NEtworksand TOpology NEtworks

Project Review MeetingProject Review MeetingLBNL, Berkeley, CALBNL, Berkeley, CA

October 5October 5thth, 2010, 2010

Personnel

• USC/ISI

• Tom Lehman

• Xi Yang

• ESnet

• Chin Guok

• Eric Pouyoul

• Inder Monga

• UNM

• Nasir Ghani

• Feng Gu

Current Network Provisioning

• Current Dynamic Network capabilities and deployments allow us to provision point-to-point "Ethernet" circuits or Layer 3 MPLS tunnels

lower layer is statically

provisioned

Current Network Provisioning

• The single layer provisioning model can be extended to Multi-Domain via the InterDomain Controller Protocol (IDCP)

• www.controlplane.net

• Using systems like OSCARS, DRAGON

• While the technology of the networks may differ, there is a single "service":

• Ethernet Framed Point-to-Point Circuit

lower layer is statically

provisioned

Desired - Vertical Intra-Domain

• Provision services at lower layer to create a topology element (link between routers) at the higher layer

• Subsequently provision any remaining bandwidth at the higher level

• may provision a 10Gbps LSC link in response to a request for 5Gbps immediate need

• remaining 5Gbps available for subsequent service requests

Desired - Multi-Service/Layer/Domain Provisioning

• Provisioning at multiple layers within a domain

• A variety of services (layer 3, layer2, layer 1, different technologies)

• Multiple peering at different technology layers between domain

ARCHSTONE Objectives

• ARCHSTONE objective is to enable this "single- layer point-to-point circuit provisioning" to evolve to "multi-layer topology provisioning"• The topology component is a very important paradigm shift

• Currently a provisioning event is single point-to-point circuit

• ARCHSTONE objective is to view service requests as topologies which may be a single point-to-point link, or a multi-node/link construct which requires multiple coordinated provisioning events to instantiate

• In addition, we want to be able to compute and instantiate topologies which allow for flexible and extensible "constraint" space or "multiple dimension computation"

• These dimensions include time domain, AAA, technology layers, performance, protection, restoration, measurement, and monitoring.

ARCHSTONE Focus and Approach

• Advanced Network Service Interface

• "Request Topology" and "Service Topology" concepts

• Common Network Resource Description schema

• Multi-Dimensional Topology Computation Element (MX-TCE)

• High Performance computation with flexible application of constraints

• Use OSCARSv6 as base infrastructure and development environment

Network Service

Requester

Network Provider

Agent

Network Resource

Description

Network Network Service Service InterfaceInterface

request

replyMX-TCE

OSCARSv6

MX-TCE

•Time Domain•AAA•Management (SNMP) Data•Administrator Requirements

Virtual Organization Specific View

Friday, 3amMaintenance View

IP View

SDN View

What paths are possible between A and B, between time i and ii, with specific policy.A B

User Specified Multiple ViewsESnet

IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

OSPF-TE

Topology

OSPF-TE

Topology

OSPF-TE

Topology

OSPF-TE

TopologyTopology

OSPF-TE

MX-TCE



Virtual Organization Specific View

Friday, 3amMaintenance View

IP View

SDN View

What paths are possible between A and B, between time i and ii, with specific policy.A BA B

User Specified Multiple ViewsESnet

IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

OSPF-TE

Topology

OSPF-TE

Topology

OSPF-TE

Topology

OSPF-TE

TopologyTopology

OSPF-TE

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

ESnet IP

ESnet IP

ESnet SDN

ESnet SDN

WDM Layer

WDM Layer

OSPF-TE

Topology

OSPF-TE

Topology

OSPF-TE

Topology

OSPF-TE

TopologyTopology

OSPF-TE

Service-Oriented MX-TCEService-Oriented MX-TCE

The core capability of the project is Multi-X Topology Computation Element (MX-TCE)

• Handling multi-dimensional information and constraints that are typically not considered in current network PCE engines

• Computing both paths and topologies for multi-layer networks

• Designed to integrate into OSCARSv6 and provide basic PCE services

• Also designed to be used as advanced Topology Computation engine for other applications and purposes

Network Service RequesterNetwork Service Requester

Network Provider AgentNetwork Provider Agent

Physical TopologyPhysical Topology

Abstracted TopologyAbstracted Topology

Service Topology (in green)Service Topology (in green)

Service TopologyService TopologyRequested TopologyRequested Topology

Abstraction Process

Path Computation

Network Services Interface

ARCHSTONE New Features/Objectives

• advanced Network Service Interface• request- and service- topology based

• calculate whole topologies (not just circuits)• service and user specific topology views• detailed explicit route objects containing all required information to

initiate and complete a multi-layer, multi-technology provisioning event

• pseudo-provisioning operations where the impacts of various potential provisioning scenarios can be evaluated.

• accept complex questions and return complex answers regarding service and topology provisioning

• operate in multi-layer, multi-technology, multi-vendor environments• maintains topology on all Layers (Layers 0-3)

ARCHSTONE New Features/Objectives

• integrate multi-dimensional information from the control, management, security, business, and administrative planes into the single computational space• utilize advanced algorithms within a complex computational space

• designed with large scalability and speed in mind

• abstracted data representations with vendor specific interface modules

• provide "services" for other “value-added” systems in science and network application spaces

Multi-Layer Networking – ARCHSTONE Role

• "Multi-Layer Networking, An Architecture Framework"• http://hybrid.east.isi.edu/twiki/pub/HybridMLN/Pubs/multi-layer-

architecture-v9.0.pdf• Capability Planes

• The CapabilityPlanes covered by this work include the following:• Control Plane requirements, design, and implementation• Service Plane Interface• Control Plane processing of information from the Service Plane, AA Plane,

Management Plane, Service Plane, and Data Plane.

DataPlane

ServicePlane

Management Plane

ControlPlane

AAPlane

ApplicationPlane

Generic DataPlane Layer

ControlPlane

AAPlane

ManagementPlane

ServicePlane

ApplicationPlane

Network Service Interface (NSI) Requirements

OGF Standards Status and ARCHSTONE Contributions

OSCARS version 6 Overview

Chin Guok (ESnet)

Requirements for Flexible Network Services

Network architectures and services - Increasing Complexity• Growing need to consider many more dimensions (or constraints) for

control and provision of network resources

• User services becoming more complex and varied - virtual organization specific resource control, on-demand creation of network topologies across multiple layers, extended user-to-network "conversations"

Essentials for multi-layer network control• Next-generation networks tend to be architected as a heterogeneous

“multi-layer, multi-technology” construct

• Multiple types of services simultaneously over common infrastructures

• Need to hide network details from users, i.e., “services virtualized”

Where are we now?• Initial renditions of user requested service provision paradigms have

been realized in DOE ESnet SDN, Internet2 DCN, and others.

• Challenging issues unresolved: rich service interface definition, dynamic topology computation, multi-layer control etc.

Atomic and Composite Network Services Architecture

Atomic Service (AS1)




Composite Service (S2 = AS1 + AS2)

Composite Service (S3 = AS3 + AS4)

Composite Service (S1 = S2 + S3)

Ser

vice

Abs

trac

tion

Incr

eas

es

Ser

vice

Abs

trac

tion

Incr

eas

es

Ser

vice

Usa

ge

Sim

plif

ies

Network Service Plane

Service templates Service templates pre-composed for pre-composed for

specific applications specific applications or customized by or customized by advanced usersadvanced users

Atomic services used Atomic services used as building blocks for as building blocks for composite servicescomposite services

Network Services Interface

Multi-Layer Network Data Plane

Examples of Atomic Services

Security Service (e.g. encryption) to ensure data integrity

Measurement Service to enable collection of usage data and performance stats

Monitoring Service to ensure proper support using SOPs for production service

Store and Forward Service to enable caching capability in the network

1+11+1

Topology Service to determine resources and orientation

Resource Computation Service* to determine possible resources based on multi-dimensional constraints

Connection Service to specify data plane connectivity

Protection Service to enable resiliency through redundancy

Restoration Service to facilitate recovery

(*MX-TCE)

Example Workflow Using Multiple Atomic ServicesNetwork Service Network Service

RequesterRequesterNetwork Service Network Service

ProviderProvider

ComputationComputation

Is there a 1GigE connection between A and Z for 2 hours within the next 24 hours?

ConnectionConnection

ConnectionConnection

MeasurementMeasurement

ComputationComputation

ProtectionProtection

Yes. That is available in the following time-windows: [1, 5], [8, 11], [19,24]. NS-ID = 1000

(SID=AZ001) Reserve 1GigE connection between A and Z at [1,3].

Reserved: GRI = es.net-A-Z-123, Service Topolgy = {A..Z}NS-ID = 1001

1+1+11

(SID=AZ001) Instantiate the Service Topology for GRI = es.net-A-Z-123.

Service Topology instantiated

(SID=AZ001) Add measurement points to Service Topology for GRI = es.net-A-Z-123.

Measurement points added to Service Topology at {A..X..Y..Z}

SID = AZ001SID = AZ001

(SID=AZ001) Are there resources for 1:1 path-protection for GRI = es.net-A-Z-123.

Yes!

Confirmed.

(SID=AZ001) Apply 1:1 protection to Service Topology GRI = es.net-A-Z-123.

End of SessionEnd of SessionSID=AZ001SID=AZ001

Standards Contribution

• Primary effort focused on Open Grid Forum (OGF)• OGF Network Services Interface (NSI) WG*

- Helped define service plane as an abstracted representation of transport plane resources (i.e. service topology)

- Convinced WG to include service types within the NSI protocol to cater for composable services model

- Develop the idea of a Service Termination Point (STP) which hides the complexity of multi-layer transport termination from the user

- Co-authors of the OGF NSI Architecture Framework document (recently submitted for general comments)

- Connection Service and NSI Protocol definition is in progress with good participation

• OGF InfraStructure On-Demand (ISOD) RG*- Explore the complex workflows in integrated scheduling of network, compute,

and storage resources

* Both WG and RG are co-chaired by people involved in the ARCHSTONE project

ESnet OSCARS

• On-demand• Private• Bandwidth guarantees• Advance reservations• ESnet SDN control

plane

OSCAR 0.6 Architecture

Notification Broker• Manage Subscriptions• Forward Notifications

AuthN• Authentication

Resource Manager• Manage Reservations

• Auditing

Coordinator• Workflow Coordinator

PCE*• Constrained Path

Computations(*MX-TCE)

Topology Bridge• Topology Information

Management

IDC API• Manages External WS

Communications

Path Setup• Network Element

Interface

Lookup• Lookup service

AuthZ*• Authorization

• Costing

*Distinct Data and Control Plane Functions

Web Browser User Interface

21

Example Graph of PCE Modules And Aggregation

Aggregate Tags 3,4

Aggregate Tags 3,4

Aggregate Tags 1,2

Aggregate Tags 1,2

PCERuntime

PCE 1PCE 1

Tag 1Tag 1

PCE 3PCE 3

Tag 1Tag 1

PCE 2PCE 2

Tag 1Tag 1

PCE 4PCE 4

Tag 2Tag 2

PCE 5PCE 5

Tag 3Tag 3

PCE 6PCE 6

Tag 4Tag 4

PCE 7PCE 7

Tag 4Tag 4

User + PCE1 + PCE2 + PCE3

Constrains (Tag=1)


Constrains (Tag=1)

User + PCE1 + PCE2 Constrains

(Tag=1)


(Tag=1)

User + PCE1 Constrains (Tag=1)


User ConstrainsUser Constrains



User + PCE4 Constrains (Tag=2) )

User + PCE4 Constrains (Tag=2) )


(Tag=4)


(Tag=4)


Constrains (Tag=4)


Constrains (Tag=4)


(Tag=3)


(Tag=3)

User ConstrainsUser Constrains

NB: Constraints = Network Element Topology Data NB: Constraints = Network Element Topology Data

Intersection of [Constrains (Tag=3)] and [Constraints

(Tag=4)] returned as Constraints (Tag =2)

Intersection of [Constrains (Tag=3)] and [Constraints

(Tag=4)] returned as Constraints (Tag =2)

ARCHSTONEMX-TCE

MX-TCE Architecture and Design

ARCHSTONE Project Documents, Status, Schedule

Tom Lehman (USC/ISI)

MX-TCE – Component Architecture

MX-TCE

PolicyManager

ResvManager

TEDBManager

TEWGTEWG

ComputationThread

TCE Client

Request Reply

ComplexPolicyEvaluator

ResourceDeltaModifier

PolicyPruningModifier

push

pull

ExternalPolicyServer

ExternalScheduler

push

pull

TEDBExternal Storage

WS, API, OSPF, SQL, XML(pull or push)

ReservationMaker

Focus is on speed and flexible incorporation of constraints into the computation process

MX-TCEMulti-Dimensional Topology Computation Engine

• TEDB Manager - manages the Traffic Engineering Database (TEDB)

• Policy Manager - manages policy data that are locally configured with the MX-TCE as well as from the rules imported from external policy server(s)

• Reservation Manager - manages reservation data that come from both the local reservations made with the MX-TCE and the reservations made with external scheduler(s)

• Computation Thread - implements path and topology computation algorithms that process the working graph and associated constraints and provide answers to questions asked by the TCE clients

For Details: Multi-X Topology Computation Element (MX-TCE) Architecture and Design

MX-TCE – Path and Topology Computation Work Flow

For Details: Multi-X Topology Computation Element (MX-TCE) Architecture and Design

• Path Computation• Constrained Search• Graph Transformation• Heuristic Search

• Topology Computation• Bridged star/tree• Meshed Multi-Connection

• Constraints• Prunable Constraints• Additive Constraints• Non-additive Constraints• Adaptation Constraints

MX-TCE – Flexible Use Cases within OSCARSv6 and by other Applications

All in One Computation

Intra-Domain Focused Computation

Topology Data Focused Computation Feeds Results to an Aggregate PCE

MX-TCEAll-in-one Constrained Path

Computation

CoordinatorWorkflow Coordinator

Topology BridgeTopology Information

Management

AuthZAuthorization Policy

Management

Resource ManagerManage Reservations and

Scheduling

Aggregate PCEAggregating Constrained Path

Computation Results



Management


Management


Scheduling

MX-TCETopology TE Constrained

Path Computation

PCE-1Policy Constrained

Computation

PCE-2Scheduling Constrained

Computation

PCE-NOther Constraints

MX-TCEIntra-domain Constrained Path

Computation



Management


Management


Scheduling

PCEInter-domain Constrained

Path Compuation

MX-TCE can also be used by other Applications

Standalone MX-TCE

NS

I A

PI

NS

I A

PI

NS

I A

PI

NS

I A

PI

Client Application

What topologies are available next week for 3 hours?

Here's Two

ARCHSTONEProject Documents

• ARCHSTONE AAA Information Model

• ARCHSTONE Multi-Dimension TEDB Adaptations Requirements

• ARCHSTONE Multi-Layer/Multi-Technology Provisioning Requirements

• ARCHSTONE Network Service Interface Requirements

• ARCHSTONE Standards Contributions

• ARCHSTONE Multi-Layer Networking Standards and Topology Definition Study

• ARCHSTONE Multi-Layer Networking Path and Topology Computation Study

• ARCHSTONE Multi-X Topology Computation Element (MX-TCE) Architecture and Design

• ARCHSTONE Multi-X Traffic Engineering Database (MX-TEDB) Architecture and Design

• ARCHSTONE Network Service Interface (NSI) Architecture and Design

http://archstone.east.isi.edu

ARCHSTONEProject Status and Schedule

• Completed first year activities

• Design and Architecture of Network Service Interface (NSI) and Multi-Dimensional Topology Computation Element (MX-TCE)

- Message formats and schema

- Topology Schemas (request topologies, service topologies, multi-layer/point/constraint)

- Software module architectures and designs

• Deployed software development and test environment on the ESnet Testbed

- OSCARS version 6 based

• Began additional detailed design and implementation

• Plans for Year 2

• Complete initial implementations of NSI and MX-TCE

• Deploy and test in Testbed

• Deploy and test on actual networks in a test and demonstration mode

Multi-Layer Networking Standards Study and Evaluation

Nasir Ghani (UNM)

Multi-Layer Network Topology Standards and Protocol Review

• Multi-Layer Network Topology Standards Study Objectives• Review state of standards bodies and industry groups in the area

of Multi-Layer Network topology descriptions and associated protocols

• The purpose of this activity was to review and understand these industry standards, as it applies to the ARCHSTONE advanced services: • and then be able to leverage this work to the maximum extent

possible

Multi-Layer Standards Overview

Internet Engineering Task Force Optical Internetworking Forum International Telecom Union

Open Grid Forum DICE (Dante-Internet2-CANARIE-Esnet)

http://www.ogf.org/

Multi-Layer Network Topology Standards Review

• Internet Engineering Task Force (IETF) Standard• Open Shortest-Path First-Traffic Engineering (OSPF-TE)• Border Gateway Protocol (BGP)• Resource Reservation Protocol-Traffic Engineering (RSVP-TE)• Constraint-Based Routing Label Distribution Protocol (CR-LDP)• Path Computation Element (PCE)• Psuedowire Working Group (PW3E)

• Optical Internetworking Forum (OIF) Standards• User Network Interface (UNI) 1.0• User Network Interface (UNI) 2.0• External Network-Network Interface (E-NNI) 1.0 Signaling• External Network-Network Interface (E-NNI) 2.0 Signaling• External Network-Network Interface (E-NNI) OSPF-Based Routing 1.0

• International Telecommunications Union (ITU-T) Standards

• Optical Grid Forum (OGF) - Network Description Language (NDL)

Standard Bodies InteractionsStandard Bodies Interactions

OIF Networking WG’sOIF Networking WG’sUNI, NNI specificationsUNI, NNI specificationsOIF Networking WG’sOIF Networking WG’s

UNI, NNI specificationsUNI, NNI specificationsITU-T SG-15, SG-13 WGArchitectures, L1 VPN

ITU-T SG-15, SG-13 WGArchitectures, L1 VPN

IETF WG’sArchitectures, protocols,

L1 VPN

IETF WG’sArchitectures, protocols,

L1 VPN

Multi-Layer / Multi-Domain Activities

Liason ActivitiesLiason Activities

High-Level Summary (IETF, OIF, ITU-T)

Multi-Layer Standards and Technology ReviewSummary

• ARCHSTONE work will leverage the industry standards• particularly in the area of the information model for topology descriptions• will also leverage the built in vendor use of standards based protocols (or

vendor proprietary versions)

• However, the R&E community needs to go further (and faster) then what is occurring in the standards bodies and associated vendor implementations. In particular, the standards bodies: • are not working on multi-layer, multi-constraint path computation details• have completed very little work on application of an Authentication,

Authorization, Accounting (AAA) model to the control plane• have completed very little work on scheduling of provisioned services• have not converged on Inter-AS interdomain E-NNI routing or signaling

protocols• are not addressing scalability and security to the degree required for the

R&E community

Grid-Networking & E-Science Community

• Open Grid Forum (OGF)• Network markup language (NML) WG to define hybrid layer networks

• Network descriptor language (NDL) for multi-layer network definition:- Use RDF web-interchange standard, define an ontology- 5 schemas: Topology, layer, capability, domain, physical

• Network Measurement Working Group (NMWG) Format- XML Based, utilized by PerfSONAR and OSCARS currently

• Dante-Internet2-CANARIE–ESnet (DICE) • Inter-domain controller protocol (IDCP) implementation:

- Domain-level controller entities defined (akin to PCE)- Signaling reservations, resource discovery, topology exchange, etc

• Based on NMWG for topology descriptions

• ARCHSTONE will utilize NMWG based framework with extensions as required for the advanced services

ARCHSTONE Use of ESnet Testbed

Xi Yang (USC/ISI)

ARCHSTONE - Use of ESnet Testbed

• Focus is on routers and WDM equipment

Full Testbed

ARCHSTONE Use


• Infinera has multiple capabilities that ARCHSTONE can leverage to enable advanced service provisioning:

• Protection/Restoration: Includes Dual-TAM Digital SNC Protection (D-SNCP), Single TAM D-SNCP, Unprotected Restorable

• Circuit Monitoring: Capabilities to monitor health of circuit via TL1, possibly other methods as well

• Unidirectional Digital Multicast: Point-to-Multipoint service allows for native broadcast services

• Topology Retrieval: Complete real-time topology can be retrieved with single TL1 command. Will also explore possible extensions to enable peering with routing protocols

• Signaling: GMPLS UNI which may support limited services (probably no explicit route specification), Network Management System API, and TL1

• For each of these we will explore how to integrate them into advanced ARCHSTONE services


• ARCHSTONE is interested in the multi-layer provisioning opportunities on the testbed

• Routers (Junipers) over WDM (Infinera)

• Main focus has been Infinera Equipment familiarization/evaluation and set up of software development/experiment/test environment

• ARCHSTONE is designed to integrate vendor capabilities into the service model

• ARCHSTONE NSI Protection Atomic Service may be realized via one of the native Infinera protection mechanisms

• An ARCHSTONE NSI "Service Topology" may utilize the native Infinera point-to-multipoint broadcast capability in response to a "Request Topology" with this capability requested

• The objective here is not to be dependent on a specific vendor feature set, but to show how unique vendor capabilities can be integrated into the ARCHSTONE service model

• OSCARSv6 has been deployed on the testbed as a development environment

Thank-you