nasir ghani

8/3/2019 nasir ghani

1/35

Advanced Cyber-Infrastructure Laboratory 1

Network Scheduling Services forNetwork Scheduling Services forEmerging ApplicationsEmerging Applications

Nasir GhaniNasir Ghani

Advanced Cyberinfrastructure LabAdvanced Cyberinfrastructure Lab

ECE Dept, University of New MexicoECE Dept, University of New Mexico

Albuquerque, NM, USAAlbuquerque, NM, USA

http://ece.unm.edu/~nghanihttp://ece.unm.edu/~nghani

Frontiers in IT (FIT) 2011Frontiers in IT (FIT) 2011Islamabad, PakistanIslamabad, Pakistan

December 19December 19thth, 2011, 2011


2/35


OutlineOutline

Background

Key Developments

Scheduled Services Design

Ongoing Efforts

Conclusions


3/35


IntroductionIntroduction

Advances in high-speed networking technologies: Packet/frame switching @ layers 2, 3:

IP/MPLS, Carrier Ethernet

Circuit-switching @ layers 1, 1.5:

Wavelength division multiplexing(WDM), next-gen SONET

Traffic engineering (TE) control standards:

Generalized MPLS (GMPLS), path computation, etc

Focus shifts to applications

Leverage infrastructures to support client needs Surge in e-science & commercial demands

Diverse service needs: scalability, flexibility, velocity


4/35


Optical DWDM Transport/Switching Large tributaries (10-40-100 Gbps lambdas)

High scalability (terabits/fiber)

OXC, OXC+DCS, R-OADM

Integrated EDFA (pre, in-line)

GMPLS control (OSPF-TE, RSVP-TE)

Next-Gen SONET/SDH, MSPP Robust non-TDM mappings (GFP)

Efficient capacity matching (VCAT)

Dynamic bandwidth control (LCAS)

Grooming, Ethernet-over-SONET

IP/MPLS or Ethernet (Layers 2-3)

Multi-service IP QoS, advanced TE protocols

Carrier Ethernet solutions (PBB-TE, T-MPLS)

Line rates up to 100 Gbps and beyond

Streamlined Architectures

Backbone NetworksBackbone Networks

IP, Ethernet core

NGS ring/mesh

Optical mesh

OC-n/STS-nV,

OTN

OC-n/STS-nV,

OTN

10, 100 GbE,

ITU-T G.709


5/35


C & L-BandsS-Band

Legacy TDM, Ethernet

band

O-Band E-BandDWDM (0.8 nm, 0.4 nm)

FiberLoss(dB/km)

0.2

0.5

1300 nm 1600 nm1400 nm 1500 nm

1.0

5

SMFwater-peak

profile

Improved low-

water-peak fibers

Single Mode Fiber (SMF) ITU-T Spectrum

DWDM Transmission (Mid-1990s)

EDFA

Mux /demux

Lasertransponders

SONET

SONET

SAN

SAN

Cable

Cable

Multiple colors per fiber:

Frequency division mux(FDM)

Massive scalability (terabits):100+ s (2.5, 10 Gb/s each)

Full-band amplifiers (EDFA):

No costly per-ch regeneration

Transparent bypass features:

Optical cross connect (OXC)

IP/Eth

IP/Eth

SMF fiber

Wavelength Division MultiplexingWavelength Division Multiplexing


6/35


ClientRouter B

Optical Cross Connect (OXC)

Mux

Fabric

Control

Demux

No electronic conversion (transparent)

Multi-Hop Lightpath Routing

Optical Network

Optical bypass (no electronics),much lower cost/complexity

ClientRouterA

Automate provisioning of wavelengths

Rapid progress in architectures:

GMPLS, UNI, OTN,ASTN, etc

Extensive research developments:Switching designs, RWA, survivability,

network design, etc

Optical Wavelength RoutingOptical Wavelength Routing


7/35


OutlineOutline

BackgroundBackgroundBackground

Key Developments

Scheduled Services DesignScheduled Services DesignScheduled Services Design

Ongoing EffortsOngoing EffortsOngoing Efforts

ConclusionsConclusionsConclusions


8/35


High energy physics,

fusion, astrophysics

Bio-informatics,

genomics

Experimentation, computation generate massive datasets

Need to transfer, visualize, remotely steer sensors

Large Hadron Collider, Spallation Neutron Source,

AdvancedLight Source, Terascale Supernova Initiative, etc

Gene expression, sequence analysis

Terabyte-level datasets common

Need to transfer, visualize, steer

Joint Genome Institute

Climate & geographicalchange modeling

Long/short-term connections

Very massive datasets

Visualization component

Some Facts

Petabytes-exabytes dataset sizes (5 yrs)

Deterministic transfer requirements

Dedicated bandwidth & low jitter (QoS)

Data backhauling, router-to-router TE

Scheduled demands/work-flows

EE--Science ApplicationsScience Applications


9/35


Global Ring Network for Adv.Applications Development

Global LambdaIntegrated Facility

DOE Energy Sciences Net (ESNet)DOE UltraScience Net (USN)Internet2

European Union

Key NREN InfrastructuresKey NREN Infrastructures

National Lambda Rail CANARIE (Canada)

Some Facts

Blend networks/computing/storage:

Grid-computing, workflow paradigms

Many technologies deployed (Gbps-Tbps):

IP, MPLS, Ethernet, DWDM, SONET/SDH

Increased inter-domain peerings to supportwider collaborations

Interconnection with state RENs (Quilt)Abilene


10/35


2005 2014

Demand GrowthDemand Growth

Traffic Increase 10x Every 4 years


11/35


Emerging Service RequirementsEmerging Service Requirements

Immediate on-demand reservation Generally suited for most mid-sized transfers (gigabytes)

Already implemented in most networks (k-SP heuristics)

Extensive standards support (routing, signaling, PCE)

Further need for network connection scheduling

Massive transfers preclude on-demand reservations

Scientific workflows/apps need timing of network connections

New advance reservation(AR) services stagger users:

I want connection service from A-B from time tstartto tend

Many commercial applications as well:

Special broadcasts, video conferencing, storage backups, etc


12/35


OutlineOutline


Key DevelopmentsKey DevelopmentsKey Developments

Scheduled Services Design

Ongoing EffortsOngoing EffortsOngoing Efforts



13/35


Overview & ChallengesOverview & Challenges

Prior work in AR Various AR scheduling algorithms proposed (IP, DWDM)

Largely heuristic (graph-theoretic) schemes, some ILP:

Maintain link BW-timeline windows, search for feasible routes

Many variants as well, e.g., variable start/stop, data sizes, etc

Open areas

Rerouting of future reservations:

Very few studies to date, no optimization work either

Real-world network implementation:Existing focus mostly on idealized algorithm design


14/35


AR Rerouting StrategiesAR Rerouting Strategies

Overview of approach Reroute non-active future reservations (non-disruptive)

Tradeoff overhead (complexity) vs optimality (effectiveness)

Existing studies: try to min. # rerouted paths

Open issues:

- Path selection (new request) to minimize disruptions

- Proper resource re-distribution, i.e., prevent future call rejections

Our contributions

Integer linear optimization (ILP) formulation for rerouting

Graph-based rerouting heuristics:

- Apply load-balancing concepts (extend gains in IR settings)

- Candidate path concept to streamline rerouting selection


15/35


RelativeRun-Time

Complex

ity

SolutionAlgorithms

No re-routing Graph-based

heuristics

Polynomial

time, O(Nk)

Global ILP

rerouting

Exponential

time, O(2N)

ExponentialO(2n),n


16/35


Global, idealized settings Assume demands known a-priori (real-world are sequential)

Use to bound performance, various AR-related studies:

- Assume full visibility (all connections inactive, schedulable)

- Segment timeline into fixed timeslots for arrival/departure

AR rerouting considerations

Timeline dimension induces huge complexities

E.g., 4-node mesh, 30 reqs, 50 timeslots 16x30x50=24k variables

Very lengthy compute times (global ILP formulation) Propose a dynamic ILP adaptation

ILP Optimization FormulationsILP Optimization Formulations


17/35


Dynamic ILP (DILP)Dynamic ILP (DILP)

Motivations Customize ILP to improve scalability

Preclude full a-priori demands, i.e., sequential on-demand

Integrate w. simulation, i.e., OPNET ModelerTM+ lp_solve

Key methodologies

Run shortened ILP foreach incoming requestw. inputs:

- BW timelines for all links (from current time)

- Pending and active reservation lists

Limit timeslots by choosing farthest look-ahead time:

I.e., based on reservation w. latest ending time

Drastically reduce variable counts (compute time):

E.g., 4-node mesh w. 3 pending reservations over 10 time slots

4x4x(3+1)x10=640 variables


18/35


Notational OverviewNotational Overview

Key variables V: Set of network nodes

E: Set of network links

rn: Reservation n, denoted by 5-tuple:

{source sn, dest dn, start time tsn, end time te

n, bandwidth bn }

ri: Incoming reservation

T: Current timeslot (when ILP triggered)

Rat: Set of active reservations at time t

Rpt: Set of pending reservations at time t

Tl: Max. look-ahead time, i.e., only re-optimize rn in [T , Tl]

pn,e,t: 1if reservation rn used link e at time slot t;0if not

ve: If node v is the egress node of link e

ev: If node v is the ingress node of link e

Objective

Minimize the total resource assigned to current pending & incoming reservations,

i.e., (bandwidthxpath length product)

te,n,

iTp

nr Ee l

Tt


19/35


Conditional ExpressionsConditional Expressions

te,n,

iTp

nr Ee l

Tt


20/35


AR Rerouting HeuristicAR Rerouting Heuristic

AR load-balancing

algorithm

AR load-balancing

rerouting

Two-Stage Heuristic Solution

AR Request

R = {src, dest, x, T1, T2}

Setup success

Feasible path found

No feasiblepaths

Setup failure

Insufficient resources

Setup success

Feasible paths found forinput request and allrerouted connections

C. Xie, et al , "Load-Balancing Connection

Rerouting in Advance Reservation Networks",

IEEECommunicationsLetters, June 2010.


21/35


AR LoadAR Load--Balancing HeuristicBalancing Heuristic

RegularAR (Non-Rerouting)1) Compute Kshortest paths from src-destoverG(V, E),

only considerfeasible links, i.e., capacity xin [T1, T2]

2) Compute bottleneck bandwidth of each path

in desired interval [T1, T2] (min. BW of all path links)

3) Select feasible path with max. bottleneck bandwidth

AR Request 5-TupleR = {src, dest, x, T1, T2}

src: Source node

dest: Dest node

x: Req. capacity

T1: Start time

T2: Stop time

A

C

G

E

DF

B

Incoming request: R = {A, E, 0.5X, T1, T2}

Path1

Path2

is bottleneck bandwidth,

Path1 feasible if 0.5XH

H

X

time

Link A-B Timeline

t T1 T2

XLink B-E Timeline

timet T1 T2

X

Path 1

t T1 T3

H


22/35


AR LoadAR Load--Balancing ReroutingBalancing Rerouting

Rerouting Phase

LetR= {src, dest,x, T1, T2}

if (found load-balancing path forR)

- Build G(V, E)where weight of a link is the

min. # of resv. to exceed (1-)x in [T1, T2]

- Compute shortest src-destpath on G(V, E),

i.e., candidate path that has smallestrerouting candidate set(RCS)

- Try to re-establish all connections in RCS,

success if all found (use temp. graph copy)

else

Reject requestR

Overall Goals Find capacity for a request by rerouting

a minimum number of reservations

Recover partial capacity,x(1)

Use modified Dijkstras SP algorithms

and previous AR LB (polynomial time)

Incoming request: R = {A, E, 0.5X, T1, T2}

Let=0.5, henceR = {A, E, 0.25X, T1, T2}

Candidate pathA-C-D-Egives smallest

RCS, i.e., 4 reservations

Minimum service disruptions

G(V , E) for interval [T1, T2]

1

A

C

G

E

DF

B

3

2

12

1

3 2

4


23/35


OPNET ModelerTM

Overview

State-of-the-art tool, widely-used

Complete GUI, ease of use

Full C/C++ coding interface

Integrated lp_solve ILP toolkit Hierarchical modeling:

Subnet, node, link, process

In-House Expertise

Full development site:10+ licenses, 30,000+ LOC

Advanced protocols suite:

IP, MPLS, Eth, SONET, optical

Extensive topology database

Performance EvaluationPerformance Evaluation


24/35


Simple 4Simple 4--Node NetworkNode Network

Testcase Parameters 30 random connection requests

10 Gbps links speeds

0.2 - 1 Gbps requests (200 Mbps increments)

Exp. inter-arrival / hold / book-ahead times

Scheme Run Time

LB Heuristic 30 sec

Dynamic ILP 45 sec

Global ILP 3 days

-4.44E-16

0.05

0.1

0.15

0.2

9 10 11 12 13 14 15 16 17

BBR(%)

Load (Erlang)

Min-hop (no rerouting)

Load-balancing Rerouting

DILP

GILP

Topology

Analysis Results


25/35


88--Node NetworkNode Network

Testcase Parameters 1,000 random connection requests

10 Gbps links speeds

0.2 - 1 Gbps requests (200 Mbps increments)

Exp. inter-arrival / hold / book-ahead times

Topology

Scheme Run Time

LB Heuristic 2 min

Dynamic ILP 6 min

Global ILP Unbounded

0.001

0.01

0.1

25 35 45 55 65

B

BR(%)

Load (Erlang)

Min-hop

LB-R

DILP

Analysis Results


26/35


0.001

0.01

0.1

50 70 90 110 130 150 170

BBR(%)

Load(Erlang)

HR

LR

LB-R =0.5

0.32

0.34

0.36

0.38

0.4

0.42

0.44

50 70 90 110 130 150 170

BandwidthUtilization(%)

Load(Erlang)

HR

LR

LB-R =0.5

1616--Node NSFNETNode NSFNET

Topology Testcase Parameters

10 Gbps link speeds

0.2-1 Gbps requests (200 Mbps increments)

Exp. inter-arrival / hold/ book-ahead times

Heuristics only, ILP schemes do not converge

Analysis Results 20-80% higher carriedload/revenues

Blocking Resource Utilization


27/35


0.0001

0.001

0.01

0.1

180 230 280 330 380 430 480 530 580

Load (Erlang)

BB

R

(%

)min-hop

LRHC-R

LB-R =0.5

2727--Node Deutsche TelekomNode Deutsche Telekom

Topology Testcase Parameters 10 Gbps link speeds

0.2-1 Gbps requests (200 Mbps increments)

Exp. inter-arrival / hold/ book-ahead times

Heuristics only, ILP schemes do not converge

Add HC-R heuristic

Analysis Results

Blocking Resource Utilization

2.9

3

3.1

3.2

3.3

3.4

3.5

3.6

200 250 300 350 400 450 500 550

Load(Erlang)

MeanPathL

ength(Hop)

min-hop

LR

HC-R

LB-R =0.5


28/35


Overhead ComparisonsOverhead Comparisons

Measure average time between rerouting events

Compare proposed LB rerouting with hop count (HC) rerouting

Analysis Results

NSFNET Deutsche Telekom

0

50

100

150

200

250

300

350

50 70 90 110 130 150 170

(1000timeunits)

Load(Erlang)

TimeUnitsperreroutedconnection

HC-R

LB-R =0.5

0

20

40

60

80

100

120

140

160

180

200

200 250 300 350 400 450 500 550

(1000timeunits)

Load (Erlang)

Time

Unitsperreroutedconnection

HC-R

LB-R =0.5


29/35


OutlineOutline


Key DevelopmentsKey DevelopmentsKey Developments

Scheduled Services DesignScheduled Services DesignScheduled Services Design

Ongoing Efforts



30/35


RealReal--World ChallengesWorld Challenges

Current implementation status AR schemes require global timelines for all links:

Real world distributed control, multiple domains

Existing protocols for IR only (GMPLS):

Routing, signaling, path computation, etc

Few centralized/proprietary AR solutions:

ESNetOSCARS, EU CRISM testbed

Our contributions

First complete framework fordistributedAR (IEEE Globecom10) Augment existing protocols w. timeline state (OSPF-TE)

Further efforts on multi-domain AR


31/35


Source

Destination

Domain 1

Domain 2

Domain 3

Domain 4

Domain 5

Domain 6

Overview

Real-world networks are complex, decentralized

A single entity cannot maintain global state:

Scalability, privacy reasons

Some standards emerging, key research area:

On demand IR operation only

Propose distributed AR solutions, limited visibility

Related ChallengesRelated Challenges


32/35


Proposed ApproachProposed Approach

Topology

Abstraction

Topology

Abstraction

Hierarchical

Inter-Domain

Routing

Hierarchical

Inter-Domain

Routing

AR Scheduling&

Signaling Setup

AR Scheduling&

Signaling Setup

Condensed

domain state

Extend GMPLS Standards

Physical topologies

and resources

Global abstract

views

Inter-domain AR

routes & schedules

Routing policies,

database MIB, timers

Skeleton path scheduling

algorithms & policies


33/35


Full Mesh

Border nodes revealed

Simple Node

Max. summarization

Objectives

Optimize externally-divulged (timeline) state

Various renditions: simple node, mesh, star, bus, etc

Well-studied for IR provisioning (IP, DWDM)

Proposed first extension for AR settings

Topology AbstractionTopology Abstraction

Full Domain Topology

G(V,E)


34/35


ConclusionsConclusions

Many cyber-infrastructure advances Maturation of key technologies (Layers 1-3)

Focus now on expanding services & applications

Many directions to build upon

Some crucial open research problems

Network scheduling: benefits from load-balancing, rerouting

Need to buid real-world proof-of-concept test-beds

Many avenues for collaboration, development, training


35/35


Thank You!Thank You!

Question

&Answer

Documents

nasir ghani