May 2002© 2002 Yoram Ofek1 Satisfying the Requirements of Applications on a Single Packet Network Yoram Ofek Synchrodyne Networks, Inc. E-mail: [email protected]

May 2002 © 2002 Yoram Ofek 1

Satisfying the Requirements of Applications Satisfying the Requirements of Applications on a Single Packet Network on a Single Packet Network

Yoram OfekYoram OfekSynchrodyne Networks, Inc.

E-mail: [email protected]

Phone: (917) 601-7180


Applications - Generic Traffic Types

Playback:Playback:Machine-to-PersonMachine-to-Person

SingleSinglePacketPacket

NetworkNetwork

Person-to-PersonPerson-to-PersonCommunicationsCommunications

Typically with RateTypically with Rate

Machine-to-MachineMachine-to-MachineCommunicationsCommunicationsTypically No RateTypically No Rate


Person-to-Person: with Rate

Applications with some notion of rate: Most demanding: interactive - streaming media - voice/video

end-to-end delay < 100 ms continuous play - i.e., periodic

Will satisfy also: non-interactive: playback, large file transfers - Machine-to-PersonMachine-to-Person

The transition from circuit to packet switching the rate per person will increase 3-4 orders of magnitude: from 104 b/s to 108 b/s


Machine-to-Machine: No Rate

(Computing) Machines are still evolving rapidly e.g., capabilities: “Moore’s Law” - new applications

General characteristic: bursty - unpredictable in time/space All bits should be transferred correctly with no “shaping”:

max. throughput (burst) AND min. delay & loss e.g., distributed/parallel computing, data processing

tt

Traffic shape Traffic shape at the sourceat the source

tt

Traffic shape Traffic shape at the Destinationat the Destination

tt

Transfer withTransfer withMinimum DistortionsMinimum Distortions

NoNo““Shaping”Shaping”



Outline How to support the two generic traffic types:

1. Ring networks

2. Convergence routing

3. Time-driven - switched networks

4. Dynamic optical networking

Machine-to-MachineMachine-to-Machine

Person-to-PersonPerson-to-Person



Integration of Machine-to-Machine using UTCIntegration of Machine-to-Machine using UTC




Rings

First token ring was introduced (e.g., IBM, FDDI) Why token rings?

Can support: 1) Bursty data (asynchronous) with

no rate, (no) loss, (low) latency, fairness, multicast 2) Periodic real-time

with rate and delay guarantees, multicast




Rings with Spatial Bandwidth Reuse

Packet are removed at destinations: slotted or insertion ring

Concurrent transmission

Throughput grows with locality all nodes can transmit

simultaneously to their neighbors

12

3

4

5

67

8

9

10

1112


SAT (token) gives predefined transmission quota Rotates in the opposite direction Held intermittently if the node is not SATisfiedHeld intermittently if the node is not SATisfied

Node 1

IB Node 6

IBNode 3

IB

Node 2

IB

SAT IB - Insertion BufferIB - Insertion Buffer

MetaRing: Fairness with Spatial Bandwidth Reuse

Slotted or insertion

ring

Node 5Node 4


MetaRing: SAT Fairness Properties

Equal throughput after each SAT rotation - with multiple variants Multiple SATs operations for simple fault recovery SAT/SAT’ for graceful degradation to (multi) bus operation

SAT signal provides for: Bounded delay with no loss of bursty data - Integration of real-time traffic with known rate -

MetaRing is the underlying network for IBM storage area network (SAN) products (also ANSI SSA - X3T10 standard)

Spatial reuse rings are currently very active: Cisco SRP/DPT and IEEE 802.17

Multi-billion business for IBM




Switched Network

Is MetaRing panacea? NO


Traffic with No Rateover Switched Network

To transfer with max. throughput (burst) AND min. delay and loss

tt

Traffic shape Traffic shape at the sourceat the source

tt

Traffic shape Traffic shape at the Destinationat the Destination

tt

Transfer withTransfer withMinimum DistortionsMinimum Distortions



TCP/IP: unstable/unpredictable throughput/delay/loss Cannot be done with over fixed routes (congestion and loss)

Dynamic routing: e.g., “Hot-Potato” (P. Baran),

Manhattan Street Network - deflection routing (N. Maxemchuk)


MetaNet Convergence Routing with Sense of Direction

Invented by Yoram Ofek and Moti Yung Virtual ring embedding

Link types: Ring - part of virtual ring/s Thread - all other links

Embeddings methods - e.g.,: Simple Hamiltonian Circuit Euler tree traversal

VN1

VN3

VN2

VN0

VN9VN7

VN14

VN15

VN6

VN8

Sequential Numberingof Virtual Nodes:

VN0, VN1, VN2, …

AAGG

FF

CC

BB

DD

EE HH

II

Multiple partial virtual rings


Packet routing paradigm: 1. Packets are forwarded to idle output link

“closer” to their destinations with: “sense of direction” - along virtual ring(s)

2. Virtual (buffer insertion) ring traffic gets priority to continue on the virtual ring links

MetaNet Convergence Routing over Switched Network


VN1

VN3

VN2

VN0

VN9VN7

VN14

VN15

VN6

VN8

SHORT-CUT Routing: Example: packet arrives

to VN6 on node C with destination H, can short-cut to VN8

Diametric routing in light load

AAGG

FF

CC

BB

DD

EE HH

II

Short-cut



VN3

VN1

VN2

VN0

VN9VN7

VN14

VN15

Broadcast-with-feedback: Requirements:

asynchronous - without arbitration losslessness correctness

complete coverage packet copied only once complete feedback to the source

When short-cuts or jumps are possible the packets are DUPLICATED

AAGG

FF

CC

BB

DD

EE HH

II



Summary: Support traffic from bursty sources with no rates:

No packet loss Bounded delay Fairness Broadcast and multicast (with feedback)

However, still limitations:1) on size - it is not a global network!2) does not support person-to-person communications with known rates





Outline How to support the two generic traffic types:

1. Ring networks

2. Convergence routing

3. Time-driven - switched networks

4. Dynamic optical networking









Time-Driven Priority over Switched Network

How to support communications with known rate on a global network?

Flow (congestion) control methods: Rate control at the network’s boundaries - e.g., ATM (J. Turner)

with statistical multiplexing inside the network Inside the network with local clocks scheduling -

deadline scheduling (D. Ferari), GPS (A. Parekh, R. Gallager) Inside the network with scheduling based on global time:

UTC - Coordinated Universal Time:

TIME-DRIVEN PRIORITYTIME-DRIVEN PRIORITYBased on pipeline forwardingBased on pipeline forwarding



Pipeline: optimal method - independent of a specific realization -successfully deployed with optimal efficiency in Factory (automotive), Computers (CPU)NOW pipeline in global networks! Thanks to GPS that provides UTCNOW pipeline in global networks! Thanks to GPS that provides UTC

Pipeline: From Henry Ford to the Internet

•Time-of-day or UTC – coordinated universal time - with accuracy of 5 s

1 2 1000

TimeCycle0

1 2 1000

TimeCycle1

1 2 1000

TimeCycle 79

Super-cycle - UTC secondwith 80k Time-frames

Time-of-Day or UTC 0beginning of a UTC second

1beginning of a UTC second

fTfTfTfT fT

Time Driven Priority


Time-driven Priority - Forwarding

ti i48 481 2 1 2

Time CycleTime Cycle

ti i48 481 2 1 2

Time CycleTime Cycle

Time CycleTime Cycle Time CycleTime Cycle

i+1

i+2

ArbitraryImmediate 2-frame

Arrive toArrive toOutputOutput

PortPort

Forward fromForward fromOutputOutput

PortPort

1. Immediate forwarding2. 2-frame forwarding3. Arbitrary forwarding

Schemeh - # of hops

BlockingProbability

Small p(q=1-p)

Immediate (1-q ) h p + O(p )

2-frame hp + O(p )

Arbitrary-frame 1-(1-p ) hp + O(p )

h k

k h

k k k+1

k+1

2k

k

k


Time-driven Priorityfor Videoconferencing

Time driven priority

videoconferencing with complex periodicity scheduling Face-to-face quality Scale the globe

Time driven priority

videoconferencing with complex periodicity scheduling Face-to-face quality Scale the globe

Video-conferencing

Sender-receiver synchronization

Node C

Receiver

Node B

Sender

UTC from GPSVideo FrameCapture

Video FrameDisplayt real

MPEGMPEGI pictureI picture



MPEGMPEGP picturesP pictures



Complex Periodicity Scheduling: The size of successive packets of the same flow changes in a repetitive manner


IP and “Best Effort” service are unchangedIP and “Best Effort” service are unchanged Time-driven priority scales the globe:Time-driven priority scales the globe:

Jitter: bounded by 2*TJitter: bounded by 2*Tf f :: Independent of the network size, traffic load, flow rateIndependent of the network size, traffic load, flow rate

End-to-end delay: 2* h*Tf + prop. DelayEnd-to-end delay: 2* h*Tf + prop. Delay No lossNo loss

Can easily integrated with:Can easily integrated with:MetaNet convergence routingMetaNet convergence routing

Optimized for interactive streaming mediaOptimized for interactive streaming media

Time-driven Priority - Summary




Objective: to utilize UTC in the optical domainObjective: to utilize UTC in the optical domain

In static opticalstatic optical networking all data units on the optical channel are switched in the same way

while,

In dynamic opticaldynamic optical networking each data unit on the optical channel may be switched differently

Fractional Switchingfor Dynamic Optical Networking


Problems of Static Switching: N 2 ’s

5 5 ss5 5 ss

NYC

LA

SF

SEA

STL


What: Fractional SwitchingSave Save s & Grooming & Small or No Memorys & Grooming & Small or No Memory

1 1 5 Fractional 5 Fractional Pipes (FPipes (FPs)Ps)1 1 5 Fractional 5 Fractional Pipes (FPipes (FPs)Ps)

NYC

LA

SF

SEA

STL

Number of s = Gb/s 10

neededcapacity Aggregate


IP

Time-based Grooming and Degrooming

IP/MPLS

ADSL DSLAM(central office)

Cable Modem Head-end

Server Farm(web, VoD)

Wireless Base Station

Smallfractions

Smallfractions

Switching

Largefractions

Edge/Access Router (POP)

Header processing only at the edgesHeader processing only at the edges


Pipeline forwarding of whole time frames No header processing Banyan based switch structure - optimal

Dynamic: Fractional Switching

See pipeline forwarding - PFanimation over FlPs

1 2 1000

TimeCycle0

1 2 1000

TimeCycle1

1 2 1000

TimeCycle 79

Super-cycle - UTC secondwith 80k Time-frames

Time-of-Day or UTC 0beginning of a UTC second

1beginning of a UTC second

fTfTfTfT fT


Why: Dynamic: Fractional SwitchingThe Optical Links are Memory

UTC A mesh of linear delay lines How to preserve pipeline forwarding? Delay between switches =

integer number of time frame


Input 1 Optical Alignment

Optical Switching

Fabric

Optical Alignment

Input N

Output 1

Output N

t+1

Time-of-Day or UTC

t-1 t-2 t-3tt+2

Idle time: Safety marginbetween two time frames

Idle time:Safety marginbetween two time frames

Switch ControllerTime-of-Day

or UTC

: Time frame payload – with a predefined number of data units

fT fT fT fT

fT : Time frame

The Optical Links are the Memory


Multiple time frames low blocking probabilityDWDM many parallel routes low blocking probability

Multistage Crossbar Switching elements a*N*lgaN N2

For N=256, a=4 4K 64KFor N=1024, a=4 20K 1,000K

(factor of 16)(factor of 50)

Low Complexity Switching FabricOptimal Speedup of 1

ScalabilityScalability BlockingBlocking


Blocking ProbabilityFour Channels per Link

0%

10%

20%

30%

40%

50%

60%

70%

80%

50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%

Average Utilization [%]

Blo

ck

ing

Pro

ba

bili

ty [

%]

1 TF 4 TFs 16 TFs 32 TFs 64 TFs 1000 TFs


Scheduling and Switching with UTC Alignment

TF1

TF2

TF3

TF4TF5

TF6

TF

7

TF8TF1

TF2

TF3

TF4TF5

TF6

TF7

TF8

PeriodicSchedule

on Switch i

PeriodicSchedule

on Switch j

Always aligned with a bounded error (typically < 1 second)Thus, delay (memory) per switch = 1 TF

Schedule s


Scheduling and Switching without UTC Alignment Circuit Switching, e.g., SONET

TF1

TF2

TF3TF4

TF5

TF6

TF7

TF8TF1TF2

TF3

TF4

TF5TF6

TF7

TF8

No alignment Thus, delay (memory) per switch = 1 Time Cycle

PeriodicSchedule

on Switch i

PeriodicSchedule

on Switch j

Schedule s


Service Interfaces

Network Processor (MPLS)

FP 2

FP 1

FP iPort

SONET DMUX (STS-1)

FP 2

FP 1

FP iPort

IP/MPLS

SONET

OC

-12/

OC

-48

OC

-12/

OC

-48

MPLS Packets

SONET STS-1 frames See Animation

UTC

UTC


Conclusion

SingleSingleNetworkNetwork

Machine-to-MachineMachine-to-MachineTypically No RateTypically No Rate

Person-to-PersonPerson-to-PersonTypically with RateTypically with Rate

Fractional SwitchingTime-driven Priority

MetaNet ConvergenceRouting

UTC can be used as the “GLUE” for combining:

Person-to-Person, Machine-to-Machine, TCP/IP “best effort” …

Documents

May 2002© 2002 Yoram Ofek1 Satisfying the Requirements of Applications on a Single Packet Network Yoram Ofek Synchrodyne Networks, Inc. E-mail: [email protected]