Lecture Note on Switch Architectures

Function of Switch

Input 1

Input N

Output 1

Output N

Naive Way

Input 1

Input N

Output 1

Output N

Bus-Based Switch

Input PortProcessor

Output PortProcessor

Input PortProcessor

Output PortProcessor

Input PortProcessor

Output PortProcessor

Controller

• No buffering at input port processor (IPP)

• Output port processor (OPP) buffers cells

• Controller exchanges control message with terminals and other controller.

• Disadvantage:– Bus bandwidth is equal to sum of

external link for non-blocking– IPPs and OPPs must operate at

full bus bandwidth– Bus width increases with number

of links

Centralized Bus Arbitration

• IPPs send requests to central arbiter• Request may includes:

– Priority– Waiting time– OPP destination(s)– Length of IPP queue

• Arbitration complexity is O(N^2)• Distributed version is preferred, but may degrade throughput.

Bus Arbitration Using Rotating Daisy Chain

• Rotating token eliminates positional favoritism

0

1

K

N

Token

Ring Switch

IPP RI

IPP RI OPPRI

OPPRI

• Same bandwidth and complexity as bus switch

• Avoids capacitive loading of bus, allowing higher clock frequencies

• Control mechanisms– Token passing– Slotted ring with busy bit

Shared Buffer Switch

Input PortProcessor

Output PortProcessor

Input PortProcessor

Output PortProcessor

Input PortProcessor

Output PortProcessor

Controller

Shared Memory

• Individual queues are rarely full.

• Shared memory needs two times of external link bandwidth

• Require less memory • Better ability to handle

burst traffic

Crossbar Switch

Input PortProcessor

Output PortProcessor

Input PortProcessor

Output PortProcessor

Input PortProcessor

Output PortProcessor

Controller

Output Buffering

Input Port Output Port

• Efficient, but needs N time speed up internally.

Input Buffering

Input Port

Output Port

• Multiple packets simultaneously transmitted distinct outputs.• Require sophisticated arbitration • No speed up required • Head-of-line blocking

Bi-partite Matching

• Require global information • Complexity is O(N^(5/2))• Not suitable for hardware implementation• May leads to starvation

0

1

2

3

0

1

2

3

0

1

2

3

0

1

2

3Stravati on for

traffi c(0 to 0 and 1 to0)

Desired Arbitration Algorithms

• High throughput– Low backlog in each input queue – Close to 100% for each input and output

• Starvation free– No queue will be hold indefinitely

• Simple to implement

Options to Build High Performance Switches

• Bufferless crossbar• Buffered crossbar• Shared buffer

Bufferless Crossbar

• Centralized arbitrator is required – Arbitration complexity is O(N*N) – O(log2N) iterations of arbitration needed for high throughput

• Synchronization in all elements• Single point failure: central arbitrator• Complex line interface

Buffered Crossbar

• Simple scheduling algorithms– Ingress: O(1)– Egress: O(N)

• Inefficient use of memory– Memory linearly increased with number of ports

Shared Memory

• No central arbitrator needed• Reduced memory requirements• Distributed flow control • Less timing constrains • Simpler line card interface

Comparisons (1)

Line Card Buffering

Arbitor Complexity

Number of iterations

Available Arbitration Time Slots

Bufferless Crossbar Yes O(NXN) O(log N) 1/O(logN)Buffered Crossbar Yes O(N) 1 1Shared Memory No O(N) 1 1

Comparison (2)

10 G ports 8 16 32 64Bufferless Crossbar 17.1 12.8 10.2 8.5Buffered Crossbar 51.2 51.2 51.2 51.2Shared Memory 51.2 51.2 51.2 51.2

Time for arbitration (nano seconds)

Assume 10G for each port and packet size is 64 bytes.

Scaling Number of Ports

• Single larger switch is less expensive, more reliable, easier to maintain and offer better performance, but– O(n2) complexity – Board-level buses limited by capacitive loading

• Port multiplexing• Buffered multistage routing

– Dynamic routing: Benes network– Static routing: Clos network

• Bufferless multistage routing– Deflection routing

Port Multiplexing

IPP

IPP

OPP

OPP

• High speed core can handles high speed links as well as low speed• Sharing of common circuitry• Reduced complexity in interconnection network• Better queueing performance for bursty traffic• Less fragmentation of bandwidth and memory

Dynamic Routing – Benes Network

1000

1001

0000

0001

0010

0011

0100

0101

0110

0111

1000

1001

1010

1011

1100

1101

1101

1111

distributionRout and copy

• Network expanded by adding stages on left and right– 2k-1 stages with d port switch elements supports dk ports

• Traffic distribution on first k-1 stages• Routing on last k stages• Internal load external load• Traffic maybe out of order: need re-sequencing

Static Routing - Clos network

d r• All traffic follows same path• r 2d-1 to be strict non-blocking.

Deflection Routing

3

4

1

2

3

4

1

2

4

1

3

22

3

3

2

Basic Architectural Components:Forwarding Decision

ForwardingDecision

ForwardingDecision

ForwardingDecision

ForwardingTable

ForwardingTable

Interconnect

OutputScheduling

1.

2.

3.

ForwardingTable

ATM SwitchesDirect Lookup

VCI

Address

MemoryD

at a(Port, VCI)

Hashing Function

CRC-1616

Linked lists

#1 #2 #3 #4

#1 #2

#1 #2 #3

Memory

Search Data

48

log2N

AssociatedData

Hit?Address{

Ethernet SwitchesHashing

Advantages• Simple• Expected lookup time can be smallDisadvantages• Non-deterministic lookup time• Inefficient use of memory

Per-Packet Processing in IP Routers

1. Accept packet arriving on an incoming link.2. Lookup packet destination address in the forwarding table,

to identify outgoing port(s).3. Manipulate packet header: e.g., update header checksum.4. Send packet to the outgoing port(s).5. Classify and buffer packet in the queue.6. Transmit packet onto outgoing link.

IP Router Lookup

Destination Address

Next Hop Link

--------

---- ----

--------

Destination Next HopForwarding Table

Next Hop Computation

Forwarding Engine

Incoming Packet

HEADER

payload

Lookup and Forwarding Engine

header

Packet

Router

Destination Address

Outgoing Port

Forwarding Engine

Lookup Data

Example Forwarding Table

Destination IP Prefix Outgoing Port

65.0.0.0/ 8 3

128.9.0.0/16 1

142.12.0.0/19 7

IP prefix: 1-32 bits

Prefix length

Multiple Matching

128.9.16.0/21 128.9.172.0/21

128.9.176.0/24

Routing lookup: Find the longest matching prefix (or the most specific route) among all prefixes that match the destination address.

0 232-1

128.9.0.0/16142.12.0.0/1965.0.0.0/8

128.9.16.14

Longest matching prefix

Longest Prefix Matching Problem

2-dimensional search • Prefix Length• Prefix Value

Performance Metrics• Lookup time• Storage space• Update time• Preprocessing time

Required Lookup Rates

12540.0OC768c2002-05

31.2510.0OC192c2000-01

7.812.5OC48c1999-00

1.940.622OC12c1998-99

40B packets (Mpps)

Line-rate (Gbps)

LineYear

DRAM: 50-80 ns, SRAM: 5-10 ns

31.25 Mpps 33 ns

0100002000030000400005000060000700008000090000

100000

Size of Forward Table

95 96 97 98 99 00Year

Num

ber o

f Pre

fixes

10,000/year

Renewed growth due to multi-homing of enterprise networks

Trees and Tries

Binary Search Tree

< >

< > < >

log2 N

N entries

Binary Search Trie

0 1

0 1 0 1

111010

Typical Profile of Forward Table

Prefix Length

Num

ber

Most prefixes are 24-bits or shorter

Basic Architectural Components: Interconnect

ForwardingDecision

ForwardingDecision

ForwardingDecision

ForwardingTable

ForwardingTable

Interconnect

OutputScheduling

1.

2.

3.

ForwardingTable

First-Generation Routers

CPU BufferMemory

LineInterface

DMA

MAC

LineInterface

DMA

MAC

LineInterface

DMA

MAC

Second-Generation Routers

CPU BufferMemory

LineCard

DMA

MAC

LocalLocalBufferBuffer

MemoryMemory

LineCard

DMA

MAC

LocalLocalBufferBuffer

MemoryMemory

LineCard

DMA

MAC

LocalLocalBufferBuffer

MemoryMemory

Third-Generation Routers

LineCard

MAC

LocalBufferMemory

CPUCard

LineCard

MAC

LocalBufferMemory

Switching Goals

Circuit Switches

• A switch that can handle N calls has N logical inputs and N logical outputs– N up to 200,000

• Moves 8-bit samples from an input to an output port– Samples have no headers– Destination of sample depends on time at which it arrives at the switch

• In practice, input trunks are multiplexed– Multiplexed trunks carry frames, i.e., set of samples

• Extract samples from frame, and depending on position in frame, switch to output– each incoming sample has to get to the right output line and the right

slot in the output frame

Blocking in Circuit Switches

• Can’t find a path from input to output• Internal blocking

– slot in output frame exists, but no path• Output blocking

– no slot in output frame is available

Time Division Switching

• Time division switching interchanges sample position within a frame: time slot interchange (TSI)

Scaling Issues with TSI

Space Division Switching

• Each sample takes a different path through the switch, depending on its destination

Time Space Switching

Time Space Time Switching

Packet Switches

• In a circuit switch, path of a sample is determined at time of connection establishment. No need for header.

• In a packet switch, packets carry a destination field or label. Need to look up destination port on-the-fly.– Datagram switches: lookup based on destination address – Label switches: lookup based on labels

Blocking in Packet Switches

• Can have both internal and output blocking• Internal

– no path to output

• Output– trunk unavailable

• Unlike a circuit switch, cannot predict if packets will block.• If packet is blocked, must buffer or drop

Dealing with Blocking in Packet Switches

• Over-provisioning– internal links much faster than inputs

• Buffers– at input or output

• Backpressure– if switch fabric doesn’t have buffers, prevent packet from entering until

path is available

• Parallel switch fabrics– increases effective switching capacity

Basic Architectural Components: Queuing, Classification

ForwardingDecision

ForwardingDecision

ForwardingDecision

ForwardingTable

ForwardingTable

Interconnect

OutputScheduling

1.

2.

3.

ForwardingTable

Techniques in Queuing

Input Queueing Output Queueing

Output Queueing

Individual Output Queues Centralized Shared Memory

Memory b/w = (N+1).R

1

2

N

Memory b/w = 2N.R

1

2

N

Input Queuing

Input Queueing Performance

Del

ay

Load 58.6% 100%

Virtual Output Queues at Input

Del

ay

Load 100%

Classification

Action

--------

---- ----

--------

Predicate ActionClassifier (Policy Database)

Packet Classification

Forwarding Engine

Incoming Packet

HEADER

Multi-Field Packet Classification

Given a classifier with N rules, find the action associated with the highest priority rule matching an incoming packet.

Field 1 Field 2 … Field k Action

Rule 1 152.163.190.69/ 21 152.163.80.11/ 32 … UDP A1

Rule 2 152.168.3.0/ 24 152.163.0.0/ 16 … TCP A2

… … … … … …

Rule N 152.168.0.0/ 16 152.0.0.0/ 8 … ANY An