OutlineOutline
A brief Historical aside
Review of Transmission (Transport) Technologies, Architectures and Evolution
Transporting Broadband across Transmission Transporting Broadband across Transmission Networks designed for NarrowbandNetworks designed for Narrowband
Current Issues:Current Issues:Broadband IP Transport AnalysisBroadband IP Transport Analysis
Ongoing Investigations in IP/OTN NetworksOngoing Investigations in IP/OTN Networks
A Brief Historical AsideA Brief Historical Aside
LDBell-Labs WEBCS ME
AT&T 1984 - 1997
LD AT&T LabsAT&T circa 1997
Bell-Labs WEBCS MELucent circa 1997
Pre 1984 AT&T
BOCs LDBell-Labs WEBCS ME
AgereAvaya
RBOCs circa 1984US WestAmeritechSouthWest BellBell SouthNynexBell-AtlanticPac Bell
Bellcore
Qwest
Telcordia
Tellium
SBC
Verizon
Bell South
AT&T Lucent
The Bell System Legacy Today
Review of Transmission(Transport) Technologies,
Architectures and Evolution
Review of Transmission(Transport) Technologies,
Architectures and Evolution
Opening Trivia QuestionOpening Trivia Question
What is the difference between a DS3 (or DS1) and a T3 (or T1)?
Asynchronous Data Rates
Digital Signal Level 0 DS0 64 Kb/sinternal to equipment
Digital Signal Level 1 DS1 1.544 Mb/sintra office only (600 ft limit)
Digital Signal Level 3 DS3 45 Mb/s intra office only (600 ft limit)
T1 Electrical (Copper) Version of DS1 1.544 Mb/srepeatered version of DS1 sent out of Central Office
T3 Electrical (Copper) Version of DS3 45 Mb/srepeatered version of DS3 sent out of Central Office
Asynchronous Digital Hierarchy
DS1 DS3
Asynchronous Optical Line SignalN x DS3s
28 DS1s = 1 DS324 DS0s = 1 DS1
DS0 (a digitized analog POTS circuit @ 64 Kbits/s)
Asynchronous Lightwave Systems typically transport traffic in multiples of DS3s i.e.... 1, 3, 12, 24, 36, 72 DS3s
DS0
Asynchronous NetworkingManual DS1 Grooming/Add/Drop
LW M13
DSX3
DS1
M13
DSX1
DSX1
DSX3
LW
• Manually Hardwired Central Office• No Automation of Operations• Labor Intensive• High Operations Cost• Longer Time To Service
DS3 DS3
Some Review QuestionsSome Review Questions
What does the acronym SONET mean?
What differentiates SONET from Asynchronous technology?
What does the acronym SDH mean?
The Original Goals of SONET/SDH Standardization
Vendor Independence & Interoperability
Elimination of All Manual Operations Activities
Reduction of Cost of Operations
Protection from Cable Cuts and Node Failures
Faster, More Reliable, Less Expensive Service to the Customer
SONET RatesDS3s are STS-1 Mapped
DS3
STS-1
51.84
Mbits
/s
SONET Optical Line SignalOC-N = N x STS-1s
N is the number of STS-1s (or DS3s) transported
28 DS1s = 1 DS3 = 1 STS-124 DS0s = 1 DS1
(= 1 VT1.5)
DS1
DS0 (a digitized analog POTS circuit @ 64 Kbits/s)
DS0
OC level STM level Line rate (MB/s) OC-1 - 51.84 OC-3 STM-1 155.52 OC-12 STM-4 622.08 OC-48 STM-16 2488.32 OC-192 STM-64 9953.28
SONET and SDH
STESTELTE
LTE
PTEPTE
PTEPTE
PTEPTE
STESTELT
ELT
E
PTEPTE
PTEPTE
PTEPTEDS-3DS-3
DS-3DS-3
DS-3DS-3
DS-3DS-3
DS-3DS-3
DS-3DS-3
OC-3 TMOC-3 TMOC-3 TMOC-3 TM
SONET Line
SONET Path
SONET Section
TM = Terminal MultiplexorDS = Digital Signal
PTE = Path Terminating ElementLTE = Line Terminating ElementSTE = Section Terminating Element
SONET Layering for Cost Effective OperationsSONET Layering for Cost Effective Operations
SONET Point-to-Point Network
Repeater Repeater
TM TM
Section
Line
Path
STS-1FrameFormat
LineOverhead
SectionOverhead Path
Overhead
STS-1 Synchronous Payload Envelope
STS-1 SPE
SONET Ring Network ArchitecturesSONET Ring Network Architectures
Unidirectional Path Switched Ring
A-BA-B
A-BA-B
B-AB-A
B-AB-A
Path Path SelectionSelection
Path SelectionPath Selection
WW
PP
fiber 1fiber 1
fiber 2fiber 2
AA
BB
CC
DD
Failure-free StateFailure-free StateBridgeBridge
BridgeBridge
Bidirectional Line Switched Ring
AACC
C C AAAACC
C C AA
WorkingWorking ProtectionProtection
2-Fiber BLSRB
A
D
C
Some Review QuestionsSome Review Questions
Which SONET Ring Network is simpler?
Which SONET Ring Network is inefficient for distributed demand sets?
Typical Deployment of UPSR and BLSR in RBOC Network
Regional Ring (BLSR)
Intra-Regional Ring (BLSR) Intra-Regional Ring (BLSR)
Access Rings (UPSR)
WB DACs
BB DACs
WB DACS = Wideband DACS - DS1 GroomingBB DACS = Broadband DACS - DS3/STS-1 GroomingOptical Cross Connect = OXC = STS-48 Grooming
DACS=DCS=DXC
Emergence of DWDMEmergence of DWDM
Some Review QuestionsWhat does the acronym DWDM mean?
What was the fundamental technology that enabled the DWDM network deployments?
First Driver for DWDMLong Distance Networks
WD
M N
EW
DM
NE W
DM
NE
WD
M N
E
• Limited Rights of Way• Multiple BLSR Rings Homing to a few Rights of Way• Fiber Exhaustion
BLSR Fiber PairsBLSR Fiber Pairs
Key Development for DWDM Optical Fiber Amplifier
120 km
OC-48
OLSTERM
OLSRPTR
OLSRPTR
OLSTERM
120 km 120 km
Fiber Amplifier Based Optical Transport - 20 Gb/s
OC-48OC-48
OC-48
OC-48OC-48
OC-48OC-48
Conventional Optical Transport - 20 Gb/s
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM
40km 40km 40km 40km 40km 40km 40km 40km 40km
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM1310
RPTR1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM1310
RPTR1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM1310
RPTR1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM1310
RPTR1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM1310
RPTR1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM1310
RPTR1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
1310RPTR
TERMTERM
OC-48OC-48
OC-48OC-48
OC-48OC-48
OC-48OC-48
Increased Fiber Network Capacity
Transporting BroadbandTransporting Broadbandacross Transmission Networksacross Transmission Networks
designed for Narrowbanddesigned for Narrowband
Transporting BroadbandTransporting Broadbandacross Transmission Networksacross Transmission Networks
designed for Narrowbanddesigned for Narrowband
T1/T3/OC3FRS and CRS
ATM
Access
ATM
Access
ATM
Switch
Public/PrivateInternet Peering
ATM
Access
ATM
Access
Access
Router
T1/T3 IPLeased-LineConnections
Core
Router
Core
Router
Access
Router
Access
Router
ATM Access
ATM Access
RAS
RAS
RAS
RAS
RAS
RAS
RAS
RAS
Access
Router
Access
Router
EtherSwitch
EtherSwitch
RAS
RAS
RAS
RAS
RAS
RAS
RAS
RAS
Core
Router
Core
Router
BackboneSONET/WDM
RAS Farms
T1/T3 FRand ATM IPLeased-LineConnections
ATM Switch
ATMSwitch
ATMSwitch
ATMSwitch
Core
Router
Core
Router
Data SP
High Capacity Path NetworkingHigh Capacity Path Networking
Existing SONET/SDH networks are a Existing SONET/SDH networks are a BOTTLENECKBOTTLENECK for Broadband Transport for Broadband Transport
Most Access Rings are OC-3 and OC-12 UPSRs while most Backbone Rings are OC-48. Transport of rates higher than OC-48 using the existing SONET/SDH network will require significant and Most Access Rings are OC-3 and OC-12 UPSRs while most Backbone Rings are OC-48. Transport of rates higher than OC-48 using the existing SONET/SDH network will require significant and costly changes. Clearly upgrading the SONET/SDH network everytime broadband data interfaces are upgraded based increased IP traffic is not an appropriate solution.costly changes. Clearly upgrading the SONET/SDH network everytime broadband data interfaces are upgraded based increased IP traffic is not an appropriate solution.
Existing SONET/SDH networks are a Existing SONET/SDH networks are a BOTTLENECKBOTTLENECK for Broadband Transport for Broadband Transport
Most Access Rings are OC-3 and OC-12 UPSRs while most Backbone Rings are OC-48. Transport of rates higher than OC-48 using the existing SONET/SDH network will require significant and Most Access Rings are OC-3 and OC-12 UPSRs while most Backbone Rings are OC-48. Transport of rates higher than OC-48 using the existing SONET/SDH network will require significant and costly changes. Clearly upgrading the SONET/SDH network everytime broadband data interfaces are upgraded based increased IP traffic is not an appropriate solution.costly changes. Clearly upgrading the SONET/SDH network everytime broadband data interfaces are upgraded based increased IP traffic is not an appropriate solution.
Existing SDH-SONET Network
IP router
IP router IP router
STS-3c
STS-12c/48c/...
IP/SONET/WDM Network ArchitectureIP/SONET/WDM Network Architecture
Core IPNode
EMS
.
.
.
SONETADM/LT
OC-3/12[STS-3c/12c]
OC-12/48
OC-3/12[STS-3c/12c/48c]
SONET Transport Network
SONETNMS
Core IPNode
EMS
.
.
.
Access Routers/EnterpriseServers
OC-48
SONETADM/LT
SONETXC
WDMLT
WDMLT1, 2, ...
OC-3/12/48[STS-3c/12c/48c]
Pt-to-Pt WDM Transport Network
OC-3/12/48[STS-3c/12c/48c]
OTNNMS
IP = Internet ProtocolIP = Internet ProtocolOTN = Optical Transport NetworkOTN = Optical Transport NetworkADM = Add Drop MultiplexorADM = Add Drop MultiplexorWDM = Wavelength Division MultiplexingWDM = Wavelength Division Multiplexing
LT = Line TerminalLT = Line TerminalEMS = Element Management SystemEMS = Element Management SystemNMS = Network Management SystemNMS = Network Management System
Optical Network Evolution mirrorsSONET Network EvolutionOptical Network Evolution mirrorsSONET Network Evolution
Multipoint NetworkWDM Add/Drop
Point-to-Point WDM Line System
Optical Cross-ConnectWDM Networking
OXC
i
WDMADM
WDMADM
k
IP/OTN ArchitectureIP/OTN Architecture
Core DataNode
EMS
.
.
.
OXC
mc: multi-channel interface(e.g., multi-channel OC-12/OC-48)
mc
mcOptical Transport Network
OTNNMS
Core Data Node
EMS
.
.
.
Access RoutersEnterprise Servers
OXC
OXC
Core Data Node
EMS
.
.
.
mc
mc
IP = Internet ProtocolIP = Internet ProtocolOTN = Optical Transport NetworkOTN = Optical Transport NetworkOXC = Optical Cross ConnectOXC = Optical Cross ConnectWDM = Wavelength Division MultiplexingWDM = Wavelength Division Multiplexing
EMS = Element Management SystemEMS = Element Management SystemNMS = Network Management SystemNMS = Network Management System
Broadband IP Transport Analysis
Credits to Debanjan Saha and Subir Biswas
Broadband IP Transport Analysis
Credits to Debanjan Saha and Subir Biswas
Architectural AlternativesArchitectural Alternatives
IP-over-DWDM: IP routers connected directly over
DWDM transport systems.
IP-over-OTN: IP routers interconnected over a
reconfigurable optical transport network (OTN)
consisting of optical cross-connects (OXCs) connected
via DWDM.
Architectural AlternativesArchitectural Alternatives
Quadruple Redundant Configuration of IP Routers at PoPsQuadruple Redundant Configuration of IP Routers at PoPs
Currently deployed by carriers to increase router reliability and perform load balancing.
Upper two routers are service routers adding/dropping traffic from the network side and passing through transit traffic.
Lower two routers are drop routers connected to client devices.
Two connections from the network port at the ingress upper (service) router to two drop ports, one in each of the lower (drop) routers. Client device sends 50% of the traffic on one of these drop interfaces and 50% on the other (it is attached to both of the drop
routers).
Not required for OXCs.
IP-over-DWDM: Pros and ConsIP-over-DWDM: Pros and Cons
IP-routers with OC-48c/OC-192c interfaces and aggregate throughput reaching 100s of Gbps.
Transport functions like switching, configuration, and restoration are moved to the IP layer and accomplished by protocols like MPLS, thus providing a unifying framework.
IP routers control end-to-end path selection using traffic engineering extended routing
and signaling IP protocols.
Supports the peer-to-peer model where IP routers interact as peers to exchange routing information.
Can router technology scale to port counts consistent with multi-terabit capacities without compromising performance, reliability, restoration speed, and software stability ? A big question mark.
ConsPros
IP-over-OTN: Pros and ConsIP-over-OTN: Pros and Cons
Reconfigurable optical backbone provides a flexible transport infrastructure
Core OXC network can be shared with other service networks such as ATM, Frame Relay, and SONET/SDH private line services.
Allows interconnection of IP routers in an arbitrary (logical) mesh topology.
Not possible in architecture A since a typical CO/PoP has two, in some cases three, and in rare occasions four conduits connecting it to neighboring PoPs.
Adding a reconfigurable optical backbone introduces an additional layer between the IP and DWDM layers and associated overhead.
Traffic engineering occurs independently in two domains -- (i) the IP router network with its logical adjacencies spanning the OXC backbone, and (ii) the optical network which provisions physical lightpaths between edge IP routers. Could lead to inefficiency in traffic routing from a global perspective.
ConsPros
Why Glass Through is not an Alternative?Why Glass Through is not an Alternative?
Removes the flexibility of dynamic switching between incoming and outgoing fibers at a PoP that comes with using a router or an OXC.
Prevents organic growth of the network. Dynamic switching allows local capacity to be used to meet traffic demands between arbitrary PoPs. With glass through, bandwidth is not available at the link level but only at the segment level whose two end PoPs terminate glass through fiber paths.
Does not allow intelligent packet processing or performance monitoring of transit traffic at a PoP.
Network Deployment Cost AnalysisNetwork Deployment Cost Analysis
Analysis of the two architectures from an economic standpoint.
Contrary to common wisdom, a reconfigurable optical layer can lead to substantial reduction in capital expenditure for networks of even moderate size.
Critical observation: Amount of transit traffic at a PoP is much higher than the amount of add-drop traffic.
Hence, a reconfigurable optical layer that uses OXC ports (instead of router ports) to route transit traffic will drive total network cost down so long as an OXC interface is marginally cheaper than a router interface.
Savings increases rapidly with the number of nodes in the network and traffic demand between nodes.
Assumptions: Network ModelAssumptions: Network Model
Typical CO/PoP has two, in some cases three, and in rare occasions four conduits connecting it to neighboring PoPs. Average degree = 2.5.
Routing uniform traffic (equal traffic demand between every pair of PoPs) on networks of increasing size.
Two traffic demand scenarios: uniform demand of 2.5 Gbps (OC-48) and 5 Gbps between every pair of PoPs.
Multiple routers/OXCs can be placed at each PoP to meet port requirements for routing traffic.
Core OXC network provides full grooming of OC-192 ports into OC-48 tributaries.
Transit traffic uses router ports in IP-
over-WDM and OXC ports (only) in
IP-over-OTN.
Quadruple redundant configuration of
IP routers at a PoP to improve
reliability and perform load-balancing.
Shortest-hop routing of lightpaths.
IP routers have upto 64 ports and
OXCs have upto 512 ports (in keeping
with port counts of currently shipped
products).
With or without traffic restoration
(diverse backup paths).
Assumptions:PricingAssumptions:Pricing
IP routers and OXCs have fixed costs and per-port costs for OC-48 and OC-192 interfaces.
Ballpark list prices for currently shipped products.
IP router:fixed cost of $200K and
per-port cost of $100K and $250K for OC-48 and OC-192 interfaces respectively.
OXC:fixed cost of $1M and
per-post cost of $25K and $100K for OC-48 and OC-192 interfaces respectively.
2.5 Gbps of Traffic between PoP Pairs Without Restoration2.5 Gbps of Traffic between PoP Pairs Without Restoration
Cross-over point at network size of about 18 nodes.
2.5 Gbps uniform traffic
0500
100015002000250030003500400045005000
0 10 20 30 40 50 60Network size (nodes)
To
tal
$-C
ost
(M
)
IP-over-WDM
IP-over-OTN
Cross-over point at network size of about 15 nodes.
5 Gbps uniform traffic
0
1000
2000
3000
4000
5000
6000
7000
8000
0 10 20 30 40 50 60Network size (nodes)
To
tal
$-C
ost
(M
)
IP-over-WDM
IP-over-OTN
5 Gbps of Traffic between PoP Pairs Without Restoration5 Gbps of Traffic between PoP Pairs Without Restoration
% of Transit Traffic in the Network Without Restoration% of Transit Traffic in the Network Without Restoration
% of Transit Traffic
0
20
40
60
80
100
0 10 20 30 40 50 60Network size (nodes)
% T
ran
sit
Tra
ffic
degree = 2
degree = 3
75-85% of the total traffic is transit traffic for a network size of 50 PoPs.
Cross-over point at network size of less than 8 nodes.
2.5 Gbps uniform traffic
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 10 20 30 40 50 60Network size (nodes)
To
tal
$-C
ost
(M
)
IP-over-WDM
IP-over-OTN
2.5 Gbps of Traffic between PoP Pairs With Restoration2.5 Gbps of Traffic between PoP Pairs With Restoration
Cross-over point at network size of less than 4 nodes.
5 Gbps uniform traffic
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
0 10 20 30 40 50 60Network size (nodes)
To
tal
$-C
ost
(M
)
IP-over-WDM
IP-over-OTN
5 Gbps of Traffic between PoP Pairs With Restoration5 Gbps of Traffic between PoP Pairs With Restoration
% of Transit Traffic in the Network With Restoration% of Transit Traffic in the Network With Restoration
80-95% of the total traffic is transit traffic for a network size of 50 PoPs.
% of Transit Traffic
0102030405060708090
100
0 10 20 30 40 50 60Network size (nodes)
% T
hro
ug
h T
raff
ic
degree = 2
degree = 3
Results and DiscussionResults and Discussion
Without restoration: Network cost breakeven point occurs at network sizes of 18 and 15 nodes for 2.5 Gbps and 5 Gbps of uniform traffic respectively.
With restoration: IP-over-OTN has lower cost beyond a network size of 4-6 nodes.
IP-over-OTN becomes increasingly attractive as amount of traffic and network size grows. Savings is much more when we consider traffic restoration.
Amount of transit traffic in the network grows rapidly as network size increases. For example, without restoration, 75-85% of the total traffic is transit traffic for a network size of 50 PoPs, and with restoration, it is 80-95%.
Carrying transit traffic over OXC ports (instead of router ports) drives network cost down so long as an OXC interface is marginally cheaper than a router interface.
Results and Discussion contd. ...Results and Discussion contd. ...
With traffic restoration, the economies of scale reaped from IP-over-OTN is further increased.
Each primary path in a network has a diversely routed backup path.
Transit port usage will increase substantially when we consider backup paths
while the number of terminating ports remains unchanged.
Case for Restoration at Optical LayerCase for Restoration at Optical Layer
Restoration in IP-over-WDM: Provided at the IP layer where backup paths consume router ports (like primary paths).
Restoration in IP-over-OTN: Can be provided at the optical or IP layers. In the former case, router ports are not consumed on intermediate PoPs.
Study shows substantial increase in savings for IP-over-OTN when restoration is taken into consideration.
IP-over-OTN has lower cost beyond a network size of 4-6 nodes.As much as 80-95% of the total traffic is transit traffic for a network size of 50 PoPs.
Ongoing Investigations in IP/OTN NetworksOngoing Investigations in IP/OTN Networks
Can IP layer provide reliable service?
How much Restoration is really required for services?
Interaction of Routing Protocols with Optical Layer Restoration
Optimal Routing with Topology of IP and Optical Layers
And many more...