Ivan Tam 2015

Fixed Network Infrastructure – An OverviewIVAN TAM

Applications, Broadband, Fixed Mobile Infrastructures

Web browsing, on-line service, peer to peer, video streaming, 4K, Cloud Storage,

Wired Access – Most cost effective technology over wire to connect home/enterprise over high speed

IP transport – Transport routing from service provider network to core/data center and Internet

Cloud, Data Center

Wavelength Division Multiplexing

Wireless Access – Most cost effective technology over limited spectrum to connect user wirelessly and maintaining mobility

Internet peering and transit - A number of carriers to the final network of the content

Caching, local content, Billing and charging

Tracking user, keeping record of service profile, authentication

Bandwidth Management

Agenda

Access Technologies Copper based xDSL, Fiber based PON technology

Optical Networking Technology

IP Networks IP Routing in carrier

Broadband Network Gateway (BNG) for Residential services

QoS

Core and Internet Gateway

Service Provider Architecture

InternetCoreService EdgeAggregationAccess

xDSL

Core RouterInternet Gateway

Transit Carrier Router

Internet Exchanges

• Access – provides cost/performance effective connectivity using copper, fiber to home and offices

• Aggregation –aggregate the traffic from various types of access terminating at CO/POP

• Service/edge –service management point where authentication, service quality, and service features are provided BNG ( Broadband Gateway)/VPN PE (Provider Edge)

• AAA – where database of customers are stored and checked by service edge

• Value added service (VAS) –IPTV server, internet cache

• Core – national connectivity between data centers and service edge to each other and to the internet

• Transit – service provider interface with a number of transit carriers via own border routers. Transit carrier is responsible for routing traffic back and forth to destinations in the worls

Data Center

Central Office

Transit

Web site’s service provider

Web site’s hosting data center/ Cloud

Physical Infrastructure

InternetCoreService EdgeAggregation/Metro IPAccess

GPON

xDSL

Metro-ethernet

Core Router

Internet Gateway

BroadbandGateway/VPN Provider Edge

Access Node

Central OfficeData Center

AAA/VAS

Transit Carrier Router

Internet ExchangesSource: ECMweb.com

• CO (Central Office) -distributed throughout a city terminating copper pairs from homes and offices within a few km

• PSTN switches (aka telephone switch) -telephony service, ADSL DSLAM -broadband service

• 10s of COs in a typical city• FTTx (fiber to cabinet,

building, home) => termination of copper pair moves out of CO and fiber from them enables CO consolidation

• CO are connected to Metro Data center using metro IP transport network

Access Technologies Leverage on existing infrastructure such as telephone twisted pair (CAT 3) or cable TV coaxial cable HFC (Hybrid Fiber Coaxial network) to deliver very high speed access Reduce or delay investment to lay new wiring (e.g., fiber)

Or simply deploy fiber all the way to home outright (FTTH) Gbps, and 10Gbps of bandwidth to homes

Shared for economics, and easy to maintain

PON (passive optical network)

Evolution of DSP enables more advanced techniques to be used Use more spectrum of the copper wire, xDSL (Digital Subscriber Loop)

Divide the spectrum into channels

Invariably due to attenuation and use of high frequencies in the spectrum Trade off of distance and bandwidth -> reduce distance of copper/coaxial

Termination equipment is closer to home, fiber need to extend further from network (FTTX)

xDSL Technologies (1) Broadband over existing telephone copper pair

E.g., Cat 3

Evolve starting from ~1997 ADSL (Asymmetric) ADSL2, ADSL2+ (u=24, d=3.3 Mbps), VDSL2 (u=100,

d=100/50Mbps), vectoring and G.Fast

VDSL2 profiles support different speed and deployment scenario

Wider and wider spectrum gradually used

DMT (discrete Multi-tone) Divides spectrum into channels (4.12135kHz)

Allow modulation scheme to be adapted according to noise situation at different part of spectrum

Quality and speed depends on: Distance sensitive due to attenuation

Copper wire quality, wire gauge, un-sealed termination, bridge taps

Cross-talk is one of the major issues

Near end cross talk (Next), Far end cross talk (FEXT)

Robustness improvement techniques – Interleaving, FEC

Binder25xTwisted Pair

Cable

DSLAM

ModemFEXTNEXT

Central Office /Cabinet Home

Home

Home

4KHz 0.14Mhz 1.1Mhz 2.2Mhz 8.8Mhz 12Mhz 17Mz 30Mhz

ADSL2+

VDSL2 frequency Profiles

ADSL2UpstreamPOTS

VDSL2 17A VDSL2 30A

0.276Mhz (annex M) VDSL – divides spectrum into multi-band for upstream, downstreamchannel width at 8K for Profile 30a

VDSL2 12AVDSL2 8A

Earlier xDSL Technologies

25Mbps

50Mbps

75Mbps

100Mbps

10 Mbps

1 km 2 km 3 km 4 km 5 km

24 Mbps

16 Mbps

8 Mbps

VDSL2 (performance subjected to FEXT)

ADSL2

ADSL

12.5Mbps

FTTC (Fiber to Curb)

FTTB (Fiber to Building)

ADSL2+

Copper connection

xDSL with Fiber uplink

powerCentral Office

xDSL Technologies (2) Vectoring – cancelling of x-talk effects

Theoretical speed of VDSL2 often not achievable due to cross talk

Computation of x-talk effect at far end and pre-compensate them, like noise cancellation

Push up the downstream speed of VDSL-2 to true ~ 150Mbps up to 400meter

G.fast – use spectrum up to 106Mhz Achieve 500-1000Mbps <100m, ~200 Mbps @ 200m

Vectoring is a must due to use of high frequency

ADSL

ADSL2+VDSL2 8b

VDSL2 17a

VDSL2

30 MHz

G.fast

106 MHz

G.fast

212 MHz

0 50 100 150 200

bandwidth [MHz]

ADSL

ADSL2+

VDSL2 8b

VDSL2 17a

VDSL2 30a

G.fast 100 MHz

G.fast 200 MHz

Modem

FEXT

FEXT from line 1 effect on Line-1 is considered, and certain pattern is preimposed

The x-talk effect from line 1 is cancelled

Line 1

Line 2

Time division (TDD) based for up and down steam

Unlike previous xDSL which is based on frequency band (FDD)

Fiber further deeper and near to home -> FTTdp (Fiber to the distribution point), ease of deployment form factor

212 Mhz coming

400-700 meters

FTTdp

Modem< 200 meters

Ease of Deployment

PON Technologies (1) PON (Passive Optical Network) ePON and GPON

GPON (Gigabit Passive Optical Networks) ITU G.984.1 – 984.6

Over single core fiber shared up to the splitting point Splitting ratio 1:32, 1:64, 1:128

Optical loss over distance, and over splitter

OLT (optical line termination) at the network side and ONT(optical network termination) at home

GPON protocol defined how the shared fiber medium is accessed Broadcast to all in downstream, but frames are identified for

individual ONT, upstream is based on TDMA (TDM Access)

DBA (dynamic bandwidth allocation) to control bandwidth and slots allocated to upstream, re-allocate every few ms

QOS by defining guaranteed, available, and best effort allocation mechanism using 5 types of T-CONT implemented by time slot allocation

Each ONT can have one or more T-CONT and identified on per ONT basis

OLT

2.5Gbps Shared

1.25 Gbps Shared

20 km

ONTFeeder Fiber (single core G.652)

1310 nm

1490 nm

Splitter

Distribution fiber

Drop fiber

GTC PayloadPCBd GTC PayloadPCBd GTC PayloadPCBd

Port_id Payload

GEM Frame

Port_id PayloadSync/OAM/Error Check US BwMap

Upstream Bandwidth Map

Alloc_Id Start Stop

125 us

Header Payload Header Payload Header Payload

Sync, OAM ONT T-CONT Buffer fill level

38880 bits

125 us19440 bits

Up stream

Down stream

Frame Structure (Simplified)

PON (Passive Optical Network) Deployment

OLT

Downstream bandwidth 2.5Gbps Shared

Upstream bandwidth = 1.25 Gbps Shared

< 20 km

ONT

Feeder Fiber (single core G.652)

Drop fiber1310 nm

1490 nm

Splitter

Distribution fiber

Fiber loss/km~0.36db

1:2 splitter~3.2db

1:32 splitter~16.5 db

Connector~0.4 db

Class B: 28.5db, Class C+: 32.5db

PON Technologies (2) NG PON1 – 10Gbps down 2.5Gbps up, coexists with current fiber and GPON, under ITU G.987.x Uses current fiber and splitter, coexists with GPON

Very important in keeping current investment

1:256 split, 40km

NG PON2 - TWDM (Time Division and Wavelength Multiplexed) PON under ITU G.989.x 4 or more wavelength in a fiber each λ runs 10Gbps

symmetrical

An ONT is allocated a single wavelength but share with other ONTs allocated the same wavelength

Start with a single wavelength and incrementally add in additional as needed

Cost effective Wavelength tunable ONT is needed

Alternative is to use fixed lambda

10Gbps applications Mostly used to backhaul other fttx equipment back to

network

Mobile backhaul for LTE-Advanced and beyond

OLT

10 Gbps Shared per λ

10 Gbps Shared per λ

40 km

ONT

1300 - 1320 1480-1500 1530 - 1540 1595 - 1625

Splitter

1260-1280 1575-1580

XGPON1 up GPON up GPON Dn NGPON2 up xGPON1 Dn NGPON2 DN

Co-exist and Mux and Demx of GPON, XG-PON1, NGPON2 λ to different line cards and

GPON, XG-PON1, NG-PON2 coexists

4 λ of NG-PON2

L Band1565 1625

C Band15301460

S Band1360

E BandO Band1260

Optical Networking Technologies

Wavelength Division Multiplexing Cost of deploying fiber is high, and takes time, e.g., in urban areas Economize on use of fiber, one pair for each router port connection is

expensive Transmit information on different wavelengths What determine how much we can transmit per wavelength, can we share

them? Define what wavelength are to be used, how much of it, and standardize

them

Optical Network Point to point fiber technology is simple but building fiber network this way is

expensive Ring provides redundancy, allow longer distance fiber to cover more stops Hence the function of add-drop is needed to share the pair of fiber But if more redundancy and larger area is needed, mesh topology is more

suitable Wavelength routing technology is needed to route a wavelength across a

mesh topology

Two Fiber pairs

DWDM

Two wavelength over one fiber pair

Point to point

Ring

Mesh

WDM Technologies (1) Carrying multiple router connections using different wavelengths

e.g., up to 8.8Tbps per fiber in case of 88 channels of 100Gbps each, or even 17.6 Tbps per fiber using 88 channels of 200Gbps

Normal 10Gbe optics transmits at 1310nm (LR -10km) or 1550nm (ER, ZR 40-80Km) over 2 core

DWDM (Dense Wavelength Division Multiplexing) Based on ITU G.694.1 grid of 12.5, 25, 50, 100Ghz channels plan at

C-Band and L-band

Up to 96 channels (currently) per fiber

High grade quality optics due to narrower channel width

Used where fiber is scarce, e.g. metro, long haul, and submarine cable

CWDM (Coarse Wavelength Division Multiplexing) Based on ITU G.694.2 grid of 20nm channel plan

Up to 16 channels, mostly 8 deployed

Relaxed channel width, less precise and cheaper optics can be used

Deployed where distance is short, economize on fiber usage is good but not at high cost, e.g., access

OTN (Optical Transport Network) Standard ◦ Used for multiplexing lower speed payload to higher speed

channel to more effectively utilize high speed λ

◦ ITU-G.709, cf. http://www.itu.int/ITU-T/studygroups/com15/otn/OTNtutorial.pdf

Mux/Demux(4, 8, 40 λ)Tx

Rcv

ProcessingMappingFEC

Tx

Rcv

ProcessingMapping

Rcv

Tx

ProcessingMapping

Rcv

Tx

ProcessingMapping

R1

R2

R3

R4

R5

R1

R2

R3

R4

R5

Amplifier for long distance

Normal B&W opticse.g., 10Gbe

Converting multiplexed OTU to Och (colored optics)e.g., 100 G λ

IP/Ethernet

10Gbe -> ODU-2

10Gbe -> ODU-2

ODU-4 OTU-4

OTU = Management overhead + ODU + FECe.g., OUT-4 > 100Gbps due to above

ODU-0=1Gbps, ODU1=2.5Gbps,ODU-2=10Gbps, ODU-3=40Gbps, ODU4=100Gbps

OchOTN

DWDM Transponder Transponder

Wavelength Division Multiplexing (C-band)

L Band1565 1625

C Band15301460

S Band1360

E BandO Band1260

19

1.3

5TH

z 1

56

6.7

2nm

19

1.4

THz

1

56

6.3

1nm

19

3.1

THz

1

55

2.5

2nm

19

3.1

5TH

z 1

55

2.1

2nm

19

3.0

5TH

z 1

55

2.9

3nm

19

6.0

5TH

z 1

52

9.1

6nm

19

6.1

THz

1

52

8.7

7nm

19

3.1

THz

1

55

2.5

2nm

19

3.2

THz

1

55

1.7

2nm

19

3.2

THz

15

55

1.7

2nm

19

5.9

THz

1

53

0.3

3nm

19

2TH

z

15

36

1.4

2n

m

193.1THz anchor

12

71

nm

12

91

nm

15

91

nm

16

11

nm

14

51

nm

14

71

nm

15

51

nm

15

71

nm

19

1.8

THz

15

36

3.0

5nm

19

6.1

THz

15

28

.77

nm

44 Channels40 channels

96 channels

16 channels

10 Gbe, 40 Gbe

10 Gbe, 40 Gbe, 100Gbe

CWDM

DWDM50Ghz

DWDM100Ghz

ITU G.694.1, 694.2 Grid

+6/7nmEach side, rest is for separation

Ring Topology and Protection

TransponderOADM

TransponderOADM

R1 R2

R3 R4

R5 R6R1 R2

R3 R4

R5 R6

Protected Unprotected(rely on dual routers)

Ring topology offers optics and fiber protection when traffic is sent via east and west direction

This can be done via using Y cable where incoming traffic is splited into two

transponders each transmit and receive in different direction

At the receiving side, only one of the incoming signal is selected and passed out of the node

Alternatively, one can simply use two routers, or a single router with two ports

Y cable show is actually deployed as a pair, one for splitting signal from input and the other for

Y-cable

Optical Network transport

InternetCoreService EdgeAggregationAccess

• Metro-transport • Transport and

aggregate the traffic from access to the core/data center

• Expected to grow very significantly as the access bandwidth increases and traffic consumption via video streaming increases

• Most cost effective ways of moving bits between access and data center

Core Router

Internet Gateway

BroadbandGateway/VPN Provider Edge

Core/Data Center

AAA/VAS

Transit Carrier Router

Internet Exchanges

GPON

Metro-ethernet

Central Office

xDSL

Access Node

• Fiber Path available ? Count increases

• Distance too far

• Port Count increases as traffic grows• One fiber per 10Gbe port?• Group them into 100Gbps and transmit

Optical Network transport

InternetCoreService EdgeAggregationAccess

• Metro DWDM transport • The example shown

here consists of two Metro-DWDM rings, each with two central offices and one core nodes

• A ring provides two paths connecting a CO to the core

• A core DWDM ring is also show connecting the two data centers and the internet exchanges

Core Router

Internet Gateway

Core/Data Center

AAA/VAS

Transit Carrier Router

Internet Exchanges

GPON

Metro-ethernet

Central Office

xDSL

Access Node

Core DWDM Ring

Metro DWDM Ring

WDM Technologies (2) Coherent Transmission

Major advancement for 100Gbps and beyond

Polarization, high order modulation, coherent detection, better DSP

ROADM (Reconfigurable OADM) Colorless – any λ can be add-drop

Directionless – traffic can go any direction

Contentionless – any λ (overlapping) can be add-drop at different direction

No manual work for above!

Based on WSS (Wavelength Selective Switch)

Enable True wavelength routing, dynamic set up, recovery in mesh network, improving redundancy and wavelength efficiency

FlexiGrid ITU defined granularity at 12.5Ghz, nx12.5GHz to form

“superchannel”, allowing more flexible allocations and improve efficiency

E.g. possible to carry 100Gbps over a 37.5Ghz superchannel, 400 Gbps is carried as 100Ghz super channel rather than 4 x50Ghz 100Gbps channel

Because the of the arbitrary width of superchannel, flexgridrequires new filter, wss and transceiver

50Ghz 100Gbps

100Gbps /37.5Ghz 400Gbps /100Ghz 1Tbps /200Ghz

WSS

4-Degree WSS

R1 R2

R3

R4

R5

R1

R2

R3

R4

R5

Site A

Site B

Site C

Site D

Site E

Site F

Internet Routing (conceptual)IP Routing among service providers (EGP)

Border routers of service provider are interconnected

Border router shares the set of IP addresses that it’s own service provider can reach

From connected foreign border routers, learn what are the IP addresses other service providers can reach

Forward Internet packet to the selected foreign border router

Not aware of how it is routed subsequently, no visibility to the topologies of other service providers

Service provider fully control the routing within is own domain called AS (Autonomous System), with unique AS number

E.g., BGP

IP Routing within a Service Provider (IGP) By having router to share IP addresses and its link costs among routers within the

same service provider

The IP addresses (network part) and network topology is visible to all

A router determine the “shortest” path to route a given IP address

IGP Protocol, E.g., OSPF, IS-IS

SP1

SP2

SP3

SP4

IP addresses that I can reach, include those from mine SP2 , and SP3

Web site’s hosting data center/ Cloud

Management Port

Control Card

Line Cards

Fan Tray

Switching Cards

Line Card – Ingress • De-framing• Classification of

traffic by QoS• Look up on

destination identify port/card for egress

• Execute any configured policy

• Buffering and transmit to the switching card

• Maintain forward table in conjunction with control card

• Collect statistics

Line Cards

Line Card - Egress• Accept packet or frame from

switching card• Queuing and buffering according to

class of service at the egress port• Framing for transmission• Collect statistics

• Run routing protocol and other service protocol

• Formulate routing table and forwarding table

• Accept management command for configuration

• Monitor the status of the equipment

• Switch packet or frame to the destination line card

Anatomy Of A Router

L2 vs. L3 L2 based on MAC addresses (48 bits) A switch keeps track of which direction to forward for MAC addresses

Broadcast to all directions when it does not know how to forward

Spanning tree is used to restrict broadcast and avoid looping and storm

Learn the direction when the destined terminal sends return frame

VLAN provides “domain” separation, L2 traffic broadcasted within VLAN, often correspond to a IP subnet (see below)

L3 based on IP addresses 32 bit of IP addresses has network and host part, routing requires knowing

where to direct based on network address (IP subnet), same for IPv6 (128bit)

IP scalability allows us to build Internet, only network part of IP address is learned (best practice)

In IGP - A router keeps routing table, exchange IP addresses and topologies via routing Protocol

When link failure is noted, router sends via other path to destination

IP Addresses can be public which is unique worldwide and used in Internet assigned by Internet Authorities to service providers and then to customers

IP Address can also be private overlapping, e.g., 10.0.0.0/8, 192.168.0.0/16, and reused by different organizations internally

B

1 2

1 2 1 2

Routing Protocol

VLAN X VLAN Y

R1 R2

R3

MAC Addresses

L3 Routing Table

IP Network Part

A

C

1

L2 Forwarding Table

A

C

1

Subnet

InternetCoreService EdgeAggregationAccess

• iBGP – shared the learned IP addresses from border gateway within the operator’s own router network, allow internet packet to be routed

xDSLIGP, e.g., OSPF/ISIS

EGP, i.e., BGPiBGP

L2

• IGP – routing of IP addresses within the operator domain, e.g., OSPF, IS-IS

• Each router has full visibility of links, routers, and what addresses are reachable behind router

• For each IP network addresses that it is aware of from the IGP, each router find the shortest path from itself to the destination router

• Any packet coming in with that IP address is routed out to the first link on the path

• BGP – allows an operator to tell another operator what IP networks addresses (Prefix) it can reach

• Exchanged between border routers of different AS (Autonomous System)

• To go to IP destination belonging to other operators, a service provider route the packet to the “right” border router, based on IP addresses it learned via BGP

• The border router takes it from there

• Access and Aggregation –normally switch packet by L2, either up or down. Simple configuration

Routing Architecture (example)

Version (4)

Header Length (4)

Priority and Type of Service (8) Total Length (16)

Identification(16)Flat (3)

Fragment Offset (13)

Time to live (8) Protocol (8) Header Checksum (16)

Source IP Address (32)

Destination IP Address (32)

Data

IP Option (0 or 32)

Bit 0 Bit 15 Bit 31

IPv4 Header

QOS Architecture (example)

InternetCoreService EdgeAggregationAccess

QOS Classes• QOS are supported at Ethernet

level (L2) by p-bits (3) in the header, also supported at IP level by DSCP bits (6) in the header

• Up to 8 classes of QoS are usually defined in deployment

• Highest two class are devoted to service provider’s own control and management traffic

• Voice is usually given highest class out of the remaining due low latency requirement

• Video such as IPTV is usually given the second highest, to avoid high lost impacting visual experience

• Important data, e.g., corporate data could also be given a separate class

• Finally, normal internet traffic (including OTT) is given BE (Best effort service)

xDSL L2

ONT remark upstream class of traffic DSCP and 802.1 p-bits when relaying incoming traffic from home. Work with OLT to schedule upstream transmission

OLT schedule upstream with ONT via DBA, manage downstream traffic on a per ONT or even per port per ONT basis based on profile and classes of traffic

BNG keeps subscriber profiles which defines the policy for up and down stream bandwidth or total data. May perform further remarking by looking into the applications type

AAA, Policy

Aggregation router follows the class of service marking and schedule the packet accordingly

Core and border router schedule the traffic according to class of service. If outgoing to overseas, the traffic is passed to transit router.

L3

References and Some Further Info:Fiber Deployment

- https://www.youtube.com/watch?v=a8bzZajwR50

Internet Statistics

- https://www.sandvine.com/trends/global-internet-phenomena/

- https://content.akamai.com/PG2061-SOTI.html?gclid=CMCc3ZCptscCFQ0rjgodTHECSA

- http://www.cisco.com/c/en/us/solutions/service-provider/visual-networking-index-vni/index.html

Internet Exchange example and International Connectivity:

- https://ams-ix.net/

- https://www.telegeography.com/

Network Vision example:

- https://techzine.alcatel-lucent.com/

- http://newsroom.cisco.com/focus

Industry and Standard Organizations

- https://www.broadband-forum.org/

- http://www.ietf.org/

- http://www.itu.int/en/ITU-T/Pages/default.aspx