LLD of AXIS Amber Project IP RAN Jabodetabek (L3VPN) v1.8 - No Yet Update

Low Level Design---- AXIS AMBER IP RAN Jabodetabek

Quidway CX600

Issue V1.7

Date 2011-09-19

CONFIDENTIAL

Huawei Technologies Co., Ltd.

LLD of AXIS Jabodetabek IP RAN Network

About This Document

Author

Prepared by Hendra Rusmin Date 2011-07-21

Reviewed by

Xu Zhao Pan Date 2011-08-05

Approved by

Date

SummaryThis document describes service transmission, protocol control, data forwarding, protection, QoS, clock synchronization, and operation and maintenance (O&M) for the CX solution, with an intention to show how the CX solution helps tackle the issues regarding costs, service quality, and O&M that will be encountered during the process of mobile network evolution to broadband. The design of the CX solution aims at mobile broadband carrier networks.

Issue v1.2 (2011-08-05)Confidential Information of

HuaweiNo Spreading without Permission

Page 2 of 122


History

Issue Details Date AuthorApproved by

v1.0 Create2011-07-21

Hendra Rusmin

v1.1 Update Sync. Eth2011-08-05

xuzhaopan

v1.1 Update Redundancy2011-08-06

xuzhaopan

v.1.2Update ip allocation table, L2&L3 connection table, Vlan & VC design

2011-08-08

Hendra Rusmin

v1.3Update some drawing in OSPF, MPLS, VLL and Sync. Eth

2011-08-09

xuzhaopan

v1.4Update some comments from cust

2011-08-16

Hendra Rusmin

v1.5

Update DCN & OAMUpdate VLL, topology, still need update security, board allocation

2011-08-20

xuzhaopan

V1.6 Update Security 2011-08-23

zhangpengfei

V1.7Update board allocationUpdate Phase1 & Phase2

2011-09-19

xuzhaopan



Page 3 of 122


Contents

About This Document........................................................................2Author...................................................................................................................................................................2

Summary..............................................................................................................................................................2

History...................................................................................................................................................................3

1 Project Planning..........................................................................121.1 Project Overview.............................................................................................................................................12

1.1.1 Jabodetabek IP RAN..............................................................................................................................13

1.2 BoQ.................................................................................................................................................................13

1.3 Requirement....................................................................................................................................................14

1.4 Products Introduction......................................................................................................................................14

1.4.1 Hardware Information...........................................................................................................................14

1.4.2 Software Version....................................................................................................................................17

1.4.3 Specification..........................................................................................................................................17

1.4.4 Hardware Redundancy...........................................................................................................................19

1.5 Network Topology...........................................................................................................................................19

1.5.1 IP RAN Topology Phase 1.....................................................................................................................19

1.5.2 IP RAN Topology Phase 2.....................................................................................................................20

1.5.3 Service Flow..........................................................................................................................................23

1.6 Physical Connection and Slot Allocation........................................................................................................24

1.6.1 Core Sites...............................................................................................................................................24

1.6.2 Single Sites (No Ring)...........................................................................................................................25

1.6.3 Ring-1 Sites...........................................................................................................................................27

1.6.4 Ring-2 Sites...........................................................................................................................................29

1.6.5 Ring-3 Sites...........................................................................................................................................33

1.6.6 Ring-4 Sites...........................................................................................................................................35

1.6.7 Ring-5 Sites...........................................................................................................................................37

2 Device Naming.............................................................................392.1 Device Name Design.......................................................................................................................................39

2.2 Interface Description Design...........................................................................................................................41

3 VLAN Design................................................................................423.1 Overview.........................................................................................................................................................42



Page iv of cxxii


3.2 VLAN Design Regulation...............................................................................................................................42

3.2.1 Node B OAM VLAN.............................................................................................................................42

3.2.2 RTN OAM VAN....................................................................................................................................42

3.2.3 OSN OAM VLAN.................................................................................................................................43

3.2.4 Special Case...........................................................................................................................................43

3.3 VLAN and VLL Planning (Access)................................................................................................................45

4 IP Address Design........................................................................474.1 Overview.........................................................................................................................................................47

4.2 Detail IP Address Table...................................................................................................................................47

4.3 IP Address for OAM........................................................................................................................................47

5 Routing Planning.........................................................................495.1 Overview.........................................................................................................................................................49

5.2 IGP Planning...................................................................................................................................................49

5.2.1 OSPF Process 100 use in Jabodetabek..................................................................................................50

5.2.2 Link type Design....................................................................................................................................51

5.2.3 Cost Design............................................................................................................................................51

6 MPLS Planning.............................................................................526.1 Overview.........................................................................................................................................................52

6.2 MPLS RSVP TE Design.................................................................................................................................53

6.3 TE Hot-standby parameter..............................................................................................................................54

7 MPLS L2VPN Design.....................................................................557.1 Overview.........................................................................................................................................................55

7.2 Phase 1: BSC/RNC Sites is Single CX600.....................................................................................................56

7.3 Phase 2: BSC/RNC Sites is Daul CX600........................................................................................................57

7.3.1 Scenario for BTS/Node B to BSC/RNC................................................................................................57

7.3.2 PW Redundancy design.........................................................................................................................59

7.3.3 VPLS design for RTN and BSC/RNC in same site...............................................................................60

8 Reliability Planning......................................................................628.1 LSP 1:1 Protection in IP RAN.........................................................................................................................62

8.2 OSPF Fast Convergence..................................................................................................................................62

8.3 BFD Detect multiple-hop................................................................................................................................63

8.4 RTN to CX600 LAG Protection......................................................................................................................64

8.5 Phase1: BSC/RNC to CX – Single CX...........................................................................................................65

8.6 Phase2: BSC/RNC to CX – Dual CX (PW redundancy)................................................................................66

8.6.1 Scenario 1: Failure in IP RAN...............................................................................................................69

8.6.2 Failure of CX600-1................................................................................................................................70

8.6.3 Scenario 3: Failure between CX600-1 and BSC/RNC..........................................................................71

9 QoS Planning...............................................................................729.1 Overview.........................................................................................................................................................72



Page v of cxxii


9.2 IP RAN MPLS Network QoS design..............................................................................................................73

10 Sync. Eth Clock Planning............................................................7710.1 Overview.......................................................................................................................................................77

10.1.1 Synchronization Requirements............................................................................................................77

10.1.2 Clock requirements of radio networks.................................................................................................78

10.1.3 Phase/Frequency synchronization.......................................................................................................79

10.1.4 Equipment support Sync. Eth State.....................................................................................................79

10.1.5 Board and device support Sync...........................................................................................................80

10.1.6 Clock delay and jitter...........................................................................................................................81

10.1.7 Packet loss for clock............................................................................................................................82

10.1.8 How to calculate Sync. Eth hop...........................................................................................................83

10.2 Sync. Eth Design...........................................................................................................................................84

10.2.1 Two Clock Source redundancy............................................................................................................84

10.2.2 Primary Source Down..........................................................................................................................85

10.2.3 Link Down...........................................................................................................................................86

10.3 Sync. Eth Deploy...........................................................................................................................................87

10.3.1 Configure Sync. Eth in CX600............................................................................................................87

10.3.2 Main Clock Trace Path in IP RAN Jabo..............................................................................................87

10.3.3 Sync. E Priority in IP RAN Jabo.........................................................................................................88

10.3.4 Configure Sync. Eth Example.............................................................................................................91

11 Security Planning.......................................................................9411.1 Overview.......................................................................................................................................................94

11.2 Access Control...............................................................................................................................................94

11.3 Tacacs Authentication....................................................................................................................................95

11.4 Route encrypt................................................................................................................................................95

12 CX-PE Connection Planning.........................................................9712.1 Overview.......................................................................................................................................................97

12.2 Physical interface..........................................................................................................................................97

12.3 Protocol deploy.............................................................................................................................................97

13 Operation and Maintenance Planning..........................................9913.1 Overview.......................................................................................................................................................99

13.1.1 Network Deployment...........................................................................................................................99

13.1.2 Service Provisioning..........................................................................................................................100

13.1.3 Service Quality Assurance.................................................................................................................101

13.2 O&M and NMS Planning............................................................................................................................102

13.2.1 NMS Overview..................................................................................................................................102

13.2.2 BTS O&M.........................................................................................................................................103

13.2.3 Node B O&M....................................................................................................................................103

13.2.4 RTN O&M.........................................................................................................................................104

13.2.5 OSN1800 O&M.................................................................................................................................105



Page vi of cxxii


13.2.6 CX600 O&M.....................................................................................................................................106

13.3 Y.1731 deployment in CX...........................................................................................................................107

13.3.1 Y.1731 overview................................................................................................................................107

13.3.2 Basic Concepts and Principle............................................................................................................108

13.3.3 Y.1731 deployment............................................................................................................................111

13.4 NTP.............................................................................................................................................................113

13.4.1 NTP Overview...................................................................................................................................113

13.4.2 NTP Design........................................................................................................................................113

14 Annex......................................................................................114



Page vii of cxxii


Figures

Figure 1-1 CX Equipment...............................................................................................................................14

Figure 1-2 System configuration list of the CX600-X8........................................................................17

Figure 1-3 System configuration list of the CX600-X3........................................................................17

Figure 1-4 Physical connection in AXIS Jabodetabek – single CX in BSC/RNC sites.................20

Figure 1-5 Network models in AXIS Jabodetabek..................................................................................21

Figure 1-6 Service Flow...................................................................................................................................22

Figure 1-7 Physical topology of Menara Dea..........................................................................................23

Figure 1-8 Physical topology of Citra Graha............................................................................................24

Figure 1-9 Physical topology of Serang Banten TV..............................................................................25

Figure 1-10 Physical topology of Menara Peninsula.............................................................................26

Figure 1-11 Physical topology of Gedung Total.......................................................................................26

Figure 1-12 Physical topology of Menara Bank Dagang Negara......................................................27

Figure 1-13 Physical topology of APart. Puri ImPerium........................................................................27

Figure 1-14 Physical topology of Manggala Wanabakti.......................................................................28

Figure 1-15 Physical topology of Wisma Indovision..............................................................................28

Figure 1-16 Physical topology of Royal Tower Riverside, Pluit..........................................................29

Figure 1-17 Physical topology of Gajah Mada Tower............................................................................29

Figure 1-18 Physical topology of Sunter Mall..........................................................................................30

Figure 1-19 Physical topology of CemPaka Putih...................................................................................30

Figure 1-20 Physical topology of Graha Amaba.....................................................................................31

Figure 1-21 Physical topology of German Center..................................................................................32

Figure 1-22 Physical topology of Condo Golf Karawaci.......................................................................32

Figure 1-23 Physical topology of Tangerang Gondrong.......................................................................33

Figure 1-24 Physical topology of Metro Pondok Indah.........................................................................34

Figure 1-25 Physical topology of Cilandak APartment.........................................................................34

Figure 1-26 Physical topology of Metro Depok Greenfield.................................................................35



Page viii of cxxii


Figure 1-27 Physical topology of Bogor Situ Cikaret Greenfield......................................................35

Figure 1-28 Physical topology of Bekasi Villa Nusa Indah..................................................................36

Figure 1-29 Physical topology of Bekasi Mulia Industri.......................................................................36

Figure 1-30 Physical topology of Bekasi Trade Center.........................................................................37

Figure 3-1 Typical Microwave Network......................................................................................................42

Figure 3-2 Example for one Arm have BTS belong to different BSC..............................................43

Figure 5-1 OSPF Area.......................................................................................................................................48

Figure 6-1 MPLS RSVP-TE................................................................................................................................52

Figure 7-1 VLL example..................................................................................................................................54

Figure 7-2 example for VLL work mode (Single CX).............................................................................56

Figure 7-3 BTS/Node B (RTN) going to Remote BSC/RNC...................................................................57

Figure 7-4 BTS/Node B (RTN) going to Local BSC/RNC........................................................................57

Figure 7-5 BTS/NodeB going to Local BSC/RNC Directly....................................................................58

Figure 7-6 example for VLL work mode (Dual CX)................................................................................59

Figure 7-7 RSVP-TE + VLL Redundancy + MC-LAG...............................................................................59

Figure 7-8 RSVP-TE + VLL Redundancy + MC-LAG...............................................................................60

Figure 8-1 BFD for non-direct CX connection.........................................................................................62

Figure 8-2 Physical connection....................................................................................................................63

Figure 8-3 Logical connection.......................................................................................................................63

Figure 8-4 One Link down..............................................................................................................................64


Figure 8-6 Logical Connection......................................................................................................................65

Figure 8-7 One link down...............................................................................................................................65


Figure 8-9 Logical connection.......................................................................................................................67

Figure 8-10 IP RAN 10G down.......................................................................................................................68

Figure 8-11 IP RAN Recovery.........................................................................................................................68

Figure 8-12 CX Down recovery.....................................................................................................................69

Figure 8-13 CX router recovery....................................................................................................................69

Figure 8-14 Access link of BSC/RNC down................................................................................................70

Figure 8-15 Access link of BSC/RNC recovery.........................................................................................70

Figure 9-1 QoS solution...................................................................................................................................72

Figure 10-1 Sync. Eth flow..............................................................................................................................76



Page ix of cxxii


Figure 10-2 Phase and Frequency synchronizations............................................................................78

Figure 10-3 1588 ACR......................................................................................................................................81

Figure 10-4 Example for Hop Calculate.....................................................................................................82

Figure 10-5 Sync. Eth Master/Slave Design Planning..........................................................................83

Figure 10-6 Network models in AXIS Amber............................................................................................84

Figure 10-7 Network models in AXIS Amber............................................................................................85

Figure 10-8 Main clock trace path...............................................................................................................86

Figure 10-9 Sync. Eth backup in CX............................................................................................................90

Figure 11-1 Remote login and control CX.................................................................................................94

Figure 11-2 CX OSPF Route Authentication..............................................................................................94

Figure 12-1 CX-PE..............................................................................................................................................97

Figure 13-1 NMS System...............................................................................................................................101

Figure 13-2 BTS OAM Service Flow...........................................................................................................102

Figure 13-3 Node B OAM Service Flow.....................................................................................................103

Figure 13-4 RTN OAM Service Flow...........................................................................................................104

Figure 13-5 OSN OAM Service Flow..........................................................................................................105

Figure 13-6 CX600 OAM Service Flow......................................................................................................106

Figure 13-7 Y.1731 overview........................................................................................................................107

Figure 13-8 Networking diagram of single-ended frame loss measurement............................108

Figure 13-9 BTS Networking diagram of dual-ended frame loss measurement......................109

Figure 13-10 Networking diagram of one-way frame delay measurement...............................109

Figure 13-11 Networking diagram of two-way frame delay measurement................................110

Figure 13-12 Y.1731 deployment in CX....................................................................................................110



Page x of cxxii


Tables

Table 1-1 BOQ Main Material.........................................................................................................................13

Table 1-2 Hardware Redundancy For CX600x3......................................................................................18

Table 1-3 Hardware Redundancy For CX600x8......................................................................................18

Table 1-4 BSC/RNC Sites.................................................................................................................................20

Table 2-1 IP Core Equipment Naming Rule..............................................................................................38

Table 2-2 IP Core Type Rule...........................................................................................................................38

Table 2-3 Sites ID and Naming Abbreviation..........................................................................................39

Table 3-1 VLAN/IP/VC ID Design...................................................................................................................45

Table 9-1 IP RAN QoS Classification guideline........................................................................................73

Table 9-2 Queue scheduling algorithm.....................................................................................................75

Table 10-1 Clock requirements of radio networks.................................................................................77

Table 10-2 Equipment support Sync. Eth Status...................................................................................78

Table 10-3 Board support Sync. Eth Status.............................................................................................80

Table 10-4 Clock requirements for Jitter, delay and Packet lost......................................................81

Table 10-5 Sync. Eth Clock Priority in IP RAN Jabo................................................................................89



Page 11 of 122


1 Project Planning

1.1 Project Overview

The requirements on network bandwidths rapidly increase with the evolution of mobile services from only voice services to integrated broadband services, such as voice, streaming media, and high speed downlink packet access (HSDPA) services. In addition, the increase of operation costs in traditional access mode brings enormous pressure on carriers. Therefore, the carriers are in urgent need of an access mode featuring low costs, flexibility, and high efficiency to face the challenges of broadband-oriented development. ALL-IP transformation over mobile services has become the trend for network development.

The RTN+CX back-to-back (B2B) mobile backhaul solution proposed by Huawei is based on Huawei RTN and CX600 devices. Based on access modes on base stations, services are classified into time division multiplexing (TDM), asynchronous transfer mode (ATM), and Ethernet+IP services. Currently, RTN devices access services. By working with the CX600, the RTN devices carry 2G/3G/LTE mobile services using the packet switching technology. LTE refers to long term evolution. In addition to the preceding services, multiple access networks (AN) services, such as Internet access, enterprise virtual private network (VPN), and Internet Protocol television (IPTV) services, is provided by carriers.

RTN devices provide various service interfaces and therefore feature flexible configuration and easy installation. They provide an integrated solution to TDM, Hybrid, and packet microwave based on network requirements, supporting hitless upgrade from TDM to Hybrid microwave and from Hybrid to packet microwave. This solution can be developed with the development of mobile network services. Therefore, it meets the requirements of not only the current 2G/3G networks but also the future LTE and 4G networks.



Page 12 of 122


1.1.1 Jabodetabek IP RAN› Jabodetabek Region use DWDM (ONS1800)+CX600+RTN(ATN) to build the IP

RAN Network, there have 5 primary Ring in phase one.

» OSN1800: 23 (serang use OSN3500 to JAVA 10G Backbone)

» CX600x8: 4 (CG, DEA)

» CX600x3: 22 (including Serang)

» BSC: 10

» RTN: 6

› BTS/Node B to BSC/RNC Service bears by IP RAN and BSC/RNC to MGW/SGSN

service bear by IP Core network.

› CX600 IP RAN use MPLS L2 VPN VLL to bear the service.

› RTN use E-Line solution, so each BTS/Node B has different VLAN, BSC/RNC use

different VLAN to separate each BTS/Node B, each different BSC/RNC can use

same VLAN Range.

› Next PO will add another CX600 in BSC and RNC site for redundancy.

1.2 BoQRelated material as follow:

IP RAN CX600 Series

No.

Part Number Model

Total Quantit

y1 02113041 2.2m Router Assembly Cabinet with Double Swing Doors 23

2 S4016684CX600-X8 Basic Configuration(Including CX600-X8 Chassis,2+1 Redundant SRU/SFU,2*2G DDR2 SDRAM,4*1G FLASH,4 DC Power,without Software Charge and Document)

2

3 S4016045 CX600-X3 Basic Configuration(Include CX600-X3 Chassis,Dual MPUs,2*2G DDR SDRAM,4*1G FLASH,Dual DC Power,without Software Charge and Document) 23

4 03038461 Flexible Card Line Processing Unit(LPUF-10,four slots) 25 5 03030KNE 8-Port 100/1000Base-X-SFP Flexible Card A(P10-A,Supporting 1588v2) 25 6 03053055 Flexible Card Line Processing Unit(LPUF-40,2 sub-slots) B 6 7 03052557 Flexible Card Line Processing Unit(LPUF-21,2 sub-slots) B 23 8 03038466 2-Port 10GBase LAN/WAN-XFP Flexible Card A(P40-A,Supporting 1588v2) 12 9 03030KKP 1-Port 10GBase WAN/LAN-XFP Flexible Card A(Supporting 1588v2) 46

10 S4015798 Optical Transceiver(XFP,850nm,10.3Gb/s,-7.3dBm~-1.3dBm,-7.5dBm,LC,Multimode,0.3Km) 70

11 34060286 Optical Transceiver,eSFP,850nm,2.125Gb/s(Multi rate) ,-9.5~-2.5dBm,-17dBm,LC,MM,0.5km 200

12 S4024281 Optical Patch Cord(PCS) 540

13 25030101 Wire,450/750V,60227 IEC 02(RV)25mm^2,blue,110A,With a package exempted from fumigating(per meter) 1350

14 25030432 Wire,450/750V,60227 IEC 02(RV)25mm^2,black,110A,With a package exempted from fumigating(per meter) 1350

15 25030431 Wire,450/750V,60227 IEC 02(RV)25mm^2,yellow green,110A,With a package 625



Page 13 of 122


exempted from fumigating(per meter)

Table 1-1 BOQ Main Material

<Please refer to Annex BOQ>

1.3 RequirementThis project objective is to provide IP RAN service use IP RAN CX600 series.

The requirement of network IP RAN and PE are:

1. IP RAN topology will be ring, use ring topology we will have east west model protection.

2. IP RAN network will deliver Connection between BTS to BSC including clocking connection.

1.4 Products Introduction

1.4.1 Hardware InformationQuidway CX600 Metro Services Platform (hereinafter referred to as the CX600) is a type of edge CX device with a 10 Gbit/s interface. It is developed on the basis of the Huawei Versatile Routing Platform (VRP) and features large capacity, high performance and high reliability. The CX600 has a powerful monitoring system. The Main Processing Unit (MPU) on the Switch and Route Processing Unit (SRU) implements the management and maintenance of the entire system. The MPU can manage, monitor, and maintain the boards, fans, and power distribution and clock modules.



Page 14 of 122


Figure 1-1 CX Equipment

1.4.1.1 MPU

As the main control unit of the CX600-X3 call MPU, the MPU controls and

manages the system and exchanges data. The CX600-X3 provides two MPUs.

The MPUs work in 1:1 backup mode. An MPU consists of the main control unit,

clock unit, and system maintenance unit. There are two CF cards on the MPU:

One is located on the PCB; the other is located on the panel.

1. Ejector lever 2. BITS0 3. BITS1 4. ETH0

5. CF OFL button 6. CF indicator 7. CF card slot 8. Reset button

9. OFL button 10. OFL indicator 11. ACT indicator 12. ALM indicator

13. RUN indicator 14. Console 15. AUX interface

1.4.1.2 SRU

As the main control unit of the CX600-X8 call SRU, the SRU controls and

manages the system and exchanges data. The CX600-X8 provides two MPUs.

The MPUs work in 1:1 backup mode. An SRU consists of the main control unit,



Page 15 of 122

Huawei CX Series Overview-JabotabekHigh GE/10GE density

Abundant Interface Card

Powerful L2&L3 function

LTE Ready

Full Service

CX600-X3

CX600-X8

32U

14U

5U4U1U

10UCX600-X16

CX600-X2

CX600-X1

CX600 MiniFirst 300mm Depth

Single platform: all product is based on routing platform - VRP

Powerful platform, support universal service: 2G/3G/LTE service, Triple play, HDTV, P4P, business MPLS-TP/MPLS

L3 Ready , meet LTE requirement

ATN 910


clock unit, and system maintenance unit. There are two CF cards on the MPU:

One is located on the PCB; the other is located on the panel.

1. Ejector lever 2. CLK/Serial 3. CLK/1PPS 4. CLK/TOD

5. AUX 6. Console 7. MGMT-ETH 8. CTL-ETH-SFP

9. LINK/ACT 10. CF OFL button

11. CF card 12. CF indicator

13. USB 14. RESET 15. ALM indicator 16. OFL button

17. OFL indicator 18. RUN indicator

19. ACT active/standby indicator

20. MGMT-ETH LINK indicator

21. MGMT-ETH ACT indicator

4.1.1.3 LPU

Line interface Processing Unit (LPU) provides various physical interfaces to

external networks.

There are three types of LPU:

Ethernet LPU: provides Ethernet interface

POS LPU: provides POS interface

ATM LPU: provides ATM interface

4.1.1.4 Power Supply Module

The CX600-X3 supports DC power input and AC power input. CX600-X3 , which

act as TDM-Gateway Router in PBoF Network use DC power. Power modules of

the CX600support hot swap and 1+1 backup.Special slots are prepared for

power modules of the CX600-X3.The DC power modules are located on the back

of the device. The AC power modules are located in front of the device.

CX600-X3 has one straight-through power, which is capable of the Surge

protection, filter, and short circuit protection Alarm function.

4.1.1.5 FAN Module

The heat dissipation system is responsible for dissipating heat for the entire

system. The heat generated by boards is dissipated through the heat dissipation



Page 16 of 122


system. In this manner, the temperature of the components on boards is

controlled within a normal range, enabling the boards to work stably.

The heat dissipation system is composed of fan modules (two fans in each fan

module), fan control boards (FCBs), temperature sensors, air filters, air intake

and exhaust vents, and a system air channel.

All fans work at the same time and the speed of all fans is adjusted at the same

time. When one fan becomes faulty, the other fans automatically rotate at full

speed. When a single fan fails, the heat dissipation system enables the system

to work in a short period of time at ambient temperature of 40°C.

Temperature sensors, located on the air exhaust vent and boards, are used to

monitor the temperature of the components on boards and adjust the fan speed

through the command delivered by the MPU to control the temperature in a

normal range.

The power modules of the system have two fans of their own for independent

heat dissipation.

1.4.2 Software VersionThe main software inventory is as follows.

CX600-X8/3: V600R003C00 (for version we will user newest version that is V6R3)

U2000: U2000 V100R005C00

1.4.3 Specification

» CX600-X8



Page 17 of 122


Figure 1-1 System configuration list of the CX600-X8

CX600-X3

Figure 1-2 System configuration list of the CX600-X3

1.4.4 Hardware Redundancy



Page 18 of 122


» CX600-x3

ModulesUnit

Redundancy type (1+1 or N+1)

Power 2 1+1LPU 2/3 / MPU 1 1:1SFU 1 No FAN 2 1+1

Table 1-1 Hardware Redundancy For CX600x3

» CX600-x8

ModulesUnit

Redundancy type (1+1 or N+1)

Power 2 1+1LPU 2/3 /SRU 2 1:1SFU 2 1+1FAN 2 1+1

Table 1-2 Hardware Redundancy For CX600x8

1.5 Network Topology

1.5.1 IP RAN Topology Phase 1

› Use The Material of BoQ Patch 1.

› BSC/RNC Sites just one CX.

› Just One 8 port GE board.



Page 19 of 122


1.5.2 IP RAN Topology Phase 2

› There have 23 CX sites.

› Jaodetabek have 5 Ring.

› CG, DEA, and some RnC/BSC site below will have 2 NE, other sites is single

node.

› All CX of IP RAN network running MPLS L2VPN.

Site Name BSC/RNCCitra Graha 3 / 1Royal Tower Riverside, Pluit 2 / 1KomPleks Polri Batu CePer (Gondrong) Tangkarang

1 / 1

DePok Greenfield 1 / 0Situ Cikaret Greenfield Bogor 1 / 1Bekasi Trade Center 1 / 1Banten TV (Serang) 1 / 1



Page 20 of 122


Table 1-1 BSC/RNC Sites

› CX and RNC support MC-LAG (LACP), But BSC cannot support MA-LAG.

› Configure two PW from edged CX to Core (BSC/RNC) CX, PW redundancy.

Figure 1-2 Physical connection in AXIS Jabodetabek – single CX in BSC/RNC sites



Page 21 of 122


Logical Topology (E2E)

Figure 1-3 Network models in AXIS Jabodetabek

› BTS/Node B to BSC/RNC service Abis, Iub bear by IP RAN Network.

› BSC/RNC to MGW/SGSN service Gb, IuPS, AoIP, IuCS, Iur bear by IP Core Network.

› IP RAN use VLL VPN and IP Core use L3VPN.

› RTN to CX600 use LAG protection mode and BSC/RNC use Master/Slave mode, RNC can support LAG LACP, but BSC cannot.

› BSC/RNC to NE40E use VRRP protection mode.



Page 22 of 122

Last Mile Access IP RAN

BTS

Node B

BTSFE

FE

FE

10GE

BITSG

E

GE

GE

GE

GE P

E

IP Core / Core Network

U2000MGW

MSC

SGSN

Abis

BSC/RNC

BSC

RNC

A/Gb IP Core


1.5.3 Service Flow

Figure 1-1 Service Flow



Page 23 of 122


1.6 Physical Connection and Slot Allocation

1.6.1 Core Sites» Menara Dea

Figure 1-1 Physical topology of Menara Dea



Page 24 of 122


1.6.2 Single Sites (No Ring)» Citra Graha

Figure 1-1 Physical topology of Citra Graha



Page 25 of 122


» Serang Banten TV

Figure 1-2 Physical topology of Serang Banten TV



Page 26 of 122


1.6.3 Ring-1 Sites

» Menara Peninsula

Figure 1-1 Physical topology of Menara Peninsula

» Gedung Total

Figure 1-2 Physical topology of Gedung Total



Page 27 of 122


» Menara Bank Dagang Negara

Figure 1-3 Physical topology of Menara Bank Dagang Negara

» APart. Puri ImPerium

Figure 1-4 Physical topology of APart. Puri ImPerium



Page 28 of 122


1.6.4 Ring-2 Sites

» Manggala Wanabakti

Figure 1-1 Physical topology of Manggala Wanabakti

» Wisma Indovision

Figure 1-2 Physical topology of Wisma Indovision



Page 29 of 122


» Royal Tower Riverside, Pluit

Figure 1-3 Physical topology of Royal Tower Riverside, Pluit

» Gajah Mada Tower

Figure 1-4 Physical topology of Gajah Mada Tower



Page 30 of 122


» Sunter Mall

Figure 1-5 Physical topology of Sunter Mall

» CemPaka Putih

Figure 1-6 Physical topology of CemPaka Putih



Page 31 of 122


» Graha Amaba

Figure 1-7 Physical topology of Graha Amaba



Page 32 of 122


1.6.5 Ring-3 Sites

» German Center

Figure 1-1 Physical topology of German Center

» Condo Golf Karawaci

Figure 1-2 Physical topology of Condo Golf Karawaci



Page 33 of 122


» Tangerang Gondrong

Figure 1-3 Physical topology of Tangerang Gondrong



Page 34 of 122


1.6.6 Ring-4 Sites

» Metro Pondok Indah

Figure 1-1 Physical topology of Metro Pondok Indah

» Cilandak APartment

Figure 1-2 Physical topology of Cilandak APartment



Page 35 of 122


» Depok Greenfield

Figure 1-3 Physical topology of Metro Depok Greenfield

» Bogor Situ Cikaret Greenfield

Figure 1-4 Physical topology of Bogor Situ Cikaret Greenfield



Page 36 of 122


1.6.7 Ring-5 Sites» Bekasi Villa Nusa Indah

Figure 1-1 Physical topology of Bekasi Villa Nusa Indah

» Bekasi Mulia Industri

Figure 1-2 Physical topology of Bekasi Mulia Industri

» Bekasi Trade Center



Page 37 of 122


Figure 1-3 Physical topology of Bekasi Trade Center

<Please refer to Annex “Physical Connection of IP RAN Jabodetabek”>



Page 38 of 122


2 Device Naming

2.1 Device Name Design

The naming will follow existing rule.

The IP Core equipment naming convention will consist of 8 characters. The first and the second characters, an abbreviation to reflect the Equipment Type, the third and the fourth characters will represent the province code, the fifth and the sixth characters will be an abbreviation of site location which the equipment to be placed. And the remaining 2 character will represent the numbering sequence of the IP Core equipment. The numbering will start from 01 through to 99.

IP Core Equipment ID1 2 3 4 5 6 7 8

Device typeProvince

CodeSite Location

CodeAscending

number 01-99

Table 2-1 IP Core Equipment Naming Rule

Character

Code

Network NE Type Definition

RN IP RAN CX600

SW Switch Switch / Catalyst

PE Provider Edge Provider Edge Router

Table 2-2 IP Core Type Rule



Page 39 of 122


No Device

Site Name Province Code

Site Location

Code

ID NE Name

1 RN Citra Graha 1 JKT CGH01

RNJKTCGH01

2 RN Citra Graha 2 JKT CGH02 RNJKTCGH02

3 RN Jakarta Menara Dea 1 JKT MAX01

RNJKTMAX01

4 RN Jakarta Menara Dea 2 JKT MAX02 RNJKTMAX02

5 RN Jakarta Menara Peninsula JKT MPL01 RNJKTMPL01

6 RN Jakarta Gedung Total JKT GTT01 RNJKTGTT01

7 RNJakarta Menara Bank Dagang

Negara JKT BDN01 RNJKTBDN01

8 RN Jakarta APart. Puri ImPerium JKT API01 RNJKTAPI01

9 RN Jakarta Manggala Wanabakti JKT MWB01

RNJKTMWB01

10 RN Jakarta Wisma Indovision JKT WIV01

RNJKTWIV01

11 RN Jakarta Royal Tower Riverside 1 JKT RTR01

RNJKTRTR01

12 RN Jakarta Royal Tower Riverside 2 JKT RTR02

RNJKTRTR02

13 RN Jakarta Gajah Mada Tower JKT GMT01

RNJKTGMT01

14 RN Jakarta Sunter Mall JKT SML01

RNJKTSML01

15 RN Jakarta CemPaka Putih JKT CPT01

RNJKTCPT01

16 RN Jakarta Graha Amaba JKT GAB01

RNJKTGAB01

17 RN Tangerang German Center TNG GCT01

RNTNGGCT01

18 RN Tangerang Condo Golf Karawaci TNG GKR01

RNTNGGKR01

19 RN Tangerang Gondrong 1 TNG GDR01

RNTNGGDR01

20 RN Tangerang Gondrong 2 TNG GDR02

RNTNGGDR02

21 RN Serang Banten TV 1 SRG BTV01

RNSRGBTV01

22 RN Serang Banten TV 2 SRG BTV02

RNSRGBTV02

23 RN Jakarta Metro Pondok Indah JKT MPI01

RNJKTMPI01

24 RN Jakarta Cilandak APartment JKT CLD01

RNJKTCLD01

25 RN Depok Greenfield 1 DEP GFD01

RNDEPGFD01

26 RN Depok Greenfield 2 DEP GFD02

RNDEPGFD02

27 RN Bogor Situ Cikaret Greenfield 1 BGR SCG 0 RNBGRSCG01



Page 40 of 122


1

28 RN Bogor Situ Cikaret Greenfield 2 BGR SCG02

RNBGRSCG02

29 RN Bekasi Villa Nusa Indah BKS VNI01

RNBKSVNI01

30 RN Bekasi Mulia Industri BKS MID01

RNBKSMID01

30 RN Bekasi Trade Center 1 BKS BTC01

RNBKSBTC01

31 RN Bekasi Trade Center 2 BKS BTC02

RNBKSBTC02

Table 2-3 Sites ID and Naming Abbreviation

2.2 Interface Description DesignEach configuration should have description, such as Interface, vlan, protocol.

The principle of Interface Description is:

Link to Device-name interface-number

For 10GE link description will mention use capital word (GE1/0/0)

For GE link description will mention use small latter (ge3/0/0)

For example: Link to MEJKMA01 GE1/0/0

<Please refer to Annex “Layer-2 and Layer-3 Connection chart”>



Page 41 of 122


3 VLAN Design

3.1 OverviewIP RAN will deliver 2G and 3G service from BTS to BSC and Node B to RNC.

Connection for One BTS/Node B to BSC/RNC will be L2. CX600 established VLL VPN tunnel for pass the vlan.

VLL Tunnel will be barrier Abis and Iub service, for BTS will use 1 vlan and for Node B will use 2 vlan. Each BSC/RNC including BTS/Node B should less than 250.

Since BTS oam will together with Service use same VLAN, so we don’t consider OAM VLAN for BTS.

3.2 VLAN Design Regulation

3.2.1 Node B OAM VLANWe design Node B OAM use one same VLAN if it until same GNE RTN (Hub RTN).

And we use VLL transfer Node B OAM to PE, each GNE can repeat use same VLAN, so we use the VLAN range 4075-4079 in RTN to CX side for Node B OAM.

Each ring of Node B OAM will go to same PE which in this ring, so we design the VLAN is 101-199 for RTN OAM in PE side.

3.2.2 RTN OAM VANWe design RTN OAM use same physical interface with Service.

Since one GNE (Hub RTN) just use one OAM IP address and one VLAN, we use VLL transfer RTN OAM to PE, and each GNE can repeat use same VLAN, so we use the VLAN range 4080-4094 in RTN to CX side for RTN OAM.



Page 42 of 122


All GNE RTN of one ring will connect to same PE which in this ring, so we design the VLAN is 201-299 for RTN OAM in CX to PE link.

3.2.3 OSN OAM VLANThere have one Ring haven’t PE sites, so OSN GNE will connect CX and pass through VLL connect to DEA PE. And we design use VLAN range 4070-4074 for OSN OAM in OSN-CX link, and same VLAN in CX-PE link.

3.2.4 Special CaseTo minimize broadcast domain, 2G and 3G services belong to different microwave arms will carry different VLAN IDs. We use different VLAN for each arm (even we can repeat use the VLAN in different GNE RTN, but in order to easy manage VLAN, we use different VLAN for each arm, until all VLAN deplenish, then we can consider repeat use VLAN range.)

In one microwave hub, it can have several numbers of arms. Different arms will have different number of BTS/NodeB and each BTS/NodeB can be connected to the same or different BSC/RNC. Diagram below shows an example of a typical microwave network.



Page 43 of 122


Figure 3-1 Typical Microwave Network

Besides that, some of the arms will connect to a large number of BTS/Node B (> 20 sites). For these kinds of arms, two VLANs will be used to minimize the broadcast domain.

Figure 3-2 Example for one Arm have BTS belong to different BSC

3.3 VLAN and VLL Planning (Access)› Each BTS/Node B have different VLAN, BSC/RNC use each VLAN to differentiate

BTS/Node B.

› In order to easy to know and maintenance network, we design VLAN and IP

address of BTS/Node B, and VC ID of VLL as follow.

› VLAN Range can repeat use for each BSC/RNC.



Page 44 of 122


Vlan and VLL planning will have some rule like below:

VLAN & IP Address of BTS/Node B and VC ID of VLL Design

Device

VLAN Range

Arm of RTN

VLAN for BTS/Node B

VC ID of VLL (master)

VC ID of VLL (slave)

Remark1nn1001-1nn20001mm2001-1mm3000

2nn1001-2nn20002mm2001-2mm3000

BSC 0002-2000

Arm-1-1 1001 1011001 2011001 example for BSC1



Arm-n 0002-2000

RNC 2001-4000




Arm-n 2001-4000

OAM 4001-4094

OSN 4070-4074

1014070 2014070 Use 4070 first

Node B 4075-4079

1014075 2014075 Use 4075 first

GNE RTN 4080-4094

1014094 2014094 Use 4094 first

Comments: 1. Generally, each Arm of RTN (hub) will have one VLAN for BTS and one VLAN for Node B.2. If the amounts of BTS or Node B are more than 20, then we will separate Arm-x-y, each sub-

Arm have one VLAN.3. Under the same microwave hub, if BTS which under the same arm are connected to different

BSC, these BTS under different BSC need to use different VLAN but these VLAN can be repeated at different arm.

4. Under the same microwave hub, if Node B which under the same arm are connected to different RNC, these Node B under different RNC need to use different VLAN but these VLAN can be repeated at different arm.

5. Under the same GNE RTN, the OAM VLAN of RTN wills same, use VLAN range 4075-4079, but in PE side will use a range 201-299 (this VLAN can repeat use, it is different with above VLAN design.)

6. Under the same GNE RTN, the OAM VLAN of Node B wills same, use VLAN range 4080-4094, but in PE side will use a range 101-199 (this VLAN can repeat use, it is different with above VLAN design.)

7. nn is the number of BSC8. mm is the number of RNC9. x is Arm number, y is sub-Arm number.



Page 45 of 122


Table 3-1 VLAN/IP/VC ID Design

Metro E will have connection to access NE (BSC, BTS, RNC and Node B) though GE interface with detail allocation like below:

<Please refer to Annex “VLAN & VLL VC ID Design”>



Page 46 of 122


4 IP Address Design

4.1 OverviewIP RAN will deliver 2G and 3G service from BTS to BSC and Node B to

RNC. Core connection between each metro will use IGP and L3 connection with MPLS protocol.

4.2 Detail IP Address TableIP for Metro E MPLS will use ip segment that provide by Axis team, ip

block that we use is X.X.X.X/YY. For IP B2B we will use /30.

Loopback IP use: x.x.x.x/32

B2B IP block use: x.x.x.x/31

For Detail please refer:

<Please refer to Annex “IP Address Design”>

4.3 IP Address for OAM1. RTN OAM ip address PER HUB RTN (GNE) use /29

VLAN Allocation use VLAN range: 4080-4094. In PE router, 1 GEN RTN will use one unique VLAN.

Below is the ip allocation for above data RTN OAM:



Page 47 of 122


a. RTN OAM Jabodetabek : 10.160.0.0/22 (please breakdown into /29 per hub RTN)

b. RTN OAM WJ :use existing spare network 10.129.0.0/16 (please breakdown into /29 per hub RTN)

c. RTN OAM CJ : use existing spare network 10.131.0.0/16 (please breakdown into /29 per hub RTN)

d. RTN OAM EJ : use existing spare network 10.130.0.0/16 (please breakdown into /29 per hub RTN)

e. RTN Balom : use existing spare network 10.140.0.0/16 (please breakdown into /29 per hub RTN)

2. BTS ip address per BSC use /23

VLAN Allocation use per ARM/VLAN

3. Node B service data ip address per RNC /23

VLAN Allocation use per ARM/VLAN

4. Node B OAM ip address per HUB RTN /25

VLAN Allocation use VLAN range 4075-4079. In PE router, 1 GNE RTN will use one unique VLAN



Page 48 of 122


5 Routing Planning

5.1 Overview Axis IP RAN network will run IP MPLS with L2 VPN and IGP as dynamic path selection of VLL.

Figure 5-1 OSPF Area

5.2 IGP PlanningIn IP Core network, OSPF is widely used. The advantage of OSPF is listed as below:

OSPF is more flexible as basing on interface supports various network

types and more mature, ISIS is better constructed and stable but only in



Page 49 of 122


one area, cannot support NBMA P2MP directly.

Technicians are more familiar with OSPF than IS-IS.

All Loopback addresses and interconnection segment should be advertised by

OSPF.

» Use multi-process OSPF in IP RAN to separate the routing domain if need.

» All CX600 can be designed in area 0 when CX600 amount is less 100 in one OSPF process.

» When RSG need communicate with IP Core, just import PE loopback IP to IP RAN OSPF process.

Some parameters need to set like below:

All Loopback addresses and point to point segment should be advertised by OSPF.

Each router should set loopback IP address which is taken from the same IP address pool allocation.

One OSPF process each IP RAN Network Regional.

Network Region OSPF Process OSPF Area

IP Core Whole Region 1 0

IP RAN Jabodetabek Jakarta 100 0

IP RAN Surabaya East Java 200 0

IP RAN Bandung West Java 300 0

IP RAN Yogya Central Java 400 0

IP RAN Malang Central Java 401 0

5.2.1 OSPF Process 100 use in Jabodetabek1. OSPF Process 100 will use for:

a. Interface between Backbone node

b. Router ID backbone node

2. Each router should set loopback IP address which is taken from the same IP

address pool allocation.



Page 50 of 122


3. Process-id use 100

4. Area use 0

5.2.2 Link type DesignAll the link type is P2P.

5.2.3 Cost Design› OSPF Cost requirement:

› 100G=1, 10G=10, 1G=100

› The calculation formula is as follows: cost of the interface = bandwidth

reference value/interface bandwidth. If the result is smaller than 1, the cost

value is 1.

› By default, the bandwidth reference value is 100<M> of an OSPF process for

an OSPF interface without a set cost.

› So, we can configure the bandwidth reference value is: 100,000

ospf 1

bandwidth-reference 100,000



Page 51 of 122


6 MPLS Planning

6.1 OverviewIn MPLS IP Core network, MPLS is used to transport L2 network though

L3 network. MPLS was first developed to help increase the forwarding

speed. Its architecture consists of:

Control plane

It is connectionless and implemented through the current IP network.

Forwarding plane

Also known as data plane, it is connection-oriented and can make use

of the Layer 2 network such as ATM network.

MPLS uses a short label of fixed length to encapsulate packets. Data

with the label is fast forwarded on the data plane. The powerful, flexible

routing function of the IP network is used on the control plane to meet the

demands of new applications for the network.

LDP is the control protocol of MPLS. It is similar to the signaling

protocol in the traditional network. LDP is in charge of packet

classification, label distribution, LSP establishment and maintenance.

Some Parameter need to set on MPLS network:

LDP is designed to be used in the bearer network. LDP should be

enabling at each interface that connect to another router.

Lsr-ID is a unique identifier of each MPLS routers. It is used to establish

the relationship with other MPLS routers. Lsr-ID is defined with

Loopback 0 address of each router.



Page 52 of 122


6.2 MPLS RSVP TE DesignL2 VPN will use PWE3 VLL for IP RAN network. The PWE3 mode

implements the L2VPN by setting up a point-to-point link. It takes LDP as

the signaling protocol to transfer Layer 2 information and VC labels. The

PWE3 VLL adopts VC-type plus VC-ID to identify a VC between RTN and

BSC/RNC.

Figure 6-1 MPLS RSVP-TE

MPLS lsr-id design

Lsr-ID is a unique identifier of each MPLS routers. It is used to establish the

relationship with other MPLS routers. Lsr-ID is defined with Loopback 0 address of

each router.

Lsp-trigger method design

In order to avoid the existence of unnecessary LSP, Lsp-trigger method is designed to be “host”. That means, only loopback addresses in MPLS IP RAN network can trigger the LSP establishment.



Page 53 of 122


6.3 TE Hot-standby parameter

TE Hot-Standby + BFD Design Rule Example

Source CX Name CX600-A RNJKTCGH01

Destination CX Name CX600-N RNJKTMAX01

Source Loopback0 ww.xx.yy.zz 10.0.12.4

Destination Loopback0 aa.bb.cc.dd 10.0.12.5

Tunnel Interface Tunnel0/0/ccdd Tunnel0/0/1205

Tunnel ID ccdd ccdd

Explicit-Path NameMaster to_cx600-n_m to_rnjktmax01_m

Slave to_cx600-n_s to_rnjktmax01_s

Tunnel-Policy Nmae to_cx600-n to_rnjktmax01

BFD NameMaster to_cx600-n_m to_rnjktmax01_m

Slave to_cx600-n_s to_rnjktmax01_s



Page 54 of 122


7 MPLS L2VPN Design

7.1 Overview The BGP/MPLS IP VPN is a PE-based L2VPN technology in the Provider

Provisioned VPN VLL. It uses MP-BGP to advertise the VPN routes and MPLS to

forward the VPN packets on the provider backbone network. The BGP/MPLS IP

VPN has flexible networking modes, good extensibility and convenient support

for the MPLS QoS. Hence, it is widely used.

Figure 7-1 VLL example

The BGP/MPLS IP VPN model contains the following parts:

Customer Edge (CE): is an edge device in the customer network. It has

one or more interfaces directly connected with the service provider network. It

can be a router, a switch or a host. Mostly, the CE cannot “sense” the existence

of the VPN, and does not need to support MPLS. In the bearer network, the



Page 55 of 122


devices of core network and intelligence network will work as CE. RTN will

work as CE.

Provider Edge (PE): is an edge device of the provider network. It is directly

connected to the CE. In the MPLS network, the PE router disposes all the VPN

processing. The CX600 device is the PE equipment in Digitel bearer network.

Traditional VPNs are based on Asynchronous Transfer Mode (ATM) or Frame

Relay (FR) where different VPNs can share the network structure of carriers.

Traditional VPNs have the Following disadvantages:

2. Dependence on special media (such as ATM or FR): The carriers must

establish ATM networks or FR networks for ATM-based or FR-based VPNs

across the country. This is a waste of network construction.

3. Complicated VPN structure: when a site is added to an existing VPN, it is

necessary to modify the configuration of all the edge nodes that access the

VPN site.

To avoid the preceding disadvantages, new solutions are introduced. Virtual

Leased Line (VLL) Based on Multiprotocol Label Switching (MPLS) L2VPN is one

of the solutions. The VLL provides Layer 2 VPN services on the MPLS network. It

allows the establishment of L2VPNs on different media including VLAN, Ethernet

and PPP. At the same time, the MPLS Network provides traditional IP services,

MPLS L3VPN, traffic engineering and QoS.

The VLL transfers Layer 2 data of the user transparently on the MPLS network.

The MPLS Network is a Layer 2 switching network used to establish Layer 2

connections between nodes. Consider ATM as an example. Configure an ATM

virtual circuit for each Customer Edge device (CE) to communicate with another

CE device through the MPLS network, similar to that through the ATM network.

7.2 Phase 1: BSC/RNC Sites is Single CX600We will propose use PWE3 VLL for IP RAN network. The PWE3 mode implements

the L2VPN by setting up a point-to-point link. It takes LDP as the signaling

protocol to transfer Layer 2 information and VC labels. The PWE3 VLL adopts

VC-type plus VC-ID to identify a VC between RTN and BSC/RNC.



Page 56 of 122


› VLL tunnel to bears Abis and Iub service.

› Generally, 1 RTN Arm 1 VLAN.

› Usually, Each BSC/RNC including BTS/Node B less than 400.

› CX600 support VLAN 4094.

Figure 7-1 example for VLL work mode (Single CX)

In Phase 1, since BSC/RNC sites just have one CX600, so we just can establish

one VLL for each VLAN service.

7.3 Phase 2: BSC/RNC Sites is Daul CX600

7.3.1 Scenario for BTS/Node B to BSC/RNC

For the BTS/Node B to BSC/RNC, there have 3 Scenario:

(i) Scenario 1: BTS/Node B and RTN communicate with Remote BSC/RNC.



Page 57 of 122


Figure 7-1 BTS/Node B (RTN) going to Remote BSC/RNC

(ii) Scenario 2: RTN and BSC/RNC under same CX600.

Figure 7-2 BTS/Node B (RTN) going to Local BSC/RNC



Page 58 of 122


(iii) Scenario 3: BTS/NodeB connect to CX directly to communicate with local BSC/RNC

Figure 7-3 BTS/NodeB going to Local BSC/RNC Directly

7.3.2 PW Redundancy designFor Scenario1 we can use PW Redundancy to backup the service:

Figure 7-1 example for VLL work mode (Dual CX)



Page 59 of 122


Figure 7-2 RSVP-TE + VLL Redundancy + MC-LAG

7.3.3 VPLS design for RTN and BSC/RNC in same siteFor Scenario2, 3: we use VPLS:

• Master RSG use native Ethernet VSI to band RTN AC sub-interface、slave RSG sub-interface and RNC master port sub-interface

• Slave RSG use native Ethernet VSI to band master RSG sub-interface and RNC slave port sub-interface

• When BSC/RNC switchover, VSI withdraws and relearns MAC address.



Page 60 of 122


Figure 7-1 RSVP-TE + VLL Redundancy + MC-LAG



Page 61 of 122


8 Reliability Planning

IP bearer backbone network should have very high reliability and availability in order to support the services. The network reliability depends on multiple factors, mainly including reliability of equipment, redundancy of design, link availability and protection mechanism of protocol layout and address layout. The network robustness can be optimized by means of fully usage of the software/hardware characteristics of the equipment, as well as the full meshed connection, partial mesh connection and dual-home in the network topology combining with the dynamic route protocol.

8.1 LSP 1:1 Protection in IP RANIn IP RAN network, we use MPLS LDP to backup the lsp, and enable ospf fast

convergence and bfd for ospf to quicken lsp switchover.

8.2 OSPF Fast Convergence

By default, the interval for originating LSA is 5 seconds.

› The OSPF protocol defines that the interval for originating LSAs is 5

seconds. The setting is expected to prevent network connections or

routing flaps from using excessive bandwidth and router resources.

› In the network environment where the network is stable and the fast

convergence is required, you can cancel the interval for receiving LSAs by

setting the interval to 0. Thus, changes to the topology and routes can be

advertised to the network in time, and routing convergence speeds up in



Page 62 of 122


the network.

› By default, the interval for receiving the LSA is 1 second.

› In a stable network, if the speed of route convergence is required to be

fast, you can cancel the interval of receiving LSA by setting it to 0.

Routers can thus feel changes of topology and route in time. This speeds

up route convergence.

› When the LSDB of OSPF changes, SPF is recalculated. Calculating the

shortest path when a change occurs consumes excessive resources and

affects the performance of the router.

› Adjusting the SPF calculation interval, however, can restrain the resource

consumption caused by frequent network changes. We suggest setting is

1000ms.

ospf 1

spf-schedule-interval millisecond 1000

lsa-originate-interval 0

lsa-arrival-interval 0

8.3 BFD Detect multiple-hop

• 1. BFD for OSPF to detect multiple hop link state.

• 2. Since the IP RAN use DWDM, the transmission quality should more stable, we

can set the BFD value as follow:

detect-multiplier 3

min-tx-interval 100

min-rx-interval 100

Figure 8-1 BFD for non-direct CX connection



Page 63 of 122


8.4 RTN to CX600 LAG Protection

• RTN and CX600 bundle two GE interface as a logical interface (Eth-Trunk)

• RTN and CX600 LAG (Eth-Trunk) should use same mode, here use 1:1

master/slave mode.

Figure 8-1 Physical connection

Figure 8-2 Logical connection



Page 64 of 122


Figure 8-3 One Link down

8.5 Phase1: BSC/RNC to CX – Single CX

BSC/RNC use Master/Slave mode, and CX600 bundle two GE interface as a logical

interface (Eth-Trunk)




Page 65 of 122


Figure 8-2 Logical Connection

Figure 8-3 One link down

8.6 Phase2: BSC/RNC to CX – Dual CX (PW redundancy)



Page 66 of 122


• PW redundancy use master/slave mode with non-revert, and keep consistent

with RNC/BSC no-share manual LAG.

• PW BFD detection timer propos to be long period 50ms.

• When link failure occur, TE Hot Standby will switchover first before PW

redundancy

• Configure BFD for MC-LAG




Page 67 of 122


Figure 8-2 Logical connection

How to configure bfd for MC-LAG

BFD for MC-LAG

# bfd hello bind peer-ip <loopback> source-ip <CX2 loopback> discriminator local <n>discriminator remote <m> commit # e-trunk 1 priority 10 peer-address <loopback> source-address <CX2 loopback> timer hello 9 e-trunk track bfd-session <n>#



Page 68 of 122


8.6.1 Scenario 1: Failure in IP RAN

Figure 8-1 IP RAN 10G down

Recovery in IP RAN



Page 69 of 122

Action on CX600-3:

Detect the Primary PW failure

Switch to PW2

Action on CX600-1 Master:

Detect the primary PW failure

Bridge the MC-LAG sub-interface with ICB PW

Action on CX600-2 Slave:

Do nothing


Figure 8-2 IP RAN Recovery

8.6.2 Failure of CX600-1



Page 70 of 122

Action on CX600-3:

PW1 is UP

Swap to PW1


PW1 is UP

Swap to PW1


Do nothing

CX600-3 can receive data from two PWs， in this scenario LDP notification will not affect CX600-3’s status.

Action on CX600-3:

Detect the Primary PW failure

Switch to PW2

PW2 status become master


Do nothing

Action on CX600-2 Slave (New Master):

MC-LAG BFD detect the failure by BFD, turns to MC-LAG master

BFD for ICB admin PW detect the failure

Send LDP notification to CX600-3


Figure 8-1 CX Down recovery

Recovery of CX600-1

Figure 8-2 CX router recovery



Page 71 of 122

Action on CX600-3:

PW1 is UP

PW1 become master

Swap to PW1


CX600-1 change to MC-LAG master wait

ICB PW is UP, PW1 is UP

CX600-1 change to MC-LAG master

Send LDP Active Message to CX600-3


Do nothing， just receive UDP message to switch to backup

This scenario should BSC/RNC support LACP, if cannot, then BSC/RNC master interface doesn’t


8.6.3 Scenario 3: Failure between CX600-1 and BSC/RNC

Figure 8-1 Access link of BSC/RNC down

Recovery in IP RAN



Page 72 of 122

Action on CX600-3:

Receive LDP notification

Switch to PW2

PW2 status become master

Action on CX600 -1 master:

Become MC-LAG Slave

Bridge remote PW with ICB PW

Bridge ICB PW with remote PW

Action on CX600-2 Slave (New Master):

MC-LAG turns to master

Bridge PW2 with MC-LAG sub-interface

Send LDP notification to ATN make PW2 to be master


Figure 8-2 Access link of BSC/RNC recovery



Page 73 of 122

Action on CX600-3:

Receive LDP notification

PW1 become master

Swap to PW1


CX600-1 change to MC-LAG master

Send LDP Active Message to ATN


Do nothing， just receive UDP message to switch to backup


9 QoS Planning

Detailed please refer to the LLD of E2E QoS solution.

9.1 Overview

In the mobile carrier solution, QoS requirements vary with traffic typess. On the

carrier network, DiffServ and HQoS must be deployed to ensure proper service

quality.

A carrier network mainly carries the following types of traffic:

» Voice, video, and data traffic originating from end users

» Message and OM traffic originating from radio and core

networks

» Protocol packets at the control layer

» Pass-through traffic

› QoS in Axis Network will be using Differentiated service (DiffServ) model.

› Appropriate DiffServ (DS) Domains will be created based on services at

BTS/Node B. Service category at BTS/Node B can be allocated with VLAN

priorities or DSCP values.

› The following tables show service priorities of 2G and 3G services. These

priorities are defined by BTS/BSC and Node B/RNC



Page 74 of 122


Figure 9-1 QoS solution

9.2 IP RAN MPLS Network QoS design

QoS Class

IP-Preceden

ce/EXP

DSCPCharacteristi

c of QoS Class

ServiceTraffic

shapingRemarks Service Description

CS7 7

Highest Priority of all class for IP Signaling

Protocol PQ Y.1731

CS6 6

Highest Priority of all class for IP Signaling

Protocol PQNetwork

Control / IP Protocol

Protocol packet

EF 5

48 Stright High Priority Class

with low delay and latency

PS Signaling PQ (limit

to 66% or 85% of total traffic)

2G Signaling

Voice 2G Voice call

46IMS SRB 3G

Signaling radio bearer

AMR 3GAdaptive Multi-

RateAF4 4 38 High Priority PS WFQ 2G/3G Video call



Page 75 of 122

PRIEXP EXP EXPEXP EXPPRI DSCP

EXP EXP EXPEXP EXPDSCP

VoIPVODBTV

HSI

VoIPVODBTV

HSI

CS7CS6EFAF4AF3AF2AF1BE

CS7CS6EFAF4AF3AF2AF1BE

PQ

WFQ

SPPWRRSPP

FEBTS/

NodeB

Microwave GE GE

RTN900

BTS/NodeB

FE

CX 600

CX 600CX 600

CX 600

RNC

10GE

10GE

10GEBSCCX 600


Class with low delay

and latency

Conversational

(18%)PS Streaming

2G/3G PPStream

34 O&M 2G/3G O&M

AF3 3High Priority Class with low delay

Reserved WFQ -

AF2 2

38

Low Priority Class for

High Data

PS High PRI

Interactive

WFQ (73%)

2GInteractive games,

IE

PS Middle PRI

Interactive


IE

PS Low PRI

Interactive


IE

PS Backgrou

nd2G

Email/Film/MP3 downloading, FTP

service

22

PS/HSDPA High PRI Interactiv

e


IE

PS/HSDPA Middle PRI Interactiv

e


IE

PS/HSDPA Low PRI

Interactive


IE

PS Backgrou

nd3G

Email/Film/MP3 downloading, FTP

service

AF1 1Low Priority

Class for Medium Data

Reserved WFQ -

BE 0

Best effort class for unknown

traffic

ReservedWFQ(9 %)

-

Table 9-1 IP RAN QoS Classification guideline



Page 76 of 122


Based on the information from RNP, 85% of 2G service is voice service and remaining 15% is PS service. Within this PS service, 20% is PS conversational and streaming service and remaining 80% is PS interactive and background service. (Detail please refer Table 5).

PHB Service RemarksPercentage

of bandwidth (%)

Bandwidth (Mbps)

EFPS Signaling 2G

85 1.7Voice 2G

AF4PS Conversational 2G

3 0.06PS Streaming 2G

AF2

PS High PRI Interactive 2G

12 0.24PS Middle PRI Interactive 2G

PS Low PRI Interactive 2GPS Background 2G

Total 100 2

Table 5: 2G Services in Percentage (%)

For 3G service, 30% is AMR service and remaining 70% is PS service. Within this PS service, 20% is PS conversational and streaming service and remaining 80% is PS/HSDPA interactive and background service. (Detail please refer Table 6)

PHB Service RemarksPercentage

of bandwidth (%)

Bandwidth (Mbps)

EFPS Signaling 3G

30 3.3Voice 3G

AF4PS Conversational 3G

14 1.54PS Streaming 3G

AF2

PS/HSDPA High PRI Interactive

3G

56 6.16PS/HSDPA Middle PRI

Interactive3G

PS/HSDPA Low PRI Interactive

3G

PS Background 3GTotal 100 11

Table 6: 3G Services in Percentage (%)

Based on this data, appropriate weight used in AF4 and AF2 in WRR queue can be calculated.



Page 77 of 122


AF4 = (2G AF4 + 3G AF4) / (2G/3G AF4 and 2G/3G AF2) * 100 = (0.06 + 1.54)/(0.06+1.54+0.24+6.16) * 100 = 20%

AF2 = (2G AF2 + 3G AF2) / (2G/3G AF4 and 2G/3G AF2) * 100 = (0.24 + 6.16)/(0.06+1.54+0.24+6.16) * 100 = 80%

From the calculation, although AF4 will consume 20% of the WRR queues (AF1, AF2, AF3 & AF4) and AF2 will consume 80% of the WRR queue for both 2G and 3G services, but since all untrusted packets will receive BE treatment, weight for BE traffic in WRR queue need to be included as well.

Below shows the new weight for WRR queue after considered BE traffic.AF4 = 20/110 x 100 = 18%AF2 = 80/110 x 100 = 73%BE = 10/110 x 100 = 9%

Table below shows the final weight for AF4, AF2 and BE in WRR queue.

Queue Scheduling AlgorithmPHB Service Class

Queue Scheduling Mode (RTN)

Queue Scheduling Mode(CX600)

CS7 SP PQ

CS6 SP PQ

EF SP PQ

AF4 WRR (weight = 18) WFQ (weight = 18)

AF3 WRR WFQ

AF2 WRR (weight = 73) WFQ (weight = 73)

AF1 WRR WFQ

BE WRR (weight = 9) WFQ (weight = 9)

Table 9-2 Queue scheduling algorithm

In order to prevent EF traffic from occupying all the bandwidth when network is congested, bandwidth for EF traffic in SP queue will be shaped to certain limit. Based on the information from IP core, EF traffic will be limited to 66% of the total bandwidth. To play at the safe side, for 2G+3G area, EF traffic will be limited to 66% while EF traffic at 2G only area and area that mixed with 2G and 3G, EF traffic will be limited to 85%.



Page 78 of 122




Page 79 of 122


10 Sync. Eth Clock Planning

10.1 Overview

10.1.1 Synchronization Requirements1. Clock source quantity and location should be Specified

2. All equipment on backhaul network support Sync Eth

3. The decline of clock signal quality should be considered:

4. Synchronization Ethernet: the maximum 20 nodes from end to end.

5. The BTS /NodeB could also run for one week normally if the clock source

shut down unexpected.

6. The clock source of working one and standby one will be finish exchange

within 1s .

Figure 10-1 Sync. Eth flow

In the case of the networks on the backbone layer and aggregation layer,



Page 80 of 122


clock protection should be configured, and the primary and secondary

reference clock sources must be set for active/standby clock switching.

The line clock source should be traced along the shorted path. In the case of

a ring network with five or fewer NEs, the reference clock source can be

traced from one direction. In the case of a ring network with six or more NEs,

the line clock should be traced along the shortest path. That is, when a

network consists of N NEs, N/2 NEs should trace the reference clock from

one direction and the other N/2 NEs trace the reference clock from another

direction.

In order to prevent clock trace loop, SSM protocol (standard) should be

enabled on every CX. When the SSM information is not configured, do not

configure the clock on the local NE into a ring.

The clock signals extracted from STM-N or synchronous Ethernet should be

used as the inter-office clock signals, instead of the tributary timing signals.

Retiming is preferred to adaptation as the synchronization solution for the

CES service.

10.1.2 Clock requirements of radio networks

Radio Mode Requirements for Frequency Synchronization

Requirements for Time Synchronization

GSM 0.05 ppm -

WCDMA 0.05 ppm -

TD-SCDMA 0.05 ppm 1.5 us

CDMA2000 0.05 ppm 3 us

WiMax FDD 0.05 ppm -

Wimax TDD 0.05 ppm 1 us

LTE FDD 0.05 ppm - (MBSFN requires 1 us.)

LTE TDD 0.05 ppm 1.5us

Table 10-1 Clock requirements of radio networks



Page 81 of 122


10.1.3 Phase/Frequency synchronization

Below is the drawing to explain what different between Phase and Frequency synchronizations.

Synchronize Ethernet is Frequency Synchronize.

Figure 10-1 Phase and Frequency synchronizations

10.1.4 Equipment support Sync. Eth StateBelow is the information for support status of each equipment in AXIS network.

Equipment support E2E Clock Status in AXIS Amber

Soft verssionSync. Eth 1588v2

Data of GA

Hardware

Software

Hardware

Software

CX600 V600R003C00 Y Y Y YOSN180

0V1R3 Y Y Y Y

RTN980 V1R3C02 Y Y Y N Q2orQ3, 2012

RTN950 V1R3C02 Y Y Y N Q2orQ3, 2012

BTS V100R003C00 Y Y Y Y

Node BV100R003C00

Y Y Y Y



Page 82 of 122

Freq_1

Freq_2

t t t t t

Freq_1

Freq_2


OSN3500

V1R11 Y Y Y YMSTP+

OSN6800

V1R5 Y Y Y Y

OSN8800

V1R5 Y Y Y Y

Table 10-1 Equipment support Sync. Eth Status



Page 83 of 122


10.1.5 Board and device support Sync.



Page 84 of 122


Produ

ct

Versio

n

Board Sync.

E

SSM BM

C

BC

CX600

x8

V6R3C0

0

CR53-LPUF-10 Flexible Card Line

Processing Unit(LPUF-

10,four slots)

NA NA NA NA

CR5M0E8GFA3

0

8-Port 100/1000Base-X-SFP

Flexible Card A(P10-

A,Supporting 1588v2)

Y Y Y Y

CR5MLPUF402

B

Flexible Card Line

Processing Unit(LPUF-40,2

sub-slots) B

NA NA NA NA

CR5M0L2XXA2

0

2-Port 10GBase LAN/WAN-

XFP Flexible Card A(P40-


Y Y Y Y

CX600

x3

V6R3C0

0



10,four slots)

NA NA NA NA

CR5M0E8GFA3

0


Flexible Card A(P10-


Y Y Y Y

CX61-LPUF-21-

B

Flexible Card Line


sub-slots) B

NA NA NA NA

CR52-P20-

1x10GBase

WAN/LAN-XFP-

A

1-Port 10GBase WAN/LAN-

XFP Flexible Card

A(Supporting 1588v2)

Y Y Y Y

NE40E

x8

V6R1C0

0SPCe0

0



10,four slots)

NA NA NA NA

CR53-P10-

8x100/1000Ba

se-X-SFP


Flexible Card

Y Y N N

CR5MLPUF402

A

Flexible Card Line


sub-slots) A

(L3VPN,MVPN,IPv6

Enhanced)

NA NA NA NA



Page 85 of 122


CR5M00L2XX2

0

2-Port 10GBase LAN/WAN-

XFP Flexible Card(P40)

Y Y N N

NE40E

x3

V6R1C0

0SPCe0

0



10,four slots)

NA NA NA NA

CR53-P10-

8x100/1000Ba

se-X-SFP


Flexible Card

Y Y N N

NE40E-

8

V3R3C0

0B697

CR52-4xPOS/

STM1-SFP

4*OC-3c/STM-1 POS-SFP

Optical Interface LPU G

N Y N N



10,four slots)

NA NA NA NA

CR53-P10-

8x100/1000Ba

se-X-SFP


Flexible Card

Y Y N N

CR52-LPUF-21-

A

Flexible Card Line


sub-slots) A

(L3VPN,MVPN,IPv6

Enhanced)

NA NA NA NA

CR52-P20-

1x10GBase

WAN/LAN-XFP

1-Port 10GBase WAN/LAN-

XFP Flexible Card

Y Y N N

Table 10-1 Board support Sync. Eth Status

10.1.6 Clock delay and jitter Sync. Eth transfer clock data based on physical signal, the Sync. E clock packet is transferred by electric/optical signal in Ethernet network, the mechanism is different with layer 2 or layer 3 protocols. In clock client side (BTS/Node B), it can make the clock frequency by Sync. Eth arithmetical, if there have jitter or delay impact the quality of network, firstly, the clock will be identified and filter by hardware, and adjust the frequency, After that it will compare new clock with existing clock, it will keep the existing clock. The Sync. Eth clock will traces clock after link recover.the BTS/Node B will keep clock 1 week.



Page 86 of 122


If deploy 1588v2 End to End (Hop by Hop), and the clock packet transfer based on per hop, then the network delay not impact the clock sync. But the Jitter should be less than 16ms.

If deploy 1588v2 ACR (there have other type of Sync. Between 1588v2 server and client), then the both delay and jitter will impact it. Delay should be less than , and Jitter should be less than 16ms.

Figure 10-1 1588 ACR

Requirements of 3rd party network

1.delay less than 100ms,jitter less than 16ms ,packet loss less than 0.5%.

2. Each 1588 ACR client requires 0.1Mbps bandwidth, that means 0.1Mbps * N bandwidth needed if there are N clients.

Once Sync.Eth or 1588v2 loss traces clock source , it will track the link status in time,1588v2 sync within 5 minutes after link recover . Sync.Eth sync within 1 minute.



Page 87 of 122

PRC

3rd party network1588ACR

1588ACR

Master

SlaveStep 1:MASTER device traces PRC and sends 1588ACR packets

Step 2:3rd party network transparently transmits 1588ACR packets sent by MASTER devices

Step 3:SLAVE device receives 1588ACR packets, then recovers frequency

Slave


10.1.7 Packet loss for clock For Sync. Eth and 1588v2 packet loss rate should be less than 5% .the clock will change to backup clock while packet loss.

　 Delay Requirement

Jitter Requirement

packet loss impact

recover time

Sync. Ethernet

no no <0.5% 1 minute

1588v2 no 16ms <0.5% 5 minutes1588v2 ACR 100ms 16ms <0.5% 5 minutes

Table 10-1 Clock requirements for Jitter, delay and Packet lost

10.1.8 How to calculate Sync. Eth hop

Figure 10-1 Example for Hop Calculate



Page 88 of 122


10.2 Sync. Eth Design

10.2.1 Two Clock Source redundancyWe design input two clock source into IP RAN Network to redundancy.

Figure 10-1 Sync. Eth Master/Slave Design Planning



Page 89 of 122


10.2.2 Primary Source DownIf the primary BITS down, the Sync. Eth clock will choose the secondary source.

Figure 10-1 Network models in AXIS Amber



Page 90 of 122


10.2.3 Link DownIf some link down in the IP RAN network, both clock we give the frequency to BTS/Node B.

Figure 10-1 Network models in AXIS Amber



Page 91 of 122


10.3 Sync. Eth Deploy

10.3.1 Configure Sync. Eth in CX600 The CX600 gets the 2Mbits clock on the primary MPU board BITS0 interface

from BITS master board as the first priority, and it will get the clock on the

standby MPU board BITS0 interface from BITS if switchover happens between

two MPU boards. For clock source redundancy, configure BITS secondary Board

which connects to the CX600 BITS1 interface on both MPU. The CX600 sends

the Synchronization Ethernet clock to IP RAN and RTN.

Every CX600 should configure primary clock and secondary clock, for example:

CX2 take the synchronization from CX1 as the first Clock Source; and take the

synchronization from CSG6 as the second Clock Source.

In case of primary clock source failure, the CX should switch to trace backup

clock source.



Page 92 of 122


10.3.2 Main Clock Trace Path in IP RAN Jabo

Figure 10-1 Main clock trace path

10.3.3 Sync. E Priority in IP RAN Jabo Sites Device ports Opposite priority

Core Site

Jakarta Citra Graha 1 RNJKTCGH01 GE1/0/0 DEA RNJKCG02 5

GE1/0/1 DEA RNJKMA01

GE8/0/0 To RTN 0(Nono Priority)


Jakarta Citra Graha 2 RNJKTCGH02 10



Jakarta Menara Dea 1 RNJKTMAX01 CLK0 5

10GE 1/0/0

10GE 1/0/1

10GE 1/1/0

10GE 1/1/1

10GE 2/0/0

10GE 2/0/1

10GE 2/1/0



Page 93 of 122




Jakarta Menara Dea 2 RNJKTMAX02 CLK0 10

10GE 1/0/0

10GE 1/0/1

10GE 1/1/0

10GE 1/1/1

10GE 2/0/0

10GE 2/0/1

10GE 2/1/0



Ring 1

Jakarta Menara Peninsula RNJKTMPL01 GE1/0/0 To DEA 5

GE1/0/1 To Gedung Total 10



Jakarta Gedung Total RNJKTGTT01 GE1/0/0 To DEA 5



GE3/0/1 To RTN 0(Nono Priority)Jakarta Menara Bank Dagang Negara RNJKTBDN01 GE1/0/0 To DEA 5




Jakarta APart. Puri ImPerium RNJKTAPI01 GE1/0/0 To DEA 5




Ring 2

Jakarta Manggala Wanabakti RNJKTMWB01 GE1/0/0 To DEA 5




Jakarta Wisma Indovision RNJKTWIV01 GE1/0/0 To DEA 5




Jakarta Royal Tower Riverside 1 RNJKTRTR01 GE1/0/0 To DEA 5




Jakarta Royal Tower Riverside 2 RNJKTRTR02 GE1/0/0 To DEA 5




Jakarta Gajah Mada Tower RNJKTGMT01 GE1/0/0 To DEA 5





Page 94 of 122



Jakarta Sunter Mall RNJKTSML01 GE1/0/0 To DEA 5




Jakarta CemPaka Putih RNJKTCPT01 GE1/0/0 To DEA 5




Jakarta Graha Amaba RNJKTGAB01 GE1/0/0 To DEA 5




Ring 3

Tangerang German Center RNTNGGCT01 GE1/0/0 To DEA 5




Tangerang Condo Golf Karawaci RNTNGGKR01 GE1/0/0 To DEA 5




Tangerang Gondrong 1 RNTNGGDR01 GE1/0/0 To DEA 5




Tangerang Gondrong 2 RNTNGGDR02 GE1/0/0 To DEA 5




Single Node

Serang Banten TV 1 RNSRGBTV01 GE1/0/0 To DEA 5




Serang Banten TV 2 RNSRGBTV02 GE1/0/0 To DEA 5




Ring 4

Jakarta Metro Pondok Indah RNJKTMPI01 GE1/0/0 To DEA 5




Jakarta Cilandak APartment RNJKTCLD01 GE1/0/0 To DEA 5




Depok Greenfield 1 RNDEPGFD01 GE1/0/0 To DEA 5

GE1/0/1 To Gedung 10



Page 95 of 122


Total GE3/0/0 To RTN 0(Nono Priority)


Depok Greenfield 2 RNDEPGFD02 GE1/0/0 To DEA 5




Bogor Situ Cikaret Greenfield 1 RNBGRSCG01 GE1/0/0 To DEA 5




Bogor Situ Cikaret Greenfield 2 RNBGRSCG02 GE1/0/0 To DEA 5




Ring 5

Bekasi Villa Nusa Indah RNBKSVNI01 GE1/0/0 To DEA 5




Bekasi Mulia Industri RNBKSMID01 GE1/0/0 To DEA 5




Bekasi Trade Center 1 RNBKSBTC01 GE1/0/0 To DEA 5




Bekasi Trade Center 2 RNBKSBTC02 GE1/0/0 To DEA 5




Table 10-1 Sync. Eth Clock Priority in IP RAN Jabo

10.3.4 Configure Sync. Eth Example



Page 96 of 122


Figure 10-1 Sync. Eth backup in CX

CX600-1 and CX600-2 get external 2Mbps clock source from OSN3500. CX600 transport Sync-E clock to RTN hub.The main clock link is following the blue direction.The protection clock link is the red direction.The CX ports priority plan is:

Device ports priorityCX600-1 CLK0 5　 g0/2/0 10CX600-2 g0/1/0 5　 g0/2/0 10　 g0/3/0 0　 g0/3/1 0CX600-3 g0/1/0 5　 g0/2/0 10CX600-4 g0/1/0 5　 CLK0 10

CX600-1:

clock ethernet-synchronization enable //Global enable SyncE

clock ssm-control on //Enable SSM protocol clock freq-deviation-detect enable //Enable freq-deviation-detectclock bits-type bits0 2mbps //Set Ext clock type to 2Mbpsclock source bits0 priority 5 //Set priority of Ext clock clock source bits0 ssm prc //option. No need to set for 2Mbpsclock source bits0 synchronization enable //Enable Ext clock SyncE

GigabitEthernet0/2/0



Page 97 of 122


clock priority 10 //Set port SyncE priority clock synchronization enable //Enable port SyncE

GigabitEthernet0/1/0 clock synchronization enable //Enable port SyncE

CX600-2:


clock ssm-control on //Enable SSM protocol clock freq-deviation-detect enable //Enable freq-deviation-detect

GigabitEthernet0/1/0 clock priority 5 //Set port SyncE priority clock synchronization enable //Enable port SyncE




CX600-3:


clock ssm-control on //Enable SSM protocol clock freq-deviation-detect enable //Enable freq-deviation-detect



CX600-4:


clock ssm-control on //Enable SSM protocol clock freq-deviation-detect enable //Enable freq-deviation-detectclock bits-type bits0 2mbps //Set Ext clock type to 2Mbpsclock source bits0 priority 10 //Set priority of Ext clock clock source bits0 ssm prc //only set for the first CXclock source bits0 ssm prc //option. No need to set for 2Mbps



Page 98 of 122


clock source bits0 synchronization enable //Enable Ext clock SyncE



System trace source State:

[CX600-1]dis clock source System trace source State: lock mode pull-in range Current system trace source: bits0 Current 2M-1 trace source: system PLL Current 2M-2 trace source: system PLL

Master board source Pri(sys/2m-1/2m-2) In-SSM Out-SSM State -------------------------------------------------------------------------- bits0 1 /---/--- prc dnu normal GigabitEthernet0/2/0 2 /---/--- dnu prc normal* GigabitEthernet0/1/0 --- /---/--- sec prc normal

Slave board source In-SSM Out-SSM State -------------------------------------------------------------------------- bits0 prc dnu normal

Remark:1. For priority of the Sync-E port and Ext clock, we use 5/10/15/20…..The low value has higher priority.2. For Trunk ports of CX , they must be set with the same priority.3. For LAG ports of RTN, we must set the main port and non-main port of LAG with different priority for Sync-E protection.



Page 99 of 122


11 Security Planning

11.1 Overview In the mobile carrier network, security is one of most important work. The IP RAN network must set security policy for proctection security of network and device:

1. The CX Router sets access policy for user login, it allow only legal user telnet and configuration.

2. The CX Route will authenticate when user was logging on and configing.

3. The route will encrypt in the IP RAN.

11.2 Access ControlHuawei suggests that it is necessary to config security policy for user access in

the CX Router. It will config acl filter base user IP network , and allow only legal user telnet Router. The acl is follow:

#

acl number 2000 description ** fore remote login user **

rule 5 permit source 10.0.0.0 0.0.31.255




rule 100 deny

#



Page 100 of 122


11.3 Tacacs Authentication

Figure 11-1 Remote login and control CX

It is necessary to authorize and authenticate when user do login and configuration the CX600 Router. Huawei recommends the tacacs solution for IP RAN network, because it is using in other AXIS network now. The detailed solution is follow:

1. When user input username and password to telnet CX Router, The CX Router forwards the username and password to Tacacs server for authentication. if the authentication is successful, the user can log on CX Router. Or the user can’t log on.

2. If user want to config the CX Router after login, It is necessary to pass authorization of Tacacs server. The tacacs server can authorize and authenticate each command of user. If the authorization is successful, the command of user can implement, or it can’t.

3. When user input the super command to modify user-level after telnet , the CX Router will verify password of user, it forwards the password of user to tacacs server for authentication. if the authentication is successful, the user can modify user-level, or it can’t.

4. The CX Router configs failure policy when the tacacs server is abnormal, it will use local authorization and authentication.

11.4 Route encrypt



Page 101 of 122


Figure 11-1 CX OSPF Route Authentication

All the CX Route will belong to same area in the IP RAN, and it is necessary to encrypt route for security. Huawei suggests that it sets up ospf neighbor in md5 algorithm between different CX Router, all Router config ospf authentication password . if the authentication is failure , the ospf neighbor can’t set up between router, and can’t get peer route.



Page 102 of 122


12 CX-PE Connection Planning

12.1 Overview› For future usage, Create a direct link between CX router to PE router in 6 sites

› Sites: CG, DEA, Pluit, Tangkarang, DePok Greenfield, Bekasi Trade

Center, and Banten TV (Serang).

12.2 Physical interface› Bundle two GE interface <optical> into Eth-Trunk

12.3 Protocol deploy› Just Configure back to back interface IP between CX600 and PE.

› Use Static Route for OAM service between this links.



Page 103 of 122


Figure 12-1 CX-PE

In PE Router:#ospf 1 area 0.0.0.0 network x.x.x.x x.x.x.x area 0.0.0.100 network x.x.x.x x.x.x.x #

In CX600 Router:#ospf 100 area 0.0.0.0 network x.x.x.x x.x.x.x network x.x.x.x x.x.x.x#

Between CX and PE use Static Route for each OAM service.



Page 104 of 122


13 Operation and Maintenance

Planning

13.1 OverviewFuture networks will be doubtlessly IP-based and more versatile mobile

data services will be available, which means increasingly complex network maintenance. A qualified operation and maintenance (O&M) system must handle more complex and flexible configurations and fault detection for multiple types of services while the operational expenditure (OpEx) is maintained at a reasonable level.

An IP-based mobile carrier network imposes the following challenges on O&M:

End-to-end service provisioning and assurance: IP-based carrier networks must allow for SDH/MSTP-like O&M and must be compatible with radio carrier networks for management synergy. To take this challenge, operators must improve the capabilities of managing end-to-end services and monitoring service quality.

Quick fault identification and performance management: The packet network makes network management invisible, which hinders fault identification and complicates performance management.

In the ATN(RTN)+CX solution, Huawei iManager U2000 can be deployed to provide an ideal solution to network deployment, service provisioning, and service quality assurance.

13.1.1 Network DeploymentOne-time site presence, free of software commissioning on site: Huawei ATN and CX600 devices can plug-and-play, which frees up software commissioning engineers or customers as the hardware installation engineers complete all operations. In other words, software commissioning on site is not required and deployment costs can be significantly reduced.



Page 105 of 122


Network topology auto-discovery and link auto-discovery

All NEs in a specified scope are automatically searched out and created on the U2000.

NEs are listed in the search result, which facilitates unified modification or maintenance on NE attributes.

All fibers in a specified scope are automatically searched out and created on the U2000.

Quick fiber search and creation: The fiber auto-search function helps quickly and accurately creates fibers, which facilitates network construction.

IP address auto-assignment for NNI-side ports

NNI-side IP addresses auto-assignment: NNI-side IP addresses can be automatically assigned while fibers are created. This simplifies network deployment.

Centralized management of NE IP addresses: IP addresses of networkwide devices are listed for easy maintenance.

Wizard, simplifying network adjustment

Network adjustment is feasible in scenarios where nodes are added on ring or chains, or where links are added.

Wizard with step-by-step operation guidelines helps improve efficiency as it shortens the learning curve of O&M engineers.

Assurance is provided for each procedure during network expansion. Network resources are checked prior to expansion; service rollback takes place in case of an expansion failure; a network comparison report is generated after expansion. This enables high reliability and easy management.

13.1.2 Service ProvisioningEnd-to-end service management and provisioning

End-to-end services can be created, simplifying service provisioning.

Services can be managed based on a profile, improving service provisioning efficiency.

Service running status and alarm status are displayed on the network topology, facilitating service status monitoring.

Real-time and historical service performance data can be collected, facilitating service status monitoring.

Quick provisioning of services in batches

Data about PWE3 services and protection for them can be imported to devices to quickly provision PWE3 services based on the service plan.

Data about tunnels and protection for them can be imported to devices to quickly provision tunnels based on the tunnel plan.

Visible provisioning of end-to-end services

Tunnels are visible.



Page 106 of 122


Tunnel protection relationships are visible.

Service topologies are visible and previewable.

Quick and accurate service auto-recovery: Services can be recovered by means of auto-discovery. NNI-side live services can be recovered to end-to-end service configurations in the system within two steps, which speeds up service provisioning.

Services can be automatically created after users select the source or sink and specify a route policy. This improves service provisioning efficiency while making few mistakes.

Services can be created in batches, enabling quick provisioning of a large number of services.

13.1.3 Service Quality AssuranceVisual management of end-to-end NodeB-RNC carrier paths: O&M engineers can obtain key device specifications based on the displayed paths and thereby quickly identify faults, improving O&M efficiency.

Service-based end-to-end OAM

All OAM configurations are completed in an end-to-end manner.

Default parameter settings are provided for one-click configurations of tunnel OAM. Wizard-like Ethernet OAM helps automatically generate remote MEPs.

Service-based fault diagnosis, quick fault identification

End-to-end service diagnosis enables quick fault identification.

Fault data is collected during diagnosis, which is helpful for problem analysis.

Visible networkwide clocks

The clock topology (IEEE 1588v2, synchronous Ethernet, and SDH clock) can be automatically discovered and networkwide clocks are displayed in a unified topology. In case of a network fault, the clock topology and clock synchronization status are updated in real time.

Clock status is monitored in real time, and clock alarms, clock tracing relationships, and clock protection status are displayed in real time.

The clock topology can be synchronized with the physical topology.

The master clock ID can be displayed.

Clock attributes can be queried.

Clock lockout status can be displayed.

Service-centered alarm monitoring

If a network fault affects service provisioning, alarms will be displayed on the end-to-end service management window.

In alarm management, the services that are affected by specific alarms can be easily identified.

When alarms occur, network wide alarms can be observed on the client or using a web browser.



Page 107 of 122


13.2 O&M and NMS Planning

13.2.1 NMS Overview

Devices should be managed via traffic in band.

In band management should be enabled on all the devices.

To achieve the NMS, address segment of OAM should be advertised by PE

router. Thus, NMS client will be able to reach every device.

The BTS, Node B, RTN980/950, OSN1800 and CX all be managed pass through

DCN (IP RAN and IP Core).

We allocate 1 NMS server U2000 in Jakarta CG. U2000 could manage every

network elements in IP Core and IP RAN network.

Figure 13-1 NMS System



Page 108 of 122


13.2.2 BTS O&M

BTS OAM will use the same IP address with Service, and pass through BSC to M2000. So we use same VLAN and same VLL for BTS oam.

Figure 13-1 BTS OAM Service Flow

13.2.3 Node B O&M

Node B OAM use different IP address with Service IP but same physical interface. So one Node B will have two different IP, one is for Service, another is for OAM.

OAM IP can communicate with M2000 directly (also can pass through RNC), but better communicate with M2000 directly, so we separate service IP and OAM IP into different VLAN.

For Node B service, one Arm one VLAN, each VLAN pass through VLL to RNC.

But for Node B OAM, we design one Hub RTN (GNE) one VLAN. And each VLAN pass through VLL to PE. (GW in PE Node B OAM sub-interface, OAM VPN)



Page 109 of 122


In Node B side, we use uniform VLAN range: 4075-4079.

In PE side, we design VLAN from 101 to 199 in one physical interface for all service which from CX to PE, including Node B OAM, RTN and some OSN OAM.

Figure 13-1 Node B OAM Service Flow

13.2.4 RTN O&MSeveral RTN chain will have a GNE (Hub RTN), one GNE just has one OAM IP address.

So for RTN OAM we use same solution with Node B.

One Hub RTN (GNE) has one VLAN for OAM. And each VLAN pass through VLL to PE. (GW in PE RTN OAM sub-interface, OAM VPN)

In RTN side, RTN use same physical interface for Service and OAM, we use uniform VLAN range: 4080-4094 (RTN default OAM VLAN is 4094)

In PE side, we design VLAN from 201 to 299 in one physical interface for all service which from CX to PE, including Node B OAM, RTN and some OSN OAM.



Page 110 of 122


Figure 13-1 RTN OAM Service Flow

Capacity and new U2000 hardware which use by RTN

Model Description Quantity

iManager U2000(V100R003)

Main Hardware Equipment

PC Server(For Windows OS)

NPCSERV09 Medium Scale PC Server ,4CPU*2.66GHZ or above,32G,6*300G 1

Optional Auxiliary Hardware Equipment

IM1B44N610E 1U/2U PC Server And Network Equipment Rack (2*AC220V) 1

Kehua FR-UK/B1110L2-E

UPS,10KVA,Online,Low Frequency,Long Delay Type,2 Hours,220V single phase,Chinese and English datasheet,Include battery,50Hz,CE certification

1

WM1P0CIKVM00

KVM 4 in 1 LCD Control Module,1U,17-Inch TFT LCD,8way KVM,With Mouse&Keyboard,110V/240V Self-adaptation,8 USB Straight signal cables,With mounting Accessories,English doc,Black

1

NMS Software

U2000 V1R5 1

Capacity

6000 equivalent NEs

13.2.5 OSN1800 O&M

Since 5 rings of OSN1800 all converge in OSN8800 of CG site, so we design the OSN8800 CG is the GNE to converge all of OSN1800 OAM.

But in order to enhance the DCN network of OSN stronger, we design each ring have a GNE as a backup GNE to converge each ring of OSN1800 OAM, if the OSN1800 OAM link which connect to OSN8800 down, then there will have a backup OAM GNE to manage each NE.



Page 111 of 122


OSN8800 CG will connect to SWJKCG21 and SWJKCG22 and to PEJKCG01/02 OAM VPN.

R2, R3, R4, R5 all have PE sites, so we choose the OSN which in PE sites as the Gateway NE, so OSN1800 OAM can connect to Switch and into PE OAM VPN.

R1 haven’t PE sites, so we connect OSN1800 OAM to CX600, and create a VLL connect to DEA PE.

In this OSN we use VLAN rang: 4070-4074, and in PE side we use uniform VLAN range design for all service, including Node B OAM, RTN and some OSN OAM in one physical interface.

Figure 13-1 OSN OAM Service Flow

13.2.6 CX600 O&M

1. CX600 use Loopback IP address for OAM.

2. We create a static route in CX600 DEA with high priority, destination is U2000 IP and next hop is PE CG. And create a static route in CX600 of 6 other



Page 112 of 122


BSC/RNC sites with low priority to backup OAM service, destination is U2000 IP and next hop is each local PE.

3. And we create static route in PE CG OAM VPAN with high priority, destination is CX600 Loopback IP. And create a static route in PE of 6 other BSC/RNC sites with low priority to backup OAM service, destination is CX600 Loopback IP and next hop is each local CX600.

Figure 13-1 CX600 OAM Service Flow

13.3 Y.1731 deployment in CX

13.3.1 Y.1731 overview

Y.1731 is an Operation, Administration and Maintenance (OAM) protocol defined by the ITUT. It is used to implement end-to-end connectivity detection, loopback detection, and link trace on Metro Ethernets (MEs). It also provides the test diagnosis and performance monitoring functions such as frame loss measurement, frame delay measurement, frame jitter measurement, and throughput measurement.



Page 113 of 122


Figure 13-1 Y.1731 overview

13.3.2 Basic Concepts and Principle

» Single-ended frame loss measurement

Frame loss measurement is performed by sending frames with ETH-LM information to a peer Maintenance association End Point (MEP) and receiving frames with ETH-LM information from the peer MEP. As shown in Figure 9-1, the process of single-ended frame loss measurement is as follows:

1. The local MEP sends an ETH-LMM (a frame containing ETH-LM request information) to the remote MEP. The ETH-LMM carries a transmit counter indicating the time at which the message is sent by the local end.

2. After receiving the ETH-LMM, the remote MEP replies with an ETH-LMR (a frame containing ETH-LM response information).

3. After receiving the ETH-LMR, the local MEP obtains corresponding measurement information based on message contents and calculates the frame loss ratio.



Page 114 of 122

10GEPrimary Ring BSC/RNC

GE

BTS/Node

B

GE/10GESecondary

Ring

GE

FE

y.1731 over MPLS

eth

Vll

LM/DM

Notes : LM/DM means loss measurement and delay measurement

CX

CX

CX

ATN

y.1731 over MPLS

eth

Vll

LM/DM

Option1:

Option2:

（ RTN+CX）

（ ATN+CX）


Figure 13-1 Networking diagram of single-ended frame loss measurement

» Dual-ended frame loss measurementFrame loss measurement is performed by sending frames with ETH-LM information to a peer MEP and receiving frames with ETH-LM information from the peer MEP. As shown in Figure 9-2, the process of dual-ended frame loss measurement is as follows:

1. Each MEP sends a frame containing ETH-LM request information to remote MEPs. Here, the frame containing ETH-LM request information is called a Continuity Check Message (CCM).

2. Each MEP processes the received CCMs and measures the number of frames lost on both the local and remote ends.

3. Each MEP obtains corresponding measurement information based on contents in the CCMs and calculates frame loss ratios.



Page 115 of 122


Figure 13-2 BTS Networking diagram of dual-ended frame loss measurement

» One-way frame delay measurementOne-way frame delay measurement is performed on end-to-end MEPs. A MEP receives a DM frame and returns a Delay Measurement Reply (DMR) to carry out the one-way frame delay measurement. As shown in Figure 9-3, the process of one-way frame delay measurement is as follows:

1. A MEP periodically sends DM frames carrying TxTimeStampf.

2. After receiving a DM frame, the remote MEP calculates the one-way frame delay based on the following formula:

Frame delay = RxTimef – TxTimeStampf

Figure 13-3 Networking diagram of one-way frame delay measurement

» Two-way frame delay measurementTwo-way frame delay measurement is commonly performed on end-to-end MEPs. A MEP receives a Delay Measurement Message (DMM) and returns a DMR to carry out two-way frame delay measurement. As shown in Figure 9-4, the process of two-way frame delay measurement is as follows:

1. A MEP periodically sends DMMs carrying TxTimeStampf.

2. After receiving a DMM, the remote MEP adds the RxTimeStampf value (the time of receiving the DMM) to the DMM, generates a DMR with the TxTimeStampb value (the time of sending the DMR), and sends the frame to the requesting MEP. Every field in the DMM is copied to the DMR, except that the source and destination MAC addresses are swapped and the essage type is changed from DMM to DMR.

3. After receiving a DMR, the MEP that sends a DMM calculates the two-way frame delay based on the following formula:



Page 116 of 122


Frame delay = (RxTimeb - TxTimeStampf) - (TxTimeStampb - RxTimeStampf)

Figure 13-4 Networking diagram of two-way frame delay measurement

13.3.3 Y.1731 deployment

Figure 13-1 Y.1731 deployment in CX

› Example for Configure Proactive Dual-Ended Frame Loss Measurement on a VLL

» CX of VLL one side: #cfm version standardcfm enable#cfm md md1ma ma1map mpls l2vc 2 taggedmep mep-id 1 interface GigabitEthernet1/0/1.1 inwardmep ccm-send mep-id 1 enableremote-mep mep-id 2remote-mep ccm-receive mep-id 2 enable



Page 117 of 122


loss-measure dual-ended continual mep mep-id 1 remote-mep mep-id 2#

» CX of VLL opposite side: #cfm version standardcfm enable#cfm md md1ma ma1map mpls l2vc 2 taggedmep mep-id 2 interface GigabitEthernet1/0/1.1 inwardmep ccm-send mep-id 2 enableremote-mep mep-id 1remote-mep ccm-receive mep-id 1 enableloss-measure dual-ended enable mep mep-id 2 remote-mep mep-id 1#

› Example for Configuring Proactive Dual-Ended Frame Loss Measurement on a VLL

» CX of VLL one side: #cfm version standardcfm enable#cfm md md1ma ma1map mpls l2vc 2 taggedmep mep-id 1 interface GigabitEthernet1/0/1.1 inwardremote-mep mep-id 2delay-measure two-way continual mep mep-id 1 mac 0001-0001-0001 interval 30000#

» CX of VLL opposite side: #cfm version standardcfm enable#cfm md md1ma ma1map mpls l2vc 2 taggedmep mep-id 2 interface GigabitEthernet1/0/1.1 inwardremote-mep mep-id 1delay-measure two-way receive#



Page 118 of 122


13.4 NTP

13.4.1 NTP Overview NTP aims at clock synchronization of all devices in a network. It keeps all the clocks of these devices consistent, and enables the devices to implement various applications based on the uniform time. Any local system that runs NTP can be time synchronized by other clock sources, and also act as a clock source to synchronize other clocks. In addition, mutual synchronization can be done through NTP packets exchanges.

NTP runs over UDP. The NTP port number is 123.

13.4.2 NTP Design IP RAN and IP Core device NTP server is set refer to M2000.



Page 119 of 122


14 Annex

Please refer to related files in the folder.

1. Port Requirement Statistic

2. BoQ

3. Sites ID & Device Naming IP RAN Jabodetabek

4. Detail IP Address for AXIS IP RAN – No yet

5. Layer-2 and Layer-3 Connection Chart

6. Physical Connection of IP RAN Jabodetabek

7. VLAN & VLL VC ID Design

8. TE Hot-Standby & BFD Parameter

9. BFD Discriminator Design for AXIS Jabo IP RAN

10. Sync. Eth Clock Priority Design

11. Device Board BOM Cord & E2E Clock support Status



Page 120 of 122

Documents

LLD of AXIS Amber Project IP RAN Jabodetabek (L3VPN) v1.8 - No Yet Update