Huawei NE5000E White Paper - Router Analysis · PDF fileLPU% (LineProcessingUnit ... !fabric!cards,!the!systemcan!connect! ... Huawei NE5000E White Paper

Copyright Steven Noble 2012

Huawei NE5000E Huawei’s Multi-‐Chassis Capable Core Router Steven Noble ([email protected])

ABSTRACT

This paper covers the hardware used for the Huawei NE5000E system including; hardware configurations currently offered, necessary pieces for the different configurations, Huawei’s naming convention and an overview of the upgrade process. Last Updated: June 24, 2010

Huawei NE5000E Core Router

Copyright Steven Noble 2012 2

Table of Contents

NE5000E Hardware Naming Cheat Sheet ........................................................................ 4 General Hardware ............................................................................................................................ 4 Software ............................................................................................................................................... 4

Document Summary ............................................................................................................... 5

NE5000E Timeline ................................................................................................................... 6 NE Series Release Dates and Backwards Compatibility ............................................. 7

Huawei Installation Base Claims ........................................................................................ 7 NE5000E Logical Overview ................................................................................................... 9 NE5000E Chassis ............................................................................................................................... 9

NE5000E Physical Overview ............................................................................................. 11 NE5000E Chassis ............................................................................................................................. 11 MPU (Master Processing Unit) ................................................................................................... 12 SFU (Switch Fabric Unit) .............................................................................................................. 12 SFUC ................................................................................................................................................................... 12 SFEA ................................................................................................................................................................... 12

LPU (Line Processing Unit) .......................................................................................................... 14 Line Card Comparison Chart ....................................................................................................... 14 Specific Line Cards ......................................................................................................................... 15 LPUA .................................................................................................................................................................. 15 LPUC ................................................................................................................................................................... 16 LPUE ................................................................................................................................................................... 16

LPUF Family of Line Cards ........................................................................................................... 17 LPUF Basic Specifications ......................................................................................................................... 17 LPUF-‐10/20 .................................................................................................................................................... 18 LPUF-‐21 ............................................................................................................................................................ 18 LPUF-‐40 ............................................................................................................................................................ 18

SPU (Service Processing Unit) .................................................................................................... 18 VRP 5.5 ..................................................................................................................................... 19 Versatile Routing Platform .......................................................................................................... 19 VRP Overview ................................................................................................................................................ 19 Software Patching ........................................................................................................................................ 19

Next Generation Technology ............................................................................................ 20 LPUF-‐100/LPUX (100G Card) ................................................................................................................. 20

Comparing the CRS-‐1 to the NE5000E ........................................................................... 22 Differences ........................................................................................................................................ 22 Processing Power ......................................................................................................................................... 22 Fabric ................................................................................................................................................................. 22 Chassis .............................................................................................................................................................. 22 ISSU .................................................................................................................................................................... 22

Similarities ........................................................................................................................................ 23 In Service Hardware Upgrade ................................................................................................................. 23 Fabric ................................................................................................................................................................. 23

Table of Contents


Appendix .................................................................................................................................. 24 Hardware ........................................................................................................................................... 24 Software ............................................................................................................................................. 24

Figure 1 -‐ Huawei ad showing stand-‐alone, dual and 4 + 2 Cluster Systems



NE5000E Hardware Naming Cheat Sheet

General Hardware

LPU (Line Processing Unit) – Line Card MPU (Main Processing Unit) – Route Processor SFU (Switch Fabric Unit) – Switch Fabric Card SPU (Serivce Processing Unit) – Services Card

Software

VRP – Versitile Routing Platform – Operating System



Document Summary

The Huawei Quidway Net Engine 5000E (NE5000E) is the current high-‐end core router offered by Huawei. The NE5000E was introduced in 2004 and has been upgraded incrementally over the last 6 years.

Huawei claims that they have sold over 1000 NE5000Es to many large network providers in many countries.

The NE5000E is composed of the same general parts that all core router are made from; a system processor, switch fabric and line cards. Huawei uses different acronyms then Cisco for their hardware. The system processor is called the MPU, the switch fabric is called the SFU and the line card is called the LPU.

There are multiple line card versions available for the NE5000E from the first generation LPU-‐A to the current LPU-‐F. For doing certain features such as Netflow, MPLS VPN and certain types of tunneling the NE5000E uses a service card called the SPU. The line cards utilize TCAMs for forwarding, ACLs, ARP Tables and other features. The NE5000E can be connected to other NE5000Es directly or through a switch fabric chassis to create a multi-‐chassis system or “Cluster” as Huawei calls it. Huawei currently claims that their largest cluster, a 4+2 cluster system is in production in China. Huawei does not support fabric Multicast replication on the NE5000E. The source line card does the first two multicast replications and the other line cards handling the other replications. Due to the fabric usage of Multicast, Unicast traffic can be dropped by the system if there are too many Multicaat replications of any one stream. The Operating System used on the NE5000E is VRP. VRP is a monolithic operating system based on VxWorks. VRP supports limited software patching and claims support process restart.



NE5000E Timeline

In 2002 Huawei introduced the NE5000 TSR, a rebadged router from another vendor. The system supported up to 10G per slot and promised performance equal to that of Juniper’s M120 and Cisco Systems GSR. This router was replaced in 2004 with a Huawei developed system, the NE5000E.

The NE5000E is based on the same chassis as the NE80E. The main difference between the two systems is the fabric. The fabric in the NE5000E is 40G while the NE80E was 10G in 2004. Currently both the NE80E and NE5000E have 40G fabrics.

The NE5000E was designed in-‐house at Huawei to compete with the Cisco CRS-‐1 and the Juniper T-‐Series.

The first release of the NE5000E had a 40G fabric but used the 10G LPU-‐A line cards from the NE80E. The LPU-‐A is a 10G capable line card that utilizes the Cisco designed Intel IXP2800 NPU. The first NE5000E specific line card was the LPU-‐C. The LPU-‐C was claimed by Huawei to support up to 20Gb/s at wire-‐rate but had limited memory and performance. The first claimed 40G capable line card was the LPU-‐E. The LPU-‐E has claimed performance of over 60Mpps and 1M IPv4 FIB entries.

The current incarnation of the NE5000E has a 40G Fabric, 8kW power supplies and 40G line cards.

The NE5000E is also capable of “Clustering” i.e. Multi-‐Chassis. With the addition of SFEA (Switch Fabric Extended) fabric cards, the system can connect back-‐to-‐back with another NE5000E or to multiple NE5000Es through switch fabric chassis.

Most of the forwarding chips are made by a Huawei sub-‐company called HiSilicon. The HiSilicon SD588 is a 10G chip and is used on many of the line cards such as the LPUB, LPUC, LPUF. The HiSilicon SD588NFC looks to be a 20G chip and is used in the LPUE.



NE Series Release Dates and Backwards Compatibility

• Pre 2004 o NE40/80 – 2.5G chassis

§ Huawei built product o NE5000 – 10G Fabric

§ Rebadged from another company • 2004

o NE80E – 10G fabric § Limited backwards compatibility with NE40/80 line cards by

utilizing the LPU-‐D adapter line card. o NE5000E – 40G Fabric

§ No 40G cards at introduction, only 20G LPU-‐C § No backwards compatibility with the NE5000 line cards

• 2005 o NE40E – 10G Fabric that accepts 20G cards (oversubscribed)

• 2007 o NE40E – Upgraded to 20G fabric o NE80E – Upgraded to 40G fabric

• 2009 o NE40E-‐X3 – First NExxE-‐X series router

§ Small form factor § Completely new fabric (claimed to be 1.2Tb)

• 2009 o NE40E-‐X8/16 – New Chassis/Cooling/Power/Fabric

§ Upgrade needed to support > 40G slot (> 20G on NE40E-‐8) § New cooling design to support multiple high-‐speed line cards § Claimed support for all previous NE-‐E series line cards (LPUA-‐

>LPUK) • 2010

o NE40E-‐X1/2 o NE5000E-‐X16

• 2011 o VRP8 o LPUF-‐100 o SFUI

Huawei Installation Base Claims Huawei claims that the NE5000E core router has been deployed worldwide.

They also claim that by the end of December 30, 2009, more than 1000 routers have been installed and more than 40 NE5000E clusters installed in over 15 backbone/metro core networks.



The following is from a press release by Huawei: “In China, China Telecom, China Netcom, China Mobile, China Unicom have

used the NE5000E routers in a large scale. Outside of China, the NE5000E routers have been used in a large scale in operators' networks of T-‐mobile in Holland, FT/Orange in Belgium and Portugal, Singtel in Singapore, Globe in Philippines, Etisalat in Saudi Arabia, TATA in India, America Movil in Ecuador, VIVO in Brazil, Central Telecom in Russian, SuperOnline in Turkey.” Note: The Press release states that NE5000E routers “have been used” outside of china, it does not specifically state that they are currently in use, or are deployed in production.



NE5000E Logical Overview

NE5000E Chassis There are three key parts that combine together to create the NE5000E; The Line Processing Unit (LPU), Switch Fabric Unit (SFU) and Main Processing Unit (MPU)

The Line Processing Unit (LPU) provides the physical interfaces to the outside world (Ethernet/POS) and connects to the Switch Fabric Unit (SFU).

The Switch Fabric Unit (SFU) connects the Line Processing Units (LPU) together allowing packets to be sent from card to card across the fabric.

The Main Processing Unit (MPU) is akin to the Route Processor of the Cisco 12000 Series Router. The MPU runs the control and management planes of the Chassis



Figure 2 -‐ NE5000E Internal Connections

SFU

SFU

SFU

SFU

MPU

LPU

MPU

LPU

LPU

LPU

LPU

LPU



NE5000E Physical Overview

NE5000E Chassis

The NE5000E has the same physical footprint as the NE80E and the CX600-‐16. Dimensions of the NE5000E are 442 mm x 669 mm x 1600 mm (width x depth x height). It weighs about 551 lbs fully loaded. Max power draw is stated at about 6800 watts.

1

LPU

2

LPU

3

LPU

5

LPU

6

LPU

7

LPU

4

LPU

8

LPU

9

LPU

10

LPU

11

LPU

12

LPU

13

LPU

14

LPU

15

LPU

16

LPU

19

SFU

20

SFU

21

SFU

22

SFU

17

MPU

18

MPU

Figure 3 – NE5000E Chassis and Line Card Numbering Diagram

(1) LCD Display (Fans install behind), (2) Cable Management, (3) LPU, (4) Mounting Hardware, (5) Air Intake (Air flows in the front bottom and out the back top) , (6) Power Supplies, (7) Handles

The NE5000E supports 16 LPUs (Line Processing Unit) in Slots 1-‐16, 4 SFUs (Switch Fabric Unit) in Slots 19-‐22, 2 MPUs (Main Processing Unit) in Slots 17-‐18, redundant fan trays and power supplies



MPU (Master Processing Unit)

The Main Processing Unit (MPU) consists of the main control unit, system clock unit, synchronous switching clock unit and system maintenance unit. The MPU controls and manages the system. As the system clock source and the management and maintenance unit, the MPU provides the functions of the control plane, switching plane, and maintenance plane. The MPU runs the VRP (Versatile Routing Platform) software and is responsible for all of the tasks of a standard route processor.

There are two versions of the MPU currently, MPU-‐A which is used in stand-‐alone systems and the MPU-‐B which is required for Clustered systems. The MPU utilizes a PowerPC processor and comes with 2GB of ram and a 512MB CF card.

Figure 4 – NE5000E MPU

SFU (Switch Fabric Unit)

There are 4 switch fabric cards in the NE5000E. The fabric runs in a 3+1 redundant configuration (N+1). Each fabric card has 64 3.125G SerDes connections giving them 200G each. The SerDes connections are shared 16 per slot across 16 slots. After encoding there is about 160G usable.

SFUC

The Switch Fabric Unit rev C (SFUC) is the standard 40G stand-‐alone fabric for the NE5000E. It is a crossbar-‐based fabric designed by Dune Networks. This fabric does not do multicast replication.

SFEA

The Switch Fabric Extended rev A (SFEA) is the multi-‐chassis 40G switch fabric for the NE5000E. It is a 3 stage switching fabric that contains Amphenol connectors to allow for either back-‐to-‐back connection between two NE5000Es or connection directly to an external switch fabric.

Each Amphenol connector has 72 fibers, 32 are used in each direction with 8 spares. Each Fabric card has 4 connectors, and there are 4 switch fabric cards in the



system for a total of 1.6Tb of raw bandwidth in each direction if all fabrics were active. Since the system is 3+1 redundant, there is only 1.2Tb of raw bandwidth available. After overhead and encoding there is about 750G left.

This fabric also does not do multicast replication.

Figure 5 -‐ SFEA



LPU (Line Processing Unit)

The LPUs designed by Huawei follow a standard formula containing the following items; physical interfaces (Ethernet/Pos), multiple NPUs, TCAM memories, a PowerPC CPU and a FAD (Fabric Access Device). The LPUs have letter designations starting from LPUA and currently going past LPUK. While most line cards are single versions, the LPU-‐F is a family of line cards.

The initial run of cards, the LPUA and LPUB contained a limited (512k) amount of TCAM. The maximum TCAM entries for IPv4 was set to 224k, which is lower then the current Internet routing table. The LPUG was the first card that claimed support for 600k IPv4 TCAM entries. Huawei has been replacing LPUA cards at customer sites with the LPUG.

The LPUH was the first card to claim support for 1M IPv4 TCAM entries. Currently the LPUE, LPUF-‐20, LPUH and a few others claim support for 1M FIB entries.

An important thing to note is that the FIB size on the system is limited by the worst-‐case line card inserted; if a LPUA is installed, the system will default to 224k IPv4 routes in the FIB.

Each line 40G line card has 8 3.125G SerDes connections to each fabric for a total of 75G of raw bandwidth (3x25). Using normal overhead calculations gives a bandwidth available of about 46G. As Huawei claims that there are 16 SerDes available to each line card, they could get up to about 93G but not 100G required to do a 100G line card.

Line Card Comparison Chart Line Card Processor(s) Forwarding

Claimed Full feature Chassis

Support IPv4 FIB Table

LPUA Intel IXP2800 25Mpps Yes 40/80/5000 224k

LPUB Huawei SD588 25Mpps No 40/80/5000 224k

LPUC Huawei SD588 50Mpps Yes 40/5000 224k

LPUD Huawei SD588 25Mpps No 40/80/5000 224k

LPUE Huawei SD588NFC 100Mpps Yes 5000 1M

LPUF-‐10/20/40

Huawei SD588 25/50/100Mpps -‐A versions only

40/80/5000 1M



Specific Line Cards

LPUA

The LPUA was the first line card introduced for the NE40E/80E/5000E series routers. The LPUA was based on the Intel IXP2800 NPU, which was originally designed by Cisco. The LPUA has limited TCAM memory allowing for only 220k IPv4 routes to be put in hardware. This is the same design that is used in all of the future cards with the main changes being increased TCAM and Different NPUs.



LPUC

The LPUC was the first > 10G forwarding line card available for the NE series routers. It works in both the NE5000E and the NE40E but not the NE80E. It has a claimed forwarding performance of 50Mpps and the ability to hold 224k IPv4 FIB entries by default.

Figure 6 -‐ LPUC 2 Port 10GE

LPUE

The LPUE was the first 40G capable card available for the NE5000E. It utilizes 2 20G NPUs in parallel. There are currently 2 versions of the card a 40x1GE



and a 4x10GE. The card claims forwarding performance of 100Mpps and up to 1M IPv4 FIB Entries. Testing has shown that the forwarding performance of the LPUE is affected by Access Lists and other features.

Figure 7 -‐ LPUE 4 Port 10GE

LPUF Family of Line Cards

LPUF Basic Specifications

The LPUF is a flexible line card that accepts FPICs (flexible plugged-‐in interface cards). There are currently 10, 20 and 40G versions of the LPUF line card. The LPUF-‐10 is a 10G line card, the LPUF-‐20 is a 20G line card, the LPUF-‐21 is an updated LPUF-‐20 and the LPUF-‐40 is a 40G line card. There are also performance and value versions of the cards (LPUF-‐XX-‐A and –B). LPUF-‐21-‐A would be a performance card (supporting IPv6, MVPN and L3VPN) that can be fitted with a 40xGE or 4x10GE adapter. The FPICs are not hot swappable.

Figure 8 -‐ LPUF-‐10



Figure 9 -‐ 10GE FPIC

LPUF-‐10/20

The LPUF-‐10 and 20 were the first flexible line cards introduced for the NE series routers. Both cards are based on the HiSilicon SD588 NPU. Both the LPUF-‐10 and LPUF-‐20 can take 2 port adapters. The LPUF-‐10 can only handle 10Gb/s of traffic while the LPUF-‐20 can handle up to 20Gb/s. Claimed forwarding performance is 25Mpps for the LPUF-‐10 and 50Mpps for the LPUF-‐20. Claimed FIB size is 1M IPv4 routes.

LPUF-‐21

The LPUF-‐21 is a slightly improved version of the LPUF-‐20, it is hard to know exactly what the improvements are as Huawei does not release detailed data sheets on their equipment. The public specs read identical in the data sheets. Claimed forwarding is 50Mpps and Claimed FIB size is 1M IPv4 routes.

LPUF-‐40

The LPUF-‐40 is a 40G card that utilizes 2 SD588NFC ASICs. Huawei claims that the card is capable of 100Mpps and 40Gb/s line-‐rate. This card should have the same physical characteristics as the LPUE just with FPICs.

SPU (Service Processing Unit)

The Service Processing Unit (SPU) is a line card with no interfaces used to implement the NetStream function and processes tunnel services related to Generic Routing Encapsulation (GRE) and multicast Virtual Private Networks (VPNs).

Since traffic must travel onto and back off of the SPU, the 10G SPU can only do a maximum of 5Gb/s of traffic processing at best. It also uses a slower CPU then the current line cards, a 800 MHz PowerPC vs. the 1000 MHz PowerPC that is utilized by the line cards.



Feature support on top of the services offered on the SPU is likely to be worse then on a normal line card. For example QoS on top of mVPN.

VRP 5.5

Versatile Routing Platform

VRP Overview VRP is a Monolithic OS based on VxWorks. VRP version 5.5 is specific to the NE5000E. Huawei claims that VRP supports most all IPv4 and IPv6 features necessary in networks. They also claim support for NFS/BFD on most routing protocols. When answering RFPs Huawei is known to answer yes to all features then build a special version of VRP supporting the necessary features for testing and/or deployment.

Because of the special versions, there is no “standard” VRP release. All software support is done directly via engineers at Huawei; i.e. you cannot go to the Huawei website and download the latest version of VRP for your platform. Due to the fact that Huawei has hundreds if not thousands of different builds of VRP (basically one for each different customer) it is hard to make a list of the features they actually support.

Software Patching VRP 5.5 Software is distributed as a single blob and loaded as such. Huawei claims that VRP is a modular OS as it supports software patching. There is no comparison to IOS-‐XR as IOS-‐XR is a fully modular OS. If a patch requires changes to hardware firmware, the affected hardware must be restarted. Huawei does not have a feature like MDR.



Next Generation Technology

LPUF-‐100 (100G Card)

The LPUF-‐100 has been announced by Huawei but will not ship until at least 2011. The LPUF-‐100 is a 100G card based on the just announced Solar 2.0 PFE ASIC. The Solar 2.0 PFE2A chip is developed by Huawei and based on their Macro Instruction for Packet Processing; a Huawei technology claimed to be good for IP/MPLS. Compared to a network processor, the Solar 2.0 PFE2A chip is claimed to be more flexible in application of services, higher performance, and to have low-‐power consumption.



Figure 10 -‐ LPUF-‐100 Logical Diagram

Figure 11 -‐ Huawei Solar 2.0 100G ASIC

Figure 12 -‐ Huawei LPUF-‐100 100GE and 10x10GE linecards



Comparing the CRS-‐1 to the NE5000E

Differences

Processing Power

The CRS-‐1 RP is SMP while the NE5000E MPU-‐B is a single CPU. The risk of bogging down the CPU on the NE5000E is much greater then on the CRS-‐1.

The NE5000E does not have DRPs, limiting its usability in Multi-‐Chassis deployments. Multi-‐chassis deployments tend to have higher feature capacity requirements such as 1000s of BGP peers, 100s of local neighbors doing different protocols, etc. A single RP with a single processor will have issues keeping up with even a single chassis.

Fabric

The NE5000E Switch Fabric Chassis is half as dense as the CRS-‐1 Fabric Card Chassis. In similarly sized systems for each Cisco FCC deployed, Huawei would need two of theirs. 2 Huawei fabric chassis will take up the same amount or more space then a CRS-‐1 fabric chassis depending on how they are mounted.

The original stand-‐alone NE5000E has a crossbar-‐based fabric (SFUC), the CRS-‐1 only has a 3 stage fabric, stand-‐alone or multi-‐chassis.

The NE5000E SFEA (Multi-‐Chassis capable switch fabric) contains all 3 stages of the CLOS fabric, while the CRS-‐1 only has S1/S3. This means the cost of goods (COGS) on the NE5000E SFEA are higher then the COGS on the CRS Fabric.

Chassis

The CRS-‐1 chassis was designed from the ground up to be next generation capable, supporting 40Gb/slot initially, upgradable to 120+ Gb/slot using the same chassis. The cooling design and chassis rigidity are the best in the business.

The NE5000E chassis is the same chassis used on the NE80E and CX600-‐16 systems; lightweight metal with minimal cooling/power. In order to support a chassis full of 100G line cards, Huawei will need to release a Next Generation NE5000E with a better cooling design and upgraded power. Huawei has already released upgraded versions of the NE40E series routers (The NE40E-‐X) to address these issues on that platform. There is a good chance that the next generation NE5000E will be based on the same chassis as the NE40E-‐X16.

ISSU Cisco offers per-‐process upgradability and restart while Huawei only offers

software patching.



Cisco has MDR (Minimum Disruptive Restart), Huawei requires that line cards and/or processors are reset if there are any microcode changes to the processing path.

Similarities

In Service Hardware Upgrade

Both the NE5000E Cluster System and the CRS-‐1 MC claim no traffic disruption during system expansion.

Huawei has a patented process called ISHE (In-‐service Hardware Expansion), which they claim to enable hitless hardware expansion from a single chassis to back to back and on to multi-‐chassis.

Fabric

Both the NE5000E Cluster System and the CRS-‐1 MC utilize a 3 stage switching fabric. Huawei uses CLOS while Cisco uses Benes.



Appendix

Hardware

LPU (Line Processing Unit) – Line Card MPU (Main Processing Unit) – Route Processor SFU (Switch Fabric Unit) – Switch Fabric Card SPU (Serivce Processing Unit) – Services Card LPU-‐A – First generation full feature1 line card LPU-‐B – Lower cost limited feature line card LPU-‐C – First 20G line card specific for the NE5000E LPU-‐D – Adapter card allowing the use previous generation line cards LPU-‐E – 40G line card for the NE5000E

LPU-‐F – Flexible line card (akin to SIP/SPA) LPU-‐F10-‐A – 10G full feature1 line card LPU-‐F10-‐B – 10G limited feature line card

LPU-‐F20-‐A – 20G full feature1 line card LPU-‐F20-‐B – Lower cost limited feature line card LPU-‐F21-‐A – Latest generation full feature1 LPUF with more memory LPU-‐F21-‐B – Latest generation limited feature line card LPU-‐F40-‐A – 40G full feature1 line card LPU-‐F40-‐B – 40G limited feature line card LPU-‐F100-‐A – 100G full feature1 line card SPU-‐C – 10G service card

Software

VRP – Versitile Routing Platform – VxWorks based OS for Huawei routers VRP 5.5 – VRP version for the NE5000E Specifically VRP 8.2 – VRP version 8 for the NE5000E NetStream – Licensed feature IPv6 – A license is required to enable IPv6 on the NE5000E Tunneling – A license is required to enable GRE/IPSEC/L2TP tunneling Layer 2 – A license is required to enable Layer 2 switching and MSTP 1 L3VPN, MVPN, IPv6 Enhanced

Documents

Huawei NE5000E White Paper - Router Analysis · PDF fileLPU% (LineProcessingUnit ... !fabric!cards,!the!systemcan!connect! ... Huawei NE5000E White Paper