Public Multi-Vendor Interoperability Event White Paper · CX600-X1-M4 CX600-X2-M8 Ixia Anue 3500 Anue GEM Anue XGEM ImpairNet IxNetwork RAD ETX-205A ETX-220A MiNID Spirent Communications

Ethernet World 2013

Public Multi-VendorInteroperability Event

White Paper

2

Ethernet World 2013 Multi-Vendor Interoperability Test

EDITOR’S NOTE

At EANTC, we are happyand proud to have completedthe ninth public interopera-bility test at IIR’s series ofEthernet World conferencesthis year, with support fromleading vendors. Packet transport networksusing MPLS, IP, and CarrierEthernet technologies havebecome mature and thus are

the dominating solutions in the market. The CarrierEthernet paradigms are leading the way in mobilebackhaul and wholesale networks.That said, there are a number of major operationaland interoperability challenges remaining which weevaluated in this event: Mobile networks require synchronized base stations.The underlying technology, IEEE 1588:2008 andaccompanying ITU-T standards, are still in flux andare non-trivial to implement. Advanced LTE deploy-ments require even stronger clock support in thenetwork (not only at its edges), achieved withboundary clocks. It was a great achievement to getfour boundary clock implementations to interoperate. In addition, a growingly mission-critical clock infra-structure requires resiliency. We successfully tested adistributed grandmaster with two vendors. As the industry is moving to best-of-breed mobilenetworks, the possibility of asingle-vendor mobile infra-structure is decreasing and withit service provider demand forinteroperable solutionsincreases. Proprietary clockimplementations or lack ofcompliance with the variousITU-T telecom profiles are nolonger viable options. EANTCwill continue to help theindustry evolve interoperabilityof clock synchronizationsolutions in future events andindividual tests for vendors and network operators.We also still see ample room for improvement onoperational aspects for Carrier Ethernet networks.Proactively monitoring performance and managingfaults efficiently is a key capability to reduce OPEX.There is a growing market of SFP-based demarcationand monitoring systems; we included one in thetests.I believe there is a gaping hole in the industry formulti-vendor service creation (provisioning) support.Many times, we have tried to encourage relevantvendors to join — some of which have really greatmarketing material describing matching solutions.None ever joined. The lesson learned is: Unless onehas a single-vendor network, or a network imple-mented with components from the top two or threeequipment vendors, the Command-Line Interface(CLI) has won the game. We have yet to witness asolution able to operate a multi-vendor packetnetwork with third-party tools. This is a pity, as

operational cost is the biggest unknown factor andrisk when deploying a packet network solution today. This is where Software-Defined Networks (SDN) arepromising major improvement. The potentialmigration of service provider networks to elasticSDN-based solutions is of great interest for theindustry. The topic was on our agenda but no vendordared to sign up for our service provider-focusedSDN tests this time yet. We look forward tovalidating SDN-based solutions in other venues inthe future.

INTRODUCTION

Mobile penetration and the proliferation of smartphones are driving the industry to deploy radios atmore and more locations. Small cells promisecellular coverage improvement as well as additionalcapacity. They fit well in dense residential andbusiness areas and could also help rural areasimprove connectivity.As small cells are still cells, they require access to therest of the mobile infrastructure such as macro-cells,clocks and controllers. This means that these devices,meant to be cost and energy effective, depend onother parts of the infrastructure to provide connec-tivity, clock synchronization information andtransport.In this year’s Ethernet World interoperabilityshowcase we took the opportunity to focus onEthernet tools to help service providers connect and

operate their cell and small cellinfrastructure using native Ethernetin the access and IP/MPLS orMPLS-TP in the network aggre-gation. One such area is the essentialclock synchronization over packet.Since our 2012 Carrier Ethernetinteroperability test event severalphase related profiles completedtheir path through the ITU standard-ization and were now officiallyavailable for us. These included forexample G.8271.1: Network

Requirements for Time/Phase Full on Path Support.We were also pleased to welcome to our interopera-bility test new Precision Time Protocol grandmasterclock functions meant to be placed closer to theirclock slaves as well as microwave transport devicesoperating on unlicensed frequencies such as 60GHzV-band and 80GHz E-band.As in every event, the team, both from the partici-pating vendors and EANTC’s own engineering staff,worked feverishly over the course of two weeks tosetup the test scenarios, perform the tests and collectthe results. Without the tight collaboration of thecomplete team the results presented herein could nothave been achieved — we would like to thankeverybody and hope that you enjoy the read.

Carsten RossenhövelManaging Director

EANTC

TABLE OF CONTENTS

Participants and Devices..............3Clock Synchronization.................3Carrier Ethernet Transport ............6Showcase Network Topology .......7Carrier Ethernet Life Cycle..........10Demonstration Network..............11

3

Transparent Clocks

INTEROPERABILITY TEST RESULTS

In the following sections of the white paper wedescribe the test areas and results of the interopera-bility event. We use the term “tested” when reportingon multi-vendor interoperability tests. The term“demonstrated” refers to scenarios where a serviceor protocol was terminated by equipment from asingle vendor on both ends.Executing the tests require traffic generation,measurement and impairment. We thank Calnex,Ixia and Spirent Communications for providing thesetools and their support during the hot staging.

PARTICIPANTS AND DEVICES

CLOCK SYNCHRONIZATION

In each and every test event since our first clocksynchronization tests in 2008 we see more imple-mentations as well as new standards, profiles andtopologies.This time we aimed to evaluate the solutions forsupport of what the group believed to be needed forsmall cell backhaul: phase support, heterogeneoustopologies traversing third party networks,advanced network design and new grandmasterclock elements in network locations where suchdevices did not exist in the past.The results presented in this section are the mostadvanced synchronization tests done in a publicforum in our industry. With LTE advanced expectedto reach deployment in the next couple of years weexpect this area to remain a staple in our events.

Transparent ClocksA transparent clock is a fundamental element of asynchronization network. Its purpose is to mitigatethe adverse effect of packet delay variation (PDV) onthe transfer of frequency and time without theprotocol complexity inherit to terminating a precisiontime protocol (PTP) session. A transparent clockupdates the correction field in PTP packets. Theupdates account for the cumulative delay introducedby transparent clocks. This helps to reduce the time aslave clock requires to lock to the grandmaster orboundary clock.In our test, the transparent clock was inserted intothe PTP path between the grandmaster and the slaveclock as we expect to see in a typical deploymentscenario. We generated bursty traffic on the linkbetween the transparent clock and the slave clockand measured the accuracy of the correction fieldupdates. The analyzers were connected oncebetween the grandmaster and the transparent clocksas well as between the transparent and slave clocks.We compared the packet delay with the residencetime — the value the transparent clock adds to thecorrection field of the PTP packets.We measured the time interval error (TIE) at a sync-out interface of the slave clock for 15 minutes. Basedon TIE, we calculated MTIE (Maximum Time IntervalError) and time of day deviation. We expected afrequency deviation of no more than 16 ppb (partsper billion) and phase deviation (time of daydeviation) within ±1.5 μs (microseconds).In two test runs (see Figure 1) we measured the floorpacket population (FPP) metrics described in ITU-TG.8260. FPP is the ratio of the received packetcount within the cluster height to the total receivedpacket count. The cluster height is a packet delayrange between the minimum observed packet delayand the minimum observed packet delay plus 150μs. The FPP is calculated over a window of 200seconds.FPP measurements evaluate the packet delayvariation (PDV) at the network boundary, immedi-ately before the packet slave clock. An FPPmeasurement lower than the FPP limit indicatesexcessive PDV. We used the 1% FPP goal as

Vendor Devices

Calnex Paragon-X

Ericsson MINI-LINK PT 2020MINI-LINK PT 3060MINI-LINK SP 110MINI-LINK SP 210MINI-LINK SP 310MINI-LINK SP 415MINI-LINK SP 420MINI-LINK TNSPO 1410

Huawei ATN905-indoorATN905-outdoorATN910IATN910BATN950BCX600-X1-M4CX600-X2-M8

Ixia Anue 3500Anue GEMAnue XGEMImpairNetIxNetwork

RAD ETX-205AETX-220AMiNID

SpirentCommunications

Spirent TestCenter

Symmetricom SSU 2000eTimeProvider 1500TimeProvider 2300TimeProvider 2700TimeProvider 5000

4


described in ITU-T G.8261.1. All FPP measurementspassed that goal successfully.

Figure 1: Transparent Clocks

In the test runs involving the Ericsson MINI-LINKPT 2020 microwave solution as transparent clock wefaced a test design challenge: the device only hadone Ethernet port that was, by nature of the testsetup, connected to the grandmaster clock. In orderto inject traffic, we added another Ericsson devicewhich also acted as a boundary clock. This allowedus to perform the test as we did in the other setup. A successful run required MTIE measurements topass the G.823 SEC mask for frequency and theG.8271 time/phase accuracy level of 4 (±1.5 μs)and above. Figure 1 shows all successful results weobserved.We measured correction field accuracy, thecorrection field value added by the transparent clocksubtracted from the actual latency, between 542 nsand 254 μs.

Boundary ClocksBoundary clocks provide a means to distribute timein larger networks by building a clock hierarchy. Theboundary clock relieves the grandmaster fromhandling a large number of ordinary clocks,segmenting downstream networks into areas.In this test we set up a PTP clock chain consisting ofgrandmaster, boundary and slave clocks. Betweenthe grandmaster and the boundary clock andbetween the boundary and the ordinary clocks weemulated network paths of ten nodes each. Theimpairment generator performed this emulation withan traffic impairment profile according to ITU-TG.8261 test case 12. This test setups aims tosimulate a partial on-path support with oneboundary clock between the grandmaster and slaveclocks.For MTIE and time/phase accuracy calculation weexcluded the clock stabilization period. The stabili-zation period following the lock acquisition on theslave clocks ranged from 15 minutes to one hour.We evaluated a successful run to pass frequencymeasurements defined in the ITU-T G.823 (SEC MTIEmask) and the time/phase accuracy to be withinG.8271 quality level 4 (±1.5 μs) at the slave clockoutput. While all frequency measurements qualifiedsuccessfully with G.823 SEC, we observed phaseaccuracy qualifying for G.8271 quality levels 3(±5μs) and 4 (±1.5μs), attributable to the widebreadth of the emulated network.The figure depicts the results we observed.

Figure 2: Boundary Clocks

In one test run (Symmetricom TimeProvider 5000 asgrandmaster, Ericsson MINI-LINK SP 310 asboundary and Ericsson MINI-LINK SP 420 as slaveclock) the transport between the grandmaster andthe boundary clock was IPv4 unicast, while thetransport between the boundary and slave clocks

Clock link

Packet Switched Network (PSN)

Reference Clock

1PPS link

PTP EVC

Data EVCPTP Grandmaster

Synchronous Node

Analyzer

12:50:00

Impairment tool

Impairment

EricssonMINI-LINK

SP 210

SymmetricomTimeProvider

5000

12:50:00

RADETX-205A

TransparentClock

SlaveClock

EricssonMINI-LINK

SP 210

RAD ETX-205A

12:50:00

TransparentClock

SlaveClock

RADETX-205A


5000

12:50:00

TransparentClock

SlaveClock

SP 110

Calnex Paragon-X

Ixia Anue 3500

Calnex Paragon-X

EricssonMINI-LINK

Monitoring link

EricssonMINI-LINKPT 2020

EricssonMINI-LINKPT 2020

HuaweiATN910B

FPP

FPP

BoundaryClock

SlaveClock

RADETX-205A


Ericsson MINI-LINK SP 310





IxiaAnue 3500


1500

2300

2300

Symmetricom TimeProvider

5000

12:50:00

5

SyncE Islands Synchronization via PTP

was Ethernet multicast. Translating between thevarious transport mechanisms available in the IEEE1588v2 standard, adds flexibility in clock transportnetworks. As we have seen in our tests over theyears, to reach a true multi-vendor clock network,sometimes this translation capability enablessolutions that otherwise would not interoperate witheach other to build an end-to-end network.

SyncE Islands Synchronization via PTPA common use case for operators using any kind ofleased lines that do not provide synchronization isthe concept of Synchronous Ethernet Islandsconnected by PTP. PTP could then be used to crossthe domain that does not support synchronousEthernet, and could be used as the clock source forthe next Synchronous Ethernet domain.In order to test this scenario we connected twoSyncE networks (Islands) over a packet network. Thenodes on the border between the SyncE-enablednetworks and the packet network ran PTP andsynchronized the SyncE islands with each other. Toemulate the asynchronous packet network we usedan impairment tool that introduced PDV to thenetwork segment between the PTP master and thePTP slave using ITU-T G.8261 test case 12.After all clocks stabilized, we started the frequencymeasurements and evaluated them against theG.823 SEC MTIE mask.Since the test setup included a large number ofdevices, we only needed to verify one test scenario.The scenario consisted of RAD ETX-205A as theSyncE master, Ericsson SPO 1410 as the PTP master,Huawei ATN950B as the PTP slave, Ericsson MINI-LINK SP 310 as the SyncE slave while SymmetricomTimeProvider 5000 acted as the frequencyreference. The test run successfully passed G.823SEC MTIE mask. Measurements were performedwith Ixia Anue 3500.

Distributed Grandmaster ClockWe have been testing PTP slave clocks since our firstMobile Backhaul interoperability event in 2008.While each of our clock tests requires a slave clock,the functionality of the slave clock is almost a given.These days we see vendors introducing new class ofgrandmaster clocks however. These clocks aredesigned to be positioned much closer (in a networktopology perspective) to the slave clocks. In a small cell backhaul scenario, positioning thegrandmaster clock closer to a geographically limitedarea reduces the risk of a catastrophic clock infra-structure failure. A distributed grandmaster clockdesign allows the positioning of the grandmasterclock close to the slaves. In this scenario, the numberof hops between the grandmaster and slave clock istypically in the range of two to five hops. Thisreduces the risk of clock infrastructure failures,reduces PDV and asymmetry, all of which negativelyeffect the timing synchronization accuracy. It alsoreduces (or possibly eliminates) the number of nodesthat need upgrade to BC/TC functionality where1588 is concerned.

Figure 3: Distributed Grandmaster Clock

We applied a packet delay variation (PDV)impairment profile according to G.8261 test case12 (which emulates ten hops), lacking astandardized model for this scenario.We started the test with PDV impairment runningand brought the slave clocks into free-running mode.We allowed the slave clocks up to 30 minutes toacquire a lock and started the measurements.We evaluated the frequency measurements with theG.823 SEC MTIE mask and G.8271 quality levels.All frequency measurements passed the G.823 SECMTIE mask and G.8271 quality level 5 (within±1 μs). The maximum time error we observed forphase/time was below 200 ns.

Best Master Clock SelectionThe Precision Time Protocol (PTP) provides amechanism for slave clocks to select the best masterclock in their respective synchronization domain.The selection is performed according to the PTP attri-butes of the grandmaster (e.g. PTP priority, clockclass).In our test, two grandmasters provided timing to PTPdomain A. The boundary clock was a slave clock indomain A and a master in domain B. The boundaryclock was synchronized with the best master in PTPdomain A and supplied synchronization informationdownstream to the slave clock in PTP domain Balong with the information about the grandmaster.As in the previous test, we started the test run withthe boundary clock and the slave clock in a free-running state. We used the Priority1 value asdefined in IEEE 1588 to dictate the grandmaster wedesignated as grandmaster A to be preferable. We then induced a switching event by modifying theclock priority of grandmaster A to be less preferablethan grandmaster B. We then used an impairmenttool to drop all PTP messages originating fromgrandmaster B, expecting the boundary clock torevert to grandmaster A after it detected that grand-master B is no longer available.

EricssonSPO 1410


12:50:00

2700

RADETX-205A

12:50:00

IxiaAnue 3500

CalnexParagon-X

HuaweiATN910I

RADETX-205A

12:50:00

EricssonMINI-LINK TN

CalnexParagon-X

Impairment SlaveClock

6


Before and after each switching event, we verifiedthat the identity of the grandmaster as shown by theboundary clock and by the slave clock matches theexpected preferable grandmaster clock.We performed one test run with two SymmetricomTimeProvider 5000 devices as grandmaster clocks,Huawei CX600-X2-M8 as boundary clock andEricsson MINI-LINK TN as slave clock. IPv4 unicasttransport was used between the grandmaster andboundary clocks, while Ethernet multicast was usedbetween the boundary and the slave clocks. Thefrequency measurements on the slave clock, whichincluded both switching events, passed the G.823SEC MTIE mask.We performed an additional test run with IxiaIxNetwork emulating the two grandmaster clocks,Ericsson MINI-LINK TN as boundary clock andHuawei ATN905-indoor as a slave clock. Althoughthe boundary clock did not lock to the grandmasterclock (since the emulated grandmaster clock was notsynchronized to GPS clock), we were able to verifythe correct selection according to the best masterclock algorithm. We observed that the slave clockwas locked to the boundary clock and reported aclock quality indicating that its source was previouslylocked into PTP (but not currently). No measurementswere taken in this test run, since the grandmasterclocks were emulated and did not have a clocksource (such as GPS).

Figure 4: Best Master Clock Selection

Precision Time Protocol Ethernet Multicast Address ScopesThe IEEE 1588 standard specifies two addresses tobe used for Ethernet multicast transport: an addressspecific to the 1588 with OUI 01-1B-19 and anaddress shared with the 802.1AB standard. The

main difference between the addresses is that the1588 address should be flooded, while the802.1AB address should not.The 802.1AB address was originally intended to beused only for peer delay mechanism messages.There have been recent discussions concerningpartial on-path support raising the question of usingeither the addresses as a mean to control forwardingof PTP packets via Ethernet multicast. The usage ofthe 1588 address allows a non-PTP node (or a PTPnode, when PTP is disabled) to be connectedbetween two PTP devices, while the usage of the802.1AB addresses provides tight control whichrequires full on-path support.Our test was designed to verify the correct floodingbehavior according to the address used. As inprevious tests, our setup consisted of a grandmasterclock, boundary clocks and a slave clock chain.We started the test with the boundary and the slaveclocks in free-running mode, allowed the clocks tolock and measured the output of the slave clock. Weproceeded by disabling PTP on the boundary clocks,resetting the slave clock to free-running mode andallowed it to lock.We performed one successful test run using the1588 address (in the order of connectivity): Symmet-ricom TimeProvider 5000 as grandmaster, EricssonMINI-LINK SP 420 as a boundary clock, HuaweiCX600-X1-M4 as boundary clock and EricssonMINI-LINK SP 415 as slave clock. All measurementspassed the G.823 SEC MTIE mask and G.8271quality level 5 (within ±1 μs). All measurements wereperformed with Ixia Anue 3500.

CARRIER ETHERNET TRANSPORT

The Metro Ethernet Forum (MEF) has beenhighlighting the benefits of Carrier Ethernet as atransport technology for mobile backhaul for severalyears. We have tested many aspects of CarrierEthernet in transport networks since our first CarrierEthernet focused event in 2005. In this event weallowed ourselves to set the focus on the variousways that Carrier Ethernet could support small cellbackhaul.

Ethernet Ring Protection Switching (ERPS)Network reliability is a major concern for serviceprovider and operator networks. Especially in amobile radio access network (RAN) scenario, the useof transporting customers voice and data, overpacket networks, require careful planning.If we accept the premise that Ethernet is a goodmatch for small cell backhaul, than Ethernet RingProtection Switching (ERPS) (defined in ITU-T recom-mendation G.8032) is also an adequate, nativeEthernet protection mechanism. Protected Ethernetrings use a closed loop topology by integratingEthernet Operation Administration and Maintenance(OAM) and an Automatic Protection Switching (APS)protocol to reach high degrees of link and nodefailure recovery.

Ixia Anue3500


12:50:00

5000


12:50:00

5000

IxiaIxNetwork

12:50:00

IxiaIxNetwork

12:50:00

HuaweiCX600-X2-M8


HuaweiATN905-indoor


Ixia Anue3500

BoundaryClock

SlaveClock

BoundaryClock

SlaveClock

7

Ethernet Ring Protection Switching (ERPS)

HuaweiATN905-outdoorEricsson

SPO 1410

EricssonMINI-LINK PT 2020

MINI-LINK SP 210

EricssonMINI-LINK SP 420

Ericsson

Symmetricom

Ericsson MINI-LINK TN

Symmetricom

Symmetricom

Ericsson MINI-LINKPT 2020

HuaweiATN950B Huawei

Ericsson MINI-LINKSP 310

Ericsson SPO 1410+RAD MiNID

Mobile Core

IP/MPLS

ERPS Ring

Access device Gigabit Ethernet

10Gigabit EthernetIEEE 1588v2 grandmaster

Microwave radiosystem


RAD ETX-205A


Frequency clock link

1PPS

HuaweiATN910B

Aggregation device

Traffic generator/analyzer

Symmetricom TimeProvider 5000

TimeProvider 2300TimeProvider 2700

ATN910IHuawei

ATN905-indoor

12:50:00

Emulator

Ethernet Access

ERPS RingRADETX-220A

MPLS-TP

Impairment generator

12:50:00

12:50:00

Ixia

SHOWCASE NETWORK TOPOLOGY

EthernetAggregation

Ixia Anue 3500Ericsson

MINI-LINK SP 110


SymmetricomTimeProvider 1500

SSU2000e

Radio access

HuaweiCX600-X2-M8

IxiaAnue GEM

RAD ETX-205A


HuaweiCX600-X1-M4

Ixia IxNetwork

IxNetwork IxiaImpairNet

SpirentTestCenter

CalnexParagon-X

12:50:00

8


The ERPS testing this year focused on a single ringinterconnecting small cell backhaul devices. In such atopology each ring node has two ring ports and allring nodes are physically connected in a closed loop.Traffic is transported over the ring in a single directionby placing a block on a ring link called RingProtection Link (RPL). Each end of the RPL isconnected to two ring nodes, called RPL owner andRPL neighbor. The activity of the protection switchingin an Ethernet ring is coordinated by the RingAutomatic Protocol Switching (R-APS). Upon linkfailure between ring nodes, the protection switchingin the ring is triggered by the Ring AutomaticProtection Switching Signal Fail (R-APS SF) messagessent by nodes connected to the failed link. The R-APSSF messages cause the RPL owner to unblock the RPLlink and traffic is then forwarded through the openRPL.In revertive mode of operation the RPL node detectsthat the failure is resolved by receiving (R-APS (NR)).This causes the RPL owner to start the Wait-to-Restore(WTR) timer. Upon expiration of the WTR timer, theRPL owner blocks the RPL port. Nodes adjacent tothe former failed link unblock their port and the ringresumes its original operation.

In all test scenarios, a single multi-vendor ring wasconstructed. The ring was provisioned with one RPLowner and RPL neighbor. The ring used a “ServiceChannel” to carry service traffic and an R-APSchannel for control messages transmission.We conducted all tests by first sending traffic on theservice configured in the ring using either Ixia

IxNetwork or Spirent TestCenter to verify the properoperation of the ring. We then emulated the failureof the link and verified that the RPL owner removedthe blocked port. We tested the restoration byresolving the link failure.Two methodologies were used to break the ringconnectivity — physical link disconnection andexplicit impairment of the Continuity CheckMessages (CCMs). When the explicit impairment ofthe CCM was used to break the connectivity in thering, Connectivity Fault Management (CFM) wasused to monitor the liveliness of link between nodes,running continuity Check Messages at an interval of3.33 milliseconds. Calnex Paragon-X and Ixia AnueXGEM were used to impair CCM messages.Figure 5 depicts the 5 test topologies that success-fully participated in the test. The simulated ringconnectivity failure generated failover time ranginganywhere from 6 to 35 milliseconds and therevertive switchover time from 0 to 16 milliseconds.In all test scenarios we also successfully tested theuse of administrative command including “ManualSwitch” and “Force Switch”, which moved the blockport as expected. The out of service time when thesecommands were used ranged from 2 to 25 milli-seconds. Once both administrative commands weresuccessfully cleared we measured restoration time upto 16 milliseconds.

RAD ETX-220A

Huawei ATN905-outdoor

Ericsson SPO 1410

Ericsson

Ericsson MINI-LINK

Ericsson

HuaweiATN950B

SPO 1410

MINI-LINK PT 3060

HuaweiATN910B RAD ETX-220A

EricssonMINI-LINK SP 310+PT 3060

Ericsson SPO 1410

Huawei ATN905-indoor

RAD ETX-220A

Huawei ATN950B

Ericsson SPO 1410

RAD ETX-220A

SP 310

Figure 5: Ethernet Ring Protection Switching

CalnexParagon-X

Ixia Anue XGEMIxia Anue XGEM

Working path

Protection path

10 GbE

ERPS-enabled networkMicrowave

device

Link failure

RPL interface

RPL neighborinterface

Traffic generator

Gigabit Ethernet link

9

Small Cell Backhaul Microwave QoS Support

Small Cell Backhaul Microwave QoS SupportMicrowave technology is increasingly used toprovide Carrier Ethernet transport. Microwaves arealso a great match for small cell backhaul due totheir ease of deployment without the need to runcables from small cells to a central location. Microwave radios use Adaptive Coding andModulation (ACM) to adapt the radio modulation tochanging transmission conditions. Variation inmodulation changes the link capacity, making itdifficult to guarantee Service Level Agreements(SLA). It is therefore desirable to reserve bandwidthto allow critical data such as synchronization, cellscoordination, signalling and voice to be prioritizedover best effort data with more relaxed SLAswhenever the link capacity is limited.The test setup connected two MPLS Label SwitchedRouter (LSR) through two microwave systemsconnected via an Ethernet link. An E-Line service wasconfigured between both LSRs on top of either IP/MPLS or MPLS-TP network. Two class of servicebased on MPLS-EXP were provisioned on all DUTs.High priority class was configured with the MPLS-EXP 6 and low priority class used to carry best efforttraffic was configured with MPLS-EXP bit 0. We firstgenerated bidirectional traffic using packet size of1,500 bytes at 80% of the maximum modulation,which was 512 QAM, corresponding to about 405Mbit/s. We did not observe packet drop and therecorded delay was up to 0.2 milliseconds.

Figure 6: Microwave QoS Support

We then used an RF attenuator to decrease themodulation to the minimum value — 4 QAM (about93 Mbit/s) to emulate a bad weather condition. Weexpected to maintain high priority traffic and losethe best-effort traffic as the latter was to be depriori-tized. In all test scenarios we observed results thatmatched our expectations: 0% traffic loss on the highpriority traffic and roughly 99% frame loss for best-effort traffic. We measured latency of 0.1–0.5 ms.

The test microwave device in all test scenarios wasEricsson MINI-LINK PT 2020. Ixia IxNetwork andSpirent TestCenter were emulating LSR. Likewise,Ericsson MINI-LINK SP 210, Ericsson MINI-LINK SP420 and Ericsson SPO 1410 acted as LSR in thetopologies depicted in Figure 6.

MPLS Pseudowire RedundancyIn cases where MPLS is the chosen technology in theradio access network (RAN), service providers havea plethora of tools to support high availability of thetransport network. When services to the cell site arerealized using MPLS pseudowires, RFC 6870 couldbe used for Pseudowire redundancy. The standarddefines a mechanism for pseudowire preferentialforwarding status bit and enables service providersto configure their MPLS network to detect a failure inthe network and reroute the traffic away from thefailure. In this test case we focused on two potentialcatastrophe scenarios: access and core link failure.In this test topology Huawei CX600-X2-M8 wasconfigured as Customer Edge (CE) device and wasdual-homed using two Attachment Circuits (AC) totwo Provider Edges (PEs) - Ericsson MINI-LINK SP415 and Ericsson MINI-LINK SP 420 — eachconfigured with a pseudowire in a primary/standbymanner. These in turn terminated on the HuaweiATN950B acting as an MPLS Provider Edge router.Each PE device was configured to independentlyselect which pseudowire is eligible to become activeas defined in RFC 6870. The test was performedwith revertible mode enabled. Once all service wasproperly configured, we sent bidirectional trafficusing Ixia IxNetwork. On failure of the primary AC, which was induced byphysical link break, we successfully verified thattraffic was redirected to the standby pseudowire asexpected. The observed failover time was 840 milli-seconds. The restoration time did not exceed 20milliseconds. We assume that faster restoration timecould be reached if fault detection mechanisms, suchas BFD, were to be used to monitor the pseudowire.

Figure 7: MPLS Pseudowire Redundancy


IxiaIxNetwork

IxiaIxNetwork



SpirentTestCenter

Ericsson MINI-LINKSP 210

SpirentTestCenter


IxiaIxNetwork

IxiaIxNetwork

EricssonSPO 1410

IP/MPLS network MPLS-TP network

IP/MPLS network

HuaweiCX600-X2-M8

HuaweiATN950B



Customer domain

Working path

Protection path

10


CARRIER ETHERNET LIFE CYCLE

Once the transport elements have been tested wechanged our focus to ways in which serviceproviders and carriers could actually manage,maintain and monitor their Carrier Ethernetnetworks. We used native Ethernet tools in all thesections presented herein — tools that are specifiedby the MEF and standardized by the IEEE and ITU.

Fault Management on a Multipoint-to-Multipoint EVCThese days the operator’s OAM (Operations, Admin-istration and Management) toolkit is filled with wellstandardized and tested tools. EANTC has beentesting various aspects of Connectivity FaultManagement (CFM) as standardized in IEEE802.1ag, since 2009. For this test event we focusedon one of the tools: Ethernet Loopback (ETH-LB). ETH-LB function is used to check the connectivitybetween a MEP and multiple peer MEPs. Thisfunction uses the Loopback Messages (LBM) and theLoopback Replies (LBR) to form an IP-like connectivitycheck. After receiving a multicast LBM, each MEP onthe receiver checks the multicast LBM and replieswith a unicast LBR.In our test scenario four devices, as depicted in thefollowing figure, were connected in a single multi-point topology. Each sent LBM messages to amulticast loopback address and received threeunicast answers from its peers. The maximumobserved response time across all DUTs was 35 milli-seconds, which met our expectation.These devices successfully participated in the test:Huawei ATN 910I, Huawei CX600-X1-M4, IxiaIxNetwork and RAD ETX-205A.

Figure 8: Multipoint Fault Management

Performance Monitoring per Class of ServicesThe key requirement in all small cell backhaulnetworks is the ability to monitor and manage thebackhaul and to enforce service level agreement.Performance monitoring help service providers byproviding the tool required to measure the SLA. Thekey SLA metrics having a direct impact on mobilesubscribers are frame loss, frame delay and framedelay variation. The ITU-T specification G.8031/Y.1731 provides a means to measure SLA metrics.

EANTC started to test the interoperability of theperformance monitoring according to Y.1731 whenthe standard was in its infancy in 2008. The initialversion of the standard specified the performancemonitoring for single class of service (CoS). Until2011 when the specification was augmented toallow multiple monitoring sessions on different Classof Service simultaneously, there were lot of imple-mentation of the standard and less interoperabilityissues between different vendors’ implementations.The main focus of our interoperability event this yearwas the performance monitoring per class of servicefor point-to-point and multipoint-to-multipoint EthernetVirtual Connection (EVC). From all performancemonitoring procedure defined in Y.1731, we testedonly frame loss, two-ways frame delay and delayvariation per EVC and CoS.We performed the test by configuring either point-to-point or multipoint-to-multipoint Ethernet servicebetween participating devices. We used eitherCalnex Paragon-X, Ixia Anue XGEM or IxiaImpairNet, inserted between Device Under Test(DUTs), to introduce controlled impairment such asframe loss, frame delay and delay variations. Foreach specific type of impairment we verified themeasurement results provided by the DUT againstthe emulated impairment. To show the baselineinteroperability between devices, we first ran the testwithout insertion of impairment.

Figure 9: Performance Monitoring per CoS for Point-to-Point EVC

To test loss measurement we sent bidirectionalEthernet traffic and used impairment generator tointroduce a constant frame loss of 15% in onedirection and 5% in the opposite direction forCoS 0. For CoS 3 we added a bidirectionalconstant loss of 10%. CoS 5 was not impaired.

Multicast Loopback

HuaweiATN910I

HuaweiCX600-X1-M4

RADETX-205A

IxiaIxNetwork

Ixia

Huawei Ericsson SPO 1410+ RAD MiNID

HuaweiCX600-X1-M4

IxiaIxNetwork

Y.1731 ETH-DM per EVC & CoS ID

Y.1731 ETH-LM per EVC & CoS ID


EricssonMINI-LINK SP 310IxNetwork

CX600-X2-M8

Maintenance association

Ixia AnueXGEM

11

Performance Monitoring per Class of Services

For the point-to-point EVC only one participantvendor successfully tested loss measurement basedon Loss Measurement Messages and Replies (LMMsand LMR): Ericsson IP Microwave (MINI-LINK PT2020 bundled to MINI-LINK SP 310).For the delay measurement on the point-to-point EVCwe added a unidirectional constant delay of 20milliseconds for CoS 0, 10 milliseconds for CoS 3and no delay for CoS 5. Our expectation was toobserve an increase average delay of 20 and 10milliseconds on CoS 0 and CoS 3 respectively andno change on the average delay for CoS 5.We also tested the delay variation by configuringthe impairment to introduce no frame delay for CoS5. For CoS 0 we added 15 milliseconds delay toevery Delay Measurement Message (DMM) and 25milliseconds to the remaining DMMs. For CoS 3, weintroduced an unidirectional frame delay of 5 milli-seconds on every seconds DMMs and 15 milli-seconds to other DMMs. We expected the averagedelay for CoS 0 and 3 to be 20 and 10 millisecondsrespectively. The following combination was successfully testedfor the frame delay and delay variation: IxiaIxNetwork, Ericsson IP Microwave (MINI-LINK PT2020 bundled to MINI-LINK SP 310), Ericsson SPO1410 + RAD MiNID, Huawei CX600-X2-M8,Huawei CX600-X1-M4. In this test case the RADMiNID, which is a Network Interface Device on anSFP, was connected to the Ericsson SPO 1410 andperformed the measurements.Some implementations could only display the valueof the maximum delay.

Figure 10: Performance Monitoring per CoS for Multipoint-to-Multipoint EVC

On a multipoint-to-multipoint EVC, the delaymeasurement was performed for a subset on UserNetwork Interfaces (UNIs) pairs. We introduced aconstant delay of 5, 10 and 15 millisecondsbetween different pairs of DUTs in the same multi-point-to-multipoint EVC service and expected theDUT to determine the delay measurement of thespecified sub-set based on the maximum value overall ordered UNI pairs. A single vendors implemen-tation performed as expected: RAD ETX-205A.Other participating vendors calculated the delayseparately for each UNI pair in the specified sub-set.We observed how these measurements provided byITU-T Y.1731 can be useful to monitor service avail-ability during modulation shifts in the Microwaveequipment.

DEMONSTRATION NETWORK

The showcase network, presented in the center ofthis document as well as in the live showcase atEthernet World in Prague, is meant to provide atangible example of a multi-vendor end-to-endMobile Network. The showcase network wasconstructed during our hot staging event based onsuccessful results depicted in the various sections inthis document.The principle guiding the showcase network isconnectivity between emulated mobile base stationsand small cells to the mobile core. In the first layer,Ethernet only solutions were used. For this reasonring topologies were used to enable add/drop cellaccess and facilitate resiliency through Ethernet RingProtection Switching (ERPS). As in the deploymentscenarios we describe in various test casesmicrowave connectivity was also used in thisnetwork area as well as a new class of distributedgrandmaster clocks.The aggregation network terminates the Ethernetaccess network into two flavour of MPLS transport:IP/MPLS and MPLS-TP. Emulated services areactually traversing both these networks towards themobile core. These services include cell site connec-tivity as well as IEEE 1588v2 PTP packets generatedby the grandmaster clock in the emulated mobile core.In our Mobile Core network typically we expect tofind Packet Gateways (P-GW), Serving Gateways(S-GW) and other support components such asHome Subscriber Server (HSS) and PCRF. Sadly, nosuch devices were available for the test, however,we did position another core element in the mobilecore: the grandmaster clock.

Huawei

Y.1731 ETH-DM per P2P EVC & CoS ID

RADETX-205A

CX600-X2-M8

HuaweiATN 910I

Y.1731 ETH-DM per MP2MP & CoS ID

Ixia

RADETX-205A

IxNetwork

RADETX-205A

Ixia ImpairNet

Ixia AnueGEMIxia Anue

GEM

Calnex Paragon-X

IxiaXGEM

IxiaXGEM


EANTC AGEuropean Advanced Networking Test Center

Hosted by:IIR Telecoms

Salzufer 1410587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 [email protected]://www.eantc.com

29 Bressenden PlaceLondon SW1E 5DR, UKTel: +44 20 7017 7483Fax: +44 20 7017 [email protected]

This report is copyright © 2013 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information containedherein. This document was edited by the EANTC team including Carsten Rossenhoevel, Jambi Ganbar, Ronsard Pene andEldad Zack, with contributions from participating vendors.All brand names and logos mentioned here are registered trademarks of their respective companies in the UnitedStates and other countries.20130920 v3

Rack View Placeholder

Documents

Public Multi-Vendor Interoperability Event White Paper · CX600-X1-M4 CX600-X2-M8 Ixia Anue 3500 Anue GEM Anue XGEM ImpairNet IxNetwork RAD ETX-205A ETX-220A MiNID Spirent Communications