View
3
Download
0
Category
Preview:
Citation preview
Carrier Ethernet WorldCongress 2011
Public Multi-VendorInteroperability Event
White Paper
2
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
EDITOR’S NOTE2011 marks the seventh yearof the Carrier Ethernet WorldCongress and its EANTCmulti-vendor Carrier Ethernetinteroperability tests. In prepa-ration of our showcase, Iwondered how muchinnovation we would be ableto witness year. Typically,vendors who were early
adopters aim to capitalize on their investment whenthe market matures, as the absence of some largebackbone equipment vendors shows this time.Substantial innovation happens in other areas thesedays, closing some gaps for Carrier Ethernet services:
• The active integration of packet-basedmicrowave solutions into Carrier Ethernetnetworks is driven by fierce competition. Wetested full-blown switching functionality withclock synchronization, Ethernet Ring ProtectionSwitching and more.
• End-to-end service activation quality assuranceprocedures is progressing to reduce the carriers‘huge provisioning cost and helping to ensureservice levels.
• Protection in provider bridging- and MPLS-TP-based access and aggregation networks,bringing them up to speed with MPLS in theaccess.
We have tested MPLS-TP since 2009. The industry isstill undecided about two alternative, incompatibleways to implement protection.As usual we asked all vendors tobring innovative solutions withnew aspects, as opposed torepeat previous showcases. Thisreduced the field to twoequipment vendors committed toBFD-based solutions. Sometimesthe difference between politicalstatements and technologyreality can be surprising.Packet-based clock synchroni-zation continues to be an activetest area. Our event provides aunique opportunity to test multi-vendor solutions in a realisticend-to-end network scenario. This time we focuses onboundary clocks, but had to deal with a number offunctional multi-vendor interoperability points.Although progress has been made, IEEE 1588:2008is still clearly a non-trivial technology.Overall the Carrier Ethernet paradigm has gainedoverwhelming support in the industry. Basic E-Lineservices are a staple worldwide. The moreadvanced concepts which enable service providersto build value-added services, permit competitivedifferentiation and reduce operational cost,however, still require care in multi-vendor environ-ments.
INTRODUCTIONSometimes diplomatic skills are required when weorganize interoperability events at EANTC. We pollvendors’ technical interests, author documentsdescribing what will be tested, provide a platform toexecute tests and when things do not work asplanned, bring the participants to the table, mitigateand help solve problems.One of the major cornerstones of our interoperabilityshowcases is that they focus on new technologysolutions and/or new products. This is why thereader will find most test cases updated and refinedcompared to previous years‘ events, or new productsbeing put to test.In 2011, our focus was on pure Ethernet transporttests. 12 out of the 14 test cases were concernedwith Ethernet testing from clock synchronization toservice activation.After months of preparation and two intense weeksof hot staging in our new offices in Berlin in whichwe created 106 different test permutations, weconcluded a successful event. As in each interopera-bility event we had long debugging sessions andmeetings to discuss test expectations and results, butthe incremental increase in implementations andadvanced features does show that Carrier Ethernet ismoving forward.
INTEROPERABILITY TEST RESULTSThe following sections of the white paper describethe test areas and results of the interoperability
event. The document generallyfollows the structure of the test plan— a document edited by EANTCand reviewed together withvendors in preparation for theevent.
Terminology. We use the term“tested” when reporting on multi-vendor interoperability tests. Theterm “demonstrated” refers toscenarios where a service orprotocol was terminated byequipment from a single vendoron both ends.
Test Equipment. In order to perform our tests wehad to generate, measure, impair, and analyzeEthernet traffic and perform synchronizationanalysis. We thank Calnex Solutions, Ixia, SpirentCommunications, Sunrise Telecom, Symmetricomand Veryx for their test equipment and supportthroughout the hot staging.
Carsten RossenhövelManaging Director
NewPhoto
TABLE OF CONTENTSParticipants and Devices ............. 3Service Activation Test................. 3Resiliency and Redundancy ......... 6Topology ................................... 8Clock Synchronisation............... 10Ethernet Microwave Tests........... 12Demonstration Network............. 14References............................... 15
3
Participants and Devices
PARTICIPANTS AND DEVICES
SERVICE ACTIVATION TESTA brand new test area this year was Ethernet serviceactivation. The test followed the ITU-T standardY.1564 “Ethernet Service Activation Test Method-ology,” a standard approved in January 2011. Thestandard provides methodologies and specific teststo allow service providers to validate that newCarrier Ethernet services coming online meet ServiceLevel Agreements (SLA) sold to the customer.An informative appendix of the standard defines anumber of tests that evaluate Frame Loss Ratio,Frame Transfer Delay, Frame Delay Variation, andAvailability service performance parameters. Theseparameters are commonly known as objectiveservice performance indicators and are also oftendescribed in SLAs. In order to meet the performanceparameters defined for a service, the provider has toconfigure certain Quality of Service parameters onthe devices used to construct the service — this isnormally where the difficulties lie.In order to support multiple service classes, a serviceprovider has to configure parameters such asCommitted Information Rate (CIR), Excess Infor-mation Rate (EIR), Committed Burst Size (CBS),Excess Burst Size (EBS) as well as color mode (color-aware or color-blind). As our testing showed, config-uring all these parameters is a challenging task sinceMEF standard nomenclature does not necessarilyalign with different vendors‘ configuration language.The test was conducted over two Ethernet VirtualPrivate Line (EVPL) services that were defined as“Services Under Tests”. In essence these were thenew services that were being activated. The Firstservice, EVPL1, was defined to operate in color-aware mode. The service was provisioned with threeBandwidth Profiles (BWP) per EVC per CoS asspecified in the MEF 10.2 standard and the MEF23.1 draft standard. The profiles were applied onthe UNIs of EVPL1, with no policing on subsequentports in the network:BWP1 – CIR 5 Mbps, CBS 16,000 bytes, EIR 0Mbps, EBS 0 bytes; The service was defined as veryimportant (we called this service Metro High) andhad one color: Green.The second bandwidth profile was used for mediumpriority traffic. Its characteristics were: CIR 10 Mbps,CBS 16,000 bytes, EIR 10 Mbps, EBS 16,000 Bytes.Here, two colors were defined: Green and Yellow.The third bandwidth profile, BWP3, was defined for
Vendor Devices
Albis Technologies ACCEED 1416ACCEED 2202
Aviat Networks Eclipse Packet Node
Brocade MLXe-4NetIron CER
Calnex Paragon-X
ECI SR9705SR9608BG9310
Ericsson MINI-LINK TNMINI-LINK LHMINI-LINK PT2010MINI-LINK CN510MINI-LINK PT6010MINI-LINK SP110MINI-LINK SP210MINI-LINK SP310SPO1460SPO1410
Extreme Networks E4G-200E4G-400
Hitachi AMN1710
Ixia IxNetworkImpairNet
Linkra WIDHOP
NEC iPASOLINK 400
Nokia Siemens Networks (NSN)
FlexiPacket MultiRadioFlexiPacket FirstMile 200FlexiPacket Hub 800
Rohde & Schwarz SITLine ETH 1G
Siklu EtherHaul-1200
SpirentCommunications
Spirent TestCenterSpirent GEMSpirent XGEMSpirent Anue 3500
Sunrise Telecom RxTSL GigE RSPDR
Symmetricom Cesium Reference CsIIITimeProvider 1500TimeProvider 5000TimeProvider 5000 EvolutionSSU 2000e
Veryx XC1000
Vendor Devices
4
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
low priority traffic and included CIR 0 Mbps, CBS 0bytes, EIR 15 Mbps, EBS 16,000 Bytes. Here onlyYellow color was defined.In order to make sure that the tests are thorough,each bandwidth profile at each end of the networkwas first separately tested to verify its configurationand performance prior to connecting the service andtesting the entire service. This procedure separatelyconfirmed the proper operation of each bandwidthprofiler. In EVPL1, the Albis ACCEED 2202 wasdefined as the customer premises equipment and theECI 9608 was defined as the Aggregation orProvider Edge device. Once the configuration of each bandwidth profilewas confirmed, a loopback test confirmed the perfor-mance of the EVC to Y.1564.
In the second, color-blind scenario, we defined asecond EVPL service with slightly different per EVCper CoS bandwidth profiles:BWP1 – CIR 15 Mbps, CBS 16,000 bytes, EIR 0Mbps, EBS 0 bytes and BWP2 – CIR 20 Mbps, CBS16,000 bytes, EIR 30 Mbps, EBS 16,000 Bytes.
We repeated the same procedure, first verifying thatthe participating vendors devices’, which includedAlbis ACCEED 1416, ECI 9608 and Ericsson MINI-LINK TN, had the correct configuration applied andthen verified that the service parameters such asFrame Loss Ratio, Frame Transfer Delay, FrameDelay Variation, and Availability reported theexpected values within the Service AcceptanceCriteria defined by Y.1564 and the Metro Class ofService Performance Tier of MEF 23.1 (under devel-opment).One of the biggest challenges of the test was toachieve a common understanding of QoS configu-ration between vendors. Since QoS architecture isnot standardized well, every vendor has its owninterpretation of QoS. For example, the committedinformation rate parameter on one device wasconfigured with an Information Rate value whileanother device configured the same parameter usingUtilized Line Rate (ULR). We also found that someimplementations did not allow the operator toconfigure burst size at the granularity we had origi-nally specified in the test plan. We actually asked for12,176 bytes of burst size, but due to some limita-tions of some of the devices we converged on avalue of 16,000 bytes. In the same configurationarea (burst size) we also found that some of theimplementations allowed a very granular configu-ration, however, the tests showed that the configu-ration was actually not enforced by the hardware.For service providers we see these initial results as apositive sign. The instrumentation and standards toverify service activation now exist and both solutionsand services can be tested before an Ethernetservice is handed over to a customer. Especially nowthat Ethernet services are becoming more and moremission critical and are replacing legacy TDMsolutions, having the standards to verify that theservices are really working as advertised.
Performance Monitoring
Once an Ethernet service has been activated,monitoring its performance in-service is one of themost desirable requirements for service providers.After all service providers sign contracts with theirbusiness customers to deliver a certain quality ofservice (defined in Service Level Agreements, SLAs).These SLAs often stipulate financial implications inthe case that the service is “out of SLA”.The metrics to perform monitoring of services isdefined in the ITU-T specification Y.1731. The speci-fication introduces techniques to measure serviceperformance objectives such as frame loss, framedelay, and frame delay variation on point-to pointEthernet services. We started interoperability testing of Y.1731 imple-mentations when the standard was young in 2008.But just around now, three years later, serviceproviders really begin to deploy performancemonitoring in production-grade Carrier Ethernetservices. Vendors brought new or updated productsto our test this time. The level of interest was so high
Color blind Test
ECI 9705
ECI 9608
AlbisACCEED 2202
AlbisACCEED 1416
AlbisACCEED 1416
EVC1
EVC2
Stream Direction
Customer Domain
Color aware and Color-Blind Test
Figure 1: Service Activation Tests
Analyzer/Traffic Generator
Loopback device
EricssonMINI-LINK TN
EricssonMINI-LINK TN
ECI 9705
ECI 9608
AlbisACCEED 2202
Carrier EthernetNetwork
SunriseSL GigE RSPDR
SunriseRxT
SunriseRxT
SunriseRxT
SunriseRxT
SunriseSL GigE RSPDR
SunriseSL GigERSPDR
SunriseSL GigERSPDR
5
Service Activation Test
that we were able to create a list of 25 vendor pairsfor the test.We distinguished between two measurement typefor the tests: Two-way frame delay and two-wayframe delay variation.We used a procedure that allowed us first to validatethat the protocol exchange between two implemen-tations worked as expected focusing on Y.1731Delay Measurement Messages and Replies (DMMsand DMRs). Once this test procedure was completedand a baseline was reached, we added animpairment generator for all further tests. We usedCalnex Paragon-X, Ixia ImpairNet and Spirent GEMimpairment generators to introduce controlled delayand delay variations so we could verify that not onlythe implementations worked with each other, butalso worked correctly. For the frame delay measurement tests we used theimpairment generators to add constant delay of20 milliseconds (ms) to all DMM packets. Weexpected that the implementations will report anaverage frame delay value of 20 ms. It was not anunrealistic expectation given that the propagationdelay in our lab approached zero. In order to verifythat the two implementations measured frame delayvariation correctly we asked the impairmentgenerator vendors to introduce packet delay of 15ms to every second DMM packet and 25 ms to theremaining packets. With this procedure weexpected to observe average frame delay of 20 ms,and frame delay variation of 10ms.The following devices successfully participated in theY.1731 delay/delay variation tests (see Figure 2):Albis ACCEED 2202, Albis ACCEED 1416,Brocade MLXe-4, ECI 9608, ECI 9705, EricssonSPO1460, Ericsson MINI-LINK SP310, IxiaIxNetwork, Spirent TestCenter.Each pair in the figure represents a vendor pairwhich constructed a Carrier Ethernet service usingVLANs (IEEE 802.1Q standard) and that havepassed the test.After we completed performance measurement testsbetween Brocade MLXe-4 and Ericsson SPO1460,as well as between Albis ACCEED 2202 and ECI9608, we inserted the Rohde & Schwarz SITLineETH 1G encryption devices and repeated the testwith the same measurements. Our goal was to verifyif the measurements could still be executed betweentwo devices over an encrypted EVPL service.We then verified that the performance measurementswere still working over the encrypted service andalso, using the delay measurements themselves, thatthe service‘s delay and delay variation was onlymarginally impacted.
Emulated PSN
Y.1731 frame delay
Y.1731 frame delay variation
EricssonSPO1460
SpirentGEMBrocade MLXe-4
Rohde & SchwarzSITLine ETH 1G
Rohde & SchwarzSITLine ETH 1G
AlbisACCEED 2202
IxiaImpairNetECI 9608
Rohde & SchwarzSITLine ETH 1G
Rohde & SchwarzSITLine ETH 1G
AlbisACCEED 2202
SpirentTestCenter
SpirentGEM
AlbisACCEED 2202
SpirentGEM ECI 9608
AlbisACCEED 1416
SpirentGEM
EricssonSPO1460
SpirentTestCenter
SpirentGEMECI 9608
EricssonMINI-LINK SP310
CalnexParagon-X ECI 9608
IxiaIxNetwork
SpirentGEMBrocade MLXe-4
EricssonSPO1460
CalnexParagon-X Brocade MLXe-4
Spirent TestCenterBrocade MLXe-4
Spirent GEMECI 9705Brocade MLXe-4
Provider/Provider Edge
Impairment tool
Aggregation
Emulator
Layer2encryption devicedevice
Figure 2: Performance Monitoring
6
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
Brocade MLXe-4 and Ixia IxNetwork also performedthe baseline performance monitoring test over100 Gigabit Ethernet (GbE) interface. As we couldnot instrument the event with a 100GbE impairmentgenerator, we had to skip the accuracy verification. Another performance measurements test that wasexecuted using the Brocade MLXe-4 over 100GbEinterface used a Virtual Private LAN Service (VPLS)service with Spirent TestCenter. Again, only the basicprotocol interoperability was verified in this caseand no delay and delay variation accuracy wasmeasured.During the performance monitoring test, weobserved three implementations for delaymeasurement. One vendor supported only on-demand delay measurement. This implementation isuseful where the delay measurement function istriggered by an operator or by an externalmanagement system. Two vendors supported scheduled measurement. Inthis case, the values of delay and delay variation arecalculated over a specific time interval. The countersused for the calculation were cleared at the end ofthe time interval, and the measurement continuedover the next interval continuously. There was also one implementation where the meanvalue of delay and delay variation could beobserved only once the DMM session was stoppedmanually. In order to follow the procedure in the testplan, a restart was required at each step. It waspossible to read the live run delay values however.We were pleased with the results we collected forthis test and believe that some of the implementationsbenefited from the opportunity to test against somany other implementations. We see that theprotocol gets used in a growing number of imple-mentations. This is a positive signal to the industrythat services could be delivered with objective andongoing quality measurements.
Hierarchical Service OAMThe intention of Hierarchical Service OAM was toverify the interoperability of Service OAM functionsbetween different vendor Maintenance AssociationEnd Points (MEPs) at multiple maintenance levels.Since no other vendor participated in this test case,Albis created a demo to show the support of EthernetAlarm Indication (ETH-AIS), and Ethernet LockedSignal (ETH-LCK) functionality in an OAM hierarchy. We verified that the ETH-AIS messages werereceived successfully at the subscriber level in casethat the transport path and the configured servicefailed. We also verified that the ETH-LCK informationwas received periodically at the subscriber level withno impact on the traffic, until the administrative lockcondition was removed. The demo was presentedsuccessfully with no issues observed.
RESILIENCY AND REDUNDANCYSome assumptions could be made these days withrespect to the way carrier packet networks aredesigned: The core of the network is likely to bebased on MPLS (with or without transport profile)and the access will be built based on availablephysical layer infrastructure. The test scenariosplanned for this area were meant to provide anoverview on the available tools for service providerto offer reliable Ethernet services to customers. Resil-ience is after all one of the five attributes thatdescribe Carrier Ethernet so we set out to investigatethe options available in the market.
Ethernet Ring Protection SwitchingEthernet has come a long way since the days inwhich resiliency was performed using spanning treeprotocols. These days the target set in the ITU-TG.8032 standard for resiliency is 50 ms — a targetas aggressive as the gold standard set by SDH.G.8032 or Ethernet Ring Protection Switching (ERPS)is a standard for fast recovery in Ethernet based ringtopologies using automatic protection switching(APS) protocol.ERPS-enabled Ethernet nodes send ring automaticprotection switching signal failure (R-APS(SF))messages as soon as they detect a failure on a ring.The failure signal causes the switch on that ring thathas one of its ports in blocking state (RPL owner) tounblock its RPL port therefore forcing a change to thering in order to bypass the failed ring segment. Thenodes adjacent to the failed link isolated the failed
Subscriber network
Service Provider network
Operator network
Site 1-Site 2 E-Line
DUTEmulator
Up-MEP
Down-MEPMaintenance Association
Subscriber ME Level
EVC ME Level
Operator ME Level
Figure 3: Hierarchical Service OAM
7
Resiliency and Redundancy
network segment by blocking their ports and trafficis then forwarded through the open RPL port.In revertive operation mode the nodes detect that thefailure is resolved and are expected to send ringautomatic protection switching no request (R-APS(NR)) message causing the RPL owner to start theWait to Restore (WTR) timer. Upon expiration of theWTR timer, the RPL owner blocks its RPL port. Nodesadjacent to the former failed link unblock their portsand the ring then resumes its original operation. Atthis point all nodes flush their Forwarding DataBases(FDBs). This whole procedure should theoreticallyoccur within 50 milliseconds. The standard allowsfor non-revertive operations, however, we did nottest this mode.
Our goal was to verify that a ring, constructed by a
number of vendor solutions, will behave asdescribed above. The assumption we made was thatthe access is exactly where carriers would require afunctioning multi-vendor rings. After all, access isoften the network area that includes switches thatwere positioned in the core of the network or newsolutions that were added at a second networkrollout phase.The following vendors participated in this test area(see Figure 4): Brocade MLXe-4, ECI BG9310,Ericsson MINI-LINK PT2010/ MINI-LINK CN510/SPO1410, Ericsson MINI-LINK SP310/MINI-LINK CN510/MINI-LINK PT2010, Extreme E4G-400, NEC iPASOLINK 400, Siklu EtherHaul-1200,and Veryx XC1000.In all test scenarios two rings were constructed: amajor ring and a sub-ring. Both were interconnectedvia a shared link. In each ring one dedicated nodewas provisioned as RPL owner. In most of the tests aRPL neighbor was configured in both major and sub-ring to block the other end of the RPL link.In order to perform the test we defined two profilesto accommodate different implementations: The firstprofile (Profile 1) was designed for vendorsallocating different VLAN ID in both ring for R-APScommunication. The second profile was intended forvendors that use the last octet of the MAC addressas RingID for the R-APS communication to distinguishthe R-APS channel in between the rings.Initially, Connectivity Fault Management (CFM) wasrequested to be used to monitor the liveliness of thering. Since vendors take different approaches torealize CFM - Some vendors used a separate VLANID for CFM and R-APS communication whereas othervendors used the same VLAN ID for both CFM andR-APS, it was difficult to find a common linklivelihood. After a long debugging session we couldnot converge on a single method and in order tomove on with the ERPS testing we decided to run thetest without CFM. Instead we triggered the failoverby pulling the cable from one of the devices.In all test scenarios, traffic was sent at 1,000frames/seconds on the service configured in the ringusing either Ixia IxNetwork, Spirent TestCenter orVeryx. We emulated link failure between the inter-connection nodes and verified the proper reaction ofthe ring.In the first test scenario, ECI BG9310 participated inthe sub-ring as ERPSv1 and all other devices partici-pated in this test scenario and in all subsequent testscenarios with their ERPSv2 implementation.The Veryx solution acted as a ring node and wasalso able to generate traffic in the ring. When wewanted to verify that the Veryx XC 1000 had the RPLport block, we realized, together with the Veryxengineers that there are no commands available onthe solution to show such port state. None the lesswe could verify, using traffic flow directionality in thering that the Veryx solution was indeed blocking theport as expected.
ExtremeE4G-400
Primary path
Port blocking
Link failure
Backup path
Figure 4: Ethernet Ring Protection Switching
Major ring
Sub-ring
ExtremeE4G-400
ECIBG9310
Ericsson MINI-LINKSP310/CN510/PT2010
SikluEtherHaul-1200
ExtremeE4G-400
VeryxXC1000
NECiPASOLINK 400
ExtremeE4G-400
ExtremeE4G-400
VeryxXC1000
NECiPASOLINK 400
BrocadeMLXe-4
Ericsson MINI-LINKPT2010/CN510
/SPO1410NEC
iPASOLINK 400
SikluEtherHaul-1200
8
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
gation
INI-LINK10
n MINI-LINK10/PT2010 M
MI
son410
CalnexParagon-X
IxiaIxNetwork
G9310
Ericsson MINI-LSP310
ECI 9608
IxiaIxNetwork
MPLS
-TP
n
E
Access device
Emulator
TOPOLOGY
Ethernet Aggre
Aviat NetworksEclipse Packet Node
Video Client
VideoSource
CalnexParagon-X
CalnexParagon-X
SunriseSL GigE RSPDR
SymmetricomSSU 2000e
SymmetricomTimeProvider 1500
PackeTime ptp
12:50:00Symmetricom
TimeProvider 5000
Ericsson MSP1
EricssoCN5
HitachiAMN1710
EricsSPO1
Evolution
AlbisACCEED 2202
IxiaImpairNet
AlbisACCEED 2202
Rohde & SchwarzSITLine ETH 1G
Traffic Generator
Ericsson MINI-LINKTN
Traffic Generator
IxiaIxNetwork
AlbisACCEED 1416
AlbisACCEED 1416
SikluEtherHaul-1200
Linkra WIDHOP
NSN FlexiPacketMicrowave
ECI B
EricssonMINI-LINK SP310
ECI 9705
12:50:00Symmetricom
TimeProvider 5000
IP/
MPLS
Ethernet Aggregatio
Connection Types
Analyzer/Traffic Generator
Device Types
Provider/Provider Edge deviceClock link
1Gbit Ethernet link
100Gbit Ethernet link
10Gbit Ethernet link
TDM linkAggregation device
Bonded SHDSL link
Topology
9
SpiGE
EricssonI-LINK SP210
Ericsson-LINK SP310
SunriseRxT
R
SpirenAnue 35
BrocadeCER
iPAS
Extreme4G-400
Eth
K
SpirenTestCent
BrocadeMLXe-4
hernet Ac
12:50:00 IEEE
Lay
Devi
TOPOLOGY
SymmetricomCesium Cslll
EricssonSPO1460
rentM
Ericsson MINI-LINKPT6010
Traffic GeneratorNECiPASOLINK 400
VeryxXC1000
SpirentAnue 3500
Traffic Generator
NECiPASOLINK 400
xpic
Ericsson MINI-LINKLH
Ericsson MINI-LINKPT2010
Traffic Generator
Aviat NetworksEclipse Packet Node
SpirentAnue 3500
ExtremeE4G-200
CalnexParagon-X
ohde & SchwarzSITLine ETH 1G
t00
NECOLINK 400
SikluerHaul-1200
ter
cess
Network Areas
Reference Clock
Impairment toolMicrowave radiosystem
Ethernet Aggregation network
MPLS-TP network
IP/MPLS network
Access network
ERPS Rings
Loopback device
1588v2 Grandmaster
er2 encryption device
ce Types
10
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
In all test scenario after pulling the cable betweeninterconnection ring nodes, we successfully verifiedthat the protection switching was triggered - RPLnode in the major ring unblocked the RPL port andthe port connected to the failed link was blocked.The RPL port in the sub-ring remained blocked. Thefailover and restoration time was asymmetric andranged from 51 to 290 milliseconds and 7 to 96milliseconds respectively.We also successfully tested the following adminis-trative commands on all major ring RPL ownerdevices: “Manual Switch” and “Force Switch” bothof which move the blocked port as desired aroundthe ring. We also tested the “Clear” command,which removes both “Manual Switch” and “ForceSwitch” commands. The failover time during theseset of tests ranged from 0 to 103 milliseconds andthe restoration ranged from 0 to 91 milliseconds.During the test we encountered several issuesranging from configuration to interoperability.Specially we were unable to perform the test usingprofile 2. Using said profile, and while emulating thelink failure on the shared link, nodes connected tothe failed link (interconnection nodes) started to sendSignal Fail (SF). After receiving this message, thesub-ring node interprets the messages and removesthe RPL block, which causes the traffic to flow on adifferent direction than expected.
MPLS-TP OAM 1:1 ProtectionOne of the areas of interest for MPLS-TP vendors hasbeen protection scenarios. For this event threevendors were ready to support MPLS-TP 1:1protection tests. Two impairment generators —Calnex Paragon-X and Spirent Anue 3500 —offered to provide impairment functions. In prepa-ration for the tests we asked vendors to bring newimplementations as usual. As a result of the poll, wefocused on BFD implementations only this time sinceall available Y.1731 implementations have beentested in our interoperability events frequently in thepast.The test scenario was straightforward from a testingperspective: Build an MPLS-TP based network using1:1 protection, fail the active link and verify that thenetwork converges in under 50 ms. This test wasperformed by statically configuring two bidirectionalLabel Switched Paths (LSP) - primary and secondary -using a single physical connection between EricssonSPO1410 and Ixia IxNetwork. BFD ContinuityCheck (CC) sessions were running on both primaryand secondary LSPs to monitor the liveliness of theLSP and the pseudowire service configured on top ofthe LSPs. BFD-CC was transmitted over the GenericAssociated Channel (G-ACh) and GenericAssociated Label (GAL). Both BFD sessions wererunning in a slow start mode of operation. Once theBFD sessions moved into the UP state the trans-mission and receive intervals changed to 3.33ms.Once both primary and secondary LSPs were estab-lished we offered load to verify that the traffic wasindeed using the primary path. We then emulated an
unidirectional link failure on the primary path bydropping all traffic and BFD packets. We verifiedthat traffic moved to the secondary path. Therecorded failover when only the Worker BFD-CCflow was impaired was 0 milliseconds. The recordedfailover when both the Worker BFD-CC and WorkerPseudowire flows were impaired was 12.4 milli-seconds. After disabling the link failure emulation thetraffic reverted back to the primary path without anyloss. The restoration time was then zero milliseconds.
Besides protection switching due to the link failure,Administrative commands “Lockout of Protection”,“force Switch normal traffic signal-to-protection”,“manual switch normal traffic signal-to-protection”were all successfully tested based on draft-ietf-mpls-tp-linear-protection-03.
CLOCK SYNCHRONISATIONEANTC started testing clock synchronization interop-erability already in early 2008. Over time we haveseen a constant increase in the number of implemen-tations and more and more positive results.Participating vendors were still interested to continueclock synchronisation tests so we obliged. We alsohear from our service provider customers that thelevel of trust in the multi-vendor interoperability of theassociated protocols (specifically IEEE 1588v2) isstill not as high as it should be. With this mandate,we set two areas of testing for the IEEE PrecisionTime Protocol (PTP or IEEE 1588v2): OrdinaryClocks and boundary clocks.
IEEE 1588 Master/Slave – Phase and Frequency SynchronizationIn certain conditions transporting clock signal over apacket based solution is the only option available forservice providers. For example, SynchronousEthernet can not be used over Ethernet servicesleased from other providers as the majority do notoffer such services. At this point a service providerrequiring synchronization services, for mobilebackhaul or other services, could use the IEEE1588v2 Precision Time Protocol (PTP). The protocolis also attractive when Time of Day and phasesynchronization is needed.
ImpairmentTool
Traffic GeneratorWorkingMPLS-TP LSP
ProtectionMPLS-TP LSP
Figure 5: MPLS-TP 1:1 Protection
CalnexParagon-X
IxiaIxNetwork
IxiaIxNetwork
EricssonSPO1410
Ethernet access
MPLS-TP networkPseudowire
Emulator
11
Clock Synchronisation
Our methodology for verifying 1588v2 focuses onprotocol interoperability as well as on clockaccuracy. The latter is only used in order to verifythat the interoperable implementations are alsodelivering useful clock information to the clients ofthe service.The measurements were performed by connectingeach slave device to a wander analyzer forfrequency, via either E1 or 2048 KHz interfaces,and a frequency counter via 1 PPS (pulse persecond) for phase (time of day, or TOD) deviationmeasurement. All test scenarios were required topass the 15ppb mask for MTIE, and a maximumtime error of 3 microseconds for the time of daydeviation.Impairment was introduced into the networkaccording to ITU-T G.8261 Test Case 12 using aCalnex Paragon-X or Spirent Anue 3500 impairmentgenerators. Frequency wander measurement wasperformed using a Spirent Anue 3500 and anothermeasurement device over a period of 4 hours ormore. Phase measurements were planned, but at theend were not tested since the slave clocks partici-pating in the test only supported frequency output.Configuration of all slave clocks was done overunicast IP/UDP on a specified VLAN, with 64 Syncmessages per second in Two-Step mode.We successfully tested two PTP slaves (see Figure 6)under impairment, the Ericsson SPO1460 and NSNFlexiPacket Hub 800, and one slave withoutimpairment emulated by Ixia IxNetwork.
In addition to successful tests, we found severalinterop issues among some vendors. These weremostly due to differing features (such as supportedsync message rates or One-Step only vs. Two-Steponly). An additional reason for the lower than expectedresults for this test was the fact that the test durationwas so long. This meant that synchronization was
lost during the measurement period, we did not haveenough time to repeat the tests.
IEEE 1588 Boundary Clocks There were two Boundary Clock scenarios run, asonly one vendor attending this event brought aBoundary Clock to be tested.Like in the Master/Slave synchronization test cases,the Symmetricom TimeProvider 5000 served as thePTP Grandmaster. Impairment was once againbased on G.8261 Test Case 12, and provided byeither an Calnex Paragon-X and Spirent Anue 3500.The Ericsson MINI-LINK SP310 served as theBoundary Clock in both scenarios tested. The SlaveClocks tested were in a second PTP domain, as aslave to the Boundary Clock. This role wassupported by an Ericsson MINI-LINK SP110 and anExtreme E4G-200 (see Figure 7). In the case of theEricsson-Extreme interop, Ericsson used the MINI-LINK PT6010 microwave as a fiber replacement inbetween the Master and Slave Clock devices.Both Boundary Clock and Slave Clock wereconfigured with 64 sync packets per second in Two-Step mode, but the VLAN and PTP Domain weredifferent on either side of the Boundary Clock torepresent a realistic real-world scenario.Frequency measurement was done with the CalnexParagon-X or Spirent Anue 3500. 1PPS (Time ofDay) measurement was done with a CalnexParagon-X or a frequency counter. Like in theMaster/Slave test case, all tests had to pass theG.823 15ppb mask or better for frequency, andhave a maximum Time of Day deviation of 3 micro-seconds or less.
Impairmenttool
Figure 6: PTP Master/Slave Clock
CalnexParagon-X
Spirent12:50:00
SymmetricomTimeProvider
EricssonSPO1460
Frequency
5000
E1/2048KHz
Ethernet link
12:50:00
PTP domain
PTP grandmaster
PTP slave
NSNFPH800Anue 3500
Analyzer
Emulator
IxiaIxNetwork
SymmetricomCsIII
Referenceclock
Impairmenttool
CalnexParagon-X
SymmetricomCsIII
Ericsson MINI-LINK SP310
FrequencyAnalyzer
PhaseAnalyzer
1PPS link
Figure 7: PTP Boundary Clock
Ericsson MINI-LINK SP110
ExtremeE4G-200
Referenceclock
12:50:00PTP Grandmaster
E1/2048KHz
Ethernet link
PTP domain 1
PTP domain 2
12:50:00SymmetricomTimeProvider
5000
Anue 3500Spirent
12
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
Ethernet Synchronization Messaging Channel (ESMC)To test ESMC, three devices were configured in alinear fashion to have varying reference clockqualities. Supplied externally to the devices arePrimary Reference Clock (PRC) and SynchronizationSupply Unit (SSU) signals. A device that does nothave a clock source connected is expected to sendDNU (Do Not Use) messages or SEC (SDHEquipment Clock), since the internal clock on mostdevices are expected to comply to SEC specifica-tions at best. Due to the clock sources attached todevices in specific locations, we will refer to thosenodes as PRC, SEC, and SSU. Clock sources were connected and disconnectedfrom each node, allowing them to adapt while wemonitored the links between them to be sure that theSSMs (Synchronization Status Messages) being sentwere changed accordingly.
Six ESMC scenarios were tested. The followingvendors all took on various roles (see Figure 8) ineach given configuration: Albis ACCEED 1416,Aviat Networks Eclipse Packet Node, Ericsson MINI-LINK LH, Ericsson MINK-LINK PT2010, Extreme
E4G-400, Ixia IxNetwork, NSN FlexiPacketMicrowave, Siklu EtherHaul-1200, and Symmet-ricom TimeProvider 5000 Evolution.The Spirent Anue 3500, Calnex Paragon-X, andSpirent TestCenter were used to evaluate theSynchronous Ethernet clock Quality Levels on theline, which were sourced from the SymmetricomCsIII.One vendor found a bug in which they were notsending messages when the device reported theywere, and one had an issue with switching to alower-priority clock source. Both vendors testedsuccessfully later with other products or bug fixes.
ETHERNET MICROWAVE TESTSOur test ideas in this area focused on Ethernetfunctionality and not on microwave physical layerfunctionality. The microwave vendors were morethan ready to demonstrate interoperability with otherswitches and routers.We understand the reason behind the inclusion ofmore and more advanced features in microwavesolutions: The microwave solutions are an integralpart of many mobile backhaul solutions and as suchhave to satisfy an extensive set of requirements.These requirements on microwave solutions couldclearly be seen in the tests in which microwavevendors exclusively participated as well as in thetests that were opened to all.We requested microwave vendors to participate asactive components in all the tests we describe belowand, as a policy, we did not focus on air interfacetesting (including capacity).
Microwave QoS SupportAs microwave systems are evolving from puretransport to a more intelligent and complex solutions,one transport capability required is support forquality of service (QoS). The nature of microwaveslink is its capacity variation that depends on weatherconditions. It is therefore often desirable, that thebandwidth is preserved for more critical data, if thelink capacity decreases.In this test we configured two classes for which wegenerated test traffic: Priority and Best Effort. Withthe BestEffort test traffic we emulated Internet traffic,with the Priority test traffic we emulated legacy TDMvoice traffic encapsulated into packets by means ofCircuit Emulation Service (CES). For each test we used two microwave systems of twodifferent vendors that were connected via anEthernet wire. Both systems were required toconfigure QoS and differentiate traffic based onEthernet Priority Code Points (PCP) - a part of IEEE802.1Q VLAN tag. We used PCP 0 for BestEffortand PCP 6 for priority class. During the test wegenerated 60 Mbit/s bidirectional IMIX (Internetmix) BestEffort traffic and 2 Mbit/s bidirectionalPriority traffic with packet size equal to 256 Bytes.During the test we changed modulation of
Figure 8: ESMC
SyncE link
Ethernet linkSyncE Node
PRC SEC SSU
ECIBG9310
Aviat EclipsePacket Node
SpirentTestCenter
ECISR9608
ExtremeE4G-400
SymmetricomCsIII
ExtremeE4G-400
IxiaIxNetwork
ExtremeE4G-400
CalnexParagon-X
E1/2048KHz
Impairment tool
SymmetricomTP 5000 Evolution
NSN FPMultiRadio
SymmetricomCsIII
SikluEtherHaul-1200
Ericsson MINI-LINK PT2010
AlbisACCEED 1416
SikluEtherHaul-1200
SikluEtherHaul-1200
Ericsson MINI-LINK LH
ReferenceClock
Symmetricom TP5000 Evolution
ExtremeE4G-400
Aviat EclipsePacket Node
EricssonMINI-LINK LH
13
Ethernet Microwave Tests
microwave links on both participating systems firstmanually then via attenuating of the microwave link -emulating bad weather condition and triggeringadaptive modulation on the systems. Our expec-tation was that we do not loose any packets from thePriority class and do loose packets of BestEffort classwhen link capacity was reduced.11 Microwave System pair combinations success-fully passed this test (see Figure 9): Aviat NetworksEclipse Packet Node and Ericsson MINI-LINK TN,Aviat Networks Eclipse Packet Node and SikluEtherHaul-1200, Aviat Networks Eclipse PacketNode and Linkra WIDHOP, Ericsson MINI-LINK TNand Siklu EtherHaul-1200, Ericsson MINI-LINK TNand Linkra WIDHOP, Ericsson MINI-LINK TN andNSN FlexiPacket Microwave, Ericsson MINI-LINKTN and NEC iPASOLINK 400, Linkra WIDHOP andSiklu EtherHaul-1200, Linkra WIDHOP and NSNFlexiPacket Microwave, Linkra WIDHOP and NECiPASOLINK 400, NEC iPASOLINK 400 and SikluEtherHaul-1200.
On the microwave links we used differentmodulation per vendor. When we emulated normalweather condition we used the following modula-tions: on the Aviat Eclipse Packet Node 256QAMwith 28MHz channel bandwidth; on the EricssonMINI-LINK TN 1024QAM with 28Mhz channelbandwidth; on the Linkra WIDHOP 256QAM with28MHz channel bandwidth; on the NEC iPASOLINK400 512QAM with 28MHz channel bandwidth; onthe NSN FlexiPacket Microwave 256QAM with28MHz channel bandwidth; on the Siklu EtherHaul-1200 64QAM with 500MHz channel bandwidth.When we emulated bad weather conditions, allvendors were stepping back to a most robustmodulation which was 4QAM with 256 MHzchannel capacity for Siklu and 4QAM with 28 MHzchannel capacity for all other vendors.On the Ericsson MINI-LINK TN and NSN FlexiPacket
Microwave combination we configured SAToPCESoIP according to RFC4553, and on the EricssonMINI-LINK TN and NEC iPASOLINK 400 combi-nation we configured CESoETH (Circuit EmulationService over Ethernet) according to MEF8. Themicrowave systems successfully converted the E1signal from and to Ethernet packets and transportedit over microwave in the priority class.
Synchronous Ethernet over MicrowaveOne of the other capabilities that we looked at forMicrowave vendors was transporting SynchronousEthernet, a physical layer technology, overmicrowave links. In that respect, the challenge for amicrowave vendor is to transform the SynchronousEthernet clock received on an Ethernet interface to aformat that could be transported across the airinterface and then recover the clock signal on theother microwave base station. At this point thesolution will convert the signal back to SynchronousEthernet and provide it further in the clock chain.As is seen in Figure 10 we defined one microwaveas a clock master, starting the clock chain and thentransporting the clock over its own microwave link,attaching to a second vendor solution using anEthernet link and then repeating the story again. Thisprovided a way to both verify the ability of thesolution to act as a clock master, slave and transportthe clock signal over two different air interfaces.Five Vendors were tested in a Synchronous EthernetMaster/Slave scenario.A PRC clock source was provided via a Symmet-ricom SSU 2000e, which also provided a referencefor measurements on a Calnex Paragon-X andSpirent Anue 3500.We waited for the master and slave clocks tosynchronize monitoring the duration the clocks tookin moving from free-running to synchronize state andthen ran measurements for a minimum of two hours.All test measurements were required to pass theG.8262 SSU Option 1 or Option 2 Mask and havea PPB (parts per billion) of under 16 nanoseconds ata sample rate of 1 second.Eight vendor pairings were successfully tested (seeFigure 10) consisting of: Aviat EclipsePacket Node,Ericsson MINI-LINK LH, Ericsson MINI-LINKPT2010/SPO1460, Ericsson MINI-LINK PT2010,NEC iPASOLINK 400, NSN FlexiPacket Microwave,and Siklu EtherHaul-1200.
Bidirectional traffic
Figure 9: Microwave QoS Support
(Priority and Best Effort classes)
Microwave radiosystem
PSN
Aviat EclipsePacket Node
NECiPASOLINK 400
Ericsson MINI-LINKTN
SikluEtherHaul-1200
NSN FlexiPacketMicrowave
Linkra WIDHOP
14
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
DEMONSTRATION NETWORKThe demonstration network was built based onselected successful results and vendor demonstra-tions. Since we could not show all results in the samenetwork topology the detailed test results aredescribed in the test sections. In the demonstration network we reflected thedifferent network technologies tested in the eventaccording to different by network areas. We tried tominic a real provider network as possible. Wecreated Ethernet Access, Aggregation Ethernet,MPLS-TP aggregation and IP/MPLS core networkareas.In the IP/MPLS area, Brocade MLXe-4, IxiaIxNetwork, and Spirent TestCenter configured IP/MPLS over 100Gbit/s Ethernet interfaces. Ixia andSpirent emulated multiple PE devices connectingthem to a VPLS service that was build with theBrocade MLXe-4 router.In the MPLS-TP area, Ericsson SPO1410, HitachiAMN1710 and Ixia IxNetwork all established MPLS-TP LSPs and pseudowire services as well as success-fully established BFD-CC and PSC sessions. TheMPLS-TP services were transporting test trafficgenerated from the emulated devices.In the aggregation area we showed two ERPS rings,synchronous Ethernet over microwave, transport ofPTP sessions from a PTP Grandmaster towards a PTPBoundary Clock over the IP/MPLS core and Ethernetaggregation network areas. We also demonstratedredistribution of the Precision Time Protocol signalfrom a boundary clock towards ordinary clocks.Another demonstration included service activationfor a service configured between ECI and Albisdevices.Based on QoS over microwave test results wecreated a chain of microwave systems that transportCircuit Emulation Services (CES), video signal, andInternet-like data in the aggregation area over 5microwave hops.We also demonstrating an encrypted EVPL trans-ported over IP/MPLS network and performancemonitoring for this service. The performancemonitoring results of Albis and ECI devices demon-strate that the encryption of the EVPL did notnegatively influence the service.
AcknowledgementsWe would like to thank Stephen Murphy from theUniversity of New Hampshire IOL for his supportduring the testing.Editors. This report has been written and edited byJambi Ganbar, Sergej Kaelberer, Stephen Murphy,Ronsard Pene, Carsten Rossenhoevel and ShimaSajjad.
Master Slave
SyncE link
E1/2048KHz link
SymmetricomSSU 2000e
FrequencyAnalyzer
ReferenceClock
Microwave systemsupporting SyncE
Figure 10: SyncE Test Results
NECiPASOLINK 400
NECiPASOLINK 400
Ericsson MINI-LINKLH
Ericsson MINI-LINKLH
SpirentAnue 3500
Ericsson MINI-LINKLH
NSN FlexiPacket
CalnexParagon-X
AviatEclipse Packet Node
AviatEclipse Packet Node
SikluEtherHaul-1200
SikluEtherHaul-1200
NECiPASOLINK 400
Ericsson MINI-LINKPT2010/SPO1460
NSN FlexiPacket
Ericsson MINI-LINKLH
SymmetricomSSU 2000e
Microwave
Microwave
NECiPASOLINK 400
Ericsson MINI-LINKPT2010
15
References
Acronyms
REFERENCES“Ethernet ring protection switching”, ITU-T G.8032version 2“MPLS Transport Profile Data Plane Architecture”,RFC 5960“A Framework for MPLS in Transport Networks”,RFC 5921“Proactive Connection Verification, Continuity Checkand Remote Defect indication for MPLS TransportProfile”, draft-ietf-mpls-tp-cc-cv-rdi“MPLS-TP Linear Protection”, draft-ietf-mpls-tp-linear-protection, work in progress“Precision Time Protocol (PTP)”, IEEE 1588-2008“Mobile Backhaul Implementation Agreement Phase2”, MEF technical specification, work in progress“Timing and Synchronization Aspects in PacketNetworks”, ITU-T G.8261/Y.1361“Precision Time Protocol Telecom Profile forfrequency synchronization”, ITU-T G.8265.1“Distribution of timing information through packetnetworks”, ITU-T G.8264“Synchronization layer functions”, ITU-T G.781“OAM Functions and Mechanisms for EthernetBased Networks”, ITU-T Y.1731“Transport Performance Metrics MIB”, RFC 4150“Service OAM Fault Management ImplementationAgreement”, MEF 30 technical specification“Ethernet ring protection switching”, ITU-T G.8032version 2“Ethernet service activation test methodology”, ITU-TY.1564“Timing characteristics of synchronous Ethernetequipment slave clock (EEC)”, ITU-T G.8262/Y.1362“Distribution of timing through packet networks”,ITU-T G.8264/Y.1364
Term Definition
APS Automatic Protection Switching
BC Boundary Clock
BFD Bidirectional Forwarding Detection
BFD-CC Bidirectional Forwarding Detection - Continuity Check
BWP Bandwidth Profiles
CBS Committed Burst Size
CC Continuity Check
CCM Continuity Check Message
CFM Connectivity Fault Management
CIR Committed Burst Size
CoS Class of Service
CPE Customer Premise Equipment
DMM Delay Measurement Message
DMR Delay Measurement Reply
DNU Do Not Use
EBS Excess Burst Size
EIR Excess Information Rate
ERPS Ethernet Ring Protection Switching
ESMC Ethernet Synchronization Messaging Channel
ETH-AIS Ethernet Alarm Indication
ETH-LCK Ethernet Locked Signal
EVC Ethernet Virtual Connection
EVPL Ethernet Virtual Private Line
FDB Forwarding DataBase
G-ACh Generic Associated Channel
GAL G-ACh Label
IEEE Institute of Electrical and Electronics Engineers
IETF Internet Engineering Task Force
ITU-T International Telecommunication Union Telecommunication Standard-ization Sector
LAN Local Area Network
LSP Label Switched Path
MA Maintenance Association
ME Maintenance Entity
MEP Maintenance Entity Point
MIP Maintenance Association Interme-diate Point
MTIE Maximum Time Interval Error
OAM Operations, Administration and Management
OC Ordinary Clock
PRC Primary Reference Clock
PTP Precision Time Protocol
R-APS Ring Automatic Protection Switching
R-APS(NR) ring automatic protection switching no request
R-APS(SF) ring automatic protection switching signal failure
RPL Ring Protection Link
SEC SONET/SDH Equipment Clock
SLA Service Level Agreement
SSM Synchronization Status Messages
SSU Synchronization Supply Unit
SyncE Synchronous Ethernet
TDEV Time DEViation
TDM Time Division Multiplexing
TIE Time Interval Error
TOD Time of Day
VPLS Virtual Private LAN Service
WTR Wait to Restore timer
Term Definition
Carrier Ethernet World Congress 2011 Multi-Vendor Interoperability Test
EANTC AGEuropean Advanced Networking Test Center
Hosted by:IIR Telecoms
Salzufer 1410587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 3180595-10info@eantc.dehttp://www.eantc.com
29 Bressenden PlaceLondon SW1E 5DR, UKTel: +44 20 7017 7483Fax: +44 20 7017 7825registration@iir-conferences.comwww.carrierethernetworld.com
This report is copyright © 2011 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information containedherein.All brand names and logos mentioned here are registered trademarks of their respective companies in the UnitedStates and other countries.20111004 v1
IntroductionInteroperability Test ResultsParticipants and DevicesService Activation TestPerformance MonitoringHierarchical Service OAM
Resiliency and RedundancyEthernet Ring Protection Switching
TopologyTopologyMPLS-TP OAM 1:1 Protection
Clock SynchronisationIEEE 1588 Master/Slave – Phase and Frequency SynchronizationIEEE 1588 Boundary Clocks
Ethernet Synchronization Messaging Channel (ESMC)Ethernet Microwave TestsMicrowave QoS SupportSynchronous Ethernet over Microwave
Demonstration NetworkAcknowledgementsAcronyms
References
Recommended