Synch in Packet Network

MEF-MBH_Synch_HaughHirdRam-Draft_101208_1.docx

December, 8 2010 Page 1 of 34 Doc Num

Synchronization for Mobile

Backhaul

A Formula for Deploying Packet Synchronization: Investigate – Test - Deploy

December, 8 2010





Contents

Abstract ......................................................................................................................................... 4 1 Introduction ............................................................................................................................ 5 2 INVESTIGATE ....................................................................................................................... 5 3 TEST .................................................................................................................................... 10

3.1 Testing IEEE 1588v2 and Synchronous Ethernet Prior to Network Deployment ........ 10 3.2 Testing IEEE 1588 and Synchronous Ethernet in the Deployed Network ................... 19

4 DEPLOY .............................................................................................................................. 23 5 Glossary ............................................................................................................................... 24 6 Summary ............................................................................................................................. 25 Appendix A – Background and Introductory Information ............................................................ 26

Evolution of Synchronization for Packet Networks .................................................................. 26 MEF22 and Synchronization ................................................................................................... 28

Appendix B – ITU-T Synchronous Ethernet and the Ethernet Synchronization Messaging Channel (ESMC) Protocol ........................................................................................................... 30 Appendix C - IEEE 1588v2 the Precision Timing Protocol (PTP) ............................................... 32



Abstract The goal of this document is to provide technical managers and network designers insight into successfully deploying packet synchronization methods for transporting packet data across their mobile backhaul networks.



1 Introduction Network operators face frequency and time distribution challenges when implementing network convergence. Ethernet interfaces will coexist with SDH/SONET interfaces as part of the network infrastructure, even as their role in the migration toward a packet-switched network (PSN) grows. Ethernet interfaces will increasingly be used to replace the SDH/SONET interfaces that are traditionally used to distribute synchronization and traffic. In mobile backhaul networks, different technology generations — 2G TDM, 3G ATM/Ethernet, 4G IP/Ethernet — are delivered over the same transport network. While offering similar end-user services, these mobile technologies have different synchronization requirements. For example, time accuracy (phase) is required in CDMA, Universal Mobile Telecommunications Systems (UMTS) Time Division Duplex (TDD), LTE TDD, and WiMAX, but it is not needed in Global System for Mobile Communications (GSM) or UMTS Frequency Division Duplex (FDD). This document takes a three step approach to aid operators as they begin to investigate, test and deploy synchronization in their packet networks. Section 2, INVESTIGATE, answers a series of questions frequently asked to identify synchronization approaches. Section 3, TEST, explores testing that can be done in the network in order to identify problems before the service goes live. Section 4, DEPLOY, summarizes the document with how the standards organizations and customers are using packet synchronization technologies. If needed by the reader, details behind these standards and synchronization technologies are included in the Appendices.

2 INVESTIGATE This section covers some of the questions that are asked when investigating the migration to Ethernet Mobile Backhaul Synchronization Why migrate to Ethernet Sync? This paper assumes the reader is familiar with the motivation to migrate to Ethernet for Mobile Backhaul and the need to maintain accurate synchronization to the base station following this migration. The paper explores some of the drivers and technical challenges in making this move but if the reader requires detailed background information on the industry and technology drivers as well as technologies such as Synchronous Ethernet and IEEE-1588v2, it is recommended that they refer first to Appendices A, B and C. What are the differences to TDM Sync? Different in nature - The TDM network and TDM equipment were designed and specified to maintain and transport accurate synchronization. Ethernet networks and equipment were not. The synchronous or plesiochronous nature of TDM networks was also more conducive to transporting synchronization than the bursty, asynchronous nature of Ethernet networks. In order to carry accurate synchronization over an Ethernet network, new protocols and technologies have been ratified by standards organizations. In addition, advanced clock recovery algorithms that can take these differences in nature into account have also been developed. TDM Sync Standards are More Evolved– A Sync network always has a tiered or hierarchical approach to deliver timing from the Core to the Access. As such, different performance recommendations have to be defined for standalone network equipment and for entire networks.



Recommendations, metrics and limits have been defined for both TDM (ITU-T G.813) and Ethernet (ITU-T G.8262) for standalone Wander performance of network equipment. Additionally, recommendations for networks were finalised and published for TDM (ITU-T G.823/824/825). With Sync-E, a combination of the equipment performance (G.8262) and the TDM G.813 recommendations can be used as guidance for network deployments. There is a potential area of confusion in that Sync-E equipment may be tested to G.8262 Output Wander specifications when in network deployments, they only need to meet the Wander Tolerance Input limits, which are less demanding. ppm or ppb? There is confusion about the fact that transport technologies such as PDH and Synchronous Ethernet are designed to meet + 4.6 parts per million (ppm) accuracy while the end applications require parts per billion (ppb) accuracy. The accuracy requirements of some mobile technologies (for the Air interface) and the backhaul network are summarized in the table below:

Mobile Technology Frequency Time / Phase

CDMA2000 ±50 ppb Goal: <3µs Must Meet: <10µs

GSM 50 ppb N/A

WCDMA 50 ppb N/A

TD-SCDMA 50 ppb 3µs inter-cell phase difference

LTE (FDD) 50 ppb N/A

LTE (TDD) 50 ppb **3µs inter-cell phase difference

LTE MBMS 50 ppb **5µs inter-cell phase difference

FemtoCell 250 ppb N/A

WiMAX (TDD)

2 ppm absolute, ~50 ppb between base stations

Typically 1 – 1.5 µs

Backhaul Network 16 ppb N/A

Table 1 Frequency and phase accuracy requirements of the base station for different mobile technologies and the backhaul network

** Under development



The backhaul network is expected to deliver frequency accuracy to 16 parts per billion, approximately three times better than the accuracy required by the base stations, shown in Table 1 above. So what does the +4.6 parts per million refer to? It actually refers to the accuracy as a standalone device or interface. In a network, the PRC is used to synchronize all elements to deliver the parts per billion (ppb) accuracy. In other words, if Synchronous Ethernet or PDH is synchronized by a PRC, they should deliver ppb accuracy. If a device or interface loses synchronization to the PRC, the standards specify that it should then still be able to deliver +4.6 ppm after 30 days. That is what ‘+4.6 ppm’ refers to – in a real network, being correctly synchronized to a PRC ensures that Ethernet or TDM can deliver ‘ppb’ accuracy. What standards are there? Standards and recommendations exist to specify the framework, the protocols as well as the performance requirements for Synchronization. ITU-T G.81x and G.82x The performance specifications for TDM networks and synchronization are contained within the ITU-T G.81x and G.82x Series of Recommendations. These specify frequency accuracy metrics of Maximum Time Interval Error (MTIE) and Time Deviation (TDEV) with limits in the order of nanoseconds, which correspond with the requirement to deliver parts per billion accuracy (ppb) mentioned above. There are different masks for the accuracy that a signal is expected to deliver. For example, there are tighter limits for a PRC than there are for a Network Interface. For more details on these metrics, refer to Section 2, Measurements and Limits. ITU-T G.826x The G.826x series broadly cover the different requirements of providing Sync over Ethernet. G.8261 describes the framework for deploying Synchronization in a packet network including recommended pre-deployment proving test cases. G.8262 included the performance requirements for Synchronous Ethernet accuracy, in parallel to the G.81x and G.82x series mentioned above. G.8264 added the Ethernet Synchronization Messaging Channel (ESMC) protocol to manage the hierarchy and deployment of Synchronous Ethernet. The development and evolution of the G.826x recommendations forms the core work of the ITU-T study group responsible for synchronization. This includes performance metrics and limits as well as deployment recommendations for Packet-based Synchronization (like IEEE 1588v2). MEF-22 MEF-22 initially described a framework for using Ethernet for Mobile Backhaul while Phase 2 of MEF-22 will add more deployment detail and recommendation, including Service Level Agreement (SLA) parameters for network performance. The MEF continues to develop this standard to refine the recommendations for deployment of Ethernet for Mobile Backhaul data and synchronization.



IEEE-1588 (2008) Also called 1588v2 or Precision Timing Protocol (PTP), the IEEE-1588 (2008) standardized a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. This protocol was adopted and further defined by the ITU-T for deployment in Telecom networks. The ITU-T G8265.1 Recommendation describes the deployment of IEEE-1588v2 in Telecom networks for frequency transfer. SyncE and/or 1588v2? This really depends on whether the application needs frequency (GSM, W-CDMA, etc.) or frequency and phase (TD-SCDMA, WiMax, etc.). Synchronous Ethernet is a physical layer technology that delivers a frequency reference. IEEE 1588v2 delivers time from which both phase and frequency can be derived.

Synchronous Ethernet cannot be deployed on legacy Ethernet networks unless the physical hardware or interfaces are upgraded. On the plus side, it is not affected by Packet Delay Variation (PDV) or Packet Jitter, inherent in Ethernet networks.

IEEE 1588v2 can be deployed on legacy Ethernet networks and equipment, but clock recovery is negatively impacted by excessive PDV.

There is growing interest in deploying both IEEE1588v2 and SyncE and also in deploying 1588v2 even when the technology only needs frequency sync. A stable SyncE frequency synchronization can be used to discipline the phase synchronization if there are 1588v2 PTP issues caused by excessive Frame Delay Variation in the network. For technologies that only require frequency synchronization, the availability of additional phase synchronization can boost the performance at inter-cell boundaries because the enhanced synchronization accuracy means less interference, hence better quality. Does it work? The technologies work and they have been proven to work. Networks across the globe have already deployed IEEE 1588v2, Synchronous Ethernet as well as vendor-specific protocols in live networks to carry synchronization over an Ethernet Mobile Backhaul network. Clock recovery algorithms are constantly being improved to deliver reliable synchronization under different network topologies and conditions. Standards and recommendations continue to be developed to give confidence to network designers and operators to migrate to these technologies. Nevertheless, there remains work to be done to truly characterize the evolution of Ethernet networks and understand their impact on Synchronization. Testing is a key part of achieving this understanding. Will it work in MY network? Implementing a comprehensive test plan prior to deployment is an important part of building confidence that a new technology will work in a live environment. This document specifies tests that should be done prior to deployment. Test tools currently exist that allow this testing to be performed. Characteristics (such as PDV) from real networks can be emulated in the lab to prove the technologies for real network deployment.



Section 3 explores testing that is done in the network to identify problems before the service goes live. These tests are also for troubleshooting when problems are experienced. As these technologies are more widely and regularly deployed, knowledge will be gained on monitoring and maintaining the accuracy and performance in the network. Will 1588v2 Transparent Clocks (TCs) or Boundary Clocks (BCs) help? There is now general acceptance in the Telecoms community that TCs and BCs are a must to deliver accurate time/phase synchronization via a packet network. Frequency synchronization has already been deployed in networks today without TC and BC functionality. However, time/phase synchronization without TCs and/or BCs is a significant challenge. The fundamental challenge to clock recovery is Packet Delay Variation (PDV). TCs and BCs effectively reduce or remove the PDV that a Slave clock has to deal with. So TCs and BCs can help accurate clock recovery in a network. An important point to note today is that 1588v2 defines the functional model for a Boundary Clock and a Transparent Clock device, but there is no performance specification defined in any Standards document. As such, test procedures are required to verify correct operation and prove the performance of TCs and BCs, and these tests are covered in Section 3.1.1 of this document.



3 TEST This section examines testing that can be done in a lab environment or network prior to turning up live service, and testing that can be done on live service already deployed in the network.

3.1 Testing IEEE 1588v2 and Synchronous Ethernet Prior to Network Deployment

Testing these technologies prior to deployment falls into two broad categories:

• Functional testing of the protocols • Performance testing to prove robustness for network deployment

3.1.1 Functional Protocol Tests

Best Clock Selection (Applicable to 1588v2 and SyncE) During the initiation phase of the network, switches must be able to choose the best clock from multiple sources and ultimately establish themselves in a hierarchy. This test case uses a test tool to simulate the protocol messages associated with various clocks, with the device under test (DUT) choosing the correct one.

The Carrier Ethernet switch under test must select the ingress clock with the highest quality level. A similar network-wide version of this test could be applied across a network of switches to establish a hierarchical loop-free tree of clocks.

1588v2 defines the Best Master Clock Algorithm The Master clock passes values that are used by the slave to determine the best master:

Priority 1 Clock Class Clock Accuracy Offset Scaled Log Variance Priority 2 Clock Identity

It is important to test these parameters and ensure the slave clock is selecting the best master. Synchronous Ethernet Quality Level (QL) SyncE passes a Quality Level (QL) in the SSM message, which is used to by each device to select the best clock source.



Clock Deterioration/Failure/Recovery (Applicable to 1588v2 and SyncE) After the initialization phase of the network and establishment of the clocking hierarchy, clocks may degrade over time due to temperature increases or outright failure. This test simulates a change in quality levels or an outright failure as expressed in the protocol messages associated with various clock sources, and ensures that the device under test switches to a backup clock accordingly.

It is important to perform a network-wide version of this test case to ensure that a new hierarchy of clocks is established across the network as whole.

Packet Message Rate (Applicable to 1588v2 and SyncE) Message rates are defined for both ESMC and PTP. As ESMC is processed as a slow protocol, the packet rate should not exceed 10 packets per second. Conversely, PTP requires a higher packet rate, and the test engineer will want to validate that PTP packets can be processed properly across the range of rates required for the network in question.

Transparent Clock (TC) Testing A Transparent Clock (TC) measures and timestamps the delay within itself and adds this value to the Correction Field in the 1588v2 packet as shown in the diagram below. The idea is that a Slave device will have the delay information of all the TC nodes that that the timing packet has passed through and therefore will have information about the Packet Delay Variation (PDV) and be able to remove this PDV from its calculation and recover the time more easily.



In order to test the performance of a Transparent Clock, the test setup shown below is used. This evaluates the accuracy of the correction factors inserted by the TCs. The PDV Meter is used to evaluate the PDV and the measured PDV in Forward and Reverse directions under different traffic loading should be close to zero when the Correction Field is used in the calculation.



Boundary Clock (BC) Testing A Boundary Clock (BC) is effectively a node that recovers timing from the incoming 1588v2 (as a Slave) and then uses this recovered timing to re-time the outgoing signal. It then acts as the 1588v2 Master for the next node. By using this mechanism, the impact of PDV is minimized as the packets will not travel through many routers or switches between Master and Slave (which would have caused PDV).

Boundary Clocks are tested in two ways. First, the PDV performance of a single node has to be measured and shown to be minimal. This is done using the test setup below.

A second category of tests addresses the fact that when Boundary Clocks are deployed in a network, they will function much like nodes in a TDM network, recovering the clock and re-clocking at every node. The tests are therefore similar to the tests for TDM Nodes as shown below. These tests evaluate the Wander output, accumulation and tolerance to make sure there will not be excessive clock inaccuracy when deployed.



3.1.2 System Performance testing

3.1.2.1 IEEE 1588v2 Performance Testing Done during the equipment system test phase or an operator pre-deployment lab trial, the performance testing of 1588v2 involves proving its robustness for network operation. The main aspect is to test the robustness of 1588v2 PTP to Frame Delay Variation (FDV). The characteristic of Frame Delay Variation in a network is key to accurate clock recovery by a 1588v2 PTP slave. The IEEE 1588v2 Performance testing creates network Frame Delay Variations and then tests that the recovered clocks are accurate and within specifications.

G.8261 – Appendix VI Published by the ITU-T in April of 2008, the G.8261 recommendation specifies a framework for transporting timing across an Ethernet network using technologies such as 1588v2. This standard also specifies (in Appendix VI) a test methodology for evaluating these technologies for network robustness. The G.8261 recommendation also acknowledges that there are different sizes of packets traversing the network, such as short packets for voice calls and longer packets for file transfers, and that these packets may get handled in different ways by the queuing algorithms in the switches. G.8261 goes on to consider the effects of congestion (i.e., packet error and packet loss) and the impact on services and timing. G.8261 Appendix VI is of particular interest to test engineers as it has the objective of emulating multiple network scenarios in the presence of different sized packets. It provides a suite of tests



to evaluate the performance of clock recovery under different kinds of network topologies, traffic characteristics and impairments.

Test cases from G.8261 Appendix VI of the G.8261 specifies 17 test cases for how to test clock recovery under various traffic characteristics, network topologies, and impairments. The test cases can be summarized as follows:

G.8261 Test Area Test Cases Adaptive Clock Recovery Tests 1-8 Differential Clock Recovery Tests 9-11 Bidirectional Tests (e.g. for 1588v2) 12-17

Traffic models G.8261 provides two traffic models for background load testing that essentially provide an IMIX-like load of different packet sizes. One model is based on 3GPP and presumes higher VoIP content (i.e., short packets), and the second model assumes a greater percentage of data traffic (i.e., longer packets). The following table describes the mixes of packet lengths: Packet Size Traffic Model 1 Traffic Model 2 Minimum Packet Size (64 bytes) 80% 30% Medium Packet Size (576 bytes) 5% 10% Maximum Packet Size (1518 bytes) 15% 60% The specification further indicates that maximum-size packets should be bursty, with bursts lasting between 0.1 and 3 seconds, and that different packet size profiles should be random.

Test Topology The original G.8261 specifies a simple linear network topology consisting of ten Gigabit Ethernet switches, or nine Gigabit Ethernet switches plus a single fast Ethernet switch, connected in a direct row, as shown in the figure below. In the case of 1588v2, a grandmaster clock (GM) is the source at the head of the network, and a slave clock is a sink at the tail end of the network. Intermediate switches are loaded with background traffic as described in the previous subsection.

Updated Test Topology There has been much discussion across the industry on the merits of the testing approach specified in the G.8261 Appendix VI. This is due primarily to the facts that a) Ethernet switches



are not designed to be testing tools and b) Ethernet switches do not provide a repeatable testing environment as there will be differences in behavior each time a test is performed. As a result, the disturbance impact on the flow under test is difficult to predict and control. The ITU recognized this issue and addressed it in G.8261 Appendix VI with the following statement:

“…in order to get a a test environment that will be simpler to be implemented and in order to remove any risk for getting different results when using Ethernet switches of different technologies, it is under discussion the proposal to replace the specification as defined in Figure VI.4/G.8261, with a new test set up where in place of the Ethernet Switches and the Traffic generator, the delay variation could be created by a test equipment with a delay variation profile as input.”

To overcome potential issues with the different behaviors of different Ethernet switches, an alternative proposal that addresses the concerns highlighted above was accepted by the ITU-T. The figure below represents the proposed revised test-bed for testing 1588v2 clock recovery equipment.

Please note that the figure is a conceptual model and does not describe a specific implementation. The Disturbance Pattern (as input to the Test Equipment) could be one of many things: a PDV (Packet Delay Variation) profile from either a file or an algorithm, a fixed latency, the introduction of Errored Packets, Repeated Packets, etc, or any combination of these effects. These effects will impair the Flow of Interest as it passes from the 1588v2 Master to the Slave under test.



Impairments Performance testing can include packet impairments (Lost, Mis-ordered, Re-ordered packets, etc.) applied to the packets used by 1588v2 for timing transfer. A network impairment device may be used to add these impairments.

Measurements & Limits A wander measurement instrument is used to compare the original clock time signature to a time signature that has passed through the network for applications that transfer frequency. Time of day (ToD) error is measured for applications that use 1588v2 to transfer time. The measurements include:

• Frequency Accuracy (Wander), based on: o TIE - time interval error o MTIE - maximum time interval error o MRTIE - maximum relative time interval error o TDEV – time deviation

• Packet Delay Variation (PDV) • Time of Day (ToD) error

Time interval error is the time difference between the network reference clock and a measured clock signal after a period of S seconds.. The measurement limits for Frequency Accuracy (MTIE/TDEV) are specified by the relevant ITU-T standards that specify clock accuracy, for example G,813, G.823, G.824, etc. The measurement limits for Packet Delay Variation are the subject of extensive work being done by the ITU-T Study Group responsible for the deployment of 1588v2 in networks. This group is working on developing limits and metrics in the packet domain under the G.8260 recommendation, to be published in the near future. The measurement limits for ToD errors are specified by the requirements of the end-application. For example, most wireless technologies that use ToD to deliver phase synchronization have a requirement of +3µs phase accuracy between adjacent base stations. Therefore a single base station must be synchronized to better than 1.5µs, which becomes the measurement limit for ToD test.

3.1.2.2 Synchronous Ethernet Performance Testing The testing of SyncE performance is governed by the ITU-T G.8262 recommendation, which specifies all test methods and measurement limits. These tests evaluate the accuracy of frequency synchronization delivered by SyncE equipment and networks. It also tests the tolerance of this equipment to network wander and their contribution to the total network performance. An application note describing these tests called “Testing SyncE Wander to ITU-T G.8262” can be downloaded from the Calnex Solutions website (www.calnexsol.com). The test setup is shown in the diagram below:



Figure 1 Test Cases for 1588v2 and SyncE

PTP Device Types When testing the IEEE 1588v2 standard it is important to define the types of devices to test:

• Ordinary Clock (Can be a Master or Slave clock) • Boundary Clock (Acts as a Master and a Slave clock) • End-to-end Transparent Clock • Peer-to-peer Transparent Clock • Management Node

There are specific test challenges for each of these devices.

PTP Test Case Configuration Options There are several PTP modes and options that are applicable to all test cases:

• Transport Type o Ethernet, IPv4, IPv6 and others

• Unicast, Multicast or mixed modes o Unicast can be signaled or static

• One-step or Two-step (Two-step uses the optional Follow-up message) • Message rates (Defined in rates from256 per second to lower rates in multiples of 2))

o Sync Message with optional Follow-up o Delay Request Message o Announce Message

Equipment vendors need to test all of the options and combinations supported. Service providers will most likely test the modes and options that they plan to use in their production network.



Since the focus of this document is on the Mobile Backhaul application, most companies will refer to the telecom profile framework, which has been defined by the ITU-T in G.8265.

ESMC Configuration Options

ESMC only runs over Ethernet on a point-to-point link. There are minimal ESMC configuration options which affect the Synchronization Status Messages:

• Quality Level • Message Rate

Additional Test Cases • PTP scalability (testing the number of slave clocks that can be supported with various

message rates) • Multiple clock-domain scalability and inter-domain interactions • Testing with other Layer 2 or Layer 3 traffic

o QoS testing verifying PTP has the proper class of service • Testing with other protocols concurrently

o Layer 2 protocols include: STP, LACP and OAM o Layer 3 protocols include: OSPF, BGP, LDP and others

• Negative testing: o Impairing/changing the message rates o Dropping protocol messages o Abnormal protocol packets o Message disabling

3.2 Testing IEEE 1588 and Synchronous Ethernet in the Deployed Network

Accurate synchronization of base stations to nanosecond accuracy is critical to minimize service disruptions and eliminate dropped connections as calls move between adjacent cells. Highly accurate synchronization also ensures that the radio spectrum is not spread into the adjacent channels. Without stringent phase synchronization, the multiple signals in LTE’s multiple-input/multiple-output (MIMO) architecture can simply cancel one another out. Without accurate synchronization, the mobile technologies will not work to specifications, resulting in failed call setups, releases, handovers, and other network issues. IEEE 1588v2 enables the precise transfer of frequency and time to synchronize clocks over Ethernet networks. It uses time stamping, with sub-nanoseconds granularity, to deliver very high accuracies of synchronization needed to ensure the stability of base station frequency and handovers.

Testing IEEE 1588v2 and Synchronous Ethernet in the Deployed Network



In the past, TDM (E1/T1 backhaul links) and GPS were used to synchronize the base stations. With the migration to Ethernet/IP backhaul, and concerns about relying on GPS, technologies like IEEE 1588v2 and SyncE have been developed to deliver frequency synchronization and (in the case of 1588v2) time synchronization over Ethernet. As we have seen previously, in addition to the technology framework, a Telecom Profile has been defined for IEEE 1588v2 for its use in telecommunications networks and a management protocol has been defined for SyncE (the Ethernet Synchronization Monitoring Channel, ESMC) for its deployment in a telecoms infrastructure. Wireless technologies such as GSM, W-CDMA and LTE-FDD require frequency synchronization, which has been traditionally delivered by a TDM backhaul. Other wireless technologies (WiMAX, LTE-TDD and TD-SCDMA) require both phase and frequency synchronization. While phase can be delivered by GPS, with Ethernet it can be delivered by IEEE-1588v2, which is also known as Precison Time Protocol (PTP). PTP provides Time of Day (ToD), which is used to synchronize the phase. Consequently, PTP delivers both frequency and phase synchronization with suitable accuracy for carrier class mobile backhaul. In order to ensure that the new Ethernet technologies deliver synchronization to the same or better accuracy compared to TDM and GPS, they must be tested to the same ITU-T limits using specialized test equipment designed for this purpose. To test the clock signals delivered to the base stations, several specifications can be referenced to determine performance limits. E1/T1 signals are tested to the limits specified in the ITU-T G.823 and G.824 recommendations for frequency synchronization. Other recovered frequencies (8kHz, 10MHz, etc.), can be tested to limits that are adapted from and traceable to the same recommendations. Time synchronization (ToD) is delivered to the base station via a 1pps (one pulse per second) signal and a timestamp. There are several formats currently used for this timestamp and measuring its accuracy requires decoding the appropriate format. However, in order to determine accuracy in the delivered time, a tester only has to measure the 1pps signal as this indicates the Top of Second (ToS) or the start of a second on a wall-clock. The error in the ToS is the error of the delivered time. The limits are determined by the requirement of the end-application. For example 1.5µs for TD-SCDMA which will ensure the 3µs inter-cell phase difference required by the technology. Apart from the recovered clock signals, the mechanisms of clock delivery (namely IEEE 1588v2 and SyncE) could also be tested for performance. When the recovered clock does not meet specifications, IEEE 1588v2 and SyncE should be tested to identify the source of the failure. Measuring both the Ethernet delivery mechanisms as well as the recovered clock correlates the performance in the Ethernet network with the accuracy of the recovered clock. For SyncE, the performance limits have been defined under the ITU-T G.8262 recommendation, while the management protocol is specified in ITU-T G.8264. Testing SyncE therefore involves testing against the limits and specifications in both ITU-T G.8262 and G.8264. For IEEE 1588v2, the ITU-T Study Group (Group 15 Question 13) is working to define metrics that will deliver the frequency and time accuracies specified above. These are generically defined as1588v2 Packet Metrics used in the test case examples are shown in Section 2.1..



The IEEE 1588v2 Certification program and the ISPCS 1588 Plugfest (www.ispcs.org ) covered in the previous section will also use limits and metrics discussed in this section. The following provides an overview of testing both PTP (IEEE-1588) and SyncE in the network to ensure performance is within the required limits:

Figure 2 IEEE-1588 and recovered clock tests

1. For frequency synchronization, measure the recovered frequency E1/T1 to determine its accuracy as defined by ITU-T G.823/G.824. Other recovered frequencies can be tested to the limits traceable to the same recommendations.

2. For measuring the time transfer accuracy, measure the recovered 1pps and check against the relevant limits (e.g. 1.5µs for TD-SCDMA).

3. It can also be valuable to determine the bi-directional Packet Delay Variation (PDV) of the IEEE1588 messages.. Measure the Forward (SYNC) PDV and Reverse (DELAY_REQ) PDV. Then, apply the 1588v2 Packet Metrics to determine the impact of the network PDV on the quality of the recovered clock.

4. Any failures found in steps 1) and 2) should be correlated with the PDV measurements found in step 3) to assist in the troubleshooting and for future network design and baselining.



Figure 3 Synchronous Ethernet and recovered clock tests

1. As described in Section 3.1.1, measure the recovered frequency for Frequency transfer accuracy specified to the ITU-T G.823/824 limits. Other recovered frequencies can be tested to the limits traceable to the same recommendations. 2. The SyncE networked should also be measured for wander and jitter to ensure the performance is within the ITU-T G.8262 limits. 3. Monitor the ESMC flow within the SyncE network and capture the SSM messages and monitor correct clock quality level (QL) as well as clock switching performance (if present).



4 DEPLOY When Carrier Ethernet was introduced as the most cost effective technology for mobile backhaul a few years ago, operators were concerned about migrating from the TDM networks that were already deployed. A significant area of concern was synchronization, including the robustness of technologies like IEEE 1588v2 and SyncE. The industry has progressed a long way since then. First of all, standards have been written and continue to be written to provide guidelines for implementing Carrier Ethernet for Mobile backhaul. MEF 22 was developed to provide such guidelines, and the ITU-T G.826x series of recommendations addresses the frameworks and metrics required to design, deploy and test synchronization over Carrier Ethernet. Furthermore, designers of Ethernet synchronisation equipment are learning more about the behavior of networks and their impact on clock recovery. This allows the designers to improve their designs and add more robustness for different Ethernet network conditions. Finally, network equipment vendors are reducing their support for legacy TDM interfaces on their gear, requiring the network operator to migrate to Ethernet for their network expansion. So which technology should be deployed? One size certainly doesn’t fit all and it is likely that a mobile network will deploy a combination of Synchronization technologies. TDM or GPS may be maintained in a minority of sites where it is the only option. Increasingly, networks and services will require not just frequency but also accurate phase synchronization. There are only two options to deliver phase – GPS or packet-based timing like 1588v2. Given the significant concern with cost and security of GPS, the growth in deployment of 1588v2 in Ethernet Mobile Backhaul networks is inevitable. There is growing interest in deploying both 1588v2 and SyncE so that the stable SyncE frequency synchronization can be used to discipline the phase synchronization if there are 1588v2 PTP issues caused by excessive Frame Delay Variation in the network. Also, even if the end-application only needs frequency synchronisation, providing an accurate phase reference can enhance radio performance and call and handover quality. There is also a fair amount of discussion on debate on how a network is deployed. Where should the Grandmasters be located? How many hops between Grandmaster and Slave? How many Slaves can a Master support? These questions will have to be answered by experimentation and trial. By conducting thorough lab testing and network testing as recommended in this document, an operator can establish the guidelines for deployment of IEEE 1588v2 and/or Sync-E in their network. Finally, it is now accepted that deployments of IEEE 1588v2 in networks to deliver time and/or phase will require the use of Transparent Clocks (TCs) and Boundary Clocks (BCs). Once the performance of these devices have been characterised, they can be deployed according to the same planning guidelines (based on ITU-T recommendations) currently used to deploy SDH/Sonet networks.



In conclusion, the IEEE, the ITU-T and the MEF provide mobile backhaul specifications that define packet network backhaul for mobile radio access networks. Many operators are using these guidelines successfully in their networks, including migration paths for packet synchronization. This document supplements MEF22 to provide design, testing and deployment insights on synchronization methods. 5 Glossary

BC – Boundary Clock CDMA - code division multiple access DUT – device under test EEC - Ethernet Equipment Clocks ESMC - Ethernet Synchronization Messaging Channel FDD - Frequency Division Duplex FDV - Frame Delay Variation GM – grandmaster clock GPS Global Positioning System GSM - Global System for Mobile Communications LTE – Long Term Evolution MBMS - Multimedia Broadcast Multicast Service MIMO - multiple-input/multiple-output MTIE - Maximum Time Interval Error PDH - Plesiochronous Digital Hierarchy PDV – Packet Delay Variation ppb – parts per billion ppm – parts per million PRC – primary reference clock



PSN – packet-switched network PTP – point to point QL – quality level SSM - synchronization status messages TC – Transparent Clock TDD - Time Division Duplex TDEV – Time deviation TDM - Time Division Multiplexing ToD – Time of Day ToS- Top of Second UMTS - Universal Mobile Telecommunications Systems 6 Summary

This document provides guidance to technical managers and network designers on key issues to investigate and test before successfully deploying packet synchronization methods across their mobile backhaul networks. Investigate There is broad acceptance that there are strong drivers to migrate to Ethernet-based Synchronization. Protocols like SyncE and IEEE 1588v2/PTP have been formulated and developed to deliver timing over Ethernet. Operators all over the world have been tasked with investigating the timescales for migrating traditional TDM backhaul networks to Ethernet. Test The industry has developed lab tests and equipment to validate and prove functionality, robustness and performance of 1588v2 and SyncE. When the technologies are deployed in a network, measurements must be made to ensure performance within limits. Bodies like the MEF and the ITU-T are continuing their efforts to specify further metrics and limits that will enable network equipment and networks to be designed and tested to a common set of rules. Deploy Networks have already been deployed and are up and running with SyncE and 1588v2. Some operators, like a large service provider in China, have implemented both protocols using SyncE for frequency and 1588v2 for ToD. Following the test guidelines covered in this document, operators can deploy these technologies with increased confidence.



Appendix A – Background and Introductory Information A carrier-class transport network must be able to meet the synchronization requirements of all the services it supports, regardless of the TDM or packet-based technologies it uses to transport and deliver these services. In addition to the technology interfaces already mentioned, a variety of dedicated synchronization interfaces are defined in standards: primary reference clock (PRC), building integrated time source (BITS) (T1, E1, 2 MHz, 6 MHz), one pulse per second (1PPS), and time of day (ToD), as listed in Table 2.

Physical Layer Packet Layer UNI/NNI SDH/SONET, SyncE, PDH IEEE 1588v2 (PTP), NTP Dedicated synchronization interfaces

T1, E1, 2MHz, 6MHz, 1PPS NTP, ToD

Table 2 Synchronization technologies and interfaces NNI network node interface NTP Network Time Protocol PDH Plesiochronous Digital Hierarchy PTP Precision Timing Protocol SyncE Synchronous Ethernet UNI user network interface Evolution of Synchronization for Packet Networks The techniques normally used for synchronization in TDM networks are E1/T1 (PDH), SDH/SONET, GPS and 1PPS+ToD. 2G cell sites are connected to the backhaul network using E1/T1 connectivity. Operators that have embedded TDM networks will maintain some T1/E1s in their network to maintain synchronicity. The same operators can deliver data services over a parallel, asynchronous full packet network. Ethernet connects these 3G/4G cell sites to the packet backhaul network. The operator can then deploy services in hybrid mode, keeping voice traffic on the TDM network and data services on the packet network (evidence of this trend is in the February 2010 Heavy Reading Ethernet Backhaul Quarterly Tracker report), which indicates that T1/E1 line synchronization will remain popular with operators at least through 2012. Figure 1 illustrates how new IP/Ethernet base stations can be deployed during both the transformation from TDM to packet transport and in hybrid networks, while existing networks continue seamless synchronization of SONET/SDH and Ethernet in mixed domains. SDH/SONET can provide both transport and synchronization across some parts of an evolving hybrid network, while SyncE provides identical functions in other parts of the same network. This maintains end-to-end synchronization across such networks by supporting both SyncE and SDH/SONET, and seamless interworking between them in any combination for network synchronization.



Figure 4 Synchronization Distribution for TDM and Ethernet Hybrid operators can deploy SyncE and IEEE 1588 as they begin to move voice traffic from the TDM to packet-based networks. TDM circuit emulation services (CES) are transported across packet-based networks. Synchronization is essential to support frequency accuracy and stability. Lack of stability (wander and jitter) or accuracy (frequency offset) will cause bit errors and/or frame slips (underflows and overflows of frame buffers), resulting in lost packets in the PDH framing, severely affecting TDM traffic performance. Figure 2 shows an evolutionary approach to timing options as networks grow and evolve from TDM to packet-based infrastructures. Variety, flexibility, and interoperability of synchronization tools provide options for a smooth evolution.



Figure 5 Evolution of Network Timing from TDM to Carrier Ethernet

Interoperability with an Ethernet over SONET (EoS) network requires using TDM circuit emulation service (CES) via a CES gateway. IEEE 1588v2 is required across the EoS cloud to maintain time of day (ToD). As Synchronous Ethernet (SyncE) is implemented in the overlay Carrier Ethernet network, the operator still supports the TDM CES traffic but uses SyncE and IEEE 1588. SyncE is used both to recover and distribute frequency at the physical layer in Ethernet-based networks, using the same principles that were developed for synchronization based on SDH/SONET. In the final stage of evolution, the full packet Carrier Ethernet network supports all traffic, including TDM CES using SyncE and 1588v2 for ToD.

Synchronization evolution is not an issue for operators in a greenfield environment. These deployments use SyncE and IEEE 1588v2 packet synchronization methods on their all-packet networks in order to support voice and data services with TDM-like quality.

MEF22 and Synchronization

Phase 2 of MEF22, the Mobile Backhaul Implementation Agreement, will develop guidelines on synchronization methods and focus on frequency methods – including how the Carrier Ethernet network operator can offer a Synchronization Service with a PRC traceable frequency reference towards the Mobile Operator’s RAN BS sites. The focus will be on SyncE and IEEE 1588.

MEF 22 Phase 2 will also recommend performance limits for different Class of Service (CoS) implementations, with Synchronization packets assigned the highest CoS for packet-based technologies, such as IEEE-1588v2. While MEF22 will help define performance limits for a Mobile Backhaul network, IEEE 1588v2 will have additional performance requirements that need to be addressed.



The fundamental issue is that most performance limits today only specify a single limit for Frame Delay (or Latency) and a single limit for Frame Delay Variation (or Packet Delay Variation or Jitter). Clock recovery using IEEE 1588v2 depends not only on the total amount of Delay Variation but more so on how this Delay Variation changes. So it is insufficient to specify a maximum limit for Delay Variation. This is illustrated in the diagram below.

These show two different graphs of Frame Delay Variation over time. They both meet a Delay Variation maximum limit of 2ms, in this example. However, the 2 conditions will affect IEEE 1588v2 clock recovery completely differently. Clock recovery may be fine under one condition but not the other.

So what is required longer-term is the ability to specify the performance of how Frame Delay Variation changes over time. To this end, the ITU-T Study Group responsible for the G.826x packet synchronization recommendations is working on metrics, which will be incorporated into the G.8260 recommendation. These metrics will specify limits on how the Frame Delay Variation can change over time and address the issue illustrated above.

Once the ITU-T metrics are finalized, these will be incorporated into an evolution of MEF22.

Meets 2ms limit and recovered clock OK

Meets 2ms limit and recovered clock FAILS



Appendix B – ITU-T Synchronous Ethernet and the Ethernet Synchronization Messaging Channel (ESMC) Protocol

The purpose of SyncE is to distribute frequency information through an Ethernet device. In this context “frequency” and “bit-rate” effectively have the same meaning, since SyncE attempts to determine frequency information from incoming bit streams. The basic operation of SyncE interfaces is deriving the frequency from the received bit stream and passing that information up to the system clock. The system clock in a switch or a router is an actual oscillator whose output is used to clock source for all the SyncE outputs from the device. Most importantly, the output frequency of the transmit interfaces are locked on to the system clock. Ethernet without SyncE interfaces only achieves a frequency accuracy of 100ppm (parts per million) in free-running or holdover conditions (without being locked to a reference). With SyncE interfaces, accuracy is improved to 4.6ppm in free-running or holdover conditions. In normal operation, the SyncE line rate should be frequency-locked to the system master reference, which should deliver parts-per-billion accuracy. It also provides frequency traceability through a network due to the outputs of each device being frequency-locked to the input selected as the master reference.

While SyncE can distribute frequency, it cannot be used to distribute time-of-day. SyncE requires upgrading line cards in existing equipment, and potentially a change to the clock distribution within the device. Requirements for SyncE clocks – also known as Ethernet Equipment Clocks (EEC) – are specified in the G.8262 recommendation. SyncE follows the PDH and SONET/SDH telecom examples, where clock information is derived from the incoming bitstream.

The Ethernet Synchronization Messaging Channel (ESMC) protocol is used to determine which port should be used as reference clock source. ESMC is based on the Organization Specific Slow Protocol (OSSP), which is used in conjunction with SyncE to determine clock selection, clock management, quality traceability, and failover. ESMC uses synchronization status messages (SSM) of various types to achieve these objectives. SSM messages are sent at a rate 10 pps (packets per second). ESMC and SSM message formats are defined in ITU-T recommendation G.8264.

A switch in a network receives clocking information in the form of SSM messages from various ingress ports, and the switch must decide which port has the highest quality. SSM messages contain a “quality level” for exactly that purpose. The algorithm selects the ingress port with the highest quality clock, which is selected as the reference clock.

As with the basic clock extraction method, SSM messages in SyncE are derived much like their counterparts in the SONET and SDH telecom worlds and provide two options for interworking. EEC-Option 1 is optimized for hierarchy based on 2048 kb/s (commonly referred to as E1), and EEC-Option 2 is optimized for a hierarchy based on 1544 kb/s (commonly referred to as T1).

SSM messages are broadly grouped into two types: events and information. All messages contain the quality level (QL) field, more precisely referred to as a TLV. A single bit, the Event flag, is used to distinguish event PDUs from information PDUs. Event PDUs are sent when there is a change in the quality level.



The structure of SSM messages is as shown in the table below:

Octets Size Field Description 1-6 6 octets Destination address = 01-80-C2-00-00-02 (hex) 7-12 6 octets Source address 13-14 2 octets Slow protocol Ethertype = 88-09 (hex) 15 1 octet Slow protocol subtype = 0A (hex) 16-18 3 octets ITU-OUI = 00-19-A7 (hex) 19-20 2 octets ITU Subtype 21 4 bits Version 1 bit Event flag 3 bits Reserved 22-24 3 octets Reserved 25-28 4 octets QL-TLV (type, length, reserve, SSM) 29-1532 32-1486

octets Future enhancement TLVs and padding

Last 4 4 octets FCS

The selection mechanism is based on quality levels and results in a hierarchy of masters and slaves, with all clocking in a network traceable to a primary reference clock (PRC). The diagram below shows a simple example of such a master/slave network as outlined in G.8261.

Note that an alternate architecture is used to distribute primary reference clocks, such as GPSs, to each device in the network – which can be a more costly approach than either SyncE or packet-based approaches like IEEE 1588v2.



Appendix C - IEEE 1588v2 the Precision Timing Protocol (PTP)

The IEEE 1588-2008 (referred to as 1588v2) standard specifies the Precision Timing Protocol (PTP) for network synchronization. PTP uniquely solves all three synchronization problems (frequency, phase, and time-of-day distribution). PTP was introduced to overcome the poor accuracy available with NTP (network time protocol) and the high cost associated with GPS (global positioning system) technology. Being a packet-based solution, PTP is susceptible to packet-based impairments such as packet delay variation (PDV). PTP exists for both unicast and multicast applications.

Unlike SyncE – which derives its clocking information from the incoming bitstream – 1588v2 is a purely packet-based solution, with the actual clock values being passed inside the payloads of special packets dedicated to that task. The packets, or messages, sent by PTP are classified broadly into either event messages or general messages.

There are four specific event messages:

Event Message Type Purpose Sync Used to generate and communicate timing

information Delay_Req Pdelay_Req Used to measure delay information between

two ports Pdelay_Resp

There are five types of general messages:

General Message Type Purpose Announce Used to establish synchronization hierarchy Follow-up Used to generate and communicate timing

information Delay_Resp Pdelay_Resp Used to measure delay information between

two ports Management Used to query and update datasets Signaling Used for all other purposes, e.g. rate

negotiation

1588v2 establishes a master-slave hierarchy of clocks in a network, with all clocks ultimately synchronized to a grandmaster clock acting as the primary time source. The Announce messages are used to establish a hierarchy. 1588v2 also separates networks into regions, with one master clock per region used as the primary time source for all clocks in that region.

1588v2 also defines four types of clocks, and one additional device type:

Ordinary clock: a PTP clock with only one PTP port Boundary clock: a PTP clock with more than one PTP port that acts as both a master

and slave clock



Transparent clock: a PTP clock that forwards all messages and adjusts them to reflect the residency time associated with the Sync and Delay_Req as they pass through this device. The delays are inserted in the Correction Field.

Management node: a P2P device with multiple ports and a human or programmatic interface

Note: Clocks can implement End-to-End or Peer-to-Peer Delay mechanism. End-to-end clocks measure and utilise the end-to-end delays while for peer-to-peer, the delay is measured section by section of the clock path.

All clocks in a network are either boundary or ordinary clocks. The diagram below shows a 1588v2 network with ordinary clocks (master & slaves) and boundary clocks.

Figure 6 IEEE 1588v2 Clock Hierarchy 1588v2 uses the “Best Master Clock” algorithm to select the best clock. The algorithm compares data sets associated with each candidate clock. The algorithm attempts to choose a clock from a better grandmaster, taking into account such things as traceability, number of steps removed, and the quality of the grandmaster clock. Besides establishing the master-slave hierarchy, the other primary task of PTP is to synchronize the clocks throughout the network. The basic challenge is that across each link, packets experience some delay, all of which must be used to adjust the clock’s value as it is propagated through the network. To determine delay information, 1588v2 uses a simple technique to effectively calculate the one-way transit time of simple stimulus and response, and then assumes that the network delays are symmetric in order to calculate the transit time through the device. The transient delay, once determined, is then added to the ingress clock value for the outgoing clock value. A Boundary clock can be considered as performing both the slave and master function. The 1588v2 message flow is terminated and the frequency and/or time recovered from the ingress messages. This is used to guide its internal reference. The boundary clock then acts as the



master to the downstream clock by generating a new 1588v2 message flow using timestamps generated from its internal reference. A second version (informally known as 1588v2) was finalized in 2008, and included the following key enhancements:

Addition of transparent clocks. The purpose of transit clocks is to improve synchronization accuracy between slave and master clocks in a network. As the name implies, such clocks are invisible to the clocks immediately upstream and downstream from the transparent clock. Timing transparency is achieved in transparent clocks by modifying the Delay_Resp, Sync, and/or Follow-up messages as they transit through a switch to account for the receive and transmit delays through a device.

Shorter sync messages. The first version of the specification used sync messages which were 165 bytes long. Version 2 shortened sync messages to 44 bytes in order to speed up processing, and broke out some of the functionality into separate Announce messages for use in hierarchy establishment.

Varied (faster) update rates. This change goes hand-in-hand with the shorter sync message change to the specification. Shorter messages are easier to process, and therefore can be sent more frequently without overloading processors. The original spec sent messages at a rate of once every 30 seconds, while version 2 sends messages up to 256 times per second.

Protocol support for improved precision. Layer 2 transport option. The messages listed in the two tables above (Event

Messages and General Messages) are deliberately not described as “frames” or “packets” because 1588 allows transport of these messages using a number of underlying technologies. Within telecom networks, 1588 messages will likely be used over UDP/IP, but the 1588 specification also supports messaging directly over Ethernet and Ethernet-based technologies for industrial applications such as factory automation.

Unicast transport option. The original IEEE 1588 specification was targeted more at industrial applications, which mapped well to the use of multicast IP technology. However, for telecom wireless backhaul applications, unicast is required to optimize network performance. The second version of the IEEE 1588 specification includes support for unicast by allowing receiving ports to optionally request to upstream ports that Announce, Sync, Delay_Resp and Pdelay_Resp messages be sent to over unicast rather than multicast.

Fault tolerance support. IEEE 1588v2 allows the occasional missed message, and does not interpret a missed message as necessitating a dramatic change to the clocking hierarchy.

Rapid reconfiguration in response to network changes. When a clock fails, PTP automatically reconfigures the network. Similar to the concept of spanning trees, the clocking hierarchy is an overlay of a loop-free tree on top of more connected (e.g. mesh) topology. If a clock fails somewhere in the hierarchy, the clocking topology can reconfigure itself to a new, loop-free topology.

Documents

Synch in Packet Network