Service Quality Management for VoIP ApplicationsVoice over IP (VoIP) is fast becoming more than hype. The primary use for VoIP a few years ago was to offer affordable long distance

Service Quality Management for VoIP Applications

Robin L. Zak

March 4, 2003

This report is confidential and proprietary property of Spirent Communications. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any means, electronic or mechanical, without the prior written permission of Spirent Communications, Rockville, MD.

Page 1

Introduction Voice over IP (VoIP) is fast becoming more than hype. The primary use for VoIP a few years ago was to offer affordable long distance service for consumers. Many startup companies providing pre-paid calling card services and free Internet voice services have come and gone. VoIP has taken the next step in deployment to deliver business-class services. Enterprises and providers are deploying systems today in order to obtain the benefits of convergence. The technology and products have stabilized such that they are reliable, interoperable, and cost-effective. Voice quality issues have been addressed, and all this momentum is pushing VoIP to the forefront of the communications industry.

Still one question remains: How Can We Manage VoIP Services?

This is usually one of the last questions to be answered. After the equipment is deployed and subscribers sign up for the service, then the management issues are uncovered and providers scramble for quick fixes. VoIP applications require service quality management (SQM) which includes the following functions:

• Prequalification Testing

• Provisioning Assurance

• Performance Monitoring

• Fault Management, Diagnostics, and Test

• Repair Assurance

SQM can be defined as the ability to monitor and manage the network to ensure a defined service level, regardless of the technology, service, protocol, or vendor. Employing the appropriate tools will enable all aspects of SQM for VoIP. This paper identifies the proper placement of these tools to provide complete SQM for the various VoIP applications.


Page 2

VoIP Carrier Applications Description Providers offer many different VoIP services. For the purposes of this paper, we address only the “real-time” or “conversational” aspects of VoIP. We will consider both consumer-oriented services and business-oriented services.

Consumer-Oriented VoIP Services

Second Line Service

From the residential subscriber perspective, VoIP may be used as the vehicle to deliver POTS-like service to the home, which may be done via Cable providers over the HFC plant (Figure 1) or via Local Exchange Carriers (LECs) over the traditional copper plant using DSL technology (Figure 2). The subscribers’ handset may or may not change, depending on where the conversion to IP takes place. One of the challenges of providing VoIP directly to the subscriber is the ability to deliver lifeline services such as identifying a caller’s location. More development is required to track and report caller locations for E-911 calls placed from an IP phone because of the complexity of mapping an IP address to a physical location, and the routing of emergency calls to the “local” public safety answering point (PSAP). Issues also arise with providing services during power outages. IP services require an external power source for an IP Phone or modem; no power means no phone service. Traditional Plain Old Telephone Service (POTS) does not require external power because the power is provided “in-line” directly through the copper that carries the voice signal. Battery backup is possible, but if the power outage exceeds the life of the battery, phone service will be unavailable. For these reasons, residential VoIP will be used mainly for second line service.

Figure 1. VoIP over Integrated Cable Access


Page 3

Figure 2. VoIP over DSL

Providers planning to offer second-line services should consider the following:

• Quality of Service (QoS)

• Features

• Remote diagnostics

• Interoperability

Long Distance

Another area of growth for VoIP has been in the long-distance and calling card markets. Subscribers purchase pre-paid cards which allow inexpensive long-distance calls for a specified number of minutes. Calls may be made domestically or internationally and the long-distance portion will be carried over an IP network. For this service, subscribers use any POTS phone to dial a local access number (or toll-free) to a gateway. They are prompted for their calling card number as well as the number they would like to call. After verifying the subscriber and the funds available, the long-distance portion of the call will be set up over the IP network.

With the emergence of Incumbent Local Exchange Carriers (ILECs) in the long-distance business, there is strong movement to use IP as the long-haul transport backbone. Using IP lets ILECs purchase softswitches instead of expensive Class 4s and thus gain quicker return on investment (ROI). The transport is transparent to the user, who dials the long-distance number as they normally would. The Class 5 switch would route the call to a gateway where it would be converted to IP for transport over the long haul.


Page 4

Figure 3. Long Distance VoIP Service

Providers offering VoIP long-distance services should consider the following:

• Up-to-date account database

• Signaling assurance to ensure proper billing/debit of account

• Quality of Service

• Geographic reach

• Installation/Provisioning of gateways and controllers

• Remote diagnostics

• Interoperability with partners’ equipment

Business-Oriented VoIP Services

Many LECs offering managed Private Branch eXchange (PBX) and Centrex services today are looking to migrate these services to the next generation to meet customer demand. Their offerings now include IP PBX, IP Centrex, and Wholesale services.


Page 5

IP PBX

Subscribers are looking to leverage their investments in their traditional PBXs, while taking advantage of some of the new capabilities and convergence benefits in next-generation systems. They want to maintain their private dial-plans and a common set of features across both delivery mechanisms. Subscribers also want assurance that the quality will not degrade when using packet-based technology, while they obtain the cost benefits of converging networks and platforms.

Figure 4. IP PBX Application

Providers offering managed IP PBX services should consider the following:

• Interoperability/interworking between traditional and IP PBX systems

• Common set of features

• Common billing and reporting features

• Managed CPE — remote diagnostics, reducing truck rolls

• Service Level Agreements (SLAs) and QoS guarantees — proving the same quality regardless of delivery mechanism


Page 6

IP Centrex

In the IP Centrex application, the Class 5 switch is used initially to deliver services to the IP-based phones via a network-based gateway. The same services are available to analog phones as well as IP phones, which enables the subscriber to migrate their platform over time while maintaining the current feature set.

Figure 5. Phase 1, IP Centrex Deployment

In Figure 5, the gateway (E) provides the interface between the IP network and the PSTN switch. The provider owns the managed IP or ATM network and must provision for QoS capabilities to ensure high-quality voice. At the customer premises, a router may be the demarcation between the customer’s network and the provider’s network. IP telephony devices will connect directly to the LAN, while analog devices (A) may connect to the LAN through a gateway (C), or may run a traditional TDM connection to the Class 5 switch.

Figure 6 shows the next phase of IP Centrex deployment. In phase 2, the Class 5 switch will be replaced by a softswitch (D), and all connections will be via the IP network. Any analog phones still in use (A) will connect through a gateway at the customer premises (B) where the signal is then converted to IP. Calls requiring the use of the PSTN will run through a central office gateway (E & F).


Page 7

Figure 6. Phase 2, IP Centrex Deployment

Providers offering IP Centrex services should consider the following:

• Interoperability/interworking between traditional and IP systems

• Common set of features

• Common billing and reporting features

• Managed CPE – remote diagnostics, reducing truck rolls

• SLAs and QoS guarantees – proving the same quality regardless of delivery mechanism

Wholesale/Peering

Because most providers do not have a global footprint, they partner with other providers to carry their traffic to locations where they lack a presence. These partnerships also result in peering agreements to determine which provider sends the traffic, and how much traffic to send.

These applications provide unique challenges to the providers. If a call is set up across multiple providers, how are problems isolated? How can a provider determine if the call even reached their network? How can a provider determine whether the call made it through their network? How can a provider track both directions of the call?


Page 8

Figure 7. Wholesale/Peering VoIP Application

Providers offering wholesale/peering services should consider the following:

• Interoperability between IP systems from peers

• Network performance

• Performance and fault isolation

• Proper call termination

• Call trace for both directions of call

VoIP QoS Expectations The user’s voice quality expectations, at a minimum, must be as good as POTS. If users notice a difference in voice quality, they will likely change service providers. In the PSTN, typical expectations for quality may be identified as follows:

• Call Setup/Post Dial Delay: < 3 s

• Latency: < 150ms

• Jitter: < 10ms

• Loss: 10**-5 BER

These generally accepted QoS parameters for voice can also be expressed in terms of Mean Opinion Score (MOS). For toll quality voice, a MOS of 4.0 or higher (out of 5.0) is generally accepted. There are various ways to determine MOS, which include human subjective testing, Perceptive Speech Quality Measurement (PSQM), Perceptive Evaluation of Speech Quality (PESQ), and Perceptual Analysis/Measurement System (PAMS). PAMS predicts overall subjective listening quality (a human's


Page 9

perception of quality) without requiring actual subjective testing. PSQM predicts subjective quality of speech codecs without requiring subjective testing. PESQ provides an objective measurement of subjective listening tests on telephony systems.

Factors that affect MOS are latency, jitter, loss, as well as codec, tandem encoding/decoding, background noise, and echo. The codec used for the PSTN is G.711 (64 kbps voice). In a packet environment, different codecs may be used to minimize the amount of bandwidth required; however, the more compressed the voice, the more noticeable the quality difference. The quality may still be acceptable, but a difference will be noticed. Table 1 shows MOS based on coding type used.

Table 1: Rudkin, S; Grace, A.; and Whybray, M. W.; "Real-Time Applications on the Internet," BT Journal, Col.15, No. 2, April 1997.

Standard Coding Type Bit Rate (kbps) MOS

G.711 PCM 64 4.3

G.729 CS-ACELP 8 4.0

G.723.1 ACELP MP-MLQ

6.3 5.3

3.8

Potential Issues Affecting Call Completion and Quality In VoIP environments, the factors that affect quality are compounded by the use of a packet network. The call setup time could be affected by congestion in the gateways, gatekeepers (controllers), or network. Latency, loss, and jitter are a function of queuing and congestion in the network. The codec used will also have a significant impact on quality, as discussed above. Many providers, who are not concerned about conserving bandwidth, will use the G.711 codec, even though in an IP environment the bandwidth requirement for the voice call is increased by as much as 50% over TDM due to protocol overhead.

Network congestion has serious adverse affects on voice quality. Congestion causes delayed or lost packets. These may be signaling packets or media packets. If the signaling packets are delayed or lost, issues may occur during call setup and teardown. If the signaling protocol uses TCP, then potential issues (such as billing for a call that has not been set up, or continuing billing for a call that has not been terminated properly) will be minimized. Other signaling protocols that use UDP will potentially suffer greater risk of incorrect billing.

If the media packets are impacted by congestion, then the quality of the voice and the conversation will suffer. Delayed packets increase jitter, which must be compensated for by buffering, which increases latency. If the packets are delayed too much (i.e., for greater than the amount of time


Page 10

allocated by the jitter buffer), then they are counted among the lost packets. Some gateway equipment may compensate for lost packets using sophisticated voice prediction algorithms, but generally speaking a loss of 10% or greater will make the voice quality unacceptable to most humans.

Network failures will have a negative effect on call setup and quality. The location of the outage will determine the impact. If one of the core routers fails, IP will reroute around the failure provided there is adequate bandwidth to support the increased load. Latency might increase slightly, but calls should stay up and the user may notice minimal degradation. Any single point of failure could terminate calls and hinder or prevent the ability to set up new calls. Possible single points of failure could include access facilities, edge routers, and gateways.

There could also be interoperability problems if using multiple technologies from different vendors in the IP space. Although vendors support common standards, each implementation may be slightly different, therefore interoperability must be thoroughly tested. If different signaling protocols are used (H.323, SIP, MEGACO), a “signaling gateway” will be required to translate.

Configuration will also play a key role in service delivery and availability. Routers have to be configured to forward packets for specific addresses. If the access control list (ACL) does not include certain addresses, then calls will not be set up. If gateways are not configured for proper IP address to phone number translations, calls cannot be set up. If databases are not accurately configured, calling card users will not be able to authenticate, and will be denied service. If the same codec is not configured for each side of the call, the call will not be completed. There are many configuration factors that are required to make the end-to-end service functional.

It is imperative that router configurations be accurate to enable optimal forwarding and QoS for voice traffic. This can be done by implementing DiffServ prioritization and queuing and MPLS traffic engineering. Adding these QoS capabilities makes configuration management that much more complex. If multiple technologies are used (i.e., ATM, Frame Relay, and IP), mapping QoS across technologies significantly complicates the roll of the network engineer. Configurations need to be monitored and any changes need to be carefully implemented.

Other considerations are the administrative issues. When operating as a transport network, call routing decisions may be based on “least cost routing” or performance of peer network, and these factors will determine which peer provider to whom to deliver the call. These variables can change at any time, which means routing information has to be updated regularly. Calls between the same source and destination may be routed to different providers at different times based on these factors which could provide irregularity in call quality over time.


Page 11

Service Quality Management – More Than Just Performance Monitoring performance for VoIP is only one aspect of Service Quality Management. The first step is identifying if the service is even available to the subscriber by doing prequalification testing. Is the right equipment in place? Are the connections adequate? Are there impairments that may inhibit the service, etc.? Prequalification testing should also include stress testing to identify how much load can be tolerated before quality degrades.

Once the service is prequalified, it is provisioned. The service provider must test to verify the service was correctly provisioned. This requires the use of equipment that is capable of placing test calls to various numbers. These test calls will identify any problems in signaling, media, and configuration of application and network elements.

When the service is turned up, then performance management is engaged, pulling statistics from the application and network elements, identifying performance of the media calls, and enabling historical trend analyses. Should a performance problem be identified, the diagnostic and test function should be activated to determine the cause of the problem and ways to resolve it. Then repair assurance is activated to make sure the problem has been resolved and no new problems have been created.

Figure 8. Service Quality Management Stages


Page 12

The Value of SQM – Problem Avoidance and Resolution Regardless of the application, the following questions must be answered to assure VoIP services:

Call Setup and Teardown

• Are call requests reaching the gateway/IP PBX?

• Are dialed digits being interpreted correctly?

• Is the called number ON-net or OFF-net?

• Are responses reaching the terminals?

• Have a representative sample of called numbers been tested?

• What is the Post-Dial Delay?

• Is the call being disconnected properly?

If an end-user is not able to set up calls, there are many potential causes that need to be checked. In order to see if the call requests are making it to the PBX or gateway, a protocol analyzer could be used to intercept signaling messages and responses from test calls, or the statistics available from the device will be able to tell if the setup message is being received. Is the setup problem only for certain dialed numbers? If so, the chances are good that there is a configuration problem in the gateway or controller, or possibly in a firewall or router access control list. These problems could be identified up front by provisioning assurance testing. If these problems occur after the service has been turned up, a configuration management tool would be helpful in identifying recent changes that could be the cause.

Call Quality

• Is delay acceptable in both directions?

• Is speech quality acceptable in both directions?

• What is the average delay, jitter, loss per call?

• What codec is being used?

• Is there any clipping or echo?

Once the call is set up, the quality of the call should be measured to ensure no problems exist. This could be done passively on a random basis or through the use of test calls.

In order to measure call quality, an in-line probe could be placed at the endpoints to monitor actual subscriber calls and gather statistical information related to call quality. These measurements would


Page 13

be highly variable due to the variability of destination and duration of calls. Also, if there is a problem, the subscriber is already experiencing it, so there is no ability to proactively improve the situation. A better approach is to place test calls periodically and measure the quality of those calls. Probes can be placed at the edge of the IP cloud and make calls to any phone (IP or standard) or to another probe, where quality can be measured.

It is important to recognize that a voice call is comprised of two unidirectional flows, and that each direction may be independent in terms of call routing and codec used, hence the quality in each direction may differ greatly. It is imperative to obtain measurements for each direction of the flow, versus a round-trip measurement such that it is possible to isolate problems. A performance management OSS could collect and correlate this information and enable SLA management, trend analyses, and proactive fault isolation.

Network Statistics

• Is there sufficient network and gateway capacity?

• What are the busy hours and the corresponding call volumes?

• Is the network fault tolerant such that calls can be completed and maintain QoS in the event of a failure?

As the network traffic patterns grow and change, historical information should be kept for trending purposes to better plan for upgrades to network equipment and facilities. Statistics should periodically be pulled from the gateways, PBXs, control devices, and network elements to compare with acceptable thresholds and identify any possible violations of those thresholds. Does call quality change when the number of calls reaches a certain threshold? How much of a difference in call quality is there during busy hours and off peak hours? Is it realistic to limit the number of calls to some percentage of maximum in order to maintain a more consistent call quality? These questions can be answered through historical trending information gathered from the elements as well as the per-call statistics.

Testing

• What is the cause of poor quality?

• How can we isolate problems by layer, network segment, etc.?

• Has the outage been fully repaired?

• How does the “fix” affect other traffic?

If the voice application experiences a performance problem such as packet loss, testing is required to determine the cause, which could be congestion in a gateway or router, congestion in the underlying Layer 2 transport network, or bit errors on a physical link. Remote diagnostic and test functionality will enable operations staff to run tests at various layers and within layers to quickly isolate the problem to a set of protocols or equipment. Testing can also be done to isolate problems to a specific network


Page 14

segment or piece of hardware. Once the problem is resolved, repair assurance testing will ensure complete resolution of the problem (as well as minimizing the possibility of any other problems caused by the “fix”) so that the service can be properly turned over to the subscriber.

Spirent SQM for Enterprise (IP PBX) Applications For an IP PBX managed solution, the provider must have visibility onto the subscribers’ LAN to properly isolate problems and assess the performance/availability of the service. The configuration tables also must be accessible and maintained to identify changes or misconfigurations that could be service-affecting.

There are tools available to assist providers in quick problem resolution, and proper placement of these tools will give a complete performance picture and the ability to segment the network for faster fault isolation.

Figure 9 shows a possible configuration for a managed IP PBX service with the use of qScope probes and the CenterOp™ OSS suite. By placing the probes at the gateway and switch points, a problem can be isolated to the IP network, to the PSTN, or segments in between. The CenterOp OSS suite will collect information from the probes as well as the elements (routers, gateways, PBXs, etc.) and track historical trending information. It is also capable of setting up and automating diagnostic tests at various layers, querying network elements, and displaying results.

Figure 9. IP PBX Application with SQM


Page 15

Figure 10 shows a possible operations flow that would be useful for provisioning assurance for call setup in an IP PBX application. Provisioning assurance should be done prior to turning over the active service to the subscriber. To verify the accuracy of the provisioned service, test calls should be placed to various internal and external locations. Collecting statistics on these calls from the various service and network elements can identify and isolate any provisioning problems.

Figure 10. Provisioning Assurance for IP-PBX Application, Call Setup

Spirent SQM for IP Centrex Applications The IP Centrex application is similar to the managed IP PBX application with the exception of the location of the equipment and how many subscribers are supported by the equipment. The provider needs visibility to the subscriber’s demarcation point, which may be a LAN switch or router. Since firewalls may be used, proper configuration of those firewalls must be verified. The subscribers gateway may also be managed by the provider, thus visibility to the gateway must be available.

If there is a problem with call setup, a determination must be made whether or not the request is making it to the Class 5 switch or softswitch. Protocol analyzers and/or element statistics at the customer premises and in the CO will be able to verify the signaling. If requests are not getting through, there may be a firewall issue or a routing issue on the customer’s network.

If the Post Dial Delay (PDD) is not acceptable, the switch, gateway, or network may be congested and overloaded. If only specific dialed numbers are having issues, then the problem is probably in a configuration file in the controller or gateway.


Page 16

Again, as with IP PBX, each direction of the call must be measured because the call comprises separate, asymmetric, one-way flows. The historical trending information is important to identify future threshold crossings and possible service degradation. This information should be collected from the network elements and voice switches to gain a better understanding of processing load and capacity with acceptable service levels.

Figure 11 shows a possible SQM configuration for IP Centrex service. By placing probes at the subscriber demarcation as well as at the CO gateway, it is possible to isolate problems and capture site-specific performance information. With access to the physical layer circuit, a Broadband Remote Test Unit (BRTU) probe with IP expert analysis can capture and decode signaling packets to identify any problems with call setup/tear down. The CenterOp OSS suite will initiate the tests, capture statistics from the elements, and will monitor performance. Performance information is collected on a regular basis and stored such that historical information can be viewed to identify performance trends. Should subscribers wish to view performance information relative to their service a Web-based portal will give them secure access to the latest performance information and reports.

Figure 11. IP Centrex Application with SQM

Figure 12 shows a possible operations flow that would be useful for provisioning assurance for the media call in an IP Centrex application. As mentioned, provisioning assurance should be done prior to turning over the active service to the subscriber. To verify the accuracy and quality of the provisioned service, test calls should be placed to various internal and external locations. By collecting performance statistics on these calls from the probes (as well as the various service and network elements), any provisioning problems can be identified and isolated.


Page 17

Figure 12. Provisioning Assurance for the Media Call – IP Centrex Application

Spirent SQM Wholesale/ Peering Applications For the wholesale and provider peering applications, it is important to identify which network is causing a problem in the end-to-end service. The only way to isolate the problem is to be able to trace the call from ingress to egress, and measure the quality between those points. If a partner reports that calls are not getting through to the endpoint, visibility is needed to ensure that the call setup request was received by your network. If the request was never received, then the problem is somewhere upstream to your network. If the request was received but the call was not set up, then there is likely a configuration or interoperability issue. If the call is set up, but not completed, then the problem is likely to be downstream.

If the partner complains about call quality on your network, tools are needed to measure the quality for this leg of the call to ensure they are within thresholds. If call quality is poor, further diagnostic tests are required to identify the source of the problem. Since each direction of the call can be routed differently (through routers on the private IP network and/or through different providers based on least cost routing), it is important to have the ability to trace the path of the call.

Call termination is also very important in this application. Should the source (upstream) provider terminate the call, but the call is not terminated with the destination (downstream) provider, the wholesale peer will be responsible for payment of the call without collecting any revenue from the source provider.


Page 18

Figure 13 shows a possible SQM configuration for the wholesale/peering application. Probes would be placed at the network edge in order to measure performance across the network and identify if problems are On-net or Off-net. The CenterOp OSS will capture performance information from the probes and statistics from the elements and provide a historical view for trending purposes. It will initiate and automate tests required to diagnose problems and query real-time statistics from the network elements, thereby providing diagnostic and repair assurance.

Figure 13. Wholesale/Peering with SQM

Figure 14 shows a possible operations flow that would be useful for diagnosing a poor performance problem in a wholesale/peering application. Basically, the goal is to isolate the problem to either the provider’s network, or the network of one of its peers. To do this, it is imperative that call quality and network performance measurements are obtained regularly and tracked from a historical perspective to identify any trends and threshold violations.

Figure 14. Diagnostic Flow for Wholesale/Peering Applications – Media Call


Page 19

Spirent SQM for Long-Distance VoIP Applications Call quality is probably the most important characteristic for long-distance VoIP, therefore, it is important to understand this quality from the customer perspective. Tools that take into account the performance of the gateway devices as well as the network will give the most accurate results. The ability to compute a MOS for calls will give an objective measurement most applicable to the customer experience. Since call quality is affected by network conditions, it is imperative to monitor the condition of that network and the elements to ensure optimal performance. Monitoring the elements will also provide fault management information in the event of failure or performance problem.

Figure 15 shows a possible SQM configuration for the Long Distance VOIP application. Note that the BRTU probes are on the TDM-side of the gateway in order to get the most accurate performance information relative to the customer experience. Statistics are also collected from the network elements to monitor performance and to assist in fault management. The CenterOp OSS correlates all the information into meaningful data which enables the Operations staff to quickly identify and resolve problems through automated testing and diagnostic functions.

Figure 15. SQM for Long-Distance VoIP Application


Page 20

Figure 16 shows a possible operations flow that would be useful for monitoring performance of media calls for the long-distance application. The goal is to monitor the customer experience and identify and correct any performance problems before the subscriber is aware of the problem.

Figure 16. Performance Management Flow for Long-Distance Applications – Media Call

Summary/Conclusions

There are many different aspects of service quality management for VoIP. Many solutions only address a piece of the SQM puzzle, but a comprehensive solution is needed to manage the end-to-end service, not only across the network, but across the life of the service. This comprehensive solution should address testing to determine if the service is deliverable, provisioning assurance to verify configurations before the service is turned up to the customer, performance monitoring to ensure the subscriber is receiving the proper level of service and to identify trends such that proactive fault detection can occur, diagnostics and testing to identify the source of problems at all layers, and repair assurance to ensure the “fix” really fixed the problem and did not cause any new problems. These are the elements of the complete “Service Quality Management” solution.

SQM for VoIP enables providers to proactively manage services, thereby gaining valuable insight into the customer experience. SQM tools will generate greater customer satisfaction by resolving problems before they become visible to the subscriber, operations efficiency by using an integrated toolset for the 5 stages of SQM, and reduction in number and length of outages by the collection and correlation of information from multiple layers, protocols, and vendors.

Spirent Communications offers the comprehensive toolset to give providers the ability to do Service Quality Management. By packaging multi-functional test probes and operations support systems, with information from the network elements and EMSs, Spirent offers a complete SQM system for any VoIP application.

Documents

Service Quality Management for VoIP ApplicationsVoice over IP (VoIP) is fast becoming more than hype. The primary use for VoIP a few years ago was to offer affordable long distance