POLQAapproved asITU-TRec.P - OPTICOM voice … Group 12 of the ITU-T, with the objectiveofdefiningatechnologyupdate for PESQ/P.862. During its development morethan100,000fixed,mobileandVoIP

N E W S F R O M

The Perceptual Quality Experts.

Largely supported by OPTICOM’s perceptual quality experts, the tech-nology upgrade and designated successor for PESQ steered its coursethrough International Standardization and received approval in January2011 as the new ITU-T Recommendation P.863.POLQA represents the next-generation of voicequality testing technology for fixed, mobileand IP-based networks and can be appliedto HD Voice, 3G and 4G/LTE.

> continued on page 2

Introducing the Next-Generation Mobile Voice QualityTesting Standard for HD-Voice, VoIP, 3G and 4G/LTE

POLQA as P.863More on the Next-GenerationMobile Voice Quality TestingStandard Page 2

Q-App for AndroidOn-device testing on Smartphonesin Real-Time Page 4

POLQA and PEVQLearn more about OPTICOM’s newVideo Dream Team Page 6

Testing Germany’sMobile NetworksFirst POLQA Benchmark of GermanMobile Networks Page 7

NEW: POLQA approvedas ITU-T Rec. P.863

POLQA®, PEXQ®, PEAQ®, PEVQ®, PEDQ®, 3SQM® and the OPTICOM logo are registered trademarks of OPTICOM GmbH; PESQ® is a registered trademark of OPTICOMGmbH and Psytechnics Ltd.; the ‘Single-sided Speech Quality Measure’ and ‘The Perceptual Quality Experts’ are trademarks of OPTICOM GmbH.This information may be subject to change. All other brand and product names are trademarks and/or registered trademarks of theirrespective owners. All rights reserved. Copyright © 2011 OPTICOM GmbH – www.opticom.de

2 Introducing POLQA

NEW: POLQA approvedas ITU-T Rec. P.863Introducing the Next-GenerationMobile Voice Quality Testing Standardfor HD-Voice, VoIP, 3G and 4G/LTE

The new POLQA standard was in develop-ment since 2006 by leading experts, as theresult of a competition carried out byStudy Group 12 of the ITU-T, with theobjective of defining a technology updatefor PESQ/P.862. During its developmentmore than 100,000 fixed, mobile and VoIPtest calls were gathered by experts fromall over the world, thus representing thelargest and most accurate database everused in the development of a perceptualquality algorithm.

POLQA – an acronym for “PerceptualObjective Listening Quality Analysis” - willoffer a new level of benchmarking capa-bility to determine the voice quality ofmobile network services. The POLQA per-ceptual measurement algorithm is a jointdevelopment of OPTICOM, SwissQual andTNO, protected by copyright and patents,and made available under license as soft-ware tools for various platforms.

OPTICOM offers standard-setting POLQAreference implementations, along withlegacy PESQ and ECHO-TQ (TalkingQuality) measurements for various plat-forms/OS:


POLQA form factors Desktop PC OEM SmartphoneUse Case: Stand-alone (End-User) OEM Libraries OEM Apps

Application Scenario: S/W-Tools with GUI for lab use To be embedded in DTT probesand stand-alone T&M-systems

On-Device Test Tools forMobiles & Smartphones

AvailablePlatforms/Versions:

PEXQ Software-Suite forWindows, incl. Voiceplus option• POLQA/P.863

• POLQA/P.863OEM lib for Windows

• POLQA/P.863OEM lib for Linux

• POLQA/P.863 Q-Appfor Android Smartphones

• PESQ/P.863 OEM lib forAndroid

Can be combined with: Voiceplus option• PESQ/P.862• ECHO-TQVideo option• PEVQ/J.247

Voiceplus option• PESQ/P.862• ECHO-TQVideo option• PEVQ/J.247

• PESQ/P.862 Q-App forAndroid Smartphones

• PESQ/P.862 OEM libfor Android

Optional: • Other platforms andO/S on request

• Conformance Testing

• Symbian and other O/Son request

3

POLQA –the PESQ/P.862 Successor


With more than 20.000 licensed test units,the Perceptual Evaluation of SpeechQuality (PESQ/P.862) algorithm, hasevolved as the most successful industrystandard for voice testing since its defini-tion in 2000. Naturally, voice coding tech-nology and IP based transmission stan-dards have further advanced, so the timingis appropriate for an update of the state-of-the-art benchmarking method. POLQA,the designated successor for PESQ wasapproved in 2011 as the new ITU-TStandard P.863.

Measurement AccuracyIn the super-wideband mode, POLQA cov-ers an extended audio frequency range upto 14 kHz, suitable to assess modern HD-Voice applications. Apart from the 2005wideband extension P.862.2, PESQ origi-nally was limited to ‘vintage’ narrowbandtelephony applications (POTS), only.Consequently, POLQA would clearly be thechoice for any modern benchmark that tar-gets at wideband and super-widebandvoice services. For “classical” narrowbanduse cases like mobile network benchmark-ing, POLQA clearly offers a huge advancein measurement accuracy. Therefore thequestion “Is it worth migrating?” shouldeasily be answered. The following com-parison gives a general idea on theimprovements.

Operating Mode NB (8kHz) SWB (48kHz)

POLQA Version Default Fast Very Fast Default Fast Very Fast

Core 2 Duo 2.13 GHz 0.13 0.11 0.10 0.35 0.25 0.22

Core i5 2.27 GHz 0.10 0.08 0.08 0.28 0.19 0.16

Core i7 3.07 GHz 0.07 0.05 0.05 0.20 0.14 0.11

Performance figures reported without file I/O and are reported in fractions of real-time (e.g. 0.20 corre-sponds to five times real-time). Performance may vary due to test call characteristics.

“One of the most motivating experi-ences from the last ITU-T SG12 meet-ing was hearing experts in the fieldrecognising POLQA as a major stepforward from PESQ, and eagerly awaitto get their hands on it. POLQA per-forms up to 56% better than PESQ,opens new application areas and elim-inates the weak points of PESQ, onesuch aspect being EVRC under-predic-tion when compared to AMR codecs”

Dipl.-Ing. Christian Schmidmer,OPTICOM’s CTO and

Standards Developer

Algorithm PerformanceOne frequently asked question in the con-text of the new POLQA algorithm is “Howdoes it compare performance-wise?” As ageneral answer, it will require some moreprocessing in narrowband mode thanPESQ. However, OPTICOM is continuouslyprogressing and optimizing the perform-ance for various operating systems.Besides the default version, which wasinput to ITU-T for standardization, thereare already fast and advanced versionsavailable that offer significant optimiza-tion (see the following performance datafor INTEL processors).

For any particular application, we wouldrecommend to check your performancerequirements directly with OPTICOM’sexperts.

rmse*

Narrowband PESQP.862.1

POLQAP.863

Improvement

Averaged rmse* 0.1857 0.1363 27%

WidebandPESQ

P.862.2POLQAP.863

Improvement

Averaged rmse* 0.3450 0.1506 56%

Comparison of the prediction error for POLQA with PESQ: The root mean square error (RMSE) is a measureof the differences between scores predicted by an algorithm and the corresponding scores obtained fromsubjective tests. It is a more suitable measure of precision than the commonly used correlation factor. Thermse* is similar to the rmse, but also takes the accuracy of the subjective experiment into account.

Q-App for Android4

NEW: POLQA Q-App for AndroidOn-device testing onSmartphones in Real-TimeRight from the start of POLQA, the newITU-T Recommendation P.863, OPTICOMalso offers its OEM reference librarytoolset in a mobile form factor for AndroidSmartphones. The POLQA Q-App comple-ments OPTICOM’s PESQ/P.862 Q-App forAndroid, Symbian and Windows Mobile,originally premiered at the 2010 MobileWorld Congress.

Check the Voice Quality of your Mobile,coverage and network transmission atyour fingertips. Should you experiencepoor speech intelligibility that you are notwilling to pay for, compare and benchmarkcompeting operators, or just generate hardfacts by standard mean opinion scores(MOS) on your screen. Where, up to now,you would have required a test engineer toshow up with high-end measurement gear,you will now be able to download an App.Quality conscious operators could evenconsider giving-away Q-Apps as part oftheir premium mobile offering, and adjustfuture business models based on quality.


Device Performance Best Case* Typical* Worst Case*

POLQA Q-App for Android

HTC Desire Z,Qualcomm MSM 7230, 800 MHz, Android 2.2.1

0.65 0.92 2.13

PESQ Q-App for Android

HTC Desire Z,Qualcomm MSM 7230, 800 MHz, Android 2.2.1

0.11 0.14 0.20

PESQ Q-App for Symbian

NOKIA 6720 C,S60, 3rd Ed., FP 2, ARM 11, 600 MHz

0.87 1.11 1.64

PESQ Q-App for Win Mobile

HTC Diamond 2 Smartphone,Qualcomm MSM7201A, 528 MHz

1.08 1.51 2.48

* Performance may vary due to test call characterisitics and is reported in fractions of real-time (e.g. 0.20corresponds to five times real-time).

5Mobile Testing


New BusinessDevelopmentOPTICOM recently welcomed Mr. GrahamRousell (centre) to join OPTICOM in thenew position of ‘Business DevelopmentManager’. He will reinforce the keyaccount team for OEM licensing ofOPTICOM’s advanced voice, audio andvideo quality algorithms. Graham brings agreat deal of professional experience pre-viously performing product managementfor voice quality enhancement systemswithin Tellabs. He is well aware of bothsides of the coin, as he used to employOPTICOM’s PESQ tools to demonstratethe benefits in network quality improve-ment of VQE products. To get in touchwith our business development team, feelfree to use the contact form onwww.opticom.de, send us an email [email protected], or just give us a call:+49 - 9131 - 5 30 20 - 0

OPTICOM’s CEO Michael Keyhl (right)and Key Account Manager LindaPolster (left) introduce GrahamRousell, BDM (centre) to the latestGerman mobile operator benchmark.

Use Cases andApplicationsQ-Apps are available under OEM licenseto mobile manufacturers, walk & drive testtool vendors, system integrators and net-work operators. Applications mightinclude indoor walk testing, premiummobile offerings based upon agreed QoElevels, network testing by clouds of friend-

ly users, unobtrusive QoE benchmarking atmission critical or high security venuesand many more.

Audio-Visual Testing6

POLQA and PEVQ –the Video Dream-TeamWith POLQA, OPTICOM’s PEVQ for VideoMOS testing of Mobile-TV, video stream-ing, video telephony and video conferenc-ing just found an adequate partner foraudio-visual testing. PEVQ was standard-ized in 2008 as ITU-T Rec. J.247, and is afull-reference picture quality measure-ment for multimedia quality testing ofQCIF, CIF and VGA image sizes. PEVQ alsoextends to SD and HD resolutions, withsimilar performance as ITU-T J.341 (a newrecommendation for HDTV digital cabletelevision).

In the HDTV use case, PEVQ should becomplemented with PEAQ, the ITU-RRecommendation BS.1387 for evaluationof audio (music) quality. An extension ofPEAQ from stereo to multi-channel homecinema is currently under development.For video conferencing and telepresenceapplications, however, the focus is onvoice communication, so PEAQ might notbe the algorithm of choice. Instead, an HD-Voice measure like POLQA will be theideal combination. It will allow audio-visu-al quality assessment of narrowband,

wideband (8 kHz) and super-wideband (14kHz) voice codecs in a communication sce-nario of a video conference.

OPTICOM offers end-users an optionalcombination of PEVQ and POLQA in thePEXQ software suite for Windows. OEMlicensees can realize the similar function-ality from bundling OPTICOM’s PEVQ andPOLQA OEM libraries.

New in PEXQ: The comprehensive graphi-cal analysis toolset now fully supportsPOLQA/P.863 in narrow-band (8 kHz) andsuper-wideband (48 kHz) mode. To easebackward compatibility, the voiceplus optioncomes with POLQA/P.863, in combinationwith legacy PESQ/P.862 and OPTICOM'sproven ECHO-TQ (Talker Quality Measure).

New in PEXQ: The composite resultview for audio-visual quality analysis

reporting the video MOS (PEVQ) (upperframe) and the audio MOS (PESQ) (mid-dle frame). The combination of perceptu-

al video and audio quality measuresallows for detailed lip-sync analysis

(lower frame).


Report of first POLQA Benchmark


7

First POLQA Benchmark ofGerman Mobile NetworksGerman P3 communications GmbH, aleading provider of engineering servicesfor network test and optimization, as wellas a long-term customer of OPTICOM’sOEM voice quality test technology,worked with OPTICOM to investigate theapplication of POLQA/P.863 to recent net-work tests. During the latter part of 2010,in cooperation with Germany’s popular“connect” magazine, P3 communicationspublished their annual results of mobilenetwork benchmarks, comparing the QoEofferings of T-Mobile, Vodafone, E-plusand O2.

As a premiere, the latest benchmark alsoincluded a new test scenario based onmobile-to-mobile call quality analysis(most common drive test tool scenariosonly assess a mobile-to-landline sce-nario). While the mobile-to-mobile testcase is deemed to take end user behav-iour into account which better reflectsincreasing adoption of mobile QoE, it isalso suitable to better reveal the networkperformance for a more challenging con-figuration.

Historically, voice quality analysis hasbeen based on the industry standardPESQ/P.862. Now, OPTICOM and P3 com-munications jointly investigated how thisbenchmark would look like when applyingPOLQA/P.863, instead. At first glance, thecomparison of POLQA and PESQ in thediagrams does not reveal any big surpris-es. While generally being the more accu-rate measure, POLQA does not show anygreat difference in an established3G/UMTS test case that is well withinthe scope of P.862.

This might change immediately for CDMAand TD-SCDMA networks, along with 4G.On the other hand these results confirmthat operators can easily migrate toPOLQA today – thus paving the groundsfor comparability with existing test data,while being readily positioned for movingtowards 4G/LTE and HD-Voice analysis.

For more details on benchmark compari-son or effects of applying POLQA, pleasefeel free to contact P3 communications.Furthermore, OPTICOM’s experts will bepleased to offer their support whereverpossible for a seamless migration toPOLQA.

First results from POLQA benchmark ofGerman mobile networks (top): Althoughminor deviations need to be furtherinvestigated, the overall trend whenpresenting the ranking of the mobile-to-mobile scenario, for all four operators,confirms a similar QoE level. In a furtherevaluation of test data, the quality gainfor a mobile-to-landline scenario wasalso verified with POLQA (below).

Contact: Peter Seidenberg,Managing Director,P3 communications GmbH,http://www.p3-group.com/communications/

3.0

3.2

3.4

3.6

3.8

4.0

OP 1 OP 2 OP 3 OP 4 OP 1 OP 2 OP 3 OP 4Average POLQA MOSon Sample Basis

Average PESQ MOSon Sample Basis

Average POLQA MOSon Sample Basis

MO

S-LQ

O

3.0

3.2

3.4

3.6

3.8

4.0

MO

S-LQ

O

Average PESQ MOSon Sample Basis

OP 1 OP 2 OP 3 OP 1 OP 2 OP 3

Mobile to Mobile

ISDN to Mobile DL

8 OPTICOM’s Sales Contacts

About OPTICOMOPTICOM GmbH is the leading vendorfor Voice, Audio and Video qualitymeasurement technology and OEMproducts for the analysis of mobileand IP based networks. Since its foun-dation as a spin-off from Fraunhofer’sMP3 development team, the pioneersin perceptual quality testing havebeen providing by now sixInternational world-class standards foraudiovisual quality evaluation, includ-ing PESQ, PEAQ and PEVQ.

After the great success with PESQ –the industry standard for telephonyvoice quality testing with more than20.000 licensed installations world-wide – the experts from Germanyhave been the driving force behindPOLQA, the new ITU-TRecommendation P.863 and designat-ed PESQ successor. POLQA forms thenext-generation mobile voice qualitytesting standard for HD-Voice, VoIP,3G and 4G/LTE, and is being offeredby OPTICOM in distinct form factorsfor PC lab testing, OEM libraries andmobile apps.

• PEXQ is the ideal ‘all-in-one’ labtest suite for developers, manufac-turers and operators. The ‘X’ justsymbolizes the ongoing evolutionof perceptual QoE metrics: Basedon POLQA/PESQ, PEAQ, PEVQ andPEDQ the software provides themost comprehensive standards-based MOS-KPI set to score Voice,Audio/Visual and Data quality asexperienced by subscribers.

• Q-Apps mark the very latestachievement in the evolution ofperceptual testing by seamlesslyintegrating POLQA/PESQ-basedMOS scoring to mobiles and smart-phones.

• OPTICOM’s proven OEM technologycan be found in most state-of-the-art products of leading T&M ven-dors, see alsowww.opticom.de/company/cus-tomers-licensing.html.

OPTICOM GmbH is a privately ownedcompany based in Erlangen, Germany.

OPTICOM GmbHHead Quarters

Nägelsbachstraße 3891052 Erlangen, GERMANYPhone: +49-9131/53020 -0Fax: +49-9131/53020 [email protected]

Telchemy Inc.Duluth, GA, USAPhone: +1 678 387 3000

North America:

YALE Systek CorporationSeoulKOREAPhone: +82 2 36 67 56 86

Transcom International Ltd.ShanghaiCHINAPhone: +86 21 6432 6888

GCOM Technologies Co.TaipeiTAIWANPhone: +886 2 29 17 32 01

Asia Pacific


OPTICOM’s Technology Portfolio

Sales Contacts

• PEAQ – Perceptual Evaluation of Audio Quality for MOS scoring of stereo soundaccompanying video streams according to ITU-R BS.1387

• PEDQ – Perceptual Evaluation of Data-Services Quality for MOS scoringof perceived data download and browsing QoE

• POLQA – Next-Generation Mobile Voice Quality Testing Standard for HD-Voice,VoIP, 3G and 4G/LTE

• PESQ – Perceptual Evaluation of Speech Quality for MOS scoring of narrow andwide-band telephony voice signals (Listening Quality) according to ITU-TP.862/P.862.1 (narrow-band) and P.862.2 (wide-band)

• PESQ-TQ – Perceptual Evaluation of Speech Quality (Talking Quality) for MOSscoring of the talker’s perception of his own voice (echo and sidetone)

• 3SQM – Single-sided Speech Quality Measurement according to ITU-T P.563

• ECHO – OPTICOM’s advanced Echo Evaluation

• PEVQ – Perceptual Evaluation of Video Quality for MOS scoring ofvideo-telephony, -streaming and -conferencing according to ITU-T J.247

Voice Quality

Audio Quality

Video Quality

Data Quality


Head Quarters:

Documents

POLQAapproved asITU-TRec.P - OPTICOM voice … Group 12 of the ITU-T, with the objectiveofdefiningatechnologyupdate for PESQ/P.862. During its development morethan100,000fixed,mobileandVoIP