Quality of Service Report: KwaZulu-Natal Province 2015 ... · Quality of Service Report: KwaZulu-Natal Province ... and WCDMA (900, 2100) ... CSSR excludes dropped calls

0 | P a g e

Independent Communications Authority of South Africa Pinmill Farm, 164 Katherine Street, Sandton

Private Bag X10002, Sandton, 2146

Quality of Service Report:

KwaZulu-Natal Province

2015/2016 Quarter 4

1 | P a g e

Table of Contents

1 Executive Summary .............................................................................................................................................. 2

1.1 Introduction ................................................................................................................................................. 2

1.2 Drive Test Route Plan .................................................................................................................................. 3

2 Methodology ........................................................................................................................................................ 4

2.1 Drive Tests Equipment ................................................................................................................................. 4

2.2 Test Plan ...................................................................................................................................................... 4

2.3 Computational Methodology ...................................................................................................................... 6

3 Results and Analysis ............................................................................................................................................. 8

3.1 Parameter Targets ....................................................................................................................................... 8

3.2 Results .......................................................................................................................................................... 8

3.2.1 Accessibility Measurement Results ............................................................................................................. 8

3.2.2 Retainability Measurements Results ......................................................................................................... 10

3.3 KwaZulu-Natal Average Measurements Results ....................................................................................... 11

3.3.1 Average Accessibility measurement data results ...................................................................................... 11

3.3.2 Average Retainability measurement data results for KwaZulu-Natal Province ........................................ 12

3.3.3 Serving Technology Distribution ................................................................................................................ 13

4 Conclusion .......................................................................................................................................................... 13

5 Appendix A (KwaZulu-Natal coverage maps) ..................................................................................................... 15

2 | P a g e

1 Executive Summary

The Authority conducted Quality of Service (QoS) measurements on the networks of South

African mobile operators; Cell C, MTN, Telkom and Vodacom. The measurements were

performed to monitor performance of voice services being offered by the operators in

KwaZulu-Natal province. The measurements were carried out during February 2016 and

covered a total distance of 2574 kilometres of drive testing.

The purpose of collecting QoS measurements was to analyse and monitor the service quality

experienced by the end user of voice services offered by South Africa’s mobile operators

around targeted areas. Measurements were conducted in typical circumstances where

mobile service usage is likely to occur such as major towns, townships, farm areas, rural

areas and major road arteries as per population densities, economic and tourism activities

including historical complaints. The sampled areas includes Cato Ridge, Pietermaritzburg,

Howick, Mpophomeni and Richmond.

A vehicle equipped with TEM Symphony measurement tool was used to collect data, in order

to measure two main Key Performance Indicators (KPIs) which are Retainability and

Accessibility. The Drop Call Rate (DCR) parameter is used to measure Retainability and the

Call Setup Success Rate (CSSR) parameter measures Accessibility. According to the End-

User and Subscriber Service Charter regulations of 2016, DCR should be less than 3% and

CSSR should be greater than 98%.

In terms of the average results of the areas covered MTN and Telkom met the accessibility

target, while Vodacom and Cell C did not meet the accessibility target. Vodacom is the only

operator that met the average retainability target and all other operators (Telkom, MTN and

Telkom) did not met the average retainability target.

1.1 Introduction

The Authority conducted Quality of Service (QoS) monitoring of the GSM voice telephony

service being offered by Cell-C, MTN, Vodacom and Telkom within the KwaZulu-Natal

Province.

The KwaZulu-Natal is located in the south-east tip of South Africa bordering the Indian

Ocean. It also borders on the Eastern Cape, Free State and Mpumalanga provinces, as well

as Lesotho, Swaziland and Mozambique. It covers an area of 94 361 km² and has a

population of approximately 10 267 300. It is the third-smallest province in South Africa by

surface area but has the second largest population size after Gauteng.1

1 http://www.localgovernment.co.za/provinces/view/1/eastern-cape

3 | P a g e

Figure 1: Map of KwaZulu-Natal Province

QoS is defined as the collective effect of service performance that determines the degree of

satisfaction of a user of the service. QoS provides an indication of what the customer

experiences when using the mobile network and is evaluated in terms of Retainability and

Accessibility. Retainability is defined as the ability for a call to stay connected through to a

normal call tear-down process, without abnormally disconnecting from the cell site that

caries the call2. Accessibility is defined as the percentage of the number of times a user is

rejected due to the unavailability of system resources when attempting to place a call3.

The KPI parameters used to measure Retainability and Accessibility are Drop Call Rate (DCR)

and Call Setup Success Rate (CSSR), respectively.

The test methodology adopted is the drive test sampling methodology which provides a

snapshot view of the mobile operator’s quality of service and provides a realistic picture of

network performance from a user’s point of view. It must be noted that drive test results do

not represent the mobile service provider's overall network performance. It is based on the

specified routes, at a particular time of day when the measurements were carried out using

a particular type of handset. All the test phones were allowed to select the network freely.

1.2 Drive Test Route Plan

The KwaZulu-Natal is divided into one metropolitan municipality, ten district municipalities

and further subdivided into 50 (fifty) local municipalities. The areas within eThekwini

Metropolitan Municipality and Umgungundlovu District Municipalities were covered during

the drive tests. The areas covered include farming areas, villages, rural towns, major towns, tourist’s

2 Annexure B.2 of ETSI EG 202 057-3 3 GSM 12.04, ETS 300 615.

4 | P a g e

areas and connecting roads with the neighbouring towns and villages. All the measurements

were performed in the areas where all the operators claim to have network coverage. In

areas where Cell C and Telkom Mobile did not have own coverage they were allowed to roam

on their respective roaming partner.

Table 1: Drive Test routes

Test Type District/Metropolitan Municipality Route Name Dates

Accessibility

and

Retainability

eThekwini Metropolitan Cato Ridge 01/02/2016

16/02/2016

Umgungundlovu and Sisonke Pietermaritzburg

04/02/2016

18/02/2016

Richmond

02/02/2016

19/02/2016

Mpophomeni

05/02/2016

20/02/2016

Howick

03/02/2016

16/02/2016

2 Methodology

A drive test methodology was used to collect samples. TEMS Symphony benchmarking tool

was used to collect the data. This method provided an external indicative snapshot of an

operator’s network performance from the user’s point of view on the selected routes. The

drive test results represent a snapshot of the mobile service provider's network performance,

based on the specified routes during when the measurements were carried out.

2.1 Drive Tests Equipment

The Drive Testing methodology is basically collecting measurement data with drive test

equipment which comprises of: TEMS Symphony 7.7 with Samsung Galaxy Note 4 (SM-

N910F), laptop with Probe Controller 1.9 installed. However, drive tests only provide data

on the downlink side of the communication, it provides a perspective to the service provider

about what’s happening from a subscriber’s point of view.

2.2 Test Plan

A test plan is displayed on Table 2 below:

5 | P a g e

Table 2: Test plan and configurations

Configuration Explanation

Antennas

Tests were carried out from a moving vehicle with roof mounted

antennas. The antennas were arranged in a well-defined fixed way with

a minimum distance to each other, reducing RF-influence to an

acceptable level in an equal radio environment. The coupling loss

between two mobile handsets was min 40.5 dB4.

Band The bands tested were GSM (900 and 1800) and WCDMA (900, 2100)

Call Samples

A minimum of 120 test samples per network operator were tested

except in the areas were the services were limited on most part of the

drive test route. Test drives were planned to ensure, as far as

practicable, that the results adequately reflect the QoS perceived by

customers for the period under review.

The drive test was designed to be representative of the population

relative to the traffic of the network. Measurements were scheduled so

as to reflect accurately the traffic variations over the hours of a day,

and user's behavior.5

Call Type and

Window Call

Long calls and Short calls were used.

Voice Telephony was tested in Mobile Originating Call (MOC)

direction. The following call durations were used:

CD1: 10 seconds for call setup testing;

CD2: 120 seconds for typical tests, default call duration;

Call Window: Call Duration + 30 seconds, (for the setup and release

phases) + 30 seconds (for the minimum pause. Interval), for the

default call duration this results in 180 seconds.6

Equipment

The equipment used for testing was the TEMS Symphony 7.7 using

Samsung Galaxy Note S4 (SM-N910F), Dell Latitude with Probe

Controller 1.9

4 Section 6.3 of ETSI TS 102 250-4 V.1.1.2 (2003-10) 5 Section 4.8 of ETSI EG 202 057-3 V1.1.1 (2005-04) 6 Section 4.2.1 of ETSI TS 102 250-5 V1.2.1 (2005-05)

6 | P a g e

KPI

The measurements focused on the following network parameters:

(i) Dropped Call Rate (DCR);

(ii) Call Set-up Success Rate (CSSR)

Log files

The log files for each test case were stored in different location with

different names. The log files were recorded per network operator.

Mobile terminal

used

The test calls were terminated on each operator’s test platform or IVR

system.

Network tested Cell-C, MTN, Vodacom and Telkom.

Sim card Test sim cards provided by each operator were used during the drive

test.

Vehicle used

Toyota Hilux Double Cab equipped with drive test equipment was used.

All road traffic rules were observed during the drive test. The speed

was maintained at an average of 60km in town and built-up areas and

100km/h on highways respectively.

2.3 Computational Methodology

a) Drop-Call Rate

The measurements of dropped call rate is described by the ratio of successful originated

calls that were found to drop, to the total number of successful originated calls that were

correctly released.

The formula to calculate DCR is shown below:

Table 3: DCR computational formula

b) The Call Setup Success Rate

The Call setup Success Rate refers to the percentage of calls that are successfully set up

and terminated as a percentage of the total call attempts. CSSR excludes dropped calls

Drop Call Rate

Successful Call

Attempts S

Drop Calls D

DCR D/S * 100

7 | P a g e

or calls that experience no network condition, low speech quality calls and calls with long

set up time. The formula to calculate CSSR is shown below7:

Table 4: CSSR computational formula

Call Success Rate

Call Attempts X

Call Success Y

CSSR (Y/X)*100

7 Annexure B.1 Measurement based on networks counters (ETSI EG 202-057-3)

8 | P a g e

3 Results and Analysis

This section provides a summary of the mobile operator’s performance results based on the

drive test route and quality of service parameters.

3.1 Parameter Targets

The Drop Call Rate should be less than 3% as per the End-User and Subscriber Service

charter regulations of 2009. The Call Setup Success Rate should be greater than 95%, which

is not regulated by the End-User and Subscriber Service Charter Regulations of 2009. This

target is based on ITU recommendations E.800 and G.1000.

3.2 Results

3.2.1 Accessibility Measurement Results

The areas covered were: Cato Ridge, Pietermaritzburg, Richmond, Mpophomeni and Howick.

Table 5: Cato Ridge Accessibility Results

Table 6: Pietermaritzburg Accessibility Results

Table 7: Richmond Accessibility Results

Cell CVodaCom-

SAMTN-SA Telkom Cell C

VodaCom-SA

MTN-SA Telkom Cell CVodaCom-

SAMTN-SA Telkom

453 492 499 413 414 439 455 369 867 931 954 782

419 479 476 387 389 426 438 355 808 905 914 742

34 13 23 26 25 13 17 14 59 26 40 40

92.49% 97.36% 95.39% 93.70% 93.96% 97.04% 96.26% 96.21% 93.19% 97.21% 95.81% 94.88%

PHASE 1 PHASE 2 TOTAL

Cato Ridge

Call Attempt

Call Established

Call Setup Failure

Call Setup Success Rate

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

454 452 451 442 441 424 448 433 895 876 899 875

432 403 411 424 428 371 429 419 860 774 840 843

22 49 40 18 13 53 19 14 35 102 59 32

95.15% 89.16% 91.13% 95.93% 97.05% 87.50% 95.76% 96.77% 96.09% 88.36% 93.44% 96.34%

Pietermaritzburg


Call Attempt

Call Established

Call Setup Failure


Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

425 586 547 303 310 406 395 315 735 992 942 618

395 562 522 286 276 380 388 303 671 942 910 589

30 24 25 17 34 26 7 12 64 50 32 29

92.94% 95.90% 95.43% 94.39% 89.03% 93.60% 98.23% 96.19% 91.29% 94.96% 96.60% 95.31%

TOTAL

Richmond

Call Attempt

Call Established

Call Setup Failure

PHASE 1 PHASE 2

Call Setup Success

Rate

9 | P a g e

Table 8: Mpophomeni Accessibility Results

Table 9: Howick Accessibility Results

Figure 2: KwaZulu-Natal Call Setup Success Rate (CSSR)

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

341 348 344 315 424 447 411 338 765 795 755 653

318 315 318 294 402 401 383 322 720 716 701 616

23 33 26 21 22 46 28 16 45 79 54 37

93.26% 90.52% 92.44% 93.33% 94.81% 89.71% 93.19% 95.27% 94.12% 90.06% 92.85% 94.33%


Mpophomeni

Call Attempt

Call Established

Call Setup Failure


Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

298 559 361 555 396 464 497 409 694 1023 858 964

267 525 349 543 355 443 490 378 622 968 839 921

31 34 12 12 41 21 7 31 72 55 19 43

89.60% 93.92% 96.68% 97.84% 89.65% 95.47% 98.59% 92.42% 89.63% 94.62% 97.79% 95.54%


Howick

Call Attempt

Call Established

Call Setup Failure


Cato Ridge Pietermaritzburg Richmond Mpophomeni Howick

CellC 93.19% 96.09% 91.29% 94.12% 89.63%

Vodacom 97.21% 88.36% 94.96% 90.06% 94.62%

MTN 95.81% 93.44% 96.60% 92.85% 97.79%

Telkom 94.88% 96.34% 94.39% 94.33% 95.54%

Target 95% 95% 95% 95% 95%

82.00%

84.00%

86.00%

88.00%

90.00%

92.00%

94.00%

96.00%

98.00%

100.00%

Cal

l Set

up

Su

cces

s R

ate

(%)

KwaZulu-Natal Call Setup Success Rate

10 | P a g e

3.2.2 Retainability Measurements Results

The areas covered were; Cato Ridge, Pietermaritzburg, Richmond, Mpophomeni and Howick.

Table 10: Cato Ridge Retainability Results

Table 11: Pietermaritzburg Retainability Results

Table 12: Richmond Retainability Results

Table 13: Mpophomeni Retainability Results

Table 14: Howick Retainability Results

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

134 148 150 81 123 132 135 112 257 280 285 193

127 140 147 71 120 127 131 102 247 267 278 173

7 0 8 18 13 2 5 26 20 2 13 44

5.51% 0.00% 5.44% 25.35% 10.83% 1.57% 3.82% 25.49% 8.10% 0.75% 4.68% 25.43%


Cato Ridge

Call AttemptCall EstablishedDropped Call

Drop Call Rate

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

135 137 140 128 131 130 132 119 266 267 272 247

127 129 137 120 121 123 128 112 248 252 265 232

12 2 5 15 11 2 5 15 23 4 10 30

9.45% 1.55% 3.65% 12.50% 9.09% 1.63% 3.91% 13.39% 9.27% 1.59% 3.77% 12.93%

Pietermaritzburg



Drop Call Rate

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

147 179 160 107 93 118 119 79 240 297 279 186

131 174 153 102 84 114 117 74 215 288 270 176

7 3 5 13 7 5 8 20 14 8 13 33

5.34% 1.72% 3.27% 12.75% 8.33% 4.39% 6.84% 27.03% 6.51% 2.78% 4.81% 18.75%Drop Call Rate

TOTAL

Richmond


PHASE 1 PHASE 2

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

104 105 105 86 128 136 126 107 232 241 231 193

97 101 102 85 115 127 121 101 212 228 223 186

11 4 6 16 7 1 7 22 18 5 13 38

11.34% 3.96% 5.88% 18.82% 6.09% 0.79% 5.79% 21.78% 8.49% 2.19% 5.83% 20.43%


Mpophomeni


Drop Call Rate

Cell CVodaCom-


VodaCom-SA


SAMTN-SA Telkom

148 160 173 123 124 139 147 89 272 299 320 212

133 152 170 99 111 135 142 79 244 287 312 178

15 11 3 20 19 7 2 21 34 18 5 41

11.28% 7.24% 1.76% 20.20% 17.12% 5.19% 1.41% 26.58% 13.93% 6.27% 1.60% 23.03%


Howick

Call Attempt

Call Established

Dropped Call

Drop Call Rate

11 | P a g e

Figure 3: KwaZulu-Natal Drop Call Rate (DCR)

3.3 KwaZulu-Natal Average Measurements Results

3.3.1 Average Accessibility measurement data results

The result below shows the average combination of the total Accessibility results in the tested areas (Cato

Ridge, Pietermaritzburg, Richmond, Mpophomeni and Howick).

Table 15: Average Accessibility Measurement Data

Province Network Operator CSSR Target (%)

KwaZulu-Natal Vodacom 93.04% 95%

MTN 95.30% 95%

Cell-C 92.86% 95%

Telkom 95.07% 95%

Figure 4: KwaZulu-Natal Average Call Setup Success Rate

Cato Ridge Pietermaritzburg Richmond Mpophomeni Howick

CellC 8.10% 9.27% 6.51% 8.49% 13.93%

Vodacom 0.75% 1.59% 2.78% 2.19% 6.27%

MTN 4.68% 3.77% 4.81% 5.83% 1.60%

Telkom 25.43% 12.93% 18.75% 20.43% 23.03%

Target 3% 3% 3% 3% 3%

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%C

all D

rop

Rat

e (%

)KwaZulu-Natal Call Drop Rate

92%

92%

93%

93%

94%

94%

95%

95%

96%

Target >= Vodacom MTN Cell-C Telkom

Average CSSR 95% 93.04% 95.30% 92.86% 95.07%

Ave

rage

CSS

R (

%)

KwaZulu-Natal Average Call Setup Success Rate

12 | P a g e

3.3.1.1 Analysis of the Average Accessibility Results: KwaZulu-Natal

Based on the average Accessibility results, MTN and Telkom met the average accessibility target, while

Vodacom and Cell-C did not met the target.

3.3.2 Average Retainability measurement data results for KwaZulu-Natal Province

The results below show the average combination of the total Retainability results in tested areas (Cato

Ridge, Pietermaritzburg, Richmond, Mpophomeni and Howick).

Table 16: Average Retainability Results

Province Network Operator DCR Target (%)

KwaZulu-Natal Vodacom 1.83% 3%

MTN 4.77% 3%

Cell-C 8.09% 3%

Telkom 19.39% 3%

Figure 5: KwaZulu-Natal: Average Call Drop Rate

3.3.2.1 Analysis of the Results

Based on the average Retainability results, Vodacom met the average Retainability target.

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

Target <= Vodacom MTN Cell-C Telkom

DCR 3% 1.83% 4.77% 8.09% 19.39%

Ave

rage

DC

R (

%)

KwaZulu-Natal: Average Call Drop Rate

13 | P a g e

3.3.3 Serving Technology Distribution

Figure 6: KwaZulu-Natal: Serving Technology Distribution

Vodacom, MTN and Telkom voice services were on GSM technology more than 50% of the time, and Cell C voice service was on WCDMA for more than 50% of the time along the

drive test route (See Appendix A: Section 5.2).

4 Conclusion

The QoS monitoring report in KwaZulu-Natal Province focused on the following KPIs,

Call Set-up Success Rate (CSSR) and Drop Call Rate (DCR) in Cato Ridge,

Pietermaritzburg, Richmond, Mpophomeni and Howick.

Table 17: KwaZulu-Natal Cape Province - Results Summary

Area Tested Network Operator

Results


Drop Call Rate

Cato Ridge

Vodacom Passed Passed

MTN Passed Failed

Cell-C Failed Failed

Telkom Failed Failed

Pietermaritzburg

Vodacom Failed Passed

MTN Failed Failed

Cell-C Passed Failed

Telkom Passed Failed

14 | P a g e

Richmond


MTN Passed Failed



Mpophomeni


MTN Failed Failed



Howick


MTN Passed Failed


Telkom Passed Failed

From the results above, it can be deduced that in terms of the two key parameters, Cell-C

achieved the Accessibility target in Pietermaritzburg only but failed the Retainability test in

all tested areas. Vodacom met the Accessibility target in Cato Ridge only and Retainability

test in all tested areas. MTN met the Accessibility test in Cato Ridge, Richmond and Howick,

but failed the Retainability in all tested areas.

All operators failed the Accessibility test in Mpophomeni. MTN, Cell-C and Telkom failed the

Retainability test in all tested areas.

Based on the average results, the Retainability KPIs, which are expressed as Average Drop

call rate (ADCR), Vodacom met the target. The Accessibility KPI, expressed as Average Call

setup success rate (ACSSR), MTN and Telkom met the average accessibility target but

Vodacom and Cell-C did not met the target.

15 | P a g e

5 Appendix A (KwaZulu-Natal coverage maps)

5.1 Signal Levels for Serving Cells

5.1.1 Vodacom

GSM UMTS

Figure 7: KwaZulu-Natal Vodacom GSM and UMTS Signal Levels

5.1.2 MTN

GSM UMTS

Figure 8: KwaZulu-Natal MTN GSM and UMTS Signal Levels

16 | P a g e

5.1.3 Cell-C

GSM UMTS

Figure 9: KwaZulu-Natal Cell C GSM and UMTS Signal Levels

5.1.4 Telkom

GSM UMTS

Figure 10: KwaZulu-Natal Telkom GSM and UMTS Signal Levels

17 | P a g e

5.2 Serving Technology Details

5.2.1 Vodacom

Vodacom MTN

Cell-C Telkom

Figure 11: KwaZulu-Natal Serving Technology Plots

18 | P a g e

Documents

Quality of Service Report: KwaZulu-Natal Province 2015 ... · Quality of Service Report: KwaZulu-Natal Province ... and WCDMA (900, 2100) ... CSSR excludes dropped calls