Independent Drive Test Report Sikkim Report... · • If Radio Link Timeout of > 40 is considered for Aircel & Idea. Call Drop Rate would be higher than the present call drop rate

Independent Drive Test Report

- May’16

Darjeeling & Sikkim

Key performance indicators for most of the operators: Most of the operators failed to meet the Call

Setup Success Rate (CSSR) where as all of the operators fall below the 2% CDR threshold

Methodology:

Independent Drive test was conducted in Darjeeling & surrounding areas from 9:00 AM to 9:00 PM from 25th to 27th of May 2016. The total

drive test covered in Darjeeling & Sikkim area was approximately 302 km over a period of 3 days. A total of ~3000 calls were made for five 2G

networks, four 3G networks and one CDMA networks covering six operators

1 2 3 4

City-Level

Performance

Area-Level

Performance Analysis Summary

and

Highlights

Date: 01 July 2016

5

Appendix I. Methodology

II. Benchmarking KPIs

III. Coverage

IV. Receive (Rx) Quality

V. City-Level Ranking

VI. City-Level KPI

The independent drive test has been carried out by M/s IMRB International on behalf of TRAI

1. City-Level Performance

City-Level Details Operator Performance against Key Performance Indicators :

1. Call Setup Success Rate, 2. Blocked Call Rate, 3. Call Drop Rate, 4. Rx Quality

Legends Threshold for each KPI are considered as per TRAI guidelines

Abbreviation / Definition: CSSR (benchmark >=95%) : Call Setup Success Rate

BCR (benchmark <=3%): Blocked Call Rate

CDR (benchmark <=2%): Call Drop Rate

Good RxQuality (benchmark >=95%): 2G (RxQual <=5), 3G (EcNo >=-15dBm), CDMA (FER <=4%)

Most of the operators failed to

meet the Call Setup Success Rate

benchmark of >=95% except

Aircel 2G & 3G, MTS CDMA and

Vodafone 3G. Refer. Fig 1

Most of the operators failed to

meet the Call Blocked Rate

benchmark of <=3% except MTS

CDMA. Refer. Fig 2

All operators failed to meet the

Call Drop Rate benchmark of

<=2%. Refer. Fig 3

Most of operators failed to meet

the Rx Quality Rate benchmark of

>=95% except BSNL 3G and

Vodafone 3G.

Refer. Fig 4

Operator meeting the benchmark Operator not meeting the benchmark

95.43% 95.00% 90.63% 91.55%

86.76% 89.74% 93.58%

99.66% 93.72% 95.35%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Fig 1. Call Setup Success Rate (%)

95%

4.57% 5.00% 9.38% 8.45%

13.24% 10.26%

6.42%

0.34%

6.28% 4.65%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Fig 2. Blocked Call Rate (%)

3%

12.87% 14.80% 15.17% 18.85% 21.05% 18.86%

5.07%

11.56% 10.79% 13.24%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Fig 3. Call Drop Rate (%)

83.61% 94.71%

74.84% 92.75%

79.19% 96.22% 87.09% 88.62% 82.36%

95.83%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Fig 4. Rx Quality (%)

95%

2%

2. Area-Level Performance

Area-Level Details

Legends

Most of the areas of Darjeeling & Sikkim have failed to meet the Call Setup Success Rate

benchmark of >=95% except Darjeeling to Kalimpong Road & Kurseong.

Fig 5. Area Level CSSR KPI Performance ( all operators inclusive)

Areas meeting the benchmark Areas not meeting the benchmark





3

Call Setup Success Rate Aircel Airtel BSNL Idea MTS CDMA Vodafone

Darjeeling 95.0% 86.9% 90.0% 95.9% 98.9% 93.8%

Darjeeling to Kalimpong Road 100.0% 90.5% 100.0% 95.2% 100.0% 93.8%

Gangtok 94.4% 91.4% 86.4% 91.1% 100.0% 93.3%

Kalimpong 94.4% 94.5% 88.1% 95.0% 100.0% 94.9%

Kalimpong to Gantok Road 97.2% 83.3% 92.3% 90.0% 100.0% 96.3%

Kurseong 97.4% 97.3% 90.5% 93.6% 100.0% 97.3%


Area-Level Details

Legends

Fig 6. Area Level Blocked Call Rate KPI performance ( all operators inclusive)

All of the areas of Darjeeling & Sikkim have failed to meet the benchmark for Blocked Call

Rate of <=3%.





4


Blocked Call Rate Aircel Airtel BSNL Idea MTS CDMA Vodafone

Darjeeling 5.0% 13.1% 10.0% 4.1% 1.1% 6.2%


Gangtok 5.6% 8.6% 13.6% 8.9% 0.0% 6.7%

Kalimpong 5.6% 5.5% 11.9% 5.0% 0.0% 5.1%


Kurseong 2.6% 2.7% 9.5% 6.4% 0.0% 2.7%


Area-Level Details

Legends

All areas of Darjeeling & Sikkim have failed to meet the Call Drop Rate

benchmark of <=2%.

Fig 7. Area Level Call Drop Rate KPI Performance ( all operators inclusive)





5


Call Drop Rate Aircel Airtel BSNL Idea MTS CDMA Vodafone

Darjeeling 11.6% 22.3% 22.2% 5.3% 8.8% 11.4%


Gangtok 21.3% 19.5% 20.9% 5.9% 15.3% 11.2%

Kalimpong 3.7% 6.7% 13.5% 3.5% 9.1% 8.0%


Kurseong 6.7% 6.8% 17.5% 2.3% 6.8% 4.1%


Area-Level Details

Legends

All areas of Darjeeling & Sikkim have failed to meet the Rx Quality benchmark of

>=95% .

Abbreviation / Definition: CSSR (benchmark >= 95%) : Call Setup Success Rate




Fig 8. Area Level Rx Quality KPI performance (all operators inclusive)

6


Rx Quality Aircel Airtel BSNL Idea MTS CDMA Vodafone

Darjeeling 87.3% 76.4% 61.9% 92.3% 90.0% 84.9%


Gangtok 81.8% 83.9% 87.0% 84.3% 89.9% 84.7%

Kalimpong 89.4% 84.6% 85.8% 92.1% 94.5% 90.1%


Kurseong 88.1% 89.0% 84.8% 82.4% 86.8% 90.5%

3. Analysis

Detailed Analysis of Operators

Legends Current Drive: May 2016

Operator meeting the set CDR threshold Operator not meeting the set CDR threshold





High RLT

• Radio Link Timeout (RLT indicates how long the call can

sustain during poor quality – Higher the value the longer

it will sustain the call). Higher the RLT, poorer the

consumer experience during the call.

• If Radio Link Timeout of > 40 is considered for Aircel &

Idea. Call Drop Rate would be higher than the present

call drop rate.

Operators No. of Cells

with RLT <= 40

No. of Cells

with RLT > 40 Current CDR

CDR, If all

Cells assume

RLT <= 40

Airtel 93.1% 6.9% 15.17% 15.17%

Aircel 15.5% 84.5% 12.87% 14.97%

Vodafone 87.5% 12.5% 10.79% 10.79%

Idea 75.2% 24.8% 5.07% 5.67%

BSNL 90.4% 9.6% 21.05% 21.05%

3. Analysis

Detailed Analysis of Operators

Legends Current Drive: May 2016

Operator meeting the set CDR threshold Operator not meeting the set CDR threshold





All 2G Networks

. • The observation shows that poor quality and poor mobility in 2G network is a clear indication of high Call drop

rate. • Another reasons for High drop is coverage holes which is common to all operators due to hilly terrain.

All 3G networks

• All 3G networks in drive area have limited coverage which allowed to maximum number of calls attempted on 2G networks which again results to drop call due to poor quality and poor Mobility on 2G networks.

MTS CDMA (CDR : 11.56%, BCR : 0.34%)

• Rx quality of MTS CDMA is poor and not meeting benchmark criteria of >=95%.

• Coverage holes in drive test area is another reasons for high call drop Rate.

• Poor signal quality is the clear indication of high call drop in network.

Parameter Value(%)

Rx Quality(Good sample <=4% of total FER sample)

88.62%

Parameter Value (%)

Aircel 2G Airtel 2G BSNL 2G Idea 2G Vodafone 2G

Rx Quality(Good sample <=5) 83.61% 74.84% 79.19% 87.09% 82.36%

CSSR 95.43% 90.63% 86.76% 93.58% 93.72%

HOSR 84.09% 81.67% 81.04% 99.24% 90.82%

CDR 12.87% 15.17% 21.05% 5.07% 10.79%

Parameter

Value (%)

Aircel 3G Airtel 3G BSNL 3G Vodafone 3G

Time spend on 3G (%) 26.59% 46.42% 30.27% 34.93%

4. Summary & Highlights

Summary & Highlights

All operators failed to meet the Call Drop Rate Benchmark of <=2%

Key Highlights

• All operators failed to meet the Call Drop Rate benchmark of <=2% on 2G and 3G networks.

• All other operators fail to meet the benchmark by a wide margin. Other than Idea 2G, all operators have a Call Drop

rate of >10%. A high call drop rate, clearly indicate urgent need for improvement in order to deliver reasonable levels

of service

Fig 9. Call Drop Rate

12.87% 14.80% 15.17% 18.85%

21.05% 18.86%

5.07%

11.56% 10.79% 13.24%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Call Drop Rate (%) CDR Benchmark (<=2%)

5. Appendix

Ancillary Details

I. Methodology & Other Details

II. Benchmarking KPI Details

III. Coverage Details

IV. Receive (Rx) Quality Details

V. City-Level Ranking and KPI details

VI. City-Level KPI details

Table of Contents in Appendix

Appendix

Ancillary Details

1. Methodology & Other Details

Overview:

Telecom Regulatory Authority of India has been entrusted with the task of laying down the standards of quality of service to be provided

by the service providers and ensuring that the quality of service is provided as per norms; and also TRAI is responsible for conducting

the periodical audit of such services provided by the service providers so as to protect the interest of the consumers of

telecommunications service

TRAI is regularly monitoring the performance of Telecom Service Providers (TSP) against the benchmarks for the various Quality of

Service (QoS) parameters laid down by the Authority. TSPs submit Performance Monitoring Reports to TRAI every quarter. TRAI also

undertakes audit and assessment of Quality of Service through independent agencies to verify the Quality of Service claimed. The Audit

agencies conduct sample ‘Drive tests’ across various cities all over the country as part of audit and assessment of the TSPs’

performance

In view of complaints on call drops and other network issues, on behalf of TRAI, an Independent Drive Test (IDT) was conducted from

25th May 2016 to 27h May 2016 covering various locations in Darjeeling, South Darjeeling, Gangtok. The performance of Aircel, Airtel,

BSNL, Idea, Vodafone, MTS was monitored across various technologies (2G, 3G and CDMA). The drive test route was defined on the

basis of several factors that include - areas from where call drop complaints are commonly received; areas of heavy usage; residential

areas away from arterial roads; office areas; areas where previous Drive tests showed network issues; etc.

For the purpose of these tests, 2G + 3G Mode (Auto) was used. The test results obtained from these drive tests were conducted to

assess the network condition more specifically in terms of Radio Frequency (RF) Coverage; Rx Quality; Call Setup Success Rate; Call

Drop Rate; Blocked Calls and Carrier to Interference ratio

Drive Route Details For Darjeeling Area:

Independent Drive test was conducted in Darjeeling area from 9:00 AM to 9:PM from 25th to 27th of May 2016. The total drive test

covered in Darjeeling area was approximately 302 km over a period of 3 days. Total of 3008 calls were made for five 2G networks,

four 3G networks and one CDMA networks covering Six operators. Roads and Areas covered in Darjeeling & Sikkim is shown in the

below map.

Below is the List of Areas Covered and Drive Routes • Singhamari • Jawahar Parbat • Naya Basti • Lebong Cart Rd • Hill Card Rd • Laden La Rd • Gandhi Rd • Darjeeling Rly

Station • Ghoom Degree

College • NH 55 Hill Card

Rd • Rangbull • Sonada • Tung • Dowhill School • St. Marys • Kurseong Rly

Station • Pankhabari Rd • old Miltary Rd,

Kurseong

• Hill Card Rd • Rishi Rd • Peshok Rd • Log Depot • Lamahatta Druk • Lamahatta Mount View • Teesta Bazar Bus Stop • Teesta River • Chitray Bus Stop,

Kalimpong • SBI ATM, Kalimpong • Rockvale Academy, • Kalimpong • Bahadur Giri Rd,

Kalimpong • College of Agriculture,

Gangtok • Singtham Chungthang rd,

Gangtok • Animal Husbendry,

Gangtok • PNG School, Gangtok • ICAR Guest House,

Gangtok • Singtam • Chisopani • Topakhani

• Ranipool-Rumtek Rd • Durga Mandir, Gangtok • Adampool sewage plant,

Gangtok • Barad Sadan Academic • Sikkim manipal Insitute • Manipal Central Hospital • Tourism & Civil Dept • Singtham Chungthang

Rd, Gangtok • Sikkam Govt College • Sikkam university • Deorali Bazar Gurudwara • Animal Husbandry • Deorali Girls School • Sikkam Assembly • Ridge Park • Bhanu Path Rd • Enchey Sen Sec School • Ganesh Tok • Hanuman Tok • Sikkam Himalayan Park • Gangtok-chungthang Rd • Upper Sichey Rd

Appendix

Ancillary Details

2. Benchmarking KPI Details

RF Coverage

RF Coverage relates to the geographical footprint within the system that has sufficient RF signal strength to provide for a call/data

session. Coverage rate of an operator is calculated basis of % of samples in which the Rx level is better than -85 dBm

Accessibility

Accessibility is the ability of a service to be obtained within specific tolerances and other given conditions, when requested by the user.

In other words, the ability of a user to obtain the requested service from the system. Accessibility is monitored by measuring Call Setup

Success Rate (CSSR) which is defined as the ratio of Established Calls to Call Attempts

Retainability

Retainability is “The ability of a service, once obtained, to continue to be provided under given conditions for a requested duration.” For

determining the Retainability the regulation prescribes three important parameters namely Call Drop Rate (CDR), Worst affected Cells

having more than 3% TCH drop and Connection with good voice quality. Drop Call and Connection with good voice quality were

monitored by the drive test

Mobility

In a cellular system a base station has only a limited coverage area. Hence it is possible for a moving subscriber to be out of range of a

base station while making a call. The process by which a mobile telephone call is transferred from one base station to another as the

subscriber passes the boundary of a cell is called a handover. The Handover success rate (HOSR) more than 95% is considered to be

good

Rx Quality

For measuring voice quality, RxQual samples on a scale from 0 to 7 for GSM operators, EcNo from 0 to -30 dBm and Frame Error Rate

(FERs) for CDMA service providers are measured. As per the QoS norms, RxQual between 0- 5 for GSM operators, EcNo >= -15 dBm and

between 0-4% FER value for CDMA operators FERs is considered to be good, where as RxQual beyond this benchmark is considered to

be bad

Coverage % Accessibility % Retainability % Mobility % Rx Quality %

Aircel 2G 74.38% 95.43% 87.13% 84.09% 83.61%

Aircel 3G 38.98% 95.00% 85.20% 82.76% 94.71%

Airtel 2G 83.77% 90.63% 84.83% 81.67% 74.84%

Airtel 3G 44.75% 91.55% 81.15% 98.42% 92.75%

BSNL 2G 83.36% 86.76% 78.95% 81.04% 79.19%

BSNL 3G 41.18% 89.74% 81.14% 88.33% 96.22%

Idea 2G 47.81% 93.58% 94.93% 99.24% 87.09%

MTS CDMA 95.68% 99.66% 88.44% 100.00% 88.62%

Vodafone 2G 87.57% 93.72% 89.21% 90.82% 82.36%

Vodafone 3G 58.49% 95.35% 86.76% 99.49% 95.83%

0.00%

25.00%

50.00%

75.00%

100.00%

KPI Distribution

Appendix

Ancillary Details

3. Coverage Details

RF Coverage relates to the geographical footprint within the system that has sufficient RF signal strength to provide for a call/data

session. The Coverage rate of an operator is calculated basis of % of samples in which the Rx level is better than -85 dBm. The details

are as follows.

Operators not meeting benchmark

Aircel 2G Aircel 3G Airtel 2G

Airtel 3G BSNL 2G BSNL 3G

Operators Coverage Rate

Aircel 2G 74.38%

Aircel 3G 38.98%

Airtel 2G 83.77%

Airtel 3G 44.75%

BSNL 2G 83.36%

BSNL 3G 41.18%

Idea 2G 47.81%

MTS CDMA 95.68%

Vodafone 2G 87.57%

Vodafone 3G 58.49%

0.00%

25.00%

50.00%

75.00%

100.00%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Coverage Distribution

-95 to -120 dBm -85 to -95 dBm -75 to -85 dBm 0 to -75 dBm

Appendix

Ancillary Details

3. Coverage Details

IDEA 2G MTS CDMA Vodafone 2G

Vodafone 3G

Appendix

Ancillary Details

4. Receive (Rx) Quality Details

For measuring voice quality, RxQual samples on a scale from 0 to 7 for GSM operators, EcNo for 3G operators on a scale of 0 to -30

dBm and Frame Error Rate (FERs) for CDMA service providers are measured. As per the QoS norms, Rx Qual between 0- 5 for GSM

operators, EcNo >= -15 dBm and between 0-4% FER value for CDMA operators FERs is considered to be good, where as quality beyond

this benchmark is considered to be bad. The benchmark should usually be > =95%.


Aircel 2G Aircel 3G Airtel 2G

Airtel 3G BSNL 2G BSNL 3G

Operator Rx Quality

Aircel 2G 83.61%

Aircel 3G 94.71%

Airtel 2G 74.84%

Airtel 3G 92.75%

BSNL 2G 79.19%

BSNL 3G 96.22%

Idea 2G 87.09%

MTS CDMA 88.62%

Vodafone 2G 82.36%

Vodafone 3G 95.83%

0.00%

25.00%

50.00%

75.00%

100.00%

Air

cel 2

G

Air

cel 3

G

Air

tel 2

G

Air

tel 3

G

BS

NL

2G

BS

NL

3G

Idea

2G

MTS

CD

MA

Vo

da

fon

e 2

G

Vo

da

fon

e 3

G

Rx Quality Distribution

RxQual 6 to 7, EcNo <-15 dBm, FER <= 4% RxQual 0 to 5, EcNo >=-15 dBm, FER > 4%

Appendix

Ancillary Details

4. Receive (Rx) Quality Details

IDEA 2G MTS CDMA Vodafone 2G

Vodafone 3G

Appendix

Ancillary Details

5. City-Level Ranking and KPI details

Accessibility is the ability of a service to be obtained within specific tolerances and other given conditions, when requested by the user.

In other words, the ability of a user to obtain the requested service from the system. Accessibility is monitored by measuring Call Setup

Success Rate (CSSR) which is defined as the ratio of Established Calls to Call Attempts. The benchmark is >=95%. The details are as

follows:

Retainability is “the ability of a service, once obtained, to continue to be provided under given conditions for a requested duration.” For

determining the Retainability, the regulation prescribes three important parameters namely Call drop rate (CDR), Worst affected Cells

having more than 3% TCH drop and Connection with good voice quality. The call drop and connection with good voice quality were

monitored by the drive test

Call Drop Rate (CDR): The parameter gives a reliable measurement of the mobile network used by the service provider for maintaining a

call once it has been correctly established

Mobility: In a cellular system a base station has only a limited coverage area. Hence it is possible for a moving subscriber to get out of

reach of a base station while making a call. The process by which a mobile telephone call is transferred from one base station to

another as the subscriber passes the boundary of a cell is called a handover. The Handover Ssuccess Rate (HOSR) more than 95% is

considered to be good

*In case of multiple call failure in similar geo location in given period of 60sec has been counted as one call failure


Operator Total Call Attempt Blocked Calls Blocked Call Rate* CSSR

Aircel 2G 350 16 4.57% 95.43%

Aircel 3G 320 16 5.00% 95.00%

Airtel 2G 320 30 9.38% 90.63%

Airtel 3G 284 24 8.45% 91.55%

BSNL 2G 219 29 13.24% 86.76%

BSNL 3G 195 20 10.26% 89.74%

Idea 2G 358 23 6.42% 93.58%

MTS CDMA 295 1 0.34% 99.66%

Vodafone 2G 366 23 6.28% 93.72%

Vodafone 3G 301 14 4.65% 95.35%

Operator Total Call Established Total Call Drop CDR CCSR

Aircel 2G 334 43 12.87% 87.13%

Aircel 3G 304 45 14.80% 85.20%

Airtel 2G 290 44 15.17% 84.83%

Airtel 3G 260 49 18.85% 81.15%

BSNL 2G 190 40 21.05% 78.95%

BSNL 3G 175 33 18.86% 81.14%

Idea 2G 335 17 5.07% 94.93%

MTS CDMA 294 34 11.56% 88.44%

Vodafone 2G 343 37 10.79% 89.21%

Vodafone 3G 287 38 13.24% 86.76%

Operators Total HO Attempt HO Failure HO Success HO Failure Rate HO Success Rate

Aircel 2G 723 115 608 15.91% 84.09%

Aircel 3G 203 35 168 17.24% 82.76%

Airtel 2G 884 162 722 18.33% 81.67%

Airtel 3G 1138 18 1120 1.58% 98.42%

BSNL 2G 733 139 594 18.96% 81.04%

BSNL 3G 514 60 454 11.67% 88.33%

Idea 2G 262 2 260 0.76% 99.24%

MTS CDMA 729 0 729 0.00% 100.00%

Vodafone 2G 599 55 544 9.18% 90.82%

Vodafone 3G 584 3 581 0.51% 99.49%

Appendix

Ancillary Details

6. City-Level KPI details


Call Events

Operators (Summary)

Aircel Airtel BSNL IDEA MTS Vodafone

2G 3G 2G 3G 2G 3G 2G CDMA 2G 3G

Call Attempt (May-16) 350 320 320 284 219 195 358 295 366 301

Blocked Call Rate (May-16) 4.57% 5.00% 9.38% 8.45% 13.24% 10.26% 6.42% 0.34% 6.28% 4.65%

CSSR (Accessibility) (May-16) 95.43% 95.00% 90.63% 91.55% 86.76% 89.74% 93.58% 99.66% 93.72% 95.35%

Dropped Call Rate (May-16) 12.87% 14.80% 15.17% 18.85% 21.05% 18.86% 5.07% 11.56% 10.79% 13.24%

Mobility HOSR (May-16) 84.09% 82.76% 81.67% 98.42% 81.04% 88.33% 99.24% 100.00% 90.82% 99.49%

Rx Quality (May-16) 83.61% 94.71% 74.84% 92.75% 79.19% 96.22% 87.09% 88.62% 82.36% 95.83%

Operator meeting the benchmark

Documents

Independent Drive Test Report Sikkim Report... · • If Radio Link Timeout of > 40 is considered for Aircel & Idea. Call Drop Rate would be higher than the present call drop rate