48

3G Troubleshooting Presentation 082509

  • Upload
    samna6

  • View
    163

  • Download
    11

Embed Size (px)

Citation preview

Page 1: 3G Troubleshooting Presentation 082509
Page 2: 3G Troubleshooting Presentation 082509
Page 3: 3G Troubleshooting Presentation 082509
Page 4: 3G Troubleshooting Presentation 082509
Page 5: 3G Troubleshooting Presentation 082509

Call SetupCall Setup

Page 6: 3G Troubleshooting Presentation 082509

Signalling Connection Handling: Call SetupSignalling Connection Handling: Call Setup

UE Node B RNC

RRC: RRC Connection Request (CCCH) The UE sends RRC Connection Request message with an establishment code (Emergency call, Registration,

Signalling, Originating Call, IRAT Cell Reselection...).

NBAP: Radio Link Setup Request

NBAP: Radio Link Setup Response

The RNC orders the RBS to reserve the necessary resources

The Admission Control algorithm checks if the new radio link can be allowed in the cell, power control algorithm sets the initial DL power, UL SIR target, code control algorithm allocates the UL-DL SC code and DL Channelization

code.

AAL2SIG: ERQ

AAL2SIG: ECF

The transport bearer (AAL2 connection) needed for signaling is set up over Iub by the RNC

RRC: RRC Connection Setup (CCCH)

The RRC Connection Setup message is used to specify the (signalling) radio bearer, the transport channel and the physical channel characteristics both in the UL and

downlink directions.

Page 7: 3G Troubleshooting Presentation 082509

Call Setup – RRC PartCall Setup – RRC Part

UE Node B RNC

RRC: RRC Connection Setup (CCCH)

L1 synchronization

NBAP: Radio Link Restore Indication

Once UE establish the radio link, the RBS

notifies the achievement of the new uplink radio

interface synchronization

RRC: RRC Connection Setup Complete (DCCH)

Integrity and Ciphering related parameters, together with the UE capabilities requested in RRC setup message are included.

These information are used by the RAB establishment procedure, UE Security Handling, and the Channel Switching function.

UE in CELL_DCH state

Page 8: 3G Troubleshooting Presentation 082509

Iu Control Plane Connection: Call SetupIu Control Plane Connection: Call Setup

UE Node B RNC SGSN

RRC: Initial Direct Transfer (CM Service Request)

The establishment is initiated by the reception of an RRC message containing the NAS message that

originally triggered the Signaling Connection setup.

RANAP: Initial UE Message (CM Service Request)

SCCP: CC

The SRNC sets up a Iu control plane connection to the CN and sends the

RANAP message to the CN. This RANAP message includes the UE NAS message, current location information

of UE and the Iu control plane connection identifier. RANAP: Direct Transfer

(Authentication Request)

RRC:DL Direct Transfer (Authentication Request)RRC:UL Direct Transfer

(Authentication Response) RANAP: Direct Transfer (Authentication Response)

SGSN/MSC performs authentication to make sure that the UE is genuine.

RANAP: Common ID

After the establishment of the Signaling Connection, the CN sends

RANAP Common ID message to inform the SRNC about the

permanent NAS UE Identity (IMSI).

Page 9: 3G Troubleshooting Presentation 082509

RAB Establishment PartRAB Establishment Part

Node B RNC

The CN initiates establishment of the Radio Access Bearer using the RAB Assignment Request message. This message includes the QoS of the call being established.

The SRNC determines the Radio Connection configuration and checks the UE capabilities if the configuration can be supported by the UE.

RANAP:RAB Assignment Request

UE CN

NBAP: Radio Link Reconfiguration Prepare

NBAP:Radio Link Reconfiguration Ready

SRNC requests Node B to prepare establishment of DCH to carry the RAB.

Node B allocates the resources and responds with the Ready message.

ALCAP:ERQ

ALCAP:ECF

ALCAP:ERQ

ALCAP:ECF The data transport bearer over the Iu CS/PS and Iub is set up by the RNC RRC:Radio Bearer Setup

RRC:Radio Bearer Setup Complete

RANAP:RAB Assignment Response

SRNC sends the RB Setup message to add the new DCH

Page 10: 3G Troubleshooting Presentation 082509

Reasons for RRC+RAB failures

- If resources are not allocated in the NodeB (Admission control, Power usage per RL, DL channelization codes, DL/UL Channel element usage)

- If resources are not allocated in the RNC subracks – related to MP load, any outages or software issues

- If Iub link is not established, Aal2 signalling issues

- Synchronization failures (link Imbalance) – TMA, feeder, jumper, swapped sectors, faulty NodeB, RAX/Tx board)

- Radio link connection reset (happens in PS domain)

- Incorrect NodeB script files, software issues

- RNC module

Page 11: 3G Troubleshooting Presentation 082509

Reasons for RAB failures

- Insufficient channel elements – is related to faulty Tx and RAX boards. May need additional boards

- RAB is assigned by CN and it depends upon kind of service requested by UE (conversational, streaming, interactive and background). So, incorrect QOS requirements can lead to RAB failures. Different RAB’s will need different protection and hense different transport channels

- Check DL BLER for different services

- During call setup, handover to 2G happens duirng RAB setup. So, IRAT neighbors and priority should be properly defined.

- Multiple data should be correlated to isolate the problem

Page 12: 3G Troubleshooting Presentation 082509
Page 13: 3G Troubleshooting Presentation 082509
Page 14: 3G Troubleshooting Presentation 082509

Importance of defining correct neighbors and Priority

Page 15: 3G Troubleshooting Presentation 082509
Page 16: 3G Troubleshooting Presentation 082509

2-Way SHO: SC 165 is the serving cell, in active set & SC221 qualifies to be added to the active set.2-Way SHO: SC 165 is the serving cell, in active set & SC221 qualifies to be added to the active set.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Temp Array149 157 52 68 323 331 221 229 124 133 141 268 395 28 36 237 117 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 2

403 345

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 6229 237 395 403 165 268 323 331 68 141 28 149 157 181 155 236 717 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 8

213 125 96 91011

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1213

17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 141516

Result - BQU4659C (SC165)

Result - BQU4841A (SC221)

Page 17: 3G Troubleshooting Presentation 082509

STEP 1 - Add both the active cells (A) and (B) to the listed set in the same position as they existed previouslySTEP 1 - Add both the active cells (A) and (B) to the listed set in the same position as they existed previously..

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Temp Array165 149 157 52 68 323 331 221 229 124 133 141 268 395 28 36 117 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 2

237 403 345

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 6221 229 237 395 403 165 268 323 331 68 141 28 149 157 181 155 717 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 8

236 213 125 96 91011

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 12165 221 1317 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 14

1516

Result - BQU4659C (SC165)

Result - BQU4841A (SC221)

Page 18: 3G Troubleshooting Presentation 082509

STEP 2 - Take the neighbor cell with the highest priority for the best active set cell, cell(A), and if it already exists in the old listed set add it to the new listed set in the same position. If STEP 2 - Take the neighbor cell with the highest priority for the best active set cell, cell(A), and if it already exists in the old listed set add it to the new listed set in the same position. If it does not exist in the old listed set, then the position does not matter, therefore store it for addition later in a temporary array.it does not exist in the old listed set, then the position does not matter, therefore store it for addition later in a temporary array.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Temp Array165 149 157 52 68 323 331 221 229 124 133 141 268 395 28 36 117 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 2

237 403 345

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 6221 229 237 395 403 165 268 323 331 68 141 28 149 157 181 155 717 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 8

236 213 125 96 91011

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 12165 149 221 1317 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 14

1516

Result - BQU4659C (SC165)

Result - BQU4841A (SC221)

Page 19: 3G Troubleshooting Presentation 082509

STEP 3 - Take the neighbor cell with the highest priority for the second best active set cell(B), and if it already exists in the old listed set add it to the new listed set in the same position. STEP 3 - Take the neighbor cell with the highest priority for the second best active set cell(B), and if it already exists in the old listed set add it to the new listed set in the same position. If it does not exist in the old listed set, then the position does not matter and it can be stored in a temporary array for later addition. Store the neighboring cell only if it does not already If it does not exist in the old listed set, then the position does not matter and it can be stored in a temporary array for later addition. Store the neighboring cell only if it does not already exist in the temporary array (avoid duplicate). If it is already stored in the temporary array, take the next neighboring cell in priority order from the next cell in the Active Set, cell(A) exist in the temporary array (avoid duplicate). If it is already stored in the temporary array, take the next neighboring cell in priority order from the next cell in the Active Set, cell(A) applying the same rules.applying the same rules.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Temp Array165 149 157 52 68 323 331 221 229 124 133 141 268 395 28 36 117 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 2

237 403 345

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 6221 229 237 395 403 165 268 323 331 68 141 28 149 157 181 155 717 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 8

236 213 125 96 91011

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 12165 149 221 229 1317 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 14

1516

Result - BQU4659C (SC165)

Result - BQU4841A (SC221)

Page 20: 3G Troubleshooting Presentation 082509

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Temp Array165 149 157 52 68 323 331 221 229 124 133 141 268 395 28 36 1 18117 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 2 155

237 403 3 2364 2135 125

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 6 96221 229 237 395 403 165 268 323 331 68 141 28 149 157 181 155 717 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 8

236 213 125 96 91011

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 12165 149 157 52 68 323 331 221 229 124 133 141 268 395 28 36 1317 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 14

237 403 181 155 236 213 125 96 1516

Result - BQU4659C (SC165)

Result - BQU4841A (SC221)

STEP 4 - Repeat until all neighbor cells have been processed or until MaxIafMonSubsetIAF neighboring cells have been selected for the listed set.STEP 4 - Repeat until all neighbor cells have been processed or until MaxIafMonSubsetIAF neighboring cells have been selected for the listed set.

STEP 5 - Take the cells that have been selected to be included in the new listed set (stored in the temporary array) and add them to the listed set by filling the spaces that have not been filled in step (4), the neighboring cells are picked from the temporary array in the order they was stored (FIFO). (This makes sure that neighboring cells stored early in the temporary array will be the first to fill out the spaces in the listed set).

Page 21: 3G Troubleshooting Presentation 082509

Future slides………….

Page 22: 3G Troubleshooting Presentation 082509

The following are some important KPI’s to be monitored on a daily basis

1. Cell Availability

2. UL Synchronization failures

3. Voice Traffic Erlangs

4. Voice Access Failures

5. Voice Drops

6. Missing Neighbor

7. Inability to add neighbor

8. Voice 3G_2G IRAT Failure

9. Soft/Softer Overhead

Page 23: 3G Troubleshooting Presentation 082509

If Cell Availability is 0, check the following

• Cell Reserved = 1 (not_reserved), all Accessclasses= FALSE. See next few slides for details

•Cell is locked

• If it has Critical or Major alarms, most likely there is a TT open. If not, open one and assign it to Switch/FOPS

• If you don’t find any thing, it could be a sleepy. Restart NodeB as shown below

Page 24: 3G Troubleshooting Presentation 082509

Go to WCDMA RAN Explorer as shown

Scroll down and select the NodeB you want to troubleshoot as shown in the next slide

Page 25: 3G Troubleshooting Presentation 082509

Go here to change Cell Reserved and Accessclass parameters

Page 26: 3G Troubleshooting Presentation 082509

All 3G onair cells should be “Not_Reserved”. Click Apply after all changes

Page 27: 3G Troubleshooting Presentation 082509

All of the following parameters [1 to 16] should be set to FALSE accessClassNBarred

accessClassesBarredCS, accessClassesBarredPS

Page 28: 3G Troubleshooting Presentation 082509

Note that “Cell Status” is automatically changed. Administrative State can be changed to “Locked” something to look out after a TT is closed

Page 29: 3G Troubleshooting Presentation 082509

To check if the site is taking traffic, Right click on NodeB and go to Node Status Analyser

Page 30: 3G Troubleshooting Presentation 082509

If these counters are 0, cell is not taking any traffic. In case of Sleepy cell, you need to restart NodeB

Click on each cell to check RRC counters below

Page 31: 3G Troubleshooting Presentation 082509

Go to Element Manager to check Alarms, external TMA parameters, Restart RU, TxB, NodeB. Do a cold restart for NodeB in case of sleepy cell. Make sure you put reasons for restart including the TT number. Its better to call Switch if you want to restart

Page 32: 3G Troubleshooting Presentation 082509

Go to SupervisionAlarm Log

Move the mouse over any alarm and click “f1” to get description and troubleshooting options. You can populate TT or WO with these steps

Page 33: 3G Troubleshooting Presentation 082509

Go to Terminal to troubleshoot T1, RU, FU and TMA

Page 34: 3G Troubleshooting Presentation 082509

Step1

Step2

Step3

Page 35: 3G Troubleshooting Presentation 082509

Enter 1 to get RSSI and x to exit. RSSI is read at RU. Normal RSSI (between -102 and -106) indicates good RU

Page 36: 3G Troubleshooting Presentation 082509

This will give TMA current and FU voltage

For RRU sites use 000900 instead of 001200

Optimum value for current is 100. 90 is the lower limit and would need FOPS to check TMA

VSWR should be above 18

Value should be 29xxx

Page 37: 3G Troubleshooting Presentation 082509

Use the following commands from NodeB to check the status of RU, FU and T1

st ru

st fu

st t1

pget t1

get radiolink

al (alarms)

Lga (alarm history for past year, to get most recent, type 24h for 24 hour period, 6h for 6 hour period, and so on).

For T1 issues, assign TT to Switch. For hardware issues, assign TT or WO to FOPS

Page 38: 3G Troubleshooting Presentation 082509

U1NC3658A> st tx 101119-11:55:25 11.166.154.67 8.0d RBS_NODE_MODEL_L_10_9 stopfile=/tmp/13776===================================================================================Proxy Adm State Op. State MO=================================================================================== 171 0 (DISABLED) Equipment=1,Subrack=1,Slot=10,PlugInUnit=1,TxDeviceGroup=1 199 1 (ENABLED) Equipment=1,Subrack=1,Slot=11,PlugInUnit=1,TxDeviceGroup=1

Page 39: 3G Troubleshooting Presentation 082509

TO READ THE ERROR LOGS FOR HARDWARE: U1NC3658A> lhsh 001000 te log read 10 – slot number of HW. From above example it is the RAX board. Typically: Slot 4-9 are RAX boards, Slot 10-11 are TX boards Slot 2,4,6 are FU boards and so on.  This allows you to further troubleshoot the HW on a board level.

Page 40: 3G Troubleshooting Presentation 082509

Service Affecting Alarms

AntennaBranch_AntennaSystemProblem

AuxPlugInUnit_PiuConnectionLost

Carrier_RejectSignalFromHardware

Carrier_SignalNotReceivedWithinTime

DbccDevice_GammaDownlinkFailure

ExternalTma_LnaFailureBranchA

ExternalTma_LnaFailureBranchB

FuDeviceGroup_ExternalUnitFailure

NbapCommon_Layer3SetupFailure

OpticalInterfaceLink_OpticalInterfaceLinkFailure

RruDeviceGroup_GeneralHwError

RuDeviceGroup_GammaUplinkFailure

RuDeviceGroup_GeneralSwError

TpaDevice_AmplificationError

TrDeviceSet_GeneralHwError

Most likely there will be a TT open, if not, call Switch.

Page 41: 3G Troubleshooting Presentation 082509

From RNC, enter this command to check raw counters

-m minushours

3 last 3 hours

-a aggregate for last 3 hrs

If you don’t enter –m 3 –a, default is last 15 minutes

Checking raw counter values from RNC – RRC Attempts

Page 42: 3G Troubleshooting Presentation 082509

RAB Establishments and Success

Check RAB Attempts and Success for 1 cell for last 15 minutes

Check RAB Attempts and Success for multiple sites using pipe (|) for last 1 hour aggregated (-a)

Page 43: 3G Troubleshooting Presentation 082509

Voice and Packet Drops

Page 44: 3G Troubleshooting Presentation 082509

Checking HS Stats

pmNoNormalRbReleaseHS – Normal Release

pmNoSystemRbReleaseHS – HS Drops

Page 45: 3G Troubleshooting Presentation 082509

Check multiple sites and multiple counters

pmx U1PH5293A|U1PH1009A pmTotNoRrcConnectReq|pmNoRabEstablish|pmNoNormalRabReleaseSpeech|pmNoSystemRabReleaseSpeech -m 1 -a

pmx U1PH5293A|U1PH1009A pmTotNoRrcConnectReq|pmNoRabEstablish|pmNoNormalRabReleaseSpeech|pmNoSystemRabReleaseSpeech -s 20090708.0000 -e 20090708.1000 –a

-s start20090708.0000 12 AM July 8-e end20090708.1000 10 AM July8-a aggregate

Page 46: 3G Troubleshooting Presentation 082509

pmr –s 20090825.1000 –e 20090825.1100 -a

pmr –s 20090825.1000 –e 20090825.1100

pmr –m 3 –a

Pmr –m 2

PMR command options

Page 47: 3G Troubleshooting Presentation 082509
Page 48: 3G Troubleshooting Presentation 082509

Future slides………….