1

Rev A

KPI CSR Workflow Description

Introduction This description is to be used by engineers as a reference when investigating CSRs reporting KPI degradation. It outlines an investigation strategy but

does not cover every scenario due to the complexity of the WRAN system. It is not exhaustive and currently covers the standard Ericsson WRAN

Retainibility and Accessibility KPIs.

Ericsson KPI formulae The following are the KPI formulae these workflows are based on.

Speech Accessibility:

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

2

Rev A

Packet Interactive Accessibility:

Speech Drop Rate

Packet Interactive Drop Rate

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

3

Rev A

CSR Common Workflow

When a CSR is first received a number of

initial actions should always be performed,

1. Check the product the CSR is

written on is still supported and

has not entered ML4. If the

product is no longer supported the

CSR should be de-escalated stating

this, after consultation with the

relevant gatekeeper.

2. Check that the CSR contains the

mandatory and if applicable

problem specific information. If

the DCG logs are not present then

they should be requested either by

email or returning the CSR with a

note.

3. After becoming Familiar with the

CSR problem / request, update the

primus case and perform a search.

DCG Link: https://ericoll.internal.ericsson.com/sites/PLM/WRAN/Pages/

DataCollectionGuidelines.aspx

Primus Link: http://e-support.ericsson.se/iview/ui/eserver.asp

CSR escalated

D

e

-

e

s

c

a

l

a

t

e

C

S

R

1. Product

Supported?

2. DCG

Attached?

3. Primus case update and

Search

No

No

Investigation Procedures Common Workflows

4

Rev A

4. If an existing primus case is found

that answers the CSR request /

problem link that case to the CSR

and fetch it into the CSR. De-

escalate the CSR.

4. Existing

primus case

found?

A

Yes

D

e

-

e

s

c

a

l

a

t

e

C

S

R

Common Workflows Investigation Procedures

5

Rev A

KPI Common Workflow

1. If the CSR reports KPI degradation

continue with this workflow.

Otherwise the CSR is beyond the

scope of this description.

2. Confirm the customer is using

standard KPI formula. Analyse the

ROP files to confirm the level of

degradation reported. Also

establish when the degradation

first began (when comparing KPIs

time of day and day of week

should be considered).

3. Check Alarm, Event, and System

logs for entries that coincide with

the start of KPI degradation.

4. Check the System logs for OAM

activates carried out on or in the

day before the degradation began.

Moshell Commands

RNC> pmr -r 3 -m <hours>

RNC> pmx utrancell <counter> -m <hours>

RNC> lgaesrm

RNC> lguoqlrm

1. CSR reports

KPI

degradation?

2. Establish exact level and

date / time when KPI

degradation began.

A

B

e

y

o

n

d

S

c

o

p

e

3. Check the System logs

around the time the

degradation started.

4. Check System logs for

OAM activities.

No

Investigation Procedures Common Workflows

6

Rev A

5. Did degradation occur after RAN

upgrade. If so was it after the RNC

or the RBS upgrade.

6. Check the Network Impact Report

and release notes for changes,

new features or TRs that could be

responsibe.

7. Check for changed parameter

values, newly enabled features

etc.. (Note: Use complete MoM).

8. Breakdown the affected KPI(s) per,

Subrack

Transmission link (ET Board)

Module MP

Iublink

Utrancell level

9. If there is a common component

that is causing the degradation

execute that investigation

procedure. If more than one

component is affected but it is not

widespread start with worst

affected component. If necessary

return and continue with step 10.

RNC> diff . kget_before_upgrade.txt

RNC> pmr -r 5,50-53 -m <hours>

RNC> pmr -r 46-49,56-57 -m <hours>

RNC> pmr -r 4,42-45 -m <hours>

RNC> pmr -r 33-37 -m <hours>

RNC> pmr -r 24,27-28,29-31 -m <hours>

Yes 9. Common

component

found?

MMP

Subrack

Iublink

TN

Cells

8. Check for worst affected

Subrack, Transmission link,

MMP, Iublink, Cells

6. Check Release notes

& NIR for new SW.

7. Perform kget diff on

before and after upgrade

kgets.

5.Degraded

after

upgrade?

Yes

Common Workflows Investigation Procedures

7

Rev A

10. If it is a Retainibility KPI that is

affected choose Drop calls. If

multiple KPIs are degraded choose

the KPI that was first affected.

11. If it is an accessibility KPI that is

affected choose degradation type,

RRC Success Rate

Iu Signalling Setup Success

Rate.

Standalone Srb Success Rate

(NAS).

Rab Success Rate

12. Currently the scope of this

description is limited to

Retainibility and Accessibility KPI

degradation

.

RNC> pmr -r 3 -m <hours>

RNC> pmr -r 3 -m <hours>

RNC> pmxhn utrancell nassignrelease -m

<hours> -a

11. Is

accessibility

affected?

RRC

Rab

Iu Sig

NAS

Yes

No

12. Beyond scope of

this description

Yes 10. Is

Retainibility

affected?

Drop calls

Common Workflows Investigation Procedures

8

Rev A

Dropped Calls

1. For the worst cell extract the Rab

specific counters and determine if it is

a single Rab type that is affected.

pmNoSystemRabReleaseSpeech

pmNoSystemRabReleaseAmrNb

pmNoSystemRabReleaseAmrWb

pmNoSystemRabReleaseCs64

pmNoSystemRabReleaseCsStream

pmNoSystemRabReleasePacket

pmNoSystemRabReleasePacketStrea

m

pmNoSystemRabReleasePacketStrea

m128

pmNoSystemRabReleasePacketUra

pmNoSystemRabReleasePsStreamHs

pmNoSystemRbReleaseEul

pmNoSystemRbReleaseHs

2. Extract Iurlink counters. Check if one

specific Iurlink is affected or the drop

rate over Iur has causing the increase.

pmNoSystemRabReleaseCs64

pmNoSystemRabReleaseCsStream

pmNoSystemRabReleasePacket

pmNoSystemRabReleasePacketStrea

m

pmNoSystemRabReleaseSpeech

Commands

RNC> pmxh utrancell=<worst cell>

pmnosystemrabrelease -m <hours> -a

RNC> pmr -r 39 -m <hours>

RNC> pmxnh iurlink pmnosystemrabrelease -m

<hours> -a

Drops

1. Check if one Rab Type is

affected.

2. Check if increased drops

are related to one or more

Iurlinks.

Common Workflows Investigation Procedures

9

Rev A

3. Check for changes to TNL and RNL

QoS.

4. Check with CN concerned for any QoS

changes related to Rab Request.

5. Check Node Synch in RNC and RBS.

and Frame Synch.

6. Check that Frame synch has not

degraded between RNC and RBS.

Check for increase in the number of

discarded or control frames being

sent.

pmNoDchDlTimingAdjContrFrames

pmNoDchUlDataFramesOutsideWindo

w

pmNoDlDchDiscardedDataFramesE

pmNoDlDchDiscardedDataFramesL

pmNoUlDchDiscardedDataFramesE

pmNoUlDchDiscardedDataFramesL

RNC> lguoqlrm RNC> sts RBS> sts RNC> pmxh DchFrameSynch Frames -m <hours> -

a RBS> pmxh IubDataStreams late -m <hours> -a

3. Check the OAM logs

specifically from changes to

QoS.

4. Check for QoS changes

performed in CN.

UEH Exception

Analysis.

5. Check Node Synch in

RNC and RBS.

6. Check Frame Synch.

Common Workflows Investigation Procedures

10

Rev A

RRC setup failure

1. Check RRC Rej counters due to

feature RRLC Early Filter, CC Sp

flow control and MMP load.

pmNoRejRrcConnRrcLc

pmNoRejRrcConnSpFlowCtrl

pmNoRejRrcConnMpLoadC

2. For the worst affected Utrancell

check if admission denied counters

have increased,

pmNoRrcReqDeniedAdm

pmNoRrcCsReqDeniedAdm

pmNoRrcPsReqDeniedAdm

3. Are the admission denied counters

stepped due to loadsharing,

pmNoLoadSharingRrcConn

pmNoLoadSharingRrcConnCs

pmNoLoadSharingRrcConnPs

4. Have RL setup failure increased

due to Transport Network (TN)

congestion,

pmNoRrcConnReqBlockTn

pmNoRrcConnReqBlockNode

Commands

RNC> pmxh utrancell=<worst cell> pmnorejrrc -m

<hours> -a

RNC> pmr -r 21 -m <hours>

RNC> pmxh utrancell=<worst cell>

reqdeniedadm$ -m <hours> -a

RNC> pmxh utrancell=<worst cell>

pmnoloadsharingrrcconn -m <hours> -a

RNC> pmxh utrancell=<worst cell>

pmnorrcconnreqblock -m <hours> -a

RRC

1. Found

RRC

Rejects?

Yes

Load Rejs

3. Denied

due to load

sharing?

AdmDenied

No

2. Check for

Adm denied?

End

Check for Admission denied counters?

Yes

Yes

4. TN

Congestion? TN

Yes

Common Workflows Investigation Procedures

11

Rev A

5. Check RBS counters for Radio Link

setup failures,

pmSetupFailuresSf128

pmSetupFailuresSf64

6. Request the traces from affected

RBS to investigate failures.

Request support from RBS

7. If issue cannot be resolved with

trace information, request support

from RBS.

RBS> pmxh uplinkbasebandpool

pmsetupfailuressf64 -m <hours>

RBS> pmxh downlinkbasebandpool

pmsetupfailuressf128 -m <hours>

RBS> lh mp te e all NBAP_RESOURCE

RBS> lh mp te e all RADIOLINK_REJECT

5. RL Failures

in RBS?

6. Enable tracing in RBS

to investigate failures.

UEH Exception

Analysis.

Yes

End

7. Request RBS support.

Common Workflows Investigation Procedures

12

Rev A

Iu Signalling setup failure

1. Check Alarm, Event, and System

logs for entries relating to RANAP

or SCCP. If alarm found execute

Alarm IP then return to next step if

necessary.

2. Check the te log on RANAP and

SCCP MPs for ERRORs or Info

traces. Execute Error IP then

return to next step if necessary.

3. Are Initial Direct Transfers being

discarded due to RANAP Overload,

pmIdtDiscarded

pmIdtDiscardedCs

pmIdtDiscardedPs

Check MP load during time of

degradation. Also RncFunction

Refused Request counter for SCCP,

pmRefusedRequestSccpMpLoad.

RNC> lgaesrm egrep -i “ranap|sccp”

RNC> lh sccpmp te log read

RNC> lh ranapmp te log read

RNC> pmxh cnoperator pmidtdiscarded -m

<hours>

RNC> pmr -r 7 -m <hours>

RNC> pmxh rncfunction

pmrefusedrequestsccpmpload -m <hours>

Iu Sig

3. Check RANAP/SCCP MPs for overload?

1. Found

Alarms &

Events?

Alarm

2. Found

Errors in te

log? Errors

Yes

Yes

Common Workflows Investigation Procedures

13

Rev A

4. Check for RANAP or SCCP

congestion. DCG logfile “<RNC

name>_dcg_m.log”, printouts

“ranap cong”

“rof_congestion_info”. Also check

that connected Ids are not

reaching maxconn value in

command “ranap conn”.

5. If MMP Ueh Exception log are

available check for exceptions that

relate to RANAP or SCCP. If the

problem is still present targeted

analysis can be performed.

Execute IP then return to next step

if necessary.

6. Check for ongoing Reset Resource

procedures between RNC and CN

no RANAP MP.

RNC> lh modX te e all UEH_EXCEPTION.

RNC> lh ranapmp te filter set "([1] = 26)"

RANAP_ASN

RNC> lh ranapmp te e bus_send bus_receive

RANAP_ASN

Yes 5. UEH

Exceptions

logs

Available?

4. Check RANAP/SCCP Congestion?

Exception

Analysis.

6. Check for ResetResource

messages on RANAP MP.

Investigation Procedures Common Workflows

14

Rev A

7. Update the primus case with new

information from investigation to

date.

8. If an existing primus case is found

that answers the CSR request /

problem link that case to the CSR

and fetch it into the CSR. De-

escalate the CSR.

9. Request support from CN to

investigate this problem in Core.

Yes

7. Primus case update

and Search

8. Existing

primus case

found?

D

e

-

e

s

c

al

a

t

e

-

C

S

R

9. Create Support

case towards CN.

Common Workflows Investigation Procedures

15

Rev A

NAS Sig failure

General: Covers failures during the

NAS signalling phase, which is the

period after the Iu Signalling

connection has been established up

until the Rab Assignment Req is

received by the RNC.

1. Check Alarm, Event, and System

logs for entries relating to

SCCP/RANAP. If alarm found

execute Alarm IP then return to

next step if necessary.

2. Check the te log on the SCCP &

RANAP MPs for Errors/Info traces.

3. Check counters, compare with

values before degradation on RNC

and then cell level. Sort so worst

affected cells appear first.

pmNoSystemReleaseSrbOnly136

pmNoSystemReleaseSrbOnly34

pmNoSystemReleaseSrbOnlyEul

pmNoSystemReleaseSrbOnlyHs

pmNoCellDchDisconnectAbnorm

Commands

RNC> LGAESRM egrep -i “sccp|ranap” RNC> lh sccpmp te log read RNC> lh ranapmp te log read RNC> pmxhn utrancell

SystemReleaseSrbOnly|DisconnectAbnorm -m <hours>

RNC> pmx utrancell pmNoSystemReleaseSrbOnly136 -m <hours> -a | sort -k3 -n -r

RNC> pmx utrancell pmNoCellDchDisconnectAbnorm -m <hours> -m <hours> | sort -k3 -n -r

NAS

1. Found

Alarms &

Events? Alarm

Yes

2. Found

SCCP Errors? Errors

Yes

3. Check SrbOnly and

Abnormal Rel counters?

Common Workflows Investigation Procedures

16

Rev A

4. Check the MMP logs for UEH

Exceptions relating to RANAP

(Pertinent ECs disconnectInd &

Release Command with non

“Normal” cause value). Compare

with logs from before degradation

began. Execute IP then return to

next step if necessary.

5. Request support from CN to

investigate this problem in Core.

4. UEH Exc

logs available? Exception

Analysis.

Yes

5. Create Support

case towards CN.

Common Workflows Investigation Procedures

17

Rev A

Rab Setup failure

1. Check for increase in Rab mapping

or UE capability failures on node

level.

pmNoInvalidRabEstablishAttemps

pmNoRabEstablishFailureUeCapa

bility

2. Check for changes to the support

Rab Combinations in the RNC. Also

check for any changes to QoS

feature/parameters.

3. Check if the degradation coincides

with the release of a new UE in the

market concerned.

4. Request support from Core wrt

any changes made to QoS

parameters sent in Rab Req.

5. If increase in Adm denied for Rabs

execute IP. (From W12B the

Number of Rab Adm denied is

calculated as total Req denied

minus Rrc Req denied)

pmNoReqDeniedAdm

pmNoRrcReqDeniedAdm

Commands

RNC> pmxhn utrancell failureUecap|invalidrabest -m <hours

RNC> lgoqlnrm RNC> pmxhn utrancell [co]reqdeniedadm$ -m

<hours> RNC> pmxh utrancell

pmNoReqDeniedAdm$|pmNoRrcReqDeniedAdm$ -m <hours> -a | sort -k3 -n -r

1. Failures in

UE Cap /

mapping?

Rab

2. Check RNC OAM log

for changes to Rab

Combination or QoS?

3. Request info from

customer wrt new UE

releases?

4. Create Support

case towards CN.

Yes

5. Adm

Denied

increased?

AdmDenied

Yes

Common Workflows Investigation Procedures

18

Rev A

6. For Speech Rabs check if number

of directed retries has changes.

pmNoDirRetryAtt

7. Check for increase in Rab failures

due to TN congestion or failure.

pmNoRabEstBlockTnCs57

pmNoRabEstBlockTnCs64

pmNoRabEstBlockTnPsIntHs

pmNoRabEstBlockTnPsIntNonHs

pmNoRabEstBlockTnPsStrHs

pmNoRabEstBlockTnPsStrNonHs

pmNoRabEstBlockTnSpeech

8. Check RBS counters for Radio Link

setup failure increase,

pmSetupFailuresSf256

pmSetupFailuresSf128

pmSetupFailuresSf64

pmSetupFailuresSf32

pmSetupFailuresSf16

pmSetupFailuresSf8

pmSetupFailuresSf4

9. Request the traces from affected

RBS to investigate failures.

10. If issue cannot be resolved with

trace information, request support

from RBS.

RNC> pmxhn utrancell pmNoDirRetryAtt -m

<hours> RNC> pmxh utrancell pmNoDirRetryAtt -m

<hours> -a | sort -k3 -n -r RNC> pmxhn utrancell pmNoRabEstBlockTn -m

<hours> RNC> pmxh utrancell pmNoRabEstBlockTn -m

<hours> -a | sort -k3 -n -r RBS> pmxh uplinkbasebandpool

pmsetupfailuressf -m <hours> RBS> pmxh downlinkbasebandpool

pmsetupfailuressf -m <hours> RBS> lh mp te e all NBAP_RESOURCE RBS> lh mp te e all RADIOLINK_REJECT

7. TN

Congestion? TN

8. RL Failures

in RBS?

9. Enable tracing in RBS

to investigate failures.

10. Request RBS

support.

UEH Exception

Analysis.

End

Yes

Yes

6. Enable tracing in RBS

to investigate failures.

Investigation Procedures Common Workflows

19

Rev A

INVESTIGATION PROCEDURES

(IP)

20

Rev A

IP: Subrack

General: This investigation Procedure (IP) is

only to be used where KPI degradation is

found to relate to a single Subrack (not a

single Transmission Link or MMP within the

subrack).

1. Check for ISL/ESL faults and Overload.

- Check for crashes on Scb(x) boards

- Check for errors on Scb(x) boards

- Check for ISL Overload

- Check the controlling MMP and ATM

port for Iublinks are in same Subrack

2. Check Subrack ETs for faults.

3. Check MMP for usage and faults.

4. Check the Subrack DcDevice

allocation, usage and faults.

5. Check the Subrack CcDevice

allocation, usage and faults. For ATM if

Ccdevice allocated in different Subrack than

ET termination can cause ISL load.

6. If ATM is used on Iu, check that

atmUserPlaneTermSubrackRef is set in IuLink

MO, if not this can cause ISL load.

7. Check RSTP configuration.

8. Additional spas related information

can be gathered with dcgx command.

Commands

RNC> lgs RNC> lgt -g scx | egrep -i

“error|restart” RNC> pmx . pmPeakBwLevel -m

<hours> RNC> lkra RNC> lgt -g et | egrep -i

“error|restart” RNC> lh modxx te log read | egrep -i

“error|restart” (xx = ms,es1,es2…)

RNC> std RNC> lgt -g dcxx (xx = ms,es1,es2…) RNC> lgt -g ccxx (xx = ms,es1,es2…) RNC> cedr RNC> get IuLink

atmUserPlaneTermSubrackRef RNC> steg RNC> dcgx

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

21

Rev A

IP: Transport Net

General: This investigation is only to be used

where KPI degradation is found to relate to

Transport Network.

1. Check TN/RSTP configuration.

2. Check alarm log from bouncing links.

3. Check the System logs for OAM config

changes associated with the TN.

4. Check the ET board(s) for restarts or

errors.

5. Check for Aal2 setup failures

6. Perform ping / ete loopback test on

suspected TN Links.

7. Check intermediate TN nodes

(Routers, RXIs etc..) for alarms or errors.

8. Check intermediate TN nodes for OAM

config changes.

9. If a single far end node is involved

(MSC, SGSN, RBS) check this for alarm or

errors.

10. Check far end node for OAM config

changes.

11. Sometimes the nature of a TN fault

means isolating the faulty transmission link

requires removing it from service. This can be

done by locking/unlocking TL in rotation until

the faulty link is found.

Commands

RNC> stv RNC> sti RNC> steg RNC> lga RNC> lguoqlrm RNC> lgt -g et | egrep -i

“error|restart” RNC> lgs RNC> pmr -r 57 -m <hours> RNC> stip RNC> acc <VclTp> eteLoopBack RXI> lga RXI> lgt -g et | egrep -i

“error|restart” RXI> lgs RXI> lguoqlrm RBS> lga RBS> lgt -g et | egrep -i

“error|restart” RBS> lgs RBS> lguoqlrm

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

22

Rev A

IP: Module MP

General: This investigation is only to be used

where a KPI degradation is found to relate to

a Module MP.

1. Check the MMP for restarts or errors.

2. Check MMP load.

3. Check the System logs for OAM

config changes associated with this MMP.

4. Check DCs associated with this MMP

for overload and faults.

5. Check if large Location/Routing Areas

could be causing high load due to paging.

Commands

RNC> lgs RNC> lgt -g modx | egrep -i

“error|restart” RNC> pmr -r 7 -m <hours>

RNC> Lguoqlrm

RNC> std RNC> pmr -r 9 -m <hours> RNC> pmxh dcdevice=<dc>

MeasLoad -m <hours> RNC> lgt -g dcx | egrep -i

“error|restart” RNC> pmr -r 8 -m <hours> RNC> get locationarea reserved RNC> get Routingarea reserved

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

23

Rev A

IP: Iublink

General: This investigation is only to be used

where a KPI degradation is found to relate to

all cells in a specific IubLink(s).

1. Check the System logs for OAM config

changes associated with this site (RNC+RBS).

2. Check system logs for recent entries

concerning Iublink/Utracells (RNC+RBS).

3. Check for errors related to this Iublink

on terminating MMP.

4. Check if Iublink credit usage going

over 80%, AvgDlCredits/AvgUlCredits.

5. Check resource usage and hanging

resources.

6. Check for high Ul RSSI in the cells for

the site.

7. In RBS check license state and

capability.

8. Check that Iublink is residing in same

Subrack as user plane resources.

9. Check for changes in nodesynch delay

and also the RBS Synch status. Also counters

for discarded frames in RBS.

10. Check NTP synch in RBS and RNC.

Commands

RNC> Lguoqlrm RBS> Lguoqlrm RNC> lgaesrm RBS> lgaesrm RNC> lgt -g modx RNC> pmxh IubLink=<Iublink>

pmdlCredits|pmulCredits -m <hour>

RNC> cedh RNC> lh ranapmp ueregprint age

3600

RNC> pmr -r 32 -m <hours> RBS> get NodeBFunction license RNC> strt RNC> pmxh

IubLink=<Iublink>,NodeSynch delay -m <hours>

RBS> get Synchronization RBS> pmxh IubDataStreams

pmDchFrames.*late -m <hours>

RNC> ntpconfig info RBS> ntpconfig info

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

24

Rev A

IP: Utrancell

General: This investigation is only to be used

where a KPI degradation is found in a

Utrancell(s) but not a complete site.

1. Check system logs for recent entries

concerning the Utracell (RNC+RBS).

2. Check the System logs for OAM config

changes associated with this cell (RNC+RBS).

3. Check for errors related to this cell on

terminating MMP.

4. Check in the RBS for errors/restarts

on RAX/TX boards for this cell.

5. Check for high Ul RSSI in this cell.

Commands

RNC> lgaesrm RBS> lgaesrm RNC> Lguoqlrm RBS> Lguoqlrm RNC> lgt -g modx | egrep -i “<Cell

Id>” RBS> lgt egrep -i “error|restart” RNC> pmr -r 32 -m <hours

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

25

Rev A

IP: Alarms

This investigation is to be used when relevant

Alarm(s) is found.

Alarms:

Where a relevant alarm(s) is found the

corresponding Operational Instruction (OPI)

found in CPI Store should be executed. The

correct OPI can be found by using the Specific

Problem text from the alarm to perform a

search in the relevant library (e.g. search for

“Fach_InternalResourceUnavailable” in library

“WCDMA RNC 3820 W12.0”)

IP: Error codes

SCCP Errors:

1. SCCP Error codes can be decoded at

cello work support tool, they are also

described in Maintenance Instructions SCCP

(1/1541-CAA 901 437 Uen).

2. It may be necessary to contact CPP

support where an SCCP Error has occurred as

the Error code description is often cryptic.

CPI Store Link: http://cpistore.internal.ericsson.com/alex

Cello Work support tool Link: http://support.dach.ericsson.se/worksupport/t

ools/Cello_err.php3

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

26

Rev A

IP: Load Rejects

1. If pmNoRejRrcConnMpLoadC is being

stepped, check if down switch due to soft

congestion is occurring. This can result in high

load on mMP.

pmNoOfSwDownEulCong

pmNoOfSwDownHsCong

pmNoOfSwDownNgCong

pmSoftCongRbsUlHw

pmNoOfSwDownNgHo

If down switch due to congestion is

confirmed, consider enabling feature RRLC

Early Filter on the affected cells.

2. If pmNoRejRrcConnRrcLc is being

stepped check alarms in RNC related to this

feature

Check also alarms in RBS related to HW

failures, refer to IP Alarms.

3. If pmNoRejRrcConnSpFlowCtrl is

being stepped, check the number of cells

allocated to that CC device.

4. Check that the CcDevice used by this

cell is in the same Subrack as the Iublink TN

termination.

5. Check the CC device load.

RNC> pmxh utrancell NoOfSwDown -

m <hours> -a RNC> altk RBS> altk RNC> std cc RNC> str -I <Iublink Id> RNC> pmr -r 8 -m <hours>

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

27

Rev A

IP: UEH Exception

UEH Exception Identification

There are a number of stratages that can be

used when analysing UEH Exception to

identify the exception associated with the

degradation. The strategy used will depend

on a number of factors including,

The size of degradation, is it 0.1% or

10%.

Nature of the degradation, is it

localised to cluster/site/area or widespread .

Available data, is data availabe from

before degradation began for comparison

purposes.

In general it is good to narrow the area of

investiagtion as much as possible. Use

targated tracing on,worst affected cells,sites

or mMP, only tracing on worst affected

UeRc(s), etc.

General analysis:

This is where all UEH Exception are included

and an analysis is done to breakdown the

number of exceptions per Exception code,

Cause code, Transition type, etc ... This is

most useful when the size of the degradation

is relatively large compared to the ”normal”

number of failures that occur for that KPI, and

data is available from before the problem

began to compare against.

This analysis is performed in 3 steps,

1. Parse exception logs from before

degradation.

2. Parse exception logs taken during

degradation.

3. Using uehExcAnalyser.xlsm import

and analysis the data in excel.

Commands

RNC> lh modx te e all UEH_EXCEPTION

Cygwin$ java -jar

ExcAnalyser.jar <Raw Exc file before>

<Parsed Exc file before>

Cygwin$ java -jar Analyser.jar

<Raw Exc file during> <Parsed Exc file

during>

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

28

Rev A

Targeted analysis:

This is where only UEH Exceptions that have

been associated with the stepping (or not

stepping in case of attempt/success counters)

of the counter that resulted in the KPI

degradation are included in the analysis.

Additional counter specific tracing are

required along with the standard UEH

Exception trace. Due to the counter tracing

requirement useful trace logs are usually not

availabe from before the problem began for

comparison.

This analysis is performed in 3 steps,

1. Identify what counter(s) within the

KPI formula that has changed resulting in the

degradation. Extract the values for the

counters concerned individually and compare

against those from before degradation.

2. Request counter specific traces to be

taken for the counter(s) identified in step 1 (if

it is an attempt without a success, trace both

counters).

3. Parse the logfile to extract only the

UEH Exceptions that are associated with the

releveant counter(s).

4. Using ExcAnalyser.xlsm import and

analysis the data in excel.

RNC> pmx <mo> <counter> -m

<hours>

RNC> lh modx ue_pm print -counter

| egrep -i <counter> RNC> lh modx ue_pm enable -

counter <counter1 Id> RNC> lh modx ue_pm enable -

counter <conter2 Id> RNC> lh modx te e trace6

PM_OBS_IND

Cygwin$ java -jar

ExcAnalyser.jar <Raw Except file

name> <Parsed Exc file name>

<counter name>

Cygwin$ java -jar

ExcAnalyser.jar <Raw Exc file name>

<Parsed Exc file name> <attempt

counter name> <”^success counter

name”>

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

29

Rev A

Filtering:

Where the degradtion has been identified as

affecting a particular Utrancell(s) or UeRc,

then it is useful to narrow the tracing,

outlined in step 2 above, to the worst

affected cell and/or UeRc. This is done using

UE Random tracing.

Examples:

1. Increase in Speech Drop Rate KPI,

with Utrancell cid 4217 being worst affected .

2. Degradation of Eul Accessibility with

UeRc=25 in Utrancell cid=4200 worst

affected. Worst affect UeRc found using

Attempt and Success UeRc MO counters.

RNC> lh modx traceCounter on

<counter1 Id> RNC> lh modx traceCounter on

<counter2 Id> RNC> lh modx te e trace9

UE_COUNTERS_GENERAL RNC> lh modx te e all

UEH_EXCEPTION RNC> lh modx uerandtrace on -cell

cid <cid> -uerc <uerc> RNC> lh modx uerandtrace max -

unlim

Worked Example: Speech drop rate

degradation.

RNC> lh mod80 traceCounter on 4284 RNC> lh mod80 te e trace9

UE_COUNTERS_GENERAL RNC> lh mod80 te e all UEH_EXCEPTION RNC> lh mod80 uerandtrace on -cell cid

4217 RNC> lh mod80 uerandtrace max -unlim

Cygwin$ java -jar ExcAnalyser.jar Inputfile.log

outputfile.csv

PMNOSYSTEMRABRELEASESPEECH

Worked Example: Eul Accessibility

degradation.

RNC> lh mod8 traceCounter on 4564 RNC> lh mod8 traceCounter on 4565 RNC> lh mod8 te e trace9

UE_COUNTERS_GENERAL RNC> lh mod8 te e all UEH_EXCEPTION RNC> lh mod8 uerandtrace on -cell cid 4200

-uerc 25 RNC> lh mod8 uerandtrace max -unlim

Cygwin$ java -jar ExcAnalyser.jar Inputfile.log

outputfile.csv

PMNORABESTABLISHATTEMPTPACKETINTERAC

TIVEEUL -crossreference

"^PMNORABESTABLISHSUCCESSPACKETINTERA

CTIVEEUL"

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

30

Rev A

UEH Exception Meaning

Where a relevant Ueh Exception is found

there are a number of options for further

investigation,

1. The exception mean and description

of the scenario in which it would occur, along

with further trace suggestion can be found in

the Common Exception Handler (CEH).

2. If the Exception is not present in the

CEH (not all Ueh Exception are added to this

tool yet) then the exception meaning can be

found in UEH Exception Definition document

(available in CDM).

3. For further explanation and details it

is also possible to submit a support request

to the UEH Subsystem within RNC.

CEH Link: https://wcdma-

commonexception.rnd.ki.sw.ericsson.se

CDM Link UEH Exception Definition: 32/1551-CRA 403 38/1

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied

31

Rev A

IP: Admission Denied

RRC Related:

1. Check the following resource specific

counters for indication of RBS HW, Power or

downlink code congestion. Compare values

with those from before degradation began.

pmNoRrcReqDeniedAdmDlHw

pmNoRrcReqDeniedAdmUlHw

pmNoRrcReqDeniedAdmDlPwr

pmNoRrcReqDeniedAdmDlChnlCode

2. Check for hanging resources in the

RBS

Rab Related:

1. Check the following resource specific

counters for indication of RBS HW, Power or

downlink code congestion. Compare values

with those from before degradation began.

pmNoFailedRabEstAttemptExceedConnLi

mit

pmNoFailedRabEstAttemptLackDlAse

pmNoFailedRabEstAttemptLackUlAse

pmNoFailedRabEstAttemptLackDlChnlCo

de

pmNoFailedRabEstAttemptLackDlHw

pmNoFailedRabEstAttemptLackUlHw

pmNoFailedRabEstAttemptLackUlHwBest

pmNoFailedRabEstAttemptLackDlHwBest

pmNoFailedRabEstAttemptLackDlPwr

pmNoFailedRabEstAttEulRateCong

2. Check for hanging resources in the

RBS.

Commands

RNC> pmxh utrancell=<worst cell>

pmNoRrcReqDeniedAdm -m <hours>

RNC> cedh RNC> lh ranapmp ueregprint age

3600 RNC> pmxh utrancell=<worst cell>

FailedRabEstAt-m <hours> RNC> cedh RNC> lh ranapmp ueregprint age

3600

Contents

Work Flows

KPI CSR Workflow

CSR Common_WF

Common_WF

Dropped_calls

RRC Setup Failure

Iu Sig Setup Failure

NAS_WF

Rab Setup Failure

Investigation

Procedures

Subrack_IP

Tn_IP

ModuleMP_IP

Iublink_IP

Utrancell_IP

Alarms_IP

LoadRejs_IP

UehExc_IP

Admission_Denied