GSM BSS Efficiency KPIs Process

8/10/2019 GSM BSS Efficiency KPIs Process

http://slidepdf.com/reader/full/gsm-bss-efficiency-kpis-process 1/18

© Nokia Siemens Networks 2011

Confidential

Presentation / Author / Date

GSM BSS R&D Efficiency KPIs:FDE_n

FDE_s

FDE_f

Scope: Internals persons

Toni Oksanen / 17.4.2013

Version 1.5



Dept. / Author / Date

Confidential This documentation is legally privileged and for initial planning purposes only. Any concrete plans and implementation decisions are subject to NSNdecision-making governance and subject to, where applicable, consultation with employee representatives and fulfillment of legal conditions.


FDE_n for BSS

Definitions

Formula FDE_n = Net Development / Total Development

▪ Total Development= total monthly capacity of all BSS R&D persons

(i.e. BTS, BSC and E2E included).

= headcount from Location Plan * (no. of nominal

working days / mon) *7.5 hrs

Example: 21 days * 7.5hrs = 157.5 hrs

NOTE: the “21 days” may be corrected to

overcome errors due to varying number o fworking

days in a month, see pages “Error terms and

fluctuations”

▪ Net Development

= monthly reported hours for New FeatureDevelopment of all BSS R&D persons (i.e. BTS,

BSC and E2E included).

= program level sum of NFD work from B34

excel

Questions & remarks

▪ Total Development- use the “official” actual SAP headcount end of

each period

▪ Net Development

- use the programs to separate NFD and non-

NFD actuals – program lookup table:

- Pool-based separation does not currently add

value at all as programs do not have pools that

separate NFD and non-NFD actuals.

- TrackView activity –based separation would not

add much value due to smallish amounts of

additional reported hours to be cathegorized as

non-NFD.






FDE_s for BSS

Definitions

Formula FDE_s = System Verification / Total Development

▪ Total Development

- see slide FDE_n for BSS

▪ System Verification

= total monthly monthly capacity of all BSS E2E

R&D persons

= headcount from Location Plan * 157.5 hrs

Questions & remarks




- using headcounts is instructed in MBB slides and is

easy=fast to collect, but then all non-BSS NFD work

(maintenance, Horizon, general work, etc.) done by SyVe

people is included. Also, it may not change monthly, i.e. it

is a very “slow” KPI.

Should “SyVe” figure exclude non-NFD ? If yes, the best

way would be to collect actual hours for all NFD

programs instead of using headcounts.

At the moment the collected data is based on

headcounts.

FOR REFERENCE P8/2012 NUMBERS:- via headcounts

24 FTE FDE_n= 10.8%

- via actuals/programs (same lookup table as for FDE_n):

10.5 FTE FDE_n= 5.1%






FDE_f for BSS

Definitions

Formula FDE_f = Bug Fixing / Net Development

▪ Net Development


▪ Bug Fixing

= total monthly reported effort of all bug fixing work

related to prontos raised within all release

programs before P8 milestone.

Questions & remarks

▪ Net Development


▪ Bug Fixing

- the efforts are collected via “pre P8 pronto fixing”

TrackView items created under the ongoing NFD

programs.- actual hours for BTS and BSC are collected

under entity programs; for E2E under system

programs.

- for the list of TracView items see attachment in

slide “FDE_n for BSS”.






FDE_n for BTS, BSC and E2E

Definitions

Formula FDE_n = Net Development / Total Development

▪ Total Development= total monthly monthly capacity of all entity R&D

persons.

= headcount from Location Plan * (no. of nominal

working days / mon) *7.5 hrs

Example: 21 days * 7.5hrs = 157.5 hrs

NOTE: the “21 days” may be corrected to

overcome errors due to varying number of working

days in a month, see pages “Error terms and

fluctuations”.

▪ Net Development

= monthly reported hours for New Feature

Development of all entity R&D persons.

= program level sum of NFD work from B34

excel

Questions & remarks 1/2

▪ Total Development- use the “official” actual SAP headcount end of

each period

▪ Net Development







FDE_s for BTS, BSC and E2E

Definitions

Formula FDE_s = System Verification / Total Development


- see earlier slide


= ??

Questions & remarks


- see earlier slide


- ??


http://slidepdf.com/reader/full/gsm-bss-efficiency-kpis-process 7/18Dept. / Author / Date



FDE_f for BTS, BSC and E2E

Definitions

Formula FDE_f = Bug Fixing / Net Development

▪ Net Development


▪ Bug Fixing

= total monthly reported effort of all bug fixing

work related to prontos raised within all release

programs before P8 milestone.

Questions & remarks

▪ Net Development


▪ Bug Fixing

- the efforts are collected via “pre P8 pronto fixing”

TrackView items created under the ongoing NFD

programs.- actual hours for BTS and BSC are collected

under entity programs; for E2E under system

programs.

- for the list of TracView items see attachment in

slide “FDE_n for BSS”.





The process and maintenance

▪ PROCESS FOR COLLECTING DATA- extracting B34 from Artemis/PRM with ready-made parameter settings – TONI OR BSC ASSISTANTS

KIRSI/JOHANNA

- updating the actual headcount figures – TONI

- picking the actual NFD hours and “pre P8 pronto fixing” hours from B34 manually with help of a filtering instructions and a

program lookup table

~3 hrs/round for BSS and entity R&D’s altogether - BSC ASSISTANTS KIRSI/JOHANNA

- put the data to BSS FDE_x excel template, check sanity and email to relevant people and/or archieve to Sharenet

- BSC ASSISTANTS KIRSI/JOHANNA AND TONI

▪ “MAINTENANCE” OF KPI DATA PARAMETERS

- a new “pre P8 pronto fixing” TrackView item must be created every time a new NFD program is established. The actual

creation via EaspLanner is done by the Program, but initiating it must be done by KPI maintenance – TONI

- the NFD Program lookup table needs to be updated every time a new NFD program item is created or an ongoing

program reaches P8, or a new “pre P8 pronto fixing” TrackView item is created – TONI

- B34 filter instructions needs to be updated every time we have ARTEMIS organisation, program code or program group

changes - TONI





Error terms and data fluctuations 1/3

ERROR TERMS

1. Reporting-% will be less than 100% when a period ends. In a big organisation 100% reporting cannot be reached all the time.

- Consequence: a typical error term (too small FDE_n) of about 2%-5% after 1st run.

- Correction 1: in practice this error term vanishes via the re-run after a month. THIS IS IMPLEMENTED

- Correction 2. Strenghtened discipline and reporting remindings may decrease the error term in the 1st run but would not

remove it. NOT IMPLEMENTED

2. The varying no. of week-days (Mon-Fri) in each month, causes an error term compared to a fixed number of working days (e.g.

21 days/mon) to be used for calculating the “Total development”

- Consequence: error term of -9.5%... +4.8% in FDE_n- Correction: use the actual no. of Mon-Fri days in each month. THIS IS IMPLEMENTED

3. The varying no. of WORKING days in each month due to national holidays causes an error term if not taken into account when

calculating “Total Development”.

- Consequence: error term = too small FDE_n

- Correction: collect calendar data of all countries where we have sites, expand the FDE_n calculation process&excels to list

each country within each entity R&S and BSS separately and calculate the accurate “Total Development” accordingly

THIS IS IMPLEMENTED

4. Headcount increase within a period can cause an error term (lower FDE_n) if a new person joins BSS R&D. As the “Total

Development” figure generated by using the official SAP end-of-month headcount figure, new persons joining somewhere in the

middle of a month cause an error term. (Note that a headcount decrease will not cause error terms or fluctuations).

- Consequence: an permanent error term for one month only

- Correcting: manual investigation of newcomes and the dates they join; then manual adjustment of the end-of-month

headcount figures for the relevant entity R&D and the whole BSS NOT IMPLEMENTED - NO NEED AS THIS WILL

VERY SELDOM CAUSE SIGNIFICANT ERROR TERMS.





Error terms and data fluctuations 2/3

ERROR TERMS (con’t)

5. The actual end-of-period headcounts are taken from SAP (=“official” figures), which usually has the the correct numbers and team

data. In case of persons are moving between teams, the moving between teams in Artemis is a separate action and normally

happens after a delay of ~days …~weeks after SAP updating. Error term appears mainly if the Artemis moves happen in the nextperiod after the SAP update. There are 2 types of error terms:

- 1. A person moves between teams and the scope of work changes. Assuming the scope changes at the moment SAP is

updated, the reporting reflecting the new scope is counted as part of the old team THE ERROR PRESUMABLY SMALL

AND HAPPENS SELDOM; NO CORRECTION PLANNED

- 2. A person moves between teams but the scope of the work does not change. The reporting of the person will be counted

as part of the old team instead of part of the new team THE ERROR PRESUMABLY SMALL AND HAPPENS SELDOM;NO CORRECTION PLANNED





Error terms and fluctuations 2/2

FLUCTUATIONS

1. The data will fluctuate due to popular holidays seasons like July in Finland.

- Consequence: not an error term but a big fluctuation by month

- Correction: Interpolate the figures somehow to eliminate the fluctuation NOT IMPLEMENTED

To smoothen the fluctuations and error terms, also ROLLING AVERAGE figures are shown in the report excel sheets





Monthly actual data extraction 1/2B34 excel creation and filtering

▪ Parameters for B34 creation from Artemis/PRM

- Program Group: select only items beginning with “MBB…”; remember to exclude items *All and <N/A><N/A>

- Start month: select the month i.e. the first calendar month shown in the B34 excel. Remember to select at least two months in the historyas the re-collection of data has to be done for the month that has ended a month ago.

▪ Filtering the B34 excel

- common filtering (do in this order)

- column H: leave only “Active”

- column Y: exclude all items starting with “closed…”

- column AR: leave only “Reported” - column M: leave only items starting with “MBB GSM BSS…”

- column A: include all NFD items, exclude all other items (see Program Lookup Table attachment on page 2)

- BSS filtering

- go to the preferred month column, e.g. 2012/P09 column AS, select all cells and take the sum that excel

calculates = Net Development (hours)

- column AA: leave only item starting “pre P8 pronto fixing…”


calculates = Fault correction efforts (hours)

- column AA: select again all items. - BSC filtering

- column M: leave only items starting with “MBB GSM BSS BSC…”






- column AA: select again all items.





Monthly actual data extraction 2/2B34 excel creation and filtering

▪ Filtering the B34 excel (con’t)

- BTS filtering - column M: leave only items starting with “MBB GSM BSS BTS…”







- E2E filtering

- column M: leave only items starting with “MBB GSM BSS E2E…” - go to the preferred month column, e.g. 2012/P08 column AS, select all cells and take the sum that excel






▪ Filling the report graph excel

- when copying the result data from B34, leave the decimals away (i.e. do “truncation” not “rounding”)





Pre P8 pronto fixing - TrackView items

Available TrackView items Dec 2012

NFD PROGRAMS ACTIVITY TEXT STRING

A0S17 MBB GSM 0S17 RG40 BSC pre P8 pronto fixing S17

AB161 MBB GSM B161 RG30 EP1 System Program pre P8 pronto fixing RG30EP1 Step1



AB16N MBB GSM B16N RG30 System Program pre P8 pronto fixing RG30

AEX51 MBB GSM EX51 RG30EP1 BTS EX5.1 SW Development pre P8 pronto fixing EX51 Step 1 AEX51 MBB GSM EX51 RG30EP1 BTS EX5.1 SW Development pre P8 pronto fixing EX51 Step 2

AEX5N MBB GSM EX5N RG30 BTS SW Development pre P8 pronto fixing EX5N

AHEXB MBB GSM HEXB BTS Crockett SW Development pre P8 pronto fixing HEXB

AS161 MBB GSM S161 RG30 EP1 BSC Development pre P8 pronto fixing S16.1 Step1



AS16N MBB GSM S16N RG30 BSC pre P8 pronto fixing S16

ASM3B MBB GSM Flexi MR ID SM DTC and TD-LTE Evolution pre P8 pronto fixing SM3B A0GF2 MBB GSM 0GF2 RG30EP1 BTS GF2.0 Development pre P8 pronto fixing 0GF2

AGFC2 MBB GSM GFC2 RG30EP1 BTS Development pre P8 pronto fixing GFC2

A0VGS MBB GSM 0VGS RG20EP2 BTS SW Vegas Program pre P8 pronto fixing 0VGS





Data from Artemis 1/2

Available PRM report excels

▪ B34

- all data shown per activity, pool and program - includes per person

- all cumulative total reported hours including hours before the monthly scope of the B34

- the reported hours per month (the number of passed months selectable via parameters)

- ETC

- EAC (what is this? Not visible as a column)

▪ A36

- includes per person- the reported hours per month per pool (for selected number of months)

- allocations per pool

- capacities

▪ A27

- includes per team (as separate rows; calendar span towards history is selectable)

- monthly capacity

- monthly actuals and allocations in same row

▪ B24

- includes list of all activities for one program

- does not show the pools

- shows also cumulative reported hours per activity





Data from Artemis 2/2

Listings from A7

▪ Program WBS/pool/ODC code list Excel extraction- shows all

- program names

- wbs codes

- pools

- ODC codes

- no TrackView activities

version 1 2 Candidate Draft Released





Category. Financial

Customer

People & Innovation

Other

E2E: Customer Process

E2E: Product Process

E2E: Delivery Process

Data

Confidentiality

Secret Confidential For Internal Use

Graph/Chart

AvailableData Levels

Product Line, Business Line

CommentsandAdditionalInformation

Accounting discipline is required to run this KPI.

Feature Development Efficiency

NSN Information Security Policy

version

number:

1.2 Candidate Draft Released

Abbreviation FDE

Business Owner Head of BL / Head of R&D

KPI/Metric ConceptOwner

MBB / PfM / R&D EfficiencyJouni Tiainen

KPI/Metric ReportProvider

Head of Planning & Support

Purpose and

Objective

• Understand and drive improvement measures to

- increase net development for new customerfeatures- reduce effort spent on fault correction- reduce effort needed to detect defects in latephase

• Improvement of KPI indicates increase indevelopment efficiency

Source System(s) • Artemis, backlog management system

Calculation FDE _n (Net Developm ent %)

= Net Developm ent / total Developm ent

• Net Development = New feature coding + moduletesting + unit testing + system component testing (done bysame SW dev. team) + creation/contributions/review offeature specs + refactoring + change notes/requests +quality improvement actions in backlogs

• Total development = Headcount of whole organization – System VerificationFDE_f (Fault Correct io n Rate )

= bug fixing effor t as % of Net Development

• Fault correction effort = Effort for fixing of reportedfaults originated typically by SW build test, EntityIntegration, NeVe and Field

• Net Development = as above

FDE_s (System Veri f icat ion Rate) = System

Veri f icat ion effort as rat io on top of total

Developmen t effort

• Calculated on basis of HC per organization

Unit of Measurement %

Aggregation Method Product line view based on monthly figures,BL view can be calculated in case of multiple PLs,release view not needed

10%

15%

20%

25%

30%

35%

40%

45%

P6 P7 P8 P9 P10 P11 P12

Net Dev.(% of totaldev.)

FaultCorrectionRate (% of

net dev.)SyVe Rate(% on topof totaldev.)

https://sharenet-ims.inside.nokiasiemensnetworks.com/Guest/Open/364586213

https://sharenet-ims.inside.nokiasiemensnetworks.com/Guest/Open/364586213




Assumption

• Initially only include internal NSN Headcount, partners is phase 2

• Net Feature Development (NFD) is a KPI to reflect R&D efficiency

• NFD is measured in headcount share of total R&D headcount (HC%)

• NFD comprises the tasks1) System-level specification

2) Entity-level specification3) SW- and HW-development

4) Module- and Component testing

5) Entity testing

• Non-NFD tasks includea) System Verification = E2E test

b) Program Management

c) Program Support and planning

d) L4/L3 Management and Admin staffe) SW- and HW development support (as tooling)

f) I&V support (as lab staff, test tooling)

g) SW- and HW-Maintenance

h) For external R&D: non-efficiency assumption is classified as non-NFD

i) Process, Audit and Efficiency Improvement activities

GSM, R&D efficiencyNet Feature Development

Documents

GSM BSS Efficiency KPIs Process