How to Get More Value From the ICT Benchmarking Data

7/25/2019 How to Get More Value From the ICT Benchmarking Data

http://slidepdf.com/reader/full/how-to-get-more-value-from-the-ict-benchmarking-data 1/32

How To Get More Value From The

ICT Benchmarking DataGuidance for Agencies



(icensing

The Department of Finance and Deregulation is licensed to use,

reproduce, adapt, modify, distribute and communicate the informationcontained in the How to get More Value from the ICT enchmar!ingData document" #ith the e$ception of the Commonwealth Coat of %rms, and sub&ect toany copyright notices contained in indi'idual documents, all materialpresented in the How to get More Value from the ICT enchmar!ingData document is pro'ided under a Creati'e Commons %ustralia

(icence )%ttribution*+onCommercial "- .nported/)http011creati'ecommons"org1licenses1by*nc1"-1/" To the e$tent thatcopyright subsists in a third party )such as in the material relating tothe denitions of cost elements and ser'ice towers at pages 3 to 4/,permission will be re5uired from the third party to reuse the material"



Contents

1. Introduction 4

2. What is Benchmarking? 5

3. Benchmarking Process 7

4. Identifying Areas for Inestigation 11

5. Briefing !enior !takeho"ders on the Benchmarking #esu"ts 14

A$$endi% A & 'etric (ategories 1)

A$$endi% B & I(* 'etrics (ata"ogue 1+



$%$ Introduction

%gencies sub&ect to the Financial Management and %ccountability %ct

participate in annual ICT benchmar!ing conducted by Department ofFinance and Deregulation"

%gencies pro'ide data about their ICT costs, personnel andinfrastructure, and Finance uses the data to calculate benchmar!ingmetrics, which it pro'ides to agencies in the form of benchmar!ingreports"

The ICT enchmar!ing Framewor! sets out the ob&ecti'es of ICTbenchmar!ing0

• to measure progress in impro'ing the e;ciency and

eAecti'eness of ICT ser'ices in the deli'ery of 9o'ernmentprograms, and

• inform other #hole*of*9o'ernment ICT policy initiati'es"

This document focuses on the rst of the abo'e dot points It pro'ides



$%) ,hat is Benchmarking-

enchmar!ing is the process of comparing your performance with thatof peer organisations, particularly organisations which are e$amples ofgood practice, with a 'iew to identifying opportunities for impro'ingyour performance"

%gencies can use the results of the ICT benchmar!ing e$ercise in anumber of ways" These include0

• To understand current performance and to be able to

communicate this on a factual basis, not guesswor!"

• To identify 'ariations in current performance le'els 'ersus other

agencies and see! to understand why there are diAerences" Thismay include areas where an agency is shown to be e;cient ornot e;cient in relation to other agencies"

• To identify areas of the benchmar!ing results that may re5uire a

d t il d l i t l t t t id t



there may be opportunities to increase e;ciency through

alternate sourcing of ser'ices"



• ?ro'iding a summary of results for senior agency sta!eholder

including !ey strengths, wea!ness and opportunities" Thisincludes results from the benchmar! that show agencyperformances against targets that are highly 'isible to seniormembers e"g" contractor to permanent staA ratio"

It is important to ensure that the comparisons are of li!e*to*li!e, both interms of the organisation)s/ with which an agency is comparing itself,as well as the metric which is being compared" 8ome of the factorswhich may be rele'ant are the sie of an organisation, the ser'ices itdeli'ers, its ICT re5uirements, and the way it manages its ICT" It maybe di;cult to determine whether an organisation has applied the sameinterpretation in calculating a metric that you ha'e"

For these reasons, one should not apply benchmar!ing analysis in asimplistic fashion" enchmar!ing can enable you to identify areaswhere your organisation diAers in its metrics from other agencies, andthese areas should be in'estigated to see whether there are

opportunities to impro'e performance" It is possible, howe'er, that thein'estigation will conclude that that there are 'alid reasons for thediAerence in the metrics"

%n agency can also compare its performance o'er time, gi'en thatbenchmar!ing data is a'ailable going bac! to 2--*-4" % !eyad'antage of this approach is that you will ha'e a detailedunderstanding of what assumptions ha'e been made in collecting thedata, and can ha'e more condence in drawing conclusions from

h i th d t > i thi th h it i



These metrics allow agencies to understand comparati'e unit

cost by ser'ice tower and enables them to compareperformance with other agencies on a per unit basis" #here thedata is a'ailable, these metrics can be segmented by costelement, to help agencies to understand the cost dri'ers of theirunit costs"



• Category $ – Service tower su%%orting metrics

These metrics pro'ide supporting information within the ser'icetower" They can be used to identify possible root causes of theagency performance in the ser'ice tower cost =?Is"

More information about the abo'e categories is pro'ided at %ppendi$ %of this document" % detailed categorisation of the benchmar!ingmetrics is pro'ided at %ppendi$ "



$%* Benchmarking 3rocess

This section sets out a general approach to benchmar!ing" It identies

the steps you would go through in underta!ing a benchmar!ingprocess, using the benchmar!ing report as a basis" It highlights theissues you should be aware of, and is intended as a guide only"

The diagram below sets out the steps in the process"



1! Identify areas for investigation

To begin with, re'iew the %9IM: enchmar!ing eport and identifythose metrics where your agency is much higher or lower than thea'erage for your cohort" The fact of the 'ariation does not necessarilyindicate that anything is amiss" It merely highlights areas wherein'estigation is re5uired"

In deciding which areas to in'estigate, you may wish to focus on thosemetrics in ser'ice towers which represent a signicant pool ofe$penditure for your agency, because these oAer most opportunity forimpro'ing performance"

ear in mind that areas where your agency appears to be performingmuch better than other agencies may also need to be in'estigated"For instance, if your agency has a much lower unit cost per des!topthan other agencies, this may indicate e;cient management" It is

possible howe'er, that it indicates your agency has been under*in'esting in the des!top area G maybe reducing the number of supportstaA or deploying ageing infrastructure" This may result in reducedperformance in future years" It would be useful to determine that thegoodB metric was due to e;cient management"

9i'en that se'eral yearsB worth of benchmar!ing data is now a'ailable,you may wish to compare your agencyBs performance o'er time,

id tif i f i t d t i ti



ou may be able to e$plain a 'ariation in your own agencyBs

performance o'er time in the same way i"e" the 'ariation may be dueto changed re5uirements, contract prices, sourcing arrangements, etc"

ou may also need to ta!e account of the impact of changes in ICTdeli'ery arrangements within you agency e"g" if it is using cloudtechnology and L*as*a*ser'ice )where LB could be softwareB,infrastructureB, etc/" This may result in increased operationale$penditure and decreased capital e$penditure" It may also result inchanges in costs across ser'ice towers" This could also be rele'ant tocomparisons with other agencies"

If you are condent that you ha'e a good e$planation for the 'ariation,you may not need to e$plore further" ut you may wish to test thehypothesis further, depending on the supporting e'idence for thehypothesis" If you need to in'estigate further, you would proceed tothe following steps"

$! Identify agencies to )enchmar* with

In identifying those agencies to benchmar! with, you may wish to focuson those agencies which perform well against metrics of interest" oumay also decide to benchmar! with peer agencies, which are generallyagencies with some of the following characteristics0

• of a similar sie,

• i th i d t



In the case of the end user infrastructure )>.I/ ser'ice tower, for

instance, the costs allocated to this ser'ice tower should include thoserelating to email and le and print ser'ers" It is possible that anagency may ha'e mista!enly allocated these costs to the Midrangeser'ice tower, which would gi'e a misleading indication of its unit costsin the >.I ser'ice tower"

ou may nd that the 'ariation is no longer signicant when you ha'ead&usted for diAerences of interpretation, in which case no furtheraction is re5uired"

%reas where an agency appears to be cost e;cient may be a result ofunderin'estment or an area where ser'ice 5uality is poor" Con'erselyan area where cost e;ciency is poor may be a result of o'er deli'ery orwhere additional comple$ity of ser'ice deli'ery is re5uired"



-! &eve'o% and im%'ement strategies and targets to im%rove

%erformance

%s a result of the pre'ious step, you should now ha'e a detailedunderstanding of the reasons your costs )or performance moregenerally/ in diAerent areas are higher or lower than a'erage" #hereyou ha'e determined that you can impro'e performance in a specicarea, de'elop detailed strategies to realise this opportunity" Impro'edperformance is, of course, the rationale of the benchmar!ing process"

The practice of the agencies which are performing well against themetric will pro'ide a guide to you in de'eloping strategies" There maybe other sources of information on good1best practice you can draw on,such as research analysts"

%s part of this process, you should de'elop targets to measureperformance against the metric" The targets should be realistic, whileresulting G assuming they are achie'ed G in real impro'ements" The

performance of better*performing agencies should pro'ide a usefulguide in setting targets"

8ome general guidance about strategies for impro'ing performancewithin specic ser'ice towers drawn from the %. ICT ?rogram ispro'ided at %ppendi$ C" This is oAered for guidance only" +one of thestrategies may be rele'ant to your agency, but they may be useful intriggering potential hypotheses for further in'estigation"



$%+ Identif4ing Areas for In5estigation

elow is an e$ample showing how an agency might use the %9IM:enchmar!ing eport to identify areas for further in'estigation" Thee$ample is based on a notional agency, called the Department of%dministrati'e %Aairs )D%%/"



The chart abo'e shows that the D%% is utilising Q4O of its a'ailablestorageP a much higher rate than the ma&ority of the cohort" Thepercentage of utilisation has also increased from QRO to Q4O between2--4*-Q and 2--Q*-"

The chart below demonstrates a more concerning scenario where theD%%Bs storage utilisation and a'ailable capacity may eliminate theorganisationBs ability to absorb additional storage demand or agency

ti it



much lower unit costs for their storage, generally under E2,---, for

similar 'olume of installed storage"



Conc'usion

The analysis abo'e suggests that the D%% storage demand is graduallygrowing and cost is e$panding" ?aralleled in this scenario is theimpending storage ceiling, which is potentially creating a signicantris! to the agencyBs ability to continue to operate normally" %lso, theD%% appears to be incurring an unnecessarily high unit cost incomparison with its peers"

This assessment is based on analysis of the charts only" The ne$t stepis to 'alidate the analysis through further in'estigation, includingthrough discussions with peer agencies"



$%. Briefing !enior !takeholders onthe Benchmarking esults

ou may wish to brief sta!eholders within your agency, such as theC>:, on the results of the benchmar!ing process" The metrics thatwould be of interest to senior sta!eholders are typically the :'er'iewmetrics )see 8ection abo'e/, which pro'ide high le'el indicators ofe$penditure and sta;ng le'els for agencies"

The benchmar!s you use to brief sta!eholders could include the %9IM:benchmar!ing reports as well as any other source data you ha'eaccess to )e"g" published e$ternal benchmar!s or benchmar!ingconducted specically for your agency by an e$ternal pro'ider/"

8ome of the !ey metrics, and the issues you would need to consider, inbrieng sta!eholders are listed below0

/etric Issues for consideration



across yearsJ

?roportion ofinternalstaA1e$ternalstaA

How has your agency performed against its contractorreduction target )if it had one/J

Does your agency ha'e a target for the proportion ofinternal1e$ternal staA for wor!force planningpurposesJ If yes, what acti'ities are you underta!ingto progress towards the targetJ

#hat cyclical changes in your agencyBs business ha'e

impacted on this metric e"g" has your agency beenunderta!ing new pro&ects which ha'e re5uired theengagement of additional contractorsJ

(argest poolsof e$penditure)by ser'icetower and costelement/

#hat are your agenciesB largest pools of e$penditureJ

#hat factors in your agencyBs en'ironment e$plainthe distribution of e$penditure across poolsJ

How does the metric for your agency compare withthose you would consider peers, based on sie,business, capability etcJ

%re there 'ariations in the metric for your agencyacross yearsJ

Do you ha'e any plans to change the distribution ofe$penditure across the poolsJ

. i f H d h i f i h



#here rele'ant, you would highlight any acti'ity you are underta!ing

to in'estigate any anomalies in your benchmar!ing results" ou mayalso wish to discuss more detailed metrics from particular ser'icetowers where these are particularly rele'ant to your agency"

The analysis of the metrics listed abo'e can be used to facilitate abroader, more strategic discussion on issues such as0

• the dri'ers of ICT e$penditure for your agencyP

• whether it is in'esting in new capabilityP

• balancing internal and e$ternal resourcesP and

• strategies for managing increased demand and costs"



A66endi7 A 8 Metric Categories

%9IM: calculates a range of benchmar!ing metrics, which 'arydepending on the sie of the agency" %9IM: categorises agencies bythe sie of their ICT e$penditure )large0 greater than E2-m, medium0E2m*E2-m, small0 less than E2m/ and collects a diAerent set of metricsfor each cohort"

The metrics ha'e been categorised according to the followingclassication, based on the granularity of the metric0

• Category 1 – Overview metrics

These metrics pro'ide high le'el indicators of e$penditure andsta;ng le'els for agencies" These metrics can be used torespond to ad hoc re5uests for information on IT e$penditure,monitor #hole*of*9o'ernment targets, and pro'ide conte$t forin'estment planning and business cases"



capacity utilisationP producti'ity of support personnelP

eAecti'eness of procurement arrangements for hardware andsoftwareP and the competi'eness of outsourced ser'ices" Inaddition the information contained in these metrics may be usedto support business cases"

=?Is for the ser'ice towers are listed below"

• ICT Management * %nnual cost per >nd user infrastructureuser

• %pplication G %nnual cost per Function ?oint

• Mainframe G%nnual cost per Installed MI?8

• Midrange G %nnual cost per :8I

• 8torage G %nnual cost per installed T

• #%+ G %nnual cost per Mbps

• 9ateways G %nnual cost per Total uploaded anddownloaded 9

• (%+ and %8 G %nnual cost per acti'e (%+ port

• >nd .ser Infrastructure G %nnual cost per end userinfrastructure user

• Voice G %nnual cost per agency FT>



A66endi7 B 8 ICT Metrics Catalogue

!ection(ate-gory

#eference to21-11 "argeagency re$ort

*y$e 'etric /arge 'edium !ma""

8cope ?age 2 Total Cost

Total ICT e$penditure split by costelement and ser'ice tower

(for small agency report, theservice tower aspect is notdisplayed)

8cope ?age N G R Total Costagency aggregate facts andgures

:'erall metric ?age 3 Total Cost Total ICT e$penditure G ope$ Sdepreciation

:'erall metric ?age Total Cost Total ICT e$penditure G ope$ Scape$

ll i l

ICT e$penditure as a proportion of

l di



!ection(ate-gory

#eference to

21-11 "argeagency re$ort


:'erall metric ?age *2 FT> Cost Cost of e$ternal ICT FT> by agency

:'erall metric ?age FT> Cost:'erall cost of ICT FT> by ser'icetower

:'erall metric ?age N FT> CostCost of internal ICT FT> by ser'icetower

:'erall metric ?age R FT> CostCost of e$ternal ICT FT> by ser'icetower

:'erall metric ?age 3, 4 FT> atio?roportion of ICT FT> that aree$ternal by agency

:'erall metric ?age , 4 FT> atio?roportion of ICT FT> that aree$ternal by 8er'ice Tower

ICT Management ?age N- Total Cost?roportion of ICT e$penditure onICT management

:'erall metric ?age N FT> atio?roportion of internal ICT FT> ateach %?8 pay*grade

%pplication ?age N Total Cost?roportion of ICT e$penditure onapplications

%pplication 2 ?age NN atio?roportion of C:T819:T81espo!esolutions



!ection(ate-gory

#eference to



Midrange ?age RN :therni$ .nit cost per installedphysical midrange ser'er againstnumber of ser'ers

Midrange ?age RR :ther+umber of operating systeminstances per installed physicalmidrange ser'er

Midrange ?age R3

:ther+umber of physical ser'ersmanaged per midrange FT>

8torage ?age R4 Total Cost?roportion of total ICT e$penditureon storage

8torage ?age RQ :ther

Total installed storage capacity

(In the large agency report, this

metric is split by #$ine, %xternal,&', '& storage types)

8torage ?age 3- :ther.tilisation of installed storagecapacity )O/

8torage ?age 3 :ther.nit cost of installed storage per T against total installed storage

8torage ?age 32 :therInstalled storage capacitymanaged per storage FT>



!ection(ate-gory

#eference to



>nd .serInfrastructure

?age 3 :therMi$ of des!tops, laptops and thinclient de'ices


?age :ther+umber of end user de'ices peragency FT>


?age 4 :ther.nit cost per des!top, laptop andthin client


?age Q :ther:'erall unit cost of ?Cs againstnumber of ?Cs


?age 4-

:ther+umber of de'ices managed per>nd .ser Infrastructure FT>


?age 4 :ther +umber of agency FT> per printer


?age 42 :ther?rinting cost per agency FT>against ratio of printers to FT>

Voice ?age 4N Total Cost?roportion of total ICT e$penditureon 'oice

Voice ?age 4R :ther?roportion of 'oice e$penditure byclassication



A66endi7 C 8 Identif4ing "66ortunities to Im6ro5e 3erformance

How to get More Value from the ICT Benchmarking Data ' )1



How to get More Value from the ICT Benchmarking Data ' )2



How to get More Value from the ICT Benchmarking Data ' *&



How to get More Value from the ICT Benchmarking Data ' *$



How to get More Value from the ICT Benchmarking Data ' *)

Documents

How to Get More Value From the ICT Benchmarking Data