WP 1767 0406 Managing It Business Risk

8/11/2019 WP 1767 0406 Managing It Business Risk

1/26

MANAGING IT BUSINESS RISK

SAFEGUARDING THE ORGANISATION FROM IT FAILURE

A SURVEY AND WHITE PAPER PRODUCED IN CO-OPERATION WITH THE ECONOMIST INTELLIGENCE UNIT


2/26


WWW.MERCURY.COM 1

Acknowledgments2

Executive summary 3

IntroductionDefining IT Business Risk 4

The link between IT and business outcomes 5

Risk management today 9

The sources of risk 13

Priorities for mitigating ITBusiness Risk16

Survey results 18



3/26


WWW.MERCURY.COM 2

ACKNOWLEDGEMENTS

This report was prepared by Mercury in

co-operation with the Economist Intelligence

Unit. The author of the report was Terry Ernest-

Jones, and the editor was Denis McCauley.

The report is based on the findings of a survey of

1,077 IT professionals based in the Americas,

Europe, the Middle East and Asia-Pacific. The

survey was designed by Mercury and the

Economist Intelligence Unit, and executed by

Vanson Bourne. Our sincere thanks go to the

survey participants for sharing their insights on

this topic.

About Mercury

Mercury is the global leader in business

technology optimization (BTO). Mercury is

committed to helping customers optimize the

business outcomes from IT.

About the Economist Intelligence Unit

The Economist Intelligence Unit is a division of

The Economist Group. Sister companies include

The Economist newspaper, CFO magazine, and

an array of other specialist publications.

The Economist Intelligence Unit has been

providing information and advisory services to

the global business community for more than

50 years through many channels, including print

publications, electronic media, and conferences

and client meetings organized under The

Economist Conferences brand.

Whilst every effort has been taken to verify the

accuracy of this information, neither Mercury

Interactive Corporation and its affiliates nor the

Economist Intelligence Unit Ltd. and its affiliates

can accept any responsibility or liability for reliance

by any person on this information.


4/26


WWW.MERCURY.COM 3

Executive summary

It is well recognised that a firms information technology (IT) infrastructure, organisation and strategy

must be aligned with its business objectives. Less understood is the reality that most of the firms

core business processes are already driven by IT, and that the ability of the firm to achieve its desired

business outcomes usually hinges on IT systems performance. All organisations bear IT Business

Riskthe danger that the failure or under-performance of IT applications, infrastructure or systems will

result in negative business outcomes for the organisation. Executives awareness of security risk is

high, but many fail to grasp the implications of IT Business Riskespecially the potential cumulative

damage of minor system faults.

This report, produced by Mercury in co-operation with the Economist Intelligence Unit, and based on

a survey of 1,077 IT executives from around the world, finds that in over half of

companies polled, no more than one in two IT projects undertaken in the past two

years has delivered positive business outcomes. In this way, IT risk translates

directly into business risk. Most companies (although substantially less in Europe

than in other regions) claim to manage IT Business Risk in a co-ordinated fashion. At

most firms, however, both accountability and day-to-day responsibility for managing

risk lie within the IT department, where good risk management practice is less well

understood than at higher levels.

Other key findings include the following:

Revenue loss and defection of customers are the most feared outcomes of IT

failure, as is reputation or brand damage from service shutdowns. At the same

time, IT Business Risk is most often measured in terms of potential cost (in terms

of incurred expenses or unrealised savings).

Executives view the supply chain and logistics as the parts of the business most

vulnerable to IT failure. Among different business initiatives, efforts to boost cost-

efficiency and customer satisfaction are seen to be most dependent on IT.

Poor project management and business requirements definition, as well as difficulties in managing

change, are the main reasons why IT initiatives fall short. Many projects also fail due to flawed

implementation.

Security initiativesparticularly when they failpose the greatest risk to the firms business

objectives. SOA, Web Services and outsourcing also entail considerable risk of generating negative

business outcomes.

The most reliable means of mitigating IT Business Risk are fundamental and long-term in nature:

building a genuine partnership between business and IT executives, based on a shared view of the

issue, and improving the effectiveness of project management.

A telephone-based survey was

conducted in the Americas, Europe,

the Middle East and Asia-Pacific from

February through April 2006. The

survey reached a total of 1,077 IT

executives drawn from a range of

small and large organisations. A total

of 42 percent of executives werebased in Europe and the Middle East

and 30 percent in the Americas, with

the rest coming from the Asia-Pacific

region. The survey includes a cross-

section of industries, including

financial services, manufacturing, high-

tech, pharmaceuticals, retail,

distribution, transport, telecoms,

utilities and business services.

ABOUT THE SURVEY


5/26


WWW.MERCURY.COM 4

IntroductionDefining IT Business Risk

A companys business and financial performance today depends on its information technology (IT)

applications, infrastructure and systems. Typically 80-90 percent of core business processes are

automatedfrom customer service and invoicing to supplier management, payroll, regulatory

compliance and many others. When IT applications and infrastructure perform to expectations, they

undoubtedly help the firm to achieve its business objectives. If they fail or under-perform, however,

the business consequences can be far-reaching and difficult to isolate.

IT Business Risk refers to risks arising from the failure or under-performance of IT that result in

negative business outcomes for the company. The latter include more than just the costs incurred to

repair systems; they may be manifested in a loss of revenue or customers, delayed product launches,

or the failure of a new application to realise projected cost savings. Egregious systems failures can

sometimes result in damage to a firms brand and reputation, or may result in the filing of legal claims

against the firm. In some cases, poor IT performance contributes to failures in integrating newly-

acquired firms, the consequences of which can be devastating for the merged companys long-term

financial health.

IT Business Risk differs from conventional definitions of IT-related risk, which focus on security threats

from viruses and hackers, and disasters from fire or terrorism. These capture the attention of the

board, and in the worst cases the media, whereas the risk from less dramatic problems is routinely

neglected. Theres a danger of too much focus on pure failure, says Eric Holmquist of Advanta Bank

Corp. and chair of the US-based Risk Management Associations committee for operational risk

management in IT. But what happens if a given technology kind of fails? He cites the example of

data corruption on customer records: businesses may not immediately recognise this, but the

consequences can be devastating.

IT Business Risk is not a new concept. Over two-thirds of IT managers worldwide already say it is

managed in an organised fashion at their companies. This is encouraging, but other survey results

suggest that recognition of IT Business Risk is not matched everywhere by well-thought-out risk

management practices. In most cases, organisations are not paying enough attention to this kind of

risk, says Michael Gough, CEO of the UKs National Computing Centre (NCC), the countrys

paramount IT industry organisation.

Based on quantitative analysis, as well as insights from interviews conducted with senior IT executives

and industry experts, this report sheds light on how companies around the world view the nature of IT

Business Risk and its chief sources, the structures they use to manage it, and the strategies that

many are employing to mitigate this type of risk.


6/26


WWW.MERCURY.COM 5

The link between IT and business outcomes

IT has delivered immeasurable value to companies over the past 15 years in

improving cost-efficiency, enhancing decision-making and, more recently, creating

new channels for revenue generation. But IT advances have also introduced greater

complexity into companies infrastructure and applications. The fact is that many IT

projects fail to deliver. Teams have developed and implemented large and small IT

projects for decades, but many repeat the same mistakes despite vastly improved

development tools and a wealth of experience won through bitter trial and error. The

survey conducted for this report reveals that, for 54 percent of firms across the

globe, no more than one in two IT projects has delivered positive business

outcomes over the past two years.

Other research supports this finding. In a recent survey by KPMG International,

nearly half of 600 organisations reported at least one project failure in the past

year1. And the Standish Group, a consultancy that tracks project failure rates in the

United States, says in a 2004 report that more than half of IT projects over the

previous two years were completed late, over budget or lacking intended features,

and almost one in five were cancelled before completion2.

The impact on business outcomes

Be it ITunder-performance or full systems crashes that cause major disruptions, the consequences of

IT failures are usually manifested in negative business outcomes. And these can be sudden:

Protection of information isnt something new; what has changed is that you can lose data much

faster now, says Mr Holmquist.

The effects of IT failure can be enormously harmful to firms financial health. Worldwide, loss of

revenue is the outcome that companies dread most from IT failure. IT has taken centre stage, says

Key points:

At over half of companiesglobally, no more than one intwo IT projects delivers positivebusiness outcomes

Companies consider improvedcost-efficiency as the businessoutcome most dependent on IT;at the same time, loss ofrevenue and customers are themost feared outcomes arisingfrom IT failure

The supply chain and logisticsare the most vulnerable areasof company operation to ITfailure

WHICH ARE THE MOST FEARED BUSINESS OUTCOMES OF IT FAILURE WITHIN YOUR ORGANISATION?

Asia-Total EMEA Pacific Americas

Revenue loss 43% 48% 34% 46%

Loss of customers 40% 36% 44% 43%

Reputation or brand damage from publicly recognised service failures 30% 27% 41% 22%

Unanticipated business expenses (eg, to repair or replace systems) 19% 17% 15% 26%

Not making planned cost savings 18% 26% 11% 13%

Delayed launches of new products/services 11% 9% 15% 12%

Legal claims (eg, from customers, shareholders) 10% 14% 6% 8%

Regulatory challenges arising from compliance failures 8% 9% 12% 4%

Failure to integrate businesses or departments in M&A situations 6% 4% 12% 2%

1 Global IT Project Management Survey, November 20052 The Chaos Report, 2004


7/26


CP Gangadharaiah, senior vice president with IT services providerWipro Technologies, based in India.

For large and small companies alike, systems failure will impact revenue immediately. In just one of

many examples, an unsuccessful upgrade to its customer relationship management (CRM) system is

estimated to have cost US mobile operatorAT&TWireless (now part of Cingular) US $100 million in

2003 and 2004. And the examples are not limited to the private sector: in another celebrated case in

the UK, software errors led the Inland Revenue to make about US $3.5 billion worth of tax-credit

overpayments in 2004 and 2005.

Loss of customers is another feared business outcome from IT failure. Most often this results from

breakdowns of CRM systems of the type mentioned above, but website downtime or other process

failures can also generate intense customer frustration leading to churn, as Amazon and other online

retailers have found to their dismay. For survey respondents in Asia-Pacific, the prospect of customer

defection from IT failures causes the greatest anxiety.

Other firms (30 percent of our global sample) cite damage to their brand or reputation among the

most feared outcomes of IT failure. Such damage often goes hand in hand with revenue or customer

loss. The US airline Comair, for example, failed to take action to replace a critical legacy system which

managed its flight crews, nor did it take into consideration the risk if the application crashed. When

this happened, it cost the airline US $20 million in lost revenue and badly damaged its reputation.

Probably the most visible and tangible effects of IT failures are the costs incurred in correcting faults

and, in the case of longer term projects, the loss of sunk investment in initiatives that go wrong. In

2004, for example, the UK supermarket chain J Sainsbury abandoned a supply chain management

(SCM) system in which it had invested US $530 million.

WWW.MERCURY.COM 6

HOW DEPENDENT ARE EACH OF THESE TYPES OF BUSINESS OUTCOMES ON IT?

1 2 3 4 5No dependence on IT Some dependence on IT Heavy dependence on IT

ABILITY TO MERGE WITH OR ACQUIRE OTHER BUSINESSES

NEW PRODUCT INTRODUCTION OR INNOVATIONS

ENTRY INTO NEW GEOGRAPHICAL MARKETS OR CUSTOMER SEGMENTS

REDUCE COST OF OPERATIONS

CUSTOMER SATISFACTION

REGULATORY COMPLIANCE

13% 14% 27% 28% 19%

10% 14% 25% 29% 22%

11% 17% 33% 24% 15%

2% 8% 23% 38% 29%

4% 9% 28% 33% 25%

5% 13% 32% 25% 25%


8/26

WWW.MERCURY.COM


7

Such IT calamities grab the headlines, but Mercurys Chief Marketing Officer, Christopher Lochhead,

believes that companies must not lose sight of where risk resides: Its often the smaller things that

get overlooked, he says, citing the loss of productivity if an HR system fails. It is impossible to hire

an employee without having the technology up and running.

When asked how dependent various business outcomes are on IT, two-thirds of respondents in the

surveyand over three-quarters of firms with more than US $5 billion in annual revenueplace

greatest emphasis on reducing operating costs. IT is regarded as crucial in increasing profitabilityfor

example, by automating processes and lowering headcount, thus boosting productivity. It is also an

enabler of outsourcing, which financial services firms have employed to particular benefit to reduce

operating costs. (At the same time, as we will discuss later, respondents view outsourcing as a

significant source of IT Business Risk.)

Customer satisfaction is also perceived to be heavily dependent on IT for the majority of surveyed

firms. As companies expand their interaction with customers to embrace multiple channels beyond

voice (to include the web, SMS texting, email and others), and as contact centres shift to IP platforms,

this level of dependence is certain to rise. Increasingly, customers judge an organisation not only on

the smooth service they receive via the website or contact centreboth of which require the highest

standards of IT performancebut also on other IT-dependent factors, for example, how

knowledgeable agents are about previous interactions.

Areas of vulnerability

Across all regions, supply chain and logistics are the areas of company operation perceived as mostvulnerable to IT failure. Not surprisingly, this vulnerability is felt most acutely by respondents from the

retail and distribution/transport industries (perhaps with the unhappy example of J Sainsbury, cited

above, in their minds), as well as by telecommunications firms and utilities. It is also more

pronounced in the largest firms in the surveythose with over US $5 billion in revenue. With the

scattered supply chain, everything grinds to a standstill when systems are down. Customer service,

IN WHICH PARTS OF THE BUSINESS ARE OPERATIONS MOST VULNERABLE TO IT FAILURE?

Firms with Firms with

over US$1bn over US$5bn Total in revenue in revenue

Supply chain/logistics 32% 34% 37%

Customer Service 24% 25% 28%

New product/service development 23% 24% 26%

Production 22% 22% 25%

Finance 20% 19% 16%

Sales 20% 18% 15%

General Management 18% 16% 15%

Compliance 10% 13% 14%

Marketing 10% 11% 12%

Human Resources Management 7% 7% 7%

Other 4% 4% 1%


9/26


WWW.MERCURY.COM 8

for the reasons cited above, along with production and financial operations, are also prominent in the

list of vulnerable areas to IT failure.

IT initiatives that matter most

IT initiatives, of course, produce positive, as well as negative business outcomes, although the latter

tend to attract greater publicity. For better or for worse, ITmanagers believe that enterprise

application deployments such as ERP (enterprise resource planning) or CRM are the IT initiatives that

are most closely linked to driving business outcomes. (Asia-Pacific and EMEA respondents cite

enterprise application deployments most frequently in this context, while those in the US include it

among their top three.)

ERP offers tempting benefits, although executives often find that new systems present them with

either over-complex or simplistic data on which they are supposed to make decisions. Failed ERP

deployments due to applications conflicts are also common, and have costly consequences. In 2004,

car hire firm Avis Europe cancelled an ERP deployment, costing the company US $54 million. In the

same year, US IT giant Hewlett-Packard was forced to report that problems with its ERP system

contributed to a US $160 million loss.

IT managers also see service-oriented architecture (SOA) and Web Services initiatives as closely

linked to business outcomes, as well as moves to centralise and consolidate IT systems. Few IT

professionals need to be convinced of the benefits that SOA offers, but the risks associated with it

are not yet widely recognised. It is still early in the implementation stage, and problems are certain to

emerge before SOA delivers in full on its potential. (More on SOA-related risks later.)

WHICH OF THE FOLLOWING IT INITIATIVES ARE MOST CLOSELY LINKED TO DRIVING BUSINESS OUTCOMES IN YOUR COMPANY?

Firms with Asia- over US$5bn Total EMEA Pacific Americas in revenue

Enterprise application deployments (ERP/CRM) 35% 33% 44% 30% 38%

SOA/Web Services 26% 26% 23% 30% 27%

Centralisation/Consolidation 26% 32% 19% 25% 40%


10/26


WWW.MERCURY.COM 9

Risk management today

Companies may understand the concept of ITBusiness Risk, but

this does not mean that all have come to grips with it. While most

IT managers globally say their firms manage it in an organised

fashion, 40 percent of respondents from the EMEA region

(Europe, Middle East and Africa) say it is not the case in their

companies. Most firms seem to equate IT Business Risk with other,

more familiar and more visible categories, such as financial risk,

security and operational risk, and managing them all under one

portfolio. Again, however, there are stark differences between

regions, with firms in Asia-Pacific and the US more likely to

manage these in a single risk portfolio than those in EMEA.

The inference is that not all firms fully appreciate the nature of IT

Business Risk and the potential consequences of systems failure.

Some practitioners believe there is less than meets the eye to

executive assertions that their firms manage this risk well.

Executives dont realise how much of the core business processes are enabled by technology, says

Mercurys Mr Lochhead. Executives tend to treat IT in the same way as an elevator. It doesnt occur

to them that it is liable to break down until they are stuck between floors waiting for help.This

attitude, says Mr Lochhead, is all the more unwise as regulatory pressures increase. The level of

acceptable risk has changed.

Establishing accountability

This view is borne out by the finding that in the majority of firms (59 percent), accountability for IT

Business Risk resides outside the executive suite. In more than half of the surveyed firms (47 percent

of those with revenue of over US $5 billion), the IT director or IT manager hold primary accountability,

rather than the CIO, CEO or other senior executive.

Responsibility for day-to-day management of risk also lies mainly with non-business managers.

Moreover, although senior IT managers are primarily responsible at a majority of firms, management

responsibility is often spread across several roles. These include line-of-business managers, compliance

officers, chief risk officers, the programme management office and even outsourced IT suppliers. (In

EMEA about one in ten companiesand about four in ten in Latin Americahas no one managing it.)

Key points:

Most companies claim to manage IT BusinessRisk in a co-ordinated fashion, butunderstanding of its nature, and of good riskmanagement practice, may not run as deep

Accountability for IT Business Risk residesbelow the most senior levels of management inmost firms, and most frequently with ITmanagers; the latter are also tasked with day-

to-day risk management, but this responsibilityis often shared with several other executivesless directly accountable for IT Business Risk

IT Business Risk is most often measured interms of potential cost, although many firmsview it through the prism of potential lostrevenue or brand and reputation damage

PLEASE INDICATE WHETHER YOU AGREE WITH THE FOLLOWING STATEMENT:

IT BUSINESS IS NOT MANAGED IN A CO-ORDINATED FASHION AT MY COMPANY

Asia-Total EMEA Pacific Americas

Agree 27% 40% 12% 22%

Disagree 68% 56% 83% 71%

Don't know 5% 3% 6% 6%


11/26


WWW.MERCURY.COM 10

This is a worry for some experts, given the potential consequences to the business of IT failure or

under-performance. IT risk is typically managed reactively in companies, according to Mr Holmquist

of the Risk Management Association. Technical people generally have a higher tolerance of risk than

business people. Placing some responsibility on the firms business managers for IT risk is desirable,

adds Mr Holmquist. IT is best at defining what could go wrong; business is best at saying what that

would mean, he says.

Around half of companies in the survey (45 percent) utilise a program or project management office

for management of IT Business Risk. Service level management is also widespread, and so are

dedicated security teams, especially in the largest IT shops, and especially in Asia-Pacific.

WHO IS PRIMARILY ACCOUNTABLE FOR IT BUSINESS RISK IN YOUR ORGANISATION?


CIO/Head of Technology 30% 24% 38% 32% 34%

IT Director 22% 12% 40% 44% 22%

IT Manager 14% 14% 17% 12% 13%

Senior IT Manager 14% 13% 15% 14% 12%

CEO 4% 2% 9% 2% 4%

CFO 4% 5% 2% 4% 4%

Nobody 3% 4% 2% 3% 1%

Chief Risk Officer 3% 4% 2% 2% 5%Line of Business Manager 2% 1% 2% 3% 1%

Programme Management Office 1% 1% 1% 2% 2%

Outsourced IT supplier 1% 1% 3% 1%

Compliance Officer 1% 1% 1% 1%

The effects of systems failures vary, of

course, from industry to industry. For the

Danish healthcare company Novo Nordisk,

they could lead to compliance problems

and possibly a shut-down mandated by the

regulatory authorities, as CIO Lars

Fruergaard Jrgensen points out.

The companys risk structure now includes

interviews with all system owners and

managers throughout the company which

can reveal, for example, that a legacy

system is over-reliant on too few staff formaintenance. We are confident that this

thorough risk management approach will

lower the overall risk of a major

breakdown, says Mr Fruergaard Jrgensen.

For overall management of risk, clear

procedures and responsibilities must be

established. The firm has an ITrisk manager,

and a cross-functional committee reviews IT

risk on a regular basis. It uses a risk matrix

to track progress on every IT project (there

are usually 20-30 running concurrently);

reports are made monthly, and the status of

each is marked by colour according to howwell things are progressing.

When it comes to introducing new systems,

Novo Nordisk recognises the danger of

project failure during the implementation

stage. To combat this, Mr Fruergaard

Jrgensen relates that project teams at

Novo Nordisk are not disbanded until user

satisfaction has reached an acceptable

level, key performance indicators have

been attained and mandatory benefits are

realised.This may require several weeks,

but the company feels it is worth the effort.

RISK MANAGEMENT AT NOVO NORDISK


12/26


WWW.MERCURY.COM 11

WHAT MANAGEMENT TECHNIQUES OR STRUCTURES DOES YOUR BUSINESS EMPLOY TO MANAGE IT BUSINESS RISK?


Programme/project management office 45% 48% 46% 39% 54%

Service level management 43% 52% 36% 39% 50%

Dedicated security team 38% 41% 45% 28% 53%

Formal change management 34% 48% 23% 27% 53%

Centralised quality assurance function 33% 35% 25% 43% 45%

Measuring risk

It is often said that if risk cannot be measured, it cannot be managed. There is a broad range of

approaches to measuring IT Business Risk, but companies most frequently view it in terms of potential

costfor example, the value of sunk investment in an application. (This is particularly the case in firms

with large IT departments, having over 1,000 staff.) But as discussed earlier, negative business

outcomes often take the form of lost revenue, and indeed nearly 40 percent of surveyed companies

HOW DOES YOUR COMPANY MEASURE IT BUSINESS RISK?


Quantified as potential cost 48% 50% 50% 42% 60%

Quantified as potential lost revenue 39% 40% 38% 38% 48%

It is assessed in qualitative terms 35% 36% 42% 27% 42%

Impact on corporate reputation/brand 33% 36% 37% 24% 45%

We do not attempt to measure IT Business Risk 18% 17% 12% 24% 5%

For Chicago Public Schools there is an

urgency to use IT systems to raise

standards that few commercial

organisations face: of the 430,000 students

in its care, 85 percent are below the

poverty line. A team of 200 full-time IT staff

supplemented by consultants run a variety

of applications for the network of schools,

including enterprise systems, HR, finance

and student information. There are 85,000

computers across the school system. Its

like a Fortune 250 company, says CIO

Robert Runcie, a Harvard graduate with abusiness background at Accenture and

Computer Sciences Corporation. To provide

transparency and drive accountability, Mr

Runcie has created an IT governance

process which presents to key C-level

officers and stakeholders. There is also an

enterprise project management office to

ensure strong project management.

Before an IT project starts, one key factor is

settled in order to mitigate riskappropriate

staffing. We dont do a project unless we

can find the right skills and experience

within the organisation, says Mr Runcie.Once started, project risk is managed on an

on-going basis. A steering committee is

given regular status reports on each IT

project.

Mr Runcie points to another critical risk

factorexecutive sponsorship. The CEO and

other key officers need to be on board and

engaged with the project on a regular basis.

IT risk is not a matter of how perfectly you

execute the project, Mr Runcie believes.

Rather, its ensuring proper alignment of the

project to the organisations strategic goals.

The biggest IT risk is in failing to impact onthe business properly.

AN EDUCATION IN ITRISK MANAGEMENT


13/26


WWW.MERCURY.COM 12

view IT risk primarily through this prism. Still other firms make qualitative assessments of the risk of an

IT projects failure, for example as the potential damage to the companys brand or reputation.

The predominance of a cost-based perspective is not surprising given the finding that so much

responsibility for management of IT Business Risk resides within the IT department, widely regarded

by management as a cost centre. But this approach to measuring risk is potentially limiting. By

taking a broader view, one which includes the potential of lost revenue, firms could focus on

prevention rather than merely remedial measures. Fundamentals must be calculated: What is the

impact of failures on external stakeholders such as customers and distributors, as well as on other

departments in the organisation? What is the toll if an application cannot be accessed? What figure

can be estimated for lost sales from a CRM fault?


14/26


WWW.MERCURY.COM 13

The sources of risk

When IT initiatives fail or under-perform, and have a negative

impact on the business, what are the key factors involved? Three

stand out for companies across the world: ineffective project

management (including resource and budget management), poor

business requirements definition, and difficulties in handling

change.

Its not software or technology that kills a project, says Robert

Runcie, CIO serving Chicagos 640 public schools. (See box on

page 11.) Its getting well-defined requirements that makes or

breaks it. You have to make sure theyre set in advance, and that

the team knows what the transformation should look like.

Theres a tendency for executives to lob an idea over the wall, and

expect IT to use osmosis to understand it, says Mr Holmquist.

Operational risk for IT projects should focus on three areas, he

says: quality of business requirements definition, quality of

implementation, and quality in the selection of tools.

Better strategic technology planning is also needed in order to

map technology strategy to business strategy, adds Mr Holmquist. But, he observes, communication

between IT and other business functions is often problematic. Typically there is a language barrier

between business and IT. One of an IT departments most valuable assets, Mr Holmquist notes, is

that rare breed of business analyst who can act as a liaison between IT and other business functions.

You need to have a business champion who is responsible for delivery of benefits, adds Mary Lacity,

Professor of Information Systems at the University of Missouri-St. Louis (in the US). IT is responsible

for delivery of products and is accountable for costs.

Seeing the projects through

Even where communication flows well, organisations find that deployment and roll-out issues

frequently wreck IT projects. (This is especially problematic in Europe.) Often projects fail at the very

end when users are not trained properly to benefit from the new system. Most people ignore the risk

at the implementation stage, says the NCCs Michael Gough. Often, the more money spent on

analysis and design, the less is spent later on training. Its vital to have enhanced training.

A case in point is the UK governments multi-million dollar Jobcentre Customer Management System,

which handles claims for income support and allowances. The system, implemented over 2003-2005,

has been the object of heavy criticism in the British parliament and press for a 40 percent failure rate

in processing claims. The system had a very poor reception from staff, and an independent report

identified inadequate training, a failure to listen to staff concerns and inflexibility in using the system

as causes for the failure.

Key points:

Ineffective project management, poor definition ofbusiness requirements and difficulties in handlingchange are most often to blame when IT initiativesfail to produce positive business outcomes

Formal change management is the factor mostblamed by large companies (over US $5 billion inturnover) in Europe and the US for IT initiativeswhich fail

Follow-through is critical: many IT projects failduring the implementation stage. Inadequatetroubleshooting and poor user training are frequentlythe culprits

Security systems deploymentsparticularly whenthey failpose the greatest risk to the firmsbusiness objectives. Newer initiatives, such as SOA,Web Services and outsourcing, also entailconsiderable risk of generating negative businessoutcomes


15/26


WWW.MERCURY.COM 14

WHEN IT INITIATIVES HAVE FAILED TO PRODUCE THE EXPECTED BUSINESS OUTCOMES IN YOUR COMPANY,WHAT HAVE BEEN THE PRIMARY FACTORS?

Firms with Firms with over US$1bn over US$5bn Total in revenue in revenue

Project management (including resource and budget management) 28% 27% 28%

Business requirements definition 24% 25% 26%

Deployment or rollout issues 19% 24% 25%

Poor quality software/technology 17% 14% 13%

Business environment change 12% 14% 15%

Quality assurance (functional, integration, and system testing) 12% 13% 12%

Development issues (design/architecture issues, code quality/developer testing, etc.) 12% 13% 10%

Requirements governance (ie, scope creep) 11% 17% 20%

Change management 11% 10% 9%

Quality of implementation also has much to do with system design, testing and performance in the

pre-launch phaseand monitoring after launch. Surveyed executives point to problems in applications

testingsuch as poor coding, quality assurance, load/performance testing and system diagnosticsas

sources of project failure and negative business outcomes. A new application could have thousands

of functions, says Mr Gangadharaiah of Wipro. End-user testing used to be relevant when

applications were simpler. A professional testing organisation is needed now.

Difficulties in coping with change figure prominently as sources of IT failure (providing three of the

top ten failure-related factors cited in the survey), and particularly for larger firms. Deployment issues

frequently arise due to employees resistance to change, both in the systems that they use and in the

business processes that they are a part of. Changed circumstances in the business environment

such as new regulatory decisionscan alter the requirements of a project; this need not be a problem

if the new requirements are assimilated and communicated but, as discussed above, this is often not

the case. Problems in these areas point to shortcomings in change management, both within the IT

department and in the business as a whole.

Sometimes performance issues at outsourcing partners are at the root of IT project failure.

Elsewhere, production application or service management may be the problem. Other causes of

failed projects stem from changes in specifications. You need to put a box around scope creep,

says Ms Lacity of the University of Missouri-St. Louis. Problems with requirements governance are

more likely to be a source of IT failure at larger organisations than smaller ones, judging by the survey

results.


16/26


WWW.MERCURY.COM 15

New initiatives, new risks

So great today is the fear of breaches of information securitysuch as theft of intellectual property

or virus attacksthat security initiatives, or rather their failure, are viewed by managers as posing the

greatest IT Business Risk to the company. Little wonder, as hacking is increasingly motivated by

criminal gain. Across all regions, security stands out as the riskiest type of project from a business

outcomes perspective. Other risk-laden IT initiatives include:

Service-oriented architecture. Although it can cut total IT expenses over the long term by as much

as 20 percent (according to analyst firm Gartner), many companies are currently struggling with

SOA implementation. There are any number of reasons: the supporting technology is still evolving,

and standards are maturing; it requires organisational change to cross former boundaries; and it

creates complexity in the form of a large number of small applications and services (a problem in

one can impact the others). Last but not least, SOA requires new competencies on the part of IT

staff.

Web Services. These types of initiatives are another new source of complexity, due in no small

measure to the multiplicity of competing standards and specifications, and the interoperability

headaches this causes. Web Services can also make conventional security methods irrelevant, as

the doors are opened to outsiders to enter and access a corporate system. Existing firewalls often

are not up to the security task.

Outsourcing. Some experts feel that companies are exposing themselves unwittingly to further

risk through outsourcing. With outsourcing, many organisations have divested themselves of

analysis and have eroded internal competencies, according to the NCCs Michael Gough. This

puts tremendous pressure on the CIO, who must make decisions without the appropriate

surrounding support. This plus loss of business knowledge, compliance problems and data

security breaches are other outsourcing-related risks which demand that firms maintain visibility

over the outsourcers business processes.

WHICH OF THE FOLLOWING IT INITIATIVES ARE INTRODUCING THE MOST IT BUSINESS RISK INTO YOUR ORGANISATION?


Security 36% 37% 39% 32% 33%

SOA/Web Services 29% 36% 21% 29% 28%

Outsourcing 28% 33% 21% 29% 34%

Enterprise application deployments (ERP/CRM) 22% 16% 29% 23% 16%

Quality Assurance 14% 14% 13% 13% 13%

Business Service Management 13% 11% 18% 13% 13%

Centralisation/Consolidation 13% 10% 19% 13% 16%

IT Governance 13% 15% 9% 12% 13%ITIL/ITSM 10% 14% 8% 7% 15%

Performance Management 10% 11% 8% 11% 8%


17/26


WWW.MERCURY.COM 16

Priorities for mitigating IT Business Risk

Aside from developing contingency plans to cope with major

crashesnormally a part of business continuity planningthere are

few back-up mechanisms to mitigate IT Business Risk. The most

reliable means of mitigation are more fundamental and long-term:

developing a genuine partnership between business and IT

executives, and improving the effectiveness of project

management. It is no accident that IT governance and project

management are two of the most important areas in which firms

will invest over the next year to improve IT performance. Projects

succeed where there is a true partnership between business and

IT, say Ms Lacity of the University of Missouri, St. Louis.

According to Mr Runcie of Chicago Public Schools, ensuring

adequate resourcing and staffing is a fundamental means of

mitigating IT project risk before it is launched. (See box on page

11.) Once it is under way, ensuring quality and performance before

the system goes live is a fundamental means of minimising risk of failure later on.

Another way to mitigate the effectsand lessen the likelihoodof IT failure, according to Ms Lacity, is

to reduce the scale of projects. The bigger the project, she warns, the lower the success rate. Ms

Lacity advocates dividing large projects into multiple smaller projects to minimise risk, making full

Key points:

The most reliable means of mitigating ITBusiness Risk are building a partnershipbetween business and IT executives andimproving project management

IT governance and project management are,accordingly, two of the most important areas inwhich firms will invest over the next year toimprove IT performance

Business service management and infrastructuremanagement are other investment priorities forfirms as they strive to enhance IT performance

Large firms (over US $5 billion in revenue) willfocus investment on IT governance, security,requirements management and business servicemanagement

There are several effective techniques and

approaches for mitigating IT Business Risk,

and best practice varies according to the

circumstances of each organisation. But

some common ground was found when

speaking to a range of international experts

in researching this report:

CHECK AGAINST STRATEGIC GOALS. Ensure that

the project and its immediate objectives

support the widergoals of the business.

GET SUPPORT ATTHE TOP. Support for large

projects must be established at the highest

level of the company, and a senior

champion is often required to ensure

focus is sustained.

DEFINE AND PRIORITISE REQUIREMENTS. Ensure

requirements are clearly articulated before a

project starts. By the time a project gets

under way, a large number of people, often

with conflicting requirements, are likely to

have given input. Also, determine the

importance of each requirement from the

business point of view.

PROTOTYPE. Dividing large projects intosmaller units and prototyping them can be

an effective way to lower risk. The divide

and rule principle applies. Also, users have

a chance to spot problems before the full

deployment.

TRY INDEPENDENTTESTS. With the complexity

of todays large-scale projects, testing by a

separate party (often the quality assurance

team) is found to be useful, enabling the

system to be put through its paces by an

objective source.

AUTOMATE. The vast majority of IT faults are

the result of human errors. Automating the

management of IT Business Risk will help

ensure that problems are caught and

addressed before they produce negative

business outcomes.

BEWARE THE IMPLEMENTATION STAGE. Many

projects stumble at the last hurdle.

Thorough training of users at the

implementation stage is required, but so is

constant monitoring to ensure systems

availability and performance. Monthly

reporting of project progress at a senior

level is also recommended.

AUDITREGULARLY. Regular audits should be

conducted across all existing systems to

detect problems, for example, a hidden

application that has the potential to bring

major systems down and cause damage to

the company.

GUIDELINES FOR MITIGATING IT BUSINESS RISK


18/26


WWW.MERCURY.COM 17

use of prototypingwhich in IT denotes a method of using small project components to confirm

requirements, define interfaces, troubleshoot problems and ensure that software actually meets

contractual agreements. (This is where SOA and Web Services, in their ability to break up projects

and thus IT Business Riskinto smaller discrete blocks, will ultimately benefit business outcomes.)

The bigger picture

At a higher level, companies need to get to grips with managing ongoing IT Business Risk. IT projects

tend to be complex and long-term, making it difficult for the firms senior business executives to

sustain interest in them. For the larger organisations surveyed (with more than US $5 billion in

turnover), IT governance (35 percent) and security (24 percent) are closely followed as investment

areas over the next 12 months by requirements management and business service management (19

percent each). On larger projects, the business requirements are almost certain to have changed(and the projects original sponsors may have moved on) during implementation, requiring effective

communication of the new objectives to project teams, a challenge for executives whose focus has

shifted. IT staff themselves are faced with reconciling complex, ill-defined, often conflicting and

changing business requirements while juggling millions of lines of code.

The risk of projects failure or their misalignment with the business objectives will never disappear.

Consistently following some basic guidelines will help mitigate the risk that such failure will produce

negative business outcomes.


19/26


WWW.MERCURY.COM 18

Appendix

Survey results

HOW WOULD YOU DESCRIBE THE JOB ROLES YOU HAVE HELD TO DATE?

Primarily IT 61% A mix of both ITand business 33%

Primarily business 6%

HOW WOULD YOU DESCRIBE THE JOB ROLES YOUR CIO HAS HELD TO DATE?

Primarily IT 32% A mix of both ITand business 56%

Primarily business 12%

HOW DOES YOUR COMPANY MEASURE IT BUSINESS RISK?

Quantified as potential cost 48%

Quantified as potential lost revenue 39%

It is assessed in qualitative terms 35%

Impact on corporate reputation/brand 33%

We do not attempt to measure IT Business Risk 18%

The survey was designed by the Economist Intelligence Unit for Mercury and conducted by Vanson

Bourne from February through April 2006. A total of 1,077 IT managers participated in the survey.


20/26


WWW.MERCURY.COM 19

PLEASE INDICATE WHETHER YOU AGREE WITH THE FOLLOWING STATEMENTS ABOUT THE MANAGEMENT OF IT BUSINESS RISK

IT Business Risk is not managed in a co-ordinated fashion at my company

Agree 27%

Disagree 68%

Dont know 5%

IT Business Risk is managed as a separate risk category

Agree 47%

Disagree 47%

Dont know 6%

IT Business Risk is managed within the firm's overall risk portofio(including financial risk, security risk, operational risk and other types of risk)

Agree 61%

Disagree 29%

Dont know 10%

WHO IS PRIMARILY ACCOUNTABLE FOR IT BUSINESS RISK IN YOUR ORGANISATION?

CIO/Head of Technology 30%

IT Director 22%

IT Manager 14%

Senior IT Manager 14%

CEO 4%

CFO 4%

Nobody 3%

Chief Risk Officer 3%

Line of Business Manager 2%

Programme Management Office 1%

Outsourced IT supplier 1%

Compliance Officer 1%

AppendixSurvey results


21/26


WWW.MERCURY.COM 20


WHO MANAGES IT BUSINESS RISK ON A DAILY BASIS IN YOUR ORGANISATION?

IT Manager 40%

Senior IT Manager 23%

Line of Business Manager 19%

CIO/Head of Technology 16%

IT Director 16%

Outsourced IT supplier 12%

Programme Management Office 12%

Compliance Officer 11%

CFO 11%

Chief Risk Officer 10%

Nobody 8%

CEO 4%

Other 3%

IN WHICH PARTS OF THE BUSINESS ARE OPERATIONS MOST VULNERABLE TO IT FAILURE?

Supply chain/logistics 32%

Customer Service 24%

Production 23%

Finance 22%

Sales 20%

New product/service development 20%

General Management 18%

Marketing 10%

Compliance 10%

Human Resources Management 7%

Other 4%


22/26


WWW.MERCURY.COM 21


HOW DEPENDENT ARE EACH OF THESE TYPES OF BUSINESS OUTCOMES ON IT?

1 2 3 4 5No dependence on IT Some dependence on IT Heavy dependence on IT

ABILITY TO MERGE WITH OR ACQUIRE OTHER BUSINESSES

NEW PRODUCT INTRODUCTION OR INNOVATIONS

ENTRY INTO NEW GEOGRAPHICAL MARKETS OR CUSTOMER SEGMENTS

REDUCE COST OF OPERATIONS

CUSTOMER SATISFACTION

REGULATORY COMPLIANCE

13% 14% 27% 28% 19%

10% 14% 25% 29% 22%

11% 17% 33% 24% 15%

2% 8% 23% 38% 29%

4% 9% 28% 33% 25%

5% 13% 32% 25% 25%

WHICH ARE THE MOST FEARED BUSINESS OUTCOMES OF IT FAILURE WITHIN YOUR ORGANISATION?

Revenue loss 43%

Loss of customers 40%

Reputation or brand damage from publicly recognised service failures 30%

Unanticipated business expenses (eg, to repair or replace systems) 19%

Not making planned cost savings 18%

Delayed launches of new products/services 11%

Legal claims (eg, from customers, shareholders) 10%

Regulatory challenges arising from compliance failures 8%

Failure to integrate businesses or departments in M&A situations 6%

WHICH OF THE FOLLOWING IT INITIATIVES ARE MOST CLOSELY LINKED TO DRIVING BUSINESS OUTCOMES IN YOUR COMPANY?

Enterprise application deployments (ERP/CRM) 35%

SOA/Web Services 26%

Centralisation/Consolidation 26%

Security 19%

Business Service Management 18%

Quality Assurance 18%

Outsourcing 16%

IT Governance 13%Performance Management 13%

ITIL/ITSM 9%


23/26


WWW.MERCURY.COM 22


WHICH OF THE FOLLOWING IT INITIATIVES ARE INTRODUCING THE MOST IT BUSINESS RISK INTO YOUR ORGANISATION?

Security 36%

SOA/Web Services 29%

Outsourcing 28%

Enterprise application deployments (ERP/CRM) 22%

Quality Assurance 14%

Business Service Management 13%

Centralisation/Consolidation 13%

IT Governance 13%

ITIL/ITSM 10%

Performance Management 10%

APPROXIMATELY WHAT PERCENTAGE OF IT INITIATIVES UNDERTAKEN IN YOUR COMPANY OVER THE PAST TWO YEARS HAVE HAD

POSITIVE BUSINESS OUTCOMES (A POSITIVE IMPACT ON THE COMPANYS BUSINESS)?

0% 2%

10% 7%

25% 19%

50% 26%

75% 35%

100% 11%

% of IT initiatives % of respondents

WHEN IT INITIATIVES HAVE FAILED TO PRODUCE THE EXPECTED BUSINESS OUTCOMES IN YOUR COMPANY,WHAT HAVE BEEN THE PRIMARY FACTORS?

Project management (including resource and budget management) 28%

Business requirements definition 24%

Deployment or rollout issues 19%

Poor quality software/technology 17%

Business environment change 12%

Quality assurance (functional, integration, and system testing) 12%

Development issues (design/architecture issues, code quality/developer testing, etc.) 12%

Requirements governance (ie, scope creep) 11%

Change management 11%

Outsourcing/offshoring failure 10%

Production application/service management 10%

Security 9%

Performance assurance (load/performance testing, application/system diagnostics & tuning) 7%


24/26


WWW.MERCURY.COM 23


WHAT MANAGEMENT TECHNIQUES OR STRUCTURES DOES YOUR BUSINESS EMPLOY TO MANAGE IT BUSINESS RISK?

Programme/project management office 45%

Service level management 43%

Dedicated security team 38%

Formal change management 34%

Centralised quality assurance function 33%

Portfolio management 27%

IT demand management 27%

Formal process framework 25%

IN WHICH REGION ARE YOU PERSONALLY BASED?

EMEA 42%

Asia-Pacific 28%

Americas 30%

IN WHICH COUNTRY ARE YOU PERSONALLY BASED?

EMEA

Belgium 2%

Denmark 2%

Finland 2%

France 6%

Germany 7%

Israel 2%

Italy 5%

Netherlands 3%

Spain 3%

Sweden 3%

UK 7%

Americas

Brazil 2%

Canada 2%

Mexico 2%

United States 23%

Asia-Pacific

Australia 4%

China 4%

Singapore 4%

Japan 4%

Korea 4%

India 4%

Hong Kong 3%

Malaysia 3%


25/26


WWW.MERCURY.COM 24


WHAT IS YOUR COMPANY'S PRIMARY INDUSTRY?

Manufacturing 21%

Retail, distribution or transport 21%

Finance 12%

Business Services 10%

Government/Public Sector 8%

Utilities or telecoms 7%

Technology 7%

Pharmaceuticals & Chemicals 6%

Materials Handling (Oil & Gas;Mining) 6%

Construction 2%

WHAT ARE YOUR ORGANISATION'S GLOBAL ANNUAL REVENUES IN US DOLLARS?

$500m or less 29%

More than $500m up to $1bn 28%

More than $1bn up to $5bn 17%

More than $5bn up to $10bn 8%

Over $10bn 12%

Not applicable 6%

HOW MANY EMPLOYEES WORK FOR YOUR IT DEPARTMENT GLOBALLY?

Between 50 and 100 48%

More than 100 up to 250 18%

More than 250 up to 500 13%

More than 500 up to 1,000 10%

Over 1,000 11%

2006 Mercury Interactive Corporation. Patents pending. All rights reserved. Mercury Interactive, Mercury, the Mercury logo, and Mercury BTO Enterprise are trademarks ofMercury Interactive Corporation and may be registered in certain jurisdictions. All other company, brand, and product names are marks of their respective holders.


26/26

Documents

WP 1767 0406 Managing It Business Risk