Benchmarking BI: How to Identify Critical Success Factors and Calculate ROI of Your BI Initiatives

© Copyright 2013Wellesley Information Services, Inc.

All rights reserved.

Benchmarking BI: How to Identify Critical Success Factors and Calculate ROI of Your BI Initiatives

Dr. Bjarne BergCOMERIT

2

In This Session …

• Attend this session to gain insight into key benchmarks for BI project staffing levels, support requirements, BI project durations (upgrades or new implementations)

• A tool to calculate ROI, NPV, IRR, and Payback for SAP BI• Estimate return on BI investments using three different metrics –

replacement cost analysis, new capabilities analysis, and decision option analysis

• Benchmark your BI performance against best-of-breed and industry averages for project timelines and execution, costs and staffing levels, and ongoing support

3

What We’ll Cover …

• Introductions• Return on Investments (ROI) of a BI project• A Tool to Calculate ROI, NPV, IRR and Payback for SAP BI• Key benchmarks — BI project staffing levels and support

requirements• BI project durations (upgrades or new implementations) • Wrap-up

4

BI Spending is Increasing and Gaining Momentum

• 39% of CIOs in a SnapLogic survey said that BI was the #1 priority

• BI initiatives are also the #1 priority, ahead of cloud computing, in Gartner's survey of 2,335 IT managers and CIOs

Corporations are looking for a competitive edge with BI. For example, IDC estimated that BI spending increased by 6.9% to $1.8 trillion in 2012

KEY FACTS• 50% of the 2,600 respondents in TechTarget’s

annual survey said that they would increase their BI spending by at least 10% TechTarget.com, 2012

http://www.gartner.com/newsroom/id/1897514

Bimeanalytics.com, “CIO Insights”, 2012

5

Why Are Companies Investing in BI?

• FACT: If you want to become a large organization, you need BI as a cornerstone in how you manage

Key Question: How is SAP BI enabling large organizations to grow and manage better than smaller companies?

• Examples 43% of retail companies with

revenues under $250 million use Excel as the core BI tool

13% of mid-sized retailers use Excel as their core BI tool

5% of retail companies with revenues over $1 billion use Excel as the core BI tool

Source: RSR 2011. Sponsored by SAP

6

The Relationship Between Sales and Use of BI

• FACT: Pushing BI and analytics to operational line managers and store managers increases sales

Saturating BI at all levels of the organization creates focus and pushes decision making downwards; it makes the company more agile!

• Examples Retail organizations who

outperformed their peers in sales growth had 53% usage of BI at line manager levels

Retail organizations who outperformed their peers in sales growth had 52% usage of BI for store managers

Source: RSR 2011. Sponsored by SAP

7

But, Is Business Intelligence Really Worth the Cost?• In the annual BI survey by the Business Application Research Center

(BARC), it was found that most BI investments are relying on soft justifications Faster BI More accurate reporting Improvements in business decision making Improvements in customer satisfaction rates

• Few were able to justify BI initiatives with hard Return on Investment (ROI) calculations, i.e., tangible cost reductions

After 20 years of BI and Data Warehousing, CFOs and CEOs are now demanding proof of benefits before investing more in these technologies

BARC, Bi-survey.com, 2012

8

But, Is Business Intelligence Really Worth the Cost? (cont.)

• BI capabilities have to have hard tangible benefits in order to sustain the current level of enthusiasm and funding

• The trick to do this is to not align benefits to reporting, dashboarding, and tools. Instead, focus on operational improvements and make the benefits measureable

CEOs and CFOs are increasingly asking for real evidence that BI and Analytics helps reduce costs, increases profits, and actually

contributes to the enterprise and is not merely a non-value cost area.

SAP BI Capability Key ROI BenefitsBetter capital utilization and cash investmentsBetter Marketing program financing of internal resourcesLess variance of budgetsEarlier directional change alertsReduced interest paymentsReduced inventoryLess obsolete inventoryLess back ordersIncreased focused customer serviceShorter OTC cycleShorter P2P cycleTargeted selling to prime customersIncreased bundling and cross-salesIncreased customer loyaltyEarlier detecting of demand changesLower marketing costs (reduced waste)Cross use of resourcesReduced downtimeManufacturing and vendor consolidationAlignment of true costs to sales and operationsAlignment of budgets to successful operationsEarlier detection of unprofitable areasBetter customer segmentation by true profitability (not sales)Better proactive issues resolution (2-way loyalty)Suggestive selling (market basket analysis)Targeted promotionsLife-time value (LTV) customer analysisEarly customer product adopter analysis

BPC - Financial Forecast

BPC - Business Planning

Marketing Analytics & sales

PCA, CCA and COPA

CRM

9

BI — ROI Calculators• There are many BI ROI

calculators and tools available on the Internet

• Some are vendor and tool specific, while others are focused on BI and Analytics in general

Depending on the size of your initiative, a more comprehensive calculator may be more

appropriate.

• Some calculators are very complex and comprehensive, while others are simplistic and easier to use

10




11

The Current BI Time-to-Market is Too Slow…

A key complaint is that BI development is simply too slowCurrently BI time-to-market is measured in months instead of the time scale businesses operate (days/week)

SAP BI Self-Service for power users and authors is key to fast delivery and higher BI ROI

Source: SAP AG

Source: B. Oosterhof, Informatica

For costs to go down, and ROI to increase, we have to get better at delivering BI solutions faster

BI Self Service is a key component to reduce time-to-market

12

Joint Application Design(JAD)

Rapid Application Development(RAD)

Extreme Programming(EP)

System development Life-Cycle based methodologies

(SDLC)

Impact of FailureLow High

Low

High

Time to Delivery

When to Select Different Methodologies

Time to Market — We Are Simply Too Slow …

• It has been estimated that 66% of development costs can be reduced by using an Agile or a Rapid Application Development approach for BI

• Furthermore, up to 500% of the time-to-market can be reduced through rapid prototyping instead of writing functional specs

For data warehousing, ASAP and JAD may be appropriate

methodologies, but for business intelligence Agile, Scrum, RAD, and interactive

prototyping are almost always the most cost

effective answers

Source: B. Oosterhof, “Agile and Operational BI,” Informatica, 2011

I.e. Scrum, Agile

Source: Dr. Berg, Data Management Review Magazine

13

BI Time to Market — Realizing Benefits Earlier

• Benefits (i.e., sales) should be measured both in terms of the time when benefits emerge and the magnitude of benefits

• Don’t just write the estimated benefits, track the benefits of previous BI initiatives

The Business Case for BI should require some tangible way to quantify what these are and when you should be seeing them in a way that can be measured

• Most business sponsors believe IT projects are full of rework and missing features

• You have to change their mentality from building systems to doing BI: New features are always added

Source: Michael Krigsman, "75% believe IT projects are 'doomed'. 2011

14

ROI on BI Projects — Replacement Cost Analysis

• The easiest form for ROI Analysis is to take the cost of a BI investment and simply compare it with the costs of the existing one

After this project analysis, we can now estimate the ROI based on a 3-year payback period

Application servers 74,000 Application servers 20,000 Database server 261,000 Database server 365,000 Total $ 335,000 Total $385,000

Database RDBMS annual fees 289,000 Database RDBMS annual fees -

RecoveryBackup and disaster recovery

172,000 Recovery Backup and disaster recovery 172,000

Upgrades Patches and upgrades 45,000 Upgrades Patches and upgrades 24,000

Performance Performance tuning and householding jobs

49,000 Performance Performance tuning and householding jobs

-

Storage SAN costs 201,000 Storage SAN costs - $1,091,000 Current total expenses $581,000

HANA Implementation costs $302,000 TOTAL NEW BI INITIATIVE $883,000

$208,000

Hardware support

Hardware support

Current Operations New HANA Initiative

Current total expenses

Project savings

15

ROI on BI Operations — Replacement Cost Analysis

• In this case, we are replacing the current RDBMS with HANA, so the past costs and future benefits are easier to quantify

To account for the cost of capital, we discount all future savings by 10% compounded annually. This is known as the Net Present Value (NPV).

We divide the NPV with the investment to get the simple ROI.

Current Future Savings Current Future Savings Current Future SavingsApplication servers 74,000 20,000 54,000 74,000 20,000 54,000 74,000 20,000 54,000 Database server 261,000 365,000 (104,000) 261,000 365,000 (104,000) 261,000 365,000 (104,000)RDBMS annual fees 289,000 - 289,000 289,000 - 289,000 289,000 - 289,000 Backup and disaster recovery 172,000 172,000 - 172,000 172,000 - 172,000 172,000 - Patches and upgrades 45,000 24,000 21,000 45,000 24,000 21,000 45,000 24,000 21,000 Performance tuning and householding jobs 49,000 - 49,000 49,000 - 49,000 49,000 - 49,000 SAN costs 201,000 - 201,000 201,000 - 201,000 201,000 - 201,000

Total 1,091,000 581,000 510,000 1,091,000 581,000 510,000 1,091,000 581,000 510,000 Net Present Value (10% discount rate) 510,000$ 463,636$ 421,488$ TOTAL NPV 1,395,124$ HANA Project Cost and HW (investment) 667,000$ Return on Investment (ROI) 209.2%

Yr 1 Yr 2 Yr 3Area

16

Option Theory — The Inherent ROI of BI• There is an intrinsic value of increasing the time from discovery to

time-when-action-is-required• This value is created by simply having more time to evaluate more

options before action is required

Option Theory: With BI, we discover opportunities earlier, get more time to make decisions, can evaluate more options, and have more time to implement

them

Issue or Opportunity Discovery

Action Required

Option found

Time toEvaluate

Decision made

Time to Implement

Time to analyze

No BI

Issue or Opportunity Discovery

Action Required

Option found

Time toEvaluate

Decision made

Time to Implement

Time to analyze

With BI

Option found

Option found

Time

Time

17




18

A Step-by Step Example of ROI Calculator

• In this example, we are using the BI ROI calculator from Hall Consulting and Research

• It is one of the most comprehensive BI calculators available and relies on a 6-step approach: Profile BI Initiative Selection BI Maturity Assessment Initiative costs Initiative Benefits Analysis and Report In this demo example we will look at the major

activities of this tool, but there is also an enterprise edition with even more capabilities

Source: Hall Consulting and research

19

A Step 1a – The Profile – Organization

• In this ROI tool we first enter the organizational profile. This include: Industry Location PC users BI users Financial Information and cost

structure

20

A Step 1b – The Profile – Financial Information

• From HR we enter the key labor cost numbers including:

Salaries Overhead Standard hours per

year

These Labor Costs are used throughout the tool for estimates of ROI, Net Present values (NPV), Internal Rate of Return (IRR) and benefits graphs

21

A Step 1c – The Profile – Preliminary capabilities assessment

Using simple scores or investments and importance, we quickly build a capability as-is and to-be BI Capability Profile that we guide us in the BI ROI benefits case

Having a basic understanding of the current environment will help us focus our BI initiatives later.

22

A Step 1d – The Profile – Preliminary BI maturity assessment• Using basic scores and answering a few questions, the maturity profile is generated

Building, or enhancing, systems in a mature BI organization is substantially less costly than in emerging organizations.

23

A Step 1e – The Profile – Results

• The profile generates a preliminary cost per user, a cost for the organization and the overall benefits.

• This will automatically be updated by the next steps where we refine hardware, software and implementation costs

• Throughout the tool, the system updates key numbers from other Excel tabs and change graphs accordingly

KPIs per users are key to benchmarking

24

A Step 2a – BI Initiative and Application Selection• The next major step is to determine the BI applications in scope for our initiative.

• Business cases and BI strategies should not be focused on a single project, but instead look at major initiatives over 24-48 months.

Strategy is long-term (1-5 years);

Tactics is short-term (1-12 months)

25

A Step 2b – BI Source System Selection

• It is estimated that as much as 40-70% of a BI and DW initiative is spent on Extracting, Transforming & Loading data (ETL).

• In this screen we are adding data sources from a high-level and making an assessment of the type, volume, complexity and integration to the DW.

This helps drive a more accurate cost picture of what is required to integrate into the BI solution using key benchmarks

26

A Step 2c – Enhancements to Data Warehouse

• In this section we are looking at changes required to the data warehouse to accommodate our BI initiatives.

• It also include process improvement, metadata management and enhancements required to data marts and data structures.

The tool also has help and explanations included in each tab to guide a users to the tasks required in these input sheets.

27

A Step 2d – Enhancements to BI and Analytics

• In the last initiatives input tab, we enter the changes required to BI and Analytics. This include: Reporting Scorecard/Dashbaords Analytical Tools Workflow and

collaboration

We are now ready to view the output from the exercise so-far

28

A Step 2e – BI Initiatives Profile• The system then generates a BI capability

profile of what the current state of the BI environment is and what capabilities the initiative will focus on.

These profiles can help communicate with the sponsor the focus of your initiative

29

A Step 3 – BI Maturity Assessment

By filling out a survey based on

objective questions, a BI maturity profile

is automatically generated.

30

A Step 4 – BI Initiative Labor Costs

• Default values workload by week are selected based on scope, scale and complexity of initiatives. However, these can be changed by planners.

• Detailed Graphs are also created based on labor estimates.

31

A Step 4 –Software and Hardware Costs for BI

• By filling out a detailed guided spreadsheet with cost numbers for storage, application servers, database, software, utilities, support, desktop software, training and more, these graphs are generated for user costs.

32

A Step 5a – BI Initiative Benefits Input - Revenue

• A key to justifying BI spending is to identify tangible and credible revenues generated by the investments.

• Work with your finance and sales groups to get numbers you can justify to your management team.

33

A Step 5a – BI Initiative Cost Savings and Productivity

• In addition to revenues, benefits can also be expressed as: Cost avoidance (easy to quantify) Cost savings (harder to quantify) Increased productivity (hard to quantify)

also known as new capabilities ROI

34

A Step 6 – Final Step – ROI Analysis and Report

• The last step is to review the final report. Key measures required in a business case include: ROI NPV Bayback period IRR Costs Benefits Total Cost of

Ownership (TCO)

35

Calculating ROI with a Spreadsheet Tool

36




37

Benchmark Your Organization Against Eight Companies

Industry Banking Manufacturing

Higher Education Energy Automobile Airline Retail Manufacturi

ngNumber of Employees 50,000 5,000 1,200 20,000 60,000 8,000 22,000 81,000 Percent knowledge workers 35% 10% 50% 30% 11% 12% 8% 6%# of knowledge workers 17,500 500 600 6,000 6,600 960 1,760 4,860 Average salary per knowledge worker $64,300 $42,160 $88,000 $63,000 $68,000 $38,000 $41,000 $71,200 Estimated Hourly saved weekly by BI per knowledge worker

2.5 1.0 3.0 1.3 1.3 1.5 2.0 1.0

Annual Cost Savings $82,181,180 $615,820 $4,627,416 $14,355,506 $17,044,315 $1,598,562 $4,216,090 $10,108,800 BI Costs per knowledge worker $4,271 $908 $2,830 $1,650 $1,340 $1,250 $1,800 $1,624 Net Benefits $7,438,680 $161,820 $2,929,416 $4,455,506 $8,200,315 $398,562 $1,048,090 $2,216,160 ROI on Saved labor Hours 10% 36% 173% 45% 93% 33% 33% 28%

• In this example we are looking at key benchmarks from eight companies that are using Business Intelligence and Analytics instead of a traditional list reporting and Excel

• The organizations range from 1,200 employees to over 80,000

38

Savings for Knowledge Worker by using BI

Industry Banking Manufacturing

Higher Education Energy Automobile Airline Retail Manufacturi

ngNumber of Employees 50,000 5,000 1,200 20,000 60,000 8,000 22,000 81,000 Percent knowledge workers 35% 10% 50% 30% 11% 12% 8% 6%# of knowledge workers 17,500 500 600 6,000 6,600 960 1,760 4,860 Average salary per knowledge worker $64,300 $42,160 $88,000 $63,000 $68,000 $38,000 $41,000 $71,200 Estimated Hourly saved weekly by BI per knowledge worker

2.5 1.0 3.0 1.3 1.3 1.5 2.0 1.0

Annual Cost Savings $82,181,180 $615,820 $4,627,416 $14,355,506 $17,044,315 $1,598,562 $4,216,090 $10,108,800 BI Costs per knowledge worker $4,271 $908 $2,830 $1,650 $1,340 $1,250 $1,800 $1,624 Net Benefits $7,438,680 $161,820 $2,929,416 $4,455,506 $8,200,315 $398,562 $1,048,090 $2,216,160 ROI on Saved labor Hours 10% 36% 173% 45% 93% 33% 33% 28%

While the hours saved per knowledge worker ranged from 1.0 to 3.0 hours in our sample, best of class knowledge workers spend 1.3 fewer hours per week looking for pertinent information compared to knowledge workers in all other organizations

Source: The Aberdeen Group, Aug. 2012

39

KPI — BI Cost Per Knowledge Worker

• This included helpdesk, training, desktop licensing, software maintenance, change requests, print costs, and estimated additional developer support

• It did not include hardware, new development, data center operations, backup, recovery, network costs, and upgrades to software components by IT staff

From a Total Cost of Ownership, you should also include the cost of executive and casual users (tends to be lower per user but you will have more casual

users). You can find the number of users in your SAP EarlyWatch report.

• A key measure of costs of ownership is the cost per knowledge worker. In our sample, that ranged from $4,271 to $908 per employee.

40

ROI for BI Investment by Knowledge Worker Labor Savings

• This ROI is for Knowledge workers alone and does not include additional benefits from better access to cleaned data, ability to do collaborative decision making, or more timely decisions and better agility of the organization

Plan for a comprehensive business case for SAP BI, but labor hours are the easiest to quantify

• Using the expected ratios of the knowledge workers relative to other workers, as well as their average salaries, labor savings per week, and the overall cost per knowledge worker, we found an overall ROI

41

KPI Benchmarks of ROI by BI Platform Type• ROI of BI depends largely on the number of BI systems and data warehouses

you maintain• “Best-of-breed” BI and data warehouses use whatever tool is best in its class.

This includes ETL, reporting, management, database, scheduling, admin, etc.

Using a single BI and DW platform instead of Federated Data Warehouses can significantly increase the ROI of BI

Source: SAP AG, Enterprise Business Intelligence ROI white paper

• Integrated BI and DW systems use a single platform

• Integrated BI has significantly less costs and higher long-term ROI (on average)

42

The Various Costs of BI by Platform Type

• Direct Costs Acquisition costs Infrastructure costs Implementation and

Deployment costs Support and Lifetime

costs Uptime, Reliability,

and Performance costs

• Indirect Costs Shadow IT costs Compatibility costs Lost Opportunity

costs

Improve ROI:

The trend for most organizations is to simplify the BI landscape, reduce the number of tools,

and go to a single DW and BI platform

Source: SAP AG

Type of Cost Best of Breed (various tools) Single-Architecture (SAP BI)

Acquisition CostsInitial outlays may be lower, can rise significantly over time

Initial outlays can be higher, more controllable

Infrastructure CostsIncremental Marginal

Implementation and Deployment Costs

Considerable, especially integration efforts and patchwork BI over time

Significant, but controllable and predictable

Support and Lifetime CostsUneven releases of upgrades multiplies support costs

Upgrades in sync with broader systems, though license fees and maintenance still significantUptime, Reliability, and

Performance CostsComplex to diagnose and resolve due to the number of parts

Single point of contact and responsibility

Shadow ITCompletely dependent on the quality of the effort

Consistent UI and integration with operational process will tend to reduce it over time

Compatibility CostsPotentially very high Likely very low

Lost Opportunity CostsLikely due to process Somewhat less likely, but the same

forces are in play

43

A Health Check: Is Anyone Using your System?

Named Users: 14,503Real Users: 7,509 (52%)Active Users: 1,309 (9%)

User trend reports are found in the SAP EarlyWatch report – It is a Health Check to see if your initiative is failing or succeeding

Source: SAP AG

Named Users: 67,534Real Users: 50,912 (75%)Active Users: 6,279 (9%)

Named Users: 1,672Real Users: 783 (47%)Active Users: 191 (11%)

Named Users: 2,330Real Users: 762 (33%)Active Users: 129 ( 6%)

Named Users: 549Real Users: 127 (23%)Active Users: 24 ( 4%)

Real Examples

44




45

BI Project Durations for ROI Planning • While there are many considerations for various BI

projects including skillsets, budget constraints, available resources, scope and number users, there are some basic rule-of-thumbs you can use in your ROI calculations

Each BI project requires its own milestone plan and detailed project plan. However, for high-level budgeting these

durations may help your business case.

Project Type Project Scope Fast Normal SlowSmall Project (500Gb) 2-4 weeks 4-12 weeks 12+ weeks

Medium Project (1-3TB) 4-8 weeks 8-16 weeks 16+ weeks

Large Project (3-100 TB) 8-16 weeks 16-20 weeks 20+ weeks

Small system/few users 4-6 weeks 6-10 weeks 10+ weeks

Medium system/ 300+ users 6-10 weeks 10-16 weeks 16+ weeks

Large system incl. other areas 10-16 weeks 16-20 weeks 20+ weeks

HANA Implementation

BW Upgrade

46

BI Project Durations for ROI Planning (cont.) • For reporting and analytics, the data scope and the

availability of existing skilled staff and existing infrastructure will drive much of the timelines

Each BI project requires its own milestone plan and detailed project plan. However, for high-level budgeting these

durations may help your business case.

Project Type Project Scope Fast Normal Slow2-4 dashbaords existing data 2-3 weeks 3-8 weeks 8+ weeks

2-4 dashbaords new data 3-8 weeks 8-12 weeks 12+ weeks

8-10 dashboards existing/new data 10-12 weeks 12-20 weeks 20+ weeks

5-8 reports existing data 1-2 weeks 2-4 weeks 4+ weeks

5-8 reports new data 2-6 weeks 6-8 weeks 8+ weeks

10-20 reports existing/new data 10-12 weeks 12-20 weeks 20+ weeks

2-3 Infocubes, ETL and DSOs 4-6 weeks 6-10 weeks 10+ weeks



Dashbaord Project

WebI Project

BW Project

47




48

Where to Find More Information

• Asim Abdel Rahman El Sheikh and Mouhib Alnoukari, Business Intelligence and Agile Methodologies for Knowledge-Based Organizations: Cross-Disciplinary Applications, IGI Global; 1 edition (September 30, 2011)

• Thomas H. Davenport and Jeanne G Harris, Competing on Analytics: The New Science of Winning, Harvard Business School Press; 1 edition (March 6, 2007)

• http://iteconcorp.com/ROICalc.html Computing The ROI for IT Projects and Other Investments

• http://hallcr.com/BI.aspx Another calculator for ROI calculator on BI projects

http://iteconcorp.com/ROICalc.html

http://hallcr.com/BI.aspx

http://hallcr.com/BI.aspx

49

7 Key Points to Take Home

• ROIs on BI projects must be based on tangible and measurable benefits

• ROI should be calculated at a program level and tracked over time• There are many tools available to help you do this – Don’t reinvent

the wheel• Benchmark your BI initiatives in terms of project duration, cost per

employee, labor savings, ROI, payback periods, and BI platform type• Be prepared to back up your business case with hard facts from the

business users and avoid a technology focus• Replacement projects and cost avoidance efforts are easiest to

quantify• Some projects are done out of operational necessities and ROI may

not be required (audits, SOX, compliance, financial SEC reporting, etc).

50

Your Turn!

How to contact me:Dr. Bjarne Berg

[email protected]

Please remember to complete your session evaluation

51

Disclaimer

SAP, R/3, mySAP, mySAP.com, SAP NetWeaver®, Duet®, PartnerEdge, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.

Documents

Benchmarking BI: How to Identify Critical Success Factors and Calculate ROI of Your BI Initiatives