27
Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company trusted by enterprises seeking an analytic advantage.

Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

Analytics 2014Industry Trends Survey

June 2014

Research conducted and written by:

Lavastorm Analytics, the agile data management and analytics company trusted by enterprises seeking an analytic advantage.

Page 2: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

2

Table of ContentsExecutive Summary

Key Findings

About Lavastorm

Appendix I: Methodology

Appendix II: Description of Survey Respondents

Page 3: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

3

Executive Summary

“Analytics 2014” is Lavastorm Analytics’ second in a series of annual surveys that explores how analytic technologies are being applied to solve business challenges. This year’s survey has been updated to reflect a growing focus on the analysis of big data.

This survey report is based on the survey responses Lavastorm Analytics received from 495 active participants in the analytics community, including business analysts, technologists, data analytics professionals, managers, and C-level executives. Survey participants came from a broad variety of industries. The top findings were:

Analytics is Growing Despite Inefficient Analytic Supply Chains

The majority of survey respondents (64.4%) indicated that their company is increasing investment in 2014. Despite this, organizations are still finding it difficult to progress an analytic insight to an implemented business improvement. This is likely hindered by the complexity of the analytic supply chain – characterized by the group of people that work on either data or analysis starting with data acquisition and progressing to insight and resulting business action. The multiple handoffs and specialists involved in the analytic supply chain slows progress.

In addition, many businesses use multiple analytic tools, demonstrating that few tools are perfectly suited for complex and multifaceted jobs. Both of these trends are driving interest in self-service or data discovery tools, which are simpler to use and bypass a good deal of the traditional analytic supply chain when business users have new questions to answer.

The World is Dividing into Big Data “Haves” and “Have Nots”

Big data is maturing, but the majority of companies are still sitting on the sidelines watching, planning and evaluating. Only 12.6% respondents in our survey report that their company has completed several big data projects that are now in production.

Those respondents with big data projects in production appear to be trying to widen the analytic gap between themselves and their competitors. They are more likely to be increasing their investment significantly (38.7%) compared to companies that are not yet working with big data (21.8%). Interestingly, the number one concern for people in organizations that are just experimenting or planning for big data is the shortage of analytic professionals, which could indicate that organizations yet to take the big data plunge may be held back because of the lack of big data skills available to them.

Page 4: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

4

Big Data Is Currently the Domain of the Data Scientist, But Needs More Business Involvement for Long-term Success

Our survey indicated that analysts (business or data analysts) are in the dark when it came to big data technologies and the work that their company was doing to leverage those sources. Data Scientists, who are generally in R&D or IT groups, are much more involved. The best business results will only be obtained, however, when the business is heavily involved in the planning and analysis.

Data Quality Concerns On the Rise

Many more people are involved with or concerned with data quality, likely due to increased use of new data sources, especially big data sources. In this year’s survey 48 percent of respondents are working on data quality, a significant increase over the 27 percent that reported working on data quality last year.

Page 5: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

5

Key FindingsAnalytics is Growing Despite Inefficient Analytic Supply Chains

There is a general consensus that investment in analytics is increasing. This is consistent with other research that indicates that analytics and business intelligence will remain a top priority of CIOs in 2014 and several years to come. As you can see in Figure 1, 64.4% of respondents indicate their company is investing more in 2014, with 20.5% indicating their company is increasing their investment significantly.

Figure 1: How would you describe your company’s investments in analytics (tools, people, data sources, etc.) in 2014?

Increasing Investment Signi�cantly 20.5%

Increasing Investment Moderately 43.9%

Staying the Same 23.4%

Decreasing Investment Moderately 2.6%

Decreasing Investment Signi�cantly 1.0%

Not Sure 8.5%

Despite the increased investment that is expected, businesses are struggling to turn analytic insights into business improvements. As Figure 2 shows, 20.5% of respondents indicated that “turning insights to action” is their biggest challenge, while 13.8% indicate that “building trust in insights” is the major challenge.

So investment is increasing even in the face of that difficulty. These challenges show that analytics is a complex operation having as much to do with human interaction and communication capabilities as technology.

Page 6: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

6

Figure 2: What is your biggest challenge in analytics today?

Turning Insights to Action 20.5%Requirements Gathering 5.1%

Publishing Results 1.8%Other 4.5%

Gleaning Insights from the Data 16.3%

Building Trust in Insights13.8%

Manipulating / Integrating Data15.2%

Resources 11.4%

Access to the Data 11.4%

Difficulty progressing insights to business improvements is likely hindered by the complexity of the analytic supply chain, or the group of people that work on either data or analysis all the way from data acquisition through insight to business action. Analytic processes typically require multiple people with specialized roles (data scientist, data analyst, business analyst, etc.) to collaborate if they are to complete an analytic task.

For example, people in our survey who identified their role as “data scientist” more often resided in a research department (27.4%) than in any other department (Figure 3). They need to work with business leaders, therefore, to get their discoveries out of their department so that it can be acted upon. That’s one reason why 25.8% of data scientists said “building trust in insights” and another 22.6% said “turning insights to action” were their major challenges (see Figure 4). Likewise, more analysts (those people in our survey who identified their role as a “business analyst” or “data analyst”) reside in IT departments (16%) and again need to work with business leaders to enable decisions and action. (see Figure 5.)

Page 7: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

7

Figure 3: For data scientists only, “what department do you reside in?”

Figure 4:

Services 8.1%

Executive/Sr. Management 4.8%

Engineering 9.7%

Other 17.7% Research 27.4%

IT 11.3%

Finance 4.8%

Marketing 9.7%

Operations 6.5%

For data scientists only, “what is your biggest challenge in analytics today?”

Resources 9.7%

Access to the Data 14.5%

Requirements Gathering 3.2%

Gleaning Insights from the Data 12.9%

Manipulating /Integrating Data 11.3%

Building Trust in Insights 25.8%

Turning Insights to Action 22.6%

Figure 5: For analysts only, “What department do you reside in?”

Sales 2.8%

Services 12.3%

Executive/Sr. Management 2.8%

Engineering 2.8%

Research 13.2%

Finance 5.7%

IT 16.0%

Marketing 15.1%

Operations 10.4%

Other (please specify) 18.9%

Page 8: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

8

Contributing to analytic inefficiencies is the respondent reliance on multiple analytic tools, sometimes within the same department. As Figure 6 shows, people use a wide array of tools to analyze data. According to the survey, respondents use more than 4 analytics tools each. As Figure 7 shows, the tools most often used by analysts (business and data analysts) are general purpose tools, including Microsoft Excel®, SQL and Microsoft Access®.

This is driven by several factors, including:

•• Respondents have complex, multi-faceted roles to play and, therefore, that they must use specialized tools for different aspects of their job;

•• Respondents don’t have the perfect tool for the job and must use several tools as a compromise solution.

For example, analysts (business and data analysts) indicated that one of their major challenges is “manipulating/integrating data” (Figure 8). This is likely because they are dependent on general purpose analytic tools (Microsoft Excel, SQL, and Microsoft Access) which are not designed for the complex data environments we have today.

Page 9: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

9

Figure 6: “What analytic tools are you currently using? Check all that apply.”

0 10 20 30 40 50 60 70

Microsoft Excel

SQL

R

Microsoft Access

Tableau

SAS

SPSS

Python

SAP BusinessObjects

Cognos

Microsoft BI

Toad

QlikView

Matlab

Lavastorm Analytics

Teradata

Oracle OBIEE

Rapid Miner

Hive

TIBCO Spot�re

MicroStrategy

Pentaho

StatSoft/STATISTICA

SAP NetWeaver BW

ACL

Alteryx

Information Builders

Actuate (Birt)

Logix

Panorama Necto

Other (please specify)

72.4%

45.5%

36.6%

32.7%

24.8%

24.2%

17.5%

17.1%

15.2%

14.8%

12.4%

11.6%

9.3%

9.3%

9.3%

8.7%

8.3%

7.9%

6.5%

5.9%

5.1%

5.1%

3.7%

3.3%

3.0%

3.0%

2.6%

1.8%

0.6%

0.2%

0.0%

Page 10: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

10

Figure 7: For analysts (business and data analysts only), “what analytic tools are you currently using? Check all that apply.”

Microsoft Excel

SQL

Microsoft Access

R

Tableau

SAS

SPSS

Python

Cognos

SAP BusinessObjects

Microsoft BI

Toad

QlikView

Other (please specify)

Lavastorm Analytics

MicroStrategy

Rapid Miner

Teradata

TIBCO Spot�re

Oracle OBIEE

ACL

Hive

Matlab

StatSoft/STATISTICA

Alteryx

Information Builders

SAP NetWeaver BW

Pentaho

Actuate (Birt)

Logix

Panorama Necto

53.8%

78.3%

42.5%

39.6%

33.0%

26.4%

20.8%

19.8%

16.0%

13.2%

13.2%

13.2%

12.3%

12.3%

10.4%

6.6%

6.6%

6.6%

6.6%

5.7%

4.7%

4.7%

2.8%

2.8%

1.9%

1.9%

1.9%

0.9%

0.0%

0.0%

0.0%

0 10 20 30 40 50 60 70 80

Page 11: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

11

Figure 8: For analysts (business and data analysts) only, “what is your biggest challenge in analytics today?”

Resources 7.5%

Access to the Data 11.3%

Building Trust in Insights 10.4%

Turning Insights to Action 22.6%Requirements Gathering 5.7%

Manipulating/Integrating Data 19.8%

Gleaning Insights From the Data 16.0%

Publishing Results 0.9%

Other (Please Specify) 5.7%

The organizational complexity is a major reason why organizations have shifted and continue to shift to a self-service approach and why data discovery requirements are expected to drive the majority of the new license spend in BI in years to come. The self-service emergence started with business groups that wanted to take matters into their own hands and cut through a great deal of the organizational issues, such as IT’s minimal bandwidth to respond to BI requests, that were preventing them from answering key business questions. We expect the self-service trend to continue with business users taking on more complex analytic challenges themselves, such as integrating more diverse data sources, or taking them on in a collaborative effort with IT. Figure 9 shows that 30 percent of respondents in our survey were investing in self-service analytic tools for business users.

Overcoming these organizational challenges should improve analytic results and help companies put analytics into action. Companies that take a top-down data or analytic view of their operations and design a more streamlined analytic process should have a more responsive organization and better insights. For example, tools that are designed for specific roles could help simplify the tool chest for analysts and tools that enable collaboration could bridge the organizational gaps that slow down action.

Page 12: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

12

Figure 9: For people whose companies are investing in analytics in 2014, “what area is your company investing in 2014? Check all that apply.”

0 10 20 30 40 50 60

59.0%Predictive Analytics

Dashboards

Big Data

Reporting

43.8%

43.5%

37.9%

35.3%

31.2%

30.3%

30.0%

28.7%

25.9%

23.0%

20.5%

19.2%

15.1%

12.9%

12.0%

7.3%

5.4%

2.8%

2.5%

Data Management (Including Integration,Quality, ETL, and Enrichment)

Data Exploration and Discovery

Interactive Visualization Tools

Self-Service Analytic Tools for Business Users

Performance Analytics

Text Analytics

Web Analytics

Databases: Relational

Process Analytics

Databases: Non-relational (Including Columnar)

Location Analytics

Streaming Analytics

Olap

Search Interfaces

None

Other (Please Specify)

With Big Data’s Maturing, the World is Dividing into Big Data “Haves” and “Have Nots”

By all accounts, big data is maturing, but the majority of companies are still sitting on the side lines watching, planning and evaluating. Our survey indicates that maturity is still low. As you can see in Figure 10, only 12.6% report that their company has completed several big data projects that are now in production. Most (54.1%) are in the experimenting or learning/planning phases, while 20.9% are not working with big data at all.

Page 13: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

13

Figure 10: “Describe the maturity of your company’s current efforts to use big data tools such as Hadoop or to analyze big data sources, such as unstructured data sources and non- relational databases, such as MongoDB.”

Learning or Planning – We have not started any projects to work with big data sources or tools to date, but are tracking the evolution

and application of the technologies31.1%

Experimenting - We are experimenting with big data sources and tools, but those projects are not in production and are not

part of an ongoing business process23.0%

Not Participating - We don’t have a current interest in using big

data sources or tools.20.9%

In Production - We have completed several projects using big data sources

and tools and those projects are currently in production and are part of an ongoing

business process12.6%

Don’t Know12.4%

More importantly, those that have big data projects in production appear to be trying to widen the analytic gap between themselves and their competitors. They are more likely to be increasing their investment significantly (38.7%) than companies that are not yet working with big data (21.8%). See Figure 11.

Not surprisingly, companies that have big data projects in production are investing most often in additional big data initiatives (66.1%) compared with companies that are less mature when it comes to big data (45.5% investing in big data in 2014). See Figure 12.

Page 14: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

14

Figure 11: For companies with big data projects “In Production” vs companies that are not, “how would you describe your company’s investments in analytics (tools, people, data sources, etc.) in 2014?”

0

10

20

30

40

50

Increasing Investment Signi�cantly

Increasing Investment Moderately

Staying the Same

Decreasing Investment Moderately

Decreasing Investment Signi�cantly

Not Sure

Those in Production – Response Percent

Those in Planning or Experimenting – Response Percent

38.7% 38.7%

48.9%

21.8%

14.5%

19.5%

1.6% 3.0% 1.6% 1.5%4.8% 5.3%

Figure 12: For companies with Big Data projects in production vs companies that are not, “what area is your company investing in 2014? Check all that apply.”

0 10 20 30 40 50 60 70

Those in Planning or Experimenting -Response Percent

Those in Production -Response Percent

Search Interfaces

Big Data

Predictive Analytics

Dashboards

Reporting

Interactive Visualization Tools

Text AnalyticsData Management (including Integration,

Quality, ETL, and Enrichment)Web Analytics

Performance AnalyticsSelf-Service Analytic Tools for Business

UsersData exploration and Discovery

Databases: Non-Relational (including Columnar)

Databases: Relational

Process Analytics

Streaming Analytics

Location Analytics

OLAP

Other (please specify)

None

Page 15: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

15

In addition, a full 41% of the respondents that say that they are in the Planning or Experimenting phase with big data have not yet acquired any specific big data tools for their initiatives (Figure 13).

Interestingly, the people in organizations that are just experimenting or planning for big data believe the shortage of analytic professionals will have the biggest impact on the analytics industry in 2014 (Figure 14). Meanwhile those respondents from companies with big data deployments did not expect as much impact from that trend (Figure 15). Probably this indicates the confidence that those organizations that are ahead of the curve are also positioned to stay there from a skills perspective. While organizations yet to take the big data plunge may be held back because of the lack of big data skills available to them.

Figure 13: For respondents whose company is Planning or Experimenting with big data, “what big data tools do you currently use? Check all that apply.”

0.0%

10.0%

15.0%

25.0%

35.0%

45.0%

5.0%

20.0%

30.0%

40.0%

Hadoop

34.6%

Hive

12.0%

HortonWorks

3.4%

Other (please specify)

6.8%

MongoDB

11.3%

None

41.4%

Don’t Know

15.0%

Page 16: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

16

Figure 14: For respondents whose company is Planning or Experimenting with big data, in 2014, “what one development or trend will occur or progress and have the biggest impact on the analytic community?”

Shifting Responsibility for Analytics from IT to the Business

14.7%

Shortage of Analytic Professionals16.2%

Maturation of Big Data Strategies and Products

12.8%

Greater Adoption of Self-Service and Data Discovery Tools

8.6%

Increased Focus on the Data Scientist Role9.8%

Increased Use of Cloud-Based Analytic Solutions

9.0%

Increased Data Variety (including the use of Unstructured Data or New Data Sources)

7.5%

Increased Data Volumes (including Volume from Machine-Generated Data)

8.6%

Increased use of Mobile Analytic Solutions4.5%

Shortage of Appropriate Analytic Tools and Training6.0%

Other 2.3%

Figure 15: For respondents whose company is in Production with big data, in 2014, “what one development or trend will occur or progress and have the biggest impact on the analytic community?”

Shifting Responsibility for Analytics from IT to the Business

16.1%

Shortage of Analytic Professionals 6.5%

Maturation of Big Data Strategies and Products

16.1%

Greater Adoption of Self-Service and Data Discovery Tools

9.7%

Increased Focus on the Data Scientist Role11.3%

Increased use of Cloud-Based Analytic Solutions11.3%

Increased Data Variety (including the use of Unstructured Data or New Data Sources) 6.5%

Increased Data Volumes (including Volumefrom Machine-Generated Data) 3.2%

Increased use of Mobile Analytic Solutions12.9%

Shortage of Appropriate Analytic Tools and Training1.6% Other 4.8%

Page 17: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

17

Big Data is Currently the Domain of the Data Scientist, but Needs to Involve the Business More for Long-term SuccessOur survey indicated that analysts (business or data analysts) are in the dark when it came to big data technologies and the work that their company was doing to leverage those sources – nearly 73% of analysts did not know what big data tools were being used (Figure 15), or were not currently using big data tools, compared to almost 39 percent of data scientists (Figure 17).

Figure 16: For analysts (business and data analysts) only, “what big data tools do you currently use? Check all that apply.”

0.0%

10.0%

15.0%

25.0%

35.0%

45.0%

50.0%

5.0%

20.0%

30.0%

40.0%

Hadoop

18.9%

Other (please specify)

6.6%

Hive

6.6%

HortonWorks

0.9%

MongoDB

4.7%

None

47.2%

Don’t Know

25.5%

Figure 17: For data scientists only, “what big data tools do you currently use? Check all that apply.”

0.0%

10.0%

15.0%

25.0%

35.0%

45.0%

50.0%

5.0%

20.0%

30.0%

40.0%

Hadoop

46.8%

Hive

24.2%

HortonWorks

1.6%

MongoDB

11.3%

None

33.9%

Don’t Know

4.8%

Other (please specify)

12.9%

Page 18: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

18

The troubling aspect of this is that big data projects may be treated as a science project and not a business-impacting capability. The best business results will be obtained when the business is heavily involved in the planning and analysis. Gartner and other industry analyst firms have indicated that the vast majority (80% by some estimates) of BI/analytic projects fail and the lack of business involvement is a primary culprit. For best results, organizations should learn from history or risk repeating it. This is likely why most data scientists consider their challenges to be turning insights to action.

It’s likely that this reflects the maturity of organizations with big data and that data scientists are involved earlier in the process. Our survey shows that while both data scientists and analysts are primarily involved in analysis, they go about it quite differently. As shown in Figures 17 and 18, data scientists focus much more on building statistical models (88.7% to 46.1%) and on research and development (69.4% to 47.8%).

Figure 18: For data scientists only, “which of the following functions do you personally perform as a regular part of your job? Check all that apply.”

0 20 40 60 80 100

Data Analysis

Statistical Modeling

Research and Development

Import Data from a Data Warehouse (or from IT)

Data Quality Assessment and Improvement

Gather Requirements

Combine Data from a Warehouse with other Data not in the Warehouse

Filter or drill down into Dashboards/Interactive Reports

Publish or otherwise provide Data to other Departments

Load Data into the Data Warehouse

Other (please specify)

91.9%

88.7%

69.4%

59.7%

59.7%

56.5%

50.0%

41.9%

40.3%

25.8%

8.1%

Page 19: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

19

Figure 19: For analysts (business and data analysts) only, “which of the following functions do you personally perform as a regular part of your job? Check all that apply.”

0 20 40 60 80

Data analysis

Gather requirements

Data quality assessment and improvement

Publish or otherwise provide data to other departments

Research and development

Statistical modeling

Import data from a data warehouse (or from IT)

Filter or drill down into dashboards/interactive reports

Combine data from a warehouse with other data not in the warehouse

Load data into the data warehouse

Other

In addition, it likely reflects a requirement for greater technical skills to use big data tools. Our survey (see Figures 20 and 21) indicates that data scientists use statistical packages and programming languages, whereas analysts use mostly Microsoft Excel (78%), SQL (54%), and Microsoft Access (42%). More technical tools would lead organizations to call on data scientists for assistance.

Page 20: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

20

Figure 20: For analysts (business and data analysts) only, “what analytic tools are you currently using?” Check all that apply.

Microsoft Excel

SQL

Microsoft Access

R

Tableau

SAS

SPSS

Python

Cognos

SAP BusinessObjects

Microsoft BI

Toad

QlikView

Lavastorm Analytics

MicroStrategy

Rapid Miner

Teradata

TIBCO Spot�re

Oracle OBIEE

ACL

Hive

Matlab

StatSoft/STATISTICA

Alteryx

Information Builders

SAP NetWeaver BW

Pentaho

Actuate (Birt)

Logix

Panorama Necto

Other (please specify)

0 10 20 30 40 50 60 70 80

78.3%

53.8%

42.5%

39.6%

33.0%

26.4%

20.8%

19.8%

16.0%

13.2%

13.2%

13.2%

12.3%

10.4%

6.6%

6.6%

6.6%

6.6%

5.7%

4.7%

4.7%

2.8%

2.8%

1.9%

1.9%

1.9%

0.9%

0.0%

0.0%

0.0%

12.3%

Page 21: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

21

Figure 21: For data scientists only, what analytic tools are you currently using? Check all that apply.

0 10 20 30 40 50 60 70 80

R

Microsoft Excel

SQL

Python

SAS

Tableau

SPSS

Matlab

Hive

Microsoft Access

Rapid Minder

Toad

TIBCO Spot�re

StatSoft/STATISTICA

Microsoft BI

Teradata

Oracle OBIEE

QlikView

Pentaho

SAP BusinessObjects

SAP NetWeaver BW

MicroStrategy

Information Builders

Alteryx

ACL

Panorama Netco

Logix

Lavastorm Analytics

Actuate (Birt)

Other

71.0%

64.5%

41.9%

40.3%

37.1%

33.9%

29.0%

25.8%

25.8%

17.7%

16.1%

12.9%

9.7%

6.5%

6.5%

6.5%

4.8%

4.8%

3.2%

3.2%

3.2%

1.6%

1.6%

1.6%

1.6%

1.6%

0.0%

0.0%

0.0%

0.0%

Page 22: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

22

Data Quality Concerns On the Rise

The growing use of new data sources, especially big data sources, is increasing the variability and quality issues that analytic value chains are facing and forcing companies to put more resources towards data quality improvement efforts. As shown in Figure 22, 48% of respondents are working toward data quality, a significant increase over the 27% that reported working on data quality last year.

Figure 22: “Which of the following functions do you personally perform as a regular part of your job? Check all that apply.”

0 20 40 60 80

Data analysis

Gather requirements

Data quality assessment and improvement

Publish or otherwise provide data to other departments

Research and development

Statistical modeling

Import data from a data warehouse (or from IT)Filter or drill down into

dashboards/interactive reportsCombine data from a warehouse with

other data not in the warehouse

Load data into the data warehouse

Other

As companies are trying to integrate these disparate sources, it is forcing them to tackle the data quality problem head on. Issues such as mismatched data, varying formats and incomplete data are more critical given the greater variety and differences between the data sources and the fact that some of this data is being analyzed for the first time by many organizations. This is why 35 percent of survey respondents who said they were increasing analytic investments in 2014 said their company is also investing in data management (including integration, quality, ETL and enrichment). See Figure 23.

Page 23: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

23

Figure 23: For only those respondents whose company is increasing their investment in 2014, “what area is your company investing in 2014. Check all that apply.”

Predictive analytics

Dashboards

Big data

Reporting

Data Management (Including Integration, Quality, ETL, and Enrichment)

Data Exploration and Discovery

Interactive Visualization Tools

Self-Service Analytic Tools for Business Users

Performance Analytics

Text Analytics

Web Analytics

Databases: Relational

Process Analytics

Databases: Non-Relational(Including Columnar)

Location Analytics

Streaming Analytics

OLAP

Search Interfaces

None

Other (Please Specify)

59.0%

43.8%

43.5%

37.9%

35.3%

31.2%

30.3%

30.0%

28.7%

25.9%

23.0%

20.5%

19.2%

15.1%

12.9%

12.0%

7.3%

5.4%

2.8%

2.5%

0 10 20 30 40 50 60

Page 24: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

24

About LavastormLavastorm is the agile data management and analytics company trusted by enterprises seeking an analytic advantage. The company’s data discovery platform provides IT with control over data governance, while empowering business professionals and analysts with the fastest, most accurate way to discover and transform insights into business improvements. Lavastorm’s solutions have identified business improvements worth billions of dollars for some of the largest corporations in the world. For more information, please visit: www.lavastorm.com.

For more information on Lavastorm Analytics or to download a desktop edition of the Lavastorm Analytics Engine, our data analytics software for business analysts, please visit www.lavastorm.com or www.lavastorm.com/resources/software-downloads-trials, respectively.

Appendix I: MethodologyIn order to identify the trends in the market, we conducted our research within major analytic communities, including LinkedIn’s Lavastorm Analytics Community Group, Data Science Central and KD Nuggets. These communities have a global reach of over 75,000 analytic professionals who play a variety of roles in analytics today and represent all of the major industries.

This survey was conducted online using an electronic survey tool. A total of 495 business analysts, technologists, data analytics professionals, managers and C-level professionals were polled across a broad variety of industries—including financial services, telecommunications, healthcare and software & internet. After the survey data was collected, the data was analyzed to identify differences between segments of the population. Every survey participant was encouraged to answer every question.

Page 25: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

25

Appendix II: Description of Survey RespondentsThe following charts describe the backgrounds of those professionals who completed the survey by role, department and industry.

How would you classify your role? 492 Responses

Data Analyst 14.6%

Other 8.3%

Manager/Director 14.4%BI Professional 5.3%

VP/Director 6.1%Data Scientist 12.6%

Business Analyst 6.9%

Consultant 11.4%

Marketing Analyst 3.9%C-level/Executive 2.2%

Solution Architect 2.0%

Software Developer 2.0%Revenue Assurance Manager 2.0%

What department do you reside in?492 Responses

IT 20.1%

Other 16.1%

Marketing 13.0%

Research 12.6%

Operations 8.3%Executive/Sr. Management 7.7%

Services 7.3%

Finance 7.3%

Engineering 4.7%

Sales 1.6%Billing 1.2%

Other includes: Analytics (2.8%), Risk (1%) and Audit (.8%)

Page 26: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

26

What is your company’s primary industry?492 Responses

Telecom15.9%

Other16.1%

Services13.4%Retail 3.3%

Computer Software11.4%

Healthcare8.9%

Finance 8.5%

Education7.3%

Manufacturing 4.5%

Government 3.5%

Insurance 2.4%Entergy 2.0%

Transportation 1.4%Utilities 1.4%

Other includes: Consulting (2.4%) and Marketing (1%).

Which of the following functions do you personally perform as a regular part of your job? Check all that apply492 Responses

0 20 40 60 80

Data analysis

Gather requirements

Data quality assessment and improvement

Publish or otherwise provide data to other departments

Research and development

Statistical modeling

Import data from a data warehouse (or from IT)Filter or drill down into

dashboards/interactive reportsCombine data from a warehouse with

other data not in the warehouse

Load data into the data warehouse

Other

Page 27: Analytics 2014 - USI...Analytics 2014 Industry Trends Survey June 2014 Research conducted and written by: Lavastorm Analytics, the agile data management and analytics company3 Executive

www.lavastorm.com

Lavastorm Analytics, 2014. All Rights Reserved.