IDC 1524

I D C A N A L Y S T C O N N E C T I O N

Carl Olofson

Research Vice President, Application Development and Deployment

Key Considera t ions for Opt imiz ing Rea l -T ime Analyt ics

June 2013

Enterprises are encountering a seismic shift in database technology and capability. This shift is expressed in a variety of ways, from columnar data organization to in-memory database software. There are in-memory real-time analytics options that are platform based and involve a transformation of the datacenter, but not every company is in a position to do this and can afford the expense entailed. This technology is rapidly evolving, but rather than wait for the ultimate solution and the wholesale datacenter transformation involved, companies can take advantage of this technology now by thoughtfully applying it to the current IT landscape. Small and medium-sized enterprises are dependent on packaged applications and existing reference architectures, so any real-time analytics solution needs to work with those architectures, not replace them.

The following questions were posed by MemSQL to Carl Olofson, research vice president for IDC's

Application Development and Deployment service, on behalf of MemSQL's customers.

Q. What are the benefits to adding a scalable in-memory real-time analytics platform to an

enterprise's data management and business analytics portfolio?

A. Classic decision support and analytics technology is based on data warehouses and other

databases that collect data at a scheduled time. However, this process is useful only for

strategic planning and backward-looking assessment of the state of the business. As the

pace of business increases, it's critical to have business intelligence available at the actual

point of decision to enable informed action. To do this, one needs to collect data from all the

relevant sources in real time and make it available immediately for the benefit of front-line

managers and staff making key decisions. This gives decision makers an informational

context that has been absent in the past.

In general, there are several broad benefits. One benefit is greater operational efficiency

because you can make decisions that avoid waste and reduce costs. Examples include a

trucking company making real-time decisions about how to plan routes to save on fuel, avoid

traffic jams, or pick up late scheduled packages and a retail organization making same-day

decisions about inventory management to ensure that products in demand are available to

the customer. Another benefit is the ability to respond to the needs of the market by acting on

opportunities as they arise. Yet another benefit is in risk mitigation and fraud detection to

reduce legal and financial exposure.

©2013 IDC 2

Q. What kinds of organizations and individuals within those organizations can benefit

from a scalable in-memory real-time analytics platform?

A. It depends on the nature of the organization. The earliest adopters of real-time analytics have

been financial services companies, gaming and mobile application providers, media and

advertising companies, and government and defense agencies. As the technology

progresses and becomes easier to use, additional verticals where information is an element

of ongoing operations, such as retail or healthcare, should derive clear benefit. You're dealing

with the ability to leverage large volumes of data being processed at high velocity, so

companies in these verticals benefit because their people have to make decisions based on

that data as quickly as possible. In retail, the specific beneficiaries would be line-of-business

managers, salespeople, or store managers. In healthcare, it might be doctors. In banking or

financial services, it would likely be portfolio managers or account managers. These are just

a few examples.

Compliance or risk officers in security and risk mitigation responsible for the integrity of

accounts could also benefit by enhancing their ability to detect external attempts to hack into

the system or malicious attacks from within. Conventionally, they could detect breaches after

the fact, but with real-time data, they can catch such breaches as they happen. Also included

are managers involved in fine-tuning marketing campaigns as well as those who oversee the

operational systems involving supply chain, inventory, or logistics. When these teams are

able to utilize real-time analytics to make important business decisions, they are able to get a

more complete picture of how that decision will impact the customer. This process can

provide companies with significant competitive advantages over other players in the market

that don't have a direct pulse on what is actually happening inside or outside their

organizations.

Q. What are the key considerations for companies thinking about real-time analytics

solutions?

A. Memory-optimized systems (sometimes called in-memory systems) where most of the data is

held in memory will provide the best performance. Some people have concerns about this

approach because they have the notion that memory-optimized systems are less stable than

disk-optimized systems. However, that's not the case. These modern systems are designed

to be quite recoverable. They employ a number of techniques for recoverability and

continuous availability, such as redundant data replication, recovery logging, and

snapshotting, and are just as reliable as any disk-optimized system. It's also important that

the system considered support standard methods of accessing data, and of course, the most

common standard is SQL. A real-time analytics solution should be SQL based so that you

can leverage existing skills and tools. Learning a new programming language delays the

benefits of real-time analytics and limits the resource pool as new use cases are identified.

The system needs to be able to work with existing tools and have elastic scalability, mainly

achievable today through horizontal scale-out. If budget is a concern, you want that scale-out

capability to work on commodity hardware so that additional resources can be ramped up

simply by adding servers to the cluster. Ultimately, any system should be nondisruptive to the

overall business. While some large enterprises will decide to rearchitect their entire

environment, most organizations quite sensibly reject massive and risky change. It is also a

good idea to add this capability to an existing system architecture rather than engage in

radical transformation now because datacenter technologies are in a state of tremendous

flux, and it's hard to know exactly what the best total system architecture should be

ultimately.

©2013 IDC 3

Q. What are the challenges driving the adoption of real-time analytics in the enterprise?

A. One challenge is looking at how to source the data you need and make it available.

Companies need to analyze their existing systems to figure out the appropriate data sources

and scheduling. Change can be difficult. Large enterprises with complex data pipelines may

not be sure how to incorporate a real-time analytics solution into that pipeline. They may

consider initially forking data into their real-time solution, in essence letting the real-time

solution augment their existing infrastructure.

Another challenge is the need to rethink ways to manage business processes, making them

more flexible and dynamic. A retailer might decide, for example, to manage inventory and

pricing during the business day in order to respond to demand dynamically rather than wait

until the end of the day and make those decisions for the next day. Or a trucking company

might switch from a fixed schedule of deliveries to a schedule that can be dynamically

adjusted during the day in order to achieve greater operational efficiency and save fuel costs.

Some of those decision points can be quite simple, and others can be quite sophisticated.

Roles within the organization may also be affected. The challenge is to think creatively about

how the business runs and to envision how much more efficient and reliable business

processes can become using this kind of technology.

Q. How are the increasing volumes and velocity of data changing the way enterprises

think about analytic solutions?

A. The classic use of analytic technologies has involved putting data into a data warehouse and

then running reports to assist with long-range decisions. The increase in the volume and

velocity of data that impacts the business means that this is still a necessary but not sufficient

approach. Companies need to be able to keep on top of current changes as the business day

progresses by making real-time intelligence available to either decision makers or systems

that make algorithmic decisions. It's no longer acceptable to base analytic solutions on fixed

operational schedules.

The key differentiators are data volume and velocity. With data warehousing, you load data

on a periodic basis. But when the volume is very large, you can't do that because the amount

of data that accumulates over the load period is so great that it can't be loaded into a data

warehouse in a timely manner. So you need to sift and sort the data, providing a select

subset for the data warehouse but making other data available for real-time decision making.

You could execute those complex processes in Hadoop, but Hadoop is batch oriented, and

batch environments can't really be responsive enough to deal with the velocity. A horizontally

scalable high-speed solution is the only way to address volume by easily and inexpensively

scaling and optimize velocity by capturing large data volumes in real time. An organization

that wants to leverage real-time analytics for the enterprise needs a memory-centric solution

to do this.

A B O U T T H I S A N A L Y S T

Carl Olofson is research vice president, Application Development and Deployment. He performs research and analysis for

IDC's Information Management and Data Integration Software service within the Application Development and Deployment

research group. Mr. Olofson's research involves following sales and technical developments in the information and data

management (IDM) markets, database management systems (DBMS) markets, data movement and replication software,

data management software, and metadata management software.

©2013 IDC 4

A B O U T T H I S P U B L I C A T I O N

This publication was produced by IDC Go-to-Market Services. The opinion, analysis, and research results presented herein

are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor

sponsorship is noted. IDC Go-to-Market Services makes IDC content available in a wide range of formats for distribution by

various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee.

C O P Y R I G H T A N D R E S T R I C T I O N S

Any IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requires

prior written approval from IDC. For permission requests, contact the GMS information line at 508-988-7610 or gms@idc.com.

Translation and/or localization of this document requires an additional license from IDC.

For more information on IDC, visit www.idc.com. For more information on IDC GMS, visit www.idc.com/gms.

Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com