IBM Software

The rise of machine data: Are you prepared?Take action in the Internet of Things through data-in-motion analytics

The rise of machine data: Are you prepared?

1 2 3 4 6 5The Internet of Things

What to do about the rise of the machine data: Analyze it

Capitalize on real-time analytics of data in motion

The connected car: Innovating for sustainability

Uncovering actionable insights through the IBM big data platform

Resources

imageblock_OL03.psd

The rise of machine data: Are you prepared?

3

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

The Internet of Things

Fueling the big data phenomenonMillions of people rely on the Internet to socialize, collaborate and conduct business worldwide. But these numbers are eclipsed by the multitudes of Internet-connected objects—more than 10 billion wirelessly connected devices today, with over 30 billion devices expected by 2020.1

This network of linked devices has the potential to transform the world, according to the Internet of Things (IoT) concept. Coined at a time when email was cutting edge and many people were still figuring out how to use the Internet (famously pegged by the “On the Internet, nobody knows you’re a dog” cartoon in the New Yorker2), the IoT

was conceived as a network of objects equipped with radio-frequency identification (RFID) chips and similar technologies so that the objects could communicate and interact with each another.

At first, proponents of IoT focused on its benefits to the retail sphere: how it could help improve inventory management, reduce waste and keep up with client demand. Now, however, the thousands of sensors, monitors and other technologies in the IoT are leading to the rise of the machine: they generate huge volumes of data that is applicable to many other industries and a broad range of business activities.

The rise of machine data: Are you prepared?

4

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

more than two million pieces of data. Since most planes take multiple flights per day, the amount of data aggregated over weeks, months and years is astonishing.

The bottom line: data automatically generated in the IoT provides a fantastic fuel source for the era of big data. Speaking of fuel, the McKinsey Global Institute estimates that the

Sensor sensibility

Sensors are devices that detect or measure a physical property and then record or otherwise respond to that property, such as acoustics, vibration, chemicals, electric current, electric potential, magnetism, radio frequencies, environment, weather, moisture, humidity, flow, fluid velocity, light, imaging and photons, just to name a few.

Machine data typically comes from logs or sensor output. But machine data can be broadly defined as any information that is automatically created from a computer process, application or other machine without the intervention of a human. Such sources include health monitors, security cameras, computer networks, call detail records, financial instrument trades and more.

Consider, for example, a Boeing 747 aircraft. The airliner generates data across hundreds of parameters every second, which means a three-hour flight creates

automotive industry will be the second-largest generator of data by 2015.3 This estimate is not surprising, since some plug-in hybrid vehicles generate 25 GB of data in just one hour. Just as unsurprising is the leader in machine data: the utilities industry, with its bevy of smart meters, usage trackers, geographic sensors and other monitoring technologies.

The rise of machine data: Are you prepared?

5

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Big data has always been with us. What’s new is our ability to capture and analyze more of it to achieve results faster.

What to do about the rise of machine data: Analyze it

Emerging big data technologies enable a new generation of applications designed to analyze large volumes of multistructured, often-in-motion machine data to gain insights. Performing analytics on machine data can help you answer the following questions:

• Do you have real-time visibility into your business operations, such as customer experience and behavior?

• Are you able to analyze all your machine data and combine it with existing security data to predict and take action on a security threat?

• Are you proactively monitoring end-to-end infrastructure, such as wireless networks, smart grids or manufacturing supply chains, to optimize expensive resources and deliver services when and where they are most needed?

Handling machine data involves several specific challenges. Formats vary and are complex, plus few standards exist. A mix of streaming and at-rest data complicates the correlation and visualization of data sets. Machine data is also likely to be time sensitive, with a mixture of data refresh rates. In addition, the data may or may not have context.

The solution? Continuous, extremely fast analysis of massive volumes of data in motion. Big data has always been with us. What’s new is our ability to capture and analyze more of it to achieve results faster.

The rise of machine data: Are you prepared?

6

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Capitalize on real-time analytics of data in motion

Real-time analytic platforms consist of three core components.

1. Development environment. Developers should be able to easily and rapidly build applications and connect to new data sources. Drag-and-drop editors, wizards, visualization tools, and runtime monitoring and debuggers are a must.

2. Scale-out architecture. The supporting infrastructure should adapt to rapidly changing data formats, types and messaging protocols. It should also read from and write to a vast number of data sources. A massively parallel architecture is designed to deliver unlimited compute potential.

3. Analytic toolkits and accelerators. Tools should facilitate sophisticated analytics, such as geospatial, voice, image and text, and also update models on the fly.

Of the three components, the analytic toolkit is the most challenging to get right—and the most important.

The rise of machine data: Are you prepared?

7

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Accelerating real-time analytic processing

InfoSphere Streams is an advanced analytic platform that allows user-developed applications to quickly ingest, analyze and correlate information as it arrives from real-time sources. InfoSphere Streams is designed to handle very high data throughput rates, up to millions of events per second. A market leader in providing sophisticated analytics for IoT, IBM received the 2013 Ventana Research award for Operational Intelligence in the IT Innovation category for InfoSphere Streams.

IBM® InfoSphere® Streams provides all three of these core components, including more than 10 toolkits and accelerators, to enable a variety of analytics out of the box.

Geospatial: Provide high-performance processing of geospatial dataGeospatial analysis requires complex mathematics such as set theory and geospatial geometry. It is used for location intelligence and location-based services for security and surveillance, geographic information systems, traffic patterns and more. The city of Dublin, Ireland, uses InfoSphere Streams to analyze 50 bus locations per second for its fleet of roughly 1,000 buses.

Capability Value

Cleansing, filtering, aggregation and analytics for sensor and log data

• Verify veracity of data as it streams into your organization

• Save and persist only pertinent data

Ultra-high velocity runtime to support fast data

• Keep pace with continuing growth of sensor capabilities

• React in real time to real-world-aware devices

Support for and analysis of unstructured data such as text, audio and video

• Leverage toolkits and samples for more rapid time to value

• Use new insights from social media to better understand customers

Powerful analytics to model real-world events with data from sensors and social media

• Reuse existing tools and models built in IBM SPSS®, SAS, R, Matlab, Java and C++

• Predict future events from current sensor data with advanced cognitive and predictive tools

• Gain customer insight to better target prospects through social media

Industry solutions

• Utilize IBM Research-developed analytic accelerators• Deploy InfoSphere Streams inside a variety of

applications, including operational detection, intelligent transportation and predictive insights

The rise of machine data: Are you prepared?

8

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Text: Parse unstructured text to detect meaning and understand contextIn the IoT, text analytics can be used to derive meaning from vast amounts of text, including social data, to determine sentiment and identify illegal or suspicious activities. InfoSphere Streams can complete unstructured text analysis faster and more precisely than traditional methods. It analyzes millions of messages in real time4 and helps ensure data is trusted and secured.

Time series: Detect and predict patterns and anomalies in real timeKnowing the order of events can have profound impacts, for example in predicting the path of a natural disaster or picking the next best stock trade. InfoSphere Streams helps insurance companies plan for natural disasters and enables real-time public alerts. It also performs real-time analysis of sensor data collected from the Hudson River, one of the most instrumented bodies of water in the world.

Telecommunications: Process call data in real timeTelecommunications service providers continue to experience a huge growth in smartphone and mobile device use. Growing text and data usage creates a deluge of context- and time-sensitive data. InfoSphere Streams enables telecommunications providers to analyze billions of call data records per day to detect fraud, ensure high asset utilization and create accurate customer profiles for heightened customer service and retention. Using InfoSphere Streams, Sprint reduced storage costs by 90 percent.

The rise of machine data: Are you prepared?

9

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

The connected car: Innovating for sustainability

Here’s a real-world example from the transportation industry to show how InfoSphere Streams makes real-time analytics possible: enabling connected vehicles.

The automotive industry is being reshaped by changing markets and new technologies (see Figure 1). Alliances are forming as consumer and environmental requirements create the need for more collaborative partner ecosystems. Increased globalization is driving more integration within and between automotive OEMs and their suppliers. Sophisticated consumers are increasing demand for innovative and sustainable vehicles.

Figure 1. Dramatic forces affecting the automotive industry require new approaches.

Increased globalization drives more integration within automotive companies

Consumer, regulatory and environmental requirements drive the creation of collaborative partner ecosystems to promote innovation

Sophisticated consumers demand innovative and sustainable vehicles

Rapidly integrating enterprises drive increasingly dynamic operations

New technologies and capabilities make vehicles more intelligent

The rise of machine data: Are you prepared?

10

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

As a result, automotive leaders are seeking new ways to quickly launch sustainable vehicles, capitalize on services opportunities, optimize the global value chain and transform the retail environment.

Modern vehicles generate myriad data: driver weight, car speed, weather conditions, road status, geospatial positioning, fuel levels, tire pressure and more. Many people may be uncomfortable knowing that their weight is out on the IoT, but that data point has been used to save lives: knowing the weight of the driver and passengers enables the car to intelligently deploy airbags during a crash.

Moreover, connected cars can give you a personalized passenger experience. Let’s say every morning Dad drops off his kids at school, goes to a coffee shop and then heads to work. Dad’s experience could be more pleasant if he knew what time to leave to optimize his commute time or if he received a coupon on his phone for a free coffee at a shop on his route. Suppose Dad has been going to a fast-food restaurant every morning as well. His insurance company may choose to send him information on healthy eating habits or a discount for a personal trainer.

Here’s another example: Mom has to run errands after work. She might want to know the optimal path between her office and five stores, given the time of day and traffic and weather conditions. She might also appreciate alerts about the stores’ closing times, so she can adjust her route accordingly. If Mom gets distracted, a connected car could automatically deploy the brakes when it senses a car nearby.

The rise of machine data: Are you prepared?

11

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Commercial freight companies also benefit from connected vehicles. Trucking companies use connected vehicles to assess the health of drivers and understand weather and road conditions. This data can inform their response to minimize insurance charges; for example, they may choose not to deploy a truck under harsh road conditions since insurance premiums are higher during inclement weather.

They also use machine data to improve driver safety and vehicle-to-vehicle communication. Wouldn’t it be useful if windshield wipers automatically turn on when raindrops fall or if an emergency vehicle was automatically routed to the scene of an accident and advised of its severity level?

Data from connected vehicles gives automobile manufacturers valuable insight into fuel efficiency, engine performance, temperature and more, helping them engineer longer-lasting, more fuel-efficient cars. Cars that monitor their own health can automatically alert drivers and service departments when service is needed, giving drivers proactive updates and enabling service teams to order parts in advance.

Moreover, service departments can quickly analyze warranty claims, diagnostic and fault codes, customer usage data and service records with traceability through the entire value chain. Dealers and car manufacturers can also analyze social media data in response to promotions, ads or trade shows.

The rise of machine data: Are you prepared?

12

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

InfoSphere Streams enables analytics for these and other use cases, empowering a broad ecosystem including car manufacturers, retailers, insurance companies, trucking companies and consumers to be safer and be more productive on the road (see Figure 2). Real-time analytics are used both during and after the manufacturing process to achieve exceptional outcomes, including:

• Profitable aftermarket services and products• Improved, interactive driving experience

and safety by real-time analysis of weather-based data or road-congestion alerts

• Integrated vehicle data available to third parties such as insurance companies, retailers and emergency medical services

• Improved quality and functionality of future products

• Optimization of the global value chain to improve the environment Figure 2. Data from connected vehicles helps improve the efficiency of the extended automotive experience.

Connected vehicle

• Business development• Alliances and partners• Sales and marketing

• Sales, marketing and customer service

• Supply-chain manufacturing• Service and warranty

• Warehouse and inventory• Engineers• Product development

Monetize telematics

Differentiate the driving experience

Manage the supply chain to optimize inventory

Improve product quality

Predict parts failures

Improve the design of future products

Improve customer care

Generate more leads and vehicle sales

Improve sales of aftermarket products and services

The rise of machine data: Are you prepared?

13

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Uncovering actionable insights through the IBM big data platform

InfoSphere Streams plays an important role in the IoT, particularly in the connected car scenario. But the primary value of the IBM approach is in the comprehensive big data portfolio available to support action in the IoT (see Figure 3).

The IBM big data platform is designed to help you gain insight from big data by delivering enterprise-class data management and advanced analytics. The platform supports ad hoc data exploration, discovery and unstructured analysis as well as structured, repeatable tasks to improve business insight regardless of the volume, variety, velocity or veracity of data.

Figure 3. Connected vehicle reference architecture for the IBM big data platform.

IBM MessageSight

Stream computing InfoSphere Streams

Visualization

Data Warehouse

IBM PureData Systems

Hadoop System

IBM InfoSphere BigInsights Software

developmentRequirements management

Discovery

The rise of machine data: Are you prepared?

14

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

The IBM big data platform effectively manages and analyzes data in its native format—unstructured, structured, at rest or in motion. To accelerate time-to-value, all platform components are pre-integrated. By leveraging this extensible set of capabilities, you can start with a single project using one capability and add others as needed. Components of the IBM big data platform include the following:

• IBM InfoSphere BigInsights™ for Hadoop brings the power of the Apache Hadoop framework to the enterprise. The open source Hadoop software is used to reliably manage large volumes of structured and unstructured data.

• InfoSphere Streams helps you turn burgeoning, fast-moving volumes and varieties of data into actionable information and business insights.

• IBM Watson™ Explorer provides federated discovery, search and navigation over a broad range of data sources, helping you get started quickly with big data initiatives and gain more value from your information.

• IBM PureData™ System for Operational Analytics—part of the IBM PureSystems™ family—is an expert integrated data system designed and optimized specifically for the demands of an operational analytic workload.

• IBM PureData System for Analytics is a high-performance, scalable, massively parallel system that enables you to gain deep insight from your data and perform analytics on enormous data volumes.

• IBM MessageSight is a full-featured messaging appliance designed for machine-to-machine (M2M) and mobile environments. It processes large volumes of events in near–real time to deliver the performance, value and simplicity you need to accommodate a growing number of mobile devices and sensors.

The rise of machine data: Are you prepared?

15

1 The Internet of Things

2 What to do about the rise of machine data

3 Real-time analysis of data in motion

4 The connected car 5 Uncovering actionable insights

6 Resources

Resources

The massive amount of machine data available today—originating from IT machines, sensors, meters and more—requires complex analysis and correlation across different types of data sets. Companies that can perform this analysis across a variety of data (and do so with speed and accuracy) stand to reap rewards in business efficiency, customer satisfaction and strategic success. The IBM approach to big data takes all of this into account to help you gain business insights and real-time visibility into the customer experience.

To learn more about Internet of Things, InfoSphere Streams and the IBM big data platform, check out these resources:

Big data at the speed of business: Operations analysis use case

Big data in action: Industry examples

Read the InfoSphere Streams Playbook for deeper technical content about InfoSphere Streams

Want a free copy of InfoSphere Streams Quick Start edition? Click here

IMM14142-USEN-01

© Copyright IBM Corporation 2014

IBM Corporation Software Group Route 100 Somers, NY 10589

Produced in the United States of America March 2014

IBM, the IBM logo, ibm.com, BigInsights, IBM Watson, InfoSphere, PureData, PureSystems, and SPSS are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml

Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.

This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates.

The performance data and client examples cited are presented for illustrative purposes only. Actual performance results may vary depending on specific configurations and operating conditions. THE INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING WITHOUT ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND ANY WARRANTY OR CONDITION OF NON-INFRINGEMENT. IBM products are warranted according to the terms and conditions of the agreements under which they are provided.

The client is responsible for ensuring compliance with laws and regulations applicable to it. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the client is in compliance with any law or regulation.

Please Recycle

1 “More Than 30 Billion Devices Will Wirelessly Connect to the Internet of Everything in 2020.” ABI Research press release, May 9, 2013. https://www.abiresearch.com/press

2 Peter Steiner. The New Yorker. 1993. http://en.wikipedia.org/wiki/File:Internet_dog.jpg

3 James Manyika, et al. “Big Data: The Next Frontier for Innovation, Competition, and Productivity.” McKinsey Global Institute. June 2011. www.mckinsey.com/insights/business_technology/big_data_the_next_frontier_for_innovation

4 IBM Redbooks Solution Guide. “Turning Big Data into Actionable Insight with IBM InfoSphere Streams.” August 2013. www.redbooks.ibm.com/abstracts/TIPS0948.html?Open