This white paper is part 1 of a two part

series where we uncover why machine

intelligence is the key to solving the data

integration challenge for the IIoT.

We’ve all heard the facts and figures

associated with the Industrial Internet. There

will be 50 billion devices connected to the

Internet by 2020. The Industrial Internet of

Things could add $14.2 trillion to the global

economy by 2030. The productivity and

economic potential of the Industrial Internet

is clear. What is less clear is how we get to this

future. The classic iceberg example comes

to mind – data visualization and KPI tracking

are only the tip of the iceberg and according

to Gartner about 20% of an analytics project

budget. The rest, the sleeping giant beneath

the surface, is a mammoth problem that

encompasses everything from integrating

data, moving data, meta data management,

data quality challenges and more. By the same

Gartner estimates, this accounts for nearly 80%

of a data analytics project budget and comes

at a significant resource cost of a large team,

consultants, architects and engineers.

1

By Mike Varney

The heaviest lift for an

industrial enterprise is

data integration, the

Achilles’ heel of the

Industrial Internet of

Things (IIoT). This is

blocking progress on the

transformations and ROI

that companies originally

envisioned.

The heaviest lift for an industrial enterprise

is data integration, the Achilles’ heel of the

Industrial Internet of Things (IIoT). This is

blocking progress on the transformations

and ROI that companies originally envisioned.

Developments have been stymied by

challenges in handling the complexity,

diversity, volume, and velocity of data as

well as in the disparity of data characteristics

such as quality, completeness and timeliness.

Companies are now recognizing the heavy-lift

involved in supporting Big Data strategies

that can handle the data that is generated by

information systems, operational systems and

the extensive networks of old and new sensors.

To compound these issues, business leaders are

expecting data to be captured, analyzed and

used in a near real-time to optimize business

processes, drive efficiency and improve

profitability. However, integrating this vast

amount of dissimilar data into a unified data

strategy can be overwhelming for even the

largest organizations.

WHY MACHINE INTELLIGENCE IS THE KEY TO SOLVING THE DATA INTEGRATION PROBLEM FOR THE IIOT

CHALLENGES WITH IIOT DATA INTEGRATION

The technologies behind IIoT have brought

significant advancements to industries such

as Manufacturing, Transportation, Oil & Gas,

Aviation, Energy, Automotive and other

industrial enterprises. These technologies

have allowed industry to remotely monitor

and control assets to optimize production

and improve yields. However, these same

technologies have exacerbated a long-standing

data integration problem by massively

increasing the volume, velocity and diversity of

data required by the business.

A Case Study

To fully understand the issues associated

with IIoT data integration, it is best to follow

case studies on companies that have been

pioneers in this space. Take for example,

one particular industrial customer with over

two million sensors deployed across their

geographic assets. They are receiving over

two billion sensor readings every day including

events, field order details, crew updates,

customer transaction details and a myriad

of other complex data that includes satellite

images and weather updates. This customer

has a need to capture and analyze this data

on a near real-time basis to centrally manage

their operations—including effective dispatch

of field crews. The customer not only needs

to manage data from sensors, but must

correlate that data with production control

systems, business transaction systems, and

communication network systems. Timing,

synchronization and quality of data across

two million sensors and all the back-end

systems was a major challenge for traditional

integration approaches—especially given the

rate of change within the environment.

AN INTELLIGENT SOLUTION WITH MIX

Bit Stew’s MIx CoreTM platform provides

a comprehensive purpose-built approach to

data integration – an approach designed for

the IIoT. The platform leverages methodology

and algorithms applied in a highly structured

manner to automate the data integration

process. Central to this methodology is

Bit Stew’s unique approach to machine

intelligence that encompasses more than just

machine learning algorithms. The underlying

technology of MIx Core provides an unmatched

capability to integrate data in even the most

dynamic and complex environments.

Intelligent Automation to Solve the Data

Integration

Solving the data integration challenge

requires a new way of thinking and traditional

data architectures must be reimagined to

support the rapid proliferation of data from

an exponentially expanding set of data types.

Bit Stew’s integration technology is designed

to rapidly ingest and integrate data to provide

a semantic understanding of information

across disparate systems. Deeper analytics can

then be applied intelligently through analysis

methods and workbenches.

The MIx Core technology has an innovative

approach to data integration based loosely on

the concept of universal connectors to any type

of system and data source such as a network.

The objective is to remove the typical point-

solution approach to integration and replace it

with a method that can be dynamically updated

and adjusted using Machine Intelligence. This

is the basics of the MIx Core integration stack

as illustrated below.

Traditional methods of integration that rely

on definitions, contracts and interface bindings

are too rigid and brittle to support the needs

2

Hidden Cost of Integration

What data scientists call “data wrangling” or

“data munging”, where they are belabored by

the mundane task of collecting and preparing

unruly data before it can be explored, accounts

for nearly 80 percent of their time. For

many industrial enterprises this is a common

issue with some large companies budgeting

anywhere from $5 million to $8 million per

project just to deal with the data integration

problems before tackling the high-value

functionality.

The tip of the iceberg is analytics while the

real toiling work remains below the surface

mired in difficulties and delays. The solution

offered by Bit Stew Systems is to eliminate the

costly integration effort and accelerate time to

value. Technical due diligence on the Bit Stew

technology has proven at least a 90% decrease

in the cost to serve by accelerating the process

of data integration (based on real-world

implementation experience).

INTELLIGENT AUTOMATION: KEY TO EFFECTIVELY MANAGING BIG DATA Consider how a Big Data solution

might analyze, query and report

data received from IIoT sensors to

streamline operations when that data

will have significant gaps and reading errors, cleansing and normalization

issues, synchronization and

sequencing problems and inherent

industrial noise affecting resolution and quality. Manual integration is far

too tedious and costly to deal with

IIoT Big Data on a continuous basis

and intelligent automation is the key

for successful automated integration.

3

of large-scale industrial environments. These

methods cannot adapt rapidly enough to

new data sources, new data types, changes in

environmental factors, revised configurations,

and near real-time business impact scenarios.

New methods for integration must be based

on a more flexible and interpretive design and

this is the leading concept behind the MIx Core

technology. The architecture and abstraction

layers designed into MIx Core allow the

integration to adapt to changing requirements,

while still meeting high-performance demands

and preserving the need for cognitive

processing of information.

Integration Approach

The MIx Core integration solution is

segmented into the following technology

layers:

• System Connectors

• Byte Sequencing & Marking

• Composite Information Patterns

• Semantic Model Mapping

• Intelligence, Analytics and Knowledge

The stack is based on several key layers that

separate connections from logic processing to

ensure sufficient flexibility for interpretation

at each layer in the stack. Some important

features of the MIx Core integration stack

include:

• Low-level byte-sequencing support

protocol/data translation, byte-level

referencing of any data type, and high-

performance access to data based on byte-

segment marking.

• Data interpretation is abstracted to

higher layers avoiding overhead in low-

level processing and allows for multiple,

parallel interpretation based on composite

information patterns.

• Composite information patterns are rapid,

adaptable and intelligent methods for

accessing the data stream and are based

on syntax notation for discovery and

interpretation. These patterns provide the

specification and adaptation needed by the

machine for intelligent processing of data

streams.

• Asynchronous Independence Assertion-

based processing allows for continuous

testing, rules processing and intelligent

control by MIx Core using injection, scope,

sequencing and frequency tracking.

The assertion-processing layer can be

responsible for the execution flow of

analytic methods and algorithms in the

examination and measurement of data

for the purposes of integration and

information analysis.

• Semantic-model mapping supports dynamic

mapping to target models and creation of

meta data for contextual understanding

• Knowledge repository and machine

learning wrapped in MIx Core supports

code generation.

PATTERN-BASED ACCESSThe Composite Information Patterns

use Data Source, Data Path and Data

Processor identifiers for accessing and interpreting data from any

source and any type. This provides a

generic framework for working across

streaming data as well as data at rest.

The Composite Information Patterns

can be constructed and understood

by humans, but more importantly

provide a machine construct that can

be automatically generated.

INTELLIGENT SEMANTIC MODELINGSemantic modeling is a process within the

MIx Core technology that will intelligently

understand and model all information within

the enterprise. This includes the automated

process of mapping new information to the

model and extending the model with new

learned entities and elements. The semantic

model within MIx Core can be trained using

pre-existing models or it can be developed

completely from scratch. With experience in

4

understanding industrial enterprise data, Bit

Stew has developed and maintains a highly

sophisticated semantic model that covers

operational controls, sensor channels, business

information, environmental data, geospatial

data and all the relations and associations. The

semantic model can be used and extended by

our customers using both supervised and un-

supervised learning methods.

Semantic modeling is a powerful feature of

the integration technology that relies on the

MIx Core AI components to interpret, discover,

learn, model and map data. The AI components

leverage machine intelligence algorithms and

methods for feature extraction, normalization,

associations, assertions and observations.

All of these are exposed through the web-

based interface and can easily be extended

by customers, partners and third parties using

the data management workbench as well as

more powerful extensions through the MIx

Developer Network.

5

SMALL TEAMS CAN SOLVE BIG INDUSTRIAL DATA INTEGRATION PROBLEMS

The MIx Core data integration technology

has been leveraged by many large customers

who have pioneered the Industrial Internet.

This includes organizations that have over

10 million disparate sensors deployed

geographically and are managing critical

operational systems. All of these customers

have shared a common problem of integrating

data from as many as 70 different backend

systems including:

• Operational systems such as production

control, historians and SCADA

• Business systems such as SAP, Oracle and

mobile workforce management systems

• IIoT communications management systems

such as external systems, routers and

gateways

• External system such as environmental

data, LiDAR data and satellite images

• Open source systems such as HDFS and

HiVE

USE CASES FOR DATA INTEGRATION IN INDUSTRIAL ENVIRONMENTS

Customers have invested years and millions

of dollars into custom-built solutions to tackle

the integration problem, in order to deliver

value through analytics and applications that

can leverage the data and analytics. With

the Bit Stew MIx Core technology, customers

have rapidly solved the integration problem

and started to leverage their investments in

Big Data to help transform and optimize their

business—with significant ROI recognized in

cost reductions, yield increases, and reliability

improvements.

Operational Asset Management

Several large organizations have utilized the

MIx Core capabilities to dynamically integrate

sensor data in order to monitor, track, analyze

and report on operational asset health. This

includes highly complex data from large

turbine assets, pipeline assets, refinery assets,

production control assets and others. Methods

such as decoder construction, pattern mining,

vertical aggregation and other algorithms

were applied in composed pattern recognition

routines to auto detect and cleanse data.

Algorithms for outlier detection and methods

such as directed acyclic graph traversal were

used for predictive failure detection and

automated decision processes.

Revenue Loss Detection

The revenue loss detection capability of

the platform has been heavily leveraged and

is applicable across industries. Many methods

are applied in this case to determine potential

revenue loss, including the use of Natural

Language Processing for analysis of textual

data that is widely available and valuable

for profiling to improve confidence levels.

Revenue loss protection uses many outlier

detection techniques on time series data, as

well as asymmetric independence assertions

and analytic “ensembles” for measurements,

observations and reasoning. Feedback loops

from investigations have proven helpful in

supervised training of the methods.

Solving the Data Integration Problem for the

IIoT Ecosystem

The MIx Core platform is the industry-

leading approach to achieve an enterprise-wide

contextual understanding of your data across

multiple disparate sources. The platform

removes the time and cost associated with

data wrangling and quickly integrates data

sources by leveraging sophisticated machine

intelligence to manage data in a scalable and

flexible framework.

CANADA - International Headquarters Suite 205 - 7436 Fraser Park Drive,Burnaby, BC, V5J 5B9(604) 568-5999 info@bitstew.com

USA 800 West El Camino Real, Suite 180 Mountain View, CA 94040 (650) 619-9000sales@bitstew.com

bitstew.comLearn more at

AUSTRALIA Rialto South Tower, Level 27, 525 Collins Street Melbourne, Australia 3000 salesaustralia@bitstew.com

EUROPE Paseo de la Castellana 141 – 8º 28046 Madrid , Spain saleseurope@bitstew.com

About Bit Stew Systems Inc. Bit Stew provides the premier platform for handling complex data integration, data analysis, and predictive

automation for connected devices on the Industrial Internet of Things (IIoT). Purpose-built for the IIoT, Bit Stew’s

MIx Core™ platform solves the data integration challenge at scale for complex industrial data environments. In

2015, Bit Stew was named to Greentech Media’s Grid Edge 20 list, as one of the top 20 innovators architecting the

future of the electric power industry, and was ranked as one of the Top 100 Analytics Companies and Top 100 IoT

Startups by Forbes. Incorporated in 2009, Bit Stew is a venture-backed private company that is headquartered in

Canada with offices in the USA, Australia and Europe. Visit www.bitstew.com to learn more.

Follow us on LinkedIn & Twitter: @BitStew

About Mike Varney, Executive Director, Product Management & Strategic Initiatives: Mike Varney spent over 20 years in the US Navy, where his experience included commanding the

most advanced nuclear-powered submarines in complex operations around the globe, leading a

special operations team in reconstruction efforts in Afghanistan, and directing a Naval Operations

Center. He has also served as a Strategic Advisor for the US Department of Defense, a Senior

Evaluation Officer at nuclear power plants, and an advisor to companies providing smart grid

technologies to the energy industry. Mike holds Bachelor of Science degrees in Nuclear and Marine

Engineering as well as Master of Science degrees in Engineering Management and National Security

Strategy. Today, Mike is the Executive Director, Product Management & Strategic Initiatives, where

he leads the strategy for Bit Stew Systems MIx Core platform, MIx Developer Network and Bit Stew

University.