Machine Data Intelligence (MDI) Capability Maturity Model
A multi-level framework for achieving advanced machine data intelligence at scale
2 • Machine Data Intelligence (MDI) Capability Maturity Model
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
MDI Capability Maturity Model Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
What’s Your Business’s MDI Capability Maturity Stage? . . . . . . . . . . . . . . . . . 6
Level 1: Basic Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
Level 2: Advanced Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
Level 3: Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Level 4: Innovation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
Level 5: Monetization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Contents
3 • Machine Data Intelligence (MDI) Capability Maturity Model
OverviewIt is estimated that within the next ten years there will be a trillion connected computers
and devices on the planet. You have many, many of these machines in your own enterprise
— some you are aware of, probably some that you aren’t. These machines range from
traditional servers in data centers and cloud infrastructure to modern day IoT sensors and
devices. They power everything from conventional computing to smart medical devices and
autonomous vehicles.
These machines produce an unfathomable amount of data — technically “telemetry” or
measurement data — that is growing so exponentially, there are no reliable estimates on
just how much is being produced. Now imagine the wealth of insights and potential business
value contained in that data if it could be harnessed — at scale and in real-time. It is data
so dense and rich with potential value, it dwarfs traditional business intelligence as the next
frontier of competitive advantage.
4 • Machine Data Intelligence (MDI) Capability Maturity Model
It’s called “machine data intelligence (MDI).” Every aspect of our lives and our world is
becoming rapidly digitized and computerized. We’ve deployed machines to do things for
us but what can they now tell us as a result? As it turns out, quite a lot. Machine data can
be used certainly for monitoring of availability and performance of IT infrastructure, but
also for optimization of devices and operations, predictive analytics and maintenance, and
innovating entirely new products and services that can be monetized.
So how do you get started? Where do you start? This eBook was developed to help
enterprises develop a strategy and a game plan to implement an MDI program and begin
to reap its benefits. It is based on a capability maturity model not unlike maturity models
for software development. It consists of five easy-to-understand levels of maturity and
practical suggestions on how to move from one level of maturity to the next. Like any good
framework, you can plot where your enterprise fits today, determine where you’d like to be,
and then develop a plan to close the gap. You may find yourself at the earliest stages of this
model. Don’t despair. You are not alone, but the competitive landscape is moving quickly.
The Circonus MDI Capability Maturity Model is a multi-level framework that helps you to
identify the current situation of your enterprise and next steps to take, including:
• What should be improved at the stage you’re currently in
• Capabilities needed to master a level
• What your MDI capability maturity should look like upon completion of a level
• The must-haves for moving to the next level
5 • Machine Data Intelligence (MDI) Capability Maturity Model
MDI Capability Maturity Model Chart
M O N E T I Z A T I O N
I N N O V A T I O N
O P T I M I Z A T I O N
A D V A N C E D M O N I T O R I N G
B A S I C M O N I T O R I N GL E V E L 1
L E V E L 2
L E V E L 3
L E V E L 4
L E V E L 5
MDI C A P A B I L I T Y M A T U R I T Y M O D E L C H A R T
6 • Machine Data Intelligence (MDI) Capability Maturity Model
What’s Your Business’s MDI Capability Maturity Stage?
Level 1: Basic Monitoring
In level 1, there is no overarching MDI strategy for the enterprise. It is likely that a complete
inventory of all machines and associated metric streams does not exist. There is some
monitoring of IT infrastructure, but there is a lack of standards and consistent processes.
No attempt is made to harness machine data for more advanced uses like predictive
maintenance. MDI is not a strategic objective for the company and does not have visibility
in the C-suite.
Level 2: Advanced Monitoring
In level 2, enterprises have invested heavily in monitoring tooling, but they are using multiple
tools and platforms — resulting in disparate data sources, no single source of truth, and the
inability to aggregate data easily across the enterprise. There is no centralized platform for
the collection, storage, and subsequent analysis of machine data. Awareness of, and interest
in, machine data intelligence is rising but there is not yet a corporate objective to build a
comprehensive MDI strategy.
Level 3: Optimization
In level 3, the enterprise has consolidated and rationalized its monitoring and data collection
capabilities across the enterprise and has built a solid foundation on which to begin deriving
additional value from machine data. At this stage, the enterprise is using machine data to
optimize device performance, implement predictive analytics and maintenance programs,
and optimize company operations. The enterprise begins to move from basic monitoring
into optimization of product and service delivery, and MDI is becoming a topic of discussion
in the C-suite.
7 • Machine Data Intelligence (MDI) Capability Maturity Model
Level 4: Innovation
In level 4, the enterprise begins to move into true intelligence mode. It’s optimizing
operations and product and service delivery and, as a result, realizing significant efficiencies
and impact to its bottom line. At this stage there is a bedrock foundation of rich data, and
multiple business units are using a sophisticated data science platform to innovate new
products and services. Entirely new potential revenue streams such as packaged information
products are beginning to be identified. A company champion emerges and work begins on
an overarching MDI strategy.
Level 5: Monetization
In level 5, there is a cohesive and comprehensive MDI strategy in place for the entire
enterprise. It is a strategic imperative of the company with executive sponsorship at the CEO
level. At this level, all machines and metric streams have not only been identified but also
mapped to strategic corporate initiatives. All business units are leveraging machine data to
drive innovation in products and services, and there are frequently new ideas for monetizing
machine data and creating entirely new revenue streams. MDI has become a strategic
competitive advantage and is driving both top line revenue as well as bottom line profitability.
8 • Machine Data Intelligence (MDI) Capability Maturity Model
Level 1: Basic MonitoringAt this level, some monitoring is likely to be occurring but conducted in an inconsistent and
patchwork way, typically according to the preference of individual staff and developers.
There is a lack of a defined and communicated organizational mandate outlining what and
how to monitor across the enterprise’s IT organization, and what the results of a monitoring
effort should be. Consequently the business is unable to monitor their systems and services
in any uniform manner which prevents them from surfacing value from those efforts.
M O N E T I Z A T I O N
I N N O V A T I O N
O P T I M I Z A T I O N
A D V A N C E D M O N I T O R I N G
B A S I C M O N I T O R I N GL E V E L 1
L E V E L 2
L E V E L 3
L E V E L 4
L E V E L 5
9 • Machine Data Intelligence (MDI) Capability Maturity Model
What this level looks like
• No defined standards on what and how to monitor across the enterprise’s IT organization.
• No inventory of systems, services, or custom applications.
• Monitoring is done inconsistently and according to individual staff and developer preference.
• Disparate tools are being leveraged by different functions within the organization.
• Systems and services are not being monitored in a uniform manner, providing little to
no value.
Capabilities required
• Technical expertise to create an inventory plan and permissions to execute upon that plan.
• Technical expertise to add monitoring for existing and future infrastructure and core services.
• Creation or identification of tooling or libraries through which to solicit or submit
application monitoring metrics.
10 • Machine Data Intelligence (MDI) Capability Maturity Model
Checklist to move to next level
Ú Identify a platform through which to consolidate your monitoring and subsequent
analytics efforts.
Ú Create a comprehensive inventory of existing infrastructure, services, and applications.
Ú Create a plan through which the business identifies and defines the specific metrics and
KPIs it expects to collect from its infrastructure, services, and applications.
Ú Establish observability standards for all physical, virtual, cloud, and container
infrastructure, along with any services and custom applications.
The completion of this level’s requirements leaves the business with a well-defined “monitoring
plan” of the assets to monitor, why to monitor a specific asset, and how specifically to monitor it.
11 • Machine Data Intelligence (MDI) Capability Maturity Model
Level 2: Advanced MonitoringWhile much of the effort from the previous level was done manually, the advanced
monitoring stage is where everything is automatically and uniformly monitored. There
is a single, consolidated monitoring platform in place for the purposes of infrastructure,
service, and application monitoring. A defined and communicated set of standards that
outline the monitoring of infrastructure, services, and applications is in place. There
is a comprehensive inventory of infrastructure, services, and applications and all are
monitored according to standards.
M O N E T I Z A T I O N
I N N O V A T I O N
O P T I M I Z A T I O N
A D V A N C E D M O N I T O R I N G
B A S I C M O N I T O R I N GL E V E L 1
L E V E L 2
L E V E L 3
L E V E L 4
L E V E L 5
12 • Machine Data Intelligence (MDI) Capability Maturity Model
What this level looks like
• Business has a monitoring plan with a well-defined set of standards on the monitoring of
infrastructure, services, and applications.
• There is a comprehensive inventory of infrastructure, services, and applications.
• There is a centralized platform for the collection, storage, and subsequent analysis of metrics.
• Monitoring is consolidated into the centralized monitoring platform and disparate,
legacy monitoring tools are removed.
• Inventoried assets are monitored in accordance with the monitoring plan.
Capabilities required
• A centralized, performant platform through which to consolidate the business’s
monitoring efforts. Specifically a platform that can monitor physical/virtual servers, cloud
services, containers, commonly encountered services, and custom application metrics.
• Fully automated monitoring for all provisioned infrastructure and services.
• Technical expertise to incrementally retrofit existing applications to emit metrics through
agreed upon tooling or libraries.
• A DevOps function to ensure that all freshly provisioned infrastructure and services are
monitored by default, while also providing enforcement of agreed upon monitoring
standards for custom applications.
13 • Machine Data Intelligence (MDI) Capability Maturity Model
Checklist to move to next level
Ú Conduct staff training on the usage of the identified centralized monitoring platform
and overall capabilities.
Ú Create a set of standards and guidelines for DevOps within the business, and create a
plan through which freshly provisioned infrastructure, services, and applications are
automatically monitored.
Ú Enforce standards and guidelines for software development within the business to
ensure consistent and comparable metrics.
The conclusion of this level effectively guarantees comprehensive, uniform, and cost-effective
monitoring even in the event of staff churn, hypergrowth, etc.
14 • Machine Data Intelligence (MDI) Capability Maturity Model
Level 3: OptimizationAt the optimization level, businesses begin to use and leverage the data that's been collected
to identify new opportunities for improvement. Workflows are created to push the resulting
intelligence to the necessary roles within the business to ensure proper budgeting and
execution. The key to capitalizing on optimization opportunities is the ability to capture
all the data at the speed and frequency it is being generated and the ability to retain that
resolution of data without limit. The greater the density and richness of the data, the greater
the accuracy, precision, confidence, predictive qualities, and insights.
M O N E T I Z A T I O N
I N N O V A T I O N
O P T I M I Z A T I O N
A D V A N C E D M O N I T O R I N G
B A S I C M O N I T O R I N GL E V E L 1
L E V E L 2
L E V E L 3
L E V E L 4
L E V E L 5
15 • Machine Data Intelligence (MDI) Capability Maturity Model
What this level looks like
• Workflow and processes are in place to enable consistent and uniform monitoring of metrics.
• There is a centralized platform for the collection, storage, and subsequent analysis of metrics.
• High volume, high frequency metrics are collected across a wide spectrum of metrics
streams or sources.
• Analysis of the collected metrics is conducted to inform efficiency and
operational improvements.
• Workflows that ensure the resulting intelligence are shared within the organization for
proper budgeting and execution.
Capabilities required
• Staff with the necessary skills and time to extract actionable insights from the
centralized intelligence platform.
• Processes and workflows in place to facilitate the identification of opportunities for
operational and efficiency improvements.
• Processes and workflows in place to communicate findings up to managers
and executives.
• Additional platform capabilities, including real-time alerting, anomaly and fault
detection, outlier detection, and trend analysis.
• SLO/SLA monitoring and altering.
16 • Machine Data Intelligence (MDI) Capability Maturity Model
Checklist to move to next level
Ú Analyze collected metrics and draw conclusions about existing workflows and processes.
Ú Facilitate platform training for engineers and data science teams to broadly leverage
the data moving forward.
Ú Develop processes and workflows to facilitate the identification of opportunities for
operational and efficiency improvements:
ȩ Create reports that show what has happened previously, and what’s happening now.
ȩ Quantify the impact of software improvements, as well as outages, etc.
ȩ Generate predictive analytics.
Ú Develop processes and workflows to communicate findings to managers and executives.
Ú Identify executive sponsorship and a company champion.
Completion of this level results in a business now able to effectively leverage the collected
data. Concrete opportunities for improvement are identified and workflows are created to
push this information around the organization to those who need it.
17 • Machine Data Intelligence (MDI) Capability Maturity Model
Real-World ExampleA global oilfield services provider uses machine data intelligence to
drive better decision-making and optimize well production.
A global oilfield services provider is collecting high frequency analytics from
sensors inside wells to measure hydraulic pressure and determine if in-fill wells
(child wells) will compromise the productivity of parent wells. This allows the
service provider to manage well interference while maximizing the density of well
spacing and minimizing the risk of damage from frac hits.
Frac hits occur when the hydraulic pressure applied in a child well impacts the
operations of the parent well. They can be very expensive in terms of damages
to production tubing, casing, and well heads, but they can also result in a
substantial reduction in parent well production, causing economic losses in the
millions of dollars.
Ultimately, machine data intelligence gives the organization extreme clarity into
its well operations so that it can improve profitability and reduce costs.
18 • Machine Data Intelligence (MDI) Capability Maturity Model
Level 4: InnovationAt the innovation stage, enterprises move into greater intelligence related activities by
employing advanced analytics and data science. Tapping into the full value of machine data
can be transformational, delivering real and measurable results. The key to unlocking this
enormous potential is the ability to harness and make sense of the wealth of machine data
already being generated in the enterprise. The ability to gather and analyze vast amounts
of machine generated data, including data from IT infrastructure, sensors, systems, and
connected devices, achieves new levels of insight that drive smarter operations, better
decision-making, and new business opportunities.
M O N E T I Z A T I O N
I N N O V A T I O N
O P T I M I Z A T I O N
A D V A N C E D M O N I T O R I N G
B A S I C M O N I T O R I N GL E V E L 1
L E V E L 2
L E V E L 3
L E V E L 4
L E V E L 5
19 • Machine Data Intelligence (MDI) Capability Maturity Model
What this level looks like
• All data science teams in all departments and functions have access to data assets and
advanced analytics capabilities.
• The company’s inventory of data assets is mapped to strategic company objectives to
identify potential opportunities.
• Opportunities are ranked by completeness and density of data assets and highest
priority objectives.
• Ideation techniques are employed to foster creative thinking.
• Time and money budgets are allocated appropriately.
Capabilities required
• Performant analytics query language with high fidelity data science functions.
• High frequency, real-time streaming analytics.
• Anomaly and fault detection.
• Outlier detection and trend analysis.
• Historical analytics.
20 • Machine Data Intelligence (MDI) Capability Maturity Model
Checklist to move to next level
Ú Executive sponsorship and buy-in at the C-suite level.
Completion of this level results in a business that is in command of all of its machine
generated data and quickly moving to the forefront among its industry peers. The business
is now not just finding new sources of competitive advantage but completely reinventing the
ways services and products are delivered and has become a disruptor in its industry.
21 • Machine Data Intelligence (MDI) Capability Maturity Model
Real World ExampleAn energy tech company uses machine data intelligence to help
buildings become more efficient and reduce operational costs.
An energy tech company uses high frequency telemetry data pulled off of
thousands of sensors within a building to improve efficiency and lower costs.
Data collected and analyzed include electrical usage, temperature, and human
movement by floor, as well as power utilization by outlet and appliance.
Insights as to when energy can be dialed back or when it’s cost-efficient to
replace appliances help “smart buildings” cut costs, reduce maintenance time,
and improve overall functioning of the building. This type of rich information is
also playing a major role in risk mitigation and driving entirely new approaches
to underwriting in the insurance industry.
22 • Machine Data Intelligence (MDI) Capability Maturity Model
Level 5: MonetizationAt this level, a business now has the requisite data stored within a platform, methods for
accessing and leveraging that data, staff to perform the necessary work, and processes
and workflows in place to communicate conclusions to the necessary staff within the
business. Businesses at this level are able to collect and ingest incredibly high volume and
high frequency data. They can retain, find, and quickly retrieve data, as well as execute
sophisticated and complex real-time, historical, and predictive analytics against that
data. Machine data can be mined at will, without compromise or constraints — unlocking
unprecedented insights and value creation opportunities.
M O N E T I Z A T I O N
I N N O V A T I O N
O P T I M I Z A T I O N
A D V A N C E D M O N I T O R I N G
B A S I C M O N I T O R I N GL E V E L 1
L E V E L 2
L E V E L 3
L E V E L 4
L E V E L 5
23 • Machine Data Intelligence (MDI) Capability Maturity Model
What this level looks like
• Businesses reap the rewards of having a centralized intelligence platform with full
visibility into existing and future operations.
• Infrastructure, services, and software are constantly monitored and improved as needed.
• Processes and workflows are developed and implemented to inform leadership about
opportunities being identified throughout the company.
• Machine data intelligence has become a source of sustained competitive advantage.
Capabilities required
• Staff with the necessary skills and time to extract actionable insights from the
intelligence platform.
• Processes and workflows in place to facilitate the identification of opportunities for
operational and efficiency improvements.
• Processes and workflows in place to communicate findings up to managers and executives.
• Performant analytics query language with high fidelity data science functions.
• High frequency, real-time streaming analytics.
• Anomaly and fault detection, outlier detection and trend analysis, and historical analytics.
This final step is not meant to be the end of the journey. Once a business achieves this level
of sophistication, it must ensure all accomplishments in prior steps remain in place, while
continuing to maintain a constant focus on data-driven improvement of processes and
workflows throughout the business.
24 • Machine Data Intelligence (MDI) Capability Maturity Model
Real-World Example A professional sports league uses machine data intelligence to
automate monitoring, optimize viewing performance, and
generate revenue.
This organization has completely automated the monitoring of its entire
infrastructure, including both its cloud and physical environments. New
infrastructure is automatically monitored as it is spun up, and the league is
collecting detailed metrics at the application layer.
This allows the league to monitor the service levels across all of its APIs. The
ability to keep track and meet the SLAs of all their clients, who depend on
the ability to provide optimal viewing experiences for their over 100 million
consumers, has a substantial direct effect on the organization’s bottom line.
25 • Machine Data Intelligence (MDI) Capability Maturity Model
SummaryThe possibilities of machine data intelligence, across all sectors including healthcare,
insurance, utilities, ad tech, manufacturing, and many others, are boundless — limited
only by our creativity and imagination. As the number of things we want to monitor and
measure grows, and sensors proliferate our world, we can only imagine what we will find
and what we can create. Ultimately, however, we know that being smarter means making
better decisions. It means that our decisions are driven by data that we can rely on for its
accuracy and timeliness. It means that we have new knowledge that can drive competitive
advantage and change the trajectory of our products, our operations, our IT infrastructure,
and our business performance.
This is the “Internet of Everything” economy. The companies that learn to harness machine
data to optimize operations, innovate new products and services, and create entirely new
revenue streams will be the clear winners.
26 • Machine Data Intelligence (MDI) Capability Maturity Model
Circonus is the machine data intelligence expert, providing the only machine data
intelligence platform capable of handling billions of metric streams in real time to drive
unprecedented business insight and value. Led by experts in large-scale distributed systems
and data science, Circonus is pioneering the way that machine data at scale is leveraged
throughout the enterprise, from operational analytics to IoT applications.
To learn more, visit http://www.circonus.com
Contact us today to learn how your organization can realize the full power
of machine data intelligence .
CONTACT US LONG-ARROW-RIGHT