Data Mining and Predictive Modeling for Condition-Based ......traditional fleet reliability management processes. Equipped with innovative text-mining Equipped with innovative text-mining

WHITE PAPER

Data Mining and Predictive Modeling for Condition-Based MaintenanceLeveraging advanced analytic technologies to improve defense logistics and sustainment operations

i

Data Mining anD PreDiCtive MoDeling for ConDition-BaseD MaintenanCe

Table of Contents

Executive Summary .........................................................................................1Data Mining and Predictive Modeling for CBM ............................................2

Revealing and quantifying hidden, implicit relationships in sensor data ............3

Revealing and quantifying relevance and statistical significance

of sensor data .................................................................................................3

Enhancing prognostics by mining unstructured narrative data .........................4

Rapid model development and validation ........................................................5

Model lifecycle management ...........................................................................6

Enhancing fleet management metrics with predictive modeling .......................7

Customer Case Examples ............................................................................8Major North Sea oil and gas producer .............................................................8

Northern European natural gas production partnership ...................................8

Military aviation fleet management ...................................................................9

Summary .........................................................................................................9About SAS .....................................................................................................10


ii

Contributor: Allan Manning is the Lead Industry Analyst at SAS for defense logistics, supply-chain and sustainment initiatives, and has extensive experience analyzing the defense and aerospace sector. In addition to writing articles and white papers, Manning often lectures and does interviews for both broadcast and print media on defense logistics-related topics. Prior to his current assignment, Manning was a Solution Architect for the SAS Global Manufacturing and Supply-Chain Practice and began his professional career as a Systems Engineer with Electronic Data Systems.


Executive Summary

Modern military logistics agencies must sustain diverse fleets of very costly, increasingly complex and indispensible weapon systems and platforms. With the constantly changing nature of today’s threats, the high cost and complexity associated with sustaining modern weapon systems will continue to be the status quo. Given that even a minimal disruption in the readiness of a single asset can cause significant negative impact to operational effectiveness, modern logistics and sustainment efforts must do everything possible to ensure maximum weapon system readiness and availability at the lowest possible cost. The traditional labor-intensive approaches of preventative and reactive maintenance are costly and inflexible – this is insufficient given the demands expected to be placed on modern weapon systems. Recent condition-based maintenance (CBM) efforts have been initiated that will be critical factors in maximizing weapon system readiness and availability for today’s war fighter.

Existing predictive maintenance solutions can provide data-collection, prognostic/predictive modeling and fleet management capabilities that are critical to effectively supporting CBM initiatives. Modern predictive maintenance solutions can integrate with the existing IT infrastructure to collect and transmit data from the various platforms to a centralized CBM database, as well as support the external logistics, reporting and data-sharing requirements. Most importantly, established predictive maintenance solutions can provide the most mature and comprehensive data analysis and prognostic/predictive model development capabilities. The need to leverage the most robust analytics available should not be understated. Any CBM initiative, regardless of scope or investment, must endeavor to use state-of-the-art technology in statistical and mathematical modeling techniques and proven methodologies for algorithm development; if the models developed do not provide better understanding or prediction of equipment failure or performance degradation in a meaningful way, then the entire CBM effort is pointless.

Equipped with modern predictive maintenance capabilities, military logistics and sustainment organizations can make better-informed, fact-based decisions regarding specific maintenance and logistics actions using data-driven analytic models that:

• Maximizesystemavailabilitycausedbyunexpecteddowntimeofplatformandsupport equipment by predicting and alerting impending failure or performance degradation.

• Minimizesystemmaintenancecostanddowntimebypredictingandalertingmaintenance-related events that cause downtime.

• Reducetotallifecyclecostandincreasefleetavailabilitythroughoptimizedmaintenance and sustainment plans.

• Improvefleetmanagementdecisionsbymodelingandpredictingthebehavioroffactors that affect fleetwide availability.

1

■ Given that even a minimal disruption

in the readiness of a single asset

can cause significant negative

impact to operational effectiveness,

modern logistics and sustainment

efforts must do everything possible

to ensure maximum weapon system

readiness and availability at the

lowest possible cost.

■ Existing predictive maintenance

solutions can provide data-collection,

prognostic/predictive modeling and

fleet management capabilities that

are critical to effectively supporting

CBM initiatives.


The ability of modern predictive maintenance solutions to provide world-class data mining and prognostic capabilities is demonstrated by the representative organizations highlighted in this paper. These organizations are multibillion dollar enterprises with massively complex, highly automated industrial operations that depend upon world-class analytics to provide powerful fleet management analytics and failure predictions to effectively manage their global fleets of complex, mission-critical systems and platforms.

Data Mining and Predictive Modeling for CBM

A capability critical to effectively supporting CBM is data mining and prognostic/predictive modeling – the essential ability to actually build prognostic models that provide useful prediction of failures and performance degradation. Data mining and prognostic/predictive modeling is a process encompassing a range of mathematical and statistical techniques dealing with the collection, classification, exploration, analysis and interpretation of data to gain insight, reveal patterns and anomalies and identify key inputs and relationships in order to predict some event or outcome of interest. In the case of CBM, that event of interest is equipment or system failure or some other undesirable degradation in performance. While easy to say, simply achieving meaningful and useful predictions of this sort are no trivial matter. In addition to the many important statistical and mathematical methods that can be applied, domain knowledge of the systems being analyzed is equally important, along with their design attributes and the nature of how they are used in the field to most effectively develop effective prognostic algorithms.

Figure 1. The overall flow of data through analysis and model development, with subsequent deployment for operational prognostics and fleet management.

2

■ Equipped with modern predictive

maintenance capabilities, military

logistics and sustainment

organizations can make better-

informed, fact-based decisions

regarding specific maintenance and

logistics actions using data-driven

analytic models.


Revealing and quantifying hidden, implicit relationships in sensor data

Very often an important phase of prognostic model development is applying the appropriate statistical and mathematical methods to extract and uncover hidden, implicit relationships between sensor readings and resultant asset performance that may provide insight into the behavior of the equipment and how subcomponents may interact to influence the performance and reliability of the overall system. This sort of exploratory data mining is a key phase in CBM model development, as very often effective and useful prognostic models require much more complex analytic approaches than “trending” and typically require data from more than a single sensor. Our extensive experience with data mining reveals that, in most cases, a specific combination of readings from a specific set of multiple sensors combined in a certain sequence or grouping is the foundation upon which useful prognostic and predictive models are built. Leveraging comprehensive statistical and mathematical tools, modern predictive maintenance solutions provide the ability to extract information hidden deep within the massive volumes of data stored in CBM data repositories to inform and enhance prognostic model development.

Revealing and quantifying relevance and statistical significance of sensor data

Another important phase of prognostic model development is analysis of sensor data to reveal which discrete sensor(s) capture data with relevance and statistical significance to the undesirable behavior being modeled. Our experience has shown that many sensor data feeds provide absolutely no predictive value to prognostic models whatsoever. Armed with this information, system designers and engineers can decide to optimize and simplify how data flows through the entire CBM analytic data stream. Given the right insight, system engineers may choose to reconfigure certain sensors to maximize the value of data being captured or even eliminate some sensors completely (i.e., optimize sensor configuration) or to make enlightened decisions about how data should be sampled or summarized on the platform or moved downstream to reduce the data volumes being captured or transmitted where appropriate (i.e., optimize sensor data extraction, transmission and storage).

In addition, based on what we have found in analyzing a multitude of complex systems in a range of different industrial domains, it is quite likely that the capture and flow of data from the embedded sensors will never be 100 percent perfect – there can be anomalous readings, dropped data packets, misconfigured sensors and so on. In this case, there are a number of important statistical tools that can be useful to understand the impact of this imperfect data on downstream CBM analysis and can actually be used to fill in or fix the data anomalies as appropriate. These sensor-data analytic insights are important to any CBM effort, as without them there is a chance that current CBM initiatives will repeat the mistakes of early CBM efforts that collected far too much extraneous data, with a majority of the data collected providing very little, if any, value to the development of useful prognostic models.

3

■ Leveraging comprehensive statistical

and mathematical tools, modern

predictive maintenance solutions

provide the ability to extract information

hidden deep within the massive

volumes of data stored in CBM data

repositories to inform and enhance

prognostic model development.


Example data mining model types for CBM model developmentAssociations Analysis Memory Based Reasoning

AutoNeural Neural Networks

Cluster Analysis Partial Least Squares

Decision Tress/CHAID Principal Components Analysis

Dmine Regression Rule Induction

DMNeural Sequence Analysis

Ensemble Support Vector Machines

Gradient Boosting Two Stage Models

Linear Regression Variable Clustering

Link Analysis Variable Selection

Logistic Regression Text Mining

Figure 2. The range of data mining model types available in modern predictive maintenance solutions.

With the rich set of statistical and mathematical tools found in modern predictive maintenance solutions, CBM efforts will be well-equipped to ensure that the CBM data repository, which will undoubtedly continue to store increasingly massive amounts of sensor data, will provide as much prognostic value as possible.

Enhancing prognostics by mining unstructured narrative data

Another interesting and fairly new phase of prognostic model development is analysis of unstructured text data, known as “text mining.” Such data – commonly found in the form of logbooks, maintenance-action comments, operator feedback and incident reports, corrective-action notes, and other forms of narrative text stored in various systems of record – can provide incredibly useful predictive insight into equipment performance.

Based on extensive text-mining experience, we have discovered that many unforeseen reliability issues and emergent failure modes are often first revealed through analysis and mining of such narrative text data. Discoveries made this way often occur weeks and sometimes months before issues are widespread enough to become apparent through traditional fleet reliability management processes. Equipped with innovative text-mining capabilities as found in modern predictive maintenance solutions, CBM efforts can be well-positioned to use any unstructured data sources to maximize the predictive insight they may offer.

4

■ With the rich set of statistical and

mathematical tools found in modern

predictive maintenance solutions,

CBM efforts will be well-equipped to

ensure that the CBM data repository,

which will undoubtedly continue to

store increasingly massive amounts

of sensor data, will provide as much

prognostic value as possible.

■ Equipped with innovative text-mining

capabilities as found in modern

predictive maintenance solutions,

CBM efforts can be well-positioned

to use any unstructured data sources

to maximize the predictive insight

they may offer.


Rapid model development and validation

Arguably, the most important and most difficult part of prognostic model development is the creation of the actual predictive algorithms. This is a field of study rich with possibilities, but it’s important to ensure your organization has an analytic technology with the richest model library, a mechanism to assist in the creation of optimized predictive models for the various model families, and a streamlined process to evaluate/compare alternative models to establish the best predictive model for a particular outcome being analyzed. In predictive modeling, a major challenge is awareness of all potential models that are available and easily establishing which specific model – for example a neural network, a Bayesian network, a logistic regression or a decision tree – or combination of models will provide the best predictive performance. In this phase of modeling, modern predictive maintenance solutions can provide CBM analysts with the ability to rapidly create, evaluate, optimize and compare candidate prognostic models, enabling the selection of the best model that provides the strongest predictive power.

Figure 3. An example analytic data flow to develop, refine, evaluate and compare candidate predictive models.

5

■ Modern predictive maintenance

solutions can provide CBM analysts

with the ability to rapidly create,

evaluate, optimize and compare

candidate prognostic models,

enabling the selection of the best

model that provides the strongest

predictive power.


Model lifecycle management

The last and most often overlooked area to consider in developing prognostic models is the idea of the prognostic model’s life cycle. It is important to recognize that a model’s life cycle exists and that there is a need to manage all its stages, including conception, deployment, performance monitoring and review, and retirement. The performance and reliability characteristics of all systems constantly change over time. Equipment will age and experience numerous maintenance cycles and upgrades. Systems will see the introduction of new technologies and subsystems. Configurations will change and adapt to support new roles and exploit new capabilities, and systems will be used in new and ever-changing ways by the inventive war fighters found in today’s military organizations. As such, the prognostic models must continue to evolve and adapt along with the system. It’s important also to understand the size of the model management challenge, as most CBM efforts will need to manage the life cycles of hundreds and likely thousands of models.

Figure 4. The progress of prognostic models, from development/refinement to deployment, including ongoing model feedback.

Our experience has shown that comprehensive model lifecycle management, such as that found in modern predictive maintenance solutions, can provide secure, centralized mechanisms for storing, organizing and documenting models which automate and streamline what is typically a manual, tedious and often error-prone process.

6

■ Comprehensive model lifecycle

management can provide secure,

centralized mechanisms for storing,

organizing and documenting models

which automate and streamline what

is typically a manual, tedious and

often error-prone process.


7

Enhancing fleet management metrics with predictive modeling

Metrics and reporting are a key component of effective fleet management. Examples of metrics include readiness rates, system availability/uptime, equipment utilization rate, mean time between failure, mean time between system aborts, mean time to repair, ratio of maintenance hours to operational hours, unscheduled maintenance, maintenance utilization rate, etc. Metrics such as these are essential to support even the most basic fleet management efforts; however, they all have a serious limitation: All are historical, backward-looking metrics that tell us only what has already happened.

Historical metrics are good at telling a fleet manager where the fleet is today and where it has been in the past, but offer little insight into where the fleet will be tomorrow or next week or next month with respect to availability or readiness. While trending lines projected into the future may suggest where the fleet is headed, such approaches provide no confidence, no analytic rigor, and certainly lack the ability to actually predict where the fleet’s performance will be in the future. This is where modern predictive maintenance solutions can dramatically enhance the effectiveness and insight gleaned from fleet management metrics. Using statistical and mathematical techniques, predictive fleet management models can be developed to provide a forward-looking capability with enhanced metrics that predict when and where critical performance metrics drivers and other factors are likely to move in the future. Armed with predictive fleet management metrics, defense logistics and sustainment organizations can make fleet management decisions with forward-looking predictive metrics based on fact-based, data-driven prognostic models.

■ Armed with predictive fleet

management metrics, defense logistics

and sustainment organizations can

make fleet management decisions

with forward-looking predictive metrics

based on fact-based, data-driven

prognostic models.

8


Customer Case Examples

Major North Sea oil and gas producer

Partnering with SAS, a major petroleum producer successfully deployed an advanced analytic solution and developed prognostic models that provided early indication of emerging systemic performance degradation and impending equipment failures in highly complex and mechanized industrial processes. The prognostic effort started with highly complex low-pressure separation equipment, which features massively intricate and not-well-understood processes. A range of advanced data mining and predictive modeling capabilities enabled this organization’s engineers to better understand and model these complex processes and to create prognostic models that provide early signals and alerts of upcoming problems. These prognostic models feature inputs from dozens of sensors capturing data continuously at various points during complex processes, with the model interpreting the data in real time to provide data-driven, condition-based event prediction.

Additional detailed information can be found in the following white paper, available from the SAS website: http://www.sas.com/resources/whitepaper/wp_6195.pdf.

Northern European natural gas production partnership

Two global natural gas producers based in the Netherlands have created a joint venture to operate the extraction and production of natural gas from Europe’s largest gas reservoir. This joint venture has partnered with SAS to implement a predictive maintenance and fleet management solution that:

• Integratesamultitudeofdatasourcesandsystemsalongwithmassiveamountsof real-time sensor data into a unified prognostic data warehouse.

• Preventsunplannedshutdownsanddowntimewithprognosticmodels.

• Optimizesequipmentoperationalefficiencyusingadvancedoperationsmodels.

• Developsaccurateequipmentprojectionsusingstatisticalandmathematicalforecasts.

• Providesfleetmanagementequipmentperformancemetricsandreports.

Additional detailed information will be available for download in 2011 from a SAS white paper that is currently under development.

http://www.sas.com/resources/whitepaper/wp_6195.pdf


9

Military aviation fleet management

A US military service has partnered with SAS to develop an advanced analytic platform that enables its aviation maintenance and sustainment organization to integrate data from disparate systems and develop advanced analytic models to gain unique perspectives on fleet operations, asset utilization and maintenance plans. The system allows maintenance and sustainment organizations to measure cost of operation and fleet performance statistics in a unified way to increase efficiency, improve value, identify significant problems and have ready access to mission-critical information.

Additional detailed information will be available for download in 2011 from a SAS white paper that is currently under development.

Summary

As illustrated by the experiences learned and the results achieved in the customer case examples, modern predictive maintenance solutions can provide unified data preparation, world-class prognostic/predictive modeling, and robust information sharing capabilities that are critical to successful CBM initiatives. These solutions, when combined with proven analytic methodologies and processes, can provide CBM initiatives with advanced analytic models that yield data-driven insights into the nature of equipment behavior and fleetwide equipment performance, enabling military logistics and sustainment organizations to make better-informed, fact-based decisions regarding specific maintenance and sustainment actions.

10


About SAS

As the leader in business analytics, SAS helps organizations understand their business drivers and create answers to complex problems. Our mission is to deliver superior software and services that give people the power and insight to make the right decisions. SAS eliminates the complexity of sharing data and applications across the organization. SAS goes beyond other vendors’ narrow definitions of business intelligence, offering business analytics – data management and predictive analytic capabilities that tell an organization not just where it has been, but where it should go next. Founded in 1976, SAS is the largest privately held software company in the world with 2010 annual revenue in excess of US$2.43 billion. SAS serves more than 50,000 government, university and private-sector business sites in over 127 countries, including 93 of the top 100 companies on the 2010 Fortune Global 500® list. SAS is used extensively by all 15 federal departments and approximately 85 percent of federal sub-agencies and quasi-governmental affiliates.

SAS Institute Inc. World Headquarters +1 919 677 8000To contact your local SAS office, please visit: www.sas.com/officesSAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. Copyright © 2011, SAS Institute Inc. All rights reserved. 105134_S71846.0411

Executive SummaryData Mining and Predictive Modeling for CBMRevealing and quantifying hidden, implicit relationships in sensor dataRevealing and quantifying relevance and statistical significance of sensor dataEnhancing prognostics by mining unstructured narrative dataRapid model development and validationModel lifecycle managementEnhancing fleet management metrics with predictive modeling

Customer Case ExamplesMajor North Sea oil and gas producerNorthern European natural gas production partnershipMilitary aviation fleet management

Summary About SAS

Documents

Data Mining and Predictive Modeling for Condition-Based ......traditional fleet reliability management processes. Equipped with innovative text-mining Equipped with innovative text-mining