Future Directions in Health Monitoringdavidc/pubs/ehm_ch7.pdfused in manufacturing operations is already taking place. For example, use of control charts to determine if a manufacturing

177

7Future Directions in Health Monitoring

7.1 Introduction

It has become very much of a cliché in referencing the significant develop-ments in recent years in the areas of computing and in general that of electronic based technology. {AU: edit for sense} However, it is worth considering for a moment the disruptive effect these developments have had on how we interact with modern technology. Consider how music has been consumed over the past 20–30 years. We have moved from purchasing music on vinyl records, to using magnetic tape in the form of cassettes, to building a collection of audio CDs, then listening to playlists in the form of digital MPEG 3 files on a com-puter disk; and now consuming music in the form of live streaming of audio content via broadband or its 3G/4G mobile equivalent; all in short amount of time. This pattern can be observed in other media, such as video where there is a move away from owning DVDs to again streaming video content over broad-band via some form of subscription service. Our mobile phones have also developed beyond the simple capability of making a phone call. It is now possible to video chat with someone with reasonable quality and clarity; drive to new destinations without use of a map, but instead be guided on the best route that’s decided from the built in sensors that identify our current position combined with information of traffic conditions in the immediate area and the roads ahead.

Figure 7.1 shows an image of the Raspberry Pi which is a low-cost linux based computer offering a quad-core ARM Cortex-A53 processor running at

king_CH07.indd 177king_CH07.indd 177 8/10/2017 2:52:53 PM8/10/2017 2:52:53 PM

178 Equipment Health Monitoring in Complex Systems

1.2GHz supported by 1GB ram. Incorporating wifi, Ethernet, a 4-port USB hub and 40 pin general purpose input/output (GPIO) port, it retails for around £30 and is very compact. Although mainly used for hobby applications, it repre-sents the computing power that would have been regarded as a decent desk-top computer specification a few years ago, albeit contained in a smaller footprint. The flexible interface demonstrates how such versatile small computing cards can be packaged and hence the ease with which machines and other devices can become network-enabled and integrated into a connected environment.

Obviously, these trends are not limited to the domestic setting. Globally manufacturing is seeing a significant level of R&D funding being made avail-able for investment in driving development of Industry 4.0. This is seen as the current trend in automation and data exchange in manufacturing technologies leading to what is regarded as the smart factory. Arguably, the rapid growth of data storage, improved network communications, and greater use of analytics for business intelligence has inspired manufacturing companies to invest in new ways to monitor all stages of the manufacturing process.

Indeed, there are many examples where health monitoring of machines used in manufacturing operations is already taking place. For example, use of control charts to determine if a manufacturing process is operating robustly and effectively is a well-established technique in the process monitoring industry. In addition, monitoring the health of cutting machines (as in drill wear monitor-ing) is often used to maximize the life of machine tools while ensuring the qual-ity of finish of the final component. However, increased connectivity within the smart factory enables monitoring at a much higher level, where it increases agility in controlling production processes, and forecasting machine down-time (along with other operational benefits) can be achieved.

In this final chapter we offer a view into the near term developments in the areas of sensing technologies, big data, and directions of advanced analytics approaches in the context of health monitoring.

Figure 7.1 Photo of Raspberry Pi.


Future Directions in Health Monitoring 179

7.2 Emerging Developments Within Sensing Technology

Equipment state can be better assessed and predicted if the following health factors are known with minimum uncertainty:

• Failure mechanism rates;

• Failure symptoms magnitude;

• Functional performance measures;

• Operational and environmental context—which may be cause the fail-ure mechanism rates.

The utility in sensing the above factors is driven by a combination of tech-nology push and pull factors. For example, as technology evolves, an increase may be seen in the capability to directly measure the failure mechanisms or to increase the diversity of environmental parameters. Technological advances and market forces also may drive down the cost to measure changes in the health related measures. In addition, as an asset ages or its usage changes, it may be-come increasingly important to perform better health management; thus, the opportunities should be regularly assessed.

To illustrate this principle consider the degradation of oil. This complex process caused by thermal stresses and oxidation, causes changes in the function-al or chemical properties (see Figure 7.2). Monitoring such degradation might be performed by monitoring the functional change in lubrication properties (e.g., viscosity, surface friction), symptoms of faults (e.g., acidty change) or the failure mechanisms (e.g., depletion of additives). These changes can be assessed through a variety of means. In the past, automotive and aerospace applications both saw the natural loss of oil and thus periodic renewal was required, negat-ing the need to monitor quality. Newer applications, where improved sealing technology and higher stress applications, such as those introduced in geared turbofans, mean that oil quality is increasingly important to monitor.

The main functional parameter representing oil condition is its viscos-ity, which also changes with temperature and water content. Laboratory based techniques (e.g., rheometers) typically assess the absolute viscosity of oil by the damping effect on a dynamic process such as timing a dropped ball, or the forces on a rotating shaft, and kinetic viscosity via measuring flow rate out of a defined capillary tube.

Existing sensing technologies are numerous, with well-known methods for monitoring position (hall-effect, encoders, LVDT), temperature (thermo-couples, RTDs), vibration (piezoelectric, laser displacement), and so on. The cost, reliability, and accuracy of these classical sensing methods are well-known. Any member of this family of sensors could be the correct choice for a health



monitoring application, but the practitioner should also be aware of the rap-idly developing capabilities of advanced sensing. This section reports emerging trends and recent advances in technology that should be noted when consider-ing a health monitoring system design.

The trends in sensor development are characterized below as low-cost and miniaturized versions of (generally) large-scale sensors, ultra-small-scale (where different behaviours emerge from the physics micron-scales) and biological sen-sors that harness understanding and materials from the natural world.

7.2.1 Low-Cost and Ubiquitous Sensing

The miniaturization and integration of electronics and mechanical struc-tures has given rise to a range of sensor technology commonly referred to as microelectromechanical systems (MEMS). Mostly constructed in semiconductor material using standard integrated circuit manufacturing techniques, it has been possible to produce high-volume and low-cost sensing solutions. The mar-ket is dominated in volume by consumer applications for microphones, accel-erometers, and gyroscopes. Low fidelity applications in industry are also com-

Figure 7.2 Representation of oil degradation process.



monplace, for example, tire pressure sensing, magnetic position sensing and strain gauges. The challenges in reliability and measurement drift are becoming better understood [1] and an increasingly wide class of industrial applications are becoming prevalent, particularly when integrated in smart sensors. Not-withstanding concerns over use in safety critical environments, the digital bus output, auto-calibration, and diagnostic capabilities of integrated transducer and electronics technology in the form of smart sensors is a growth area. As a consequence, access to point measurements of symptoms, device performance, and the operating environment for assets is increasingly commonplace.

In personal healthcare, wearable devices such as Google Glass, FitBit and the Nike+ FuelBand are representative of the opportunities to take advantage of advanced sensing to create new markets. Electrically-conductive polymers, ul-trathin inorganics and organic semiconductors have enabled flexible electronic sensor systems that can conform to patient everyday use. Using state of the art fabrication techniques, substrates and circuits approaching microns in thick-ness, and bending radii with weights of weighing less than 1 mg/cm2, electronic devices can potentially be truly imperceptible.

Activity monitors are means to measure the functional performance of a patient. Largely overlapping with industrial sensing modalities, accelerometers, gyroscopes, force and pressure sensors, can provide more detailed insight into patients’ movement characteristics, such as gait, falls, tremor, and dyskinesia.

A patient’s physiological condition can give insight to the failure mecha-nisms and symptoms of emerging failure. Physical and electrical measurements can be made, including heart rate, breathing rate, blood oxygen saturation (us-ing microfluidic sensing), and blood pressure. Chemical analysis is emerging using semiconductor based lab-on-a-chip approaches. Using changes in electri-cal properties, real-time sweat analysis can provide information on pH, electro-lytes and hydration levels, for example.

Environmental (i.e., operating context) monitoring of hazards is also made more feasible through low-cost MEMS. Wearable sensors able to detect exposure to contaminants (such as explosives or high concentrations of toxic gases) by monitoring of pollutant through IR absorption or photo-acoustic phenomena integrated into MEMS packages {AU: edit for sense}. Techniques relying on optical property changes represent a class of sensors exploiting active and passive electromagnetic and acoustic waves.

Thermal imaging in the long-wave electromagnetic spectrum has dramat-ically reduced in cost in recent years, and provides opportunity for wide area sensing from a single device. Health monitoring opportunities are expansive for IR sensing, particularly when low-cost devices are used, these include:

• Fluid/gas leak incipient failure detection;



• Multipoint sensor for temperature measurement assuming compensa-tion for component emissivity;

• Friction or wear symptom detection;

• Electronic and electrical systems symptoms of failure (e.g., electronic component overheating, electrical arching, electrical machine winding faults);

• Heat transfer rate changes can give indications of fluidic flow rate through a channel (thus indicate blockages, etc.), as well as the thickness of deposits or film (detect onset of injector blocking);

• Environmental changes including hazards such as fi re detection.

There are several candidate technology classes for low-cost thermal imag-ing: photon-detectors, uncooled bolometers, and thermos-pile arrays.

The photo-active element of a photon-detecting sensor (and front-end filtering) determines the sensitivity to different wavelengths including UV and IR. Near-IR is emitted from bodies with temperatures above several hundred degrees Celsius and this can be detected with standard silicon photo sensors. For shortwave IR, detection is possible using InGaAs photo-detector materials. To detect emissions from bodies at cooler temperatures, (longer wavelengths [7.5 to 14 um]), extensive cooling of the photo-active elements is tradition-ally employed making the technology unsuitable for low cost applications. On the other hand uncooled (resistive) micro-bolometers, require no complex and expensive cooling methods, making the devices much more practical. A single pixel is constructed from a thermally isolated infrared absorbing material such as amorphous silicon and vanadium oxide. Measuring resistance change in an array of pixels can be used to create an image. State of the art pixel size is now at 12 µm in arrays of up to one million pixels. Thermopile arrays use the pyro-electric effect to generate a voltage proportional to the incident heat radiation. It is noted that there exist robust higher temperature variants in single sense ele-ments—this is hardly surprising since these can be thought of as thermos-cou-ples connected in series. Future developments include quantum wells, carbon nano-structures, and SOI technology, as regularly reported at the conferences of SPIE, the international society for optics and photonics.

The detection of photons can also be exploited for material fatigue esti-mation. Second harmonic generation occurs where photons incident on a ma-terial are reflected back at a higher frequency, where the proportion of conver-sion can be affected by material conditions. One example of the phenomena’s use in health monitoring is lasers shone on metal that produce reflections at high-frequency ultra-violet dependant on material condition.



A further example of electromagnetic spectrum exploitation is in fiber-optic sensing. A fiber-optic has the inherent advantage of allowing signals to be carried over high-temperature, noise immune, low weight and volume con-duits to a benign environment for electronic processing. As such, applications in oil and gas, as well as high-temperature propulsion, are well suited to this technology. The sensors may be intrinsic, where the fiber itself responds to the physical environment, or extrinsic where the fiber acts only as a communication medium (e.g., visual and infrared imaging). For intrinsic sensing, as well as ex-ploiting the changes in the optical material (such as increased scattering at high temperatures), often Fibre-Bragg gratings (FBG) are integrated at one or mul-tiple points along the cable. FBG are sensitive to interference pattern changes that occur since the grating element size is affected by temperature, pressure, and strain of the structure to which it is mounted or embedded in. The design of FBG to have different frequency response characteristics allows multiple si-multaneous measurements to be made; such as in distributed acoustic sensing.

Passively, high-frequency acoustics (acoustic emission) has been used to detect rub, material micro-cracking, and the flow of turbulent liquids due to leaks. In addition the potential for active ultrasonics for sensing is a growth area. Ultrasonic impulses can be used for position measurement (acting as an echo-locator), and the more sophisticated integration of piezoelectric transduc-ers and their excitation can infer the interface properties such as viscosity and oil film thickness.

Surface acoustic wave (SAW) devices consist of two interlocking comb-shaped arrays of metallic electrodes deposited onto a piezo-electric substrate, one acting as a source, the other as a receiver. The transmission of the surface wave generated by the source is modulated by physical phenomena affecting the received amplitude, phase, frequency, or time-delay. Pressure, strain, torque, and temperature acting on the substrate between the source and receiver cause the distance between to change resulting in a shift in phase. The mass of de-posited contaminates (e.g., fuel deposition) on the device can be detected by a change in resonance or wave propagation speed. The addition of other material sensitive to chemical or magnetic phenomena into the transmission path can affect the surface wave in a similar way. Generally, the devices can be locally ex-cited and measured, or activated by an external field stimulus in a passive mode. The removal of electronics in this passive mode allows very high temperature operation making products for gas turbine blade temperature measurement in excess of 1,000 degrees Celsius possible.

Patch antennas can also be used to sense changes in an underlying sub-strate. Expansion of the antennae or resistivity changes in the material substrate of the antennae changes the Q-factor and distorting the reflected RF signal [2]. Temperatures >1,280K have been reported.



As MEMS technology advances mm^3 sensors start to push the boundar-ies of the fabrication processes.

7.2.2 Ultra-Minaturization—Nano and Quantum

Advancing on miniaturization of MEMS sensors such as accelerometers, which, while small, rely on familiar mechanics to large-scale sensors, nanoscale and quantum sensing rely on principles only observed at the atomic or subatomic scales.

A demonstrated application of nanoparticles is seen in protective coatings with corrosion-sensing (e.g., nanocontainers of quantum dots). The increase in acidity, a typical symptom of a corrosion process, affects the photoluminosity of the quantum dots. When a stimulating radiation source (e.g., a tuned laser) is directed at a quantum dot the frequency (e.g., color) of the reemitted radiation may be measured optically. Some practical problems with the technology exist; not least, the use of Cadmium is generally a concern for the technology.

Nanomaterials, such as carbon structured as nanofibers or nanowires or in 2-D as graphene, are being extensively investigated for their sensitivity and ease of assembly, along with the remarkable response of their electrical and me-chanical properties as they interact with other materials. These nanomaterials are increasingly being utilized for various sensing applications, including bio-molecules, chemicals, mechanical strain, and pressure. Moreover, the biocom-patibility of graphene opens up further possibilities for its use as a fundamental element of implantable biophysical sensors. In one example, researchers have combined a MEMS gas chromatograph (spreading different chemicals over time) with nanotube receptors (to measure chemical quantities). The promise of full fidelity, online chemical decompositions is an emerging possibility, in both patients, and industrial machinery.

At a yet smaller scale, quantum phenomena have been laboratory demon-strated to offer sensing capability. A UK quantum sensing [3] institute envisage maturation to industrial and commercial markets over the next 10–20 years. Quantum sensors use a variety of properties of the small-scale world to enhance measurement, for example:

• Atoms to sense changes, since the atoms can be controlled and measured precisely;

• Quantum superposition is highly sensitive to the environment, thus can be used as the basis for precise sensors—for example, an atom interfer-ometer.

• Atoms embedded in materials such as diamond and silicon, are particu-larly suited for magnetic sensors.



• Photonic sensors can detect the optical properties of molecules to mea-sure, for example, faint chemical traces.

The potential for the technology is to outperform (up to 1,000-fold) con-ventional sensors in the measurement of: gravity (e.g., pipeline leak localisation and tomography), rotation (rotating machine speed fluctuation), clocks (data alignment), magnetism (flow sensing, brain imaging, material defects), and im-aging (multispectral photon detection). The promise is also there to diagnose dementia and heart disease from magnetic field changes and cancer through increasing the sensitivity of mm wave technology.

7.2.3 Bio-Inspired

The natural tendency for human designed artificial sensors is to replicate na-ture: simple examples, such as microphones, echolocation, electronic noses, cameras and temperature, abound. Often practical alternatives to biological electrochemical responses are used based often on the purely electrical response of materials to stimulation. The wholesale use of natural systems, like a caged canary in a mineshaft, or lichen growth as a sign of air quality is a most basic example of sensory exploitation of biological systems. This section explores the emerging field of drawing inspiration from nature and the integration of natu-ral systems into artificial sensors.

Depending on the method of signal transduction, biosensors can also be divided into different groups: optical, thermometric, electrochemical methods based on voltammetry and impedance, or magnetic.

Optical biosensors are the most commonly reported class of biosensors. The detection typically relies on an enzyme system that catalytically converts the substance to investigate into products that can be oxidized or reduced at a working electrode. The generated electrical potential is then measured to infer properties of the material. An optical biosensor is a compact, and potentially low-cost, analytical device. It is composed of a biological sensing element, inte-grated or connected to, an optical transducer system. An example is the use of yeast cells to detect a TNT-associated compound when the cells turn a fluores-cent green. Electrochemical effects are demonstrated in a contact lens contain-ing biosensors to measure and monitor a plurality of physiological symptoms. The biosensors generate an electrical change in their semiconducting substrate: indium gallium zinc oxide (IGZO). IGZO proves useful in testing blood glu-cose levels for diabetic people. Glucose oxidizes changing the acidity at the sensor surface affecting the electrical current in the IGZO transistor. The tech-nology promises the ability to place 2,500 biosensors in a 1 mm square patch of contact lens; each monitoring a different bodily function. It is clear that chemi-



cal sensing of a wide variety of failure mechanisms seems a promising area for these sensor types.

In recent years, advances in systems biology have allowed a closer explora-tion of biological sensory systems. The sensory systems from nature have been used as the inspiration of sensing modalities as diverse as vibration to tem-perature, often implemented in a nonbiological manner. It is recognized that a spider senses vibration and stress through the interlocking and separation of zip-like slits on the surface of their leg joints, presumably exploiting the discon-tinuous impulses as the slits open and close. Nonlinearity is a common feature of nature that contributes to complex behaviour emerging from simple struc-tures. The bioinspired sensor uses conductance changes as slits in a platinum layer open and close to measure strain. Increasing sensitivity and reducing cost could be an ultimate outcome.

Reuse of fundamental building blocks for different purposes is a key facet of nature. Hair cells, for example, provide a variety of sensing abilities for differ-ent animals: they help humans hear, they help insects detect vibration, and they form the lateral line system that allows fish to sense the flow of water around them. The reuse and processing of sensory input is a potential area for future application.

7.2.4 Summary

Sensors that are cheap and simple to integrate are revolutionizing the availabil-ity and breadth of data suitable for assessing the performance and environment of patients and industrial machines. As novel arrangements of MEMS devices are being developed and the atomic scale is being exploited, the sensing of a wider range of failure symptoms and mechanisms is becoming ever more pow-erful possible. The increased sensitivity from biologically-inspired and quan-tum-sensing systems suggests the very early-warning signs of emerging failure will become increasingly measureable. The exploitation of high-dimensional, continuously supplied data, has the potential to add value only if it is handled and analysed effectively. Analysts desperate for more data may rue their wishes.

7.3 Sensor Informatics for Medical Monitoring

With the ever-increasing availability of wearable physiological monitors for consumers, it might be surprising that very few devices exist for tracking the health of ambulatory patients. Most patients in a hospital are not in the Inten-sive care unit; many hospital patients are encouraged to walk around, to speed their recovery on the ward. Likewise, few examples exist of wearable monitors being used in the home to care for patients with long-term conditions. In both



hospital and home environments, there has been very little penetration of wear-able devices into actual practice [4, 5].

Wearable sensors offer the possibility of ubiquitous monitoring of pa-tients by being unobtrusive.The authors have found, unsurprisingly, that pa-tients tend to dislike wearing obtrusive conventional medical sensors (such as finger-worn pulse oximeters or chest-mounted ECG electrodes), and often re-move them—even if their health is at stake [6]. Certainly, it would not be fea-sible to ask patients to wear such sensors for prolonged periods. Thus, there is a great need for wearable sensors that accurately monitor physiology for the sick and elderly patients that would use them.

However, a key reason for the lack of use of wearables in healthcare is the lack of robustness of such sensing systems when used with patients, rather than with consumers: while the latter tend to be young and healthy, the former tend to be (on average) elderly and unhealthy. The physiology of the latter can be very different to that of the former in the manner in which it affects the waveforms acquired via transducers in medical sensors. For example, re-spiratory rate is one of the most important vital signs to observe to identify physiological distress, and is acquired by wearable sensors through the PPG (photoplethysmogram).The acquisition of this latter waveform involves shin-ing an LED on the subject’s skin, with sensors commonly found in wrist-worn wearable sensors and smart watches. The act of breathing modulates the PPG in a number of ways, and these are typically obvious in their manifestation for young and healthy subjects. However, the physiology of elderly patients typically degrades the strength of some or all of these modulations, and this degradation varies from patient to patient according to the changing health of the individual. Hence, algorithms in consumer devices that calculate respiratory rate from the PPG tend to work well for the consumer market for which they are intended,but they are insufficiently reliable for use in healthcare.

A key step to achieving the goal of wearable reliable sensors for use with patients is to capitalize on the rapidly-growing field of medical sensor informat-ics. Repositories of medical data acquired from mobile sensors are growing in size and quality, and are reaching the stage where new algorithms can be con-structed that depend on complex probabilistic models for tracking physiology [7]. Such models can take advantage of the benefits of, for example, Bayesian inference by coping with the noise and artifact that are commonplace with wearable sensing. On-going research into medical sensor informatics aims to develop systems that can distribute inference for handling wearable sensor data; this distribution of inference involves processing data (i) on the device itself; (ii) on the patient’s smartphone (or similar device dedicated to the patient), which communicates frequently via Bluetooth with the wearable; and (iii) on the hospital cloud, which communicates via Wi-Fi with the patient’s smartphone. Moving from (i) to (iii), inference may take place using greater computational



resources, and therefore the complexity of models used to process the data may increase as one moves from wearable, to smartphone, to cloud. However, to maximize battery life, there is a desire to transmit from the wearable only neces-sary data; most patients are stable most of the time, during which times data need not be streamed from the wearable to the cloud [4]. This notion of bal-ancing inference between the sensing device and the cloud is at the heart of the coming generation of wearable sensor informatics for healthcare, and offer the potential for augmenting human-acquired gold standard data with more frequently-acquired data from automated, wearable systems [8].

7.3.1 Deep Learning for Patient Monitoring

The majority of medical applications discussed in this book so far have involved time-series analysis, and the need to cope with noise, artifact, and corruption of data that occurs as a result of patient movement and deteriorating physiology. The branches of time-series analysis that have made significant progress in the medical applications have, to date, often been based on probabilistic inference, and Bayesian inference [9, 10], which are methods well-suited for performing inference with noisy and artifactual data. However, the nascent field of deep learning offers a different approach, and is already transforming some fields of medical informatics.

Within recent years, the convolutional neural network (or convnet) has become the standard for many image processing applications [11]. These net-works are similar to previous generations of neural networks (NNs, which peaked in popularity in the 1990s), but which include convolutional layers that filter local regions of the image, and which have fully-connected layers that learn representations of data at ever higher levels of abstraction. First demon-strated for very large-scale classification of images, these models have improved their classification accuracy to the point at which they can outperform humans for large-scale image classification. The success of such an approach depends on the exploitation of very large numbers of hidden layers (and which therefore have many millions of free parameters) to build classification capacity. With such large numbers of parameters, extremely large training sets are required. Various applications in medical image processing have arisen, and the use of convnets can now be considered standard practice in such fields.

While the monitoring of patient physiology can sometimes touch on image processing [6], the majority of the field is closer to the field of signal processing, and of time-series analysis as outlined above. Different varieties of neural network architecture have been proposed over the years for tackling such tasks—often involving recurrent neural networks (RNNs). Shallow versions of the latter were well-understood during the first wave of NNs, and which, as with convnets, are experiencing a renaissance as deep RNNs [11]. The recurrent



neural network, as the name suggests, presents time-lagged versions of sequen-tial data (which are often time-series) as inputs to the network, and where units link the analysis of these different lagged versions of the signal up to some maximum delay. For example, the most recent 16 samples of a sensor time-series might be presented to the 16-node input layer of an RNN. Examples of well-understood architectures in the literature for deep RNNs include the long short-term memory (LSTM) network and the gated recurrent unit (GRU) network. Again, as with convnets, the difference between such models and those from earlier generations of machine learning is characterized by the much greater number of free parameters in deep networks, and, therefore, the need to provide very large training sets to construct effective models.

RNNs are beginning to find application in medical monitoring; early ex-amples include the analysis of sensor ECG data, whereby the various morpholo-gies of the waveform pertaining to different cardiac conditions, may be learned by the network. However, such approaches are in their infancy, and their use at scale (and for use in real healthcare systems, beyond the limits of preliminary proof-of-principle studies) has yet to be demonstrated.

Perhaps some of the most interesting developments for medical informat-ics within the area of deep learning are those related to variational autoencoders (VAEs). The autoencoder was proposed in the 1990s as a network architec-ture that comprised an encoder (which mapped the input to a much lower-dimensional encoded state) and a decoder (which mapped the encoded state back onto a reconstruction of the input.) Noting that the encoded (or latent) state is a lower-dimensional representation of the input, this represents a coding scheme in which the encoded state aims to represent the latent structure of the data, such that the data can be reconstructed as accurately as possible from that encoded state. This may be seen as a dimensionality-reduction procedure, in which the aim of the autoencoder is to find that dimensionally-reduced latent representation of the data that captures the majority of features of interest in the data; ultimately, only the encoder is typically retained after training (because it represents the dimensionality-reducing mapping from input to latent space), the decoder is discarded. The advent of deep learning permits the encoder to comprise many layers, with many free parameters. VAEs [11] are an interesting model in which the latent state variables are multivariate Gaussian, and where the encoder seeks to generate the mean and covariance of that multivariate Gaussian.

Applications within image processing have been demonstrated using VAEs, but their application within the medical domain is, again, in its infancy. However, where there is great potential for automatic feature generation: the latent state variables, once obtained from a model trained using a sufficiently large dataset, can be used for subsequent inference—in the same manner as conventional dimensionality-reduction techniques might be used to preprocess



a training set. As medical datasets become larger, it becomes harder for humans to label the data to permit fully-supervised model construction to take place. Thus, complementary techniques for discovering features that are potentially relevant for separating data from, for example, stable and deteriorating patients will become increasingly useful. These methods represent an unproven future for medical informatics, but one that promises to permit the analysis of datasets that far exceed the limits of existing methods.

7.4 Big Data Analytics and Health Monitoring

Before discussing how big data analytics may influence health monitoring so-lutions, it is worth considering what big data is. One simple definition is that it is a term used for data sets that are so large or complex that traditional data processing methods (e.g., any one of analysis, data capture, searching, storage, transfer, retrieval) are inadequate to deal with them. It is estimated that our technological capacity to store information has approximately doubled every 40 months since 1980 (Wikipedia){AU: provide exact source info with reference number}. This trend can be easily seen in new higher resolution digital cameras coming onto the market every year, greater use in industry of wireless sensor networks, ever increasing technical capability of mobile devices, and increasing storage density, which doubles in capacity every 12 months with a net outcome of lower cost per byte.

7.5 Growth in Use of Digital Storage

At the time of writing, the web page at http://www.martinhilbert.net/world-infocapacity.html shows a cartoon similar to that shown in Figure 7.3 above, representing the estimated world’s technological capacity to store, communi-cate, and compute information by tracking 60 analogue and digital technolo-gies between the years 1986 to 2007. The corresponding publication [12] sug-gests that telecommunications has been dominated by digital technologies since 1990. From the crude growth curve in the above figure, the year 2002 can be taken as the reference point at which a significant level of data consumed glob-ally was no longer held in analogue form (that is, magnetic media such as video tape, vinyl, etc.). It also coincides with an initiative by the UK government to fund various e-Science projects related to grid computing [13]. The term is inspired by the electrical power grid which implements standards for electrical power transmission that:

• Allows for the decoupling of consumer and provider;



Figu

re 7

.3

{AU

: pro

vide

fi gu

re c

aptio

n}



• Provides a mechanism to link diverse providers into a managed utility.

The vision for the grid was therefore to provide a software infrastructure that would enable flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources including computational systems, data storage resources, and specialized facilities.

Much of the technology developed at that time now forms key compo-nents of what is often referred to as cloud computing with many elements avail-able as open-source solutions.

Cloud computing is networked-based often utilizing internet connectiv-ity. It provides on demand services anywhere, anytime such that consumers pay for use as and when needed with elastic capability. This flexibility refers to hard-ware and software services which can be scaled up and down in capacity and functionality depending on requirements. Cloud computing also provides a comprehensive framework to enable data streaming from remote connected de-vices (e.g., internet of things). There are a number of cloud providers each offer-ing services to the general public, corporate enterprises, and commerce alike. In the domestic market, most consumers will be familiar with data storage services offered by cloud providers for remote storage of digital photographs. How-ever, industrial consumers will be more interested in the flexible computing services that provide analytical support. Big data analytics involves the process of examining large data volumes to identify hidden patterns, unknown correla-tions trends, and other descriptive statistics that can provide useful insight from which business value can be derived. There are many industrial and commercial examples where such techniques provide value including, but not limited to: decision support, financial analysis, social media monitoring, telecoms network monitoring, traffic management, text mining, language translation, and so on.

Big data may therefore be considered to be high volume, often high veloc-ity and likely to consist of different data types (variety). These are often referred to as the three key V’s of big data. However, veracity is another important factor and relates to the quality of captured data, and its variation between sources, which can impact any subsequent analysis. Although considered disruptive technology, much of the analysis methods underpinning big data capability are based on established techniques (e.g., signal processing, machine learning/data mining, statistical methods). Scalability of computing resources provided by cloud technologies, and the mechanics of plumbing these tools together are key enablers for handling big data.

7.5.1 Example Health Monitoring Application Utilizing Grid Capability

Successful equipment health management of fleet-wide assets involves a reli-able flow of data and information. As discussed in Chapter 3, such data will



originate from several alternate distributed locations, some of which consist of monitored data direct from the asset, others relating to maintenance action, operators logs, strip reports, and so on. The totality of this data clearly has value in the support of tracking individual assets effectiveness in service operation and can also be utilized in understanding background problems when a fault requires investigation. When combined with information from other assets, this total collection of data offers an effective fleet-wide view supporting key decisions that maintains overall availability to all operators.

This section illustrates the use of a number of connected services, de-veloped under the BROADEN grid project, as an example of a fleet-wide health management capability operating in an early version of the cloud. The BROADEN project (Business Resource Optimization for Aftermarket and De-sign of Engineering Networks), funded by the Department of Trade and Indus-try (2005–2008), provided an example of how capabilities of grid technology, and hence cloud computing, could be harnessed to address many of the chal-lenges identified above. The project consisted of four work packages and was funded for a three year period; however, the relevance for this section is the work package that specifically focused on integrated diagnostics and prognostic tools for health monitoring. The project demonstrated how legacy applications could be effectively combined within a service environment utilizing web-service and distributed storage, such that engineers could initiate complex queries in a sim-plified manner, without any concern for where the data was stored, or the fact that data would migrate through the data supply chain initially from the asset and finally rest at a major data center. This capability was demonstrated within a pilot Gird environment as depicted in the Figure 7.4.

The pilot grid architecture (Figure 7.4) allowed rapid searching across vast distributed data repositories of engine condition monitoring domain data. The aim was to demonstrate the feasibility of seamless access to a variety of remote overhaul centers and development test-beds in such a way that the engineer does not need to be burdened with maintaining knowledge of where data is located at any given time. In this pilot system, three virtual centers have been defined representing an overhaul plant in an offshore country; a production pass-off bed located at engineering headquarters; and an engine fleet monitor-ing center located elsewhere in the United Kingdom. These are shown in the Figure 7.4 as distributed nodes 0, 1, and 2 respectively. Each of these nodes contains an identical set of grid components.

Facilities provided by the data orchestrator monitor the arrival of new en-gine data and initiate automatic processing for the detection of abnormal events, via the supporting XTO service which primarily extracts key features previously defined as relevant for engine diagnostic purposes and compares these against a series of data-driven models. Any detected abnormalities are then provided as a summary report available to any valid user via a portal interface.



Figu

re 7

.4

Pilo

t grid

dev

elop

men

t arc

hite

ctur

e.



A pattern match controller (PMC) maintains search operations at each node. This service reacts to incoming search requests such that all nodes process submitted queries in parallel. The PMC is also responsible for correlating the results, and returning them back to the relevant client. In this project, the focus is on the search capabilities offered by the AURA technology (Advanced Un-certain Reasoning Architecture [14–16]) however, the PMC is flexible enough to integrate any search technology that can be exposed as a web service. Storage and access to data at each node is provided by the storage request broker service. This provides a metacatalogue of files held within the local storage system and provides access via logical file handles so that no knowledge of the file’s physi-cal location is required on the client side. This means that any data migration between nodes on the network is completely transparent to the end user; hence the result of any repeated query will generate the same result to the end-user regardless of how data has migrated around the grid environment irrespective of time period.

A diagnostic workbench portal provides the front-end interface to the domain expert. Here the user has access to a range of tools for initiating search requests either by textual descriptions in engineering terms, using case-based reasoning, or by characteristic shapes in displayed graphical data. The portal also provides access to a range of client tools offering legacy diagnostic and analysis capability spanning both engine performance (performance chart tool [PCT]), and vibration data. The kernel of this architecture is a middleware stack, implemented in the form of an enterprise service bus based on the open source ServiceMix package. This provides the necessary messaging components to support effective communication and connectivity between client and ser-vice tools within the system. Quality of service within the entire grid environ-ment is handled by the workflow manager. This incorporates run-time predic-tion techniques so that requested operations can be optimally scheduled for all active services.

In addition to the components described above, researchers at Univer-sity of Sheffield demonstrated how hand-held devices (tablets, mobile phones, PDAs etc.) can also be incorporated within a grid environment to provide auto-matic problem alerts, and request remote search/interrogation operations from the case base reasoning (CBR) service.

A number of use-case scenarios were used during the project to demon-strate different analysis work-flows and user-interaction. The following focuses on the vibration diagnostic process involved in the identification and subse-quent search of a bearing problem. In this scenario it is assumed an expert has been informed of a new vibration problem during an engine overhaul pass-off test with supporting data. The vibration expert uses a range of traditional support tools to investigate tracked order characteristics and vibration spectral components. In this test case, the supplied data relates to a phenomenon known



as bearing no load cross-over [17]. Gas turbine rotors comprise a compressor and turbine joined by a shaft, and supported by bearings to limit their radial and axial movements. Axially, the aerodynamic load of the compressor is largely countered by the aerodynamic load of the turbine. Clearly there is a desire to ensure that the load passing through the bearing structure lies below certain limits. In addition, although initially counter intuitive, the load through the bearings must never equal zero at any high power condition, otherwise con-siderable damage can occur at zero axial loads due to discontinuous contact between the ball and bearing races. Evidence of this no load condition can be detected in the tracked order profile by observing a significant drop in vibration amplitude over a specific, but short, range of engine shaft speed. In the upper trace of Figure 7.5, it can be seen that in the identified region engine speed remains steady at around 96% shaft speed over a short interval. The corre-sponding highlighted region in the lower trace shows a sudden drop in tracked order vibration amplitude, which is characteristic of this bearing phenomenon. Having diagnosed the characteristic signature as bearing cross-over, the expert is able to raise a new case entry within the case base reasoning system and attach all relevant supporting evidence (e.g., reports, diagrams, screen-shots, summary diagnosis, and work-scope instructions) for future reference.

In the second stage of this scenario it is assumed that a vibration engineer has been alerted to a new problem via summary results from the automatic data

Figure 7.5 Example tracked order profi le depicting bearing load cross-over condition: en-gine speed in % (top); tracked order amplitude (bottom).



orchestrator tool. As previously described, any new engine overhaul pass-off data arriving at a grid node will be automatically processed using a standard set of feature detectors to establish any possible anomalies in the data. Obviously, the alert may also arise via a more conventional mechanism if the test-crew is having difficulties clearing the quality pass-off tests. In either event, the vibra-tion engineer, who may have different skills/experience to the person who dealt with the initial investigation, loads the relevant data within the signal data ex-plorer tool and highlights relevant regions of interest (see example in Figure 7.6 below) and submits a search request to establish if a corresponding pattern has been seen before across the available fleet archive.

This search request is federated out across all pattern match controllers at each node connected to the grid environment. These in turn initiate the search in their local environment and return details to the node that initiated the re-quest of any corresponding matches. The AURA technique [15], incorporated by the SDE tool and PMC, achieves rapid pattern matching as follows:

• All vibration features (e.g., tracked order components), which are rep-resented as time-history series, are encoded as a highly compressed data stream using differential binary-threshold encoding and stored within the AURA database for future reference. This operation only needs to be performed once.

Figure 7.6 Example search for bearing cross-over problem.



• Data segments, representing regions of interest selected by the engineer, are then processed using the same encoding scheme as above.

• The encoded query is then used as an input to the AURA search engine, which performs the comparison against bulk reference data using the reduced binary representation. This results in a set of candidate matches that are close to the query in binary form.

• A fi nal back-check operation, using conventional comparison tech-niques, is then performed on the candidate list to determine the fi nal set of results each of which is tagged with a corresponding confi dence measure of similarity to the target pattern.

Any returned matches to the engineer can then be further investigated by drilling down into the corresponding CBR references. Alternatively, if the engineer already suspects the problem to be related to a previously known bear-ing problem, then a textual search can be performed immediately within CBR for past cases matching that condition. Previously attached references to re-trieved cases provide the engineer with supporting background information on how to deal with the current problem and offer recommendations on potential solutions.

7.5.2 Cloud Alternatives

It is clear that the above example can be generalized to any application domain where there is a need to access a number of connected services to provide a complete fleet monitoring service whether it be electrical power generation, monitoring of rolling stock, or aircraft. Although not discussed, it should be apparent that extending services to include queried access to maintenance and past service records, such as the facilities offered by enterprise asset manage-ment systems can provide a comprehensive view of fleet status with appropriate scheduling tools that enable operators to maximize utilisation of their assets. Hence, the scenario described above could operate in current cloud environ-ments, given the similarity between grid technologies available at the time of the BROADEN project and modern IT capabilities. Indeed GE already offers their Predix platform, which is based on Amazon Web Services, to industrial customers in a range of services that support predictive maintenance opera-tions. One of the major components in this environment is the analytics servic-es which enables the creation and orchestration of various analysis services that is claimed to create insight of industrial assets being monitored. A number of established cloud providers, including Microsoft and Amazon, also offer their own computing environments in various leasing formats which provide data storage services, analytic pipelines and machine learning environments.



The diagram in Figure 7.7 indicates how data can flow from a monitored asset (top left) into a cloud environment, in this case based in Microsoft’s Azure platform, and extract relevant features which are then assessed by a data-driven model and finally displayed in a dash-board environment to the user to indicate current health status. In this example data is received from the monitored asset in the cloud environment via its IoT interface. The lower-left image describes the analysis performed in Azure ML (machine learning environment) which al-lows users to define and connect different machine learning models (including some of those described in Chapter 3) to establish an analysis work-flow which can then be exposed as a web-service. Azure ML also allows user-defined ‘R’ or Python language scripts to be incorporated within the work-flow. Results from the Azure ML model are then stored in another cloud storage container and then accessed by Microsoft’s Power BI dashboarding tool for output as shown in the lower right of the figure.

In an earlier chapter, a recommended frame-work was proposed for estab-lishing a health monitoring solution based on statistically based models. Obvi-ously this can be extended to any type of data-driven model including those in-volving machine-learning techniques. Some would regard the process of feature extraction to be a major bottle-neck in the development process. The initial steps may be regarded as straight forward in so much as the relevant features will often relate directly to the physics of the problem domain (e.g., a pressure sensor parameter). However, further manipulation of these features is often re-quired and can entail extraction of additional statistical properties (for example, means, percentiles, etc.) and subsequent feature aggregation. In applications involving several hundred sensors it is possible that these derived features will expand significantly, and deciding which to use, and in which combination, can involve a significant level of effort. Various vendors have responded to this problem by offering capability that is claimed to automatically extract key fea-tures from monitored data and additionally automatically generate analytical models that provide asset performance trending and automatic detection of emerging problems in operation.

One approach that has been developed to automatically identify anoma-lies in time-series data has been proposed using hierarchical temporal memory [18] using unsupervised associative learning mechanisms, which are biologically inspired, to assess each data point and adapt a model to changing statistics with-in the data. The developers claim this technology is well suited for prediction, anomaly detection, classification and ultimately sensorimotor applications.

Alternative solutions seem to adopt a wider speculative approach by using what is sometimes referred to as dynamic self-learning. In these systems raw data is processed to automatically create so-called dynamic engineered features which then undergo some form of dynamic feature selection. The selected fea-tures are then fed to multiple analytic models of varying types (e.g., neural



Figu

re 7

.7

Exam

ple

heal

th m

onito

ring

anal

ytic

s w

ork-

fl ow

with

in A

zure

.



network, decision trees, support vector machines, logistic regression, random forests, and so on) where an optimization process then evaluates these models (in parallel) and selects the most favorable combination of extracted features, model class, and corresponding model configuration to best describe the data.

Cognitive computing is a recent data discovery service available in the cloud environment. It is claimed this approach is able to guide data exploration, automates predictive analytics, and enables effortless dashboard and graphical creation with claims of being able to rapidly obtain answers and derive new insights that support confident decisions. Certain vendors offer such solutions as on premise products able to access and analyze both structured and unstruc-tured content by automatically performing a data crawl across the contents of the company’s data silos. This would then be followed by automatic analysis and interpretation to provide summary views of cognitive insight and attribute rela-tionships within a single environment. This type of capability is mainly targeted at large manufacturing organizations (e.g., automotive) where rapid resolution of in-service problems may be required by offering a complete 360o view of information related to components including design properties, reliability rate, service records, and dependencies from a single view point available to the engi-neer. There is also an emerging trend to incorporate data storage solutions that enable engineers to issue data queries with little or no detailed knowledge of a query language such as SQL. In some cases the product will even recommend that the user may be interested in certain discovered information identified as a potential correlation. Additional services such as text-to-speech, speech-to-text, natural language classification, concept insight, relationship extraction, visual recognition, data insights, data analytics, and many others may also be offered as API components that developers can incorporate within their own bespoke service applications.

The theme of this section has been concerned with big data analytics and its relevance to health monitoring. However, one thing that needs to be made clear is that although this term is often taken to mean extremely high-volumes of data, it is in fact a relative term. Compared to the consumer and financial sec-tors where huge levels of transactional data is generated on a daily basis, many industrial applications are only dealing with relatively small data volumes, say hourly snapshots of plant activity. Nevertheless, the capabilities discussed, par-ticularly those relating to cloud services, can provide benefit where high-valued assets (including human health) are involved. Hence, having provided some examples of big data analytics and advanced analytic capability, hopefully the reader will now have an understanding of how some of the methods described in this volume can be applied to a wide variety of health monitoring applications.



References

[1] Dommann, A., Antonia N., 2011. “Reliability of MEMS,” Proc. SPIE 7928, Reliability, Packaging, Testing, and Characterization of MEMS/MOEMS and Nanodevices X, edited by Sonia Garcia-Blanco and Rajeshuni Ramesham, International Society for Optics and Pho-tonics. doi:10.1117/12.876574.

[2] Rumpf, R C., Coggin, J.M., Poland, S., 2010. Ultra-High Temperature Distributed Wireless Sensors, US20100290503, issued May 13, 2010.

[3] Bongs, K., et al., 2016. “The UK National Quantum Technologies Hub in Sensors and Metrology (Keynote Paper),” SPIE 9900, Quantum Optics, edited by Jürgen Stuhler and Andrew J. Shields, 990009. doi:10.1117/12.2232143.

[4] Orphanidou, C., et al., “Signal Quality Indices for the Electrocardiogram and Photople-thysmogram: Derivation and Applications to Wireless Monitoring,” IEEE Journal of Bio-medical and Health Informatics, 19(3), 2015, pp. 832–838.

[5] Pimentel, M.A.F., et al., “Towards a Robust Estimation of Respiratory Rate from Pulse Oximeter,” IEEE Transactions on Biomedical Engineering, 2017.

[6] Tarassenko, L., et al., “Non-contact Video-Based Vital Sign Monitoring Using Ambient Light and Auto-regressive Models,” Physiological Measurement, 35, 2014, pp. 807–831.\

[7] Zhu, T.T., et al.,“Fusing Continuous-Valued Medical Labels Using a Bayesian Model,” Annals of Biomedical Engineering, 43(12), 2015, pp. 2892–2902.

[8] Luca, S., Clifton, D.A., and Vanrumste, B., “One-Class Classification of Point Patterns of Extremes,” Journal of Machine Learning Research, 17(191), 2016, pp. 1–21.

[9] Johnson, A.E.W., et al., “Machine Learning and Decision Support in Critical Care,” Pro-ceedings of the IEEE, 104(2), 2016, pp. 444–466.

[10] Clifton, D.A., Machine Learning for Healthcare Technologies, IET, London, UK, 2016.

[11] Goodfellow, I., Bengio, Y., and Courville, A, Deep Learning, MIT Press, USA, 2016.

[12] Hilbert, M., López, P., (2011). “The World’s Technological Capacity to Store, Communicate, and Compute Information,” Science, 332(6025), 60–65. doi:10.1126/science.1200970.

[13] Foster, I., “What is the Grid? A Three Point Checklist,” July 2002.

[14] King, S., Ramos-Hernandez D., Pilot Evaluation of GRID Technology for EHM Applications, {AU: fi nish reference}.

[15] Austin, J., Kennedy, J., Lees, K., “The advanced uncertain reasoning architecture,” Weightless Neural Network Workshop, 1995.

[16] Austin, J., et al., “DAME: Searching Large Data Sets Within a Grid-Enabled Engineering Application,” Proceedings of the IEEE, Vol. 93, No. 3, March 2005.

[17] Tipton, L.P.J., “Bearing Load Measurements on a 3-shaft Gas Turbine,” Applied Mechanical and Materials, Vols 1-2, pp 225-232, 2004.



[18] Cui, Y., Ahmad, C.S., Hawkins, J., “Continuous Online Sequence Learning with an Unsupervised Neural Network,” Neural Computation, November 2016, Vol 28. No. 11, 2016/11/0.



Documents

Future Directions in Health Monitoringdavidc/pubs/ehm_ch7.pdfused in manufacturing operations is already taking place. For example, use of control charts to determine if a manufacturing