Intelligent Innovation - AI & HPC in A&D Manufacturing · challenges and opportunities are increasingly shifting towards the ubiquity and complexity of data that is being collected,

Intelligent Innovation: AI & HPC in Aerospace & Defence Manufacturing | November 2018

1

Intelligent Innovation

AI, Machine Learning & High-Performance Computing (HPC) in Aerospace & Defence Manufacturing November 2018 Digital Technologies in the UK


2

Table of Contents

1. AI, HPC and Manufacturing ........................................................................................ 3

2. Computational Fluid Dynamics .................................................................................. 8

3. Predictive Analytics................................................................................................... 13

4. Digital Twins .............................................................................................................. 17

5. Conclusion ................................................................................................................. 20 Copyright © Digital Catapult. 2018. All rights reserved. Cray and the Cray logo are registered trademarks of Cray Inc. All other trademarks mentioned herein are the properties of their respective owners.

This report is for general information only and does not constitute advice, should not be relied upon for business or other purposes, and no duty is owed in respect of any use of its contents. Whilst steps have been taken to try and ensure the presentation is appropriate, no responsibility is taken for the accuracy, quality or completeness of the material nor the use of the contents for any purpose. No liability is accepted for any use of the contents and the presentation is protected by copyright laws.


3

1. AI, HPC and Manufacturing The ongoing trend of digitalisation and datafication is transforming industries globally, driven by the promise of improved products at decreased costs, through a better understanding of people, markets and processes, and the power of foresight and automation. Some sectors, such as media, entertainment, communications, insurance, retail and travel, are almost fully digitalised (in other words, having all goods and services accessible via digital means, rather than entirely digital). Others, such as health and banking, are well on their way, such as health and banking, whilst some sectors have been laggard, including legal and manufacturing.

Although manufacturing has undergone pockets of digitalisation, the bulk of the industry has been slower to adopt. This is likely due to a number of reasons: the physical nature of the processes involved require the design of cyber-physical interfaces for digitisation to occur; the industry involves expensive tools that have a relatively long lifespan; and there is an underlying culture centred around consistency and tradition, which can often be the enemy of innovation.

In addition, manufacturers have often viewed IT as a business support function rather than one of the core business drivers. This perception is beginning to shift. Within aerospace and defence and advanced manufacturing, the challenges and opportunities are increasingly shifting towards the ubiquity and complexity of data that is being collected, processed, analysed and visualised. It is for this reason that the sector is well positioned to seize on the growing momentum and potential applications of machine learning and artificial intelligence (AI) driven by high-performance computing (HPC), with these technologies being a natural fit for the sensor-filled and data-rich sector.

The narrow list of tasks at which machines can outperform human experts is growing, and now includes chess, the Chinese board game Go, a raft of video games from Pong to Doom, recognising numerical digits and images of objects, through to translating text. More recently, highly applied examples have arisen, such as optimising the power consumption of a large scale data centre (a technology in principle applicable to factories), the design and manufacture of hyper-complex structures using algorithms and additive manufacturing, and the planning of computer numerical control (CNC) milling routines.

These examples are beginning to make good on early assertions regarding the value of big data. In parallel, the Internet of Things (IoT) movement has begun making the notion of cyber-physical systems a reality. The barriers for the digitalisation and datafication of manufacturing are being deconstructed, paving the way for the application of both data visualisation and advanced data-driven processes.

This transformation is being enabled by HPC, where flexible, heterogenous computation platforms can tackle both traditional high-precision arithmetic calculations and the lower-precision data-parallel workloads commonly used in ML. HPC also offers other benefits such as the ability to do large-scale data processing and the expertise to optimise systems and code for economic and efficient deployment.

With this backdrop, this paper will explore three key applications of machine learning and HPC in aerospace and defence manufacturing. It will explore: (i) computational fluid dynamics (ii) predictive analytics (iii) digital twins, identifying the need, potential use cases and value that HPC and AI can have across the sector.


4

Building a Data-Driven Culture in Aerospace & Defence Manufacturing It is crucial for aerospace and defence manufacturers looking to deploy innovative use cases of HPC-enabled machine learning to first adopt a data-driven culture. The process underlying this culture is summarised in figure 1 below. It demonstrates that in order to reach the goal of building a sophisticated machine learning model, there need to be a number of processes and supporting infrastructure in place to lay the foundations for effective and useful machine learning. The triangular shape of the schema is also an indication of the effort required to ensure that each of the layers is operating satisfactorily. In essence, more effort is required to collect, move and store data in a manner that makes it easily digestible and therefore allows for smoother operations in the initial stages of data handling in a data scientist’s pipeline. Often the production of a simpler ML model is adopted, rather than a deep learning model. This approach is typically undertaken as a result of a cost-benefit analysis, with the processing power and/or data requirements of the model being assessed against the increased accuracy gain that such a model provides.

Fig. 1: The Data Science Hierarchy of Needs - Hackernoon - The AI Hierarchy of Needs1

What is Machine Learning? Before introducing machine learning, it is beneficial to describe the wider context in which it sits. Machine learning is a subset of the field of AI, which is the study of intelligent agents and has roots in machines mimicking human cognitive functions. AI encompasses machines behaving in an intelligent manner. However, the underlying mechanism can often be very different from human cognition. For example, certain chatbots may appear on the surface to be intelligent as they are responding to queries that you pose, but in fact they may be following a lookup table of set responses in a series of ‘if-else’ statement loops.

1 Rogati, Monica. “The AI Hierarchy of Needs.” Hackernoon. August 2017. https://hackernoon.com/the-ai-hierarchy-of-needs-18f111fcc007


5

Fig. 2: Overview of the evolution of AI starting in the 50’s - Buzz Robot - Difference Between AI, ML & DL2

Recently, AI has come to the forefront of technological developments as the result of three key advancements: 1) the advancement of algorithms, particularly in the field of neural networks, that has led to new techniques such as deep learning 2) the increasing amounts of data that are now available to train data-hungry models; and 3) the surge in processing power that has facilitated the training of such models. Machine learning itself builds on the original AI models by removing the need to explicitly program a machine to follow particular rules in a rigid manner. Instead, machine learning is a computer’s ability to learn the rules of a system without explicit instruction. For example, a machine learning model that was shown a large number of games of chess can learn how to win the game. This is just one application of machine learning, which spans the whole gamut of human intelligence, from being able to segment a dataset into groups with meaningful differences that enables us to identify potential outliers and therefore likely machine failures, through to predicting the likelihood of machine failure based on a number of known factors and data about past failures.

2 Tuples, Edu. “Difference Between Artificial Intelligence, Machine Learning and Deep Learning.” buZZroot. December 2017. https://buzzrobot.com/difference-between-artificial-intelligence-machine-learning-and-deep-learning-ccfd779eca7b 3 Brynjolfsson, Erik and Carmichael, Sara Green. “How AI Is Already Changing Business”. Harvard Business Review. July 2017. https://hbr.org/ideacast/2017/07/how-ai-is-already-changing-business

Building machines that perform better than humans without having to explicitly program them to do so allows us to free up cognitive load and concentrate on other tasks instead. Examples of this include a computer vision model that sorts agricultural produce, such as cucumbers of different sizes and shapes, into pre-defined categories for sale; and retinotopic diagnoses of diabetes, where physician reliability is only 65%.3 In the latter example, the use of a computer vision model to selectively filter cases that are relevant can reduce a physician’s efforts in making diagnoses. The goal of machine learning is often more than just automating cognitively expensive tasks; it aims to uncover insights that might never have been discovered had the task been completed by humans. For example, Google DeepMind’s AlphaGo computer program beat Lee Sedol, widely considered to be the greatest player of Go in the last decade, by a resounding 4-1 in 2016.4 As well as being an amazing achievement in itself, during these games, AlphaGo played a number of novel moves that have reshaped how professional players view the game and have led to advanced players, including Lee Sedol himself, adopting these new styles of play and experimenting with new moves.


6

These powerful new models using neural network architectures and deep learning algorithms differentiate themselves from traditional machine learning models by doing away with the need to have domain knowledge in order to explicitly programme the decision-making process, instead allowing a more straightforward mechanism of solely showing training examples. There are, however, costs associated with this new approach to developing models. The tradeoff is that these methods tend to be very data-hungry. For example, the ImageNet Large Scale Visual Recognition Challenge has a database of over 14 million manually annotated images that required a huge manual effort to collate. The lack of such large and freely available datasets with high-quality curation is currently one of the main barriers to implementing machine learning models at scale. The Value of Data in Aerospace and Defence and Advanced Manufacturing With lower costs of sensors, data storage and computation power, manufacturing data within the aerospace and defence sector has become significantly more accessible and abundant. Although Internet of Things technologies and sensors being deployed across the factory floor means the capability of advanced manufacturing facilities in capturing and sharing data is pervasive, it requires data analytics to create real value. This includes:

● Integrating data sets from various sources and overcoming compatibility and interoperability challenges

● Labelling, cleaning and manipulating data ● Identifying patterns in large volumes of data using machine learning and AI ● Communicating and visualising analytical insights ● Identifying cause and effect, such as modelling and simulation technologies

In an advanced manufacturing setting, data analytics are imperative to ensure that data can be integrated from various silos, creating a single source of truth across data sets and a data lake to enhance decision making across multiple geographic locations. This, in turn, will allow for more accurate analyses and predictive services using autonomous systems to provide insights into demand, supply, quality and yield of production and use performance. This can be utilised in areas such as enterprise resource planning (ERP) systems, customer relationship management (CRM) systems, warranty systems and so on, bringing the data together and combining it with external data such as weather, traffic, demographics to provide a comprehensive analytical layer for decision-making that can alleviate the challenges of data complexity and volume that have emerged in recent years.4

4 “A Cross Sectional Map of the Digital Manufacturing Ecosystem in the UK” Digital Catapult & Institute for Manufacturing


7

The UK’s Artificial Intelligence & Machine Learning Ecosystem: Complementing a World-Leading Aerospace & Defence Sector The UK is viewed as a global thought leader in AI and machine learning. The country has three of the top 20 global universities for computer science, along with being second in the world after the US for influence of AI academic publications (h-index), fourth globally by number of publications in AI and machine learning (behind China, the US and Japan) and third globally by number of citations in AI and machine learning (behind the US and China).5 When this is combined with a concerted investment in R&D activities in this area, with over 159 EPSRC grant-funded projects across 46 institutions, alongside setting up the Turing Institute as a world-class research centre, the Leverhulme Institute for the Future of Intelligence, and AISB: Society for the Study of Artificial Intelligence and Simulation of Behaviour that was founded in 1964, it is clear the UK has a rich history and an exciting future in this space.

This base of talent and interest in the field has led to a growing AI/machine learning SME ecosystem, an active VC community and some of Europe’s most high-profile acquisitions and investments taking place over the past few years. This includes Google’s purchase of DeepMind in 2014 ($500M); the Series B investment into Improbable in 2017 ($554M); Microsoft’s purchase of Swiftkey in 2016 ($250M); Investment in Benevolent AI in 2018 ($100M); Twitter’s purchase of Magic Pony in 2016 ($150M); and Apple’s purchase of VocalIQ in 2015 ($50M).6

The UK leads by a fair margin across Europe too and is at the forefront of the EU’s leading

5 Scimago H-Index Country Rankings for Artificial Intelligence https://www.scimagojr.com/countryrank.php?category=1702&order=h&ord=desc 6 Most Well Funded/Acquired AI Companies in the UK, Pagan Research, https://paganresearch.io/most-well-fundedacquired-ai-companies-in-the-uk/ 7 “The State of AI 2017” MMC Ventures & Numis - October 2017 https://www.mmcventures.com/wp-

position just behind North America in this field. Taking the European AI landscape into consideration, the UK has the strongest market with over 400 AI/machine learning SMEs7, according to MMC Ventures, while according to ASGARD there are just 81 AI/machine learning companies in Germany, just over 50 in the Nordic countries and France and fewer than 30 in the Benelux region. Furthermore, on a comparative scale of city hubs in Europe, London leads the way with more AI startups than Berlin and Paris combined.8 On the global stage, although the UK is behind Silicon Valley, which has nearly double the number of startups in this space, the EU as a whole is ahead of Asia. However, with the significant investment in deep learning research by China as demonstrated in the White House report on AI last year, it is only a matter of time before Asia catches up on the commercial front and, as such, the UK must build on its existing advantages.9

The wealth of expertise and growing global reputation for the UK in AI and machine learnin is a perfect complement to the UK’s leading position in the aerospace and defence sector. The UK’s aerospace, defence, security and space sectors are critical to the UK economy and future growth. As a world-leading sector that has approximately 123,000 direct employees and £35bn in revenue annually, the A&D sector is well primed to become an early adopter of AI and ML technologies as the sector grows.

Indeed, with 35,000 new aircraft expected to be required over the next 20 years, worth an estimated $5.3 trillion, the need and potential use cases of AI technologies alongside HPC will help to keep the UK at the forefront of the global market and ensure its future growth and competitiveness.

content/uploads/2017/10/The-State-of-AI-2017-Inflection-Point-Summary.pdf 8 “The German AI Landcape” ASGARD VC – Feb 2017 https://asgard.vc/the-german-artificial-intelligence-landscape/ 9 Artificial Intelligence, Automation & the Economy, Executive Office of the President, U.S. Government https://obamawhitehouse.archives.gov/sites/whitehouse.gov/files/documents/Artificial-Intelligence-Automation-Economy.PDF


8

2. Computational Fluid Dynamics Computational fluid dynamics (CFD) is a branch of fluid mechanics that uses numerical analysis and data structures to solve and analyse problems involving fluid flows. Intensive computations are required to simulate the interaction of fluids with surfaces defined by boundary conditions. CFD analysis is typically part of multi-disciplinary design and optimisation processes, and hence involves a number of iterative steps which are computationally expensive, memory-intensive and time-consuming. A typical CFD-based acoustics analysis requires the creation of models with 10s of millions of mesh points and hundreds of millions of processing steps, taking up to 3 months of processing time on a standard 8-core workstation. Such limitations prohibit rapid design space exploration and are one of the major pain points in the high -value design industry. This section will provide an overview of the nature and applicability of the HPC infrastructure needed to enable full-scale CFD analysis in the aerospace and defence sector. HPC and CFD Use of HPC in computationally intensive areas improves the speed and accuracy of CFD simulations by running them on multi-core computers alongside a cluster of computers (i.e. a group of computers connected together using a high-speed network). Cluster computing uses multiple computers and all of their CPUs to solve the CFD analysis. HPC infrastructure splits the large and complex computations into smaller chunks so they can be computed simultaneously. An HPC cluster comprises many compute nodes, where each node is an individual server with its own CPUs, memory and network connections, connected to a shared storage system. Many larger HPC clusters are enabled by a high-performance, low-latency interconnect network, such as InfiniBand (IB):

Fig. 3 Example HPC Architecture

Most CFD applications make use of MPI (Message Passing Interface) as a framework to allow a single application to span the resources of the compute nodes and to access the application data in parallel:


9

Fig. 4 MPI framework for communication

Using the latest advances in numerical modelling, CFD has clear applications across a variety of advanced manufacturing sectors. With this in mind, it is important to understand the required level of accuracy in this context, as it drives HPC requirements. A number of different CFD applications are presented in figure 5 below. The top left shows the turbulent flow field over an aircraft during landing. The top right represents the vortical flow field over a generic car model. These two test cases highlight the need for computer simulation, as it can drive the improvement of performance during the early design stages. Additional applications are related to marine engineering (bottom left) and civil sectors (bottom right). In order to reduce the energy consumption of a building, it is paramount to design smart buildings based on natural ventilation and therefore vital to understand how the civil structure performs under atmospheric conditions.

Fig. 5. Different applications of CFD. Top left: Flow over a high lift configuration; Top right: Flow over a generic car mode; Bottom left: Flow over a suboff configuration; Bottom Right: atmospheric flow over over an airport terminal at pedestrian leve


10

Within the aerospace and defence sector, CFD simulation can be used in aircraft design where the noise generated during takeoff and landing is one of the critical design factors. The noise generated leads to an acoustically hazardous environment for the airport crew as well as for the areas surrounding the airport. Solving this problem by collecting real data is almost impossible due to the extreme and hazardous environment. Hence significant effort is spent on iterative design and optimisation of aircraft shapes and performing the corresponding virtual CFD simulation to ensure that the noise is eliminated at the source. While such CFD simulation on standard workstations can take 2-3 months to process, using a HPC infrastructure with 4,000 cores and 180 compute nodes can reduce the simulation lead time to 0.5 hrs.10 The reduction in processing lead time using HPC is beneficial but the requirement from the industry is for such CFD simulations to be completed in real time to allow for rapid design space exploration. This has resulted in the need for a reduced order model or a surrogate model that can be solved with significantly less computational resource and lead time. An AI-driven surrogate model is one potential approach to meet this industry need. Artificial Intelligence in CFD Recent advances in AI, particularly in terms of its use of graphical processing units (GPUs) and highly effective model architectures, makes it an attractive hardware alternative for industry. Traditionally, software computations have been performed on central processing units (CPUs) that tackle calculations in a sequential order, which means that each mathematical operation will have to wait for the previous one to complete. A CPU with multiple cores may marginally speed up the calculation by offloading the operations to each core, but these kits can be very expensive. This is where GPUs are very effective. GPUs have processors with thousands of cores capable of performing millions of mathematical operations in parallel. GPUs have traditionally been used for graphics rendering and have also proven to be very effective for deep learning as both these scenarios deal with a huge number of matrix multiplication operations per second. Deep learning uses deep neural networks (DNN), which aspire to emulate the processing steps of the human brain. They were originally used as a method to study brain function, albeit as an overly simplistic analogy that has nonetheless yielded some interesting findings in neuroscience. There has, however, been more progress with neural nets in the field of computer science, where a combination of algorithmic advancements and increases in the availability of data and processing power has led to some compelling applications for information processing, and therefore AI. These neural networks consist of multiple layers of stacked artificial neurons (non-linear processing units) which enable the learning of a ‘mapping function’ between inputs (boundary conditions) and the outputs (e.g. CFD solver results). Once the mapping function is learnt, it is then used to make predictions about the outputs for a new set of input conditions. These predictions are usually made within a few seconds, albeit at a small cost in terms of accuracy.

10 These specifications have been set out by the UK’s Centre for Modelling & Simulation (CFMS). You can find more information about CFMS at http://www.cfms.org.uk


11

Fig. 6 Example of Deep Neural Network Convolutional neural networks (CNN) are a type of DNN that are very effective in learning mapping functions in cases requiring strong spatial and temporal dependencies using fewer parameters that are shared within the network. This sharing of parameters makes this kind of neural network very memory-efficient. In a CFD application, memory requirement is usually the bottleneck in building, say, a full-velocity field surrogate model for large design shapes. CNN-based surrogate models estimate the velocity field orders of magnitude faster than a GPU-accelerated CFD solver. U-Net is a type of CNN architecture that is already in use for AI-driven medical diagnosis, but is also fully applicable in the CFD domain. U-Net uses convolutional autoencoders1112 and residual connections1314 to compress the state size of a simulation and learn the dynamics on this compressed form. The result is a computationally and memory-efficient neural network that can be iterated and queried to reproduce a fluid simulation.

Fig. 7. Example of U-Net Architecture15

11 Chablani, Manish. “Autoencoders – Introduction and Implementation in TF”. Towards Data Science. June 2017. https://towardsdatascience.com/autoencoders-introduction-and-implementation-3f40483b0a85 12 Autoencoders are a family of neural networks for which the input is the same as the output (they implement a identity function). They work by compressing the input into a latent-space representation, and then reconstructing the output from this representation. 13 He, Kaiming;Zhang, Xiagya; Ren, Shoaqing; Sun, Jian. “Deep Residual Learning for Image Recognition”. eprint arXiv:1512.03385, https://arxiv.org/abs/1512.03385 14 Residual connections ease the training of networks that are substantially deeper than those used previously. https://arxiv.org/abs/1512.03385 15 Example of U-Net Architecture: https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/


12

This kind of network architecture is currently being researched at the UK’s Centre for Modelling & Simulation (CFMS) to design and optimise an aerofoil cross-section shape. Such a shape optimisation technique is applicable to various industries like aerospace, Formula One racing, energy, etc. However, the key difference is in its optimisation objective, i.e., in the aerospace industry the aerofoil cross section is optimised to generate lift, whereas in Formula One racing industry it is optimised to reduce lift. The data for training the CFMS neural network was generated by running 1,000 CFD scenarios, which completed in one week using the full capacity of the CFMS HPC cluster. The CFD input and output data was converted into images in such a way that all the CFD domain knowledge (e.g. pressure, temperature, Mach number) was captured within each pixel of the images. The deep learning model using the U-Net architecture was then trained to learn the mapping between the inputs and the outputs. This training phase used four K80 GPUs and completed in 12 hours. Once the mapping was learnt, it was then used to make predictions about CFD solutions. Predictions for 250 CFD scenarios were completed in under 250 seconds with over 90% accuracy. Although this result is based on a ‘toy example’ given that the experiment was conducted using only 1,000 CFD scenarios – real world examples need to be at least 10 times more – it demonstrates that the combination of AI and HPC for CFD holds promise for much faster exploration and optimisation of the design space.

Fig. 8 Example AI model for CFD


13

3. Predictive AnalyticsData-driven manufacturing is beginning to draw more interest as users increasingly demand comprehensive maintenance services for their production equipment. There are numerous applications of predictive analytics. Predictive and preventative maintenance through condition monitoring16, which is now feasible with the availability of a plethora of new sensor technology, will be able to maximise operating times by predicting time of failure, limiting out-of-service time, eliminating hidden bottlenecks and identifying root causes within a complex assembly line. Together, these improvements will lead to higher machine availability and a

reduction in downtimes, particularly at critical phases of the production process, which will help to significantly reduce customer delivery times.

By analysing the interactions between machines and processes, using profit-per-hour analyses that account for data over a wide variety of parameters and conditions, an accurate model of the total profitability of the supply chain can be obtained and used to drive optimisation processes, including the identification and elimination of unprofitable product

ion lines. This sort of analysis can also include yield-energy-throughput, whereby yield is maximised by optimising a machine’s efficiency.

Key to effecting this would be changes in people, process and technology. From the manufacturer’s point of view, it is vital to optimise and to improve the service provided to the final user, allowing appropriate maintenance planning and responding to demand. The main objective of predictive maintenance or condition-based maintenance (CbM) is to identify and avoid anomalies by means of the use of predictive analytics. This is one spin-off to the continuous characterisation of the health status of the equipment. In this way, the maintenance actions are aligned with the real needs of the equipment and tasks can be planned, taking into account production and exploitation constraints.

Maintenance timelines are critical for scheduling maintenance actions before equipment failures. For predictive maintenance (PdM) to be effective, condition monitoring (CM) must be applied, evaluating the health of a machine and picking up any early signs of potential failures and how they may evolve over

16 Condition monitoring applies description and corrective analytics to inform the user of a process or system to see the condition of each machine using statistics and visual analytics. A common set of technologies typically used in condition monitoring are: Vibration Analysis, Lubricant

time to accurately predict when they can be rectified.

HPC & Predictive Analytics

A typical framework for shop floor data collection that is eventually used for predictive analytics is shown below.

Fig.9 Typical Framework for Shop Floor Data Collection

IoT data generated from shop floor devices in advanced manufacturing is acquired for in-place monitoring and control via SCADA (supervisory control and data acquisition). However, there is a need to store the IoT data from all the shop floor devices into a unified data repository. The data repository has to be such that it can handle the heterogeneous nature of shop floor data (e.g. structured/tabular data, semi-

analysis, Acoustic emissions, Infrared thermography and ultrasound testing.


14

structured data, log files, images, audio files etc.). The process of populating the data lakefollows a sequence of steps known as ETL: extract the source data, transform it and finally load it into the data lake.

The architecture choice for a data lake and distributed computing could be traditional enterprise, commodity compute or HPC infrastructure

Traditional Enterprise Traditional enterprise data solutions (e.g. SQL databases) make use of expensive technologies such as hardware RAID (redundant array of independent disks) storage, resilient networks and enterprise RAS (reliability, availability, scalability) functions, to ensure that a system remains available, and so encourage a system to ‘scale up’ and capitalise on these expensive investments. There is often either a hard limit to how far a system will scale, and adding additional compute resources only increases redundancy. Commodity Compute Technology stacks such as Hadoop were designed to work with inexpensive ‘commodity’ computing services, which allow workloads to scale out. Rather than investing in expensive reliability solutions (i.e. traditional enterprise approach), data is replicated in shards across a cluster, with each individual server being regarded as disposable. Scale-out systems like Hadoop will use the aggregated processing power of the whole cluster, so as the workload grows, the cluster can grow in parallel. Although Hadoop is often referred to as a monolithic application, it is actually comprised of several components:

● HDFS – a distributed file system that shards data over all members of the cluster ● YARN – a data-location-aware workload scheduler ● MapReduce – a programming model for large-scale data processing.

High Performance Computing Many organisations that make use of Hadoop often also make use of HPC clusters. HPC clusters blend enterprise technologies (high-performance centralised RAID storage, high-performance networking) with the scale-out disposability of compute resources to run tightly coupled parallel applications for engineering and science applications. HPC clusters’ use of job scheduling software like SLURM, LSF, UGE and PBS, and high-performance parallel file systems such as Lustre or IBM Spectrum Scale, can replace the YARN and HDFS components of Hadoop, and tools such as LLNL’s ‘Magpie’ ease deployment of many big data tools including Hadoop, Spark and Kafka onto HPC clusters. As a result, an organisation looking to make use of ‘data lake’ type applications, would ideally consider the types of IT infrastructure and skills that they have and select the appropriate toolset, especially considering whether a dedicated Hadoop environment is necessary. Examples of Predictive Analytics Inside of Advanced Manufacturing Intel has been using predictive analytics to reduce the vast numbers of tests it must perform on every chip on its production line, which is typically in the region of 19,000. By analysing the data collected


15

throughout the manufacturing process, Intel was able to achieve a $3 million saving for a single line of Intel Core processors.17 Lufthansa Technik is a leading provider of maintenance, repair and overhaul (MRO) for civil aircraft, engines and components. The company offers a condition analytics platform to support customers with respect to condition monitoring and predictive maintenance, enhancing procedures to mitigate potential faults. Lufthansa Technik also has a research project, APOSEM (Advanced Prediction Of Severity effects on Engine Maintenance), that models engine behaviour over the entire life cycle in order to optimally plan for maintenance measures by better forecasting the engine behaviour over time. Examples of Predictive Analytics Outside of Advanced Manufacturing Google

A substantial cost to any cloud provider is the cooling required to keep server rooms at the right temperature for optimal performance. Many companies try to mitigate these costs by having data centres placed close to cheap cooling resources, or in colder climates. Google has used the expertise from the acquisition of its DeepMind startup to build a neural network model that predicts the usage of its data centre resources and, as a result, can manage power usage. They claim that on average, Google data centres use 50% less energy than a typical data centre. This was achieved as a result of optimising power usage to reduce their overall electricity requirements by 40%. After accounting for electrical losses and other inefficiencies, overall energy savings was 15%, which will translate to millions of dollars over the years.

GE Predix

GE’s Predix platform is the vision of former CEO Jeffrey Immelt, who anticipated the advent of the industrial internet and developed it as part of his strategy to transform GE into a digital industrial company in the last decade. In essence, the Predix platform connects equipment such as turbines and elevators to computers in order to predict failures and operating costs.

There are three main products: asset performance management, focussing on optimising maintenance costs, mitigating operational risk and reducing total cost of ownership; operations performance management, focussing on optimizing yield, quality and efficiency; and service max, focussing on field service management using IoT to support technicians on-site with real-time data and analytics.

Ai Build

Ai Build is a company developing AI and robotic technologies for large-scale additive manufacturing. Its aim is to empower factories of the future to make manufacturing easy, smart, sustainable and affordable. Ai Build’s autonomous large-scale 3D printing technology has wide applications including construction, high-performance products, custom molds, interior finishes and furniture.

Using existing robotic technology to operate an extruder, Ai Build makes real-time updates to the manufacturing process by combining information from offline simulation with online feedback. This allows the company to reactively build products to a high specification despite changes in environmental conditions affecting production, such as material shrinkage as a result of cooling. This information is also used to inform production of the next item on the assembly line.

17 “4 Big Data Use Cases in Manufacturing Industry”. Sept. 2017. Ingram Micro Advisor. http://www.ingrammicroadvisor.com/data-center/4-big-data-use-cases-in-the-manufacturing-industry


16

GE: GE used Kaggle competitions as a way to release flight history, flight status, weather, route and no-fly-zone data in a competition in which winners were asked to choose the most optimal route. The prize-winning solution was evaluated in a flight simulator and had a 12% increase in efficiency and costs over actual flights.18 Northwestern University: Metallic glass is a material that can act as a superior replacement for stainless steel, being stronger, lighter and more resistant to corrosion and wear. However, in the last 50 years, only a few thousand out of millions of possible combinations of materials have been developed into useful products. A group of scientists at Northwestern University’s Department of Energy’s SLAC National Accelerator Laboratory and the National Institute of Standards Technology (NIST) are using AI with experiments to quickly make and screen hundreds of sample materials at a time. This has allowed the team to quickly discover three new combinations of materials and short-cut the discovery process by a factor of 200, where it would typically take a decade to discover a new material.19

18 GE announces the winners of Flight Quest 2: Kaggle GE Announcement 19 “Artificial intelligence accelerates discovery of metallic glass”. Northwestern Now. April 2018, https://news.northwestern.edu/stories/2018/april/artificial-intelligence-accelerates-discovery-of-metallic-glass/


17

4. Digital Twins

The term ‘digital twin’ refers to a computerised model of a physical industrial asset, such as an instrument, wind turbine or vehicle. Digital twins provide accurate and precise visualisation of cumulative real-time data that allows for detailed analysis and insights that can help in decision-making and actions for planning, optimisation and product design across manufacturing, maintenance and operational processes throughout the product’s life cycle. Digital twins can be especially useful in the aerospace and defence industry, where assets can spend long amounts of time in the field and be subject to a wide range of limitations such as weather damage and mechanical exertion, away from a central location where thorough diagnosis can be carried out.

The term digital twin has emerged in recent years with many competing definitions, due to different sectors having their own terminology and the classic silos of design, manufacturing and operation having their own viewpoints and areas of focus. One commonality between sectors and departments is that the ability to effectively amalgamate from various data sources is critical to success. A digital twin combines a variety of product design elements, namely:

• 3D models (using CAD systems) • System models (using system engineering product development solutions, such as systems

driven product development) • Bill of materials • 1D, 2D and 3D analysis models (using CAE systems) • Digital software design and testing (using ALM systems) • Electronic design (using systems such as Mentor Graphics)

Ideally a digital twin would cover all aspects of the design life cycle of a product, blending design, simulation and analysis models with physical asset telemetry to form a live unified model. Using the unified model it is possible to perform analytical, predictive maintenance and design optimisation operations all in one location. This is crucial in the aerospace industry, where assets travel worldwide and cannot be fully serviced after every flight. The digital twin allows for design of assets across multiple territories, manufacture of parts for aeroplanes takes place across multiple sites in many countries, allowing designers to work together across vast distances enables for more opportunities for collaboration-enhanced design.


18

The implementation of a digital twin identifies when problems could potentially occur, enabling the manufacturer or carrier to alert ground crews of a requirement while an aeroplane is still in flight in order to assess and correct a problem as soon as the plane lands, reducing downtime of the asset dramatically. Enterprise silos typically prevent this unified model, hence a set of ‘digital twins’ are created at various stages of the product development. For example product design departments produce 1D, 2D and 3D simulations of the product to meet product requirements and performance criteria. Aerospace and defence manufacturing produces artifacts such as a discrete event simulation of the factory, a bill of process and a bill of assembly. Service, maintenance and operation will produce operating models, maintenance schedules, parts and spares inventories and service centre availability. Combining this with any live telemetry from the product or factory results in a data flood. Designing and making a product are not, by nature, concurrent processes. All sectors can identify the issues with designing and making a product in the traditional ‘waterfall’ process and agree in principle to a more agile and iterative process. However, legacy enterprise systems, software tools and processes are cemented in the traditional methodologies, making change very difficult. Moving to a new methodology is an all-or-nothing step, which introduces significant business risk. However, if the whole enterprise can be represented digitally, the opportunity to understand and affect the business is enormous. Hence why the term ‘digital twin’ is also being used as a marketing term to bring a step change in the way businesses operate. HPC & Digital Twins The rise of affordable and available computing power, networking and embedded sensors has greatly reduced the cost of the traditionally expensive and bespoke domain of IT, virtual product development and simulation. The manufacturing machine suppliers already provide access to key machine performance data, and factories are becoming more ‘digital’. The aerospace and defence sector is no different, with a wealth of data being generated. The challenge is therefore to link this manufacturing data to the design-stage simulations and models and to the operational-stage models and data. The missing ingredient for business decision-makers is the appetite for change and management of risk. Digital twins exist in two forms: product twins and plant process twins. Product twins refer to a system that monitors a product following its purchase by a consumer, for ongoing support and sales, this type of digital twin is commonly applied to products such as vehicles and aeroplanes. The automotive industry is adopting the idea of product digital twins for after-sales maintenance and support. The most prolific digital twin system currently on the market is that of Tesla. Tesla provides an excellent example of the predictive maintenance concept. The electric car manufacturer has a digital twin for every car it builds, tied to the car’s vehicle identification number (VIN). Data is constantly transmitted back and forth from the car to the factory. A plant process twin exists as an online version of a piece of machinery at a plant or on a factory floor. The most common plant process twins today exist as wind turbines and small pieces of machinery. Through combining these elements, a “comprehensive computerised model of the product - enabling almost 100 percent of virtual validation and testing of the product under design.” This is hugely beneficial for product design as it avoids the need to prototype, reducing the iterative design phase time, improving quality and reducing the time to market for the final manufactured product. This could


19

lead to 10% effectiveness improvement in large industrial companies adopting the digital twin approach20. Digital twins are being utilised across a number of industries to optimise operations for the end user. Some of the most interesting cases are below:

● Maserati: The automaker is using digital twin technologies to accelerate product design. Virtual modeling and simulation are reducing the number of expensive, real-world prototypes required, as well as the need to launch physical wind tunnel tests and test drives, cutting vehicle development time by 30 percent21

● Intermarché: The French supermarket uses data from IoT-enabled shelves and sales systems to create a digital twin of brick-and-mortar stores, enabling managers to get real-time insight on stock and test the efficacy of different store layouts22

● GE: The tech giant is using digital twins to model supply chain and factory processes at its Nevada facility in order to improve inventory management23

● Dassault Systèmes: The healthcare company is building a library of realistic human heart simulations that physicians can consult to better understand a patient's condition in real time24

● U.S. Military Sealift Command: Digital twins of variable frequecy drives, propulsion motors, diesel engines and generators compare the real-time asset performace against its digital twin’s data profile, spotting data variance – often a sign of performance degradation that leads to potential failure – and informing operators who can solve the problem before it occurs, increasing asset reliability and availability, and reducing maintenance needs and associated operational expenditures24

● Lockheed Martin: In 2017 Lockheed Martin unveiled plans to expand on its Digital Tapestry strategy. The company is actively creating digital twins for most of its products, processes and tools. By doing so, Lockheed Martin will be able to fine-tune every step in the life cycle of an aircraft

Analysts point to 2020-2021 as the time when digital twins will be leveraged in key business applications. Gartner sees the main place of digital twins in an IoT project context until around 2020. The company expects half of large industrial firms to use digital twins by 2021. Advances in various technologies supporting digital twins over the coming years will help to improve the reliability and usefulness of digital twins. The advent of 5G (expected from 2020) will enable massive machine-type communications (mMTC) that allow low-latency data transfer between the physical asset and the digital twin, providing real time changes that are not possible with existing 4G networks. Existing technologies such as Wi-Fi and 4G can’t process the large variety of sensor data that is produced by the multitude of sensors required for a complex digital twin. The advent of 5G will allow large amounts of real-time data to be delivered to high-powered computers to simulate complex assets on a manufacturing floor or a complex asset in the field. 20 Petty, Christy. “Prepare for the Impact of Digital Twins”. Smarter With Gartner. Sept. 2017, https://www.gartner.com/smarterwithgartner/prepare-for-the-impact-of-digital-twins/ 21 “Maserati and the Future of Manufacturing”. Online Video Clip. Siemens. April 2015. https://www.siemens.com/stories/cc/en/driven-by-data/ 22 Carboni, Brian. ”The ‘Connected Store’ Reimagines Retail”. December 2015. https://blogs.3ds.com/northamerica/connected-store-reimagines-retail/ 23 GE Look ahead, “The Digital Twin”, The Economist, September 2015. http://gelookahead.economist.com/digital-twin/ 24 Grieves, Michael. “Virtual Twin: Manufacturing Excellence through Virtual Factory Replication”. Dassault Systemes. 2014. https://ifwe.3ds.com/media/virtual-twin-whitepaper


20

5. Conclusion

AI and HPC are beginning to transform the aerospace and defence manufacturing sector by offering new and innovative data-driven decision-making capabilities. The key enablers for this transformation are the ubiquity of data across the industry, advances in high performance computing, intelligent algorithms and an open-source software ecosystem. The strength of the UK’s AI and HPC ecosystem, coupled with recent leaps in the capabilities of these technologies, offers huge opportunities for the sector across its product lifecycle (design; manufacture; operate; maintain). In order to undertake the journey to adopting HPC-driven ML products, services and applications, aerospace and defence manufacturers must take active steps to build a data-driven culture that will allow them to ensure both accuracy and effectiveness of algorithms by assuring that data is collected, with a reliable and clean data pipeline, including labelled and aggregated data that underpins algorithm training. With these foundations, aerospace and defence manufacturers – who already a significant wealth of data – will open greater opportunities for AI and deep learning. Alongside this, HPC will play a significant role in specific deep learning use cases and offer the sector opportunities to utilise developing innovative data science techniques in areas such as CFD, predictive maintenance and digital twins. In CFD, AI has the potential to help design and optimise areas such as aerofoil cross-section shapes by taking large data sets, converting that data into images, and capturing CFD domain knowledge such as pressure, temperature, Mach number etc. into each pixel of the image. This allows for significant time reduction in predictions for CFD scenarios and for faster exploration and optimisation within the design space. In the predictive analytics space, HPC and ML can capitalise on IoT data – generated from the shop floor of an advanced manufacturing facility – to train algorithms via a data repository. Given the significant amount of data being generated in the creation of data lakes, HPC is often used alongside tools such as Hadoop. Further, with the evolving nature of and interest in digital twins, aerospace and defence manufacturers are understandably beginning to explore the potential of real-time data to link manufacturing data to design-stage simulations and models and to the operational stage models and data. Although this requires HPC, it is also critical to ensure reliability and accuracy of the data through effective connectivity and infrastructure. It is for this reason that next-generation future networks such as 5G need to be explored to enable mMTC that will allow low-latency data transfer between the physical asset and the digital twin. This, when combined with HPC, will enable AI and ML algorithms to intelligently analyse, predict and optimise processes accurately in a complex digital twin of a product or plant process. It is clear from these findings that the future of aerospace and defence manufacturing will rely on a combination of IoT sensors for data gathering and HPC to drive intelligent analytics and visualisation of data, alongside ultra-reliable and ultra-fast connectivity infrastructure.

Documents

Intelligent Innovation - AI & HPC in A&D Manufacturing · challenges and opportunities are increasingly shifting towards the ubiquity and complexity of data that is being collected,