Implementation and testing of an online fault isolation methodology in a real industrial scenario

Ferrarini Luca Dipartimento di Elettronica e

Informazione Politecnico di Milano

Milan, Italy ferrarini@elet.polimi.it

Allevi Massimo Dipartimento di Elettronica e

Informazione Politecnico di Milano

Milan, Italy allevi@elet.polimi.it

Dedè Alessio Dipartimento di Elettronica e

Informazione Politecnico di Milano

Milan, Italy dede@elet.polimi.it

Abstract— The diagnosis of manufacturing systems plays an important role in the safety of both systems and operators. The paper presents a real application of systematic approaches to fault identification problem in the manufacturing field up to the online implementation. The diagnostic method comes from the classical diagnoser one, with the explicit modeling of time-out and the explicit modeling of nominal and non-nominal control behaviour. The aim of this novel diagnostic algorithm is to isolate faults that can occur in typical devices of machining centres. The diagnostic model and the approach has been defined and inserted in the model-driven architecture defined in the EU-MEDEIA project with the aim to describe a generic industrial plant as set of components.

Keyword: manufacturing systems, diagnosis, discrete-event systems, machining centres

I. INTRODUCTION Today’s industrial plants are designed and implemented as

a set of heterogeneous components which must be integrated in order to satisfy the requirements. These new architectures have increased the general complexity of the automatic systems making difficult to design and put into practice robust and flexible solutions. In this scenario a set of new instruments and approaches must be introduced in order to dominate the complexity achieving a good level of reliability and flexibility. The integrations of heterogeneous subsystems and the reduction of the plants complexity could be achieved with a distributed model-driven approach where the complete model of the plant must be defined starting from a set of sub-models which describes different components [5]. This idea has been developed in different academic and industrial projects. In particular, the paper presents the MEDEIA [7] approach which follows a distributed model-based design for industrial plants. To divide and model a plant could be good solutions to obtain a reduction of the design times and costs. However this process must be sustained by an integrated active diagnostics and a structured alarm management for a fast isolation of faults in order to increase the safety of the system and its availability. Furthermore starting from the fault isolation it is possible to define recovery strategies extending the life time of the system. To answer to this question in the last years the industrial and academic research have proposed and synthesized many diagnostic approaches.

In industrial practice, error handling algorithms are introduced into the control code, in order to understand, online and within a pre-established amount of time, whether or not the behavior of the system under control is correct. In this way, the designer understands quickly if something goes wrong, avoiding to investigate and understand which precise component is under fault [10]. This diffuse practice should be reviewed and improved: as a matter of fact, a real and concrete advantage for control design is the systematic integration of modelization and design processes generating diagnostic functionalities, which should be more effective in faults isolation and alarms management. MEDEIA introduces in its design approach a new model-based definition of the diagnostics functionalities. Each component, in fact, has its own diagnostic model used to specify and design fault isolation, alarm management and each function useful to increase its reliability.

On the other hand, in the academic research little space has been given to application of systematic approaches to fault identification problem in the manufacturing field, in spite of the relevant research work available. In this perspective, the present paper intends to provide elements to bridge the gap between the sound theoretical results available and the industrial needs in the manufacturing automation field. In particular a new model-based diagnostic methodology has been developed and inserted in the MEDEIA framework, allowing an automatic code generation for the fault isolation which has been tested within a real demonstration scenario. The new approach, called TiDiaM (Timed Diagnostic Model) has been developed starting from lot of theoretic results reported in academic and industrial research activities. In particular, it is based on the DES (Discrete Event System) methodologies which have been developed in academic studies. [9] provides the description of the DES diagnostic approach used in TiDiaM where the system is modeled as a DES, in which faults as unobservable events and the diagnosis is the process of detecting occurrences of such events from observed event sequences. The events observer is called diagnoser and it is basically an automaton whose states are associated to a set of fault system state estimates describing the possible faults occurred in the system. An application of this method is described in [2].

978-1-4244-8970-1/11/$26.00 ©2011 IEEE 13

The classical diagnoser method has not been thought as an online isolation method and it is difficult to adopt it in real industrial cases, mainly because the condition of diagnosability is not fulfilled in most of the production systems. To solve this problem various methods and new definitions have been introduced, a example is presented in [1] . TiDiaM is one of them. It can be seen as an extension of the original diagnoser with the insertion of time-out information associated to a special modeling technique. An application of this method, using a Java software executor called TiDE (Timed Diagnosis Executor), is shown in [2]. In the test case presented by this paper a code generator for an industrial softPLC has been implemented and used to automatically generate TiDiaM code implementation for the Orchestra controller, [11], which has been employed to control the project demonstrator.

The paper is organized in five different sections. After the introduction, section 2 describes the MEDEIA project, its methodology and the demonstrator scenario used to test the novel diagnostic algorithm. The section 3 presented the diagnostic method TiDiaM, which it is divided in three parts: the first described the role to synthesize the diagnostic automaton, the second introduce the time-out concept and the timed analysis for isolation of the faults and last part described the real-time diagnostic executor. The different application tests of the TiDiaM and relative results and observations on are discussed in section 4, before conclusions in section 5.

II. MEDEIA The presented research work has been carried out within a

European project, MEDEIA (Model-Driven Embedded Systems Design Environment for the Industrial Automation Sector), focused on the development of a methodology and an engineering framework for the design of complex automatic systems. In particular, the application here presented has been used as final MEDEIA demonstration for the manufacturing field, applying the new design methodology to automatically generate the control code and the diagnostic one for a machining cell.

MEDEIA proposes a modular and hierarchical approach where the design phase of a plant is based on a modelization of its subsystems and on a final aggregation of them. The result is a model composed of a set of components organized in a hierarchical architecture. This methodology allows a reuse of the designed models and a reduction of the considered complexity during the design phase. In fact each component can be thought alone focusing on its particular section before the final aggregation to obtain the overall plant.

Figure 1. MEDEIA Automation Component

Figure 2. MEDEIA design approach

A. Automation Component As described before the MEDEIA methodology is based on

a modular and hierarchical model approach to design and realize complex industrial plant. This idea is based on a structure composed by components called Automation Components (AC). Each component defines and models a sub part of the final plant and it contains a plant model, a control model and a diagnostic one. These models can be edited and modified using particular DSVs (Domain Specific Views) and they can be used to implement the relative functionalities for a set of PSIs (Platform Specific Implementations). According with this approach the AC can be considered as a bridge between the design phase and the implementation one, useful to obtain a standardization and a reuse of the all processes and activities involved in the design and realization of a new plant.

B. Industrial scenario: SPI Cell The MEDEIA approach and the code generators for control

and diagnostics have been used to design and implement a manufacturing centre called SPI Cell. The considered plant is composed of an operator station, where a human operator loads and unloads pieces into and from the production system, a robot with 6 degrees of freedom, used to move the pieces into the plant, a local buffer of the machining centre, called pallet changer, able to load and unload pieces into and from the work area of the machine, and of a machine with 5 axis used to refine motorcycle engine aluminum parts. The layout of the overall production system is reported in Fig. 3.

Figure 3. Cell SPI Layout. 1) Operator Station, 2) Robot, 3) Pallet Changer, 4) Machine

1

23

4

14

Figure 4. Architecture of TiDiaM

The particularity of this demonstration is the application of a Mixed-Reality solution [3] in order to simulate some parts of the plant which are not available integrating them with the available real ones. In particular, the machine and the operator station are simulated. Instead, the robot and the pallet changer device are real. Furthermore the overall production cell is controlled with the softPLC Orchestra. Also the diagnostic implementation, as it will be described in next paragraphs, has been generated for this particular platform. This solution has permitted to test the result of the MEDEIA project on a real manufacturing example without having an entire real plant.

III. TIDIAM TiDiaM follows the diagnoser approach method, isolating

faults only through observable events of the system. In a typical feedback control system, observable events are events generated by sensors in the plant and by the controller. This framework is described in the Fig. 4.

The main improvements of TiDiaM with respect to the classic diagnoser are the insertion of the non-nominal behavior in the control model, related to the error handling, which improves the isolation ability of the model, and the diagnostic time-out insertion for every state, which permits to promptly isolate faults performing a particular timed analysis on the fault estimates. The next sub-sections show the basic rules to build the diagnostic automaton, to insert the diagnostic time-out and to execute the timed analysis.

A. The diagnostic automaton TiDiaM is automatically synthesized starting from a system

model S, computed as synchronous composition of control (C) and plant model (P), (1).

S=P||C (1)

The plant model P is an automaton which represents the model of physical devices to be controlled, without control. It can be described with the automaton ),,,(

0ppp xXP δΣ=

where: Xp is the set of states, Σ is the set of events, δp is the transition function, PP Xx ∈0 is initial the state. The events set can be partitioned: uoo Σ∪Σ=Σ , where the oΣ is the set of the observable events and uoΣ is the set of unobservable events. The first contains the plant events which can be observed, these are associated to the measures provided by the sensors or the commands generated by the control. The second set contains

the events that can not be observed such as faults or events modeling cause-effect relations among plant devices (e.g. when an actuator is activated, sooner or later a sensor output will change). The automaton P is typically computed as the parallel composition of each component models iZ (micro sensor, electro valve, etc.) and the actuator of the system Y (hydraulic piston, electric motor, etc.), as described in (2);

P=Z1||Z2||...||Zn||Y (2)

The actuator model Y describes the behavior of the physical actuator of the device (for example the behavior of a pneumatic piston respect to the variations of the inlet pressures) instead each component Zi describes the nominal behaviour and the behavior in a presence of a fault of a “controlled component” (such as an electric valve, a micro sensor, etc.). A set of particular states are added for each component contained in plant model in order to model the dynamic transitions of the system. These states are defined transient states. To differentiate them from other states, a conventional notation is used for their name: the string “-t” is appended. The components Zi of the system can be modeled through two different methodologies: one-to-one and aggregation. The first uses a relation one to one between the real components and models, the other permits to join more real components in a unique model. Latter method has allowed to reduce the complexity of the plant model, and therefore of the final diagnostic automaton.

The control model C is modeled with the automaton ),,,( 0ccoc xXC δΣ= where the Xc is the set of control

states, oΣ is the set of observable events, δc is the transition function, CC Xx ∈0 is the initial state. The set oΣ is given by

ceoeo Σ∪Σ=Σ . oeΣ is the set of measures, observable events that the sensors of the system can generate, and ceΣ is the set of commands, controllable events that the “logic” actuators of the system can accept. Clearly, =Σ∩Σ ceoe ∅. In the control model, the nominal cycle and the error handling model are clearly identified. The first one represents the desired behaviour of the system, which is composed by a cyclic sequence of events that follow a typical pattern. As a matter of fact, the control may send one or more commands to the plant, and then waits for the reaction of the plant, measured in terms of sensor events. In the applications investigated so far, i.e. automated devices belonging to machining centres, the commands appear always in a sequence, while the measures events come in any order. This led to the introduction of the useful notion of “actions” iA used to decompose a nominal control.

An action iA is composed of a sequence isc of command events followed by all the possible combinations of the events belonging to a set of measure events isc

eoΣ , taken in any order.

A fundamental element of the TiDiaM to isolate the fault is the non-nominal behaviour of the control model, also called

15

error handling [5] and [10], which is composed of a set of observable events that describe the system evolutions after a fault occurrence. The specific modeling of a non-nominal behavior in the control model allows to improve the isolation ability of the final diagnostic automaton. The error handling is automatically added to the nominal cycle model through a specific algorithm implemented in Java language. The system model S is finally obtained as parallel composition S=P||C. Starting from the system model S and from its considered faults the fault partition must be defined. This is the set of the considered system faults grouped in order to isolate a set of faults families. The last step is the synthesis of the diagnostic automaton which is done through the classic diagnoser technique presented in [9].

B. Diagnostic Time-out and timed analysis The diagnostic time-out is a fundamental element in the

method here presented. For each state of the diagnostic automaton a time-out is associated, which represents the maximum time necessary to expire the transient dynamic of the system. When a time-out expires during the execution of an action, it provides the information “something went wrong”. Such a fault information is used by the diagnoser automaton to accelerate the diagnostic analysis and so the isolation of the possible fault which has caused the time-out expiration. For each set of measure events related to an action Ai, a value of diagnostic time-out must be given. Starting by this information, it is possible automatically set the diagnostic time-out for each control state.

The timed analysis is the real algorithm able to isolate the faults that can occur in a system using the information modeled by the TiDiaM. This analysis begins when the time-out expires in a state of the TiDiaM. It works under the following hypotheses:

• the faults which can occur in the system must be persistent, so that the effect of the faults remains in the system until a recovery operation is done by an external intervention;

• component faults can not occur in transient states;

• when the time-out expires the control is freezed, so that it can’t generate new controllable events;

• only one fault can occur before the expiration of the diagnostic time-out;

• the size of the measures sets isceoΣ must be equals or

more of two elements from different components;

• a component used in an action, it can not be used in any other action of the plant at the same time.

The analysis is executed on the equilibrium faults estimates associated to the state where the time-out expired. Such estimates are those which do not contain transient states of the plant models by definition. Basically, when the diagnostic time-out expires, the intersection of the equilibrium estimates of faults is performed. If this operation isolates a set of faults, the method ends. Otherwise, we should wait for some other event to occur, to improve our estimation.

Figure 5. Architecture of the TiDiaM implementation

As opposed, the timed analysis is able to reduce the admissible fault estimates looking for one sensor able to change its state, then TiDiaM is investigated as if that sensor would change its state. This is possible if and only if the above hypotheses hold (and in particular, the last two). The choice of the sensor depends on the previous history of the system and allows a reduction of the fault estimates. This permits certain isolation of the occurred fault or to obtain a smaller set of possible faults which have caused the expiration of the diagnostic time-out.

C. Real-Time Diagnostic Execution In order to provide an executable version of the TiDiaM, a

code generator for the softPLC Orchestra has been developed. The architecture of this implementation is shown in the Fig. 5. The generated code, in the figure called DiagnosticExecution.cpp, is a particular C task of the PLC which, sampling the PLC global variables, follows the evolution of the diagnostic automaton and, if the time-out expires, it executes the timed diagnostic analysis trying to isolate the occurred fault. This analysis has been implemented in a set of functions stored in a C library, TiDiaM.h, called by the TiDiaM task, which is automatically generated starting from the diagnostic automaton. The latter is synthesized with the DESUMA/UMDES, a software developed from the University of Michigan, which permits the generation of the automatons of the system (plant and control model) and the diagnostic automaton. The aim of the generated code and relative library of function is to produce the diagnostic results in a “small” amount of time, according with the timed analysis described above. The obtained results are shown in the user interface of the softPLC and forwarded to the upper level diagnostic methods and alarms managers.

IV. THE APPLICATION TESTS The TiDiaM has been applied on the Pallet Changer, Fig. 6,

which is a subsystem of the industrial scenario described in previous paragraph.

Figure 6. Image of the real pallet changer

16

Figure 7. Multi-level Control schema for the pallet changer

This subsystem is a part of the SPI Machine used to load / unload pieces to / from the workarea. It is composed of three different equipments: a vertical translation device (pneumatic), called ZTranslator, a vertical rotation one (electrical), called ZRotator, and a piece locker (pneumatic) in turn composed of four plates/clamps to lock/unlock pieces, called LockParts.

The control of this sub-system has been developed with the model-based approach as described in the previously paragraphs. The structure of the control for the pallet changer is described in the Fig. 7. The TiDiaM has been applied to each pallet changer subsystem. This decomposition has permitted to reduce the complexity of the final diagnostic system following the one made for the control model.

Each plant component of the pallet changer has been modeled with two persistent faults. For each component Xi (a “logic” actuator or a sensor) two faults are modeled: the “stuck open” fault (unobservable event stuck_X_0), and the “stuck closed” fault (unobservable event stuck_X_1). The summary of the pallet changer TiDiaMs are showed in the Tab. 1. An example of real case with the relative model mapping is presented in the Fig. 8. To reduce the complexity of the diagnostic automaton in the ZRotator TiDiaM has been used the algorithm to model components of the same type as a unique equivalent component. In this case, in the real system there are two strobe sensors, which signal the movement of the table: when the table is in maximum or minimum positions the strobe sensors signals are 1, while in other positions the signals are 0. The two sensors are redundant, so they signal the same values and, differently from the micro sensors which measure the table positions, they measure the transmission positions. To reduce the complexity of the diagnostic automaton the strobe sensors are modeled as a unique equivalent strobe sensor.

Figure 8. Example to mapping model in the ZTranslator device

Instead the LockParts device has been divided in two sides which have the same diagnostic model. In this way we have two TiDiaMs for this pallet changer subsystem, LockPartsSide1 and LockPartsSide2 which are instances of the same type. In each instances there are three micro sensors for each plate that measure the unlock state of the piston. To reduce the complexity of the diagnostic automaton a new modelization has been introduced gathering the set of micro sensors of each plate into a unique equivalent piston.

Each TiDiaM has been generated using DESUMA/UMDES that is a software used to edit and to operate with the finite state machine. The obtained models text files have been automatically converted in the C code for the Orchestra platform using a Java program. The tests have been made in two different scenarios: the simulation-based and the real one. For the first scenario the architecture described in the Fig. 8 has been developed

In this case the pallet changer has been simulated using a SIMBA (Simulation of Model Based Automatic systems) which is a simulation architecture that permits to create and execute real-time simulations with a 3D graphical representation of the systems. Instead, the control has been implemented and executed with the softPLC Orchestra using the model-based approach of MEDEIA project. The serial communication has been applied in the testing architecture, Fig. 9; this choice has been driven by the facility to use it in an academic environment. Instead the real test has been executed using the architecture described in Fig. 10.

In this architecture, the real pallet changer with the same control code of the previously described scenario has been used. The communication, in this case, has been realized with Profibus (Process Field Bus), major details on this fieldbus are reported in [8].

Many tests, executed both with simulation test architecture both with real test architecture, have shown that the TiDiaM, under the hypothesis described before for the timed analysis, is a good method for isolation and identification of the faults in the manufacturing centre. Each fault that can occur in the whole pallet changer system (stuck open or close in the sensors or the “logic” actuators) has been isolated with the diagnostic methodology presented in this paper.

TABLE I. DIAGNOSTIC AUTOMATA FOR PALLET CHANGER DEVICE.

Equipment Diag. States

Components Model # I/O

# fault

ZTranslator 74 2 electrovales 2 switch sensors

4 8

ZRotator 627 2 relays 2 switch sensors 1 strobe sensor

5 10

LockParts Side1 377 2 electrovalves 2 switch sensors 1 pressure sensor

5 10

Side2 377 2 electrovalves 2 switch sensors 1 pressure sensor

5 10

Moreover the real tests have helped to discover many limits

on the application of the TiDiaM. The first encountered

17

problem is the synchronization through the controlled system and the TiDiaM. In fact, for a correct execution of the diagnostic automaton, the TiDiaM implementation must know precisely the initial state of the system. This doesn’t permit the start of the TiDiaM in any moment of the system execution, and it requires a rephasing procedure at each system restart. During the described tests a manually re-phasing of the system has been required in order to avoid this problem. Another fundamental point for the correct functioning of the TiDiaM application is the communication with the system. Obviously a fast and stable fieldbus communication is necessary in order to sample correctly the system signals avoiding loss of information. In the practical experience here described this topic has been achieved choosing Profibus as fieldbus.

Another issue of the TiDiaM is the complexity of the synthesized diagnostic automatons. The increase of the system components and of their control sequences entails a direct increase of the number of the models states. In addition, it’s clear that the increase of the complexity of the diagnostic automaton implies the increase of the dimension of the generated code for the specific platform implementation, in this case for the softPLC Orchestra platform. For example, the dimension of the Orchestra control code for the pallet changer is about 600 kilobytes, instead the dimension of the control with the relative TiDiaM implementations is about 2700 kilobytes. The diagnostic code dimension is about two or three times larger than the control code dimension. In the MEDEIA demonstration this has not been an important hazard because Orchestra is a softPLC so that it hasn’t the same hardware restrictions as a classical industrial PLC.

V. CONCLUDING REMARKS The paper describes the application of a real-time diagnosis

system into a real industrial automated device. The method, originally tested in simulation, is based on the theoretical results of the diagnoser automaton, improved in the modeling – especially the control model, divided into nominal and non-nominal – and in the time-out insertion. Such improvements, suitably combined in a dedicated timed analysis, allow to obtain a practical application (TiDiaM) which has been implemented for a commercial PLC. Many tests have showed that the TiDiaM allows to isolate the faulty component even when all sensors and actuators are affected by faults, under the hypotheses of persistent faults, only one fault possible before the first time-out and faults from steady states. The main advantage of the TiDiaM is the automatic generation of the diagnostic code for the controller (such as the softPLC used in the test case), starting from library components for basic automation equipments.

Figure 9. Architecture of the simulation testing

Figure 10. Architecture of the real test case

As expected, experimental tests have shown also some weakness, and that putting a conceptually simple algorithm into practice and especially in a control loop for automation application is never trivial. So future research steps are needed to solve those limits; especially the identification of the initial state of the diagnoser, the minimum level of reliability and performance needed in the communication channel. For sure, a better formalization of the defined method and its isolation capabilities is required in order to understand under which conditions its performances can be replicated considering similar contexts.

ACKNOLEDGEMENT This work was supported in part by the MEDEIA

Consortium within the MEDEIA EU project (FP6-IST-016649), and, in particular, by MCM S.p.A., providing the real application example and its CAD models.

[1] Ferrarini, L., Brusa, R. and Veber, C. (2008). A Pragmatic Approach To

Fault Diagnosis In Hydraulic Circuits For Automated Machining: a case study, CASE 2008, 23-26.

[2] Ferrarini, L., Allevi, M. and Dede, A. (2010). Design and implementation of an automatic on-line diagnosis with TiDiaM and Tide, ETFA 2010.

[3] Ferrarini, L., Dedè, A. (2010). A model-Based Approach for Mixed Hardware In the Loop Simulation of Manufacturing Systems, CASE 2010, Toronto (Canada)

[4] Lafourtune, S., Teneketzis, D., Sampath, M., Sengputa, M. and Sinnamohideen, K. (2001). Failure Diagnosis of Dynamic Systems: An Approach Based on Discrete Event Systems. American Control Conf., 2058-2071.

[5] Lee, Seungjoo, and Tilbury, Dawn, (2008). A modular control design method for a flexible manufacturing cell including error handling Decision and Control, 2005 and 2005 European Control Conference. CDC-ECC '05 19(3), 308-330.

[6] Lin, E. (1993b). Diagnosability of discrete event systems and its application, Discrete Event Dinamic Systems, 4(2): 197-212.

[7] MEDEIA project, 2008, Model-Driven Embedded Sytems Design Environment for the Industrial Automation Sector, <http://www.medeia.eu>

[8] Profibus International 2011, PROFIBUS, < http://www.profibus.com/> [9] Sampath, M., Sengupta, R., Lafortune, S., Sinaamohideen, K. and

Teneketzis, D. (1995). Diagnosability of discrete event systems. IEEE Transactions on Automatic Control, 40(9), 1555–1575.

[10] Shah, S.S., Endsley, E.W., Lucas, M.R., Tilbury, D.M., 2002, "Reconfigurable Logic Control Using Modular FSMs: Design, Verification, Implementation and integrated error handling", Proceedings of the 2002 American Control Conference, p. 4153 - 4158 vol.5.

[11] Sintesi SpA 2009, Orchestra Control Engine, <http://www.orchestracontrol.com >

18