Diagnosis Of Industrial Robots Using Case Based … · Diagnosis Of Industrial Robots Using Case Based Reasoning ... ABB Robotics is one of the worlds leading manufacturers of industrial

Diagnosis Of Industrial Robots Using Case

Based Reasoning

Erik Olsson [email protected]

A Master of Science thesis in Computer Science performed at Mälardalen University and

ABB Robotics during the summer and fall of 2003.

Supervisors:

Mats Jackson, ABB Robotics and Mälardalen University, Dep. of Design and Product Development

Peter Funk, Mälardalen University, Dep. of Computer Science and Engineering

Marcus Bengtsson, Mälardalen University Dep. of Design and Product Development

mailto:[email protected]

Abstract

ABB Robotics is one of the worlds leading manufacturers of industrial robots. The manufac-

turing process of an industrial robot involves procedures such as assembly, testing and paint-

ing. The testing procedure is critical when it comes to fault detection.

Faults can often be identified as abnormal sounds coming from one of the gearboxes on the

robot during the testing procedure. The decision whether such a sound should be regarded as a

fault or not can be difficult to make by just listening to it. Today that decision is made by the

testing personnel. This means that inexperienced testing personnel runs a risk of making a

wrong diagnosis and consequently let a faulty robot pass as normal.

This thesis explores automatic fault detection in production of robots. This is accomplished by

letting a computer classify the sound that comes from the gearboxes during the testing proce-

dure. A prototype based on wavelet analysis and case based reasoning has been implemented

in attempt to show that this is possible. The prototype has managed to identify two kinds of

faults on axis 4 on an industrial robot.

This report is a ABB internal version of a thesis required for the degree of Master of Science

in Computer Science at Mälardalen University. It was performed by Erik Olsson during the

summer and fall of 2003.

Acknowledgments

This thesis was written at ABB Robotics AB during the summer and fall of 2003. It was writ-

ten by Erik Olsson and forms the required thesis for the degree of Master of Science in Com-

puter Science at Mälardalen University. I want to thank my supervisors: Peter Funk and Mar-

cus Bengtsson at Mälardalen University and Mats Jackson at ABB Robotics. I also want to

thank Mats Åhgren and Christian Dahlberg for their compliance and the testing personnel for

putting up with all my recordings. At last I want to thank Ian Kaplan at Bear Products Interna-

tional [6] for his excellent implementation of the Wavelet Packet Algorithm.

2

1 Introduction............................................................................................................. 6 1.1 Overview .................................................................................................................. 6 1.2 Background............................................................................................................. 6 1.3 Aim ............................................................................................................................ 7 1.4 Related work ........................................................................................................... 7 1.5 Limitations............................................................................................................... 7

2 Theory .......................................................................................................................... 8 2.1 Introduction............................................................................................................. 8 2.2 Case based reasoning ......................................................................................... 8 2.3 Wavelets................................................................................................................. 10

3 The recorded sound....................................................................................... 16 3.1 Introduction........................................................................................................... 16 3.2 Choice of robot model and axis ...................................................................... 16 3.3 The ABB Industrial robot................................................................................... 16 3.4 Choice of axis....................................................................................................... 17 3.5 The testing procedure ........................................................................................ 18 3.6 Choice of recording device............................................................................... 18 3.7 Attaching the recording device ....................................................................... 18 3.8 Recording the sound .......................................................................................... 20 3.9 The recordings ..................................................................................................... 20

3.9.1 Conclusions .................................................................................................. 22 3.10 Analysis of the axis4 sound ............................................................................. 23

3.10.1 Introduction................................................................................................... 23 3.10.2 The period ..................................................................................................... 23 3.10.3 Basic features............................................................................................... 23

4 The prototype ...................................................................................................... 25 4.1 Introduction........................................................................................................... 25 4.2 Problem description ........................................................................................... 25 4.3 Problem analysis ................................................................................................. 25

4.3.1 Methods ......................................................................................................... 26 4.4 Design..................................................................................................................... 26 4.5 Implementation .................................................................................................... 27

4.5.1 Dividing the signal into windows ............................................................ 27 4.5.2 Analysing the windows and extract frequency information ............ 28 4.5.3 Assemble a feature vector from the frequency information............ 28 4.5.4 Matching the feature vector and produce a guess............................. 29

4.6 Testing.................................................................................................................... 30 4.6.1 Introduction................................................................................................... 30 4.6.2 A normal sounding axis............................................................................. 30 4.6.3 Fault1: Loosely tightened gear wheel fault. ......................................... 32 4.6.4 Fault 2: Notch fault...................................................................................... 33 4.6.5 Faults 3: Notch fault ................................................................................... 34 4.6.6 Three candidate faults ............................................................................... 35 4.6.7 Example1 ....................................................................................................... 35 4.6.8 Example2 ....................................................................................................... 37

5 Conclusions .......................................................................................................... 38 5.1 Result...................................................................................................................... 38 5.2 Future work ........................................................................................................... 38

3

5.3 Summary................................................................................................................ 39 References ..................................................................................................................... 40 Appendix A, Graphical plots of sound recordings ..................... 41 Table 1: Example of cases representing car failures......................................................... 8 Figure 1: Matching cases. ...................................................................................................... 9 Figure 2: The CBR model. (Agnar Aamodt, Norwegian University of Science and

Technology).................................................................................................................... 10 Figure 3: Graphs of the soundwaves produced by a tuning fork (left) and the spoken

word “greasy” (right). (Ofer Levi, Stanford University)............................................. 11 Figure 4: The Mexican Hat wavelet. ................................................................................... 11 Figure 5: A cosine wave. ...................................................................................................... 12 Figure 6: The wavelet passes along the cosine wave and different integral values are

produced. ........................................................................................................................ 12 Figure 7: The wavelet is stretched and shrinked.............................................................. 13 Figure 8: The Daubechies wavelet. .................................................................................... 13 Figure 9: Input with 16 samples. ......................................................................................... 14 Figure 10: The input is divided into a low and a high pass version............................... 14 Figure 11: The result is a tree.............................................................................................. 15 Figure 12: The ABB industrial robot ................................................................................... 17 Figure 13: The Palm handheld computer is mounted in a box with magnets on. ....... 19 Figure 14: Position 1. ............................................................................................................ 19 Figure 15: Position 2. ............................................................................................................ 20 Figure 16: Axis 4 sound........................................................................................................ 23 Figure 17: Axis 4 sound at 6600Hz. ................................................................................... 24 Figure 18: Axis 4 sound at 1140Hz. ................................................................................... 24 Figure 19: The design of the prototype. ............................................................................. 27 Figure 20: Flowchart of the signal division ........................................................................ 27 Figure 21: The matching function ....................................................................................... 29 Figure 22: The frequency spectra of a normal sounding axis. ....................................... 31 Figure 23: The frequency spectra of a loose gear wheel fault. ...................................... 32 Figure 24: The feature vector for a loose gear fault......................................................... 32 Figure 25: Feature vector for the notch fault..................................................................... 33 Figure 26: The frequency spectra of a notch fault. .......................................................... 33 Figure 27: A profile of a notch fault..................................................................................... 34 Figure 28: Feature vector of a notch fault. ........................................................................ 34 Figure 29: A cross section of the tree is taken. ................................................................ 35 Figure 30: A_0729_1235_NoPayloadPos1 ....................................................................... 41 Figure 31: A_0721_NoPayloadPos1 .................................................................................. 42 Figure 32: A_0721_1301_NoPayloadPos1 ....................................................................... 42 Figure 33: A_0721_NoPayloadPos1_NotchFault ............................................................ 43 Figure 34: A_0824_PayloadPos1 ....................................................................................... 43 Figure 35: A_0824_PayloadPos2 ....................................................................................... 44 Figure 36: A_0826_PayloadPos1 ....................................................................................... 44 Figure 37: A_0826_PayloadPos2 ....................................................................................... 45 Figure 38: A_0907_0928_PayloadPos1 ............................................................................ 45 Figure 39: A_0907_0928_PayloadPos2 ............................................................................ 46 Figure 40: D_0907_PayloadPos1 ....................................................................................... 46

4

Figure 41: A_0918_NoPayloadPos1 .................................................................................. 47 Figure 42: A_0918_NoPayloadPos2 .................................................................................. 47 Figure 43: C_0825_PayloadPos1_LooseGearFault ........................................................ 48 Figure 44: C_0825_PayloadPos2_LooseGearFault ........................................................ 48 Figure 45: C_0827_PayloadPos1 ....................................................................................... 49 Figure 46: C_0827_PayloadPos2 ....................................................................................... 49 Figure 47: C_0906_NoPayloadPos1 .................................................................................. 50 Figure 48: C_0906_NoPayloadPos2 .................................................................................. 50 Figure 49: C_0918_PayloadPos1 ....................................................................................... 51 Figure 50: C_0918_PayloadPos2 ....................................................................................... 51 Figure 51: B_0716_NoPayloadPos1_NotchFault ............................................................ 52 Figure 52: B_0730_NoPayloadPos1_NotchFault ............................................................ 52 Figure 53: B_0816_1242_NoPayloadPos2 ....................................................................... 53 Figure 54: B_0914_PayloadPos1 ....................................................................................... 53 Figure 55: B_0914_PayloadPos2 ....................................................................................... 54

5

1 Introduction

1.1 Overview

The report consists of five chapters. Chapter 1 covers the background and the purpose of the

thesis. Chapter 2 gives a brief introduction to the underlying theories of the thesis. If the

reader already has the theoretical background, chapter 2 can be skipped. Chapter 3 considers

aspects of the robot sound and the recording devices used. Chapter 4 describes the design,

implementation and testing of a prototype system that is implemented as a part of the thesis.

Chapter 5 covers the conclusions made during this work and presents visions and ideas of a

further development of the prototype system.

1.2 Background

ABB Robotics AB is one of the worlds leading manufacturers of industrial robots. Their first

robot model, the IRB 6, was manufactured 1974 and was the worlds first microcomputer con-

trolled electrical robot. Since then they have produced and sold over 100 000 industrial robots

all over the world. The model range spans from the IRB 140 with a lifting capacity of 5 kg to

the IRB 7600 that can handle payloads up to 500 kg. Industrial robots are used in many kinds

of industries, from consumer goods industries to automotive and foundry industries.

The main part of the robots are assembled and tested at Finnslätten in Västerås, Sweden. The

assembly process of an industrial robot involves four main steps: assembly, testing, painting

and dressing. It is of great importance that the robots shipping from the factory in Finnslätten

are free from faults.

The testing step is critical when it comes to fault detection. If a robot has any faults due to

poor assembly routines or bad parts etc, it most likely will be discovered during the testing

procedure. When a robot is tested, it is placed in a testing area and a special test program runs

the robot to control its function. During the test program, strange noises sometimes can be

heard from the robot. Noises that can be the results of various defects in gears, bearings and

motors on the robot. Because there is no equipment to analyse these noises - the decision

whether such a noise comes from a faulty bearing, gear or electrical motor or if the noise is in

fact normal is very much up to the sense and feel of the testing personnel.

6

This thesis considers the idea of automatic fault detection. It gives a suggestion to hand assist

fault detection with a computer. A computer is more consistent than a human and a computer

has better memory. This report considers the underlying theory and implementation of a pro-

totype witch goal is to make a diagnosis of the condition of the robot with the aid of sound.

The following pages gives a detailed description of how the work was performed and how the

prototype was implemented.

1.3 Aim

The aim of this thesis is to find out whether sound recordings can be used to separate a faulty

industrial robot from a normal robot with a “smart tool” used by an engineer. Methods and

techniques from artificial intelligence and mathematics, such as case based reasoning and

wavelet analysis will be used for the implementation of a prototype with the aim to solve this

problem. If the problem can be solved, a system based on those principles can be used to aid

fault detection in industrial robots.

1.4 Related work

Several works in related areas has been performed. Anréus and Grante [1] has performed a

thesis at Volvo Wheel Loaders AB. The thesis covers spectral analysis of the sound produced

when wheel loader gearboxes are tested. Benko et al [2] covers fault detection in vacuum

cleaner motors. Mäkinen [3] discusses the advantages of vibration analysis on cooling com-

pressors. Vallin [4] has used electric current measurements in attempt to see if any indication

is given when the gears in axis 5 and 6 on are adjusted to tight or to loose. This thesis goes

one step further with the aim to build a prototype of a flexible and learning “smart tool”, used

by e.g. field engineers working in production.

1.5 Limitations

Only one axis on one robot model will be analysed for validation of the proposed method.

The design and implementation of the prototype will be focused on the sound from that axis.

In chapter 5 there is a discussion of how the ideas and solutions implemented in the prototype

can be extended to other axes and robot models.

7

2 Theory

2.1 Introduction

This chapter gives a short tutorial of case based reasoning and wavelet analysis. The aim with

this chapter is to provide the background theory needed to understand the design and imple-

mentation of the prototype system discussed in chapter 3. This chapter starts by giving a short

tutorial of case based reasoning which is a method used in the prototype. It continues to de-

scribe the theory behind wavelet analysis. In this thesis, wavelet analysis is used as a tool to

extract features that comprise cases in the prototype.

2.2 Case based reasoning

Case Based Reasoning (CBR) is an Artificial Intelligence (AI) I technique that is based upon

the way a human is learning things. If you run into a problem, the first thing you do is to try to

remember if you have solved a similar problem before. If you have, there is a big chance that

you can solve the problem by applying the same technique that you did the last time the prob-

lem occurred. If the problem is slightly different, there is still chance that you can solve it by

applying your old knowledge and alter it to fit the new problem. In this way you can learn

new things by reusing old knowledge. This is the fundamental idea of CBR – applying old

knowledge of problem solving to solve new problems.

A case is an abstraction of a procedure. The procedure can be anything that produces a useful

result. One example is the diagnosis of a car failure:

Case Starter engine

OK?

Is there gas? Headlight OK? Action

Case1 Yes Yes ? Call an expert

Case2 Yes No ? Fill up gas

Case3 No ? No Charge battery

Case4 No ? Yes Call an expert

Table 1: Example of cases representing car failures.

8

Here the case is the car repair procedure. It involves symptom, diagnosis and action.

Cases are stored in a case library for later reuse when similar cases occur. The case library is

updated with newer cases as they pop up. Similar cases might be merged together and old

cases that are no longer used might be deleted. When new cases occur, they are matched with

cases in the case library. One way of matching cases is to use the nearest neighbour algorithm.

Similar case

Border case

Figure 1: Matching cases.

The cases witch are nearest the new case has the best probability to present a correct solution

to the new case. If no cases in the case library are close enough, the new case becomes a new

contribution to the library. If different cases in the case library are within the same distance to

the new case they might be combined to present the correct solution. The new combined case

also becomes a new contribution to the case library. The framework for CBR is a cognitive

model for problem solving that is based on past experience, retrieve-reuse-revise-retain.

9

Figure 2: The CBR model. (Agnar Aamodt, Norwegian University of Science and Technol-

ogy).

2.3 Wavelets

The Wavelet theory is a further development of the Fourier theory. The Fourier theory says

that a periodic signal can be expressed as the sum of a, possibly infinite, series of sines and

cosines.

As an example, the tone of an instrument is a combination of its lowest frequency tone, called

the fundamental frequency of the tone, and the higher-frequency ones, called overtones. The

Fourier theory has the limitation that it cannot represent non-periodic signals. Periodic signals

are signals that repeat itself over a period of time like, for example, the tone from an instru-

ment. Non-periodic signals are for example a speech sound. The word “grease” is not a peri-

odic signal, it contains a mix of frequencies with sharp peaks and slopes. But the sound from

a tuning fork is periodic.

10

Figure 3: Graphs of the soundwaves produced by a tuning fork (left) and the spoken word

“greasy” (right). (Ofer Levi, Stanford University)

A periodic signal is said to be localised in frequency whereas a non-periodic signal is said to

be localised in time.

A wavelet has the advantage that it can do both, the wavelet package transform separates a

wave into “time-frequency packets”. A wavelet consists of a basis function, the mother wave-

let. It can be any function witch positive and negative areas cancel out.

One example is the Mexican hat wavelet. It is called so because it looks like looks like an up-

side down sombrero.

Figure 4: The Mexican Hat wavelet.

The function describing the Mexican hat is:

11

As an example, let’s see how the Mexican hat wavelet is used to analyse a cosine wave.

Let be the cosine wave.

Figure 5: A cosine wave.

The wavelet is shifted alongside the signal and a match is calculated as the integral between

the signal and the wavelet at any given time.

As the wavelet slides along the signal, the absolute value of the integral between the signal

and the wavelet will increase when the wavelet is in phase with the signal and decrease when

the wavelet slides out of phase with the signal. The opposite will occur when the wavelet

slides in phase again.

Figure 6: The wavelet passes along the cosine wave and different integral values are pro-

duced.

12

Now lets stretch and shrink the wavelet along the time axis by changing the duration time of

the wavelet function. When sliding the stretched and shrinked versions of the wavelet func-

tion along the signal, different integral results will be produced.

Figure 7: The wavelet is stretched and shrinked.

As can be seen in the picture, line number three gives the best match against the original sig-

nal. The wavelet transform is shown in dashed lines and the wavelet function is shown in dot-

ted lines. In the end the result will be a collection of time-frequency representations of the

signal, all with different resolutions. The stretched wavelet will match signals with lower fre-

quencies and the shrinked wavelet will match higher frequencies.

The wavelet algorithm that is used in this thesis project is called the Daubechies wavelet

packet transform. The Daubechies wavelet is named after its inventor, the mathematician

Ingrid Daubechies.

Figure 8: The Daubechies wavelet.

13

Wavelet analysis can be used to analyse the frequency spectra in various measurement data,

for example stock market curves, temperatures or, as in this case, sampled sound. Sampled

sound consists of a number of measurements that spans over a certain time. The number of

samples taken each second is called the sampling rate, or the sampling frequency (measured

in Hz). The number of bits used to represent each sample is called the resolution (measured in

bits).

The Daubechies wavelet packet transform algorithm separates the original signal into “time-

frequency-packets”, this is done by applying the wavelet function recursively over and over

again until the input signal is of length 2. As an example, lets assume that the input signal to

the algorithm consists of 16 samples.

Figure 9: Input with 16 samples.

The nature of the algorithm forces the size of the input data to be in a power of two. The algo-

rithm applies the Daubechies wavelet function to the input data and thereby separates the

original data into two units.

Figure 10: The input is divided into a low and a high pass version.

hese units can be seen as a high and a low pass filtered version of the original data. The low

he wavelet function is again applied to each filtered unit and the result is copied into the

T

pass filtered data is in the left of the original data array and the high pass filtered data is in the

right of the original data array.

T

original array. This process is then repeated until the high and low pass units are of length 20

and no more splitting is possible. The result can be seen as a tree with the original data as the

root of the tree and the smallest high and low pass units as the leaves of the tree.

14

Figure 11: The result is a tree.

The depth of the tree is determined by the length of the input data. If the input data is of

length 2n the tree depth will be n. A suitable collection of time frequency packets can be se-

lected by taking a cross section of the tree at a depth. Each sibling in the cross section of the

tree spans the entire time of the original data set. This means that going deeper in the tree

gives at better resolution in frequency but a poorer resolution in time. The best compromise

between time and frequency resolution is to take a cross section in the tree where the length of

each sibling is the same as the number of siblings in the cross section. At a given depth n and

with original data size S, the length of a sibling (or leaf) is S/2n and the number of siblings is

2n.

When a low or high pass filtered version of the data is calculated, the data is interpolated on

the mother wavelet function. The difference between the interpolated values and the original

values is the result that is saved in the array. To learn more about the wavelet packet algo-

rithm, see Ian Kaplan’s page [5] or Amara’s links and tutorials [6].

15

3 The recorded sound

3.1 Introduction

This chapter covers the sound recordings that are used by the prototype for analysis. It first

describes the robot model and axis that is chosen for the recordings. All sound recordings

where made during the testing procedure of the robots. This procedure is also described. The

recording device is explained and after that a discussion of the characteristics of the recorded

sound is given.

3.2 Choice of robot model and axis

Three problems must be solved to obtain the sound data:

1. Choose a robot model and axis to record

2. Choose a recording device

3. Find a way of attaching the recording device on the robot

The robot model selected for sound recording is an industrial robot produced by ABB Robot-

ics. Several robots have been recorded. A robot model that tended to be more error prone dur-

ing testing than other robot models was selected. The axes of this model are exposed to great

torque during operation.

3.3 The ABB Industrial robot

The ABB Industrial robot can handle five different sizes of payloads.

It comes in five versions:

A, B, C, D and E

It is used for process applications in many types of industries. Typical areas are spotwelding,

material handling and machine tending.

16

Figure 12: The ABB industrial robot

The ABB industrial robot has six axes witch gives it a wide area of operation. The axes are

numbered from bottom to top as axis 1 to 6.

3.4 Choice of axis

Axis number 4 was selected for recording. It was selected because the first and second fault

occurred on it (see chapter 4). It therefore became natural to focus on it. Axis 4 rotates the

wrist. Its working range is +300° to -300° and its max speed is 150°/s.

17

3.5 The testing procedure

All robots endure the same testing procedure before they are delivered to the customer.

The testing procedure contains of six steps:

1. Manual operation

2. Separate axis test

3. Brake test

4. Automatic operation test

5. Battery test

6. Dip-control

3.6 Choice of recording device

A Palm handheld computer was selected for recording. It is a small handheld computer with

an integrated microphone.

The selected recording software was Audacity Digital Voice Recorder [7].

3.7 Attaching the recording device

The PDA (personal digital assistance) handheld computer is mounted in a small box. A hole

was drilled in the box to let the sound reach the microphone. Magnets are attached to the

backside of the box to provide a way of attaching the box on the robot. This solution was se-

lected due to some technical problems with the wireless blue tooth microphone that would

enable the user to make the recording by attaching a small magnetic microphone.

18

Figure 13: The Palm handheld computer is mounted in a box with magnets on.

Position 1:

This position is close to the motor that operates the axis 4 but not equally close to the gear

wheels in the gearbox.

Figure 14: Position 1.

19

Position 2:

This position is equally close to the motor as to the gear wheels.

Figure 15: Position 2.

Recordings where performed in both positions se next subchapter for a discussion about the

positions.

3.8 Recording the sound

All recordings were made during the testing procedure.

A button on the handheld starts the sound recording. The test cycle is set to separate axis test

and axis number four is selected. The recording format is WAV, the sampling frequency is

8kHz and the resolution is 16 bits. The recording is in mono. The sound file is then trans-

ferred to a stationary computer in witch the prototype is implemented.

3.9 The recordings

17 different robots have been recorded. 11 of them have two recordings made with two differ-

ent positions of the microphone (the Palm handheld). This was done to find out if a change in

position of the microphone affects the features of the recorded sound. 9 robots had the testing

payload attached and 8 robots had not.

The file names are to be interpreted as robot model_date of recording_payload/no payload

position of microphone_type of fault (if any)

20

Two types of faults have been identified in the recordings, the fault types are:

1. Notch fault

2. Loose gear wheel fault

See chapter 5 for more information about the recorded faults.

All recordings can be seen as plots in the appendix.

The following recordings was performed in position 1:

Recordings performed on robots with no payloads:

A_0721_NoPayloadPos1_NotchFault.WAV

B_0716_NoPayloadPos1_NotchFault.WAV

B_0730_NoPayloadPos1_NotchFault.WAV

0729_1235_NoPayloadPos1.WAV

A_0721_1301_NoPayloadPos1.WAV

A_0721_NoPayloadPos1.WAV


C_0906_NoPayloadPos1.WAV

Recordings performed on robots with payloads:

C_0825_PayloadPos1_LooseGearFault.WAV

A_0824_PayloadPos1.WAV


A_0907_0928_PayloadPos1.WAV


C_0827_PayloadPos1.WAV


B_0914_PayloadPos1.WAV

D_0907_PayloadPos1.WAV

21

The following recordings are performed in position 2.

Recordings performed on robots with no payloads:


C_0906_NoPayloadPos2.WAV

B_0816_1242_NoPayloadPos2.WAV

Recordings done on robots with payloads:

C_0825_PayloadPos2_LooseGearFault.WAV







B_0914_PayloadPos2.WAV

3.9.1 Conclusions

Recordings made in position 1 are not as good as recordings in position 2, this includes both

recordings with payloads and recordings without payloads. Even though, some faults have

been identified in position 1. This is because position 1 was the original position and some

recordings were made only in that position.

Robots with payloads have more outstanding peaks in the plots compared to robots without

payloads (see appendix). There is a risk that some of those peaks can be misinterpreted as

faults in the matching function.

22

3.10 Analysis of the axis4 sound

3.10.1 Introduction The sound made from axis4 when the arm rotates from start position to end position has cer-

tain characteristics. These characteristics make the foundation of the way the prototype is im-

plemented and how it behaves.

3.10.2 The period The sound made from axis4 when the arm rotates from start position to end position is called

a period. The period is characterised by a bump in the plot below. Two periods are separated

by a short time of silence. In the plot below the silence can be seen as a thin line between two

bumps. The silence comes from when the robot arm changes its direction of rotation.

-40000-30000-20000-10000

0 10000 20000 30000

0 20000 40000 60000 80000 100000 120000 140000

"vekt.txt"

Figure 16: Axis 4 sound.

The period is taken directly from the raw wav data. This makes up the input to the prototype.

When the period looks like this, it is unfiltered, it contains all frequencies from 1Hz (and be-

low), up to 8kHz witch is the max recording frequency.

3.10.3 Basic features

There is much information hidden in the period that cannot directly be obtained when the pe-

riod is in this format.

The wavelet analysis algorithm, filter out various frequency information from the period and

provides a way of finding “hidden” features in it.

23

At 6600Hz the bumps are easy to distinguish from each other.

-20000-15000-10000-5000

0 5000

10000 15000 20000

0 100 200 300 400 500 600

"plot220"

Figure 17: Axis 4 sound at 6600Hz.

At 1140Hz there is easy to distinguish in which way the arm is rotating. The higher amplitude

bumps rotates the opposite way as the lesser amplitude bumps.

-100000-80000-60000-40000-20000

0 20000 40000 60000 80000

100000

0 100 200 300 400 500 600

"plot38"

Figure 18: Axis 4 sound at 1140Hz.

24

4 The prototype

4.1 Introduction

This chapter covers the implementation of a prototype system. It first describes and analyses

the given problem. Methods are given to solve the problem and a design is presented. The im-

plementation of the design is described and some examples of the testing procedure are pre-

sented.

4.2 Problem description

The task is to implement a prototype that uses sound to diagnose faults in industrial robots.

The prototype must be able to take a recorded sound as input and make a diagnosis of the

sound. To make a diagnosis, the prototype must be able to find features in the sound that cap-

tures the faults in the sound.

The time for the project is limited to 20 weeks. To be able to solve the task within that time,

only one axis on a specific robot model is to chosen for recordings.

4.3 Problem analysis

The implementation of the prototype involves:

1. Design

2. Implementation

3. Testing

25

4.3.1 Methods

Two well known methods will be used in the prototype:

1. Wavelet analysis

Wavelet analysis will be used for feature extraction. Wavelet analysis provides a tool

of making a spectral analysis of the sound. This will aid the feature extraction process

as hidden features can be identified in the various sound spectra.

2. Cased based reasoning

Cased based reasoning will be used for diagnosis. Case based reasoning has an advan-

tage over neural nets because it requires no training data in order to make a diagnosis.

Features of a fault can instantly be saved in the case library and later faults can be

matched to these features.

The prototype is implemented on a regular IBM PC. The operating system is Windows NT

and the programming environment is Microsoft Visual Studio 6.0.

The programming language is C++.

4.4 Design

The prototype takes a wav sound file as input and produces a diagnosis of the sound file as

output. This is achieved in a pipe design involving four steps. Each step in the pipe can be

seen as a function with an input and an output.

The prototype consists of four steps

1. Dividing the signal into periods

2. Analysing the periods and extract frequency information

3. Assemble a feature vector from the frequency information

4. Matching the feature vector and produce a guess

26

1. Dividing

the signal

into peri-

ods

-GuessSignal

Figure 19: The design

4.5 Implementation

4.5.1 Dividing the signal in

The first step before the perio

cording. A function: “calibrat

amplitude level of silence and

When the calibration is finishe

riod is a rotation of the robot a

This is performed in three step

3.

2.

1.

Figure 20: Flo

Explanation of figure 20:

1) Sound data are

2) The period is re3) The rest of the w

2. Analys-

ing the pe-

riods and

extract fre

quency i

formation

n-

of the prototype.

to windows

d division starts is

e” calculates a mean

sound in the recordin

d - the next step is

rm.

s:

AddZero(Signal)

ReadSound(Signal)

FindNoice(Signal)

wchart of the signal

read from the file un

ad from the file. indow is filled with

3. Assem-

ble a fea-

ture vector

from the

frequency

information

to calibrate the amp

value of the record

g.

to divide the signal i

division

til a period is found.

zeros.

4. Match-

ing the fea-

ture vector

and pro-

duce a

guess

litude level of the re-

ing and estimates the

nto periods. Every pe-

27

The length of each audio data block (window) that is passed to the next step in the pipeline is

set to 32768 (215), witch is about 4 seconds of continuous sound. One rotation of the robot

arm lasts for about three seconds. The rest of the window is filled with zeros.

4.5.2 Analysing the windows and extract frequency information

The window has a length of about four seconds. It contains one period of robot arm move-

ment. Every period contains a rotation of the robot arm.

Each window is then sent to the wavelet analysis algorithm for /frequency analysis.

The wavelet analysis algorithm makes a time/frequency packet tree of the input data window.

A cross section at level eight of the wavelet packet tree is taken. Level eight is chosen because

it makes a good compromise between time and frequency resolution. At level 8, there are 256

(28) time/frequency packets.

The sampling rate is 8kHz. This means that every time/frequency packet has a frequency in-

terval of about 30Hz (8000/256).

4.5.3 Assemble a feature vector from the frequency information

One value is taken from each packet in the cross section. The value is the maximum absolute

value in each packet. Each value is saved in the feature vector.

4.5.3.1 The feature vector

The feature vector is a way to represent the sound from a period in an easy and comparable

way. Another important feature of the feature vector is to reduce the dimension of the original

data. The feature vector has a dimension of 256 values. Each value is the maximum absolute

value taken from each packet in the wavelet packet tree.

28

Feature vector data type:

[value1,value2, …, value256]

4.5.4 Matching the feature vector and produce a guess

When a feature vector is assembled, it is normalized, matched and stored in a case library.

The case library consists of vectors from older recordings.

The matching function calculates the Euclidian distance between the new feature vector and

the cases stored in the case library. The result is a list sorted by the distance between the new

feature vector and the cases in the case library.

Case library

Case

Sorted list with results

Retrieve

Case Feature vector

Reuse

Match

Feataure vector

Figure 21: The matching function

29

4.6 Testing

4.6.1 Introduction Two types of faults have been recorded. Here, the faults will be explained in detail and the

special features contained in them will be discussed. The special features in the faults can

later be used to identify similar faults. The faults have been identified by experienced testing

personnel before they were recorded.

The faults are:

1. Loosely tightened gear wheel fault. Caused by a slack between gear wheels in axis 4.

Can be heard as four bumps in a short time interval between each rotation in a period.

2. Notch fault. Caused by a notch on the big gear wheel in axis 4.

Can be heard as a bump in the second rotation of the period.

3. Notch fault. Caused by notches on the big gear wheel in axis 4.

Difficult to hear

4. Three candidate faults (not verified).

Sounds like a loosely tightened gear wheel.

4.6.2 A normal sounding axis

The features of the faults are easier to spot if being compared to a normal sounding axis. A

normal sound from axis 4 during a period of the separate axis test is characterised by a con-

tinuous humming sound. Sometimes (but not always) a light bump can be heard at the end of

the continuing sound. This bump is made when the axis changes its direction of rotation. The

amplitude of this bump is a good starting indication whether the sound from a period is nor-

mal or not. The picture below shows a typical normal sounding axis.

30

Figure 22: The frequency spectra of a normal sounding axis.

The plot shows the frequency spectrum of two normal periods. The z-axis shows the period

length, the y-axis shows the amplitude and the x-axis shows the frequency. The bump at the

end of each period is prominent from the rest of the sound in the period. The amplitude of the

bump is low and consistent in the entire frequency spectra and there are no other prominent

features in the plot. This is an ideal sound. The plot shows a robot with no payload attached.

If a payload is attached, some outstanding peaks in the lower frequency spectra can occur but

the amplitude of the peaks is not as high as in the plots displaying a loose gear fault (see the

appendix for a full collection of plots).

The recorded faults all have some basic characteristics:

1. The loosely tightened gear wheel faults have peaks of abnormal amplitude at the be-

ginning and end of each rotation in the lower frequency spectra.

2. The notch faults have peaks caused by the notches at various positions and frequencies

within a rotation of a period.

In the rest of the chapter, the faults will be explained more in detail.

31

4.6.3 Fault1: Loosely tightened gear wheel fault.

This fault is caused due to a slack between the gear wheels in the gearbox. The sound can be

heard as four bumps in a short interval of time at the end of each rotation in the period. Below

is a plot that shows the sound from a recording performed on a robot of type B.

Figure 23: The frequency spectra of a loose gear wheel fault.

The first thing to notice in the plot is the abnormally high peaks in the lower frequency spec-

tra at the end of each period The peaks coincide with the recorded sound in which fours

bumps in a short interval of time can be heard at the end of each rotation.

The feature vector representing this fault captures the abnormalities in the sound:

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0 50 100 150 200 250 300

line 1

Figure 24: The feature vector for a loose gear fault

The frequency of the most prominent peaks occurs in packets #5 to packet #10 that is in the

frequency spectra from 150Hz-300Hz (5*30Hz-10*30Hz).

32

4.6.4 Fault 2: Notch fault

This fault is caused by a notch on the big gear wheel in the gearbox of axis 4. It can be heard

as a bump in the beginning of the rotation of the period. Equal to the case of the loose gear

wheel fault, this fault is easy to discern in the period. Except from the abnormal amplitude of

the bumps at the end of each rotation, there is a very prominent peak in the beginning of the

rotation. It can be seen in the plot at the lower frequency spectra of the period.

The feature vector representing this fault:

0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.1 0.11 0.12 0.13

0 50 100 150 200 250 300

line 1

Figure 25: Feature vector for the notch fault.

The position of the peaks agrees with the spectral plot and with the bump in the recorded

sound.

Figure 26: The frequency spectra of a notch fault.

33

4.6.5 Faults 3: Notch fault

These faults are caused by (probably several) notches on one of the gear wheels of axis 4. The

faults are difficult to hear in the recordings and can only be discovered by trained testing per-

sonnel. In the plot they are quite easy to spot if seen from the right angle.

Fault 3 has at least one abnormal peak in each period in the spectral plot. The plot shows two

periods:

Figure 27: A profile of a notch fault.

The peaks are captured in the feature vector. Mainly at frequency #175:

0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.1 0.11

0 50 100 150 200 250 300

line 1

Figure 28: Feature vector of a notch fault.

34

4.6.6 Three candidate faults

These faults are not verified but the matching function matches them close to a loose gear

fault. See the example in the next chapter. Their sound and plots resembles the verified loose

gear fault.

See appendix for plots on:

D_0918_PayloadPos1.WAV



4.6.7 Example1

A robot with a loose gear fault, was recorded on the 25th of August. The name of the sound

file is C_0825_PayloadPos2_LooseGearFault.WAV The sound from a loose gear is character-

ised by four bumps at the end of each rotation.

The sound is copied from the Palm handheld computer into the PC with the prototype.

In the prototype the wav sound file is first divided into windows of a length of 4 seconds.

Each window is analysed in the wavelet packet algorithm and each packet in the cross section

of the time/frequency packet tree at level 8 is examined and 256 values are assembled to a

feature vector.

Figure 29: A cross section of the tree is taken.

The matching function then matches the vector with all the other vectors stored in the library.

35

Beneath, the top ten matches are displayed. The left column displays the name of the re-

cording. The right column shows the Euclidian distance between the vectors. The maximum

distance between two normalized vectors is 1.

Matching with C_0825_PayloadPos2_LooseGearFault.WAV:

C_0825_PayloadPos1_LooseGearFault.WAV = 0.143330

D_0918_PayloadPos1.WAV = 0.144296

A_0907_PayloadPos2.WAV = 0.145872

C_0827_PayloadPos2.WAV = 0.148169

A_0824_PayloadPos2.WAV = 0.149204

A_0721_NoPayloadPos1_NotchFault.WAV = 0.150612

C_0816_1242_NoPayloadPos2.WAV = 0.152333

A_PayloadPos1.WAV = 0.155065

A_0918_NoPayloadPos2.WAV = 0.156066

C_0918_PayloadPos2.WAV = 0.156285

A comparison between the feature vector from C_0825_PayloadPos2_LooseGearFault.WAV

is displayed in the plot below:

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

0 50 100 150 200 250 300

line 1

The feature vector from D_0918_PayloadPos1.WAV displayed in the plot below:

36

0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.1 0.11 0.12

0 50 100 150 200 250 300

line 1

The plots show that they both have prominent peaks at frequency #70. The dip around fre-

quency #165 is also represented in both plots.

4.6.8 Example2 A robot with a notch fault, was recorded on 16th of July. The name of the sound file is

C_0825_PayloadPos2_LooseGearFault.WAV The sound from a loose gear is characterised by

four bumps at the end of each rotation.

Beneath, the top ten matches are displayed. The left column displays the name of the re-

cording.

Matching with 175-2.8_0716_NoPayloadPos1_NotchFault.WAV:

175-2.8_0730_NoPayloadPos1_NotchFault.WAV = 0.011827

125-3.2_0721_NoPayloadPos1.WAV = 0.148176

125-3.2_0824_PayloadPos1.WAV = 0.150532

0729_1235_NoPayloadPos1.WAV = 0.151999


175-2.55_0918_PayloadPos1.WAV = 0.154718

175-2.8_0914_PayloadPos2.WAV = 0.156401

175-2.8_0816_1242_NoPayloadPos2.WAV = 0.157412


175-2.8_0914_PayloadPos1.WAV = 0.159770

37

5 Conclusions 5.1 Result

A prototype has been implemented as proof the concept. The prototype takes a recording as

input and produces a list of matches as out put. The matches are older recordings stored in a

case database. The prototype has been successful in detecting both types of the recorded faults

A total of 17 different robots have been recorded and used in the evaluation of the prototype.

11 of them have two recording done with two different positions of the microphone. In both

positions of the microphone, faults have successfully been identified.

The prototype is not a “research prototype” and not a complete application and it is not ready

to be used as one either. It is limited to one robot model and one axis and it is not tested on

recordings performed with other types of microphones. The rotation speed of the axis is fixed

in the prototype. It means that other rotation speeds of the robot axis cannot be used.

5.2 Future work

A further development of the prototype is suggested to make it more general.

Desirable improvements include:

1. Variable axis speed in recordings.

2. The possibility to diagnose recordings of other axes and other robot models.

3. A modification of the feature vector to improve the ability of representing faults and

matching of faulty sounds.

38

5.3 Summary

This thesis shows that sound analysis proves useful in detection of mechanical faults in indus-

trial robots. Two types of faults have been detected and identified by a prototype system im-

plemented as a part of the thesis. The prototype is implemented in C++ and uses wavelet

analysis combined with case based reasoning as methods to extract features and make a diag-

nosis of the recorded sound. The prototype is more flexible since one recording of a faulty

sound is sufficient to identify similar faults in the future. No further training or interaction

with experts in sound analysis is necessary. This makes the “smart tool” flexible and it will

improve performance as more recordings of faulty robots are made on field. The aim of the

thesis project where met.

39

References

[1] Anreus H., Grante A. Utveckling av mätmetod för ljudprovning av transmissioner i

anslutning till produktionen. Examensarbete MMK 2003:46 MME 735.

[2] Benko U., Petrovcic J., Juricic D., Tavcar J., Rejec J. Quality End-Test Of Vacuum

Cleaner Motors Based On Sound Analysis.

[3] Mäkinen E. J., Vibrationsanalys på kylkompressorer. Examensarbete YTH-Underhåll,

Högskolan i Karlstad 1994,

http://www.kylteknikern-em.se/vibrationsanalys.htm

[4] Vallin K. Mätdatorer. ABB Robotics avd ROP/PA 97.

[5] Ian Kaplan´s wavelet packet algorithm

http://www.bearcave.com/misl/misl_tech/wavelets/index.html

[6] Various wavelet tutorials

http://www.amara.com/current/wavelet.html

[7] Audacity sound recording software

http://www.audiost.com/advr.htm

40

http://www.kylteknikern-em.se/vibrationsanalys.htm

http://www.bearcave.com/misl/misl_tech/wavelets/index.html

http://www.amara.com/current/wavelet.html

http://www.audiost.com/advr.htm

Appendix A, Graphical plots of sound recordings In this section, all sound recordings are presented as graphical plots. The plots are intermedi-

ate results extracted from within the prototype. They are extracted between step 2 and step 3

in the system model (see chapter 4).

Figure 30: A_0729_1235_NoPayloadPos1

41

Figure 31: A_0721_NoPayloadPos1

Figure 32: A_0721_1301_NoPayloadPos1

42

Figure 33: A_0721_NoPayloadPos1_NotchFault

Figure 34: A_0824_PayloadPos1

43



44


Figure 38: A_0907_0928_PayloadPos1

45

Figure 39: A_0907_0928_PayloadPos2

Figure 40: D_0907_PayloadPos1

46



47

Figure 43: C_0825_PayloadPos1_LooseGearFault

Figure 44: C_0825_PayloadPos2_LooseGearFault

48

Figure 45: C_0827_PayloadPos1


49

Figure 47: C_0906_NoPayloadPos1

Figure 48: C_0906_NoPayloadPos2

50



51

Figure 51: B_0716_NoPayloadPos1_NotchFault

Figure 52: B_0730_NoPayloadPos1_NotchFault

52

Figure 53: B_0816_1242_NoPayloadPos2

Figure 54: B_0914_PayloadPos1

53

Figure 55: B_0914_PayloadPos2

54

Documents

Diagnosis Of Industrial Robots Using Case Based … · Diagnosis Of Industrial Robots Using Case Based Reasoning ... ABB Robotics is one of the worlds leading manufacturers of industrial