Journal of Materials Processing Technology

ELSEVIER Journa l o f Materials Process ing Technology 48 (1995) 43--49

The development of a machine vision system for adaptive bending of sheet metals

S.K. Kwok and W.B. Lee

Department of Manufacturing Engineering, Hong Kong Polytechnic, Hung Horn, Hong Kong

Press-brake bending is a common forming process widely used in the sheet metal industry. One of the major problems in the bending operation is the control of the amount of springback. A sheet metal bending machine which is capable of on-line measurement of the punch load and punch displacement data has been built. A computer vision system is incorporated in the machine to capture the image of the bend, carry out features extraction and image analysis to find out the profile of the bend. The image data are fed into a plastomechanical model which will compute the required punch travel to achieve a given angle of the bend. These image data can also be used to monitor the bending process and control the product quality. The inherent variation in material properties is taken into account during each bending operation.

1. INTRODUCTION

In press-brake forming, a flat sheet is placed over a die and a punch is lowered into the sheet to form the bend. The sheet undergoes elastic recovery and springback. An accurate prediction of the springback problem is often difficult as sheet metal properties often vary within a coil and from coil to coil. Recent development in unmanned flexible sheet metal manufacturing cell leads to the research interest in intelligent bending machines that can take into account the inherent variation in the mechanical properties of the materials during the bending operations [ 1 ].

Many research works have been done in the past to overcome the springback problem in bending. The notable one is the work by Stelson [2]. Based on the punch force and displacement data, Stelson suggested an adaptive control algorithm to control the maximum punch travel to give a desired bend angle. The bending behaviour of the workpiece was described by the moment-curvature (M-K) relationship. Although Stelson's method is simple and fast, it works only for small rotation only (i.e., when the flank angle is small), as his model is based on small deflection of a beam cantilever [3]. When the deformation in the workpiece is large, the M-K relationship thus derived from small rotation would

0924-0136/95/$09.50 © 1995 Elsevier Science S.A. SSDI 0 9 2 4 - 0 1 3 6 ( 9 4 ) 0 1 6 3 1 - A

not be accurate. A control algorithm of the bending process

based on a plastomechanical model which takes into account the strain hardening characteristics of the materials, and the large bend angle encountered in industrial bending operations has been developed. A computer vision system is incorporated in the bending machine. The system captures the image of the bend, carries out features extraction and image analysis to find out the equation of the flank angle of the bend, which is used for on-line determination of the punch travel to give the desired bend angle.

2. THE MATHEMATICAL BENDING MODEL

A schematic diagram of the microcomputer control of the press-brake bending process is shown in Fig.1. The sheet is assumed to be deformed by bending alone and the shear and tensile deformation are neglected. The degree of springback for a given punch travel is predicted from the moment-curvature relationship which can be derived from the shape function (Ycx)) of the profile of the bend, the punch load (Fp) and the punch displacement (Yp). The general equation relating the M-K curve to the punch load displacement characteristic for a given material is given by [4],

All rights reserved

44 S.BL Kwok, W.B. Lee / Journal of Materials Processing Technology 48 (1995) 43-49

C f~rnmand l~sued ~_ I P I Bent Specification L U} MateriM Property ~-"

Intell igent Bending

Machine Controller

Control __Signal L a s e - - V- I Displacement- .~ ] I Sensor

Yr, M

Punch

B e n t ~ Sheet- /- Metal

D i c ~

Fig. 1

dF@ (YPF~)- K(~'%)MOL 3K(Mt')Me2 jofL~dXl+y' (1)

A schematic diagram of the intelligent bending machine

where Y' is the first derivative of the shape function of the profile of the bend.

M(~) = F v x L ( l - L ) (2)

M r, = Fp x L (3)

The punch load is measured by a force transducer and the deflection of the sheet metal by a displacement sensor. The analog output data from the load transducer and the displacement sensor, which are transformed into digital output via AD/DA conversion, are collected at regular time intervals during the early stages of the bending process. When the punch advances to a preset point, the image of the bend is captured into the computer.

To solve eqn.(1), the shape function needs to be

known. Yix) is given by the profile of the bend

Y(x), and is determined by on-line image analyzing of the bend profile.

3. THE MACHINE VISION SYSTEM

A software called Machine Vision System for Sheet Metal Bending (MVSSMB) has been developed for capturing, analyzing the shape of the bend, and determining the flank angle and equation for the curved portion of the bend, These data are used in a plastomechanical model developed for on- line modelling of sheet metal bending [4]. The major features of the vision system are described below.

(i) lmage pre-processing The image is captured by a conventional CCD

camera. The image pre-processing subroutine allows

S,K. Kwok, W.B. Lee / Journal of Materials Processing Technology 48 (1995) 43--49 45

for a choice of a number of combinations of image conditioning techniques, such as median spatial filtering, thresholding and/or edge detection [5]. The choice of combinations depends on the conditions and quality of the image captured. The captured image can be previewed to help the user to choose the image processing techniques.

(iO Locating the datum points To relate the captured image to the real physical

geometry, some datum points are employed. In the tool set, six circular marks are marked on the punch and die. The positions of the marks are shown in Fig.2. After identifying the position of the masks in the captured image, reference can be made on them to establish the real coordinate system relative to the punch and die, by comparing the image coordinates with the physical coordinates of the marks.

Ma~3

Markl

+ _ ~ Mark2

, • Punch

/ S Die

• ~ - Mark4

~ - Mark6

Fig.2 Datum marks on tool set

(iii) Point image determination To get the flank angle and profile of the curved

part of the bend, the midpoints of the thickness of the bend need to be determined. In other words, the picture is reduced to a point image which contains only the points that lie in the centre line of the bend.

Three processing regions are determined from the datum points and the tool geometry (Fig.3). Region 1 and 2 are defined based on the x and y coordinates of the left top die point and the right top die point respectively. Region 3 is defined based on the coordinates of the tip point (T1) of the bend and the right top die point. By emitting a number of straight lines within each region at a constant angle of 45 ° in region 1, and 135 ° in region 2 and 3 with different offsets, the points along the centre line of the bend at each region can be determined.

I ~ X " i Li . . . . Rted to ' ~ \ \ Region 2 Region I - " cut the bendimage / \ \ \,

Fig.3 Image processing regions in point image determination

(iv) Coordinate system transformation Due to errors in the tool setup and the limitation

in the hardware, there will be some distortions in the shape and size of the captured image. There are two sources of major distortions : angular distortion (square object becomes parallelogram), and linear distortion (square object becomes rectangular). The point data provided by the midpoint determination refers to the screen coordinate system which is useless for our purpose, as these data will change from image to image. These point data are required to go through a three stages transformation (Fig.4) and to be converted to a real coordinate system (i.e., in scale with the real object) which is used later for image analyzing.

The angular distortion is corrected in stage 1. The centre coordinates of three datum points Mark 3 (Do), Mark 4 (D~) and Mark 5 (D2), as shown in Fig.2 and Fig.3, are taken as references. In stage 1 as shown in Fig.4, mt and m2 are the slopes of lines DoD ~ and DoD2 respectively, u(X3,Y3) is a point to be convened to a new coordinate system, and A(XA,YA), and B(XB,YB) are the intercepts of the lines which pass through the point u and parallel to the line DoD~ and DoD 2 respectively.

With D O as the origin, DoD~ as the x-axis and D0D 2 as the y-axis in the new coordinate system, it is easy to verify that the projected coordinate u(Dx,Dv) of u(X3,Y3) is given by

46 S.K. Kwok, W.B. Lee / Journal of Materials Processing Technology 48 (1995) 43--49

X screen

Screen coordinate

system

i

.i Y screen

X screen

iD2

Y screen

Do(X ~Y o) m t B (XB,Ys} DI(XI,Y I )

=: r) ? - - +~) m e l

¢' A (N,Y^)

" u ( X , Y Q

t

D2(X 2,Y 2 )

i Stage 1 JJ

X screen _ _ )

Y new

" - - : : " ~ X new i T1 !

Y screen

Y real

/

I

! Stage 3 I

Fig.4 A schematic diagram showing the three stages in the transformation of the co-ordinate system

i )2 Y 2 -Y 3 + mlX3-m 2X2 . + Xnew = Dx-DTx X3 mt - m 2

D x = I ( Y 3 ( X 2 - X3)m2ml +m2Y3-mlY2 ) 2 m 2 -- ml . (4) Y ..... = DTy - DY

I )2 YI -Ya +m2X3 - m i X i - +

X3 m 2 -m~ = ~ (5)

DY I I Y3 (Xl - X3)mlm2 + mlY3 - m2Yl) - m r - m 2

In stage 2, the coordinate system in stage 1 is converted to the one that is defined based on point TI as shown in stage 2 of Fig.4. From the above operations, (Dvx,Dvv) thus becomes the projected x,y-coordinates of the point T1. The transformation is simply a linear transformation of the coordinate system using the following equations :

(6)

(7)

The linear distortion is corrected in stage 3 of Fig.4. The coordinate system in stage 2 is converted to the real coordinate system. Let fc×, fcv be the scaling factor for the x and y axis respectively.

measured real distance between

centre points D O and D I (8)

fcx - ~ ( X , - X o ) ~ +(V, -Vo)=

measured real distance between

centre points D o and D 2 (9)

fcy - /(x2-x0) 2 + (v2-v07

S.I~ Kwok, I'KB. Lee / Journal of Materials Processing Technology 48 (1995) 43-49 47

The conversion is carried out by multiplying fc x and fc v with X .~ and Y.,,, respectively, and the real coordinates of point u(X~¢,t,Y~J are given by

Xr~al = X,e w x fc x (10)

Yreal = Y . ~ XfCv (11)

(v) Image data analysis After obtaining the point image, the final stage is

to analyze the points in the point image. For flank angle determination, the point data in the two straight portions of the bend (i.e. region 1 and 2 in Fig.3) are used. The determination is based on a linear regression method.

The method of least squares is used for determining the best fit of the data to either one of the following equations,

I I x 3 3 x ~ ' ] y = Yp --~(~-) + ~ ( ~ ) - J or (12)

y = Yp f l - cos (2 -~x) ] or (13)

y = Y p [ ¼ ( 1 - - ~ ) 4 + ~ l (14)

with the following boundary conditions, x = 0 , y = 0 , y ' = 0 x = L , y=Yp, y " = 0

Fig.5 shows a captured image of the bend. A comparison of the measured data with the machine vision data of the bend is shown in Fig.6. The measured data is obtained by tracing the magnified profile of the bend from a large-screen projector. The processing time for one complete image analysis at tow resolution image (320x200 pixels) takes only a few seconds on a 486DX microcomputer. The high accuracy and short processing time that can be achieved will make the vision system very suitable for on-line applications.

4. ON-LINE DETERMINATION OF SPRINGBACK

The shape function (Y(~)) determined from the

\~ /

e Fig.5 The computer captured image of the bend

profile

: . . . . . . . . . i . . . . . . . . . . . . . . . . . . . .

~ J3"° ............. - ............ ~ ........... := "" := i i

>- a.o ............ 4- ............ ~ ........ ~--~-'- .......... ~ ............

ii o o X/rnm

~--. Experiment ..... Computer Vision

Fig.6 The profile of the bend comparison of

measured data with machine vision data

vision system is used to adjust the measured punch force-displacement data. Referring to Fig.7, the point of application of the forces at the die-sheet interface will change as the sheet rolls and slides on the die. To reduce such kind of systematic differences between the model and practice, calculations are made relating the measured values of punch force (FpM) and punch displacement (YPM) to the idealized punch force (Fp) and punch displacement (Yp). Since the bend profile varies according to the shape function (Y(,)), the idealized values (Fp and Yp) are found by iterative procedures. Referring to Fig.7, at the neutral axis,

L* = L - rtsin0

Ye = Ypp + rtcos0

where 0 < r < l and Ypp=Yp•+(l-r) t

(15)

( t6)

48 &K Kwok, W.B. Lee / Journal of Materials Processing Technology 48 (1995) 43-49

%

Fig.7 Relationship of the various punch displacement parameters, YPM, Yp, YPP and L" at the die-sheet interface

Since the flank angle 0 is not known, eqn.(16) can be written as

r~ + YPI = YPI-, 1/1 +(y,e,., )2 (17)

where Y' is the first derivative of the shape function At the start, Yp,., is set equal to YeP and the iteration

r t < 1 0 -3 ' is repeated until )2

then Yp = Yp, (18)

and L' = L - _ _ ~ Y'p (19)

! N< T" F

I- C -I

[ Fig.8 Real force diagram in bending

t

Referring to Fig.8, the horizontal force (FH) can be calculated from the normal force (I'4) and tangential force (F) at the die-sheet interface, i.e.,

~Fpu = tan0 ~ FpH = FpM Y'p (20) Fr, M

N = ~/FpM ~" + FpH 2 = F[,M~ + y, 2 (21)

F = gN = p.FpM~/l + y,p2 (22)

where g is the coefficient of friction. Hence,

F v = FpM + Fsin0 = FpM (1 + p.Y't, ) (23)

F H - FpH - F c o s 0 = FpM(Y' e -I-t) (24) From eqn.(23), eqn.(24) and Fig.8,

M = FvL* + FnY P (25)

- FpM[(I+}.ty,p)L* + (Y'p-g)Yp]

Referring to the force diagram in Fig.9,

M

J Fp

"1

Fig.9 Idealized force diagram in bending

M = F e xL (26)

From eqn.(25) and eqn.(26),

Fp - M _ FpM L L [( l+gY'p)L* +(Y'p-bt)Yp] (27)

Eqn.(18) and eqn.(27) can be used to convert the measured values (FpM and YPM) to the idealized

S.K Kwok, W.B. Lee I Journal of Materials Processing Technology 48 (1995) 43-49 49

force-displacement data (Fp and Yp). The relationship between Fp and Yo can be approximated by a parabolic equation of the form

Yp = aFp +h(Fp) c (28)

where a, h and c are material constants, and a is a parameter corresponding to linear elasticity; h and c are the coefficient and exponent of the power-law strain-hardening equation respectively.

To solve the coefficients in the equation, the idealized data are fitted to eqn.(28) by the Area- Coarea method [6]. Substitute eqn.(28) into eqn.(1),

Mp 3a + h(c+2) L

K(MI') = L 2 / 5 - ~ 3 fL2LJ0 l + y ' 2 ) d x ~ (29)

Once the profile of bend is fixed by an equation and all coefficients in eqn.(28) are found, eqn.(1) can be used to find out the M-K relationship. For example, assuming that

Y ~ ) = Y p I 1 - c o s ( 2 - ~ x ) ]

~YP s i n ( ~ x) (30) Y'(x) = 2 L 2 L

and substitute this to eqn.(29), the calculated M-K relationship is given by eqn.(3 l),

K(Mp) =

/" M "xC-I" MPL 3a+h(c+2)tE~- j

i

3 I L2 2 2

After several iterative steps, the geometrical solution of the bend profile which fulfills the moment-curvature relationship is found and (31) the correct punch penetration position can thus be calculated [7].

5. CONCLUSION

The incorporation of a machine vision system in sheet metal bending process enables a different approach to solve the classical springback problem based on a new plastomechanical model in which the moment-curvature relationships is derived from the profile of the bend shape. The property of each sheet is determined on-line to compute the required punch travel to achieve a given bend angle. No arbitrary parameters need to be input into the model and any variation in the mechanical properties of the sheet is taken into account. The ability of the bending machine to give commands to give the required punch travel according to individual material property need is the first step towards what can be called intelligent forming.

ACKNOWLEDGMENTS

The authors wish to thank the Research Committee of the Hong Kong Polytechnic for the award of a scholarship to one of the author (SKK) and Mr. C.F.Cheung for his technical support during the course of the work.

REFERENCES

1, S.Pickering, Journal of Materials Processing Technology, 36(1993)447.

2. K.A.Stelson, ASME Journal of Engineering for Industry, 108(1986)127.

3. K.A.Stelson, ASME Journal of Engineering for Industry, 105(1983)45.

4. W.B.Lee, S.K.Kwok and H.L.Li, a paper submitted to Int.J.Mech.Sci., 1993.

5. Robert J.Schalkoff, Digital Image Processing And Computer Vision, John Wiley & Sons, Inc., Singapore, 1989.

6. T.Y.Peterson and K.A.Stelson, ASME Journal of Engineering for Industry, ! 11 (1989)295.

7. S.K.Kwok and W.B.Lee, Proc. of International Conference on Manufacturing Automation, Hong Kong University, Hong Kong, 1992, p.601.