Fast Reflectional Symmetry Detection Using Orientation Histograms

Ž .Real-Time Imaging 5, 63]74 1999Article No. rtim.1998.0135, available online at http:rrwww.idealibrary.com on

Fast Reflectional Symmetry Detection UsingOrientation Histograms

simple and fast reflectional symmetry detection algorithm has been developed in thisApaper. The algorithm employs only the original gray scale image and the gradientinformation of the image, and it is able to detect multiple reflectional symmetry axes of

an object in the image. The directions of the symmetry axes are obtained from the gradientorientation histogram of the input gray scale image by using the Fourier method. Both syntheticand real images have been tested using the proposed algorithm.

q 1999 Academic Press

C. Sun1 and D. Si2

1 CSIRO Mathematical and Information Sciences, Locked Bag 17, North Ryde, NSW 2113, AustraliaE-mail: [email protected]

2 Department of Electronics Engineering, Beijing Uni ersity of Aeronautics and AstronauticsBeijing 100083, China

Introduction w xtion. Burton et al. 2 considered a simple indexingscheme to implement the exponential pyramid data

It has been proposed that symmetry plays a major rolein object recognition by providing a canonical axis for

w xthe representation of shapes 1 . Many objects aroundus are strongly constrained. For instance, many cultural

Žartifacts and natural objects are reflectionally bilater-.ally or mirror symmetric. A reflectional symmetry has

a reflection line, for which the left half space is amirror image of the right half. One of the goals of animage understanding system is to identify and locate aspecified object in the scene. In such cases, the systemmust have some knowledge of the shape of the desiredobject. Symmetries are good candidates for describingshapes. It is a powerful concept that facilitates objectdetection and recognition in many situations. Theserepresentations can be used in robotics for recognition,inspection, grasping, and reasoning.

Most of the work carried out on symmetry detectionhas been based on edge, contour or point set informa-

1077-2014r99r010063 q 12 $30.00

w xstructure for particular symmetries. Wolter et al. 3described exact algorithms for detecting all rotationaland involutional symmetries in point sets, polygons and

w x w xpolyhedra. Atallah 4 and Davis 5 used evaluationtechniques for symmetry detection on images com-

w xposed of line segments, circles and points. Highnam 6presented an asymptotically optimal algorithm to locateall the axes of mirror symmetry and optimal algorithmsfor finding rotational symmetries of a planar point set.

w xMarola 7 presented an algorithm for finding the num-ber and position of the symmetry axes of a symmetricor almost symmetric planar image. This methodrequired the evaluation of some rational functions. Healso presented a recognition procedure based on themeasurements of the degree of symmetry of planar

w xintensity images by superposition or by convolution 8 .w xZabrodsky et al. 9 defined a Continuous Symmetry

Measure to quantify the symmetry of objects. They alsow xpresented a multi-resolution scheme 10 that hierar-

chically detects symmetric and almost symmetric pat-

q 1999 Academic Press

C. SUN AND D. SI64

w xterns. Parry-Barwick and Bowyer 11 developedmethods that can detect both hierarchical and partial

Ž . w xTI x, y , the surface gradient vector I , I is definedx yby:

Figure 1. An illustrative shape of the gradient orientationhistogram for a reflectionally symmetric object.

Ž .symmetry of two-dimensional 2D set-theoretic modelswith components constructed with a few straight edgesor polynomials. This method had the disadvantage of

w xbeing computationally intensive. Zielke et al. 12, 13only looked at vertical or near vertical symmetry axesin an image for car-following applications. Masuda et

w xal. 14 described a method of extracting rotational andreflectional symmetry by performing a correlation withthe rotated and reflected images. But the method had ahigh computational cost and memory requirements.

w xBowns and Morgan 15 presented a method of extract-ing facial features using natural orientation informa-tion. But this method is not suitable for general shapedobjects when the peak orientation is not related to theorientation of the symmetry axis. O’Mara and Owensw x16 used the directions of the principal axes of anobject as the initial values of the symmetry axis. Sunw x17 used a direct correlation method on the orienta-tion histogram to obtain the most likely symmetry axis.

In this article we investigate the use of gradientinformation for symmetry detection in a gray scaleimage by using the Fourier technique. Our purpose forusing the Fourier method is to increase the computa-tion speed further compared with the method used inw x17 . We also try to find multiple symmetry axes in anobject rather than just obtain one of the most likely

w xsymmetry axes, as in 17 . The following sectionsdescribe the gradient orientation histogram, give thealgorithm for finding the orientations and position ofthe symmetry axes, and show the results of the algo-rithm on both synthetic and real images. Comparisonof several existing methods and the method developedin this paper is also described. The final section pro-vides some conclusions.

Gradient Orientation Histogram

The image formation includes imaging geometry, sur-face photometry, and surface contours. The imageitself is often ambiguous in its representation of sceneinformation, and the interpretation of such an imagerequires a combination of analytical tools and knowl-

Ž .edge or assumptions about the scene objects and thew ximage production 18 . The observed brightness of

objects corresponds to the image irradiance. For ouranalysis, we will only consider Lambertian surfaces.Neither specular reflection nor shadowing effects willbe treated here. For an object surface described by

Ž . Ž . I x , y I x , yŽ .I s , I s . 1x y x y

The orientation of this gradient vector is:

Ž . Ž .f s arctan I rI . 2y x

w .The domain of f is 0, 2p . It is expected that thegradient orientation information of an image can beused for symmetry detection. Details of the method willbe described in later sections. This method can bemisled by the presence of a periodic texture within the

Žobject if the texture pattern does not have the same.symmetry axis with the object , as these repetitive pat-

terns may give high peaks in the orientation histogramcompared with those from the object’s global shape.However, as long as the predominant gradient orienta-tions are symmetric, the algorithm will be able to findthe symmetry axes.

Figure 1 gives an illustrative shape of the gradientorientation histogram for a reflectionally symmetricobject. The shape should be mirror symmetric, or nearlymirror symmetric, because of the digitization errors.

Ž . Ž .Figure 2 a shows a symmetric object, and Figure 2 bis a circular gradient orientation histogram with theorientation angle starting from the center-to-right

Ž .direction 08 and increasing clockwise, and the radiallength showing the number of pixels at this orientationwithin this image. The line passing through the center

Ž .of Figure 2 b indicates the symmetry orientation. It iseasier for us to perceive the 2p periodicity ofthe gradient orientation histogram in this circularrepresentation.

Orientation and Position of Symmetry Axes

Our algorithm is based on the gradient orientationdistribution of the input gray scale intensity image. It

FAST REFLECTIONAL SYMMETRY DETECTION 65

Ž . Ž .Figure 2. A symmetric object and its circular gradient orientation histogram. a An object with reflectional symmetry; b theŽcircular gradient orientation histogram of the object with the symmetry orientation overlaid the line passing through the

.center .

will also work for binary images in which the usefulgradient information appears at the boundaries of

only one symmetry axis is chosen, by simply selectingŽ .the maximal value of Eqn 4 . In practice, the integral

objects. From the previous section, the histogram ofthis gradient orientation image can therefore be

w Ž . Ž .xobtained using Eqns 1 and 2 . The range of thegradient orientation is from 0 to 360 degrees. Users areable to specify the number of bins for the histogram. Inorder to have better angular resolution, the bin numbercan be chosen to be some larger numbers than 360. Itcan be observed that for a symmetric object in theimage, its orientation histogram is also symmetric. It isalso clear that this histogram function is periodic with

Ž .period 360 or 2p . That is,

Ž . Ž . Ž .h u s h u " 2np n s 0, 1, 2, ??? 3

Ž .where h u is the gradient orientation histogram of thew . w xinput image, and u is in 0, 2p . Sun 17 used a

window at position x of the histogram with length pfrom the left and p from the right to calculate thefollowing function:

pŽ . Ž . Ž . Ž .c x s h x q u h x y u du 4H

us0

The orientation of the symmetry axis is the value xŽ .which gives the maximal value for c x . In his paper

in the above equation is approximated by a sum of theŽ .discretized orientation histogram h i , i s 0, 1, . . . ,

Ž .N y 1. The signal processing operation in Eqn 4 is acorrelation between the segments of the gradient ori-entation histogram. Since portions of the histogram aremirror images, a reflection prior to a shift and multiplyis carried out, as is done when performing a convolu-tion. In this paper we use the Fourier method to obtain

Ž .the symmetry information. The Fourier transform H uŽ . Ž .of h i as defined in Eqn 5 will be used to obtain the

convolutions of two signals. In our case, the two signalsŽ .are the same, i.e. h i .

Ny1Ž . Ž . � 4H u s h i exp yj2p iurN , u s 0, 1, . . . , N y 1Ý

is0

Ž .5

For a reflectionally symmetric object, its gradientŽ .orientation histogram h i will be reflectionally sym-

metric or nearly so at one or more orientation angles.Ž .The convolution of histogram h i with itself will pro-

duce peaks at those symmetry angles. This convolutioncan be easily obtained by the Fourier transform. Based

C. SUN AND D. SI66

on the following con¨olution theorem: 1. Perform a gradient operation on the image;2. Obtain the gradient orientation histogram;

Ž . Ž . Ž . Ž . Ž .f x ) g x m F u = G u 6

the convolution in the space domain can be obtainedby taking the inverse Fourier transform of the productŽ . Ž .F u G u . By doing so, the computational cost can be

reduced to the order of N log N rather than the22 Ž . Ž . Ž .order of N . In our case F u s G u s H u . There-

Ž . Ž . Ž . Ž .fore, we have h i ) h i m H u = H u , and we can2Ž .use the inverse Fourier transform of H u for obtain-

Ž .ing the convolution shown in Eqn 4 , i.e.:

Ž . Ž . 2 Ž . Ž .h i ) h i m H u 7

w xWe also extend the work of Sun 17 to the caseswhere there exist multiple symmetry axes about theobject. If there are multiple symmetry axes, the convo-lution function should have multiple peaks. So by set-ting an appropriate threshold, multiple peak positionswhich correspond to the orientations of multiple sym-metry axes can be obtained.

The accuracy of the orientations of the symmetryaxes obtained by searching the peak positions in theconvolution function can be improved by fitting aparabola function around the local region of each peak.The optimal positions of the peaks can be obtainedanalytically from the fitted parabola function.

After the directions of the symmetry axes have beenobtained, it is also necessary to determine the position

Žof these lines. The center of gravity of the image or.the object of interest can be used for determining the

position of the symmetry axes.

The operations for obtaining the symmetry axisdirections and position are the following:

Figure 3. Flow chart of the algorithm for obtaining symmetry ax

3. Search for the convolution peaks of the gradientorientation histogram by using the Fourier trans-form to obtain the orientations of symmetry axes;

4. Obtain sub-angular positional accuracy by fitting aparabola function, and finding the peak positionanalytically;

5. Perform symmetry check or evaluation about theobtained symmetry axes;

6. Obtain the position of this symmetry axis by usingthe center of mass of the object; and

7. Draw the symmetry axes based on the orientationand position information obtained.

The flow chart of the algorithm is shown in Figure 3.

Experimental Results

The results for the described algorithm on both syn-thetic and real images are given in this section. Thegradient of an image is obtained by using a 5 = 5 Sobel

w xfiltering operation on the input gray scale image 19 .The filtering splits the kernels into 2 = 1 sub-kernelsand iteratively calculates the responses. The results aretwo gradients for the x- and y-directions, respectively.Then the gradient orientation at this point of the image

Ž .can be obtained by using Eqn 2 . When both I and Ix yare zero at a point in the image, the gradient at thispoint is treated as not defined. By having a set of bins

Ž .in the range of 0, 360 , the orientation histogram ofthe image can therefore be obtained. No initialsmoothing was applied to the original image beforethe gradient operation, as the process of obtaining theorientation histogram has the effect of cancelling thenoise contribution.

es.


Because of the digitization effects, the boundary ofan object consists of mostly zigzagged short line seg-

Figure 4. Sub-angular positional accuracy by finding the peakof a fitted quadratic parabola equation.

sufficient condition for symmetry detection. That is, forcertain non-symmetric objects, its orientation his-

ments, and very often these line segments are eitherhorizontal or vertical, with diagonal segments connect-ing the horizontal or vertical short lines. Therefore, inmost of the gradient orientation histograms, peaks of-

Žten appear at multiples of 458 08, 458, 908, 1358, 1808,.2258, 2708 and 3158 . This effect can be reduced by

increasing the kernal size in the gradient operation tosome extent. However, it is hard to eliminate the effectcompletely just by increasing the kernel size. The his-togram is circularly smoothed with a median filter ofsize 5; that is, the smoothing window is wrapped aroundat the ends since the angular data are circularly contin-uous. The convolution peaks are found from thesmoothed data. Only those peaks that are within 90%of the maximal value are chosen. The positions of thepeaks are further refined by fitting a quadratic function

Ž .around the local region say five points of each peak.The refined positions are obtained analytically by find-ing the maximal value of the quadratic function. There-fore, higher angular resolution can be obtained. Thegeneral form of the quadratic parabola equation is:Ž . 2f x s a q b ? x q c ? x . The maximum can be found

where the slope is zero in the quadratic equation. Theposition of the peak can be found at x s ybr2c, asillustrated in Figure 4.

The histogram property is a necessary but not a

Ž . Figure 5. a A counter-example when the proposed algorithmŽ .where g s a q b s c q d .

togram might still be reflectionally symmetric. OneŽ .example is shown in Figure 5. Figure 5 a shows a

Ž .non-symmetric object, while Figure 5 b displays itsidealized gradient orientation histogram. Although theorientation histogram is reflectionally symmetric, theobject does not have the reflectional symmetry prop-erty. However, as long as we know that the object inthe image is reflectionally symmetric, we can apply thealgorithm to detect reflectional symmetries. If the con-dition is not sufficient, i.e. when the orientation his-togram shows a reflectional symmetry but the actualobject is not reflectionally symmetric, a further stepmight be necessary to check whether the object is

Ž . Ž .might not work; b The ideal orientation histogram for a

C. SUN AND D. SI68

Ž . Ž .Figure 6. a Two symmetry axes were initially detected; b the incorrect symmetry axis was eliminated.

Figure 7. Symmetry detection result. Two symmetry axes have been detected by searching the high peaks in the convolutionŽ . Ž . Ž .function. a Original image and the symmetry axes obtained; b gradient orientation image normalized from 0]360 to 0]255 ;

Ž . Ž . Ž .c median smoothed histogram of the gradient orientation image b ; and d the convolution function obtained using Fouriertransform.


Ž . Ž .Figure 8. Symmetry detection results for several synthetic images. a A range image of an adapter; b a range image of a halfŽ . Ž . Ž . Ž .sphere; c a square with four symmetry axes; d an aeroplane; e a telephone; and f a rotated human face.

C. SUN AND D. SI70

Ž . Ž .Figure 9. Symmetry detection. a an original image with the detected symmetry line overlaid; b the gradient orientationŽ .histogram; and c the convolution function obtained from Fourier transform.

Ž . Ž .Figure 10. Symmetry detection process for a real image. a Original image and the symmetry axis obtained; b the convolutionfunction obtained from the inverse Fourier transform. The symmetries are obtained by searching the peaks of this function.


actually reflectionally symmetric or not. These parame-ters or symmetry hypothesis can be verified by obtain-

function. There is only one peak within 90% of themaximum value. Figure 10 illustrates the steps for

Ž w xing a symmetry measure see Zabrodsky et al. 9 orw x.O’Mara and Owens 16 . If this symmetry measure

gives a low value, we say that the object is not symmet-ric. If an object is convex, the histogram property willnot only be a necessary, but also a sufficient conditionfor symmetry detection. Figure 6 gives an example of anon-convex object. In this image, two symmetry axeswere initially detected. One of the incorrect axes waseliminated after considering the symmetry measure.

Ž .Figure 7 a shows an original image and the symme-Ž .try axes obtained; Figure 7 b is the gradient orienta-

tion image normalized from 0]360 to 0]255 for displayŽ .purposes; Figure 7 c gives the histogram of the gradi-

Ž . Ž .ent orientation image b ; and Figure 7 d shows theconvolution function of the histogram obtained byusing the Fourier method. The two high peaks corre-spond to the two symmetry axes. Figure 8 shows theresults of the symmetry detection algorithm on severalsynthetic images. Figure 9 gives another example of

Ž .symmetry detection. Figure 9 b is the gradient orienta-Ž .tion histogram, and Figure 9 c is the convolution

Figure 11. Symmetry detection for two noise added synthetic imŽ .25% noise added to the image of Figure 8 f .

obtaining the symmetry axes for a real image, an imageŽ .of Jupiter. As shown in Figure 10 b , there are two

peaks that are within 90% of the maximal value. Thesymmetry evaluation step indicates that the symmetry

Ž w x.measures using the measure described in 16 forthese two peak positions are very similar. Although ourhuman perception may prefer to choose the axis run-ning from the top of the image to the bottom of the

w Ž .ximage see Figure 10 a , the algorithm actually gives aslightly stronger symmetry measure for the other axis.After careful examination about the image we find thatthe change of intensity in the image around the leftboundary of Jupiter is smaller than the change aroundthe right boundary of Jupiter. It is this difference in thechange of intensity that makes the algorithm pick uptwo symmetry axes. Figure 11 shows the results of thealgorithm on two noise added synthetic images. Thesymmetry axes are very close to the ones obtained inthe noise free images. Figure 12 gives the results of thealgorithm on some real images.

The typical CPU running time for a 256 = 256 image

Ž . Ž . Ž .ages. a 25% noise added to the image of Figure 8 a ; and b

C. SUN AND D. SI72

Ž . Ž . Ž . Ž .Figure 12. Symmetry detection results for several real images. a Welcome gate; b slice of CT x-ray image; c ceramic; dŽ . Ž .human face; e square pattern; and f fluctuating crosses.


is less than 0.1 s on a Sun Sparc10. The time spent onthe calculation of symmetry orientation from the

Conclusions

We thank the reviewers for their valuable comments.The authors would also like to thank Professor N.Wade for his kind permission to use one illustrationfrom his book: Visual Allusions: Pictures of Perception,1990.

References

Ž .1. Marr, D. 1982 Vision. San Francisco, CA: Freeman.Ž .2. Burton, F. W., Kollias, J. G. & Alexandridis, N. A. 1984

An implementation of the exponential pyramid datastructure with application to determination of symmetriesin pictures. Computer Vision, Graphics, and ImageProcessing, 25: 218]225.

Ž .3. Wolter, J. D., Woo, T. C. & Volz, R. A. 1985 Optimalalgorithms for symmetry detection in two and threedimensions. The Visual Computer, 1: 37]48.

obtained histogram is about 2.6 ms, while the methodw xused in 17 takes about 72.7 ms. The code has not been

optimized for speed-up of processing.

Two other related methods for symmetry detectionwere implemented and the results were compared withour method. The reason for choosing these methodsfor comparison is that they mainly detect symmetryfrom intensity images rather than using contour pointsor isolated point sets. Table 1 shows some of the results

w xobtained. The methods by Zielke et al. 12, 13 werenot compared here, as they only looked at vertical or

w xnear vertical symmetry axes. Masuda et al. 14 useddirectional pattern matching of edge feature to detectrotational and reflectional symmetry. The main disad-vantages of this method were its computational costand memory requirements.

It is difficult to compare different methods in allaspects, as each method has its own emphases. Theimplementation details may be different for differentpeople as to data IrO, memory handling, etc. Thefollowing comparison is made on the basis that thesymmetry axes pass through the center of mass ofthe objects, and the algorithms work on the gray levelimages. All the methods listed in Table 1 are theoreti-cally simple and fast for gray level images comparedwith other methods mentioned in the Introductionsection.

w xThe method by O’Mara and Owens 16 is fast andsimple to implement, but the accuracy of the algorithmdepends on the results of the principle axes, which aresensitive to imperfect symmetries. The method onlyselects the dominant symmetry. The method by Sunw x17 is also fast and simple to implement. It only detectsthe most likely symmetry axis. The method proposed inthis paper is fast to compute and simple to implement.It is also able to detect multiple symmetry axes.

Table 1. Comparison of three methods for reflectional
symmetry detection
Ž .CPU time s Multiple axes

Ž .O’Mara & Owens 1996 ;0.05 NoŽ .Sun 1995 ;0.15 No

Our algorithm ;0.10 Yes

Based on our observation about the relationshipbetween object symmetry and the gradient orientationhistogram, a simple and fast reflectional symmetrydetection algorithm has been developed which employsonly the original gray scale image and its gradientorientation information. Fast Fourier transform hasbeen used to obtain the convolution function from thegradient orientation histogram of the input image. Thepeaks in the obtained convolution function correspondto the orientations of symmetry axes. The positions ofthese peaks can be refined by fitting a parabola func-tion in the neighbourhoods of these peaks. The refinedpositions are obtained analytically. This method candetect multiple symmetry axes of an object. The CPUtime for a 256 = 256 image takes less than 0.1 s on aSun Sparc10. The information about the symmetry

Žaxis can be used for many applications such as robot.operations, further image segmentation, etc .

Work has only been performed on a single object inan image. The symmetry axes for separated multipleobjects can be obtained after the image segmentationof these objects. For every single object, the algorithmdescribed in this paper is applied to obtain thesymmetry axes.

Acknowledgements

Ž .4. Atallah, M. J. 1985 On symmetry detection. IEEETransactions on Computers, C-34: 663]666.

Ž .5. Davis, L. S. 1977 Understanding shape: II. Symmetry.IEEE Transactions on Systems, Man and Cybernetics, pp.204]212.

Ž .6. Highnam, P. T. 1986 Optimal algorithms for finding the

C. SUN AND D. SI74

symmetries of a planar point set. Information ProcessingLetters, 22: 219]222.

Ž .7. Marola, G. 1989 On the detection of the axes of symme-try of symmetric and almost symmetric planar images.IEEE Transactions on Pattern Analysis and MachineIntelligence, 11: 104]108.

Ž .8. Marola, G. 1989 Using symmetry for detecting andlocating objects in a picture. Computer Vision, Graphics,and Image Processing, 46: 179]195.

Ž .9. Zabrodsky, H., Peleg, S. & Avnir, D. 1992 A measure ofsymmetry based on shape similarity. In: Proceedings ofComputer Vision and Pattern Recognition, pp. 703]706.

Ž .10. Zabrodsky, H., Peleg, S. & Avnir, D. 1992 Hierarchicalsymmetry. In: Proceedings of International Conference onPattern Recognition, III: 9]11.

Ž .11. Parry-Barwick, S. & Bowyer, A. 1993 Symmetry analysisand geometric modelling. In: Fung, K. K. & Ginige, A.Ž .eds. Digital Image Computing: Techniques and Applica-tions, I: 39]46. Australian Pattern Recognition Society.

Ž .12. Zielke, T. Brauckmann, M. & von Seelen, W. 1992CARTRACK: computer vision-based car-following. In:Workshop on Applications of Computer Vision, pp. 156]163.

Ž .13. Zielke, T., Brauckmann, M. & von Seelen, W. 1993

Intensity and edge-based symmetry detection with anapplication to car-following. Computer Vision, Graphics,and Image Processing: Image Understanding, 58: 177]190.

Ž .14. Masuda, T., Yamamoto, K. & Yamada, H. 1993 Detec-tion of partial symmetry using correlation with rotated-reflected images. Pattern Recognition, 26: 1245]1253.

Ž .15. Bowns, L. & Morgan, M. J. 1993 Facial features andaxis of symmetry extracted using natural orientationinformation. Biological Cybernetics, 70: 137]144.

Ž .16. O’Mara, D. & Owens, R. 1996 Measuring symmetry indigital images. In: Proceedings of TENCON ’96]IEEERegion Ten Conference: Digital Signal Processing Applica-tions, volume I, pages 151]156.

Ž .17. Sun, C. 1995 Symmetry detection using gradientinformation. Pattern Recognition Letters, 16: 987]996.

Ž .18. Bracho, R. & Sanderson, A. C. 1985 Segmentationof images based on intensity gradient information. In:Proceedings of Computer Vision and Pattern Recognition,San Francisco, CA, pp. 341]347.

Ž .19. Danielsson, P. E. & Seger, O. 1990 Generalized andŽ .separable Sobel operators. In: Freeman, H. ed. Machine

Vision for Three-Dimensional Scenes, pp. 347]379.Academic Press.

Documents

Fast Reflectional Symmetry Detection Using Orientation Histograms