Mic John - UWA Research Repository - University of Western ...research-repository.uwa.edu.au/files/3248336/Robins_Michael_John... · ters and the ridge tracing algorithm applied to

LOCAL ENERGY FEATURE TRACING IN DIGITAL IMAGES

AND VOLUMES

This thesis is

presented to the

Department of Computer Science

for the degree of

Doctor of Philosophy

of the

University of Western Australia

By

Michael John Robins

June ��

c� Copyright ��

by

Michael John Robins

Contents

Abstract ix

Preface xi

Acknowledgments xiii

Glossary xiv

� Introduction �

�� General introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Thesis overview � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Finding image features � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Finding features using phase information � � � � � � � � � � � � � � �

�� The local energy function � � � � � � � � � � � � � � � � � � � � � � � �

�� Calculation of phase congruency and local energy � � � � � � � � � � �

�� Features at multiple scales � � � � � � � � � � � � � � � � � � � � � � � �

�� Features in multiple dimensions � � � � � � � � � � � � � � � � � � � � �

�� A recursive implementation of local energy � � � � � � � � � � � � �

�� Finding peaks and ridges � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Gradients and nonmaximal suppression � � � � � � � � � � � � � � � ��

�� A new ridge tracing algorithm � � � � � � � � � � � � � � � � � � � � ��

�� Finding surfaces and �laments � � � � � � � � � � � � � � � � � � � � ��

�� Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

iii

� Local Energy Feature Detection ��

�� A brief history of local energy � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Image feature detection � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Overview of image feature detection � � � � � � � � � � � � � � � � � ��

�� The phase congruency model � � � � � � � � � � � � � � � � � � � � � �

�� The local energy model � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Calculation of local energy by convolution � � � � � � � � � � � � � � ��

�� Extension of the local energy model � � � � � � � � � � � � � � � � � ��

�� Filter pairs for local energy analysis � � � � � � � � � � � � � � � � � � � � � ��

�� The Gaussian function � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Wavelets for local energy � � � � � � � � � � � � � � � � � � � � � � � ��

�� Review of �lter pairs � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Digital �ltering techniques � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Frequency domain convolution � � � � � � � � � � � � � � � � � � � � ��

�� Spatial domain convolution � � � � � � � � � � � � � � � � � � � � � � ��

�� Conclusion � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

� Recursive Filters for Oriented Local Energy ��

�� Introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Design of In�nite Impulse Response Filters � � � � � � � � � � � � � � � � � ��

�� The ztransform � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Recursive Structures � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Design by approximation � � � � � � � � � � � � � � � � � � � � � � � ��

�� Numerical Approximation � � � � � � � � � � � � � � � � � � � � � � � ��

�� Approximate IIR �lters and local energy � � � � � � � � � � � � � � � �

�� Freeman s Filters � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Separability � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Basis functions separable in x and y � � � � � � � � � � � � � � � � � ��

iv

�� Third derivative of Gaussian � � � � � � � � � � � � � � � � � � � � � ��

�� Steerable quadrature pairs � � � � � � � � � � � � � � � � � � � � � � � ��

�� L� normalisation � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� E�ect of scaling � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Steerable scalable recursive �lters � � � � � � � � � � � � � � � � � � � � � � � ��

�� Recursive G� implementations � � � � � � � � � � � � � � � � � � � � ��

�� Recursive Morlet Wavelets � � � � � � � � � � � � � � � � � � � � � � � ��

�� Recursive local energy � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Quadrature �lters � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Orientations � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Scales � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Recursive local energy in three dimensions � � � � � � � � � � � � � � � � � � ��

�� Steering equations for multiple dimensions � � � � � � � � � � � � � � ��

�� Steering G� and H� in three dimensions � � � � � � � � � � � � � � � ��

�� Recursive implementation of the G� and H� basis functions � � � � �

�� Local energy � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Conclusion � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

� Peaks and Ridges ��

�� Introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Peaks � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

�� Extrema in onedimensional data sets � � � � � � � � � � � � � � � � �

�� Extrema in multiple dimensions � � � � � � � � � � � � � � � � � � � � ��

�� Finding extrema in twodimensions � � � � � � � � � � � � � � � � � � ��

�� Ridges � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Various attempts to �nd ridges � � � � � � � � � � � � � � � � � � � � ��

v

�� Geometric methods N dimensional images � � � � � � � � � � � � � � ��

�� Watershed image segmentation � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Calculating digital watersheds � � � � � � � � � � � � � � � � � � � � � ��

�� Drawbacks of watershed segmentation � � � � � � � � � � � � � � � � ��

�� Skeletons and key point anchors � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Keypoints � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Skeletonisation � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Conclusion � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Summary � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

� Ridge Tracing ��

�� Introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Energy Maps �Revisited� � � � � � � � � � � � � � � � � � � � � � � � ��

�� Ridge Tracing �Qualitative� � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Algorithms � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Ordered Queues of Pixels � � � � � � � � � � � � � � � � � � � � � � � ��

�� Flooding Algorithms � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Seed and Marker Pixels � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Implementation � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Watershed Transform � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Skeleton Transform � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Ridge Tracing � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Filling � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Oversegmentation � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Watershed vs ridge tracing � � � � � � � � � � � � � � � � � � � � � � ��

�� Conclusion � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

vi

� Surface Tracing ��

�� Surface tracing algorithm � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Features in threedimensions � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Homotopy in threedimensions � � � � � � � � � � � � � � � � � � � � � � � � ��

�� An alternative to the Euler characteristic � � � � � � � � � � � � � � ��

�� Anchors in threedimensions � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Algorithm � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Normalisation and quantisation � � � � � � � � � � � � � � � � � � � � ��

�� Preprocess �lling � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Seeds and anchors � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Pruning and the borders � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Flooding the energy �eld � � � � � � � � � � � � � � � � � � � � � � � ��

�� Re�ooding the energy �eld � � � � � � � � � � � � � � � � � � � � � � ��

�� Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Discussion � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Conclusion � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

� Conclusion ��

�� Summary � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Contributions � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

�� Further work � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

Bibliography ��

A Review of local energy lter pairs ��

A�� and � point masks � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

A�� Derivatives of Gaussian � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

A�� Gabor functions � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

A�� Modi�ed Gabor functions � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

vii

A�� Log Gabor functions � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

A�� Morlet Wavelets � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

A�� Steerable derivative of Gaussian �lters � � � � � � � � � � � � � � � � � � � � ��

B Recursive lter implementation ��

B�� Fitting a polynomial to Hilbert transform � � � � � � � � � � � � � � � � � � ��

B�� Filter implementation � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

B�� Threedimensional local energy � � � � � � � � � � � � � � � � � � � � � � � � ��

B�� Read Filter Design � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

B�� Convolve XYZ Filter � � � � � � � � � � � � � � � � � � � � � � � � � � ��

B�� Kernel � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

B�� Steer � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

B�� Energy�d � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C Surface tracing implementation ��

C�� Minima and maxima � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Fill shallow lakes � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Watershed � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Skeleton � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Homotopy � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Homotopy �D � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� CheckConnection � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� CheckSeparation � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Propagate � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

C�� Ridge and surface tracing � � � � � � � � � � � � � � � � � � � � � � � � � � � ��

D Direction cosines for orientations ��

D�� Regular dodecahedron � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

D�� Regular icosahedron � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

viii

Abstract

Local energy feature tracing in digital images and volumes

Digital image feature detectors often comprise two stages of processing� an initial �ltering

phase and a secondary search stage� The initial �ltering is designed to accentuate speci�c

feature characteristics or suppress spurious components of the image signal� The second

stage of processing involves searching the results for various criteria that will identify the

locations of the image features�

The local energy feature detection scheme combines the squares of the signal convolved

with a pair of �lters that are in quadrature with each other� The resulting local energy

value is proportional to phase congruency which is a measure of the local alignment of the

phases of the signal s constituent Fourier components� Points of local maximum phase

alignment have been shown to correspond to visual features in the image�

The local energy calculation accentuates the location of many types of image features�

such as lines� edges and ramps and estimates of local energy can be calculated in multi

dimensional image data� by rotating the quadrature �lters to several orientations�

The second stage search criterion for local energy is to locate the points that lie along the

ridges in the energy map that connect the points of local maxima� In threedimensional

data the relatively higher energy values will form �lms between connecting �laments and

tendrils�

This thesis examines the use of recursive spatial domain �ltering to calculate local energy�

A quadrature pair of �lters which are based on the �rst derivative of the Gaussian function

and its Hilbert transform� are rotated in space using a kernel of basis functions to obtain

various orientations of the �lters� The kernel is designed to be separable and each term

is implemented using a recursive digital �lter�

ix

Once local energy has been calculated� the ridges and surfaces of high energy values are

determined using a �ooding technique� Starting from the points of local minima we per

form an ablative skeletonisation of the higher energy values� The topology of the original

set is maintained by examining and preserving the topology of the neighbourhood of each

point when considering it for removal� This combination of homotopic skeletonisation

and sequential processing of each level of energy values� results in a well located� thinned

and connected tracing of the ridges�

The thesis contains examples of the local energy calculation using steerable recursive �l

ters and the ridge tracing algorithm applied to two and three dimensional images� Details

of the algorithms are contained in the text and details of their computer implementation

are provided in the appendices�

x

Preface

The author has previously published some sections of this thesis in various conference

proceedings� These are referred to at appropriate points in the text� Speci�cally these

are�

� Local energy estimation using recursive �lters ��

� Flooding the energy �eld ��

� Surface tracing of local energy density values ��

The following paper was published jointly and contains some of the original ridge and

surface tracing material�

� Surface detection in �D confocal microscope images ��

Several sections of background information presented in this thesis are closely based on

their source� Such sections are referenced at the point of presentation and speci�cally

include�

� Section �� The phase congruency model� Owens ��

� Section �� Morlet wavelets� Pudney et al� ��

� Section �� Digital �ltering techniques� Gabel and Roberts ��

� Section �� The ztransform� Stark ��

� Section �� Prony s method� Lanczos ��

� Section �� The nonlinear leastsquares problem� Lim ��

xi

� Since the �lter design and implementation presented in Chapter � is based on the

work of Freeman �� and Deriche �� selected formulae and workings from

these papers are reproduced here�

Original material presented in this thesis is the work of the author�

xii

Acknowledgments

I would like to thank my supervisor Robyn Owens for her support and patience through

out my long tenure as her student� Her skill in both professional and family life is an

inspiration to all who know her�

My thanks go to all the members of the Robotics and Vision reseach group especially

Peter Kovesi� Ben Robbins and Matthew McDonald who have been especially helpful with

technical dicussions and practical assistance on computer coding� A special thanks goes

to Chris Pudney who has found practical application of the ridge tracing with confocal

microscope images and whose assistance and encouragement has been tremendous�

I would also like to thank my friends and colleagues in the Department of Computer

Science� in particular Bruce Backman and Chris McDonald for their help and support

during my research and the writing of this thesis� In fact I d like to acknowledge all of

the people whom I have come into contact with through the department over the years�

I must also express eternal gratitude to my parents John and Joan Robins for their

support� skills and e�orts in proof reading my written work�

Words can not express my thanks to Catherine� my wife� Her love and encouragement

have kept me going� Writing this thesis has taken a great deal of our time and I appreciate

her forbearance and support� I am looking forward to spending much more time with

her and our beautiful young daughter� Sophia�

This work was supported by an Australian Postgraduate Research Award�

xiii

Glossary

PC Phase Congruency�

HVS Human Visual System�

FIR Finite Impulse Response � discrete digital �lter speci�ed as a �nite set of coe�

cients�

IIR In�nite Impulse Response � discrete digital �lter speci�ed recursively�

DFT Discrete Fourier Transform�

FFT Fast Fourier Transform�

abscissae Distances of points from the yaxis measured parallel to the xaxis ��

CT Computed Tomography�

MRI Magnetic Resonance Imaging�

MPEG Motion Picture Experts Group � video compression standard �named after the

de�ning committee��

xiv

Chapter �

Introduction

�� General introduction

Digital image feature detectors often comprise two stages of processing� an initial �ltering

stage and a second search stage� The initial �ltering is designed to accentuate speci�c

feature characteristics or suppress spurious components of the image signal� The second

stage of processing involves searching the results for various criteria that will identify the

locations of the image features�

The local energy model of image feature detection follows this pattern� An initial �ltering

is performed with pairs of quadrature �lters and this is followed by a search for the highest

energy points which correspond to the feature locations� The search criteria include local

maxima and the ridges of high values in the energy map that trace out features in the

original image signal�

This thesis examines both the �ltering and ridge tracing required for a local energy based

feature detection scheme� Recursively implemented steerable �lters are used to calculate

local energy in twodimensional images and threedimensional volumetric data sets� An

ablative skeletonisation process is then developed with which to trace the ridges in the

energy map� Ridge tracing is extended to surface tracing in order to locate the locus of

points of high energy in the energy density maps of volumetric data�

�

CHAPTER �� INTRODUCTION �

�� Thesis overview

Chapter � introduces the subject and provides an overview of the work presented in sub

sequent chapters� It introduces the topic of image features� their detection and the local

energy model� and goes on to discuss the ridge and surface tracing and the algorithms

developed�

Chapter � gives a thorough review of the local energy model and the concept of phase

congruency� Calculation of local energy and its �lter requirements are discussed� In

particular� �lters based on the derivatives of the Gaussian function are considered�

Chapter � develops the steerable recursive �lters with which local energy can be calculated

across a range of scales in two and threedimensions� These �lters are used throughout

the thesis�

Chapter � is another review chapter which introduces the concept of peaks and ridges and

methods to �nd them� The watershed transform is introduced here along with various

topological concepts that are used in the ridge tracing algorithm�

Chapter � describes the ridge tracing algorithm� The algorithm is a blend of binary

skeletonisation and a watershed algorithm that uses anchor points provided by the local

energy model� Ridge tracing uses a shallow �ll value rather than thresholds to reduce

noise response�

Chapter � extends the ridge tracing to threedimensions where modi�ed homotopy rules

and a second stage of skeletonisation are used to generate suitable anchor lines� This

allows the tracing algorithm to trace the surfaces of high energy values that correspond

to threedimensional features�

Chapter � brie�y summarises the topics presented in the thesis and itemises the original

components of the work� A short section outlines possible applications and further work�

�� Finding image features

In order to process and analyse digitised images� computer vision systems need to be able

to automatically detect features� The digital pro�les of lowlevel features such as edges

and lines have been studied and modelled in great detail and yet a truly general feature

detection scheme has not yet been developed� Part of the di�culty lies in the de�nition


Umbra

Intensity

x

Illumination

Figure �� The image re�ected intensitypro�le at the occluding edge of a curvedLambertian object in front of a uniformbackground contains a graded dark band�due to the oblique surface� rather than thesimple step function that is usually takenas a model for image features�

Figure �� A step function and a roofpro�le shown with their principal Fouriercomponents� Notice the alignment of thephases of the component frequencies atthe points that correspond with the fea�

tures in each pro�le�

of the problem as di�erent initial assumptions lead to di�erent optimal solutions�

The Canny �� edge detector is a classic and widely used example of this� Canny s

objective was to optimise a linear �lter to locate a step function in a noisy data signal�

His optimisation criteria were chosen to achieve maximum signal to noise response and

location accuracy� and to produce only a single response to a single edge� The result was

a �lter closely resembling the �rst derivative of a Gaussian function� Convolution of a

signal with the �rst derivative of a Gaussian function has the e�ect of calculating the

gradient of the smoothed signal� Such a gradientbased approach to edge detection is

very widely used and most such procedures follow from Canny s work�

The step edge was considered a reasonable model for common image features because it

arises from occlusion� one object blocking the view of another� but is this correct� Most

images of natural scenes are of three dimensional objects with Lambertian and specular

re�ection characteristics� Objects do not always have sharp corners� they often have a


marked degree of curvature near an extremity� A pro�le showing shading caused by less

intense re�ection due to oblique scattering� as illustrated in �gure �� would seem to be a

more general model of an image feature� Feature pro�les are discussed in more detail by

Ronse �� in his study of the phase congruency model�

Natural images also contain many line features� Lines arise from thin objects such as

branches� power cables and ropes� and surface features like cracks� grooves or decoration�

The gradientbased edge models completely ignore these features� Edge detectors often

respond to a line feature as if it were two edges� giving multiple responses that are

incorrectly located�

�� Finding features using phase information

Rather than rede�ne feature detection optimisation in terms of an alternative intensity

pro�le� a more general model of features has been developed� Morrone and Owens ��

�� noted that when the phases of the Fourier components of the intensity pro�les are

examined� the phases are aligned or maximally congruent at points corresponding to those

that the human visual system �HVS� identi�es as features in the image� It does not matter

what form of feature or intensity pro�le is present� the congruence of phase reaches a

local maximum at the feature location� Examination of the actual phase angles involved

can be used to characterise the type of pro�le present �� Figure � illustrates the step

edge and roof functions with their Fourier components� When the Fourier expansion is

expressed as a sum of sinewaves� the components of the step edge are aligned at a phase

angle of �� and the roof pro�le at �� A negative going step would be aligned at �� and

an inverted roof� or trough� at ��

The phase congruence model of feature pro�les constitutes a generalised model of image

features� It is invariant to illumination� scale� orientation and translation� In fact it is a

dimensionless parameter and Kovesi �� has demonstrated that this is a major advantage

of using phase congruency�

�� The local energy function

Phase congruency is a cumbersome parameter to calculate� However Venkatesh and

Owens �� have shown� by simple trigonometric manipulation� that phase congruency

is closely related to a measure called local energy� Local energy is basically the magnitude


Figure �� Local energy �middle� andphase congruency �bottom� functionshave local maxima at locations corresponding with features in the image pro�le �top�� For local energy� only the location of the local energy peaks is of interest as the magnitude is proportionalto the contrast across the feature� Themagnitude of the phase congruency peakson the other hand provides a normalisedand dimensionless measure of feature signi�cance� Note that phase congruency ismore sensitive to noise in the signal�

of an envelope function of the signal �see Bracewell �� page �� Peaks in the local energy

function occur at the same locations as peaks in the phase congruency function� This is

illustrated in �gure ��

However� unlike phase congruency� local energy is not a normalised measure of feature

signi�cance and the magnitude of the energy signal is proportional to the intensity con

trast across the feature� They both locate the same features with maximal values in the

output signal�

By using local energy instead of phase congruency� one is trading independence from

intensity contrast for simpli�cation of calculation� However� because phase congruency

is so cumbersome to calculate� this tradeo� is essential� which is why this thesis is based

on local energy analysis�

�� Calculation of phase congruency and local energy

For a onedimensional signal� Morrone and Owens �� de�ne the local energy to be the

square root of the sum of the squares� of the signal� F � and its Hilbert transform� H�

LE � jjE�x�jj �qF ��x� �H��x��

The Hilbert transform of a function has the same magnitude of Fourier components but

the phase of each component is shifted by �� for positive frequencies and ��

� for negative

�Some literature de�nes local energy without the square root� i�e� LE � F��x� �H

��x�� Since it isonly the relative magnitude and not the absolute magnitude that we are interested in� this is immaterialand the result is equivalent�


frequencies� H has no D�C� component� A function and its Hilbert transform are said

to be in quadrature when they both have zero D�C� component and have the same L�

norm� that is�

ZF �x�dx �

ZH�x�dx � � ��

and ZF ��x�dx �

ZH��x�dx� ��

Also� the Hilbert transform is commutative over convolution so the Hilbert transform

of a signal convolved with some �lter is equivalent to the convolution of the signal with

the Hilbert transform of the �lter� This means that to calculate local energy it is not

necessary to explicitly calculate the Hilbert transform of the digitised signal�

Estimation of local energy

The two components of local energy can be estimated by convolving the signal with a

pair of quadrature �lters� This means that one �lter will be even symmetric� Fe� and the

other odd symmetric� Fo�

Local energy of the digitised image pro�le� I� can be estimated by�

LE � jjE�x�jj �qMe

� �Mo��

where Me � I � Fe and Mo � I � Fo� that is� the signal convolved with even and odd

quadrature �lters� Filters typically used to calculate local energy are �rst or second

derivatives of the Gaussian� Gabor and wavelet functions and their Hilbert transforms�

In e�ect� one �lter of the pair� Me� forms a bar � detector and the other� Mo� an edge

detector�

Chapter � contains a thorough review of the local energy and phase congruency literature

presenting a range of techniques and �lters that have been used to calculate local energy�

It examines the derivation and implementation of local energy and phase congruency

feature detection in more detail�

�A bar detector responds to narrow pulses or bars in the input signal� In two�dimensions this istypically the pro�le across line features in an image�


This thesis uses the second derivative of a Gaussian and its Hilbert transform as convo

lution �lters although many pairs of quadrature �lters could have been used� The second

derivative of a Gaussian was chosen because it is suitable for the recursive steerable �lter

implementation described in Chapter ��

�� Features at multiple scales

The e�ect of using �lters to estimate local energy is to reduce the bandwidth and apply

a spatial extent to the analysis� limiting the constituent frequencies of the image features

that are analysed� This reduced frequency response limits the scale of features that are

detected� A study by Field �� of natural images� indicates that images tend to contain

features whose frequencies drop o� at �� and that the HVS responds to approximately

three octaves of feature sizes� Features are analysed across a range of scales by combining

the output of numerous geometrically scaled �lters� To ensure that the energy contributed

by each �lter is constant� the �lters are scaled geometrically in magnitude and frequency�

The total response of the system is tuned to evenly cover the spectrum across about

three octaves�

The phase congruency calculation requires an estimate of the magnitude of the signal s

frequency components in quite narrow frequency bands� Wavelet analysis is well suited

to this task and in his thesis� Kovesi �� presents a thorough exposition of the phase

congruency calculation� The phase congruency model is reviewed in Chapter � of this

thesis� Figure � illustrates the spectrum coverage of several Morlet wavelet functions

that might be used to calculate phase congruency�

�� Features in multiple dimensions

The phase congruency and local energy models have been de�ned for one dimensional

signals� In order to analyse multidimensional signals such as images or volumetric data�

the signal must be analysed at several orientations� Onedimensional quadrature �lters

are rotated to various orientations and smoothed in the orthogonal directions� Results of

these individual convolutions are then squared and summed to derive an oriented local

energy� The various oriented energy results are combined to form a congruency function

or local energy map�

CHAPTER �� INTRODUCTION

ω

Figure �� In order to provide an even response to features across a range of scales�several �lters are used� The sum of theirenergy spectra should form a band passtransfer characteristic�

Figure �� The spatial extent and orientations of the energy �lters must combineto have an even coverage for the orientationally selective �lters used to calculateoriented local energy�

The degree of smoothing and number of orientations is selected to provide even coverage

in all dimensions� A number of researchers have used this approach including Perona ��

Kovesi �� Kubler et al� �� Pudney et al� �� and the original work by Canny ��

A discussion of the requirements for smoothing functions and angular selectivity can

be found in Ronse s study �� of the phase congruency model� Figure � illustrates the

notional spatial coverage of six orientations of a circular �lter�

�� A recursive implementation of local energy

Chapter � of this thesis presents a new implementation for local energy using steerable

recursive �lters� This new implementation provides �exibility in the number of orien

tations and spatial scale of the �lters used� In addition the recursive implementation

uses a �xed number of operations per point regardless of �lter width� The �ltering is

performed in the spatial domain which has advantages in certain applications� lending

itself to coarse grain parallel implementation or a foveated analysis system�


Figure �� Image of a dark square on a light background� its energy function and anorthographic view of the energy function� Along the edges of the square� scanned perpendicular to the edge there will be a peak� At a corner� where there is more informationin the image� there will be a local maximum� peaking at the location of the corner�Depending on the scale of the �lters and the degree of smoothing� the corners may beslightly rounded and the local maximum less pronounced�

�� Finding peaks and ridges

In order to identify the location of each feature once a congruency function or energy

map has been calculated� the peaks in the function must be found� In one dimension

this is simply �nding the local maxima� for which there are many techniques and gener

ally a unique solution�� In multiple dimensions the problem of �nding extrema is more

complicated�

For example� in the energy function of a dark square on a light background� as illus

trated in �gure �� the local maxima corresponding to the corners can easily be identi�ed�

but what mathematical criteria can be used to describe the highest points along the

connecting ridges�

Chapter � considers the de�nition of ridges and provides a review of various techniques

that have been used to determine where the peaks and ridges of a surface topology lie�

Finding an answer to the question of how best to determine the location of the peaks and

ridges of this two dimensional energy function forms the basis of the bulk of this thesis

and is presented principally in Chapter ��

�There is an ambiguity when two or more neighbouring ordinates are of the same maximum value�

CHAPTER �� INTRODUCTION ��

�� Gradients and non�maximal suppression

A gradientbased technique is commonly used to �nd ridges in twodimensional data sets�

For each point� a signi�cant orientation is determined� This direction is then examined

for a one dimensional maximum� Thresholding heuristics have been invented to join these

ridge points and to avoid spurious maxima due to noise� The orientation to be used is

usually determined from consideration of surface gradients but� unfortunately� gradients

can become ambiguous at corners and junctions�

The common technique� referred to as hysteresis thresholding� was introduced by Canny ��

�� in conjunction with his edge detector� Hysteresis thresholding uses two thresholds in

order to link maxima in a continuous fashion and to avoid noise� Various other ap

proaches have been used� based on surface curvature to de�ne and �nd ridges or creases

in surfaces�

These techniques all rely on choosing an orientation and �nding some particular char

acteristic in the crosssection of the function� In some ways this is reintroducing the

problem of feature detection� albeit in a more constrained� gradient map domain�

Rather than alter the model of the ridge� it would seem to be preferable to try some other

technique� Greylevel morphological algorithms such as the watershed� as used here� o�er

a whole new class of solutions�

�� A new ridge tracing algorithm

Chapter � of this thesis describes a new ridge tracing algorithm� The algorithm is based

on a �ooding simulation �� related to a watershed algorithm �� Importantly the ridges

are thinned and connected contours which join at junctions� An overview of the procedure

is presented here�

The ridges are traced by an ablative skeletonisation procedure� Starting at the points of

absolute minima� the algorithm sequentially erodes upwards with the erosion restricted

to preserve the topology� of the peaks� The absolute peaks that correspond to keypoints

in the original image are used to anchor the erosion� Homotopic� erosion ensures that the

ridge is preserved regardless of its absolute height� The sequential nature of the erosion

�Topology refers to the connectedness and hole enclosure of sets of pixels��Homotopic operations do not change the topology�


Fill depth

Figure �� An initial �ll operation is performed to reduce the segmentation due to noiseand to �ood low ridges� Each minimum is �ooded to a small �ll depth as illustratedabove� The topology of the resulting islands is preserved during the erosion phase of theridge tracing algorithm�

ensures that a ridge is centrally located� close to the medial axis� no matter how shallow

or gentle the curvature�

Like Beucher s �� and Vincent s �� watershed algorithms� the ridge tracing starts �ood

ing from every point of local minimum� Each start point will lead to its own catchment

basin which can lead to severe oversegmentation of the image� We also �nd with the

basic procedure that noise� close proximity of features and the �nite spatial extent of

the �lters all lead to the detection of many small ridges that do not correspond to real

features in the image� It is also found that a small ridge� only one or two energy quanta

high� will often appear to join two features together but in fact is an artifact of the spatial

extent of the �lters rather than a feature�

To overcome these problems an initial processing step is introduced which involves �lling

each basin in the topography to some predetermined depth� This shallow �ooding will

immerse small ridges and peaks thereby joining many catchment basins together� The

resulting segmentation is much less cluttered but still contains the higher ridges from

signi�cant features because only small ridges and peaks disappear� Figure � illustrates

this �lling process� It is implemented using the same �ooding simulation technique as

the ridge tracing algorithm� and is e�cient�

The �ll depth is an �arbitrary threshold that must be applied to the algorithm� but it


has been found to be robust� and reduces in an intuitive way the features that are found�

Filling rather than thresholding is more appropriate for the local energy model because

it is the relative heights of the peaks that are important rather than the absolute heights�

The ridge tracing technique presented in this thesis is well suited to the local energy

model of feature detection� It forms the missing link for application of the local energy

feature detection to twodimensional images� it is e�ective� producing a thinned result

that includes junctions and isolated features� the �ll depth provides an intuitive noise and

information reduction parameter� and the algorithm is simple and e�cient to implement�

The implementation uses ordered queues to hold references to pixels that are awaiting

processing� This means that only the pixels that are possible candidates for erosion

are considered at each step� The algorithm is sequential but� unlike many serial image

processing algorithms� there is no unnecessary raster scanning of the image data�

�� Finding surfaces and �laments

Chapter � goes on to extend the ridge tracing algorithm to the three dimensional case

where the energy function contains �lms and tendrils� The topology is one of density

values rather than of heights as was the two dimensional case� The erosion phase in this

case must be anchored� not only by peaks� but also by the �laments that delineate the

�lms or surfaces of high energy values� A preliminary erosion is introduced with slightly

modi�ed homotopy rules� which produces a network of anchor lines that anchor the edges

of surfaces�

As was the case in twodimensions� the technique is well suited for use in conjunction

with the threedimensional local energy calculation� It produces thin� accurately located

results with the correct topology� The preprocessing �ll density is a useful and robust

parameter and the implementation is e�cient and relatively simple�

�� Results

The ridge tracing technique requires the availability of suitable anchor points� These are

present in phase congruency and energy maps� but not in many image domains where

ridges are sought directly from the intensity values� Typical unsuitable applications

include searching for ridges in MRI or Xray data� interferometry and �nger print analysis�


Figure � A threedimensional surfacerendering of the feature locations identi�ed by the local energy model and threedimensional ridge tracing of a CT imageof a human neck�

These direct searches for ridges su�er from noise� scale and intensity variation problems

that would perhaps be removed or reduced if the features were analysed within the

framework of the phase congruency model� In such cases the ridge tracing would then

be applicable for locating the features from the congruency function or energy map�

Figure is a rendered image of the threedimensional feature locations found using the re

cursive implementation of local energy and ridge and surface tracing algorithms described

in this thesis�

Various appendices are presented which contain details of various �lter designs� mathe

matical calculations and details of computer code used and developed for this thesis�

Chapter �

Local Energy Feature Detection

This chapter discusses feature detection and reviews the local energy model of image

features� Image processing in the local energy feature detection model requires digital

�ltering of the data with speci�c pairs of �lters� We examine what additional constraints

must be placed on the frequency response and orientational selectivity of the �lters in

order to extend the local energy model to treat multidimensional data and incorporate

a multiscale analysis�

This chapter also discusses the implementation details and e�ciency of digital �ltering

in the spatial and frequency domain� It introduces the concept of a recursive �lter� which

is used in the following chapter where recursive �lters are designed that are applicable

to the calculation of local energy�

Two important classes of �lters are described� the derivative of the Gaussian functions

and Morlet wavelet �lters� The derivative of Gaussian functions will be used in chapter ��

where a novel recursive spatial domain �ltering scheme is developed�

It is found the Morlet wavelet functions are not suitable for use with the recursive �lter

design methodology� however they are an important multiscale �ltering technique and

are included here for completeness�

A review of several other classes of �lters that have been used by others to perform

energybased feature detection is attached as Appendix A�

��

CHAPTER �� LOCAL ENERGY FEATURE DETECTION ��

�� A brief history of local energy

The phase characteristics of image features were �rst noted by Morrone and Owens ��

who proposed the use of local energy as a feature detector �� A number of papers

followed� exploring various aspects of the local energy model� such as idempotence ��

and culminating in Venkatesh s practical implementation and study of energy based mod

els for detection and classi�cation of images ��

Energy based models have been used by various researchers such as Freeman� Kubler�

Ronse and Aw� Kovesi has concentrated on using the phase congruency itself� rather

than local energy� to develop a measure of edge signi�cance� His use of phase congruency

is a measure without dimensions and is therefore independent of the particular class of

image �� Ronse� in his study �� provides an extensive review of phase congruency

techniques�

The e�ect of scale on local energy feature detection has been investigated by Venkatesh�

Cooper and White �� where they present re�ned �point �lter masks with which to

calculate local energy� This report also contains an eloquent exposition of basics of local

energy�

In his thesis� Aw �� further consolidates the theory and mathematical construction of

phase congruency and also presents a scheme for catalogue based image compression

using a neuralnet to classify image features found using local energy�

Recent attention has been focused on the extension of the local energy model to two

and threedimensional data sets� Work in this thesis follows from that of Heitger et

al� �� Freeman �� Perona �� and from twodimensional considerations pointed out

by Canny �� and Marr and Hildreth �� Deriche s �� and Monga et al� s ��

�� implementation of two and threedimensional edge detectors using recursive �ltering

motivated some of the work presented in this thesis�

Recent research at The University of Western Australia into the detection of two and

threedimensional features includes a thorough presentation of the oriented energy model

by Robbins �� Applications using oriented energy for surface detection have been

developed by Pudney et al� �� and Kovesi et al� ��

Two PhD theses dealing with various aspects of the multidimensional local energy model


have recently been presented at The University of Western Australia� Kovesi �� devel

oped a method of calculating phase congruency using wavelets to estimate an oriented

congruency measure� He investigated di�erent �lters and presented a noise estimation

technique to reduce the impact of noise on feature detection� Robbins �� developed the

oriented energy concept and examined its e�ectiveness as a higher order feature detec

tor� He concentrated on detecting higher order features such as corners in two and three

dimensions� He also presented a number of optimisations that can be used to improve

the e�ciency of frequency domain �ltering�

In chapter � of this present thesis� I investigate the use of steerable recursive �lters to

calculate oriented energy in the spatial domain� Chapters �� and � are devoted to

presenting a new method of tracing image feature locations from ridges in the energy

map� The ridge tracing technique that is developed is e�cient and applicable to both

two and three dimensions�

�� Image feature detection

�� Overview of image feature detection

The objective of low level image processing is to process digital images in such a way as

to identify points of visual interest� We refer to these points as features and group them

to form contours� Points of visual interest include boundaries between distinct regions of

an image� as well as discontinuities inside these regions� and also the lines in line drawings

and sketches� They are measured and classi�ed by assigning qualities to them such as

salience and direction� and descriptive labels such as lines� edges� surfaces and junctions�

Early work by Marr and Hildreth �� has been the basis of extensive work de�ning�

analysing and detecting various forms of features in images� Elaborate and elegant models

of features have been developed and have been classi�ed into three groups by Ronse ��

These are the physical� psychophysical and mathematical models�

Physical models construct an image as a two dimensional projection of light re�ected from

object surfaces� Image features are formed by surface curvature� occlusion� orientation

and illumination�

Psychophysical models have been developed from studies of the primate and human visual

system �HVS� by examining both the physical processes and perceptual manifestation


inherent in vision� Inferences from visual illusions and conundrums are used to investigate

models of the processes involved� Dissection� surgical probes and some lessinvasive

techniques such as MRI are used to investigate the physical processes of the brain that

are involved with the process of vision�

Mathematical models treat greylevel images as mathematical functions� Feature detec

tion becomes a study of derivatives� discontinuities and� more recently� phase information�

Features are described as the locus of points that satisfy certain mathematical criteria�

Typical criteria are� discontinuities in the function or its derivatives� zerocrossings in

second derivatives� or local maxima in gradient or phase congruency�

Typical of the mathematical genre� the Canny �� edge detector uses a �rst derivative of a

Gaussian �lter to approximate an optimum edge locating �lter� As discussed previously

this is a popular edge detector with reasonable implementation e�ciency� The associated

gradientbased approach has been used by many researchers and extended to multiple

dimensions in various ways�

The phase congruency model for image features takes feature detection one step further�

It notes that features occur at points in a signal where there is order in the phase infor

mation� that is� where the Fourier components are maximally in phase with one another�

This was illustrated in �gure � in Chapter �� Notice that the phase congruency model

does not presume any particular luminance pro�le for a feature and is therefore very

general� Its drawback is its computational complexity� Fortunately� phase congruency is

related to local energy� which can be calculated more simply�

Many researchers have studied the local energy model� particularly Morrone� Burr and

Owens �� and Owens and Ross �� and Venkatesh �� The mathematics of

phase congruency has been studied by Kovesi �� and Ronse �� Ronse s study of phase

congruency contains a thorough review and exposition of the mathematical theory�

Subsequent work has concentrated on extension of the energy model to features in multi

ple dimensions� Papers by Robbins �� Robbins and Owens �� Kovesi �� Pudney et

al� �� Perona �� and Heitger et al� �� all concentrate on applying the onedimensional

results at a number of orientations� Canny �� Deriche �� and Freeman �� have

also applied multiorientation analysis to edge detection� a technique that was suggested

much earlier by Marr and Hildreth ��

The following three subsections �� to �� discuss the development of the phase

CHAPTER �� LOCAL ENERGY FEATURE DETECTION �

congruency model of feature detection and the associated local energy calculations� Sec

tion �� describes the extension of the local energy model to multidimensioned data

and includes an overview of the procedures that will be developed later in Chapter ��

�� The phase congruency model

Morrone and Owens �� suggested that the location of features in images coincides with

points where the deviation of phases of the Fourier components of an image is at a

minimum� The minimum deviation is expressed mathematically as the maximum of a

phase congruency measure� The Fourier components of some simple feature pro�les were

illustrated in �gure � of chapter ��

This model of features is appropriate for step and impulse discontinuities as well as for

ramps and other luminance pro�les� It also predicts features� such as mach bands� that

are perceived by the human visual system� It is general and robust and is also supported

by psychophysical evidence�

Phases of a step function

The following examination of the Fourier components of a step function follows Owens ��

Consider a onedimensional signal f�x�� representing a slice through an image� This

signal can be reconstructed from its Fourier series by�

f�x� ��Xi��

An cos�n�x� �n��

where� for each frequency n� An is the amplitude of the cosine wave and n�x��n is the

phase o�set of that wave� The term � is a constant frequency scaling parameter �often

�� related to the size of the image window�

If the image signal were a simple step edge� then

f�x� ��Xn��

�

�n� �cos�nx�

�

��

and� at the point of the step edge� where x � �� all the phase terms� are aligned at ��

This is the only place in the signal where there is congruency in the phase values� At

�In Chapter � the alignment was in terms of sine functions whereas here the cosine function has beenused to reconstruct the signal� This results in a constant phase oset of �

��


all other points� the phase values of individual frequency components assume di�ering

values between � and ��

Similar congruency of phase values occurs at the apex if the image signal has a triangular

pro�le� Morrone and Burr �� point out that in general� the points in a signal where

there is local maximal congruency� or order in the phase values� coincide precisely with

points where human vision perceives features�

Measuring phase congruency

The minimum of phase deviation is measured as the maximum in a measure of phase

congruency �� The phase congruency function can be expressed as

PC�x� � max��

Pn�An cos�n�x� �n � ��P

nAn� ��

whereAn are the magnitudes of the Fourier components of the function� � is the frequency

parameter and �n are the phases of the Fourier components�

The value of � that maximises equation � is the weighted mean phase angle of all the

Fourier terms at the point being considered� Taking the cosine di�erence between the

actual phase angle of a frequency component and this weighted mean� �� generates a

quantity approximately equal to one minus half this di�erence squared �� Finding where

the phase congruency is a maximum is approximately equivalent to �nding where the

weighted standard deviation of phase angles is a minimum ��

Equation � is illustrated in �gure � showing the construction of the phasor from the

sum of component phasors� We can see that the magnitude of this phasor will be a

maximum when the phases are all aligned� i�e� congruent� We can also see that the

phase congruency measure is normalised� being the ratio of the actual magnitude to the

maximum possible magnitude� The denominator�P

nAn� normalises the expression to

the range �� which means that phase congruency gives a dimensionless measure of

feature signi�cance� No other measure of feature signi�cance gives such an absolute�

independent and fundamentally derived value�

�the Taylor expansion of cos�x� � �� x�

�for small x�


Re

Im

Im

Re

Figure �� Phasor diagram illustrating how the phase and amplitude of the various Fouriercomponents combine� We can see why phase congruency� being the length of the resultantphasor divided by the total length of component phasors� is maximum when the phasesare aligned i�e� congruent�

Problem with dominant noise

Featureless regions of an image can have a dominant frequency present that is insigni�cant

on a wider scale� but locally gives the impression of congruency� In these regions most of

the energy comes from what is basically noise� which led Kovesi to propose a modi�cation

to equation � that cancels the noise by subtracting a noise estimate calculated for the

region or image�

Unfortunately phase congruency is quite awkward to calculate for each point in an image

and the normalisation is sensitive to noise in featureless regions of the signal� The closely

related local energy function of a signal� on the other hand� is more robust and simpler

to calculate�

�� The local energy model

The local energy function is de�ned for a onedimensional signal F �x�� as

E�x� �qF ��x� �H��x��

where H�x� is the Hilbert transform of F �x��


Venkatesh and Owens �� show that local energy is proportional to phase congruency�

E�x� � PC�x�Xn

An� ��

where An are the Fourier amplitudes of the components of F �

Since local energy is directly proportional to phase congruency the peaks in local energy

will correspond to peaks in phase congruency�

The di�erence between local energy and phase congruency is that local energy is easier

to calculate� but the value of local energy is a�ected by the intensities of the various

frequency components present in the image signal�

Since phase congruency is conveniently normalised to the range �� the value of phase

congruency is directly related only to the signi�cance of the corresponding feature� Local

energy is not normalised and its value is in�uenced by image intensity� Feature locations

are indicated by peaks in the energy function with the relative heights of the peaks related

to the feature signi�cance�

�� Calculation of local energy by convolution

The signal s Hilbert transform need not be directly determined� as the two functions

F �x� and H�x� can be formed by convolution of the signal� I�x�� with two �lters that

form a quadrature pair� one being the Hilbert transform of the other� The pair of �lters

comprise one evensymmetric and one oddsymmetric �lter� The even �lter� Meven� is

designed to alter F as little as possible and remove any D�C�� from the signal� It is usually

a band pass �lter and therefore restricts the resulting analysis to speci�c frequencies or

scale of features in the image� The odd �lter� Modd� is its Hilbert transform� The Hilbert

transform requires that Meven has no D�C� term and frequency consideration requires

that the �lter be tapered towards D�C� and the Nyquist frequency�� A function and its

Hilbert transform are in quadrature� that is

ZMeven�x�dx �

ZModd�x�dx � � ��

�Direct Current �D�C�� refers to any zero frequency component in the signal��The Nyquist frequency is the highest frequency signal that could be completely reconstructed from

the sampled data series� This is usually half the sampling frequency�


and ZM�

even�x�dx �

ZM�

odd�x�dx� ��

Convolution of the image with these �lters� often referred to as masks� gives two signals

each being a band passed version of the original� with one being a �� phase shift of the

other� Squaring and summing the results of these two convolutions computes local energy

de�ned by�

E�x� �q�I�x� �Meven�� I�x� �Modd��

where � represents the convolution operator�

�� Extension of the local energy model

The local energy and phase congruency concepts discussed above are only de�ned for

onedimensional functions� To apply them to twodimensional image data and three

dimensional volumetric data the concept has been extended to incorporate oriented en

ergy�

The single �lter pair used above also limits the analysis to a range of feature scales�

Kovesi �� describes a multiscale approach for the analysis of local energy across a

broad range of feature scales�

Oriented Energy

There is no Hilbert transform for twodimensional functions� which leaves a dilemma as

to the extension of local energy into higher dimensions� Simply taking a onedimensional

estimate in orthogonal directions is insu�cient because the response to features that are

not aligned with the axes is nonuniform� This is particularly evident as the size or scale

of the �lters is increased�

Both Canny �� and Deriche �� use a single �lter as an edge detector in two and three

dimensions by taking the onedimensional �lter and smoothing it in the orthogonal di

rections� This produces a �lter with the desired characteristic along one direction and

symmetric about this axis� During the early stages of this present work �� I applied

this approach to calculate local energy using the pair of �rst and second derivatives of


0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

�a� 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

�b�

Figure �� Contour plots of the sum of squares of G� function evaluated at � and �orientations� Note the slightly nonuniform response evident with three orientations� thiswould result in slight variation in response to features at various orientation�

the Gaussian �lter smoothed in the orthogonal direction but found that the nonuniform

response with respect to orientation caused problems� Using only two axially aligned

�lter orientations meant that the energy response from diagonal features was lower than

for the same features aligned with an axis�

Freeman �� designed �lters that were in quadrature in one direction and Gaussian

smoothed in the orthogonal direction� These �lters were designed to be steerable and he

applied them at several orientations to determine the most signi�cant orientation� He

then looked for junction characteristics at a point by searching for a second dominant

orientation� He also formed an oriented energy measure as the sum of energies in each

of the component directions�

This oriented energy approach has been adopted here� where I calculate a value of local

energy using smoothed and rotated �lters for several orientations and sum the partial

energies� To give an even response to features at all angles� su�cient orientations must

be used and the angular response of the �lters must sum to an even value around the

circle� This can be checked by applying the process to an image that contains a clear

circular object�

The �lters used in this thesis need to have a symmetrical component� in order to be

steerable using Freeman s technique� so I have used the same width Gaussian for the

orthogonal smoothing as is present in the pair of quadrature masks� Using this rather


broad smoothing gives a wide angular spread and only three or four orientations need to

be used� This is illustrated in �gure ��

Since the oriented energy result is an even function� due to the quadratic nature of the

calculation� the orientations are spread across �� rather than across the full circle�

The energy calculation is now

E�x� �X

�v�OrientE�v�x��

where the orientations are de�ned by unit vectors v and speci�ed as the set Orient�

The quadrature �lters used to calculate the threedimensional value of energy E�x� are

formed by rotating the �lters�

Modd�x� � Modd�x�e��y��z��

��

and

Meven�x� � Meven�x�e��y��z��

��

to align the axis of symmetry� in this case x� to the direction given by the unit vector v�

The nomenclature for an odd and even quadrature pair of mask �lters rotated in multiple

dimensions would be Modd�v

�x� and M even�v

�x��

Figure �� illustrates the development of a twodimensional axially symmetric �lter from

the onedimensional second derivative of the Gaussian function� Figure ��a� shows the

onedimensional second derivative of the Gaussian �lter� Figure ��b� shows a two

dimensional �lter formed from it by smoothing in an orthogonal direction� Figure ��c�

illustrates the same �lter but with a shaded intensity plot and Figure ��d� shows the

same �lter rotated by � � The intensity plots �c� and �d� use darker shades for negative

values and lighter shades for positive� Greater detail could be shown by using a false

colour scheme where a spectrum of colours would be used to represent the range of values�

Shaded intensity images are usually normalised to take full advantage of the dynamic

range of available shades� which can mean that the zero level �the grey background

shade� may di�er from plot to plot�


�a�

-1

-0.5

0

0.5

1

�b�

�c� �d�

Figure �� Four views of a twodimensional �lter� �a� shows the pro�le of the function��x� � ��e�x

�� b� shows an orthographic view of this function smoothed in the orthog

onal direction� �c� contains an intensity plot of the two dimensional function� Darkershades represent negative values and lighter shades positive� �d� illustrates the samefunction rotated through an angle of �

�

Energy across scale

The local energy model of image features is based on an analysis of the relative phases of

the constituent frequencies� A local analysis is performed at each point of interest in the

image� in computer vision this generally means separately at every point in an image� The

calculations involved are simpli�ed by �nding the local energy through convolution of the

image signal with a pair of �lters� The design of these �lters necessitates a compromise

which restricts the frequency response of the analysis�

The e�ect of this constrained frequency response is to limit the e�ectiveness of the analysis

as a feature detector� The nature of the features detected by the local energy analysis


will be determined by the frequency response of the �lters�

A particular �lter will respond to features that comprise frequencies that match the

�lter� Small high frequency �lters will match small sharp features while broader �lters

will match spatially larger� possibly more blurred� features�

All �lterbased feature detection schemes exhibit this limited response� Traditionally a

scale of analysis has been chosen that matches the detection task at hand� Images are

usually photographs� and digitisation techniques determine the scale of interest� Many

researchers have looked at the e�ect of scale on feature analysis� These include Perona ��

and of course Kovesi �� with his wavelet analysis�

Ideally we would choose a �lter with a broad� �at and uniform frequency response�

The local energy calculation requires a band pass characteristic and researchers such as

Venkatesh �� have investigated the traditional Chebechev and Butterworth �lter design

techniques� Field �� has examined the frequency characteristics of natural images and

has determined that they tend to exhibit a �� structure�

With the advent of the wavelet theory another technique can be applied whereby the

results of many scaled� but similar� �ltering operations are summed to provide an anal

ysis over a broad range of frequencies� Kovesi has used a modi�ed Morlet wavelet in his

implementation of the phase congruency measure� He points out that the HVS has a fre

quency range of approximately three octaves and combines the results to incorporate the

�� characteristic of natural scenes� This technique can also be applied to the calculation

of local energy� Energy is calculated using �lters scaled in frequency and amplitude in

such a way as to provide a �at� even response across a three octave range� Thus

E ��octavesXf�f�

afEf � ��

where

Ef �q�I �M even

f �� I �Moddf ��

with M evenf and Modd

f being the even and odd �lter masks scaled in frequency�

For example the derivative of the Gaussian function G��x� � �x�� e

� x�

�� is tuned to various

frequencies by changing the scale parameter ��

One very important aspect of the local energy feature detector is that phase congruency


0 100 200 300 400 500 600

FIL

TE

R R

ESP

ON

SE

Components of Local Energy

Input signal fxkg

Edge detector F ��x�

Bar detector H��x�

Local energy E�x�

Figure �� Local energy response� � � �� to step edge features with various separations�For the given input waveform� the two components of the local energy function and theircombination as local energy are shown�

occurs at the same location regardless of which constituent frequencies are being exam

ined� This means� for example� that a step edge will be detected at the same location by

both high frequency and low frequency �lters�

The exception to this is when features interact� Depending on the scale of analysis� two

nearby step edges could be seen as separate edges of a rectangular pro�le� or be detected

as a single bar feature� The human mind is very skillful at subjectively choosing a scale to

interpret such conundrums but computers are much more literal in their interpretation�

Fortunately the local energy model is not too ambiguous� Two edges are �rst detected

at the correct location and then as the scale is increased the result changes� in a fairly

sharp transition� to display only a single feature� The feature locations are not drawn

together as the scale changes� A local energy detector using a single �lter is quite robust

in terms of its application�

Figure �� illustrates the e�ect of �lter scale on the location of detected features� A local

energy operator� of �xed scale� is applied to a onedimensional function comprising step

edges separated by various distances� This is equivalent to using a �xed separation and

altering the scale of analysis�

When the separation is much greater than the detector resolution� two features are clearly

detected as shown on the left of �gure �� As the separation is decreased� the local energy

responses to the features begin to interact� but the two features continue to be detected


100

101

102

103

0

50

100

150

200

250

300

�a�10

010

110

210

30

50

100

150

200

�b�

Figure �� Summation of frequency responses of� �a� geometrically scaled second derivative of the Gaussian functions� and �b� a weighted sum of these functions� This empiricallydetermined weighting over three octaves is �� and � with geometric scaling by a factorof three�

at their true location� When the separation is below the resolution of the detector the

interaction results in an apparent single feature located between the two original features�

This transition from two accurately located features to one combined feature is a very

sharp transition� which is in accord with earlier observations that the feature location

is invariant across scales� As the separation becomes very small the energy function

diminishes to the point where the response is indistinguishable from the signal noise�

The Morlet wavelets used by Kovesi form a �at frequency response when scaled and

summed�

The derivative of the Gaussian functions� that are implemented in the following chapter�

unfortunately do not scale so well� The frequency responses� plotted on a log scale to

take account of the �� nature of the required �ltering scheme� of these functions� as

shown in �gure ��a�� contain signi�cant energy at low frequencies� This low energy tail

is present in all scales of the �lter� which makes it di�cult to determine a combination

of scales and amplitudes that give a uniform result across the three octave range that we

are aiming for� Figure ��b� shows an empirically determined combination of scales and

amplitudes that gives a reasonably �at response� The local energy model is very robust�

which means that the Gaussian based �lters are still useful even if not as elegant as the

Morlet wavelets�


�� Filter pairs for local energy analysis

Venkatesh and Owens �� investigated the use of various quadrature pair �lters that can

be used to calculate local energy� The results showed that the choice of �lter was very

robust as long as appropriate quadrature constraints are met� One �lter must be the

Hilbert transform of the other� that is their spectra are the same but all frequency com

ponents are shifted by �� It is also important that neither �lter has any D�C� component

and that the L� norm of the two �lters is identical� One �lter will be even and one odd�

Venkatesh and Owens �lters were all onedimensional� When applied to images they were

simply applied in the x and y directions and separate x and y features were detected�

When small scale �lters are used this does not cause much problem with feature location�

but the results with larger �lter scales obviously are limited by having no information

about the location except along the single line of the �lter�

Ronse �� considered the functions in two dimensions and speci�ed extra requirements�

Firstly� he stated that the Fourier transform of the even function has nonnegative values

and that the pair form an oriented quadrature pair�� Secondly� the pair of �lters must

be continuous and integrable and their integrals over all space must be zero� These

requirements lead to the qualitative guidelines that the zeros of the two dimensional

functions form a discrete set of lines perpendicular to the chosen orientation� and that

the pro�le perpendicular to the chosen orientation is otherwise generally �bell shaped��

These constraints ensure that multiple responses do not result for ideal edge pro�les�

The Hilbert transform

The Hilbert transform �� of a function F �x� is de�ned by the singular integral

H�x� ��

�

Z �

��F �x��dx�

x� � x��

It can be seen that H�x� is a linear functional of F �x�� In fact it can be obtained by

convolution with ��x � that is

H�x� ��x

� F �x��

�Ronse develops the mathematics and notation for multiple dimensions�


The Fourier transform of ��x is i sgn�s�� The Hilbert transformation is a �ltering� in

which the amplitudes of the spectral components are unchanged but their phases are

altered by �� positively or negatively according to the sign of s�

Because all cosine components transform into negative sine components and all sine

components transform into cosines� the Hilbert transform of an even function is odd and

that of an odd function is even� The Hilbert transform of a real function is also a real

function� The Hilbert transform is only de�ned for onedimensional functions� which

necessitates the use of summations of oriented energy to estimate local energy in images�

�� The Gaussian function

The Gaussian function�

G�x� � e�x��

is widely used in statistics and signal processing� The function has several useful charac

teristics including smooth decay and in�nite di�erentiability� It is analytically tractable

and for practical purposes has limited support� decaying quickly to near zero values�

The Gaussian function is the function that minimises spread in both the spatial and

frequency domains� The Fourier transform of the Gaussian function is itself a Gaussian�

The normalised equation for the Gaussian function is

G�x� ��p��

e�x�

��

where �� is known as the variance and � the standard deviation of the �normal distri

bution in probability and statistics� It is worth noting that while the function extends

to in�nity� for practical purposes the function has a support which measures only a few

standard deviations� Three standard deviations covers �� of the area under the curve�

In image processing we often want a smoothing procedure to reduce noise� assuming noise

is high frequency� and use a Gaussian to low pass �lter the signal�

The function is radial in multiple dimensions and separable� so it is e�cient to implement

as a digital �lter� A multiple dimensioned Gaussian function is often referred to as


G�x� G��x�

G��x�

G�x� � e�x�

��

G��x� � �xe�x� ��

G��x� � �x� � �

��e�x

��

Figure �� The Gaussian function and its �rst and second derivatives� The equationshave been normalised to illustrate the essential nature of the functions as a product oftwo terms� a polynomial in x and a Gaussian function�

a Gaussian Ball� This is often used to modulate some other function to give a well

localised response�

Scaling the Gaussian function

In most cases throughout this work the scaling factors � and �p��

have been dropped to

permit the arguments to be developed with the simple equation G�x� � e�x�� Scaling and

normalisation factors are determined and applied separately� Scaling can be considered

as a change of variable x� x� so the simpli�cation can be made without loss of generality�

The normalisation factor is simply a multiplicative constant which can be carried through

the workings if required� The use of the Gaussian function in image processing produces

results that are used in relative terms rather than absolute� unlike in statistics� and these

constants can often be dropped altogether�


Derivatives of the Gaussian function

The derivative of the Gaussian function has been widely used as an edge detector for two

reasons� Firstly� Canny has shown that it is a good approximation to an optimally derived

step edge detection function� Secondly since di�erentiation highlights discontinuities and

changes in the signal� and moreover since di�erentiation is commutative over convolution�

both smoothing and di�erentiation can be performed by a single convolution with a

derivative of the Gaussian �lter�

The Gaussian function and its �rst and second derivatives are illustrated in �gure ��

The equations have been normalised and all scaling factors removed to highlight the

structure of the functions� The equation of the odd derivatives comprise an odd polyno

mial multiplied by the Gaussian function and the even derivatives contain polynomials

in even powers of x�

When considering multidimensional use of these �lters the function s symmetry must be

considered� The �rst derivative of the Gaussian function is an odd function so it must

be applied in particular directions�

The second derivative of the Gaussian function is an even function which when applied

in two dimensions� forms the Laplacian operator� The idea is that zero crossings in the

second derivative correspond with peaks and troughs in the �rst derivative� These would

coincide with edges in the image� A problem with this operator is that it always produces

closed contours�

A recursive implementation for the Laplacian is given by Deriche �� and another in

teresting recursive implementation provided by Jin and Gao ��

Another common Gaussian based image �lter is the di�erence of Gaussians �DoG�� This

�lter uses the fact that taking the di�erence of two suitably normalised Gaussian functions

of di�erent scales �� results in a function which closely approximates the Laplacian

operator� This is usually only used to reduce the level of computational complexity in

an image processing system�

Gaussian based quadrature pairs

First and second derivatives of the Gaussian are almost in quadrature and can be used

as an approximate quadrature pair� These functions were used by Venkatesh to calculate


local energy� Venkatesh �� also based pairs of small masks on these functions to derive ��

� and � point masks which can be used as a surprisingly good local energy based feature

detector� Over such a small scale the fact that these masks are not twodimensional does

not matter very much�

The polynomial nature of the Gaussian function and its derivatives has been utilised by

Freeman �� to numerically derive approximations to Hilbert transforms� and to develop

steerable kernels to implement rotated derivative of Gaussian functions� These techniques

are explored in detail in the following chapter�

�� Wavelets for local energy

The wavelet transform is a tool well suited to local frequency analysis of signals� Wavelet

analysis functions exhibit limited extent in both the spatial and frequency domains�

Their use for local analysis removes the need for separate windowing functions� which

have inherent e�ects on the analysis� Wavelet analysis involves convolution of the signal

with banks of �lters� These �lters are a geometric progression of rescalings of one

waveshape� Each scaling will respond to certain frequencies within the signal� and the

frequency response by each rescaling is geometrically related to the others leading to a

logarithmic frequency scale�

Wavelet theory was initially developed by Morlet �� for geophysical analysis� The

theory has been further developed by Grossman �� Meyer �� and Daubechies ��

Mallat �� in particular has investigated their use for image analysis and compression

where he is interested in signal reconstruction and therefore uses orthogonal wavelets�

He did also use an energy measure derived from his wavelets to develop a compression

scheme�

The wavelet transform provides a mathematical tool for multiresolution analysis that is

ideal for the estimation of local energy� Each �lter will respond to a particular frequency

range in the image� This approach gives good spatial localisation and control over the

frequency range�

Kovesi �� and Pudney et al� �� have developed a wavelet system for phase congruency

and local energy calculation� Using oriented wavelets they have extended this system to

multiple dimensional analysis ��


Morlet wavelets

The Morlet wavelet is a complexvalued wavelet made up of a real cosine function �even�

and an imaginary sine function �odd�� both modulated by a Gaussian� Using the two

functions in quadrature we can determine the amplitude and phase of frequency compo

nents at a given spatial location� The Morlet wavelets are not orthogonal� which means

that the original signal can not be uniquely reconstructed from the wavelet coe�cients�

To calculate local energy from these wavelets it is necessary to obtain F �x� and H�x� by

convolution of the image with a bank of the odd and even Morlet wavelets�

F �x� �X

f�FreqI�x� �M even

f �x��

H�x� �X

f�FreqI�x� �Modd

f �x��

where I�x� is the original image� Meven�oddf �x� is an even or odd wavelet tuned to fre

quency f � Freq is the set of frequencies covered by the wavelets� and � denotes convolu

tion� Convolution is commutative over addition� so that

F �x� � I�x� �X

f�FreqM even

f �x��

H�x� � I�x� �X

f�FreqModd

f �x��

Oriented Energy

The local energy model is extended to multiple dimensions by calculating the one dimen

sional local energy at a number of orientations� evenly spaced to cover all directions� and

then summing the resulting energies� The process of combining oriented onedimensional

local energy has been used to detect features in two and three dimensions�

Pudney together with the present author and others �� present the following implemen

tation for threedimensions� that is nicely generalised for multidimensions� Their �lters

are banks of oriented Morlet wavelets� Each wavelet comprises a quadrature pair of a

onedimensional sine and cosine function oriented to a threedimensional direction� and

modulated by a threedimensional Gaussian� The spatial and frequency domain forms of


�a� �b�

Figure �� A twodimensional odd Morlet wavelet� k � �� and a plot of the magnitude ofits Fourier transform�

an evensymmetric �lter oriented along the unit vector v� and tuned for frequency f are

given by the following�

M evenf��v �x� �

�

�p��

e�kxk�

�� cos��vT � x��

M evenf��v �u� �

�

�

�e��ku�f�vk� � e��kuf�vk�

��

respectively� with

� �k

f� ��

� � ��f� ��

The constant k controls the ratio between the �lter s width � and frequency f � The odd

�lter has a sine in place of the cosine in equation �� and is imaginary� The frequency

domain expression for the odd �lter is the di�erence of the two exponential terms in

equation �� The �lter is rescaled by changing f � Note that the �lters have amplitudes

that are di�erent to those of traditional Morlet wavelets� In equations � and �� the

lower frequency �lters are attenuated compared with Morlet wavelets� This is done to

reduce blurring in the local energy result� Filters based on the traditional form of the

Morlet wavelet have been used previously ��


-1

-0.5

0

0.5

1

�a�

-1

-0.5

0

0.5

1

�b�

Figure �� Even and odd Morlet wavelets� with k � ��

As stated above� local energy in multiple dimensions is computed as the sum of one

dimensional local energy values computed at a number of orientations� The set of orien

tations� Orient� is chosen to provide even coverage of the multidimensional space� That

is�

E�x� �X


where�

E�v �qF ��v�x� �H�

�v�x� ��

F�v�x� � I�x� �Geven�v �x� ��

H�v�x� � I�x� �Godd�v �x� ��

Geven�v �x� �

Xf�Freq

M evenf��v �x� ��

Godd�v �x� �

Xf�Freq

Moddf��v �x� ��

and I�x� is the multidimensional image data set�

These Morlet wavelets are easily constructed in the frequency domain where they can

also be rotated to calculate oriented energy values� A problem is that high frequency

�lters require a large frequency space in order to avoid problems with aliasing�


�� Review of �lter pairs

Several other �lter pairs have been used in the calculation of local energy and in conjunc

tion with the phase congruency model� Appendix A contains a review of the following

�lter pairs�

� and � point masks � Venkatesh showed in her thesis �� that small convolution

masks based on the �rst and second derivatives of the Gaussian function can be

used to create a simple feature detector�

Derivatives of the Gaussian function � Numerous researchers including Canny ��

Deriche �� and Venkatesh �� have used �lters based on the Gaussian function

and its derivatives�

Gabor functions � The Gabor function is formed from a sine wave modulated by a

Gaussian function� This is very easy to construct in the frequency domain and has

been used to calculate local energy� for an example see Pudney et al� ��

Modied Gabor functions � The stretchedGabor functions of Heitger et al� ��

have similar spatial characteristics to the Gabor function but have a zero D�C�

mean which gives a wider bandwidth over which these functions are in quadrature�

Log Gabor functions � Field �� suggests that the log Gabor function more closely

re�ects the frequency response required to analyse natural images�

Perona � Perona �� proposes an approximation technique to generate a steerable and

scalable kernel which approximates the desired function within a given tolerance

with a minimum number of components� He uses an elongated second derivative

of Gaussian with �x � �y ratio of � � �� The kernel uses complex exponentials in �

and is axially symmetric�

Freeman � Freeman �� uses various derivatives of the Gaussian function for his

energy �lters but numerically calculates the Hilbert transform of the �lter to use as

the quadrature pair� These functions are all polynomials in x extended to multiple

dimensions through modulation by a multidimensional Gaussian Ball� The �lters

are steered in the spatial domain using a basis kernel� which is discussed in detail

in Chapter ��


�� Digital �ltering techniques

Computer analysis of images involves processing discretely sampled data� The samples

are taken in multiple dimensions with a �nite quantisation in both location and intensity�

The spatial sampling of the signal should actually be measured in radians or even stera

dians� after consideration of the focal nature of the optics involved� However this is often

ignored and the image simply considered in terms of a regular grid of pixels� The discrete

signal is treated as a traditional time sampled signal for which there is a great deal of

theory� The book by Gabel and Roberts �� is an excellent text for the analysis tools of

linear systems including Fourier theory and discrete system analysis�

The Fourier transform and related analysis tools are very important in image processing

although the concept of frequency of a spatially sampled signal can be somewhat confusing

at �rst� It need not be if it is recognised that frequency refers to periodic components in

the signal� which can be periodic in space as easily as in a time series data set� A large

spatial extent in the image corresponds to a low frequency� thus a low pass �ltered image

is blurred removing small features� Reference to frequency in this thesis is to spatial

periodicity� not the time varying behavior of the signals� Temporal considerations such

as image sequences are not part of this work�

A sampled signal can be fully reconstructed as long as it did not originally contain any

frequencies more than half the sampling frequency� This is called the Nyquist frequency�

Any frequency components that are above the Nyquist frequency will appear in the

sampled data as an aliased lower frequency� Aliasing will not be a problem so long as no

processing introduces frequencies above the Nyquist frequency�

A continuous function f�x� sampled periodically at intervals T forms a series� which is

referred to as a sequence� denoted�

ffkg � ff�� f�T �� f��T �� f�kT �g� ��

In general k can take any integer value and T is the sampling period� which in the image

processing application has dimensions of length�

The sum of two sequences fakg�fbkg is the sequence fckg where ck � ak�bk� The prod

uct fakgfbkg is the sequence fckg where ck � akbk� A discrete linear system is described

in terms of three primitive elements� adders� multipliers and unitdelay elements� A


unitdelay element shifts a sequence by one place so that fakg after a unit delay becomes

the sequence fak��g�

Discrete linear systems can be described by linear di�erence equations in the same way

that continuous systems are discussed in terms of linear di�erential equations� A discrete

system can also be characterised by its impulse response� The impulse response is the

output that results from an impulse input sequence� f�kg where

f�kg �

��

�� k � �

�� k ��

The system s response to input f�kg is the impulse response fhkg� that is�

f�kg �� fhkg� ��

The systems are linear so multiplication by a constant c gives�

cf�kg �� cfhkg� ��

and because of the shift invariance of the linear system a shift in the input sequence

simply shifts the output� so

cf�k�ng �� cfhk�ng� ��

The response to an arbitrary input sequence fukg is the sum of impulse responses scaled

and delayed to correspond with fukg� Since

fukg ��X

j��ujf�k�jg� ��

the output response fykg is

fykg ��X

j��ujfhk�jg� ��

The sum in equation �� is known as a convolution sum� Convolution is a commutative

and linear operation� The shorthand notation used for convolution is

fykg � fukg � fhkg� ��

In signal processing we are interested in passing the signal through some �lter in order

to accentuate or suppress speci�c components of the input signal� We are therefore


interested in the transfer characteristics of the �lter� In fact the �lter is speci�cally

designed to exhibit particular transfer characteristics�

While the impulse response is a useful analytical tool for linear systems� �lter characteris

tics are often better described in terms of frequencies� The Fourier transform transforms

a description of the �lter s response between the frequency and spatial domains�

The Fourier transform decomposes the transfer function �or impulse response� into its

constituent frequency components� each frequency of which will have an associated value

for magnitude and phase� usually expressed as a complex number� If the signal analysis is

performed analytically the transfer function will be a continuous function and the Fourier

transform of this will also be a continuous function�

A discrete analysis will entail the use of the Discrete Fourier transform �DFT�� in which

case the resulting frequency response will also be discrete with frequencies ranging from

� to � the Nyquist frequency in increments of ��N where N is the number of samples�

The discrete Fourier transform treats the signal as periodic� repeating before and after

the actual sample window� Frequencies above the Nyquist frequency will spill over into

the repeated frequency sequence as aliased frequency components�

For example� if the transfer characteristic is an ideal rectangular pulse the frequency

response will in�nitely decay with both positive and negative frequencies� This means

that in practice the sample must be su�ciently wide that e�ects such as sharp transitions

do not cause signi�cant aliasing�

Convolution of an image and a �lter can be performed in either the frequency or spatial

domains� Each method has advantages and disadvantages�

�� Frequency domain convolution

Under the Fourier transformation the operation of convolution becomes multiplication�

So a convolution can be performed by taking the Fourier transform of both functions�

multiplying them together and taking the inverse Fourier transform of the result� Compu

tationally� multiplication is much more simple to perform than convolution which raises

the question as to the computational complexity of the Fourier transform and its inverse�

The Discrete Fourier Transform itself takes order O�N�� operations� Fast Fourier trans

form �FFT� techniques �� make the transform to and from the frequency domain more


e�cient� An FFT takes order O�N logN� operations� For large data sets� this can be a

signi�cant saving� The Fourier transform is separable so that a multidimension Fourier

transform is computed as successive transforms in each of the orthogonal dimensions� An

FFT in twodimensions will take O�N� logN� operations�

For example the twodimensional convolution of an M �M �lter with an N � N data

set can be performed as�

fhijg � fuijg � FFT��FFT �fhijg�FFT �fuijg��

which will require O�M� logM � �N� logN� operations as compared to the O�M�N��

operations required for a convolution by direct implementation of equation ��

Filters are often speci�ed in terms of a frequency response which makes their design in

frequency space straight forward� The required frequency response is constructed and

multiplied by the Fourier transform of the data to be �ltered�

The Fourier transform of a data set that has been rotated in multiple dimensions is the

Fourier transform of the unrotated data set� rotated by the same amount� An oriented

�lter is thus easily constructed by rotating its frequency domain representation�

Discrete Fourier techniques are performed on sampled data and the �nite sample length

imposes an implied periodicity on the signal� The signal is treated as if it was repeated

before and after the sample� We can see that a discontinuity between the last and �rst

sampled data values could cause extraneous components to be apparent in the transform�

A sample set is usually tapered at the beginning and end to remove these e�ects� Such

a windowing function must be chosen to minimise any further introduction of artifacts�

Alternatively a step discontinuity to zero can be used at the border of the sample set and

a border of su�cient width left to avoid aliasing and introduced artifacts� The width of

the border is dependant on the spatial and frequency extent of the processing �lters�

The FFT algorithm gets its e�ciency by recursively combining the results of two Fourier

transforms each of half the data� This factoring by two means that it works best with

data sets whose size is a power of two� Modi�cations to handle arbitrarily sized data

are available but are not often used� The data set is usually padded out with zeros to a

suitable size� The need to pad the sizes to powers of two is a disadvantage�

The pattern used to combine the partial results� often referred to as bit reversal� leads to


an almost pathological case for the communications requirements in parallel implemen

tations of the algorithm� It is di�cult to parallelise� and quite complicated to code in

general� Most programs that implement the FFT are based on published code such as

the �Numerical Recipes book ��

The Fast Fourier transform is a good method of performing digital convolution as long

as care is taken with regard to aliasing and data windowing� It is such a useful data

analysis tool that most digital signal processors have special hardware instructions that

facilitate the coding of FFT algorithms�

In an appendix to his thesis� Robbins �� points out that the two convolutions required

to calculate local energy can be performed in one operation which utilises the quadrature

relationship of the �lters�

Also� when considering the convolutions for local energy we note that a spatially narrow

�lter� that will detect �ne features� has a large frequency domain representation and the

multiplication will need to be done over a large area of the frequency domain� In such

a case� a spatial convolution may be more e�cient� With a large frequency spread more

care must be taken of signi�cant �lter components at the Nyquist frequency� which is set

by the sampling of the data set�

�� Spatial domain convolution

Convolution by large masks� particularly in multiple dimensions can be quite computa

tionally expensive� and is usually cited as the reason for the use of frequency domain

techniques� By using separable �lters and recursive �lters �IIR� signi�cant savings can

be made�

Deriche showed that his scalable recursive �lters could be designed to convolve quite

wide �lters with a �xed number of operations per point� A further advantage of spatial

domain �ltering comes from reduced storage requirements� particularly when dealing with

three dimensions� An ability to process data locally is another aspect of spatial domain

convolution that is particularly useful for active vision systems� adaptive processing or

parallel decomposition of algorithms�


Discrete convolution

The direct computer implementation of the convolution sum� equation �� of a signal

uk by a transfer function hj where uk contains N samples and hj contains M sam

ples� involves a pair of nested loops and N � M operations� If these data sets are

multidimensional the number of operations increases exponentially� Three dimensional

convolutions of images are almost untenable even on modern computers�

The direct implementation of the convolution sum is referred to as the Finite Impulse

Response �FIR� form of the digital �lter� so called because the response sequence must

be �nite for the result to be calculable�

Recursive lters

An alternate form of digital �lter is constructed when the output sequence is expressed

as a weighted sum of not only the input sequence but also the previous output results�

Not all transfer functions can be expressed in this form but it does lead to a situation

where an in�nite response can be obtained from an impulse excitation� Hence the name

In�nite Impulse Response �IIR� �lters� although they are also referred to as recursive

�lters�

The dependence on previous results imposes a causality requirement on the �lter� which is

reasonable for physical systems� Also� for the system to be stable a diminishing response

is necessary� but again most physical systems have some degree of damping which causes

a diminishing response�

The task of parameterising a transfer function in terms of a recursive �lter can be di�cult

but techniques are available� One such technique is explored in Chapter � for �lters based

on the derivatives of the Gaussian function and the resulting IIR �lters are applied to

local energy�

So long as the �lter is suitably oscillatory and diminishing in nature a recursive �lter

can often be designed with a surprisingly small number of parameters� The number of

weighted terms is referred to as the order of the �lter� For example the �lters designed

in Chapter � use fourth order �lters to apply a derivative of the Gaussian based function

at various scales� ranging over two orders of magnitude� from � � � to over ��


Separable lters

Multidimensional convolutions that can be expressed as a product of terms� each in a

single variable� can be performed as a series of onedimensional convolutions� For exam

ple� a separable convolution of a twodimensional image data set could be performed as a

onedimensional convolution in x down each column followed by a rowwise convolution

in y by the intermediate result of the �rst convolution�

The computation required to perform a twodimensional convolution by a separable func

tion is greatly reduced from that required for the direct convolution�

�� Conclusion

This chapter has introduced the local energy model of feature detection and investigated

various aspects of its calculation� The phase congruency characteristics of image features

have been examined together with a consideration of quadrature pairs of �lters such as

those based on the derivative of the Gaussian and Morlet wavelets� that can be used

to calculate local energy� Digital �ltering has been discussed and the computational

complexity of the competing frequency and spatial domain convolution techniques has

been reported�

In the following chapter� the derivative of the Gaussian function is examined in more

detail and a method of calculating local energy using steerable and recursive digital

�lters is developed� This method incorporates oriented energy and calculation across

multiple scales� It can be generalised to multiple dimensions and this is illustrated for

two and threedimensional data sets�

The �lters designed in the next chapter are powerful new tools and can be applied directly

as a means of calculating local energy� No further derivation or adaptation is necessary�

The resulting �lters are then used in subsequent chapters as the means of calculating

local energy for multiple dimension feature detection�

Chapter �

Recursive Filters for Oriented

Local Energy

�� Introduction

Local energy is estimated by convolving a signal with a pair of suitable �� lters that are

in quadrature with one another� This convolution of digital signals can be accomplished

in the spatial domain either directly or by using recursive techniques� When used with

multidimensional signals� local energy requires the use of oriented multidimensional

�lters� This chapter investigates the design of a class of steerable �lters that are suitable

to calculate local energy� and their implementation using a recursive digital �ltering

technique� The steerable �lters are also separable which means that convolution can

be performed as separate onedimensional convolutions in each of the variables� This

greatly reduces the number of arithmetic operations required when compared with a

direct multidimensional convolution�

Recursive �lter implementations give the ability to apply an arbitrary sized �lter with a

small� and �xed� number of operations per point� There are a number of restrictions and

requirements on �lters that can be implemented in this way and these will be examined

below� The �lter design used in this thesis has a parameter� �� which determines the

scale or size of the resulting �lter� this is independent of the number of operations per

point�

Spatial domain �ltering opens the way to partial processing of regions of an image and

��

CHAPTER �� RECURSIVE FILTERS FOR ORIENTED LOCAL ENERGY ��

avoids problems with frequency aliasing that can occur when applying high frequency

�lters in the frequency domain�

Filter design is somewhat restricted when using recursive �lters but suitable �lters can be

synthesised� We will use �lters akin to Freeman s steerable �lters �� to produce oriented

�lters with which to estimate oriented local energy� We follow Deriche s numerical design

procedures �� to determine suitable recursive approximations of the required �lters�

The resulting �lters are steerable� separable� scalable and suitable to calculate oriented

local energy�

�� Design of In�nite Impulse Response Filters

Recursive �lter implementations are derived from the di�erence equation description of

a system transfer function as described below in Section �� Di�erence equations

themselves are closely related to the ztransform of the function but unfortunately the

ztransform of many functions is not easily determined� One solution to this problem

is to �t the required transfer characteristic with an approximation function for which a

closed form of the ztransform is known� A recursive �lter can then be synthesised which

has the required characteristics�

Deriche �� uses this technique to derive recursive implementations of the Gaussian

function and its derivatives� functions which are commonly used in image processing�

The approximation function �to be shown later as equation �� contains a parameter�

�� which determines the scale of the Gaussian response� This parameter carries straight

through into the analytic form of the ztransform and is applied independently to the

approximation �tting procedure� The ztransform is described in most signal analysis

texts such as Bracewell �� or Stark and Tuteur ��

Our approximation function is a sum of complex exponentials for which the literature

dates back to work by Prony in �� The problem is di�cult due to the �at nature of

the solution space but the application of numerical methods and nonlinear leastsquares

techniques enable a solution to be determined�

�This work is described by Lanzcos �� pp � � �� and Hildebrand �� pp��


�� The z�transform

The ztransform and the Discrete Fourier Transform �DFT� are closely related and share

many properties� Like the DFT and Fast Fourier Transform �FFT�� which are principally

used for numerical computation of Fourier transforms� the ztransform is an analytical

tool used in the analysis of discrete systems� The discrete real frequency variable k of

the DFT is replaced in the ztransform by a continuous complex variable z� The DFT

can be regarded as a weighted average of the input data� while the ztransform may be

regarded as a resolution of the input data into signals of the form z�n� where z is a

complex variable� The onesided ztransform of a discrete signal x�n� is de�ned by

Z��x�n�� X��z� ��Xn��

x�n�zn� ��

while the twosided ztransform is de�ned by

Z��x�n�� X��z� ��X

n��x�n�zn� ��

Here we consider only the onesided ztransform therefore expressions such as X�z� or

Z�x�n�� all refer to onesided ztransforms �

The variable z is an arbitrary complex variable� but for the ztransform to exist� z must

lie inside the region of convergence that depends on the particular sequence x�n� being

transformed� If x�n� has nonzero values only over some �nite range of values of n� the

summation in equation �� converges for all z� Also if x�n� � � for n � � and is otherwise

reasonably well behaved� the series X�z� converges if jzj is larger than some radius of

convergence r� �

One of the main reasons for using the ztransform is that it possesses the convolution

multiplication property of all Fouriertype transformations� This permits the replacement

of the relatively complex procedure of convolution by the much simpler operation of

multiplication� The convolution of two discrete signals� assuming x�m� � � for m � ��

can be written as

�For the one�sided z�transform the conditions for convergence are �� jx�n�j �� for all �nite n� and� � jx�n�j � kr

n if n � N � for some �nite k� r and N � The smallest value of r for which this inequalityis satis�ed is the radius of convergence�

CHAPTER �� RECURSIVE FILTERS FOR ORIENTED LOCAL ENERGY �

y�n� �nX

m��

x�m�h�n�m��

where the transfer function h�� is a causal response� Taking the ztransform of both

sides of equation �� gives

Y �z� � X�z�H�z��

where

H�z� ��Xk��

h�k�z�k� ��

The usefulness of the ztransform depends in part on the possibility of �nding simple

closedform expressions for commonly used sequences x�n� and h�n�� Then the resulting

Y �z� can be analysed or the equation inverted to study the properties of the response�

In this respect the ztransform di�ers from the DFT� where closedform expressions are

mostly irrelevant� The ztransforms of some common functions are examined in texts�

including Stark and Tuteur �� pp��

In a linear shiftinvariant system� the linearity implies that the output sequence y�n�

can be expressed as a weighted sum of the input sequence x�n�� and the shift invariance

implies an input sequence x�n � n�� produces output y�n � n�� The transfer equation

is then the discrete convolution

y�n� ��X

m��x�m�h�n�m��

where h�n� is the discrete impulse response�

�� Recursive Structures

Linear shiftinvariant systems can be divided into recursive and nonrecursive structures�

A recursive system is one in which the current output value depends on preceding values

of both the output and input� Recursive linear shiftinvariant systems are describable by

linear constantcoe�cient di�erence equations of the form


y�n� �kX

m��

amx�n�m��kX

m��

bmy�n�m��

This is a fairly direct analog of ordinary continuous �lters that are described by linear

constant di�erential equations� and their properties are similar� Their frequency response

�actually the ztransform � has poles and zeros� and the response will be unstable if the

poles lie outside the unit circle� The location of the poles determines not only the stability

but also the inherent character of the transient response of the system�

Recursive �lters are frequently referred to as IIR �in�nite impulse response� systems to

distinguish them from nonrecursive FIR ��nite impulse response� systems whose impulse

response is �nite by design and equal to the number of delay elements employed in their

construction� Note that in practice the impulse response time of stable IIR �lters is in

fact also �nite because of �nitewordsize e�ects and the converging �decaying� nature of

the �lter�

There are a number of standard realisations for IIR systems� A realisation that imple

ments equation �� directly is the so called direct form � realisation� in which separate

delay elements are used to store previous inputs and outputs�

Figure �� shows a thirdorder� direct form � realisation� It is called a thirdorder system

because it realises a thirdorder di�erence equation in y�n�� i�e� the present value of y�n�

depends on y�n� �� y�n� �� and y�n� ��

It is easy to derive H�z� from the direct form � realisation by rewriting equation �� as

kXm��

amx�n�m� �kX

m��

bmy�n�m��

where b� � �� The ztransform then yields

X�z�kX

m��

amz�m � Y �z�

kXm��

bmz�m� ��

or� equivalently �using equation ��

H�z� �Y �z�

X�z��

Pkm�� amz

�mPkm�� bmz

�m ��

where the am and bm are the gain parameters in the direct realisation�


Σ

-1z -1z-1z

-1z -1z-1z

x(n)

a a a

0a

1 2 3

2 1

y(n)

-b -b -b3

H�z� �a� � a�z � a�z

� � � � �� anzn

� � b�z � b�z� � � � �� bmzm

Figure �� Example of a direct form � realisation of a thirdorder recursive digital �lter�The coe�cients for a recursive �lter are determined from the ztransform of the transferfunction�

Thus we can see the relationship between the ztransform of the transfer function and

its recursive implementation� The coe�cients for a recursive �lter are determined from

the ztransform of the desired transfer function� When the transfer function is written

in the normalised form�

H�z� �a� � a�z

�� a�z�� anz

�n

� � b�z�� b�z�� bmz�m� ��

the coe�cients ai form the nonrecursive component and bi the recursive component of

the �lter�

The recursive system described by equation �� is a causal system� i�e� it has no negative

time response� Our �lters exhibit symmetry about the origin and are in general a non

causal sequence� The solution is to split the impulse response h�k� into two parts h�k�

and h��k� such that h�k� � h�k� � h��k� where�


h�k� �

��

h�k� k � �

� for k � ��

h��k� �

��

� k � �

h�k� for k � ��

h�k� and h��k� are causal and anticausal� The ztransforms corresponding to these

impulse responses are

H�z� �a� � a� z

�� a� z�� an z

��n��

� � b� z�� b� z

�� bmz�m� ��

H��z� �a�� z � a�� z

� � � � �� a�n zn

� � b�� z � b�� z� � � � �� b�mzm� ��

For symmetric� even� �lters the coe�cients are related by the following equations�

b�i � bi i � ��

a�i � ai � bi a� i � ��

a�i � �bi a� i � �

��

and for antisymmetric� odd� �lters�

b�i � bi i � ��

a�i � ��ai � bi a� � i � ��

a�i � bi a� i � ��

��

Since we have the entire image data before we start processing� it can be processed in both

directions and the results combined� This is not necessarily the case in some realtime

�lter applications where the �lter is by necessity causal�

�� Design by approximation

Determination of the ztransform from an arbitrary function is not necessarily an analyt

ically tractable problem� To circumvent this problem we follow Deriche s approach and

approximate our �lter with a function for which the ztransform is known�


Deriche �� proposes the approximation function

ha�k� �nXi��

ie��i k� ��

using n � � and constraining i and i to be complex but conjugate in order to get real

coe�cients� That is�

� � ��

and then by substitution and application of Euler s rule �� ha can be written

ha�k� � �a� cos��

�k� � a� sin�

��

�k��e�

b��k � �c� cos�

��

�k� � c� sin�

��

�k��e�

b��k� ��

The ztransform of this function can be determined from the following ztransform as

tabulated in Bracewell ��

z�transformf�x� F �z�

e��n cos�n ��z��e�� cos��z��e�� cos�z��e��

e��n sin�n ��z��e�� sin��z��e�� cos�z��e��

Collecting the terms it can be rewritten in the form of equation �� thereby determining

the values for the recursive �lter coe�cients� The algebra involved is straightforward�

although �ddly� and we get�

�Euler�s rule �� ei� � cos � � i sin �


n� � a� � c�

n� � �c� sin�� c� � �a�� cos�

�� e�

b�� a� sin�

�� c� � a�� cos�

�� e�

b��

n� � ��a� � c�� cos�� cos�� a� cos�

�� sin�� c� cos�

�� sin�� e�

b�� b�

�

�c�e��

b�� a�e

��b��

n� � �c� sin�� c� cos�

�� e�

b��

b�� a� sin�

�� a� cos�

�� e�

b��

b��

d� � �� cos�� e�b�� cos�� e�

b��

d� � � cos�� cos�� e�b�� b�

� � e��b�� e��

b��

d� � �� cos�� e�b��

b�� cos�� e�

b��

b��

d� � e��b��

b��

��

Note that we have changed the notation slightly to use ni �numerator� and di �denomina

tor� rather than ai and bi� in order to avoid confusion with the parameters of equation ��

The anticausal coe�cients are determined from these values� as shown above in sec

tion �� and depend on the symmetry of the function�

With a little care� these equations are easily coded into the �lter initialisation subroutine

of a recursive �lter computer program� The program will then be able to implement a

�lter characterised by a�� a�� b�� c�� c�� and b� for some given value of ��

It is worth noting that our functions have a Gaussian envelope which gives a range of

validity� That is� if we wish to apply the �lter to a region of the image� a border of

su�cient width� say �� must be processed around the region before the �lter values

will be valid at the boundary of the region� When processing the entire image we treat

external pixel values as zero which will often mean a step function exists at the boundary�

This is preferable to the frequency domain e�ect of wrapping the image around in a

periodic fashion forming a toroid�

�� Numerical Approximation

Fitting a general function with a sum of exponentials is a nontrivial task� made all the

more di�cult by the fact that the basis functions are extremely nonorthogonal �� This

means that there are many equivalent approximations and that the �tting is extremely

sensitive to initial conditions�


The procedure for approximating the �lter is outlined by Deriche �� and will be pre

sented again here with more detail than was contained in his paper� It involves two

stages� �rst a straight forward method �Prony s method� which produces a poor result

that is then used as initial conditions for a more exact iterative �tting procedure�

Prony�s method

Prony s method �� is a technique for determining the parameters of exponential!sinusoidal

signals� which are supposedly contained in a sequence of noiseless signal samples� That

is�

xn �pX

k��

Akeskn� ��

The problem Prony addressed was to determine the unknown Ak s and sk s from the N

given samples� Since there are �p complexvalued unknowns in equation �� at least �p

samples of xn are needed to determine them� Prony observed that the determination of

the Ak s and sk s can be decoupled and that they can be determined by solving two sets

of linear simultaneous equations and �nding the roots of a polynomial�

In order to approximate a set of data points fykg� of a function f�x� given at equidistant

ordinates xr � x� � rh� with a sum of exponentials of the form

f�x� � �e��x � �e

��x � �� me��mx� ��

we �rst substitute vi � e��ih and cr � re��rx� � Then expanding equation � at x�� rh

for r � �� m gives the following system of equations�

��

c� � c� � �� cm � y�

c�v� � c�v� � �� cmvm � y�

c�v�� c�v�

� � �� cmvm� � y�

��

c�v�m � c�v�

m � �� cmvmm � ym

��

This system of equations is known as the problem of weighted moments and the solution

of this system can be found by constructing the fundamental polynomial �� p��


Fm�v� � �v � v��v � v�� v � vm� � vm � s�vm�� sm� ��

The roots of this polynomial are the values vr which can easily be determined numerically

once we have determined the values of sr� To �nd these coe�cients� s�� sr� we

multiply the �rst equation in �� by sr� the second equation by sr�� the rth equation

by s� and the �r � ��th equation by �� then add the results� If use is made of the fact

that each v satis�es equation �� the result is seen to be of the form�

ym � s�ym�� sry� � ��

A set of N � n� � additional equations of similar type is obtained in the same way by

starting instead successively with the second� third� � � � � �N � n�th equations� In this

way we �nd that equations �� and �� imply the N � n linear equations

��

yn��s� � yn��s� � �� y�sn � yn

yns� � yn��s� � �� y�sn � yn�

��

yN��s� � yN��s� � �� yN�n��sn � yN��

��

Since the ordinates of yk are known� this set can be solved directly for the m values of s�

if N � �n� or solved approximately� by the method of least squares� if N � �n�

After the sr s are determined� the n v s are found as roots of equation �� They may be

real or complex� The equations �� then become linear equations in m c s� with known

coe�cients� The c s can be determined from the �rst m of these equations� or preferably�

by least squares from the entire set�

Iterative approximation

Once an approximate solution has been found it can be improved using an iterative tech

nique� Many such techniques have been developed as computer algorithms� They seem to

di�er in the sophistication of the convergence technique� swapping between various meth

ods to avoid local minima in the solution space� For problems such as the exponential

expansion� the solution space is very �at with many equivalent solutions�


The approximation technique chosen here is the Marquardt Method� presented in �Nu

merical Recipes in C �� This was chosen in place of the lnag library routine used by

Deriche as it was freely available and su�cient for the task�

The non�linear least�squares problem

The problem of �tting a parameterised model to a set of data points is approached as a

minimisation� minimising the squared di�erence between the model and the data ��

The nonlinear least squares problem is one that uses an iterative numerical solution to

�t a parameterised function to a set of ordinates xi� Least squares is a scalar measure of

the �t that is to be minimised� i�e� we minimise f�� with respect to �� where the function

f�� is a scalar function of the vector of parameters� �� which we wish to estimate�

An iterative descent algorithm for estimating � can be expressed as

�i� � �i � ipi ��

where �i� is the result of the ith iteration� pi is the direction of search and i is a

positive scaling factor that can be used to adjust the step size� For socalled gradient

methods the direction of search� pi� can be expressed in terms of the gradient of f�� at

� � �i as

pi � �Rirf��i��

where Ri is an N �N matrix and rf��i� is the gradient of f�� at � � �i given by

rf��i� ��f��

��f��

��

�f��

��N

�T��i

� ��

From equations �� and ��

�i� � �i � iRirf��i��

It is desirable that Ri be positive de�nite� Then if rf��i� �� it is guaranteed that

f��i�� f��i� for su�ciently small i�


Three basic variations on this scheme are illustrated by Lim �� the steepest descent

method� the NewtonRaphson �NR� method and the DavidonFletcherPowell �DFP�

method� These methods can be expressed in the form of equation �� but di�er in how

i and Ri are speci�ed�

The steepest descent method is the simplest technique which uses Ri � I� where I is the

identity matrix� so

�i� � �i � irf��i��

The step size of i is chosen carefully so as not to overshoot or oscillate around the

solution but not so small that convergence is too slow� Many reasonable methods of

choosing i exist�

The steepest descent method relies on the �rst order derivative and so is simpler to

compute than the NR and DFP methods which require second order derivatives� This is

e�ective in decreasing the function in the early stage of the algorithm but converges only

linearly and can be slow as the iterative solution approaches the true solution� Steepest

descent always searches in the direction of negative gradient and may have di�culties

in the presence of local extrema in f�� Consequently� this method is often used only

in the initial stages and is replaced by another method� with better convergence� as the

solution is approached�

The NewtonRaphson method is based on the Taylor series expansion of f�� at � � �i

where the second order term is determined by the Hessian matrix Hf �� given by

Hf ��

�BBBB

��f��

� � � ��f��N

��

��f��N��

� � � ��f��N��N

CCCCA �

Then �i� is chosen as

�i� � �i �H��f ��i�rf��i��

Comparison of equations �� and � shows iRi is chosen to be H��f ��i�� In this case Ri

need not be positive de�nite and the algorithm may not converge if the iterative solution

�i is not su�ciently close to the true solution�


The DavidonFletcherPowell method is in a sense a combination of the steepest descent

and NR methods� Initially it behaves as a steepest descent and later like the NR method�

The DFP method updates �i� from �i by

�i� � �i � iRirf��i��

where Ri is also updated recursively� The idea is to recursively build Ri into an approx

imation to the inverse Hessian matrix using the �rst order derivatives of f�� R� is

usually chosen to be the identity matrix I� Refer to Fletcher and Powell �� or Lin ��

pp �� for details�

�� Approximate IIR �lters and local energy

The preceding sections have shown how to design a recursive IIR �lter by using a non

linear least squares �tting algorithm to approximate the desired �lter with one for which

the ztransform can be determined in an analytic form� The coe�cients of the recursive

�lter are determined directly from this expression of the ztransform � In the rest of this

chapter we use this design technique to implement the pairs of �lters that are used to

calculate local energy�

We follow Freeman s �� derivation of a set of separable basis functions to develop

steerable multidimensional �lters that can be used to estimate local energy� Using the

steerability� we calculate several oriented energy values which are summed to form the

local energy estimate� Each basis function is implemented as a recursive �lter which

results in a recursive spatial domain technique for the calculation of local energy�

�� Freeman�s Filters

Freeman describes the theory required to steer a function� f�x� y�� to an arbitrary angle�

�� using a set of basis functions� Furthermore� a set of separable basis functions is derived

which facilitates the calculation of numerical convolution by the function� His work is

reviewed below�

Freeman examines the constraints on functions that can be steered with a basis set formed

by rotated versions of the function itself and shows that a function of the form�


f ��x� y� � G�r�QN �x��

where G�r� is a radially symmetric function and QN �x�� is a polynomial of order N in

one variable� can be written as a weighted sum of N � � basis functions�

f ��x� y� �NXj��

kj��f�j �x� y��

and that an equivalent basis set can be determined where each basis function is separable�

The variable x� is derived by rotating the axis of abscissa� by the angle �� i�e� x� �

x cos � � y sin ��

For local energy processing where the Hilbert transfom of the Gaussian and its derivatives

are of interest these Hilbert transform functions are numerically approximated with a

polynomial of appropriate degree� In contrast to the approximation by a sum of non

orthogonal exponentials� as discussed above� a polynomial �t is easily achieved using a

linear �tting procedure such as singular value decomposition�

�� Separability

A separable function can be expressed as the product of single variable terms� Since

convolution is commutative over multiplication� convolution by a separable function can

be performed as successive convolutions in each of the separate variables� For example a

separable function� such as G�x� y� � e�x��y�

�� can be written as a product of two single

variable functions� e�g� G�x� y� � e�x�

�� e�y�

�� g�x��g�y�� The convolution f � g�x� y� canbe performed as �f � g�x�� g�y��

Convolution is distributive over addition� so that convolution of a function that is the

sum of separable terms can also be performed as a series of separate convolutions with

the results summed�

Since direct spatial domain convolution involves n�m operations per point� separability

considerably reduces this to n�m operations� e�ectively from n� to �n operations� Each

convolution by a single variable term can be implemented recursively to further reduce

computation�

�The axis of abscissa� �� is generally the x�axis in the equations in this thesis�


�� Basis functions separable in x and y

A minimal basis set may not necessarily be x y separable but it may be possible to derive

an equivalent separable basis set� If the basis set can be rewritten in terms of separable

functions then�

f ��x� y� � G�r�NXj��

kj��Rj�x�Sj�y��

The interpolation functions kj�� are found by equating the highest order products of x

and y in equation � with those of equation �� i�e�� equating the coe�cients of x�N�j�yj

for � j N � Substituting x� in equation �� the �x��N term in QN �x�� will give rise to

N � � di�erent products of x and y of order N since

�x��N �NXj��

��j�B N

j

CA cos�N�j�� sinj��x�N�j�yj��

Each basis function Rj�x�Sj�y� would be a polynomial in x and y of higher order than

N � Therefore�

Rj�x�Sj�y� � c �x�N�j� � � � � ��yj � � � � ��

where c is a constant� The coe�cient of the highest order terms x�N�j�yj in equation �

is kj�� Using equation � in equation � gives those same coe�cients in terms of sines

and cosines� Equating the two gives

kj�� j�B N

j

CA cos�N�j�� sinj��

Steering equation � for the separable basis functions gives the separable basis func

tions Rj�x�Sj�y� as a linear combination of the original �lter f ��x� y� rotated to speci�c

orientations �j� That is

�BBBBBBB

f ��x� y�

f ��x� y��

f �N �x� y�

CCCCCCCA

� G�r�

�BBBBBBB

k�� k�� kN ��

k�� k�� kN ��

��

k��N � k��N � � � � kN ��N �

CCCCCCCA

�BBBBBBB

R��x�S��y�

R��x�S��y��

RN �x�SN �y�

CCCCCCCA� ��


The Rj�x�Sj�y� can be written as a linear combination of the f �j �x� y� by inverting the

matrix of k�� s on the righthand side of equation ��

�� Third derivative of Gaussian

Freeman presents basis functions for the second and fourth derivative of the Gaussian�

modulated in the orthogonal direction by a Gaussian� In a technical report �� I have

presented the calculations involved in determining a set of separable basis functions for

the third derivative of a Gaussian� G�� and its approximate Hilbert transform� H��

The energy calculations used throughout this thesis are performed using the second

derivative of the Gaussian function� G�� and its Hilbert transform� H�� rather than G�

and H� because although the general characteristics of the functions G� and H�� and

the functions H� and G� are similar� the degree of the polynomials used in Freeman s

method is higher for G� and H� thus requiring more basis functions� This is the reason

Freeman only presents results for even derivatives of the Gaussian function�

�� Steerable quadrature pairs

The local energy calculation uses a pair of �lters that are in quadrature with one another�

Starting with a derivative of the Gaussian function we �t a polynomial to its Hilbert

transform� Using Freeman s technique� we determine a set of separable basis functions

to create a steerable �lter that is in quadrature to the initial �lter�

We calculate the Hilbert transform of the function f�x� y� along the axis of abscissa by

numerical convolution with ��x � This convolution can be performed either in the spatial or

frequency domain� In the frequency domain it is performed by multiplication by i sgn�s��

Conversion to and from the frequency domain is performed using an FFT routine�

The numerical Hilbert transform data series is then �tted to an even or odd polynomial

of order one greater than� and of opposite parity to� the original polynomial QN �x��

Fitting a polynomial is a straight forward leastsquares problem� solved using the singular

value decomposition technique� Details of the computer code used are presented in

Appendix B��

For example� using this procedure to �nd a function� H��x�� that will form a quadrature

pair with the second derivative of the Gaussian function G��x� � �� x��e�x� � we �t a


third order polynomial to get the function�

H��x� � ��x � x��e�x��

Note that the scale and sign of these functions is not crucial because they will be nor

malised and the results of the convolutions with them are squared then summed�

�� L� normalisation

For the �lters to be in quadrature the L� norm must be the same� To simplify the

numbers in the calculation of basis sets in this thesis� most of the scaling constants have

been dropped from the functions� We normalise all the �lter designs to have an L� norm

of � so that the �lters can be used as quadrature pairs� This normalisation factor cf can

then be applied to the parameters a and c of the recursive �lter design �see equation ��

The L� norm of a function f in two variables x and y is given by�

kf�x� y�k� �

sZ �

��

Z �

��f��x� y� dx dy� ��

Since the �lters we are dealing with are separable� i�e� f�x� y� � R�x�S�y�� we can

separate this integral� then

cf ��pkR�x�k�kS�y�k�

� ��

Because the �lters used here are formed from a polynomial multiplied by a separable and

radially symmetric Gaussian function� and only a rotational transformation is applied�

part of the normalisation factor is always�

ke�y�k� � ��

For example�

kG��x�k� �

Z �

��

�

�� x��e�x

�� dx ��

� ��

so the normalisation factor is�

cG� ��p

��

� ��

Similarly the normalisation factor for H� is�

cH� � ��

Both of the factors agree with the values applied by Freeman �� Appendix G� Table II��


�� Eect of scaling

As was discussed in chapter �� the size of the �lters determines the bandwidth of spatial

frequencies to which the local energy feature detector is sensitive� When combining the

various local energy components calculated at di�erent scales� the e�ect on the norms of

the scaling must be taken into account�

With regard to these �lter designs� if the x and y axes are scaled by a factor of � the only

change will be to the normalisation� Expanding the scale of one axis by � will increase

the L� norm by �� scaling two axes results in a factor of �� change in the L� norm�

�� Steerable scalable recursive �lters

Having seen how to design a scalable recursive �lter using the method described by

Deriche� and how to derive a separable kernel for a steerable �lter� as described by

Freeman� these two results can be combined to form steerable scalable recursive �lters�

Derivation of a recursive implementation of each of the basis functions gives a method

to apply a steerable �lter at any chosen orientation� Since the recursive �lter design is

scalable� the scale of the �lter can be altered without introducing any extra computational

complexity� Each of the basis functions in the kernel is arranged to be separable so the

implementation is quite e�cient�

Consider the implementation of the G� �lter� The steerable kernel for this function

contains three separable basis functions� shown later by equation �� each of which can

be applied using two single variable recursive �lters� Deriche �� presents results for

�nd� �rd and �th order �lters� however since in the application here� the computational

e�ciency of the recursive �lters is not crucial� I have chosen to use �th order recursive

�lters but lower order �lters may su�ce in many applications�

The main drawback of this separable �lter implementation scheme is that the second

convolution is done on the transpose of the intermediate result� This makes incremental

processing di�cult� The intermediate result is the same size as the initial image�

There is virtually no way around this� The image could possibly be broken into smaller

regions that are then recombined after being processed separately� Overlapping borders

su�ciently large for the �lter response to decay away would need to be allowed between


the regions� For memory e�ciency this scheme may not really be bene�cial but in terms

of parallel processing this is how the problem could be split� In modern computers

memory requirements may not be a problem but when we consider threedimensional

volumetric data the memory problem returns�

One advantage of a recursive structure is that the �lters themselves take insigni�cant

space� regardless of their spatial extent� This contrasts starkly with a Fourier domain

implementation where a spatially small high frequency �lter will occupy signi�cant space

in the Fourier domain� In a Fourier domain implementation oriented copies of each �lter

are usually held to provide for the many oriented convolutions that are required�

Assuming that memory constraints are not a problem� a straightforward implementation

is to convolve the image with each of the four x basis functions� transpose the intermediate

results then convolve with the y component of the four basis functions� A steered result

can then be calculated by a simple linear combination� see equation �� of the four values

from the basis images� If we require multiple orientations� as for local energy� each

orientation is simply calculated as a di�erent weighted sum of these same four values�

Freeman used the ability to calculate easily the response at various orientations� to char

acterise junctions and other features by examining the �lter response as orientation was

varied� His analysis was only applied at speci�cally chosen locations in the image�

In the remainder of this section the technique is applied to derive recursive implemen

tations of the G� and H� functions� These results are then used throughout this thesis

when twodimensional local energy is calculated� The results tabulated in Table � can

be used in computer programs for local energy �lter calculations at various scales�

�� Recursive G� implementations

Using Freeman s procedure as detailed in Robins �� the three basis functions for the

second derivative of the Gaussian� G�� function are

G��x� y� � ��

�� x��e��x�y��

G��x� y� � ��xye��x�y��

G��x� y� � ��

�� y��e��x�y��


with steering weights

ki�� f�� g� ��

where and � are the direction cosines of the required orientation� This di�ers slightly

from Freeman s twodimension notation where he presents his results in terms of the

rotation matrix s sin�� and cos�� in which case � cos�� and � � � sin��

Note that pairs of the functions are symmetric in their variables� x and y� so recursive

implementations only have to be derived for three functions� For the class of �lters that

we are considering these functions take the form P �x�e�x�where P �x� is a polynomial

in x� For the G� �lter these functions are

G��x� � ��

�� x��e�x

��

G��x� � xe�x�

��

G��x� � e�x��

The factor e�x�is incorporated in each of the recursive �lter designs� rather than factoring

it out for separate implementation� because the diminishing nature of the function is

required to achieve a proper �t in the approximation process of the recursive design�

A more compact notation

We now introduce a notation whereby the basis functions required to steer the �lter

G��x� y� � �� x��e��x�y�� through some rotation �� are described by the ordered set

G�i�x� y� f�

�� x��

�

�� y��xyg� ��

with the steering weights

ki�� f�� g� ��

where and � are the direction cosines of the required orientation� Note that each func

tion is separable and can be factored into terms involving a single variable� A recursive

�lter is designed for each of these terms�


P �x�e�x� a� a� �� b�c� c� �� b� sym�

�e�x� ��

�� even

xe�x� �� odd

�� x��e�x� ��

�� even

�� x��e�x� ��

�� even

��x � x��e�x� �� odd

��x � x��e�x� ��

�� odd

Table �� The recursive implementation of the separable terms of the set of separable basisfunctions that form the steered G� and H� �lters are parameterised by the coe�cientsa�� a�� b�� c�� c�� and b�� These parameters in conjunction with equation �� andequation �� describe the separable terms of the basis functions of equations �� and ��Only the causal part of the recursive �lter is described by these equations� the anticausalpart is determined as per section �� and depends on the symmetry of the function�

Using the same procedure on the quadrature �lter H��x� y� � ��x � x��e��x�y��

we get the following set of basis functions�

H�i f��x � x��y � y��y�� x��x�� y��g� ��

with the steering weights

ki�� f�� g� ��

Fitting Deriche s approximation function� equation �� to each of these functions we get

the coe�cients shown in Table ��

The parameters given in Table � characterise the recursive �lter design� The actual

�lter implementation requires one more step� that of determining the coe�cients of the

numerator and denominator of the ztransform of equation �� evaluated for a given value

of �� This is a simple numeric substitution to determine the coe�cients ni and di� from

equation �� These coe�cients will be the weights used in the di�erence equation �� This

evaluation is only done at runtime when the �lter arrays are initialised� The anticausal


-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

�e�x�-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

xe�x�-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

�� x��e�x�

-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

�� x��e�x�

-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

��x � x��e�x�

-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

��x � x��e�x�

Figure �� Recursive implementations of the separate terms of the G� and H� basisfunctions� These �lters are parameterised in Table ��

-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

-4 -3 -2 -1 0 1 2 3 4

-1.0

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1.0

Figure �� Oddsymmetric Morlet Wavelet functions� showing the e�ect of increasing k�the ratio between the frequency of the sinusoid and the Gaussian envelope� A value of�� center plot� has been suggested by Kovesi to provide a good frequency spread forcalculating local energy�

�lter values are determined from the causal ones using the symmetry considerations

described in section ��

�� Recursive Morlet Wavelets

Many quadrature pairs of �lters can be used to calculate local energy� The derivative of

Gaussian functions and their Hilbert transforms are popular �lters but are not necessarily

the best� The family of Morlet wavelet functions has been shown by Kovesi �� to have

useful frequency characteristics when summed to cover a broad bandwidth of features� It

would be useful if Morlet wavelets could be implemented using the steering and recursive

algorithms developed in this thesis�

Kovesi s modi�edMorlet wavelet is a sinusoid under a Gaussian envelope� as illustrated by


�gure �� and described in section �� The ratio� k between the frequency of the sinusoid

and the width of the envelope determines the number of oscillations that the function

exhibits� This is important for the polynomial �tting stage of our design procedure in

which the degree of the polynomial will determine the number of basis functions required

to steer the �lter�

Consider the Taylor expansion of the sine function�

sin�x� � x� x�

�"�x�

�"� � � � � x�n��

��n� ��"� � � � ��

It would appear that only a few terms would be required� but even using the �fth order

polynomial requires six basis functions to steer it� We must also remember that the

Taylor expansion is only good over a limited region and as the ratio� k� increases the

region of close approximation is increased� requiring more terms in the series�

Unfortunately the number of steering terms required becomes prohibitive when the ratio

k is in the band that makes the Morlet wavelet function useful for calculating local energy�

�� Recursive local energy

All the components of a recursive implementation of the local energy calculation for

image feature detection have now been developed� This section describes my use of this

technique to calculate local energy in twodimensions using the steerable recursive �lters�

From a quadrature pair of �lters� twodimensional �lters were constructed by smoothing

in the direction orthogonal to the axis of abscissa� The particular �lters I have chosen

to use are the second derivative of the Gaussian function and an approximation of its

Hilbert transform both of which can be factored into two terms� one a polynomial and

the other itself a Gaussian function� The same Gaussian function has been used for the

orthogonal smoothing� Using Freeman s steering formulae these twodimensional �lters

have been rotated to several orientations�

Oriented energy values were then calculated using these rotated �lters at several evenly

spaced orientations and summed to calculate local energy� The design of the �lters allows

an arbitrary scaling factor to be applied to the �lters with no computational penalty� This

facilitated a �exible choice of scale for the processing�


Figure �� The three G��x� basis functions� �� y��e��x�y�� x��e��x�y�� and

��xye��x�y��

Figure �� The four H��x� basis functions� ��y � y��e��x�y�� x �x��e��x�y�� x�� y��e��x�y�� and ��y�� x��e��x�y��

�� Quadrature �lters

The pair of �lters chosen for the calculation of local energy in this chapter are the second

derivative of the Gaussian function� G�� and a numerical approximation to its Hilbert

transform� H�� These functions were discussed in section �� The �lters are normalised

in the recursive implementation to have identical L� norms for proper quadrature�

Figure � displays the form of the separable terms which are implemented using the

recursive technique�

Figure �� displays the three basis �lters used to implement the steerable second derivative

of the Gaussian �lter� G��

Figure �� displays the four basis �lters used to implement the steerable Hilbert transform

of the second derivative of the Gaussian �lter� H��

To make multidimensional �lters the onedimensional �lter is smoothed in orthogonal


dimensions with a function that is a Gaussian function of the same width as the Gaus

sian term in the G� and H� functions� This function is also one of the basis functions

eliminating the need for extra �lters�

�� Orientations

The local energy value is the sum of the oriented energy values calculated at several

orientations� These orientations are chosen to give a uniform response to features at all

angles in the image�

Orientations are spaced equally across the range � to �� but it is not necessary to du

plicate the second half of the circle� because of the symmetry in the �lters and the

quadratic nature of the energy formula� Using a relatively broad Gaussian smoothing

function means that only a few orientations are required� As illustrated by the contour

plots in �gure �� three or four orientations produce a quite even response� Researcher s

such as Robbins �� use �lters with a much narrower angular spread� This provides

better angular resolution but needs many more orientations� Robbins suggested that ten

orientations were required for his stretched Gabor functions�

�� Scales

The scale of features detected depends on the width of the �lters used to calculate local

energy� Filter width� �� is easily controlled in the recursive implementation of the �lters

presented here� In general� computer vision tasks use high resolution to get the most out

of limited digitisation but some applications need to detect features across a wide range

of scales�

The recursive technique described in this thesis implements �lters with values of � valid

from about � up to �� or more� This range is determined by the accuracy of the

numerical �tting procedure in the �lter design�

Just as the oriented energy values are summed to get a local energy value� multiple scales

can be calculated and summed to give a broad coverage of feature sizes� To get an even

response across scale� the sum of the frequency responses of the various �lters should be

�at across the relevant frequencies� Unfortunately the frequency responses of the scaled

derivative of the Gaussian functions all contain signi�cant low frequency energy� making


it di�cult to get a �at response� In contrast� the Morlet wavelet frequency responses

sum nicely to form an arbitrarily wide passband�

Kovesi suggests that a frequency range of three octaves matches that of the human

visual system and is therefore a reasonable range to apply� We apply our �lters with

� f�� g although this may not be optimal� Empirical experimentation suggests

that scales of � f�� g with weights of �� and � respectively may be more suitable

for the second derivative of Gaussian �lters�

As mentioned in section �� a change in �lter scale is re�ected in a proportional factor

in the response� When local energy results of di�erent scales are summed this factor must

be divided out� For a twodimensional �lter the factor is ��

and for threedimensional

�lters it is �� This factor is combined with the weights�

�� Results

The recursive local energy is shown here for three images� one synthetic� the commonly

used �Lena image� and a more detailed image of children s blocks� The images are

displayed with high energy values as black�

Figure �� shows an example of the local energy of a synthetic �Test image calculated at

several scales and the resultant combined energy�

Figure �� shows an example of the local energy of the �Lena image calculated at several

scales and the resultant combined energy�

Figure �� shows an example of the local energy of the �A block image calculated at

several scales and the resultant combined energy�

Discussion

A problem when using steered �lters for local energy estimation is that the number of

convolutions and copies of the image that are required in order to store intermediate

results for the calculation of local energy increases with the number of orientations and

scales� The nonlinear nature of the combination of partial results�� that is implicit in

summing local energy components� means that the operations can not be combined� All

�ltering operations must be performed separately� then squared and then added�

�The combination F � �H� is quadratic in nature�


Synthetic test image�� greylevels�

Combined energy image over � octaves�

�D local energy � � � �D local energy � � �

�D local energy � � � �D local energy � �

Figure �� Recursive implementation of local energy� estimated using G� and H� �ltersover a range of scales � f�� g� of the synthetic test image �test �


�Lena image�� greylevels�




Figure �� Recursive implementation of local energy� estimated using G� and H� �ltersover a range of scales � f�� g� of the image �Lena �


�A block image�� greylevels�




Figure �� Recursive implementation of local energy� estimated using G� and H� �ltersover a range of scales � f�� g� of the image �A block �


Consider the case of the third derivative of the Gaussian function� G��x�� Steering this

function requires four separable basis functions� Its Hilbert transform requires �ve and

we will need to calculate an oriented energy and a total energy across all orientations�

This means �� oating point copies of the image will be required" For large images or

small memory computers this would be unacceptable�

The proliferation of intermediate storage could be reduced by using G��x� and H��x��

And also� since this �ltering technique is in the spatial domain� and the �lters have a

limited support� the input image could be processed in chunks and the results reassem

bled� Such segmentation might also lend itself to a coarse grain parallel implementation

of the �ltering algorithms�

The static analysis of the entire rectangular image does not really illustrate the potential

of this system of �lters� The advantage of the system is its �exibility� Scales and ori

entations can very easily be changed and spatial domain processing means that various

regions of the image could be processed with di�erent parameters�

Unlike frequency domain �ltering where e�cient processing often requires rectangular

regions whose dimensions are powers of two� the spatial domain recursive system could

be arranged to process an odd shaped region� for example one that follows some contour�

The scheme was originally intended as part of an active analysis system� The whole image

would be simply processed at one scale and a foveation mechanism would direct more

detailed analysis to speci�c regions of the image� Initially this was intended to assist

MPEG compression �� encoding of moving pictures� Another conceivable application

would be an autonomous vehicle whose vision system may need to identify �on a broad

scale� then read �at a �ne scale� a road sign or track a pedestrian s progress towards a

crosswalk�

�� Recursive local energy in three dimensions

In multiple dimensions� an oriented energy is calculated in several directions and summed

to give a measure of local energy� Oriented energy is calculated by taking the one

dimensional quadrature �lters smoothed along orthogonal axes and rotated to vari

ous orientations� This has been done in twodimensions by Kovesi �� Robbins and

Owens �� Ronse �� Xie �� and Robins �� More recently threedimensional data


have been analysed using local energy by Pudney et al� �� and by the present author�

Robins ��

The set of orientations� Orient� is chosen to provide even coverage of the multidimensional

space� Local energy is then

E�x� �X


where�

E�v�x� �q�I�x� �M even

�v�x�� I�x� �Modd

�v�x��

The �ltersM even�v

�x� andModd�v

�x� are axially symmetric functions comprising quadrature

�lters along one axis� smoothed in orthogonal directions and rotated so that the axis of

symmetry is aligned with the unit vector v�

�� Steering equations for multiple dimensions

The oriented �lters are designed by steering a �lter of the form f�x� y� z� � W �r�PN �x�

using a kernel of separable basis functions� that is�

fR�x� y� z� �NXj��

kj��Rj�x�Sj�y�Tj�z��

where Rj�x�� Sj�y� and Tj�z� are separable basis functions� and kj�� are weightings that

steer the �lter to an orientation �� PN �x� is a polynomial of order N and W �r� is a

rotationally symmetric function such as the Gaussian function� e��x�y�z��

The kernel consists of only separable basis functions� Starting with a basis comprising the

original function oriented to several evenly spaced directions� the kernel and weightings

are rearranged as outlined by Freeman ��

Each term of the separable basis functions is then implemented as a recursive �lter�

Separability means that the convolution can be performed one dimension at a time�

which dramatically reduces the computation required to perform the total convolution�

For the local energy calculation we take W �r� to be a Gaussian smoothing function and

the PN �x� to be even or odd polynomials in x� The symmetry of the polynomial halves

the number of basis functions required to implement it�

A similar summing of component energies can be applied to provide broad frequency coverage�


For local energy we start with a second derivative of the Gaussian function� which can

be written as a polynomial times a Gaussian� A quadrature pair is formed by taking the

Hilbert transform and approximating the result with a polynomial�

The recursive �lters are also designed using a numerical approximation technique which

was described by Deriche �� The coe�cients of a recursive �lter are determined from

the ztransform of the transfer function� but not all functions have a ztransform that can

be determined analytically� We approximate the required function with one for which

the ztransform is known� In this case the approximation is�

ha�k� �nXi��

ie��i k� ��

using n � � and constraining i and i to be complex but conjugate in order to get real

coe�cients� Then by substitution and application of Euler s rule� ha can be written

ha�k� � �a� cos��

�k� � a� sin�

��

�k��e�

b��k � �c� cos�

��

�k� � c� sin�

��

�k��e�

b��k� ��

The ztransform of this function can be determined and is given by Deriche ��

Note that the scaling parameter� �� is carried through to the last stage of the �lter design�

A recursive �lter for a range of scales can be implemented from the same numerical

approximation result� The author s computer program energy�d� key components of

which are presented in Appendix B�� takes � as an input parameter to select the scale

of the local energy result�

Note that a multiplicative scaling factor can be applied to the function by multiplying

the coe�cients a�� a�� c� and c� by the required factor�

�� Steering G� and H� in three dimensions

Results in this thesis are based on the second derivative of the Gaussian function and its

Hilbert transform� In twodimensions three orientations are used and in threedimensions

ten orientations are used�

To obtain evenly spaced orientations in three dimensions we need to consider the regular

polyhedra as other con�gurations will give anisotropic results� The tetrahedron and cube


provide too few orientations� The dodecahedron gives six unique orientations along which

to estimate local energy and the icosahedron gives ten evenly spaced three dimensional

orientations�

The normalised equations implemented here are�

G��x� y� z� � �� x

��e

��x��y��z��

��

H��x� y� z� � ��x��

x

��e

��x��y��z��

��

The separable basis functions for the quadrature pair G� and H� oriented in three

dimensions are� for G� � �� x��

G�i�x� y� z� f�

�� x��

�

�� y��

�

�� z�� xy��xz��yzg� ��

with steering weights

ki�� f�� g�

and for H� � ��x � x��

H�i�x� y� z� f��x � x��y � y��z � z��

��y�� x��z�� x��x�� y��

��z�� y��x�� z��y�� z��

�xyzg� ��

with corresponding steering weights

ki�� f�� g�

where � � and � are the direction cosines of the required orientation�

�� Recursive implementation of the G� and H� basis functions

Table � contains the recursive �lter design parameters necessary to implement the G�

and H� basis functions� These parameters are used in conjunction with equations �� to

determine the coe�cients used in the di�erence equation �� The anticausal �lter coe�

cients are determined from the causal ones using the symmetry considerations described

in section ��

A multiplicative scaling� such as a normalisation factor� can be applied by multiplying it

into the a and c parameters� refer to equation �� above�


�� Local energy

We can now estimate local energy for two and threedimensional signals using the sep

arable recursive �lters by summing the oriented energy calculated at several evenly dis

tributed orientations�

In twodimensions� evenly distributed orientations could be determined by simply divid

ing �� by the number of orientations desired� In threedimensions we are restricted to a

few regular polyhedra� Oriented energy can be calculated at six evenly distributed ori

entations using the dodecahedron � Using the icosahedron gives ten evenly distributed

orientations� We prefer not to use the cube as the response is not su�ciently uniform�

The dodecahedron and icosahedron are discussed in appendix D where the relevant di

rection cosines are tabulated�

Combination of energy across multiple scales is required as in the twodimensional calcu

lations� The same scales and weightings are relevant since we are summing the frequency

responses of the same �lter functions�

�� Results

Visualising the results of a threedimensional local energy calculation is much more di�

cult than with twodimensional images� The human visual system has evolved to detect

opaque objects in a transparent medium� If local energy density values are interpreted

as opacity and the volume rendered by computer� signi�cant detail is obscured� The

image is a blurry mess� In twodimensions the eye is very good at interpolating and

we interpret the centre of blurred lines and instinctively identify the darkest or lightest

contours in the image as a sketch of some sort which allows humans to easily interpret

the twodimensional energy result� which is not the case in threedimensions�

In threedimensions a common volume rendering technique is to choose some speci�c

voxel values and render just these values as solid voxels� Depending on the nature of the

volume data� particular values can be rendered in di�erent colours but the interpretation

and success of this technique depends very much on the image domain�

Often a threshold is used to select the voxels to render� but the problem with thresholding

local energy values is that it is the relative height� rather than the absolute height� that

Local energy response is symmetric� halving the number of unique calculations�


is important� In the latter chapters of this thesis I develop a technique to trace out the

points of maximum local value along with the ridges and surfaces of relatively maximum

points that lead up to these points� My ridge and surface tracing can be used to create

excellent visualisations of local energy results from both twodimensional images and

threedimensional volumes�

In this section threedimensional local energy maps are displayed as a series of two

dimensional slices through the volume� Synthetic images have been designed so that

areas of particular interest are illustrated in these slices�

Figure �� shows slices of the threedimensional local energy of a synthetic threedimensional

image calculated at several scales and the resultant combined energy� The image is a

thin square and an intersecting� o� centre� perpendicular line�


image calculated at several scales and the resultant combined energy� The image contains

two intersecting planes and a line�


image calculated at several scales and the resultant combined energy� The image con

tains a sphere around a cube all centred on three intersecting planes� The sphere and

planes are one pixel thick and the cube is solid� and they have varying intensity values�

The background intensity is shaded from zero to �� diagonally across the volume� This

variation can cause problems for some feature detectors�

Figure � shows a CT image of a neck� incorporating windpipe� vertebrae� arteries and

various soft tissue� Only a few slices through the volume are shown� The full energy map

is shown after surface tracing in Chapter ��

Discussion

The excessive memory usage problems discussed in the twodimensional case are exacer

bated in three dimensions� In three dimensions � and �� basis functions� �N��N��

are needed for G� and H� respectively with copies of the entire volume required for each�

This potentially consumes huge amounts of computer memory� Virtual memory can

provide su�cient space but the run time can then become untenable�

In order to calculate energy for the large CT image� the data volume has been cut into


smaller blocks for processing and the results combined� Overlapping regions� wider than

the extent of the �lters� have been allowed so that there is no edge e�ect at the adjoining

boundaries�

Memory requirements could be reduced by not using the steerable kernel to steer the

�lters� an ordinary rotation matrix could be used instead� Great e�ciencies are gained�

by using the recursive �lters but they have proved cumbersome to steer�

The alternative would be to rotate the original image through some angle� apply the

separable recursive �lters� then rotate the result back� The �ltered image could then be

squared and summed with the quadrature result and totalled into the energy map� Such

a scheme would dramatically reduce the number of copies of the volume that must be

kept in memory�

�� Conclusion

This chapter has described a new technique for calculating local energy� In order to

estimate local energy for multidimensional data� oriented �lters have been designed that

can be rotated to any direction through the use of a set of basis functions� Each of these

basis functions is arranged to be separable�

Separability means that multidimensional convolutions can be performed as a series of

single variable convolutions� which is computationally e�cient� The convolutions are

performed using a recursive �lter structure which further reduces computation�

The particular �lter design procedure enables the spatial width or scale of the �lter to

be changed easily with no computational penalty� The �lters used are based on the

derivative of the Gaussian function and its Hilbert transform� in particular the second

derivative referred to here as G� and its Hilbert transform H��

The steerable �lter theory was developed by Freeman �� and the recursive �lter design

technique was presented by Deriche �� I have combined the two techniques to develop

recursive steerable �lters and applied the resulting �lters to local energy� This is the �rst

e�cient spatial domain implementation of the local energy calculation� Calculating local

energy as the sum of a number of oriented energy values is relatively novel� in that it is

a recent approach that has only been done using frequency domain �ltering techniques

in the past�


The results have been very successful� producing a reasonably simpletoprogram frame

work for calculating local energy in two and threedimensional data� Flexibility� in terms

of size of regions� orientations and scale of the �lters is the main advantage� although a

simple structure and reasonable computational e�ciency are also bene�ts� Unfortunately�

the technique can require large amounts of computer memory although this can be traded

o� at the cost of more computation�

The local energy images and volumes that are produced identify the location of features

such as corners� lines� edges and other pro�les� as the points� ridges and surfaces of

relatively high energy values� The second part of this thesis examines a new technique to

automatically trace out the locations of these peaks and ridges in the local energy data�


�Square �D image�� greylevels�

Figure �� Recursive implementation of oriented local energy forthe synthetic volume �Square estimated using threedimensionalG� and H� �lters over a range of scales � f�� g� and ��evenly spaced orientations� Slices are �hoizontal and the numbering begins from the bottom�

Slice �� Slice �� Slice � Slice ��

Slice �� Slice �� Slice �� Slice ��




�planes �D image�� greylevels�

Figure �� Recursive implementation of oriented local energy forthe synthetic volume �Planes estimated using threedimensionalG� and H� �lters over a range of scales � f�� g� and ��evenly spaced orientations�






�Solid �D image�� greylevels�

Figure �� Recursive implementation of oriented local energy forthe synthetic volume �Solid estimated using threedimensionalG� and H� �lters over a range of scales � f�� g� and ��evenly spaced orientations� Note this image has been renderedwith some transparency to show internal structure�






�CTcarotid slice �� of �D image�� greylevels�

Figure �� Recursive implementation oforiented local energy for the CT image �CTcarotid estimated using threedimensional G� and H� �lters over arange of scales � f�� g� and ��evenly spaced orientations�

Slice �� Slice ��

Slice �� Slice ��

Chapter �

Peaks and Ridges

This chapter examines the notion of peaks and ridges in multidimensional functions and

reviews some of the methods used to identify these extrema� I discuss the twodimensional

case where an analogy to the geographical topography of a mountain range can be drawn�

and show that the same concepts can be extrapolated to multiple dimensions�

This thesis is primarily concerned with tracing ridges in a local energy map� in which the

valleys are usually broad and �at and have little signi�cance� I will generally ignore the

valleys in this discussion� If it were necessary to consider valleys� they could be traced

using related procedures by noting that they are the converse of ridges and applying

similar arguments to them with �greater than conditions swapped for �less than and the

term �maxima changed to �minima �

I will begin by considering what constitutes a ridge� and then examine some previous

work on the problem in order to ascertain what is involved in tracing out image features

from a local energy map� It will be shown that whilst the watershed algorithm promises

a solution� it fails to identify certain aspects of the topology� Finally I will point out

that the local energy function identi�es keypoints which can be used to anchor the ridge

topology� This allows me to develop a new ridge tracing algorithm which will be described

in detail in the following chapter�

Figure �� illustrates the ridges produced by the local energy feature detector� The

�gure shows the original image� a greyscale representation of the energy map and an

orthographic view of the energy function� The orthographic view clearly shows the peaks

in the energy at the corners of the square� These keypoints occur at corners and at the

�

CHAPTER �� PEAKS AND RIDGES

�a� �b� �c�

Figure �� a� Image consisting of a dark square on a light background� b� The energymap� rendered with dark tones representing higher values� c� An orthographic view ofthe energy function� Note the peaks of energy that occur at the corners and the ridgesthat connect them�

extremities of line features and are crucial to my ridge tracing algorithm�

�� Introduction

Feature detection usually involves �nding maxima in some function� which can be di�cult

when the function is multidimensional� A common solution is to identify a signi�cant

orientation and search for a onedimensional maximum along the normal to this orien

tation� In order to join such maxima into continuous curves the scheme of nonmaximal

suppression is often employed which entails the use of an orientation� and two hysteresis

thresholds�

The assumption that a suitable orientation exists holds true for edge detection but not for

higher order features such as junctions and corners� In the past� this has not necessarily

been a problem as edge detecting �lters were not expected to perform well at junctions or

corners� However these higher order features exist and are important in the local energy

model and any feature location procedure is required to identify them�

In the local energy paradigm� the feature location problem becomes one of tracing the

ridges that join peaks in a local energy map� Of the various schemes proposed to solve this

problem� one technique that is currently receiving attention uses the watershed algorithm�

A signi�cant part of the present work has been to develop a related skeletonisation

technique that can be used in conjunction with the local energy model to identify these

ridges�

CHAPTER �� PEAKS AND RIDGES �

�� Peaks

The term peaks is used to mean the locus of points in an array of data that have a higher

value than all of their neighbours� The term ridge is less clearly de�ned to be the locus

of points that have greater value than some subset of their neighbours� e�g� neighbours

that lie along a line perpendicular to the ridge�

�� Extrema in one�dimensional data sets

Finding extrema in a one dimensional data set is quite straight forward� we can step

along examining each value to �nd those that are greater than either of their neighbours�

In the event of a plateau we can recursively investigate neighbours of equal value� to see

if any higher values are encountered� It is then possible to mark one or all of the equal

valued neighbours as maxima� or perhaps mark the centre point� or apply some other

labelling� Alternatively by �tting a curve through the data points� the maxima of this

function could be located� Questions would then still arise as to how many data points

should be considered� what weightings applied to each and what type of function to �t�

Another method to �nd extrema in a function is to take derivatives and look for zero

crossings� Since numerical derivative operators are inherently a�ected by noise they are

usually combined with a smoothing operation to reduce the noise� The di�erentiation is

commutative over convolution� i�e��

d

dx�f�x� � gsmooth�x�� f�x� � d

dxgsmooth�x��

Convolution with the derivative of a smoothing function is a very common image pro

cessing operation� The �rst derivative of the Gaussian function is very often used for

this purpose� The Laplacian of Gaussian operator takes this one step further by con

volving with the second derivative of the Gaussian function� Zero crossings of the second

derivative operator would then coincide with extrema in the derivative� hence edges in

the intensity image� The Laplacian of Gaussian operator is rotationally symmetric and

separable� which leads to an e�ciently implemented operation exempli�ed by Jin and

Gao s �� recursive implementation�

The zero crossings of the Laplacian operators always form closed curves and don t respond

well to features other than edges�

CHAPTER �� PEAKS AND RIDGES ��

�� Extrema in multiple dimensions

Similar techniques can be used to �nd local extrema in multidimensioned data sets� The

data points are systematically scanned and the values of all the neighbours compared to

the point under scrutiny� A point greater than all of its neighbours can be marked as a

maximum�

In multiple dimensions the connectivity of the local neighbourhood needs to be consid

ered� It is usually de�ned by the problem domain with most data sets de�ned on a

regular grid� For a hexagonal grid the �connected neighbourhood is natural� For a

rectangular grid � or connectivity could be chosen� Commonly the complementary �n

and ��n � ��connectivities are chosen for ndimensioned data� but others such as the

threedimensional �connectivity may be used�

As with the onedimensional case� a plateau can be dealt with by recursive investigation�

which can be used� by careful labelling� to optimise and cut short subsequent recursive

searches through the data when they reach this point�

As with the onedimensional case� marking the points included in a plateau is problematic�

all points could be marked� one point� a weighted average location marked or some

function �tted to avoid the granularity of the data set�

Fortunately the peaks in the local energy maps are quite sharp and the marking is not

critical�

As we can see� local extrema are relatively simple to de�ne and an algorithm to �nd them

is straight forward� Ridges on the other hand are more nebulous�

�� Finding extrema in two�dimensions

Finding absolute extrema in �D is quite simple� We conduct an exhaustive search for

pixels equal to� or greater�less� than all of the connected neighbours� A recursive search

can be conducted to determine the status of equal valued neighbours� There are recur

sive scanning algorithms that can label absolute extrema� but these are quite complex�

requiring extensive label equivalence tests�

It is worth examining the ordered queue label propagation scheme that Vincent uses in

his algorithms� as similar techniques are used extensively in the algorithms described

later in this thesis�


Ordered lists of pixels

Vincent �� incorporates the determination of minima with his ordered pixel lists� but

this still requires two scans and recursive label propagations�

In his algorithm the pixel values are scanned to generate a tally of the number of pixels

that have each particular value� Pixel values are quantised by being restricted to integer

or byte values� A second pass though the image then enters a reference to each pixel in

the appropriate section of an array of all pixels ordered by value� The sections can be

dimensioned because we know� from the �rst scan� the number of pixels associated with

each value� The pixels are not ordered within a given value s section in the array�

This ordered array of pixel references is used as the basis for a �ooding paradigm of pixel

processing� An index is set to the start of the array and the pixel dereferenced� This

pixel will belong to a minimum plateau� A label is assigned to a label array corresponding

to the image pixels� This label is then propagated to all neighbours� Any neighbours

that have the same value also propagate the label to their neighbours and so on� When

this propagation is complete� the index is incremented and the next pixel examined� If

the label array shows this pixel to be unvisited� it must belong to another minimum� a

new label is generated and propagated� If the label array shows that the pixel has been

visited no processing is required�

Vincent s algorithm uses this propagation to calculate a watershed of the image seg

mented around all the minima that it �nds onthe�y� Maxima can also be determined

in a similar way by indexing downwards through the ordered array of pixel references� A

similar but di�erent �ooding paradigm is used in the next chapter� where the ordering

of pixel references is determined onthe�y from predetermined start points�

�� Ridges

Ridges in images have always been recognised as important in computer image analysis

because interesting image features are often marked with high intensity values�

Early attempts to segment images by tracing ridges in raw intensity images �� produced

poor segmentation and highlighted the di�culty in determining the ridge points� It is

only in certain classes of images that the ridges of raw intensity values are meaningful�

Examples of such images are Xrays� some types of MRI and interferograms�


Figure �� Stylised surface relief andcontour diagrams with the watershedshown on each� Note that there is nosingle orientation at the junction whichwould be used to align a one dimensional search to �nd the ridge point inthe vicinity of the junction�

A more general analysis technique is to use the intensity gradient to indicate image

structure� The ridges in a gradient image then identify discontinuities� i�e� edges� in the

intensity image� The Laplacian of Gaussian operator has been a popular feature detector

which acts like a second derivative operator�

The Laplacian can be expressed as the sum of partial second order derivatives and can

be rewritten as the sum of directional second derivatives along a feature contour and

normal to the contour� A zero crossing of this function will be e�ected by curvature

along the edge� unlike pure gradient based schemes which search soley along the gradient�

The Laplacian results are thinned and tend to be closed contours but su�er from false

responses due the the fact that the magnitude of the underlying intensity gradient is

ignored �� pp��

With gradient based operators� the obvious ridge detection technique is to search for

a maximum in a direction of the steepest gradient� The problem is that the gradient

operator does not respond well to many of the complex image topologies that occur�

such as in the vicinity of the junction of two edges� Edge map images are intuitively

good representations of a natural image but this is largely due to the extrapolative skill

of the HVS� The nonmaximal suppression with hysteresis thresholding heuristic has

become a standard technique to extrapolate and connect ridges in gradient based feature

detection algorithms�


�� Various attempts to �nd ridges

Part of the problem in devising an algorithm to trace ridges is de�ning a ridge� It is

reasonably clear to a human that a ridge is the path taken between two peaks� along

which at each step there is no higher ground that could have been taken� But this is

not mathematically rigorous� For example� what about the ridge that runs out along a

promontory that does not actually terminate in a headland but runs o� into the sea�

An obvious solution is to consider gradients in various directions� and this is the ap

proach taken by Haralick �� and also by Eberly �� They use the rate of change

of gradient to identify ridges and Eberly also uses it to �nd creases in a surface� This

gradient de�nition fails at complex surface curvature such as junctions of ridges as there

is no predominant orientation at such points� which requires extra contingencies in the

de�nition �see �gure ��

The problem could also be considered in terms of a mountain range� which introduces

the concept of watersheds� A watershed would be de�ned as the locus of points at which

there is an ambiguity as to which way a drop of water placed on the surface would �ow�

The watershed concept would certainly solve the problem at junctions�

Attempts to trace ridges

Early on� in order to ascertain if the �lters were producing useable results� I detected

ridges by marking a ridge point as one that was a maximum in any direction� taking the

four directions associated with a point s connected neighbours� This maximum in any

direction was performed over a distance of � points� The result is a superset of the real

ridge set� It was obvious from the results of this test that the ridge information was in

fact in the energy map but it was not clear how to get it out�

Another attempt to trace the ridges involved simulating a mountaineer� where at each

step a panorama of the view from the current pixel was generated and a step taken

towards the highest point� Bifurcations were dealt with recursively and obviously a

previous path should not be revisited� It quickly became obvious that a one pixel horizon

was insu�cient but extension to two or more pixels only shifted the problem� This

technique fails at �at spots and is not much use when a hill peters out�

Another approach would be to consider the surface as being covered in paint before a


block of deformable rubber is pressed down on it from above� The ridges would deposit

paint onto the block� thus producing a ridge map� I did consider a deformable model

as a method of implementing this ridge tracing technique but questions regarding the

coe�cients of elasticity and sti�ness make it impractical�

Finally� various morphological operators were considered� For example� if the ridges are

su�ciently sharp� a threshold would produce a set whose skeleton could be used as a

ridge map� I tried to normalise the ridge heights by counting the number of smaller

neighbours within some region� but without success� Another thought was to consider

�tting a maximal sphere under the surface� but the intractability of turning a two di

mensional problem into a three dimensional one exacerbates the problem� Rolling small

balls upwards under the ridges is analogous to the watershed solution discussed below�

Non�maximal suppression

Nonmaximal suppression is the �rst step in a thresholding heuristic suggested by Canny ��

�� which is followed by hysteresis thresholding to inhibit spurious responses but still al

low an edge strength to vary along its contour� The hysteresis tends to join the points

into a longer contour� Each point in an image is examined to see if it is a maximum

along the normal to the gradient� These points are then grouped together into contours

by thresholding with hysteresis� Hysteresis thresholding is often used without reference

to a de�nition so I will quote Canny s description�

If any part of a contour is above a high threshold� those points are immediately

output� as is the entire connected segment of contour which contains the

points and lies above a low threshold� The probability of streaking is greatly

reduced because for a contour to be broken it must now �uctuate above a high

threshold and below the low threshold� Also the probability of isolated false

edge points is reduced because the strength of such points must be above the

higher threshold� The ratio of the high to low threshold in the implementation

is in the range two or three to one�

He goes on to comment on the use of a smooth projection function� which is relevant to

the extension of the local energy �lters from one dimension to components of oriented

energy�


When nonmaximal suppression is applied to this rough signal we �nd that the

contours tend to wander or that in severe cases they are not even continuous�

The solution is to use a smooth window function� In statistics� the Hamming

and Hanning windows are typically used for moving averages� The Gaussian

is a reasonable approximation to both of these� and it certainly has very low

bandwidth for a given spatial width��

The problem with nonmaximal suppression with regard to local energy is that it requires

a gradient orientation� Local energy� particularly at or near junctions and corners� does

not have a de�nite orientation� In fact Freeman �� analyses multiple dominant

orientations at a junction as a method of classifying the junction�

Morphological operators

Morphological operators can be employed to highlight various image characteristics such

as peaks� lines and boundaries� Only three morphological techniques are described here�

and the reader is referred to texts such as Dougherty �� Giardina and Dougherty ��

or Serra �� for a more thorough treatment of this extensive topic�

Morphological dilation� A � B involves expanding the set� A� �or function� using some

structuring element� B� Morphological erosion� A�B� contracts the set� For dilation of

binary images the re�ection of the structuring element� #B is passed over every point in

the set� dilation gives the locus of all points that can be reached in an intersection of #B

and the set� Erosion of A by B is the locus of points at which B is contained within A�

Morphological openning� A B� and morphological closing �A�B� are two common image

processing operations� Openning constitutes successive erosion and dilation� whereas

closing constitutes dilation followed by erosion with a structuring element� In image

applications� opening tends to remove foreground detail and closing tends to �ll gaps

and join foreground sets�

Greylevel morphology is more involved �see Serra �� but basically considers each grey

level of the image function as a binary set and uses a single twodimensional structuring

element� Using threedimensional structuring elements is of dubious value since mathe

matically it results in a mixture of units from the various dimensions�

�The Gaussian is the unique function with minimal product of bandwidth and frequency�


The geodesic distance function can be used to identify extrema in a function� This

is discussed by Beucher and Meyer �� p�� This algorithm is interesting� but a more

e�cient and simpler recursive search is used in this thesis� Some morphological operators

lend themselves to parallel implementation� which motivates a lot of the work in image

processing using morphological operators�

The watershed transform �� p�� identi�es the dividing lines between catchment basins

in a function� These are a subset of the ridges that we wish to identify as part of the

local energy feature detection scheme� Section �� discusses the watershed transform in

detail�

�� Geometric methods N�dimensional images

A thorough and rigorous analysis of the de�nition of ridges is presented by Eberly in

his Ph�D� thesis �� and subsequent book �� and the reader is referred to these for a

detailed review of the literature and discussion of ridges� Eberly is interested in segment

ing raw medical intensity images such as MRI� angiogram and CT scans� He discusses

a number of formal ridge de�nitions which I shall summarise here� where I adopt his

notation�

His thesis is that a ridge is a feature related to a function s surface curvature� not neces

sarily just an extremum of the function s value� For our purpose� with respect to tracing

local energy maps� it is only the maxima that have interest and therefore not all of his

analysis or argument is applicable�

One aspect of ridge analysis that he espouses is the invariance of the de�nition with

respect to various transformations� Although his emphasis is on ridges� the argument

and construction naturally include valleys� the counterparts of ridges�

The transformations of interest are translation� rotation� magni�cation and a monotonic

intensity change� The local energy function is invariant across all these transformations

except that the scales will change with magni�cation� The fundamental ridge tracing

operation will be invariant as well� In practice� a monotonic change in intensity may

interact with the selected thresholds causing slight discrepancies between detected ridges�

His principal direction de�nition and level de�nition are based on the ideas of di�erential

geometry involving local measurement of curvatures�


Slope district boundaries

In Eberly s procedure the graph of the intensity values is searched for maxima� minima

and saddle points� Slope lines are drawn from minima to saddles �course lines� and from

saddles to maxima �ridge lines�� The regions enclosed by the slope lines are called slope

districts and are the primitive regions of segmentation� This is similar to the ridges

determined by the ridge tracing algorithm� which is basically a watershed segmentation�

and the result is analogous to the zones of in�uence developed by Beucher and Meyer ��

Eberly complains that �an undesirable consequence of the slope district construction is

that the ridge and course lines are determined by a nonlocal process � This is not quite

true because� were one to take an arbitrary region around a point and then determine

the slope lines� they would generally be the same as determined for the whole image�

Certainly� they would only di�er in a minor way� or for pathological cases� His criticism

of this de�nition is that it will ignore small creases that lie on the �ank of a larger ridge�

Are these ridges� It depends on one s de�nition of a ridge�

In my thesis ridges are loci of local maxima in a function whereas Eberly s consideration

and that of Monga and Benayoun �� is of any ridge on an arbitrarily oriented surface

including creases� lumps and bumps� They use a curvaturebased de�nition of a ridge�

Height denition

Eberly describes the height de�nition for ridges as more successful� with a history dating

back to �� and more recently used by Haralick �� The de�nition is based on com

puting local maxima for the intensity function in special directions� Ridge construction

by this process is local� and invariant under translation� rotation� and magni�cation� but

not monotonic intensity change�

Principal direction denition

The principal direction de�nition applies to ndimensional hypersurfaces� Ridges are

constructed as local extrema of principal curvatures� where the di�erentiation is taken in

principal directions� A geometrically intuitive discussion of this type of ridge is given by

Koenderink �� The de�nition provides a local construction� invariant under translation�

rotation and magni�cation� where the transformation is performed on the full space Rn��

CHAPTER �� PEAKS AND RIDGES �

�Vdu image� Synthetic �test image� Xray �neck image�

Figure �� Watersheds detected in raw image data� with lighter tones representing highervalues� Considering the Xray �neck image� we can see that the result is more meaningfulfor some speci�c image domains�

which contains the hypersurface� In regard to graphs of functions with domain Rn� the

ridges are not invariant under monotonic changes of the de�ning function�

Level denition

The level de�nition considers the curvature properties of level sets for smooth functions�

Treating the graph as a mountain range� a person walking along a path of constant

altitude might label the ridges as those points where the change in her direction changes

the most�

The ridge construction is local and ridges are invariant under all four transformations�

translation� rotation� magni�cation and monotonic transform of the intensity function�

Eberly discusses this de�nition in detail and points out its equivalence to the principal

direction de�nition applied to level surfaces of the graph instead of the entire graph�

Non�metric denition

Finally� Eberly gives a ridge de�nition called the non�metric de�nition� similar to the

principal direction de�nition� but only applicable to graphs of functions� The ridges

produced are often qualitatively similar to the principal direction de�nition� but the

ridge construction does not use the metric of the graph�


�Vdu energy image� �test energy image� Xray �neck energy image�

Figure �� Taking watersheds of the local energy map of the images traces most of theimage features� however several important image features are missed� In particular thepromontory features in the �test image are not marked�

�� Watershed image segmentation

If we view an intensity image as a geographic topography or mountain range this suggests

that ridges are synonymous with watersheds� A watershed is the dividing line that sep

arates two catchment basins� Rain falling either side of the watershed will �ow downhill

and eventually collect in one or the other catchment basin� This operational de�nition

would seem to su�ce as a ridge detector but on closer examination there are ambiguities

and pitfalls�

We should note that some authors refer to the catchment area as a watershed but I

feel that reference to the boundary is more useful and that the terms catchment area�

catchment basin� �ow region or zone of in�uence are more appropriate for regions�

The watershed transform is being increasingly used as a segmentation technique� It has

been applied directly to raw intensity images� as illustrated in �gure �� but more often

to the gradient image� as illustrated in �gure �� with energy maps� In the latter case it

replaces nonmaximal suppression and hysteresis thresholding and its main advantage is

that it extends contours right up to junctions� just so long as the initial �ltering scheme

is su�ciently e�ective� Whilst the watershed algorithm produces nicely thinned and

connected boundaries� it can su�er from noise as it does not have the hysteresis in its

thresholds�


�� Calculating digital watersheds

There are two methods used to calculate watersheds of digital spaces� both based on

simulation of water �owing on the digital surface topology� One involves investigating

the path water would follow from each point in the image to its eventual collection

point �� In this case the watersheds are detected as the boundary between catch

ment basins� The second method �� uses a �ooding paradigm where the surface is

�ooded from the catchment points and the watershed marked at the meeting points of

�oods from di�erent catchment basins� Vincent and Soille �� present a very e�cient

implementation for �ooding paradigm algorithms� using ordered queues of pixels� Their

algorithm is described below and discussed in detail in the following chapter�

Water ow simulation

The following implementation is described by Gauch and Pizer �� who refer to the

whole catchment area as a watershed� Their use of the watershed transform is part of a

multiscale image analysis using symmetrybased shape descriptions to track individual

ridges and valleys through scale space�

By following the image gradient downhill from each point in the image� the

set of points that drain to each local intensity minimum can be identi�ed�

� � � gradients can be followed uphill to local intensity maximum in the image�

de�ning the watershed duals of the image�

Their gradient calculation scheme is illustrative of the computational cost of these sorts

of watershed calculation�

The �rst step in calculating the watersheds for an image is identifying the local

intensity minima that de�ne the bottoms of watersheds� Since an integer

valued image is a poor approximation to a smooth surface� the input image

is converted to �oating point and very slightly blurred� This eliminates the

plateaus in the image and simpli�es the process of identifying maxima and

minima� To distinguish between these critical points� each pixel is compared

with its eight neighbors� � � �


Figure �� The watershed calculated bythe �ooding paradigm is the locus ofpoints where lakes arising from neighbouring minima meet�

The next step is calculating the image gradient� Since the goal here is to

identify the drainage directions for each pixel in the image� the eight neighbors

of each point are searched to determine the maximal and minimal intensity

directions � � �

These directions are then stored and a path determined from each pixel in the image

to a minimum� Gauch and Pizer then detect the watershed boundaries by scanning

the labelled image of watershed regions� from left to right and top to bottom� detecting

changes in labels� Where to actually locate the boundary pixels and what the connectivity

and thickness of the resulting boundary contour is remain unresolved questions�

Immersion Simulation

Buecher and Meyer �� proposed a watershed de�nition based on simulated immersion

of the image topology� Consider the surface� with holes pierced at each local minimum�

incrementally lowered into a bath of water� Starting from the minima of lowest altitude�

the water will progressively �ll up the di�erent catchment basins� As the water coming

from two di�erent catchment basins meets� we build a dam� At the end of the immer

sion� each minimum will be completely surrounded by dams� delimiting its associated

catchment basin� These dams correspond to the watersheds of the image�

Vincent and Soille �� present an e�cient implementation of this immersion simulation

using a �ooding paradigm and ordered queues of pixel references to control the sequence

of processing� The queues allow processing to propagate outwards and upwards from the


minima� so that at each stage only pixels on the �shoreline are considered�

No gradients need to be calculated as the ordered queues delay processing of higher pixels

until all lower pixels have been done� The algorithm is serial so some of the e�ciency

would be lost in a parallel processing environment� This algorithm is investigated in

detail in the following chapter�

�� Drawbacks of watershed segmentation

If the image has a plateau� the question arises as to where to draw the watershed� For

image segmentation purposes we tend to use thinned boundaries so maybe thinning a

wide watershed will be required� Another ambiguity arises when a damlike structure

exists on the side of a hill� Should the dam s watershed extend up the hill� or cease at

the high water mark and be contained within a larger catchment basin�

The main problem with the watershed as far as feature detection goes is that an isolated

feature or a line feature that extends into a catchment basin will not be marked �see� for

example� �gure ��c�� Such promontories do not separate di�erent catchment basins and

are therefore not marked as a watershed� Trying to overcome this with some extremity

heuristic leads to an ugly mess�

�� Skeletons and key point anchors

The problem with the watershed algorithm is that it uses the ridges to segment the

image into regions� rather than directly identifying the ridges as we would wish� The

resulting segmentation boundaries do not include the ridges that form promontories or

are isolated islands� Rather than abandon the approach� I decided to look more closely

at the mechanisms involved�

Firstly� I note that the local energy model can be used as a corner or keypoint detector�

Keypoints mark features such as corners and the ends of lines that are the extremities

of the types of features that are lost with the watershed segmentation�

Secondly� I look at skeletonisation algorithms� There has been a great deal of interest in

skeletonisation or thinning of binary sets� Binary images are often obtained by applying

a threshold to an image� The resulting binary sets are then thinned and reduced in some

way to get easily characterised image parameters� Some work has also been applied to


�a� �b� �c�

Figure �� a� Image of a dark cross overlaid on a grey square� b� The watershed of theenergy function does not include the arms of the cross� c� Using the keypoints that aredetected by local energy at the ends of the arms as anchors preserves them from erosionin the ridge tracing algorithm�

skeletonisation of greylevel images� although to be e�ective� greylevel skeletonisation

must be applied to a regularised image�

It is my proposal that energy maps from the local energy model are suitably regularised

and a skeleton can be determined that coincides with image feature locations�

�� Key�points

The oriented local energy model detects all orders of features� and higher order features

have higher energy� We can see this intuitively since at a higher order feature more of

the oriented �lters are responding than at adjacent points� so the total energy will be

greater� For example at a corner point the energy will result from the response of at least

two orientations whereas at points on the edges leading up to the corner the response

will be less� since it results from only one orientation�

This has an important e�ect in that keypoints have a higher energy than their neigh

bours� We can therefore easily �nd keypoints as absolute local maxima in the oriented

energy function� The wonderful thing about keypoints is that they bound features� they

mark corners� ends of lines� and isolated points�

I will show in Chapter �� by using keypoints to anchor the extremities of lower order

features we can apply an erosive skeletonisation to trace the connecting ridges�

This anchoring extends to three and higher dimensions� Keypoints still only indicate

the highest order features� i�e� points� but analogous energy ridges trace out the edges of


surfaces �see Chapter ��

This result is slightly di�erent from the work of Heitger et al� �� Henricsson and

Heitger�� and Robbins and Owens �� where a much more rigorous de�nition is placed

on the term keypoint� In their work� all keypoints are detected� including junctions that

are not peaks in the simple application of the oriented energy map�

Robbins keypoint detection scheme works by calculating the oriented energy in some

direction� and then calculating oriented energy of this �rst energy map in an orthogonal

direction� This calculation of energy of energy in orthogonal directions will eventually

result in onedimensional �point� responses in an energy map� The process is repeated

from several starting orientations and the results summed� Keypoints are then identi�ed

as the points of local maximum in the total result�

His process uses many angularly narrow �lters to provide an arbitrarily complete spatial

coverage and can be applied at various scales� The keypoints so identi�ed could be used

to anchor my ridge tracing algorithm but since straight local energy �nds the features

that are missed by the watershed algorithm� the exhaustive keypoint processing was not

deemed necessary�

�� Skeletonisation

There has been much interest in the skeletonisation of binary sets� since thresholding

reduces the problem domain of image processing and then thinning or skeletonisation

removes another dimension� Greylevel skeletonisation involves consideration of pixel

values at each stage and is therefore more complicated�

In a binary image each pixel has a value of either �� or �� which indicates membership

of the foreground or background� The pixels are grouped into foreground �background�

sets each consisting of foreground �background� pixels that are adjacent or connected��

The foreground pixels are referred to as black and the background as white� complimentary

connectivities are necessary for foreground and background sets� Pixels that have both

black and white neighbours are said to be border or boundary pixels� Pixels with all

neighbours of the same colour are said to be interior pixels�

�In practice foreground sets are often labelled using the pixel value� with zero reserved for thebackground�


Various schemes are used to skeletonise binary sets� These are referred to as binary thin

ning or skeletonisation algorithms� The thinning is done to obtain some representative

axis whilst still maintaining the structure and topology of the sets�

The basic technique is to remove pixels from the set leaving only a representative skeleton�

Pixel removal is usually based on an analysis of the pixel s local neighbourhood� Two

styles of analysis are parallel and serial erosion� Parallel removal means that all the

removals are determined at each step before any changes are made� A serial algorithm

removes one pixel at a time then examines the next removal in the new context�

Parallel removal can be implemented in a parallel computer and the process is generally

deterministic in that it will always produce the same results� A serial algorithm may

have di�erent results depending on the exact order of processing�

A parallel system will require scanning each pixel in the set at each step� whereas serial

algorithms can be arranged to propagate and only consider boundary points�

It is interesting to note that Eberly s ridge analysis produces a nonthinned set of ridge

points� He then proceeds to go through a full skeletonisation procedure to determine a

thinned set of ridge lines�

Vincent �� provides an e�cient skeletonisation algorithm based on the ordered queue

mechanisms that he developed for the watershed algorithm�

Set topology

Thinning algorithms are required to maintain the topology of the original sets� This

topology refers to the number of connected sets and the number of holes contained

within the set� Many books have been written about topology and the reader is referred

to Chinn and Steenrod �� for a thorough treatment of the subject� or the survey of

digital topology by Kong and Rosenfeld ��

An operation on the set that preserves the topology is said to be homotopic� The basic

operation performed in a thinning process is the homotopic removal of individual points

from the set�

In �at twodimensional sets� topology is quite simple� There are two considerations�

simply connectivity and hole enclosure� For a pixel to be homotopically removed from a

set�


�� removal must not alter the number of connected sets in the image� and

�� removal must not introduce a new hole into a set�

These two conditions are su�cient to determine the validity of removing the pixel from

a set� i�e� changing its status from black to white� The homotopy of a removal can

be determined from consideration of the local neighbourhood of the pixel and parallel

algorithms have been developed that ensure homotopic invariance�

In threedimensions the situation is more complicated because along with connectivity

and enclosed cavities one must also consider the existence of tunnels through the set�

Researchers have tackled the problem in threedimensions and in particular we will ex

amine the work of Tsao and Fu �� Toriwaki et al� �� and Watzel et al� �� in

threedimensional thinning� This work will be examined in detail in Chapter ��

Medial axis and maximal balls

Since skeletonisation is an erosive process� knowing when to stop can be a problem�

Vincent s binary set skeletonisation algorithm uses the concept of anchor points to control

the structure of the skeleton� These points are selected prior to starting and are not

eroded� By preserving the anchor points and the set s topology� the shape of the set is

maintained in the skeleton�

Selecting anchor points is very important to the resulting skeleton� Vincent shows how

his one algorithm can be varied to �nd di�erent types of skeleton simply by using di�erent

heuristics to select the anchor points�

The medial axis transform is an important skeletonisation that can be described as the

locus of points lying equidistant from their nearest border pixels� Consider a set of grass

�res started at each border pixel� The �refronts would propagate inwards at constant

speed and the points where two or more �refronts meet would form the medial axis�

Another way of considering the medial axis is to think of circles subscribed in the set

such that they touch the border of the set at two places� These will be the biggest circles

that can be drawn at that point in the set� hence the name maximal balls� The centres

of these maximal balls will lie on the medial axis�

Unfortunately these centres do not entirely cover the medial axis� as there may be gaps�

Vincent uses the centres of the maximal balls as anchor points� The centres of maximal


balls can easily be determined as the points of local maximum in the distance transform

of the binary set�

The distance transform is created by labelling each pixel with its distance from the nearest

border pixel� As discussed previously� such labelling can be done using a propagation

type of algorithm and local maxima can be found by scanning with recursive labelling�

�� Conclusion

This completes the discussion of keypoints and skeletonisation� Further details and a

review of the algorithms and techniques that have been introduced are included in the

following chapters�

�� Summary

This chapter has introduced the problems involved in �nding peaks and ridges in digital

images� It has reviewed various approaches to the problem that have been taken in the

past� and in an attempt to solve the problem with an e�cient morphological algorithm

rather than di�erential geometry� I examined the watershed algorithm but found it to be

de�cient in some respects�

The watershed algorithm fails to identify the important features represented by promon

tories and isolated islands� However� the local energy model produces keypoints that

mark these features in the energy map� I have examined binary skeletonisation and in

the next chapter will combine these results in order to develop an algorithm that can use

these anchor points to trace the ridges in the energy map of an image�

Chapter �

Ridge Tracing

Algorithms based on �ooding simulations provide a novel and e�cient approach to a

number of image segmentation problems� The paradigm considers an image as a topo

logical surface and orders the pixel processing as if the surface were being immersed in

water�

This chapter describes the implementation of various algorithms using the �ooding paradigm

in the context of image segmentation using a local energy map� where peaks and ridges

in the local energy map correspond with features in the image� Watershed and skeleton

morphological transforms are used to trace ridges in the local energy map� The skeleton

concept is also extended for application to greylevel images�

�� Introduction

In the extension of the local energy model of image feature detection to two dimen

sions� features are located not just at the local maxima but also along ridges in the

twodimensional energy function� Finding these ridges is the objective of the algorithms

presented in this chapter� In three dimensions it would be necessary to locate the surfaces

between the ridges� and this is discussed in Chapter ��

A two dimensional energy function can be considered as a topographic relief map rep

resenting a mountain range� The task of tracing the ridges of this mountain range� can

then be couched in terms of �nding the watershed of the terrain� the watershed being the

boundary line between various rainfall catchment basins� However this is not the com

plete solution since spurs or promontories extending out into catchment basins would

��

CHAPTER �� RIDGE TRACING ��

not be marked as dividing lines and the algorithm would fail to locate the corresponding

image features�

A very e�cient technique exists for �nding the watershed of a digital image �� This al

gorithm simulates �ooding the topographic relief map to determine where the catchment

basins meet� The e�cient procedure to perform this simulation uses an ordered queue

of candidate pixels to sequence the processing� I have called this technique the �ood�

ing paradigm� It represents a class of propagation algorithms that operate on gradient

structured images� such as the energy map�

The �ooding paradigm is investigated here and used to develop a ridge tracing technique

that is fast and e�ective� It can detect small hills and lofty peaks� Importantly it

is capable of detecting open features� analogous to the spurs and promontories of the

topographical model� A �lling procedure is also developed to prevent the complication

of oversegmentation� To achieve this� algorithms are presented that perform various

morphologic transforms which are related back to the energy map that represents the

original image�

In the following subsection relevant characteristics of the local energy model are restated�

A qualitative review of ridge tracing is then presented in section �� together with the

basic concepts of the watershed and skeleton transforms� A discussion of the �ooding

paradigm and general structure of this class of algorithm is presented in section ��

Section �� contains details for implementing the established watershed and skeleton

transforms and the new procedures for ridge tracing and �lling� The results of applying

these procedures to various image data are presented in section ��

�� Energy Maps Revisited�

It will be recalled from Chapter � that an energy map is an image formed by an energy

transform applied to a digital greylevel image� The energy transform calculates a mea

sure of local energy at each pixel of the image� Peaks in the energy map have been shown

to correspond with visual features in the image� but some technique is needed in order

to identify these peaks in two dimensions�

The concept of oriented energy has been used in extending the local energy model to two

dimensions� At each point a measure of energy is calculated in a number of orientations

using angularly tuned twodimensional �lters� The total energy is the sum of the oriented


energies�

The energy images used in this chapter were calculated using the G� and H� �lters

determined in Chapter �� These two �lters are steerable� scalable and separable� one is

derived from the second derivative of a Gaussian� extended into two dimensions� and the

other is a function which approximates its Hilbert transform in the given orientation�

The Hilbert transform H� of G� is approximated by �nding a least squares �t to a

polynomial times a Gaussian� The �lters are applied using three orientations� This has

been found empirically to show a reasonably uniform response to various edge orientations

in the image�

The energy from each orientation is summed to form an energy map� At this stage in the

processing� each energy map is converted to a greylevel image by scaling the minimum

energy to � and the maximum energy to �� Peaks in the energy map are su�ciently

sharp that this quantisation to �� levels tends not to obscure or �atten the peaks� The

quantisation may obliterate some local minima but� as discussed in section �� this is

actually desirable�

�� Ridge Tracing Qualitative�

The problem of �nding peaks in two dimensions is easily solved for absolute local maxima�

Local maxima or mountain summits are simply the pixels whose values are greater than

or equal to all surrounding pixels� Various algorithms exist to �nd these points ��

p�� Finding local minima is an analogous task� However� with greylevel images�

peaks are often connected regions or plateaux due to the quantisation�

Finding pixels that lie on the ridges of an image surface is not trivial and was discussed

in section �� The �ooding simulation paradigm suggested by Buecher �� implemented

by Vincent and Soille �� and presented in section �� o�ers an approach in which ridges

are identi�ed as the watershed between two di�erent catchment basins� As the surface

is �ooded to successively greater depths� the locus of points where �oods from di�erent

catchment basins meet forms the watershed marking the boundary of the catchment

basins� This watershed analogy is discussed in more detail below�

A related problem of �nding a representative morphological skeleton of some connected

set of pixels �� suggested the new approach for de�ning and tracing ridges which is


described in this chapter�

Some of the important concepts which arose out of the discussion in the previous chapter

are the following�

� The watershed transformation is an important tool for image segmentation� It can

be visualised by considering the topographic relief formed by the image s values to

be pierced at each point of local minima and lowered into water� The surface will

be �ooded by water entering through the minima and during the �ooding� water

coming from di�erent minima will meet� These meeting points form the watershed�

marking the boundary of the catchment basins and hence delineating the ridges�

This �ooding process can be e�ciently simulated using queues of pixels to succes

sively propagate the �ood to neighbouring pixels�

� The skeleton transform arises from recognising that the problem of �nding ridges in

an energy map is analogous to determining a skeleton for a connected set of points�

The skeleton of set X can be de�ned as the locus of centres of maximal balls which

are the largest circles that can be drawn within the set which touch the boundary

at two or more points� The skeleton by maximal balls can be obtained as the set

of local maxima of the distance function of the set X� Unfortunately this skeleton

is not necessarily connected even though the original set is�

To be useful in practice� skeletonisation needs to be a homotopic transformation

�� A homotopic transformation is one that preserves the connectedness of a set

of points� neither creating isolated sets nor removing holes from within the sets�

� The distance function is an important morphological operation� Each pixel con

tained in a connected set of points is assigned a value representing its distance from

the boundary of the set� Various distance metrics can be used for the distance func

tion including Euclidean and geodesic� Distances may depend on the underlying

grid used to represent the pixels� The distance function of a binary image can be

e�ciently calculated using the queue of pixels technique as described below�

The similarity between the form of a distance function and an energy map motivated

the investigation of mathematical morphology as a means of tracing ridges that is

presented in this thesis�


5 3 2 1 04

to queue[ 4 ]put

from queue[ 1 ]get

Figure �� Ordered queues of pixels� illustrating put and get operations� When a pixelis �rst examined a reference to it is placed in the queue corresponding to the pixel sgreylevel value� The order of pixel processing is controlled by sequentially emptying thequeues�

�� Algorithms

This section examines the use of ordered queues of pixels as an image processing paradigm

as described and illustrated by Beucher and Meyer �� Many morphological operations

can be e�ciently implemented using this approach� as shown here and in the next section

where the same paradigm is used in the development of a new ridge tracing algorithm�

�� Ordered Queues of Pixels

The basis of this image processing paradigm is a set of queues ordered by greylevel value�

from which pixels are retrieved and processed one at a time� The queues are emptied one

at a time� starting with the lowest valued queue� In this paradigm� the pixel value and

those of its neighbours are used to determine changes to the image� As each neighbour is

examined it may be entered into a queue of pixels so that it will be processed at a later

time� In this way these algorithms successively propagate the processing outwards until

all the queues are empty and no more processing is required�

Advantages of using ordered queues of pixels are as follows�

� Each pixel need be processed only once�

� Only those pixels that may change are examined at each step�


� There is an implicit order to the processing�

� Low valued pixels will be processed before higher valued neighbours�

� Processing will propagate across regions of equal value�

� Termination occurs when all queues are empty�

The representation of a pixel in a queue is simply a reference coordinate that can be used

to index an array of pixel values� A label image is often used to �ag the status of a pixel

since operations such as check if a pixel is already in a queue are not trivial operations�

Usually this label image can hold the output values of the algorithm so memory is used

e�ciently�

Since the values of the pixels are limited to the greylevels� an array of queues can be set

up where the pixel value is used to index the array and select a queue� These ordered

queues provide the mechanism whereby processing of low valued pixels precedes that of

higher valued pixels� Figure �� illustrates an ordered queue structure�

Queues could be implemented using dynamic storage but this can be ine�cient� A better

method is to use an array to implement a ring bu�er� Using the operator overloading

facilities of C � �� dynamic array bounds can be implemented to handle such queues

quite e�ciently �� section ��

Some algorithms require only one pass though the pixels of the image� In this case an

array can be generated that contains references to all of the pixels arranged in order�

Such a structure is actually quite e�cient in storage and computation� The array is built

using two passes through the image� one to determine a histogram of pixel values� the

second to place a reference to each pixel in the array using an index calculated from

cumulative histogram values� Vincent uses this method in his implementation of the

watershed algorithm ��

�� Flooding Algorithms

Flooding algorithms are directly implemented using an ordered queue structure� Seed

pixels� see section �� are entered into the ordered queue to start the process� A pixel

is then retrieved from the lowest nonempty queue and its neighbours are examined� If

the neighbour has not yet been visited it is placed in the queue corresponding to the


for �level � �� level � maxPixelValue� level��

while �get p from queue�level � EMPTY� �

for �each neighbour n of p��

if �n�label �� UNVISITED� �

n�label � p�label �� propagate the label

put n on queue�n�value �� queue the neighbour for

processing

if �n�value � level�

level � n�value �� select a lower queue

Figure �� Pseudo code for the basic �ooding algorithm� illustrating the use of orderedqueues of pixels to propagate processing and values�

neighbour s pixel value and the label image marked appropriately� probably with a label

propagated from the original pixel� Processing continues with the retrieval of another

pixel from the lowest nonempty queue�

The code fragment in �gure �� e�ectively labels all the catchment basins of the watershed

transform� With the addition of a little code to detect and mark the boundary points

the watershed implementation is complete�

The �ooding paradigm is not restricted to two dimensional processing� In three dimen

sions the cost of scanning algorithms is signi�cantly higher so the �ooding paradigm s

e�ciency makes its use attractive�

�� Seed and Marker Pixels

Some points must be initially entered into the queues in order to start the �ooding al

gorithms� These pixels are referred to as seed pixels� Seeds may be determined as� local

minima or maxima� as the result of other morphological operations� by some heuristic

using related images� or randomly� The choice of starting points determines the inter

pretation of the �nal segmentation�

Many algorithms simply use the local minima of the image as seed pixels� Vincent s


implementation of the watershed algorithm �� demonstrates an elegant technique that

automatically determines the local minima as it processes from the ordered set of image

pixels� However� his technique does not facilitate seeding based on any other criteria�

Algorithms such as Vincent s skeleton thinning �� require certain pixels to be pre

marked so as to receive special treatment when they are encountered� This is achieved

by labeling these marker pixels prior to starting the �ooding algorithm�

�� Implementation

This section describes the implementation of a number of transforms using the �ooding

paradigm and ordered queues of pixels� These transforms have been used in order to

trace the ridges of a local energy map�

The watershed and skeleton morphologic transforms are now standard algorithms after

the implementations presented by Vincent �� The ridge tracing and �lling algo

rithms are original to this thesis�

The algorithms all use the �ooding paradigm� and follow the same general form as pre

sented in �gure �� They are expressed here in terms of their starting conditions and

the pixel processing operations performed on each pixel� Label values are shown in

UPPER CASE with their meaning inferred by the name� The label image starts out

initialised to the value UNVISITED�

�� Watershed Transform

The watershed algorithm �lls catchment basins from seed points� labeling each pixel in

the catchment basin� Pixels at the boundary between catchment basins are labelled

separately and form the watershed�

Start conditions

basin �� BOUNDARY � �

for �each pixel p in Local�Minima�Set� �

p�label �� basin��


put p on queue�p�value

Pixel processing

label �� p�label

for �each neighbour n of p� �


n�label �� label

put n on queue�n�value

else if �n�label � label� �

p�label �� BOUNDARY

�� Skeleton Transform

This skeleton algorithm� reported by Vincent �� performs successive peelings of the

distance function of the binary sets contained in the image� Pixels are removed from the

set if their removal does not alter the homotopy of the set but not if they belong to the

set of premarked anchor points�

The homotopy of each point is determined through a lookup table using a binary code

word comprising the state of its neighbours� A point can not be removed if its removal

disconnects connected neighbours or if it creates a hole in the set�

Various types of skeletons can be generated from this algorithm depending on the anchor

points� The full skeleton is calculated using the skeleton created by maximal balls as the

anchor points�

The seed list used for the skeleton algorithm is the set of boundary pixels of the sets

to be skeletonised� These can be determined by examining pixel neighbourhoods in one

scanning of the image�


Start conditions

for �each pixel p in Boundary�Set� �

p�label �� EXCLUDED

put p on queue�p�distance

for �each pixel p in Anchor�Set� p�label �� ANCHOR

Pixel processing

code ��



n�label �� INCLUDED

put n on queue�n�distance

if �n�label �� EXCLUDED� �

code �� code SHL ��

else �

code �� code SHL �� OR �

if ��p�label � ANCHOR�

AND �homotopy�table�code �� REMOVABLE��

p�label �� EXCLUDED

changes �� changes � �

else �

put p on pruning�list


Figure �� Regions of similar energy values may lead to extraneous pixels being left aftererosion� Pixels marked �x require removal during the pruning phase�

Post process pruning

Due to the incremental nature of the processing� it is possible for some pixels to be

INCLUDED when they should in fact be EXCLUDED� When examining the homotopy�

only the immediate neighbours are considered� which can mean that the processing order

may alter the result� In general� pixels are processed in order of energy value and inwards

within regions of the same energy but the exact order is indeterminant for pixels at a

given energy value� If a structure such as that illustrated by �gure �� is processed the

outer pixels will be examined at a time when the neck pixels must be preserved in order

to maintain connection to the as yet unexamined interior pixels� The internal pixels will

be removed later since they are not anchored� but the previously retained neck pixels will

still be left� The e�ect is much more noticeable in threedimensional energy data�

These extraneous tendrils are removed by repeatedly scanning a list of the remaining

INCLUDED pixels� removing those that are no longer valid according to the homotopy

and anchor point rules� This scanning is reasonably e�cient since the number of skeleton

pixels is quite small� Scanning stops when no changes are made during one scan�

repeat �

check�list �� pruning�list

pruning�list �� EMPTY

changes ��

for �each pixel p in check�list� �


perform pixel processing� as above

until �changes ��

�� Ridge Tracing

This ridge tracing technique is a new approach derived by combining the morphologic

watershed and skeleton transform for application to energy map images� Neither the

watershed transform nor the skeleton transform alone was completely suitable for tracing

the peaks in an energy image� The watershed algorithm �nds closed curves and hence

�oods spurs or promontories which represent features in the original image� The skeleton

algorithm requires a binary segmentation of the image on which to operate� The technique

described here combines them both� removes their limitations and retains their e�cient

implementation�

The key components of this algorithm are�

� The energy map has local maxima at extremities of ridges�

� Pixel removal starts at local minima�

� Homotopy of the ridges is preserved�

� Successive peelings remove lower contours �rst�

The local energy values are slightly higher at the extremity of a spur or promontory than

at points midway along a contour� Because of this� the local maxima of the energy map

provide valid anchor points for the skeleton algorithm� The higher energy values result

from there being more information in the vicinity of an end point of a feature than at a

point midway along the image contour or feature�

By starting the �ooding process at the minima of the energy function we will be sure

to �nd all ridges� regardless of their absolute height� This is important since the energy

value is not directly proportional to the feature s signi�cance ��

The homotopy consideration results from the recognition that the connected sets are


those points whose value is still above the current processing level� just as they were for

the distance function in Vincent s skeletonisation algorithm�

Start conditions

for �each pixel p in Local�Minima�of�Energy� put p on queue�p�energy

for �each pixel p in Local�Maxima�of�Energy� p�label �� ANCHOR

Pixel processing

as per Skeleton Transform

Post Process Pruning

as per Skeleton Transform

�� Filling

The �lling algorithm simply �oods minima to a given depth� somewhat as if a constant

depth of rain fell on a mountain range� This e�ectively �lls shallow catchment basins�

removing their minima from the startup lists�

The operation of this algorithm is slightly unusual in that the �ooding mechanism prop

agates an energy value out from the minima and� if a cusp or deeper catchment basin is

encountered� a lower value may be propagated back inwards towards the original min

imum� Propagating back inwards is achieved by reentering a pixel in a higher level

queue�

The processing proceeds in the correct order because the queues are a �rst in �rst out

structure and are processed in ascending order� This algorithm determines the minima

as it goes� by initially sorting all of the pixels in the ordered queue�

Start conditions

for �each pixel p� put p on queue�p�value


Pixel processing

if �p�label �� UNVISITED� �

p�label �� min�maxPixelValue� p�value � fill�depth�


if �n�value � p�label� �

n�label �� n�value

else if �n�label �� UNVISITED� �

n�label �� p�label

else if �n�label � p�label�

n�label �� p�label

put n on queue�n�label

�� Results

In order to evaluate the results of these techniques we compare watershed and ridge

trace output against the energy map and original image� The comparison is subjective

but satisfactory for the purposes of discussion�

Figure � illustrates an oversegmentation problem and the result of applying various

reduction techniques� Figure �� demonstrates the di�erent results obtained by applying

the watershed and ridge tracing algorithms for one synthetic and two natural test images�

�� Oversegmentation

Applying the watershed transform to an image immediately uncovers a problem � that

of oversegmentation� There are two solutions to oversegmentation� One is to merge

segments in a postprocessing phase� The other is to preprocess the image to reduce


a� Oversegmented b� Smoothed twice c� Low threshold ��

d� Fill shallow minima �� e� Manual selection f� Random seeds� �� !��

Figure �� Illustration of the oversegmentation that results from too many seed values to the watershed transform� Various seed reduction techniques include smoothing�thresholding� �lling� manual selection and random seeding� �See original image �lena inFigure �� The randomly seeded watersheds are weighted by the number of times theywere identi�ed� This image used �� seedings with a density of � in �� seeds�

the segmentation� In the case of the watershed and ridge tracing algorithms the latter

operation is accomplished by reducing the number of seed points�

Oversegmentation� as illustrated in �gure ��a�� results from too many seed points� This

can arise from noise and quantisation in the image� both of which result in large numbers

of spurious minima and maxima which in turn contribute to the clutter in these images�

Rather than perform complicated postprocessing� it is preferable to preprocess the

energy map to reduce the number of seed and marker points�

A number of options are available� the most obvious being a smoothing operation but this

reduces the resolution� Alternatives that have been investigated include lower threshold

ing� �lling shallow minima� heuristic seed selection� and random seeding�

To this point in the processing� apart from the �lter size in the energy calculation� no

arbitrary constants have been used� A preprocessing step must however reject seeds


based on some criterion so we attempt to select a criterion that is unbiased� robust and

easy to apply�

Smoothing

A certain degree of bandwidth limiting �ltering has already been applied during the

energy calculation phase� Further smoothing will continue to reduce the noise and �ne

detail present in the energy signal� This reduced resolution of the features depends on

the size of the feature� rather than its signi�cance� and can alter the actual location of

the energy peak associated with a feature� As illustrated in �gure ��b� the segmentation

has been reduced but the structure in the image has not been enhanced�

Lower thresholding

Applying a lower threshold� see �gure ��c�� to the image signi�cantly reduces the number

of spurious minima and maxima because the energy map is quite sparse� Many energy

values are extremely low and the peaks quite sharp� However� there is a problem in

that the energy function can contain plateau areas due to illumination di�erences in the

image� Lower thresholding does not e�ect these areas� and so does not reduce the small

�uctuations superimposed on the larger signal� The D�C� o�set� of small �uctuations on

the plateaux� means that a lower threshold is selective in its operation and biased against

minima in lower energy areas�

Filling shallow minima

Filling shallow minima� see �gure ��d�� produces results similar to using a lower thresh

old but without the bias problem� In this process each minimum is �lled to a given depth

which causes many shallow minima to run together� The �lling is illustrated in �gure ��

It can be e�ciently implemented using the �ooding paradigm and has proved to be a

most successful solution to oversegmentation� Because the peaks in an energy image are

generally quite sharp� �ooding to an arbitrary value of around � percent� or less� of the

maximum energy value has proved suitable to produce what is intuitively a reasonable

segmentation of the image�


Heuristic seed selection

Various ideas for selecting seeds based on previous frame or multiscale considerations

were considered and investigated by manually selecting such seeds� The results� see

�gure ��e�� of this procedure were rather disappointing� it actually being rather di�cult

to predetermine the e�ect of a seed at a particular place�

Random seeding

A surprisingly good result� see �gure ��f�� is obtained by a random selection technique

initially suggested by a colleague �� Random seeds are generated and the watershed

determined� This is repeated a number of times and a sum is kept� for each pixel� of the

number of times a pixel is included in the watershed�

The resulting segmentation can then be thresholded � a value around �� percent was

found to be reasonable� The resulting ridges are not connected and� because it is a water

shed� promontories have been lost� In addition it is unfortunate that there are a number

of arbitrary constants involved� namely� the seed density� the number of repetitions and

the threshold�

As expected it was found that the seed density determined the average size of the segments

and the number of repetitions simply found more of the less signi�cant ridges� Also having

to repeat the whole process a number of times is undesirable in terms of e�ciency�

This technique worked surprisingly well with images containing clouds�

�� Watershed vs ridge tracing

Figure �� provides an illustration of the capabilities of these algorithms and a comparison

between the watershed and ridge tracing results�

Three images are presented� of which one is a synthetic �test image with no noise but

some rather low level intensity transitions� The �lena test image is a fairly clean high

quality natural image showing some areas of texture and others of sharp edges� The third

�A block image is a noisy low contrast image� that also contains a ghosting artifact from

the imaging system which is detected by the local energy processing�

�My thanks to Matthew McDonald who suggested this technique after a seminar�


�test original image �lena original image �A block original image

�test energy map �lena energy map �A block energy map

�test watershed �lena watershed �A block watershed

�test ridge trace �lena ridge trace �A block ridge trace

Figure �� Original� energy map� watershed and ridge traced versions of the �test � �lena and �A block images� Fill shallow minima �� used for watershed and ridge images�


Figure �� Ridges detected in phase congruency images using the ridge tracing technique described in this chapter� The phase congruency algorithm normalises the localsigni�cance of a feature which enhances the response in relatively feature free areas ofthe image necessitating the use of a deeper �ll value� A value of �� has been selectedrather than the �� value that I have generally used�

Particular features which highlight the di�erence between watershed and ridge tracing

are the line segments in the �test image and the mouth region of the �lena image�

Phase congruency and ridge tracing

This thesis mainly considers tracing the ridges of a local energy image and has been

limited to use only the recursive �lter techniques introduced in chapters � and �� The

ridge tracing technique is also applicable to phase congruency images which are closely

related to local energy� Since phase congruency is proportional to local energy the same

sorts of key points are identi�ed by local maxima and the ridge tracing algorithm can be

used�

Figure �� shows the results of applying the same ridge tracing technique to phase con

gruency images rather than the local energy maps that have generally been considered

in this thesis� These images have been generated using the same shallow �ll process to

reduce segmentation for the purposes of equal comparison� however since phase congru

ency is a normalised measure of feature signi�cance a simple low threshold may well be

su�cient or superior�

Figure �� shows the ridges detected using a nonmaximal suppression and hysterisis


Figure �� Ridges detected� by Kovesi �� in phase congruency images using nonmaximal suppression and hysterisis thresholding�

thresholding technique� Whilst the images appear �cleaner many contours are not con

tinuous and there are gaps at most feature junctions� The pair of threshold values were

selected by hand by Kovesi�

�� Conclusion

The �ooding paradigm has been shown to provide an e�cient and e�ective implemen

tation framework for various image segmentation algorithms� In combination with the

local energy transform we have developed an excellent image feature detection scheme�

Although an arbitrary �ll value is required to reduce the oversegmentation� the technique

is not sensitive to this value and the normalised scaling of the energy map� to the range �

to �� means that one value suits most images� Use of the local energy model provides

high feature sensitivity and good noise immunity�

Chapter �

Surface Tracing

This chapter extends the ridge tracing algorithm to trace the surfaces� tendrils and iso

lated points of higher energy in threedimensional local energy density data� In three

dimensions the topology preservation rule of simply not disconnecting neighbours is insuf

�cient since the algorithm must not remove or insert tunnels through the neighbourhood�

Otherwise the surface tracing algorithm proceeds as for twodimensions� starting from

local minima eroding upwards to higher density levels� A shallow �ll operation can be

used to avoid oversegmentation as in the twodimensional case and this is implemented

in essentially the same manner as described in Chapter �� using a �ooding paradigm and

ordered queues of pixels�

�� Surface tracing algorithm

The surface tracing algorithm described here involves a skeletonisation of the local energy

density data� This skeletonisation by erosion is limited by anchor points which maintain

the structure of the high density features� The erosion is performed using an e�cient

�ooding technique that uses ordered queues of pixels so as to only consider points that

are currently on the shoreline� The local energy model locates features at points in the

image where the various Fourier components of the image are maximally in phase� Since

local energy is proportional to phase congruency the feature locations coincide with high

values of local energy� This surface tracing algorithm �nds these peaks� ridges and sheets

of higher energy values in the volumetric energy map�

The steps involved in the surface tracing are as follows� Firstly a local energy map is

��

CHAPTER �� SURFACE TRACING ��

18-connected6-connected 26-connected

Figure �� Illustrating the voxels corresponding to � connected� �� connected and the �connected neighbourhoods�

constructed by convolving the data with a number of quadrature pair �lters� at various

scales and various orientations� The resulting energy measure is then preprocessed to

reduce the amount of feature segmentation by �lling shallow regions� We then �nd all of

the points of local maxima and minima� A �ooding process is used to erode the data set�

starting from the minima seed points� Erosion is limited by homotopy considerations and

by the nonremovable maxima anchor points� Homotopy refers to the connectivity and

hole enclosure of a set� A post processing pruning phase is then applied to remove some

extraneous points which were left due to the order of processing� With threedimensional

data� two erosion stages are required� the �rst determines the outline of the surface

structures� and the second marks all of the tendrils� isolated points and surfaces of high

values present in the local energy map�

This surface tracing technique is ideally suited for use in conjunction with the local

energy model because it does not require feature orientation information and works well

with the anchor points identi�ed by local maxima in the energy map� The technique is

straight forward to program and computationally e�cient�

�� Features in three�dimensions

In three dimensions features are located� as in two dimensions� at the points of local

maxima and along connecting ridges� However in three dimensions a new type of maxi

mum must also be considered� namely the �lm of higher energy values� resulting from a

twodimensional step function in the density of the volume�


Features can be classi�ed according to how many dimensions are required to identify

them� For example a surface is a onedimensional feature� the edge formed where two

planes meet or along the edge of a cube is a twodimensional feature� A feature cor

responding to a line intersecting a surface or vertex or a cube could be classi�ed as a

full threedimensional feature� This is analogous to corners and junctions being called

twodimensional features in an image�

The surface tracing problem does not simply entail �nding surfaces� it incorporates lines

and points as well� Such features would form by tendrils and spots� Gradient based

surface detectors just don t cope with these features� The local energy model highlights

all these features but once again leaves us with the problem of automatically tracing out

the high value pixels!voxels�

The ridge tracing technique developed for twodimensions can be extended to three

dimensions but there are a few considerations to be made� A lookup table de�ning the

homotopy of a point would need to have of the order of �� entries� so homotopy must be

calculated for each voxel as it is considered� Homotopy requires two tests rather than the

one necessary in twodimensions� Anchors in threedimensions need to be more than just

the points of local maxima� as these points identify threedimensional features but we

also need anchors to delineate the boundaries of surfaces� Artifacts arising from three

dimensional gradients in the energy map mean that feature locations can be distorted near

junctions with planes� but using ��connectivity for boundaries avoids these artifacts�

�� Homotopy in three�dimensions

Homotopic thinning �� of threedimensional sets has been considered by several re

searchers �� and is surveyed by Kong and Rosenfeld �� In multiple dimensions�

parallel removal criteria are of particular interest �� but the general considerations are

the same� Homotopic removal requires that there be no change to the connectivity of the

sets and no alteration of the number of tunnels through the sets� Three tests are used to

determine whether a voxel is removable�

� Connection� removal must not disconnect foreground voxels that were previously

connected�

� Separation� removal must not connect previously disconnected background voxels�


6 connected

Central voxel

Diagonal voxels

Bit pattern representation

27 neighbourhood voxels

Hexadecimal 0x77707570777

0x01002520010 6 connected excluding central voxel

26 connected excluding central voxel

Figure �� neighbourhood voxels �exploded view� showing bit pattern used to representthe neighbourhood� illustrating both the �connected and ��connected case� The bitunder examination� in the center� is marked indicating its exclusion from the propagation�

� Euler Characteristic� removal must not change the Euler characteristic� of the

neighbourhood� The Euler characteristic �� is a topological invariant which for a

threedimensional set S is�

��S� � Ncomponents �Ncavities �Ntunnels� ��

where Nfeature is the number of such features in the set�

The tests are performed locally and consider only a voxel and its immediate neighbours�

In practice the connection and separation tests� when performed in conjunction with the

Euler test� can be shown to be equivalent� so only one need be performed� Tokawaki et

al� �� present a means of computing the change in Euler characteristic resulting from

the deletion of a voxel�

The tests use complementary connectivities� ��connected for connection and Euler char

acteristic� and �connected for separation� Removability can be evaluated using bitwise

logic operations� A bit pattern representing the status of the neighbourhood is calculated


as illustrated in �gure �� To do a connection test� one bit is propagated ��connectivity�

through the foreground neighbours using shift and or operations� If after propagation�

there are still some foreground neighbours that have not been reached then the voxel

cannot be removed� Taking the complement of the bit pattern and using a �connected

propagation� the same process can be used for the separation test�

�� An alternative to the Euler characteristic

When dealing with a cubic lattice it can be shown that it is not necessary to calculate

the genus or number of holes� Instead only two numbers of connected components in the

neighbourhood need to be computed� These numbers are easily calculated using the bit

propagation technique described above�

A point may be removed� i�e� is a simple point� if and only if in the point s �

neighbourhood there is only one ��connected foreground component and one �connected

background component�

The one connected foreground and one background tests ensure that the point does not

connect of separate regions� The genus test is required to ensure no tunnels would be

introduced by removal� Consideration of the point s ��neighbourhood and all possible

con�gurations of an intersecting plane that would lead to a tunnel being introduced

should the point be removed indicates the involvement of one of the neighbourhood s

eight corner voxels� Excluding these voxels from the bit propagation means that the

homotopy preservation tests can be performed using two pit propagations� A proof of

this proposition is given by Bertrand and Malandain ��

�� Anchors in three�dimensions

In three dimensions the concept of anchor points needs to be reexamined� As illustrated

in �gure ��a�� the minimum connected sets that include the anchor points do not really

represent the full nature of the features� We need to preserve the edges of surfaces to

avoid them being eroded� The edges have high values but the pruning stage erodes them�

Allowing holes to develop� by ignoring the separation and Euler test� generates a wire

frame outline of the object� Surfaces are crossed by a tracery of lines but the edges

are de�nitely marked� The lines along the edges are assured because the edges have


�a� �b� �c�

Figure �� Skeletonisation of the threedimensional local energy density of a squaresurface with a line through it� a� When only the keypoints are anchored the surface iseroded� b� By allowing holes to develop as an initial skeletonisation progresses� a set ofanchor lines is produced� c� A �nal skeletonisation using the anchor lines and homotopicthinning preserves the full surface as well as line and point features�

higher values just as keypoints do in the twodimensional case� This is illustrated in

�gure ��b�� These points are used as anchor lines and are marked so as to be preserved�

The skeletonisation is then performed again without holes being allowed to develop and

the full surface is detected as expected� The full surface and intersecting line are shown

in �gure ��c��

Note that Eberly �� overcame this problem by having three classes of pixels while doing

an erosion� He di�erentiates between internal and boundary pixels that are part of a

surface� With local energy this di�erentiation is not necessary because the energy map

indicates which pixels are on the boundary with higher energy values� Once these tendrils

are traced out� they anchor the surfaces that they delineate�

�� Algorithm

Whilst the surface tracing algorithm is essentially the same as described in Chapter �

there are several comments and observations that can be made about the individual

stages as implemented for this thesis� The following sections discuss the various stages

of processing in approximately the order that they are applied�


�� Normalisation and quantisation

The energy map is usually calculated using �oating point arithmetic but implementation

of the ordered queues used in the �ooding algorithm requires quantised levels� usually �

to �� We perform two operations on the energy map data� �rst we normalise it to a

dynamic range of �� then we quantise the �oating point values to small integers in the

range � to �� Energy is always a positive value so no consideration of negative numbers

is needed�

Peaks in the energy map tend to be quite sharp so the process of quantisation does

not shift the location of detected features� However quantisation does slightly reduce

the segmentation present in the image by merging many of the shallow regions together�

Finally� by normalising the energy values� contrast di�erences between images are reduced

which makes the �lling or thresholding value applied in the next stage quite robust� The

threshold value depends on the actual �lters� their orientations and the number of scales

which are combined as well as some consideration of the image characteristics�

�� Preprocess �lling

Tracing the peaks and ridges of an image results in a separate region for each seed point�

The ridges are a superset of the watershed� If all local minima of the image are used

a very large number of small regions are found� and this oversegmentation obscures

the more signi�cant feature structure that is present in the image� Various options are

available to reduce the segmentation such as smoothing� thresholding and a �lling of local

minima to some depth� These were discussed in section ��

The �lling algorithm developed for twodimensional images �section �� is directly

applicable to threedimensional images and is used here� For the �lling stage� regional

connectivity of � connectedness is used�

�� Seeds and anchors

This surface tracing algorithm uses points of local minima as seeds for the �ooding process

and points of local maxima to anchor the extremities of features from erosion� The local

energy pro�le has sharp peaks and large� quite �at regions of low energy values� Regions

of texture in the image produce �at plateaux�


The algorithm used to mark the minima and maxima was chosen to be reasonably simple

to program and similar in form to the rest of the processing� We scan the image in a

raster fashion examining a pixel s neighbours to determine if that pixel is a minimum

�maximum�� If all neighbours are greater �less� than or equal to the pixel then we

recursively examine the neighbours of equal value� Pixels and neighbours are labelled

whenever possible to improve e�ciency�

As a consequence of the preprocess �lling� the minima tend to be large lakes in which

all points are marked as minima� Maxima on the other hand tend to be sharp isolated

peaks� It is possible for these to be regions� in which case they should be skeletonised

using a technique such as Vincent s �� algorithm� In practice such regions are rare and

simply taking one point in the region of maxima will provide a su�cient anchor� Taking

only one point ensures a thinned result�

�� Pruning and the borders

As discussed in section �� after an erosion stage has been performed we must repeat

edly examine all of the retained pixels to determine if any can now be removed� This

scanning is repeated until no removable pixels are found� The list is reasonably short

and the process� which is called pruning� is not too ine�cient�

The skeleton that results if the border pixels outside the image are considered to be

included is called the geodesic skeleton� This may be applicable if the image is a sub

section of a larger image� This does tend to produce an artifact whereby tendrils reach

out from the features to the border along lines of energy only slightly higher than the

surroundings� Consequently it pays to exclude the border pixels�

Spatial domain �ltering� as described in Chapter �� and used here� provides phase infor

mation right up to the border� whereas frequency domain �ltering needs to have a wide

border at the edge of the image to avoid aliasing in the Fourier transforms�

We include the borders in the erosion phase then exclude them for the pruning phase�

This combination of inclusion and exclusion produces the fewest extraneous artifacts�


�� Flooding the energy �eld

The �ooding propagates from the seed points outwards and upwards to higher energy

values� The sequence of processing is determined using ordered queues to hold reference

to the pixels until that level of energy is being processed� Pixels are removed if they

are not marked as anchor points and can be removed without disturbing the homotopy

of the set of included pixels� which includes the pixels of higher value that have not yet

been examined�

The �rst �ooding stage is performed with the homotopy test modi�ed to allow tunnels

to develop in surfaces or �lms of energy� This means that the surface boundaries are left

intact and correctly located� Surface features are de�ned by a tracery of �laments which

lie within the surface and have locations which are determined by where the processing

�rst perforated the surface� The indeterminance of the order of processing pixels at some

given energy level means that these �laments have a rather random structure�

�� Re�ooding the energy �eld

The �rst stage �ooding �see above� results in �laments that trace out the structure of

the image features� Having marked all these points as anchors� we then start the �ooding

again but this time use the true homotopy test that preserves the connectivity and tunnels

through the set�

Again the border is considered to be included during the �ooding phase and excluded

during the subsequent pruning phase�

�� Results

Figures �� and � show the results of the surface tracing algorithm� as described

above� applied to energy maps of volumetric data that have been calculated using the

recursive �lters described and illustrated in Chapter �� For each image� energy was

calculated over a range of scales using ten evenly spaced orientations� The only arbitrary

parameter used is the �ll factor�

A deeper �ll value was required for the �planes image in �gure �� because the energy

values in the region between the two planes and the line prevented the line being detected


Figure �� A threedimensional surfacerendering of the feature locations identi�ed by the recursively implemented localenergy and threedimensional ridge tracing of a synthetic image �square � Filldepth is ��

Figure �� A threedimensional surfacerendering of the feature locations identi�ed by the recursively implemented localenergy and threedimensional ridge tracing of a synthetic image �planes � Filldepth is ��

Figure �� A threedimensional surfacerendering of the feature locations identi�ed by the recursively implemented localenergy and threedimensional ridge tracing of a synthetic image �sphere � Filldepth is �� One quadrant of the imagehas been removed to expose the internalstructure of detected features�


Figure �� A threedimensional surfacerendering of the feature locations identi�ed by the recursively implemented localenergy and threedimensional ridge tracing of a CT image of a human neck� Filldepth is ��

as a separate feature� This is a function of the small image and relatively broad �lters�

Figure �� correctly detects the surface of the enclosed cube and the intersection planes

through it� The cube is solid in the original image� The planes� cube and sphere are

each constructed from di�erent intensity voxels� In order to avoid problems tracing the

planes right up to the boundaries� a blank �� pixel border has been added to this image

which necessitated the removal of the background shading that was present in the image�

as illustrated in Chapter �� This does not fundamentally change the energy result that

is presented there�

Figure � shows the surface traced result for a natural image� A deeper �ll value is

required to reduce segmentation due to texture and noise in the data� The local energy

feature detection has identi�ed several di�erent structures in the image� particularly skin�

bone and cartilage� Other features in soft tissue have been ignored due to the �ll depth�

�� Discussion

The twodimensional renderings of the surface tracings do not highlight the extent of the

threedimensional structure that has been identi�ed� An interactive rendering program

is required� preferably with the ability to peel away or select layers for display� The

connected and thinned nature of the surface traced result would lend itself to such a

process�

The �square and �planes renderings su�er a little from the discrepancy involved with

precise feature location in synthetic images� The images are constructed with step edges


and transitions perfectly aligned with pixel boundaries� however the feature detector uses

a pixel to indicate the feature location which is ambiguous by one half a pixel� This can

lead to detection of apparently jagged edges� Figure �� shows evidence of extra erosion

of the edges of the diagonal plane� This is probably an artifact of aliasing in the synthetic

image� This artifact was not evident when Morlet wavelets were used to calculate the

local energy�

The construction of synthetic images with pixel aligned structures introduces perfect step

edges hence maximum bandwidth features in the images� The advantage of this construc

tion is simplicity and precise location� albeit ambiguous� of the structures� Test images

could perhaps be constructed analytically so as to better represent sampled images� The

increasing use of fuzzy fonts for computer text display is an example of a similar problem�

Further threedimensional images calculated using an ablative skeletonisation technique

and local energy are presented in papers by Pudney �� and Pudney et al� �� where

energy is calculated using Morlet wavelets and frequency domain convolution�

�� Conclusion

There are two advantages to this local energy based surface detection technique� Firstly

the local energy model performs better than gradient based detectors such as Deriche s

threedimensional edge detector �� or the threedimensional Sobel edge operator pro

posed by Zucker and Hummel �� Secondly� using the �ooding paradigm to identify

the peaks� ridges and surfaces is superior to using a nonmaximal suppression and hys

teresis thresholding technique �� with its associated pair of thresholds and requirement

for orientation information� The technique described in this chapter �nds all classes of

features� is applicable to many scales� and performs well at junctions and complex feature

structures�

Chapter �

Conclusion

�� Summary

This thesis has looked at two aspects of image feature detection using the local energy

model of image features� Firstly� having introduced the local energy theory it exam

ines the �ltering required to calculate a measure of local energy� In particular the use

of recursive digital �lters is developed from a combination of Freeman s steerable �lter

techniques and Deriche s approach to recursive �lter design� This recursive digital �lter

ing leads to a new method for spatial domain calculation of local energy� The method is

demonstrated to be applicable in two and threedimensional image analysis�

The second aspect of image feature detection is to solve the problem of identifying the

locations of high local energy� These ridges of energy trace the location of features in

the original image� The solution is an algorithm adapted from the concept of watersheds

and Vincent s binary image skeletonisation algorithms� The solution utilises a particular

characteristic of the local energy model to provide anchor points which preserve many of

the features that are eroded by the watershed transform�

The ridge tracing algorithm uses a �ooding paradigm whereby processing starts at seed

points and progresses outwards� This results in an e�cient processing of the image pixels�

avoiding the repeated scanning of the data set that is inherent in the more common raster

scan based algorithms�

Using the same �ooding paradigm an initial shallow �ll operation is developed which

takes the place of the simple threshold so often encountered in image processing� The

��

CHAPTER �� CONCLUSION ��

�ll mechanism is well suited to use with local energy because it is the relative height of

energy peaks that is related to feature signi�cance rather than the absolute height�

The shallow �ll operation is applied after the energy map is normalised and it is found

that the depth parameter forms a robust mechanism to select the level of �ne detail or

noise to exclude from the ridge tracing�

The ridge tracing algorithm is then extended to threedimensions maintaining the same

e�ciency bene�ts and continuing the use of a �ll value to reduce the degree of segmen

tation� The algorithm detects points� tendril and surface features in threedimensions

giving a well connected and thinned result�

Homotopy in threedimensions is somewhat more complicated than in twodimensions and

a novel approach is proposed to generate a suitable set of anchors for surface detection�

This involves two stages of �ooding� In the �rst �ooding� holes are allowed to pierce

the surfaces and so generate a nice tracery of �laments which outline and cross the

surfaces� The second stage� disallowing holes� uses these anchor lines to preserve the full

threedimensional structure�

�� Contributions

This thesis makes two main contributions� The �rst is the development of a recursive

spatial domain implementation for calculating local energy in two and threedimensions

and at various scales� The second is a new approach to the problem of tracing the ridges

and surfaces of higher energy values that coincide with image features� Both solutions

build on existing techniques with several insights and novel contributions including the

following�

� I have tried to make the review of local energy presented in chapter � a useful

and informative pr$ecis on the subject� Its discussion of multidimensional oriented

energy brings together information that has previously been implied or reported as

an experimental method rather than in its own right�

� The use of recursive spatial domain �ltering to calculate local energy is novel� Local

energy is usually calculated using the frequency domain or through convolution with

�xed bandwidth masks�


� The combination of steerability and the scalable recursive implementation for the

�lters is novel� Tabulation of the �lter design parameters should prove useful for

their subsequent incorporation into other computer programs�

� Extension of the watershed transform to ridges would appear to be unique� as is

the extension� albeit straightforward� of these algorithms to three dimensions�

� The observation that maxima in the local energy map could be used to anchor

ablative skeletonisation techniques is unique�

� In the extension of ridge tracing to the threedimensional surface tracing algorithm�

the use of modi�ed homotopic removal to develop a set of anchorlines is de�nitely

original�

� The introduction of a shallow �ll parameter to reduce oversegmentation was original

when �rst reported ��

�� Further work

The subject of this thesis has developed from a close examination of the initial stages

of an image compression scheme �� with a view to an improved MPEG ��

encoding algorithm� Spatial domain �ltering using various scales was considered for

a foveated active image analysis� similar to that described by Zabrodsky �� where

di�erent regions of the image could be considered at di�erent scales� Under such a

regime the spatial domain techniques described in this thesis may be expected to prove

superior to the frequency domain �FFT� techniques�

More work could be done to investigate the design of recursive wavelet �lters or log Gabor

�lters which would have better frequency characteristics for the local energy calculations

than the derivative of Gaussian functions that have been used here� The question of

how best to combine the energy results across scale and orientation also needs further

attention�

The ridge and surface tracing is very reliant on the anchor points provided by local

energy making it inapplicable to many general ridge analysis tasks� Separate provision

of keypoint information from an algorithm such as Robbins twodimensional feature


detector �� could be investigated� particularly in the context of some multimodal ap

plications�

The output of the surface tracing algorithm could be investigated for surface rendering

applications using triangular segmentation� Since the detected voxels are well connected

and thinned� various representations based on surface junctions� surface edges and trel

lised lines across the membranes could pro�tably be utilised as a data reduction or visual

rendering mechanism�

Kovesi s �� development of the phase congruency calculation techniques should be ex

plored� particularly in conjunction with the ridge and surface tracing algorithm� Phase

congruency is well suited to the ridge tracing since� like local energy� its local maxima

naturally provide keypoints which can be used as anchors�

This author feels that a threedimensional phase congruency algorithm in conjunction

with this practical and e�cient surface tracing would form a robust and general image

and volume analysis technique that should �nd widespread use in the future�

Bibliography

�� C� J� Avis� A visual information processing system� Technical Report !�� The

University of Western Australia� ��

�� Y� K� Aw� Exploring and Utilising Features in Natural Images� PhD thesis� Depart

ment of Computer Science� The University of Western Australia� ��

�� Y� K� Aw� R� Owens� and J� Ross� A catalogue of �D features in natural images�

Computer Vision� Graphics and Image Processing� �� Mar� ��

�� G� Bertrand and G� Malandain� A new characterization of threedimensional simple

points� Pattern Recognition Letters� ��

�� S� Beucher and F� Meyer� The morphological approach to segmentation� The wa

tershed transformation� In Dougherty �� chapter ��

�� R� N� Bracewell� The Fourier Transform and Its Applications� McGrawHill� ��

�� L� Brown� editor� The New Shorter Oxford English Dictionary� Oxford� ��

�� J� Canny� A computational approach to edge detection� IEEE Transactions on

Pattern Analysis and Machine Intelligence� �� Nov� ��

�� J� F� Canny� Finding edges and lines in images� Tr�� MIT AI Lab��

�� W� G� Chinn and N� E� Steenrod� First Concepts of Topology� Random House� ��

�� I� Daubechies� The wavelet transform� timefrequency localization and signal anal

ysis� IEEE Trans� Information Theory� ��

�� J� E� Dennis� D� M� Gay� and R� E� Welsch� An adaptive nonlinear leastsquares

algorithm� ACM Transactions on Mathematical Software� �� Sept� ��

��

BIBLIOGRAPHY ��

�� R� Deriche� Separable recursive �ltering for e�cient multiscale edge detection� In

Proc� International Workshop on Machine Vision and Machine Intelligence� pages

� �� Feb� ��

�� R� Deriche� Using Canny s criteria to derive a recursively implemented optimal edge

detector� The International Journal of Computer Vision� �� May ��

�� R� Deriche� Fast algorithms for lowlevel vision� IEEE Transactions on Pattern

Analysis and Machine Intelligence� �� Jan� ��

�� R� Deriche� Recursively implementing the Gaussian and its derivatives� Rapports

de Recherche �� INRIA� ��

�� E� Dougherty� editor� Mathematical Morphology in Image Processing� Marcel Dekker�

Inc�� New York� ��

�� D� Eberly� Ridges in Image and Data Analysis� Kluwer Academic Publishers� ��

�� D� H� Eberly� Geometric Methods for Analysis of Ridges in N�Dimensional Images�

PhD thesis� Department of Computer Science� The University of North Carolina at

Chapel Hill� ��

�� D� J� Field� Relations between the statistics of natural images and the response

properties of cortical cells� Journal of the Optical Society of America A� ��

�� Dec� ��

�� R� Fletcher and M� J� D� Powell� A rapidly convergent descent method for mini

mization� Computer Journal� �� July ��

�� E� A� Fox� Digital multimedia systems� Comm� of the ACM� �� Apr�

��

�� W� T� Freeman� Steerable Filters and Local Analysis of Image Structure� PhD thesis�

Architecture and Planning� MIT� June ��

�� W� T� Freeman and E� H� Adelson� The design and use of steerable �lters� IEEE

Transactions on Pattern Analysis and Machine Intelligence� �� Sept�

��

BIBLIOGRAPHY ��

�� R� A� Gabel and R� A� Roberts� Signals and Linear Systems� Wiley� international

edition� ��

�� J� Gao and J� S� Jin� Improving LoG �ltering using thirdorder recursive �lters� In

DICTA�� Digital Imaging Computing Techniques and Applications� pages ��

�� Brisbane� Queensland� December ��

�� J� M� Gauch and S� M� Pizer� Multiresolution analysis of ridges and valleys in grey

scale images� IEEE Transactions on Pattern Analysis and Machine Intelligence�

�� June ��

�� C� R� Giardina and E� R� Dougherty� Morphological Methods in Image and Signal

Processing� Prentice Hall� ��

�� A� Grossman and J� Morlet� Decomposition of Hardy functions into square integrable

wavelets of constant shape� SIAM J� Mathematical Analysis� ��

�� R� M� Haralick� Ridges and valleys on digital images� Computer Vision� Graphics

and Image Processing� ��

�� F� Heitger� L� Rosenthaler� R� von der Heydt� E� Peterhans� and O� Kubler� Sim

ulation of neural contour mechanisms� From simple to endstopped cells� Vision

Research� ��

�� O� Hendricsson and F� Heitger� The role of keypoints in �nding contours� In �rd

European Conference on Computer Vision� volume �� pages �� Stockholm�

Sweden� ��

�� F� B� Hildebrand� Introduction to Numerical Analysis� McGrawHill� New York�

��

�� F� B� Hildebrand� Introduction to Numerical Analysis � nd edition� McGrawHill�

New York� ��

�� J� S� Jin and Y� Gao� Recursive implementation of LoG �ltering� Real�Time Imaging�

��

�� Y� Kawata� N� Niki� and T� Kumuzaki� Threedimensional imaging of blood vessels

using cone beam CT� In IEEE Internations Conference on Image Processing� pages

��

BIBLIOGRAPHY ��

�� J� J� Koenderink� Solid Shape� MIT Press� ��

�� T� Y� Kong and A� Rosenfeld� Digital topology� Introduction and survey� Computer

Vision� Graphics and Image Processing� ��

�� P� Kovesi� A dimensionless measure of edge signi�cance from phase congruency cal

culated via wavelets� In Image and Vision Computing NZ�� pages � �� Auckland�

New Zealand� Aug� �� IRL�

�� P� Kovesi� Invariant Measures of Image Features From Phase Information� PhD

thesis� Department of Psychology� The University of Western Australia� ��

�� P� Kovesi� Image features from phase congruency� Videre� to appear�

�� M� Kunt� A� Ikonomopoulos� and M� Kocher� Secondgeneration imagecoding tech

niques� Proc� of the IEEE� �� Apr� ��

�� C� Lanczos� Applied Analysis� Sir Isaac Pitman and Sons Ltd�� London� ��

�� J� S� Lim� Two�Dimensional Signal and Image Processing� Prentice Hall� ��

�� S� Mallat� A theory for multiresolution signal decomposition� The wavelet represen

tation� IEEE Transactions on Pattern Analysis and Machine Intelligence� ��

��

�� D� Marr� Vision� Freeman� San Francisco� ��

�� D� Marr and E� C� Hildreth� Theory of edge detection� Proc� R� Soc� London B�

��

�� Y� Meyer� Orthonormal Wavelets� pages �� SpringerVerlag� Marseille� ��

�� O� Monga and S� Benayoun� Using partial derivatives of �D images to extract

typical surface features� In �rd Annual Conf� AI� Simulation and Planning in High

Automony Systems� pages �� Perth� Australia� July ��

�� O� Monga� R� Deriche� G� Malandain� and J� P� Cocquerez� �D edge detection

by separable recursive �ltering and edge closing� In ��th Int� Conf� on Pattern

Recognition� pages �� Atlantic City N�J�� June ��

BIBLIOGRAPHY ��

�� O� Monga� R� Deriche� G� Malandain� and J� P� Cocquerez� Recursive �ltering

and edge tracking� Two primary tools for �D edge detection� Image and Vision

Computing� �� Aug� ��

�� J� Morlet� G� Arens� E� Foureau� and D� Giard� Wave propagation and sampling

theory�Part II� Sampling theory and complex waves� Geophysics� ��

��

�� M� Morrone and R� Owens� Feature detection from local energy� Pattern Recognition

Letters� �� Dec� ��

�� M� C� Morrone and D� C� Burr� Feature detection in human vision� A phase depen

dent energy model� Proc� R� Soc� London� B� �� Nov� ��

�� M� C� Morrone� J� R� Ross� D� C� Burr� and R� A� Owens� Mach bands are phase

dependent� Nature� ��

�� E� J� Muth� Transform Methods With applications to Engineering and Operations

Research� PrenticeHall� ��

�� H� J� Nussbaumer� Fast Fourier Transforms and Convolution Algorithms� Springer

Verlag� ��

�� R� Owens� Computational models for biological vision� In Australian Winter School

in Computer Science� Magnetic Island� QLD Australia� July ��

�� R� Owens� Feature detection via phase congruency� Computer Vision IT�� Lecture

�� Department of Computer Science� The University of Western Australia� ��

�� R� A� Owens� S� Venkatesh� and J� R� Ross� Edge detection is a projection� Pattern

Recognition Letters� ��

�� P� Perona� Steerablescalable kernels for edge detection and junction analysis� In nd

European Conference on Computer Vision� pages � �� Santa Margherita Ligure�

Italy� ��

�� I� Pohl� C for C programmers� Benjamin Cummings� Inc��

�� W� H� Press� S� A� Teukolsky� W� T� Vetterling� and B� P� Flannery� Nonlinear

models� In Numerical Recipes in C �� chapter �� pages ��

BIBLIOGRAPHY ��

�� W� H� Press� S� A� Teukolsky� W� T� Vetterling� and B� P� Flannery� Numerical

Recipes in C The Art of Scienti�c Computing� Cambridge University Press� UK�

second edition� ��

�� C� Pudney� P� Kovesi� and B� Robbins� A �D local energy surface detector for confo

cal microscope images� InANZIIS�� Third Australian and New Zealand Conference

on Intelligent Information Systems� pages � �� Perth� Western Australia� ��

�� C� Pudney� M� Robins� B� Robbins� and P� Kovesi� Surface detection in �D confocal

microscope images via local energy and ridge tracing� Journal of Computer�Assisted

Microscopy� ��

�� C� J� Pudney� Feature detection in volumetric images via local energy and ordered

thinning� In Proceedings of MIUA�� Medical Image Understanding and Analysis�

pages �� Leeds� U�K�� July ��

�� C� J� Pudney� P� D� Kovesi� and D� Stretch� Surface detection using local energy

for �D confocal microscope images� In Image Analysis �� pages � �� Fremantle�

September ��

�� K� R� Rao and J� J� Hwang� Techniques and Standards for Image� Video and Audio

Coding� Prentice Hall� ��

�� T� R� Reed� T� Ebrahimi� F� Marqu$es� and M� Kunt� New generation methods for

the highcompression coding of digital image sequences� In H� Burkhardt� Y� Neuvo�

and J� C� Simon� editors� From Pixels to Features II� Elsvier Science Publishers�

��

�� B� Robbins and R� Owens� The �D local energy model� Technical Report ��!��

Department of Computer Science� The University of Western Australia� ��

�� B� Robbins and R� Owens� �D feature detection via local energy� Image and Vision

Computing� ��

�� B� J� Robbins� The Detection of D Image Features Using Local Energy� PhD thesis�

Department of Computer Science� The University of Western Australia� ��

BIBLIOGRAPHY ��

�� M� J� Robins� Local energy estimation using recursive �lters� In Image and Vision

Computing NZ�� pages �� Auckland� New Zealand� Aug� �� Industrial

Research Limited�

�� M� J� Robins� Flooding the energy �eld� In Western Australian Computer Science

Symposium� Claremont� W�A�� Sept� �� The University of Western Australia�

�� M� J� Robins� Surface tracing of local energy density values� In Proc� �� Depart�

ment of Computer Science Research Conference� The University of Western Aus

tralia� July ��

�� M� J� Robins� Steering the third derivative of Gaussian function and its Hilbert

transform� Technical Report �!�� Department of Computer Science� The University

of Western Australia� ��

�� C� Ronse� The phase congruence model for edge detection in twodimensional pic

tures� A mathematical study� http�!!dptinfo�ustrasbg�fr��!�ronse!� Universit$e

Louis Paster� Sept� ��

�� J� Serra� Image Analysis and Mathematical Morphology� Academic Press� ��

�� H� Stark and F� B� Tuteur� Modern Electrical Communications � Theory and

Systems� PrenticeHall International� ��

�� J� Toriwaki and S� Yokoi� Algorithms for skeletonizing threedimensional digitized

binary images� In SPIE Architecture and Algorithms for Digital Image Processing�

volume �� pages �� SPIE� August ��

�� J� Toriwaki� S� Yokoi� T� Yonekura� and T� Fukumura� Topological properties and

topologypreserving transformation of a threedimensional binary picture� In �th

International Conference on Pattern Recognition� pages ��

�� Y� S� Tsao and K� S� Fu� A parallel thinning algorithm for �D pictures� Computer

Vision� Graphics and Image Processing� ��

�� S� Venkatesh� A Study on Energy Based Models for the Detection and Classi�cation

of Image Features� PhD thesis� Department of Computer Science� The University of

Western Australia� ��

BIBLIOGRAPHY ��

�� S� Venkatesh� J� Cooper� and B� White� Properties of local energy and separability of

features� Technical Report ��!�� Department of Computer Science� The University

of Western Australia� ��

�� S� Venkatesh and R� Owens� An energy feature detection scheme� In The Interna�

tional Conference on Image Processing� pages �� Singapore� ��

�� L� Vincent� E�cient computation of various types of skeletons� SPIE Image Pro�

cessing� ��

�� L� Vincent� Morphological algorithms� In Dougherty �� chapter �

�� L� Vincent and P� Soille� Watersheds in digital spaces� An e�cient algorithm based

on immersion simulations� IEEE Transactions on Pattern Analysis and Machine

Intelligence� �� June ��

�� R� Watzel� K� Braun� A� Hess� H� Scheich� and W� Zuschratter� On the deletability

of points in �D thinning� In Image Analysis Applications and Computer Graphics�

volume LNCS �� pages �� ICSC �� Hong Kong� Dec� �� Springer�

�� Z� Xie� Multi�Scale Analysis and Texture Segmentation� PhD thesis� Department of

Engineering Science� Oxford University� ��

�� H� Zabrodsky and S� Peleg� Attentive transmission� J� Visual Communication and

Representation� �� Nov� ��

�� S� W� Zucker and R� A� Hummel� A threedimensional edge operator� IEEE Trans�

actions on Pattern Analysis and Machine Intelligence� �� May ��

Appendix A

Review of local energy �lter pairs

This appendix presents details of several �lter pairs that have been investigated for use

with the calculation of local energy� Section A�� and A�� are reproduced from the main

text of the thesis�

A�� and point masks

In her thesis �� Venkatesh investigated various �lters and their suitability for calculation

of local energy� After investigating Chebechev� Butterworth and Gaussian �lters she

showed that a pair of simple � or � point masks taken from a �rst and second derivative

of the Gaussian function when suitably normalised� formed a good feature detector� The

scale of the features detected is obviously quite restricted but for simple processing this

is a remarkably good feature detector�

The �lters are applied separately to columns and rows so the search for a maximum is

straight forward� For the � point mask� features are located at maxima and also at the

edge point of plateaux�

This simple system was used as the basis of an image compression technique �� that

Filter symmetric antisymmetric

� point �� p�� p�

� point �� Figure �� Simple �lter masks can be applied to the rows and columns of an image toestimate local energy� Features are located at local maxima in the rows and columns�

��

APPENDIX A� REVIEW OF LOCAL ENERGY FILTER PAIRS ��

characterises the image pro�le at feature points� A catalogue of feature types was devel

oped and a runlength transmission used to achieve compression� The system probably

su�ered from not considering the scale of features and the separate row and column

analysis� resulting in double treatment of two dimensional features�

A�� Derivatives of Gaussian

The derivative of the Gaussian function has been widely used as an edge detector for two

reasons� Firstly� Canny has shown that it is a good approximation to an optimally derived

step edge detection function� Secondly since di�erentiation highlights discontinuities and

changes in the signal� and moreover since di�erentiation is commutative over convolution�

both smoothing and di�erentiation can be performed by a single convolution with a

derivative of the Gaussian �lter�

The Gaussian function and its �rst and second derivatives are illustrated in �gure �� The

equations have been normalised and all scaling factors removed to highlight the structure

of the functions� The equations of the odd derivatives comprise an odd polynomial

multiplied by the Gaussian function and the even derivatives contain polynomials in

even powers of x�

When considering multidimensional use of these �lters the function s symmetry must be

considered� The �rst derivative of the Gaussian function is an odd function so it must

be applied in particular directions�

The second derivative of the Gaussian function is an even function which when applied

in two dimensions� forms the Laplacian operator� The idea is that zero crossings in the

second derivative correspond with peaks and troughs in the �rst derivative� These would

coincide with edges in the image� A problem with this operator is that it always produces

closed contours�

Jin and Gao �� provide an interesting implementation for this �lter by deriving a

recursive implementation of the Laplacian�

Another common Gaussian based image �lter is the di�erence of Gaussians �DoG�� This

�lter uses the fact that taking the di�erence of two suitably normalised Gaussian functions

of di�erent scales �� results in a function which closely approximates the Laplacian

operator� This is usually only used to reduce the level of computational complexity in


an image processing system�

Gaussian based quadrature pairs

First and second derivatives of the Gaussian are almost in quadrature and can be used

as an approximate quadrature pair� These functions were used by Venkatesh to calculate

local energy� Venkatesh �� also based pairs of small masks on these functions to derive ��

� and � point masks which can be used as a surprisingly good local energy based feature

detector� Over such a small scale the fact that these masks are not twodimensional does

not matter very much�

The polynomial nature of the Gaussian function and its derivatives has been utilised

by Freeman �� to numerically derive approximations to Hilbert transforms� and to

develop steerable kernels to implement rotated derivatives of Gaussian functions� These

techniques are explored in detail in Chapter ��

A�� Gabor functions

Gabor functions are often used in image analysis because of their limited spatial and

frequency spread� In contrast to the Gaussian� they have a band pass characteristic�

They also have the e�ect of not altering the phase information which is necessary for use

in local energy calculations� The Fourier transform of a Gabor function is a Gaussian

function o�set by the centre frequency of the Gabor�

Gabor functions are formed from a sine wave modulated by a Gaussian� The two functions

used for local energy analysis are� the oddsymmetric Gabor function�

Godd�x� � e�x�

�� sin��x��

and the evensymmetric Gabor function�

Geven�x� � e�x�

�� cos��x��

These functions have all the requirements of a quadrature pair� The choice of parameters

� and � determines the scaling and shape of the frequency responses� A � to � ratio of

one is commonly used� giving a bandwidth of approximately one octave�


A�� Modi�ed Gabor functions

If a wide bandwidth is required from a Gabor �lter� a signi�cant nonzero DC component

is manifest in the evensymmetric �lter� This nonzero mean in the spatial domain

destroys the quadrature nature required of the pair of �lters� In order to overcome this

Heitger et al� �� developed the StretchedGabor �or SGabor� �lters� These functions

have similar spatial �ltering properties to the Gabor functions but zero DC mean�

The SGabor functions are generated in the frequency domain and are given in polar

separable form to facilitate orientation selective multidimensional �lter design�

The oddsymmetric SGabor �lter is�

Godd�r� � e�r�

�� sin��r��r��

and the evensymmetric �lter�

Geven�r� � e�r�

�� cos��r��r��

where � is the spread� �� is the frequency at the origin and ��r� gives the frequency sweep

of the �lter� ��r� is given to be�

��r� �e��

x��

��

where is chosen numerically such that the evensymmetric SGabor function integrates

to zero�

An oriented twodimensional �lter is constructed using a power of a cosine function to

express the angular variation� The Fourier de�nition of the �lter in polar coordinates is�

F �r� �� G�r� � cos�m��

where �� determines the orientation� m is a positive integer determining the orientation

selectivity and G�r� is the even or oddsymmetric SGabor function de�ned above�

These �lters have been used by Ben Robbins in his recent thesis to calculate �D Local

Energy� Robbins suggests that an orientation selectivity of m � � over �� orientations


provides a suitably even response over all orientations� His choice of � is made such that

the Gaussian envelope is as broad as possible but still negligible at the Nyquist frequency�

A�� Log Gabor functions

The log Gabor function has a Gaussian transfer function when viewed on a logarithmic

frequency scale in contrast to the Gabor function which has a Gaussian transfer function

when viewed on a linear scale� Field �� points out that this transfer function more

closely matches the frequency characteristics he has measured for features in natural

images� The log Gabor function is also consistent with measurements of the mammalian

visual system ��

The log Gabor function is de�ned by its transfer function�

G � e

��log� ��

��

��log� ��

��

where �� is the �lter s centre frequency and � controls the bandwidth of the resultant

�lter� Log Gabor wavelets can be constructed by keeping the ratio ��

constant for varying

��

The log Gabor function has a singularity at the origin which means that an analytical

expression for it can not be constructed in the spatial domain� These �lters must therefore

be designed in the frequency domain and numerically constructed in the spatial domain

via an inverse Fourier transform� See Kovesi �� for examples and further discussion� The

quadrature pair is constructed in the same way from the Hilbert transform of the function

which is simply multiplication by isgn�� in the frequency domain� see Bracewell ��

A�� Morlet Wavelets

The Morlet wavelet is a complexvalued wavelet made up of a real sine function �odd�

and an imaginary cosine function �even�� both modulated by a Gaussian� Using the two

functions in quadrature we can determine the amplitude and phase of frequency compo

nents at a given spatial location� The Morlet wavelets are not orthogonal� which means

that the original signal can not be uniquely reconstructed from the wavelet coe�cients�


To calculate local energy from these wavelets it is necessary to obtain F �x� and H�x� by

convolution of the image with a bank of the odd and even Morlet wavelets�

F �x� �X

f�FreqI�x� �M even

f �x��

H�x� �X

f�FreqI�x� �Modd

f �x��

where I�x� is the original image� Meven�oddf �x� is an even or odd wavelet tuned to fre

quency f � Freq is the set of frequencies covered by the wavelets� and � denotes convolu

tion� Convolution is commutative over addition� so that

F �x� � I�x� �X

f�FreqM even

f �x��

H�x� � I�x� �X

f�FreqModd

f �x��

Oriented Energy

The local energy model is extended to multiple dimensions by calculating the one dimen

sional local energy at a number of orientations� evenly spaced to cover all directions� and

then summing the resulting energies� The process of combining oriented onedimensional

local energy has been used to detect features in two and three dimensions�

Pudney� together with the present author and others �� presents the following im

plementation for threedimensions that is nicely generalised for multidimensions� Their

�lters are banks of oriented Morlet wavelets� Each wavelet comprises a quadrature pair of

a onedimensional sine and cosine function oriented to a threedimensional direction� and

modulated by a threedimensional Gaussian� The spatial and frequency domain forms of

an evensymmetric �lter oriented along the unit vector v� and tuned for frequency f are

given by the following�

M evenf��v �x� �

�

�p��

e�kxk�

�� cos��vT � x��

M evenf��v �u� �

�

�

�e��ku�f�vk� � e��kuf�vk��


Figure �� A twodimensional odd Morlet wavelet� k � �� and a plot of the magnitude ofits Fourier transform�

respectively� with

� �k

f� ��

� � ��f� ��

The constant k controls the ratio between the �lter s width � and frequency f � The odd

�lter has a sine in place of the cosine in equation �� and is imaginary� The frequency

domain expression for the odd �lter is the di�erence of the two exponential terms in

equation �� The �lter is rescaled by changing f � Note that the �lters have amplitudes

that are di�erent to those of traditional Morlet wavelets� In equations �� and �� the

lower frequency �lters are attenuated compared with Morlet wavelets� This is done to

reduce blurring in the local energy result� Filters based on the traditional form of the

Morlet wavelet have been used previously ��

As stated above� local energy in multiple dimensions is computed as the sum of one

dimensional local energy values computed at a number of orientations� The set of orien

tations� Orient� is chosen to provide even coverage of the multidimensional space� That

is�

E�x� �X


where�

E�v �qF ��v�x� �H�

�v�x� ��


-1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

Figure �� Even and odd Morlet wavelets� with k � ��

F�v�x� � I�x� �Geven�v �x� ��

H�v�x� � I�x� �Godd�v

�x� ��

Geven�v �x� �

Xf�Freq

M evenf��v �x� ��

Godd�v �x� �

Xf�Freq

Moddf��v �x� ��

and I�x� is the multidimensional image data set�

These Morlet wavelets are easily constructed in the frequency domain where they can

also be rotated to calculate oriented energy values� A problem is that high frequency

�lters require a large frequency space in order to avoid problems with aliasing�

A� Steerable derivative of Gaussian �lters

Freeman describes the theory required to steer a function� f�x� y�� to an arbitrary angle�

�� using a set of basis functions� Furthermore� he derives a set of separable basis functions

which facilitates the numerical calculation of convolution with the function�

Freeman examines the constraints on functions that can be steered with a basis set formed

by rotated versions of the function itself and shows that a function of the form�

f ��x� y� � G�r�QN �x��

where G�r� is a radially symmetric function and QN �x�� is a polynomial of order N in


one variable� can be written as a weighted sum of N � � basis functions�

f ��x� y� �NXj��

kj��f�j �x� y��

and that an equivalent basis set can be determined where each basis function is separable�

The variable x� is derived by rotating the axis of abscissa by the angle �� i�e� x� �

x cos � � y sin ��

The local energy calculation uses a pair of �lters that are in quadrature with one another�

Starting with a derivative of the Gaussian function Freeman �ts a polynomial to its

Hilbert transform then determines a set of separable basis functions to create a steerable

�lter that is in quadrature with the initial derivative of Gaussian �lter�

He calculates the Hilbert transform of the function f�x� y� along the axis of abscissa by

numerical convolution with ��x � In the frequency domain the convolution is performed

by multiplication by i sgn�s�� Conversion to and from the frequency domain is performed

using an FFT routine�

The numerical Hilbert transform data series is then �tted to an even or odd polynomial of

order one greater than� and of opposite parity to� the original polynomial QN �x�� Fitting

a polynomial is a straight forward leastsquares problem� solved using the singular value

decomposition technique�

For example� using this procedure to �nd a function� H��x�� that will form a quadrature

pair with the second derivative of the Gaussian function G��x� � �� x��e�x� � we �t a

third order polynomial to get the function�

H��x� � ��x � x��e�x��

Note that the scale and sign of these functions is not crucial because they can be nor

malised and the results of the convolutions with them are squared then summed�

Appendix B

Recursive �lter implementation

In this appendix� various code fragments are presented which are used to derive and

implement the recursive �ltering technique described in Chapter ��

The entire code and data sets used for this thesis are available via FTP from the Depart

ment of Computer Science at the University of Western Australia�

ftp � cs�uwa�edu�au pub robvis theses�

B�� Fitting a polynomial to Hilbert transform

This SciLab code is used to derive the polynomial approximation to the Hilbert transform

of a function� in this case the third derivative of the Gaussian function�

��

APPENDIX B� RECURSIVE FILTER IMPLEMENTATION ��

��x��

��g�exp��x��

��g��x��x��g�

��plot�g��

��F�fft�g��

��H��i�sign�x� �� F�

��plot�real�fft�H��

��A��g��i��g��i��g��

��u�s�v�svd�A��

��p�v��s��u��real�fft�H��

p �

� ��

��

� ��

��plot�p��p��x��p��x��

��p�p��

ans �

��

� ��

��

So the numerical �t of a �th order even polynomial to the Hilbert transform of G��x� is�

H��x� y� � c�p� � �

�x� � x��e��x�y��

where c is a constant to normalise the L� norm of the �lter to ��

B�� Filter implementation

ApplyFilter� applies a recursive �lter to a data vector returning two result vectors con

tained in the result structure� that is� the causal and anticausal �lter results� The �nal

result is given by�

Y k � � Norm � �Y �k � Y �k � Integral�� k � ��count

This is a direct implementation of Recursively Implementing the Gaussian and its Deriva�

tives ��


void Apply�Filter�int count� float �x� FILTER �filter� RESULT �result�

�

�� some local macros to simplify the coding ��

�define CHECK�BOUNDARY�i� vector� ��i�� i��count� � � vector�i��

�define X�i� CHECK�BOUNDARY� i� x �

�define RESULT�Y��i� CHECK�BOUNDARY� i� result��y� �

�define RESULT�Y��i� CHECK�BOUNDARY� i� result��y� �

int i�

int p� q�

int order � filter��order�

�� proccess the data from both ends with the causal and anticausal

filters at the same time� ��

p � �

q � count��

while �p � count� � �� filter� considering the boundary conditions ��

result��y��p� � ��

result��y��q� � ��

for �i�� i�order� i��

result��y��p� �� filter��n�i��X�p�i�

� filter��d�i��RESULT�Y��p��i�� eqn ��

result��y��q� �� filter��n�i�order��X�q��i��

� filter��d�i�order��RESULT�Y��q��i�� eqn ��

�

p�� q��

�

� �� Apply�Filter ��

B�� Three�dimensional local energy

The following code fragments come from the author s program energy�d�c which is used

to calculate the energy results shown in this thesis� The program is written in C and uses

the VIP image processing library written by the Department of Computer Science at the

University of Western Australia �� Extensive use is made of macros in an attempt to

make the code re�ect the descriptions of the algorithms presented in Chapter ��

The code fragments are provided in the order that they would be called� i�e� the actual

energy calculation routine energy�d is last�


B�� Read Filter Design

float �� ReadFilterDesign�FILE � file� int size�

�

float �� design � CALLOC�size� float ��

float � p�

float scale�

int i�

for �i�� i�size� i��

design�i� � CALLOC�� float��

p � design�i��

if �� fscanf�file��f�f�f�f�f�f�f�f�f�f��

�scale�p��p��p��p��p��p� �p�!�p�"�p��

� �

Report�FATAL��ReadFilterDesign� failed to read �

�filter parameters� line �i�#n�� i��

� else �

char comment� ��

fgets�comment� � file�� ignore to end of line ��

Report�DEBUG��p ��f �f �f �f �f �f �f �f �f �f#n��

p� scale� p��p��p��p��p��p� ��p�!��p�"��p��

�

p�� scale� �� a ��

p�� scale� �� a� ��

p�� scale� �� c ��

p� � �� scale� �� c� ��

�

return design�

�

B�� Convolve XYZ Filter

ConvolveXYZFilter� convolves the �D image with the separable recursive �lters described

by �lterx� �ltery and �lterz�


IMAGE � ConvolveXYZFilter�IMAGE � im�

FILTER � filter�x� FILTER � filter�y� FILTER � filter�z�

�

RESULT � result � NewResult�MAX�MAX�ROWS�im��COLS�im��LEVELS�im��

float � column � CALLOC�MAX�MAX�ROWS�im��COLS�im��LEVELS�im�� float��

IMAGE � image � SameAs�Image�im��

int r�c�v�

� define APPLY�FILTER�size� data� filter� result� #

if �size � filter��order� � Report�FATAL��ConvolveXYZFilter� �#

�image dimension� �d� smaller than filter�#n�� size�� #

� else � Apply�Filter�size� data� filter� result��

ForEachLevelIn�image� v� �

ForEachRowIn�image� r� �

APPLY�FILTER�COLS�im�� im��i�f�r � v � ROWS�im�� filter�x� result��

ForEachColIn�image� c� �

VOXEL�FLOAT�image�r�c�v� � result��y��c� � result��y��c��

�

�

ForEachColIn�image� c� �

ForEachRowIn�image� r� � column�r� � VOXEL�FLOAT�image�r�c�v��

APPLY�FILTER�ROWS�image�� column� filter�y� result��

ForEachRowIn�image� r� �

VOXEL�FLOAT�image�r�c�v� � result��y��r� � result��y��r��

�

�

�

ForEachRowIn�image� r� ForEachColIn�image� c� �

ForEachLevelIn�image� v� � column�v� � VOXEL�FLOAT�image�r�c�v��

APPLY�FILTER�LEVELS�image�� column� filter�z� result��

ForEachLevelIn�image� v� �

VOXEL�FLOAT�image�r�c�v� � result��y��v� � result��y��v��

�

�

DisposeResult�result��

free�column��

return image�

�


B�� Kernel

Kernel� reads the given recursive �lter �le containing design parameters for a steerable

kernel of recursive �lters then applies these �lters to the image� generating a set of basis

images from which a steered result can be obtained�


��

�� ref� table �� Ch� �Recursive filters for oriented local energy� ��

�� w� a� a�� b� w�� c� c�� b� ��

� define PARAMETERS�p� p��p��p��p��p�"��p��p� ��p�!�

�� necessary due to change in order between original code and thesis ��

� define SYMMETRY�p� ��p�� IIR�ANTI�SYMMETRIC � IIR�SYMMETRIC�

��

KERNEL � Kernel�IMAGE � image� char � name� float sigma�

�

FILE � file � ReadFilterOpen�name��

int size � ReadFilterSize�file��

float �� design � ReadFilterDesign�file� �size��

KERNEL � kernel � NewKernel�image� size��

int i�

Report�INFO��Kernel� �s contains �d filters�#n�� name� size��

Report�INFO��Kernel� image �s� �d rows� �d cols� �d levels�#n��

image��info� ROWS�image�� COLS�image�� LEVELS�image��

for �i�� i��size�� i��

kernel��filter�x�i� � NormaliseFilter�sigma�

Duplicate�Symmetric�Coefficients�

SYMMETRY�design�i��

New�DFourthOrderExpApprox�sigma� PARAMETERS��design�i��

�

��

kernel��filter�y�i� � NormaliseFilter�sigma�




�

��

kernel��filter�z�i� � NormaliseFilter�sigma�




�

��

�

for �i�� i��size�� i��

kernel��basis�i� �

ConvolveXYZFilter�image�

kernel��filter�x�i�� kernel��filter�y�i�� kernel��filter�z�i��

�

for �i�� i��size�� i�� free�design�i��

free�design��

fclose�file��

return kernel�

�

�undef PARAMETERS

�undef SYMMETRY


B�� Steer

Steer applies the steering formula to the basis images to produce an oriented convolution�

The kernel�image is used to hold the result and a pointer to this is returned by the

function� This is NOT a copy and must not be freed or altered� The kernel of basis

functions has been calculated and is stored in the array of images kernel�basis�


IMAGE � Steer�float cosines�� KERNEL � kernel� char � name�

�

� define image�r�c�v� #

kernel��image��i�f�r � v�ROWS�kernel��image��c�

� define basis�n�r�c�v� #

kernel��basis�n��i�f�r � v�ROWS�kernel��basis�n��c�

float a � cosines�� a� � a � a� a� � a � a��

float b � cosines�� b� � b � b� b� � b � b��

float c � cosines�� c� � c � c� c� � c � c��

float k�kernel��size��

int i�x�y�z�

switch �kernel��size� �

case � � �� for test and debug ��

k��

break�

case ! �

k�� a�� k�� b�� k�� c��

k�� a � b� k�� b � c� k� � � c � a�

break�

case � �

k�� a�� k�� b�� k�� c��

k�� a� � b� k�� a� � c�

k� � � a � b�� k�!� � b� � c�

k�"� � a � c�� k�� b � c��

k�� a � b � c�

break�

default �

Report�FATAL��Steer� no formula for size �i�#n��kernel��size��

exit��

�

�� Clear the image prior to linear combination ��

ForEachVoxelIn�kernel��image� x� y� z� �

image�x�y�z� � ��

�

�� Rotated convolution result of linear combination

of individual basis convolutions ��

for �i�� i�kernel��size� i��

Report�DEBUG��Steer� applying kernel �d#n�� i��

if �k�i� �� continue�

ForEachVoxelIn�kernel��image� x� y� z� �

image�x�y�z� �� k�i� � basis�i�x�y�z��

�

�

return kernel��image�

� undef image

� undef basis

�


B�� Energy�d

This is the main threedimensional energy routine� It calculates the kernel of both func

tion and its Hilbert transform� It then calculates local energy oriented to each of the

directions speci�ed by a set of direction cosines� either � or �� evenly spaced directions

can be selected� Each of the oriented energy components are summed together to produce

a total energy result�

IMAGE � energy�d�IMAGE � image� char � even� char � odd�

int orient� float sigma�

�

IMAGE � out � SameAs�Image�image�� initialised to � ��

�� Calculate both banks of kernel images ��

KERNEL � function � Kernel�image� even� sigma��

KERNEL � hilbert � Kernel�image� odd� sigma��

�� For each orientation ��

COSINES � cosines � Orientations��orient��

int angle�

for �angle�� angle � orient� angle��

�� Calculate �D oriented energy components ��

IMAGE � func � Steer�cosines�angle�� function� �func��

IMAGE � hilb � Steer�cosines�angle�� hilbert� �hilb��

�� Sum energies sqrt�F$��H$��

int r�c�v�

Report�DEBUG��energy�d� warning� hard coded normalisation factors�#n��

ForEachVoxelIn�image�r�c�v� �

float f � ��! � VOXEL�FLOAT�func�r�c�v��

float h � ��"� � VOXEL�FLOAT�hilb�r�c�v��

VOXEL�FLOAT�out�r�c�v� �� sqrt�SQR�f� � SQR�h��

�

DisposeKernel�function��

DisposeKernel�hilbert��

return out�

� �� end of energy�d ��

Appendix C

Surface tracing implementation

The two and threedimensional watershed and ridge tracing programs are constructed

in a very similar manner� Consequently only the threedimensional code fragments are

presented here�

The entire code and data sets used for this thesis are available via FTP from the Depart

ment of Computer Science at the University of Western Australia�

ftp � cs�uwa�edu�au pub robvis theses�

Several of the functions use ordered queues to hold references to individual pixels in the

order that they await processing� A union is used so that the voxel structure may refer

to its value or be a reference to its row� column� level coordinates� The queue package

provides the following functions�

NewQueue�size� returns an array of size empty QUEUE structures� The pixel s value

is the queue s array index�

PutQ�queue� voxel� append a voxel to the speci�ed queue�

GetQ�queue� returns an extracted voxel from the head of the queue�

ScanQ�queue� iterates through all the voxels on the queue without altering the queue�

Iteration is reset by SizeQ and is not reentrant�

SizeQ�queue� returns the number of voxels currently on the queue� and also resets

the iterator pointer�

��

APPENDIX C� SURFACE TRACING IMPLEMENTATION ��

The following macros� are used extensively to simplify the coding�

ForEachVoxelIn�image�r�c�v� nested for loops that scan through each level� row and

column of the given image�

ForEachNeighbour�connectivity�list�voxel�neighbour� this loop steps through

the voxel s neighbours by successively incrementing the voxel by amounts held in the

connectivity list� A separate function initialises an array for � and ��connectivity�

BOUNDARY provides a conditional wrapper to determine if a voxel is inside the

image�

LABEL�voxel� dereferences the given voxel in the label image�

VALUE�voxel� dereferences the given voxel in the value image�

C�� Minima and maxima

Minima and maxima in the image are found by scanning the image and performing a

recursive search for each potential minimum �or maximum� voxel� The routines are the

same except the comparison tests are reversed� The routine returns a list containing all

of the minima �or maxima� voxel locations�

QUEUE � Minima��D�IMAGE � value�

�

int ROWS�COLS�LEVELS�value��

IMAGE � label � Similar�Image�value� BYTETYPE��

QUEUE � minima � NewQueue��

QUEUE � equal � NewQueue��

PATH � region � REGION�CONNECTIVITY�

int min�count � �

VOXEL p� n� start�

int p�value� n�value�

if �BYTETYPE �� value��type� �

VipReport�ERROR��Minima��D� image must be BYTETYPE�#n��

exit��

�

�It should be remembered that C macros are purely text substitution devices and do not provide anyscoping of variables etc�


ForEachVoxelIn�value� start�voxel� �

int minimum � TRUE�

if �UNVISITED �� LABEL�start�voxel�� continue�

LABEL�start�voxel� � VISITED�

PutQ�equal� start�index��

do �

p�index � GetQ�equal��

p�value � VALUE�p�voxel��

ForEach�DNeighbour�region� p�voxel� n�voxel� �

if �BOUNDARY�n�voxel�� continue�

n�value � VALUE�n�voxel��

if �n�value � p�value� �

LABEL�p�voxel� � NON�MINIMUM�

minimum � FALSE�

� else if �n�value �� p�value� �

switch �LABEL�n�voxel��

case UNVISITED �

LABEL�n�voxel� � LABEL�p�voxel��

PutQ�equal� n�index��

break�

case NON�MINIMUM �

LABEL�p�voxel� � NON�MINIMUM�

minimum � FALSE�

break�

case VISITED � �� ignore �� break�

default �

VipReport�ERROR� �Minima��D� �

�unrecognised label #��d#�#n�� LABEL�n�voxel��

�

� else �� n�value � p�value ��

LABEL�n�voxel� � NON�MINIMUM�

�

�


� while �SizeQ�equal��

if �minimum� �

PutQ�minima� start�index��

min�count��

�

�

VipReport�DEBUG��Minima��D� found �d minima�#n�� min�count��

DisposeQueue�equal� ��

DisposePath�region��

Free�Image�label��

return minima�

�

C�� Fill shallow lakes

The �ll operation used to reduce oversegmentation is easily implemented using the or

dered queues and �ooding paradigm described in this thesis�

This routine returns an image in which each voxel has its �lled value�


IMAGE � Fill��D�IMAGE � value� int fill�depth�

�



QUEUE � queue � NewQueue�MAX�VALUE��

PATH � region � REGION�CONNECTIVITY�

VOXEL start�

int priority�


VipReport�ERROR��Fill��D� image must be BYTETYPE�#n��

exit��

�

ForEachVoxelIn�value� start�voxel� �

PutQ�queue � VALUE�start�voxel�� start�index��

�

for �priority � MIN�VALUE� priority � MAX�VALUE� priority��

while �SizeQ�queue � priority��

int p�label�

int n�label�

VOXEL n�

VOXEL p � � GetQ�queue � priority� ��

if �UNVISITED �� LABEL�p�voxel��

LABEL�p�voxel� � MIN�MAX�VALUE� VALUE�p�voxel� � fill�depth��

�

p�label � LABEL�p�voxel��



n�label � LABEL�n�voxel��

if �VALUE�n�voxel� � p�label� �

LABEL�n�voxel� � VALUE�n�voxel��

� else if �n�label �� UNVISITED� �

LABEL�n�voxel� � p�label�

� else if �n�label � p�label� �


PutQ�queue � p�label� n�index��

priority � MIN�priority� p�label��

�

�

�

�

DisposeQueue�queue� MAX�VALUE��


return label�

�


C�� Watershed

The watershed algorithm is implemented using the �ooding paradigm and ordered queues

of pixels� The seed list is used to initiate the �ooding so that various methods of seed

selection can be investigated� The returned image has each catchment basin labelled

with a di�erent value� The separating watershed lines are labelled FRONTIER and the

original seed points are labelled SEED�

Note that the use of BYTETYPE images severely restricts the number of catchment

basins� This routine reuses labels� see function NEXT�BASIN which could lead to erro

neous merging of regions in the event that two neighbouring basins have the same label�

In practice for the investigative applicatin used here this is not a problem�

�define MAX�LABEL �

�define SEED

�define FRONTIER �

�define MIN�LABEL �FRONTIER � ��

�define NEXT�BASIN�b� �b � �b � MAX�LABEL� b�� MIN�LABEL�

IMAGE � Water��D�IMAGE �value� QUEUE �seed�

�



PATH � region � BOUNDARY�CONNECTIVITY�


int basin � MIN�LABEL�

int count � SizeQ�seed��

int water � �

int level�


VipReport�ERROR��Water��D� image must be BYTETYPE�#n��

exit��

�

while �count��

VOXEL p � � GetQ�seed� ��

PutQ�seed� p�index��

PutQ�queue � VALUE�p�voxel�� p�index��

LABEL�p�voxel� � NEXT�BASIN�basin��

�


for �level � � level � MAX�VALUE� level��

while �SizeQ�queue � level��

VOXEL n�

VOXEL p � � GetQ�queue � level� ��

int p�label � LABEL�p�voxel��



if ��UNVISITED �� LABEL�n�voxel��

�� FRONTIER �� p�label��


PutQ�queue � VALUE�n�voxel�� n�index��

if �VALUE�n�voxel� � level� �

level � VALUE�n�voxel��

�

� else if ��p�label �� LABEL�n�voxel��

�� FRONTIER �� LABEL�n�voxel��

�� VALUE�n�voxel� �� VALUE�p�voxel��

LABEL�p�voxel� � FRONTIER�

water��

�

�

�

�

while�SizeQ�seed��


LABEL�p�voxel� � �

�

VipReport�INFO��Water��D� �d watershed pixels#n�� water��



return label�

�

C�� Skeleton

The skeleton routine performs the ablative thinning that is the basis of the ridge and

surface tracing algorithms� Given an image� label image and array of ordered queues it

examines each voxel in the queues at a given level for homotopic removal� As it examines

a voxel s neighbours the routine will add these to the appropriate queue if necessary�


Skeleton is called repetitively from Ridge�D and builds a list of all nonremovable voxels

which form the eventual skeletonisation of the data set�

�define ANCHOR �

�define INCLUDED ��!

�define EXCLUDED !

�define BOUNDARY�MODE boundary�mode

static int boundary�mode � � �� INCLUDED ��

int Skeleton��D�

IMAGE � value� IMAGE � label�

QUEUE � queue� int level�

QUEUE � list� int � changes�

int connect

�

�


PATH � region � Connected��D��!��


while �SizeQ�queue � level��

VOXEL n�

VOXEL p � � GetQ�queue � level� ��

long code � � �� must be more than �! bits ��


code � code ��

if �BOUNDARY�n�voxel��

code � BOUNDARY�MODE� �� boundary points ��

continue�

�

switch �LABEL�n�voxel��

case UNVISITED �

LABEL�n�voxel� � INCLUDED�

PutQ�queue � VALUE�n�voxel�� n�index��

if �VALUE�n�voxel� � level� �

level � VALUE�n�voxel��

�

break�

case EXCLUDED �

code � �� mark EXCLUDED neighbours ��

break�

default �

�� do nothing ��

�

�


if �EXCLUDED �� LABEL�p�voxel�� continue�

�� adjust to mark INCLUDED neighbours ��

code � �%code� � x�FFFFFF�

�� adjust to include the point itself ��

code � ��code �� code � x�FFF��

if ��ANCHOR �� LABEL�p�voxel��

�REMOVABLE ��

��connect �� !� Homotopy��D�!�code� � Homotopy��D��!�code��

� THEN �

PutQ�list� p�index��

� else �

LABEL�p�voxel� � EXCLUDED�

��changes��

�

�

VipReport�DEBUG��Skeleton��D� ��d �d changes�#n�� level� �changes��

return level�

�

C�� Homotopy

The homotopic removability of a voxel is determined from a bit pattern that represents

its FOREGROUND neighbours� For clarity� only one connectivity version of the Check

Connection and CheckSeparation routines are shown here�

C�� Homotopy �D

This routine returns REMOVABLE if the central point is a simple point as described

in Chapter � and by Bertand and Malandain �� This technique uses two connectivity

tests� which are valid for sets on a cubic grid� rather than the more general calculation

of an Euler characteristic�


int homotopy�mode � NO�HOLES� �� global variable � ��

int Homotopy��D��!�long code�

�

int one � � two � � three � �

if �x �� code� return ��

�� NB the leading indicates OCTAL ��

if ��"""" """" AND code� �� """" """"� return ��

if ��"""" """" AND code� �� return ��

CheckCodeToInts�code� �one� �two� �three��

�� make sure the point is not marked ��

two � two AND �NOT POINT��

if �CheckConnection��!�one� two� three��

�� Removal of this pixel would �! disconnect foreground regions ��

return FIXED�

� else if �ALLOW�HOLES �� homotopy�mode� �

�� Don&t care if removing this point connects background regions ��

return REMOVABLE�

� else if �CheckSeparation�!a�x OR one� two�x OR three��

�� Check if removal would ! connect adjacent background regions ��

�� Setting the corner pixels stops tunnels from developing� ��

return FIXED�

� else �

�� Point can be removed without ! connecting background regions� ��

return REMOVABLE�

�

�

C�� CheckConnection

CheckConnection�� checks homotopic connection properties� returning � if the point

does not N connect foreground regions or returns nonzero otherwise�

int CheckConnection��!�int one� int two� int three�

�

return Propagate��!�one� two� three��

�

C�� CheckSeparation

This routine performs a background connectivity test using the �neighbourhood�


CheckSeparationN checks homotopic region separation properties� returns � if the point

does not N separate EXCLUDED regions or returns nonzero otherwise�

�define BIT�PATTERN �x"""�

�define POINT �x��

int CheckSeparation�!a�int one� int two� int three�

�

one � �NOT one� AND BIT�PATTERN�

two � �NOT two� AND BIT�PATTERN AND �NOT POINT��

three � �NOT three� AND BIT�PATTERN�

�� Ensure that there is at least one !�adjacent neighbour ��

if ��ONE TWO THREE�� return ��

return Propagate�!�one� two� three��

�

C�� Propagate

Propagate checks connection properties of the neighbourhood described by the bit pat

terns one� two and three� We select a nonzero bit to start from and propagate this bit

using � or �� connectedness� throught the nonzero neighbour bits� It returns � if all of

the set bits have been visited or nonzero if some are still unlabelled� that is the bits are

N connected via the central point�


�� Bit propagation operations ��

�define LEFT�RIGHT�r� r � ��r LEFT �� OR �r� OR �r RIGHT ��

�define UP�DOWN�r� r � ��r LEFT �� OR �r� OR �r RIGHT ��

�define IN�OUT�q�r�s� ��q� OR �r� OR �s��

�define ONE �one AND x��

�define TWO �two AND x� ��

�define THREE �three AND x��

�define FIND�START�r�o� for�r � �� r � o�� r ��

int Propagate��!�int one� int two� int three�

�

int alpha � �

int beta � �

int gamma � �

int count � !�

if �one� � FIND�START�alpha� one��

else if �two� � FIND�START�beta� two��

else if �three� � FIND�START�gamma� three��

else �

fprintf�stderr��s �d Propagate��!� �

��X �X �X failed to find a start�#n��

��FILE�� LINE�� one� two� three��

exit��

�


int a�b�c�

LEFT�RIGHT�alpha��

LEFT�RIGHT�beta��

LEFT�RIGHT�gamma��

UP�DOWN�alpha��

UP�DOWN�beta��

UP�DOWN�gamma��

a � IN�OUT�ZERO� alpha� beta��

b � IN�OUT�alpha� beta� gamma��

c � IN�OUT�beta� gamma� ZERO��

alpha � a AND one�

beta � b AND two�

gamma � c AND three�

�

return �one XOR alpha � �two XOR beta� �three XOR gamma��

�


C�� Ridge and surface tracing

The Ridge�D routine has two stages� The initial stage allows holes to develop in surfaces�

which results in threedimensional anchor lines� These anchor lines are used in a second

stage in which holes are not allowed to develop� Ridge�D calls Skeleton�D to perform

ablative errosion of the label image� The erosion is ordered by voxel value using an array

of voxel queues to store voxels until their level is processed�


IMAGE � Ridge��D�IMAGE �value� QUEUE �seed� QUEUE �anchor�

�




QUEUE � prune � NewQueue��

PATH � region � Connected��D��!��

PATH � flood � REGION�CONNECTIVITY�

QUEUE � border � NULL�

int change � �

int count�

int level�

label��ThreeD � ��

label��levels � value��levels�


VipReport�ERROR��Ridge��D� image must be BYTETYPE�#n��

exit��

�

�� Put all the UNVSITED greater�than neighbours of the seed points

into the ordered queue array� ��

count � SizeQ�seed��


VOXEL p � � ScanQ�seed� ��


Flood��D�value� label� flood� queue� p��

�

�� Label all of the anchor points� ��

count � SizeQ�anchor��


VOXEL p � � ScanQ�anchor� ��

LABEL�p�voxel� � ANCHOR�

�

VipReport�INFO��Ridge��D� stage �� d seeds �d anchors#n��

SizeQ�seed�� SizeQ�anchor��

�� Flood out from the seed pixels� ��

boundary�mode � � �� INCLUDED ��

homotopy�mode � ALLOW�HOLES�

for �level � � level �� MAX�VALUE� level��

level � Skeleton��D�value� label� queue� level� prune� �change� �!��

�

�� Prune extraneous pixels from the list� ��

boundary�mode � �� EXCLUDED ��

do �

QUEUE � list � prune�

prune � NewQueue��

change � �

level � Skeleton��D�value� label� list� level � � prune� �change� �!��

DisposeQueue�list� ��

� while �change��


�� Start stage � ��

VipReport�INFO��Ridge��D� stage �#n��

�� Mark all of the Ridge points as ANCHORS ��

count � SizeQ�prune��


VOXEL p � � ScanQ�prune� ��

LABEL�p�voxel� � ANCHOR�

�

border � prune�


change � �

�� Note that the queue structure is empty at this point ��

�� Note that the prune structure is empty at this point ��

�� Re�initialise the label image ��

�

VOXEL p�

ForEachVoxelIn�label� p�voxel� �

if �EXCLUDED �� LABEL�p�voxel��

LABEL�p�voxel� � UNVISITED�

�� start with everything included ��

�

�

�

�� Put all the seed points into the ordered queue array ��

count � SizeQ�seed��



PutQ�seed� p�index��


Flood��D�value� label� flood� queue� p��

�

�� Flood out from the seed pixels� ��

boundary�mode � � �� INCLUDED ��

homotopy�mode � NO�HOLES�

for �level � � level �� MAX�VALUE� level��

level � Skeleton��D�value� label� queue� level� prune� �change� �!��

�

�� Prune extraneous pixels from the list� ��

boundary�mode � �� EXCLUDED ��

do �

QUEUE � list � prune�


change � �

level � Skeleton��D�value� label� list� level � � prune� �change� �!��

DisposeQueue�list� ��

� while �change��


�� Start output Stage ��

VipReport�INFO��Ridge��D� �d ridge points�#n��

SizeQ�prune� � SizeQ�border� � SizeQ�anchor��


DisposeQueue�prune� ��


�

Appendix D

Direction cosines for orientations

The vertices of the regular dodecahedron and icosahedron are used to acheive an even

distribution when the �lters are oriented in threedimensions� The direction cosines used

for these two polyhedra are given below� Note that only half the directions are needed

due to the symmetry of the shape and the local energy �lters�

D�� Regular dodecahedron

� �

�� p�

�� p�

�� p�

�� p�

�� p�

�� p�

Table �� The direction cosines for a dodecahedron�

D�� Regular icosahedron

�

APPENDIX D� DIRECTION COSINES FOR ORIENTATIONS ��

� �

��

p��

��

�p�

p��

�� p

�

p��

��

��

��

��

��

��

Table �� The direction cosines for an icosahedron�

Documents

Mic John - UWA Research Repository - University of Western ...research-repository.uwa.edu.au/files/3248336/Robins_Michael_John... · ters and the ridge tracing algorithm applied to