Canonical cortical module as a spatial-frequency filterYu. D. Kropotov and I. Z. Kremen

Institute of the Human Brain, Russian Academy of Sciences, St. Petersburg~Submitted March 17, 1999!Opticheski� Zhurnal66, 81–84~September 1999!

The authors earlier suggested a realistic neural network~called a canonical cortical module! thatsimulates the structure and functions of the visual system in the brain. Computer experimentsshowed a wide range of possibilities of using this network for automatic image analysis. This paperconcentrates on the abilities of the network to filter spatial frequencies, to discriminatetextures, and to suppress high-frequency spatial noise. ©1999 The Optical Society of America.@S1070-9762~99!01109-4#

lanm

hithdercer

hi

1nt,aler

c

Indi

ioor

f

tse

le

ule

f,of

eldersseto

tep

I. INTRODUCTION

In Ref. 1, the authors suggested a mathematical modea canonical cortical module that imitates the structurefunction of an element of the visual system of the brain, frothe retina to a module of the striate cortex, inclusive. Tmodel simulates the processes of input image filteringtake place in mammalian visual systems. During recentcades, enough experimental data have been gathered ovworld to prove that the mammalian brain carries out piewise spatial frequency analysis~Fourier analysis or Gabodecomposition! of the input visual signal.2,3 In this paper, weconcentrate on the abilities of our network to carry out tspecific function.

II. NETWORK STRUCTURE

The network structure is shown schematically in Fig.The network consists of several layers or levels, represeby parallel planes~Fig. 1a!. Layer R simulates the retinalayers LGBon and LGBoff represent two parts of the latergeniculate body~LGB!: the layer of on neurons and the layof off neurons.

The striate cortex is simulated by two modules. Eamodule contains four layers of inhibitory neurons~In1..In4!,four layers of simple cortical neurons~S1..S4!, and one layerof complex cortical neurons~C!. These two modules differ inthe spatial organization of the inhibitory neurons in theplanes~Fig. 1b!. In one module, they are assembled asvergent sectors~the sector module!, in the other in the formof concentric rings~the ring module!.

For all the planes except the In-planes, the distributdensities of the model neurons are unity. For the inhibitneurons in planes Ini( i 51..4), it is described by

for the sectors,

nIn~ i !on512sin~vc1p i /2!,

nIn~ i !off511sin~vc1p i /2!, ~1!

for the rings,

nIn~ i !on512sin~2pnr 1p i /2!,

nIn~ i !off511sin~2pnr 1p i /2!.

832 J. Opt. Technol. 66 (9), September 1999 1070-9762/99/09

ofd

sate-the-

s

.ed

h

--

ny

Here, v is the number of sectors,n is the number ofrings in the mosaic andr andc are the polar coordinates oan inhibitory neuron in plane Ini .

III. OUTPUT SIGNALS OF THE MODEL CORTICALNEURONS

The formula for the input signals and for the outpu~responses! of model cortical neurons of different levels havbeen derived in Ref. 1.

The input signal for a simple neuron of the ring moduis determined from

I Si~r !~X,Y!5E E exp~2a~r !2@~X2x!2

1~Y2y!2#/R2!sin~2pn~x21y2!1/2

1p i /2!OR~x,y!dxdy, ~2!

and the input signal for a simple neuron of the sector modis

I Si~s!~X,Y!5E E exp~2a~s!2@~X2x!2

1~Y2y!2#/R2!sin~vc~x,y!

1p i /2!OR~x,y!dxdy. ~3!

Here OR(X,Y)5s(X,Y)2^s& is the signal at the output othe retina,s(X,Y) is the input signal arriving at the retinaand^s& is the input signal averaged over the field of viewthe network,

c~x,y!5 H arccos~x/~x21y2!1/2!,p1arccos~2x/~x21y2!1/2!,

y>0;y,0.

The integration is performed over the circular receptive fiof a simple neuron; the field width is defined by parameta (r ) anda (s). The weighting factors by which the responof the retina in the integrands is multiplied are similarGabor functions.

The output signal for a simple neuron is defined as a sfunction with zero threshold:

OSi~s!~X,Y!5I Si

~s!~X,Y!H@ I Si~s!~X,Y!#,

OSi~r !~X,Y!5I Si

~r !~X,Y!H@ I Si~r !~X,Y!#, ~4!

8320832-04$15.00 © 1999 The Optical Society of America

pith

he

om

lex

u-theeu-

ing.on-

ar-ctortheod-flarhebee-

ec-ec-

atialnsea

heer

ori-lelums intinge of

to

s.ti-

rons.

,

whereH(z)51 if z.0, otherwiseH(z)50.The output signal for a complex neuron with~X,Y! coor-

dinates is equal to the sum of the output signals of the simneurons of all four layers of the corresponding module wthe same coordinates:

OC~s!~X,Y!5(

1

4

I Si~s!~X,Y!H@ I Si

~s!~X,Y!#,

OC~r !~X,Y!5(

1

4

I Si~r !~X,Y!H@ I Si

~r !~X,Y!#. ~5!

IV. NETWORK RESPONSES FOR SINUSOIDAL GRATINGS

To investigate the spatial-frequency sensitivity of tnetwork, sinusoidal gratings with different orientationsF,spatial frequenciesV, and spatial phasesQ were used asinput signals:

s~X,Y!511sin@~2V sin F!X1~V cosF!Y1Q#. ~6!

The dependences of the input signals of simple and cplex neurons~for sector and ring modules! on the parametersof the input grating were derived in Ref. 1@polar coordinates(R,w) are used#:

I Si~s!~R,w!5~pR2/2a~s!2!@cos~vw1p i /22Q

2VR sin~w2F!!exp~2~v21V2R2

22vVR cos~w2F!!/4a~s!2!2cos~vw

1p i /21Q1VR sin~w2F!!exp~2~v2

1V2R212vVR cos~w2F!!/4a~s!2!#,

FIG. 1. Structural diagram of canonical cortical module:~a! general schemeof network levels;~b! sector and ring mosaics of cortical inhibitory neuronR—retina, LGB—lateral geniculate body of thalamus, In—inhibitory corcal neurons, S—simple cortical neurons, C—complex cortical neurons.

833 J. Opt. Technol. 66 (9), September 1999

le

-

I Si~r !~R,w!5~pR2/2a~r !2!@cos~2pnR1p i /22Q

1VR sin~w2F!!exp~2R2~4p2n21V2

24pnV sin~w2F!!/4a~r !2!2cos~2pnR

1p i /21Q2VR sin~w2F!!

3exp~2R2~4p2n21V214pnV

3sin~w2F!!/4a~r !2!#. ~7!

To compute the responses of the simple and compneurons, Eqs.~7! were substituted into Eqs.~4! and ~5!.

High orientational selectivity of the model cortical nerons was demonstrated in Ref. 1. It was also shown thatresponses of complex neurons, unlike those of simple nrons, are invariant to the spatial phase of the input gratTherefore, the responses of the complex neurons will be csidered below to be responses of the modules.

Figure 2 shows examples of the spatial frequency chacteristics for the responses of complex neurons of seand ring modules. The results of the study proved thatoptimal input spatial frequency for neurons of a sector mule ~simple or complex! is proportional to the number osectors in the module and inversely proportional to the poradius of the neuron. As for neurons of a ring module toptimal spatial frequency of the input signal is found toproportional to the number of rings in the module and indpendent of the position of the neuron in the module.

Therefore, a sector module with a fixed number of stors is able to decompose the input image over a wide sptrum where each spatial frequency is encoded by the spradius of the specific neuron that gives the maximal respoto that frequency. As for a ring module, it can be calledspatial-frequency filter only in a very limited sense of tword, if we consider that a module with a different numbof rings selects specific spatial frequencies.

Figure 3 shows the responses of both modules to hzontal gratings. If the input signal is a sum of two paralsinusoidal gratings, it is possible to select areas of maximresponse corresponding to each of two spatial frequenciethe response of a sector module. The response to a grawith a higher frequency cannot be selected in the responsa ring module.

In what follows, the term ‘‘responses of the networktextures’’ will mean the responses of a sector module.

FIG. 2. Spatial frequency characteristics of responses of complex neu~a! for a complex neuron of a sector module~V—frequency of the inputgrating,R—polar radius of the neuron,v—number of sectors in the modulea—width parameter of receptive field!; ~b! for a complex neuron of a ringmodule~V—frequency of the input grating,n—number of rings in the mod-ule, a—width parameter of receptive field!.

833Yu. D. Kropotov and I. Z. Kremen

isis

atee-a

ss

ler’’ul

in,q

antauhes

eri-g

the. Itd by

con-of

ialres,k toith axed,isyver-

tial

hatchon

theose

-gnalble

es;.

V. TEXTURE DISCRIMINATION

The function of the discrimination of surface texturesclosely associated with spatial frequency filtering. In thcase, the surface texture is considered as a set of repeelements filling a surface in a more or less periodic mannwhich can be interpreted as a set of certain spatial frequcies and orientations.3 Our canonical cortical module performs a piecewise Gabor decomposition of the input signThe position of the regions of maximal response makepossible to define the spatial frequencies and orientationthe harmonics selected by the filter.

Figure 4 shows examples of the responses of compneurons of a sector module to ‘‘checkered’’ and ‘‘cellulatextures with different frequencies. In practice, the modperceives these textures as sums of several gratings.

VI. SPATIAL NOISE FILTERING

Gaussian filtering is a common method of suppresshigh-frequency spatial noise.4,5 Our model cortical neuronshaving a Gaussian factor in their weighting functions, E~3!, perform a similar function. However, unlike Gaussifilters, the canonical cortical module maps different orientions and spatial frequencies to different areas of the modThis allows our network to distinguish the signal from tnoise much more effectively than is done by ordinary Gauian filtering.

To study the noise sensitivity of the network, checkerand cellular textures with white noise with a normal distbution added to them were used as input images. Imawith different signal-to-noise ratio~SNR! were used.

FIG. 3. Responses of the network to spatial gratings.~a! input images;~b!responses of complex neurons~s corresponds to a sector module, andr to aring module!; ~c! response intensity scale. Model parameters:v519, n59,a (s)55.0, a (r )53.0.

834 J. Opt. Technol. 66 (9), September 1999

ingr,n-

l.itof

x

e

g

.

-le.

s-

d

es

The signal was defined as the difference betweenintensities of the light and dark section of the input texturewas set equal to 64 for each image. The noise was defineits rms value.

Four sets of input textures were generated. Each settained an image of a texture without noise and six setsnoisy images~for SNRs of 1.5, 2, 3, 4, 5 and 10!. Each setconsisted of ten noisy implementations.

The criterion for the network to be sensitive to spatnoise was chosen as follows: For each of four input textuthe maximal response of the sector module of the networthe noise-free image was calculated, and then neurons wresponse no less than 0.67 of the maximum value were fidetermining the maximal response area. Then, for all noimplementations of the same texture, we calculated the aage response for the maximal response area (Omr) and forthe remaining neurons of the module (Ob). For each set withfixed SNR, the mean valuesOmr and Ob over ten noisyimplementations were calculated. The value ofOmr /Ob

served as the criterion of sensitivity of the network to spanoise.

The results are shown in Table I. It can be seen tOmr /Ob increases with SNR in the input image for eatexture, so thatOmr /Ob may be used as an adequate criterifor noise sensitivity.

Figure 4 illustrates the responses of the neurons ofsector module to noise-free textures and Figure 5 to thwith noisy textures.

Any definition of the sensitivity threshold of an imageprocessing system has some uncertainty when defining sidetection and when determining the minimal detecta

FIG. 4. Discrimination of textures by the network~sector module!. ~a!checkerboard input textures,~b! cellular input textures~the numbers give thespatial frequencies!; ~c! responses of the network to checkerboard textur~d! responses of the network to cellular textures;~e! response intensity scaleModel parameters:v519, a (s)55.0.

TABLE I. Sensitivity of the network to spatial noise.

Imageset

Texture~with spatialfrequency, cycles per

module!

Omr /Ob

withoutnoise SNR51.5 SNR52 SNR53 SNR54 SNR55 SNR510

1 Checkerboard~12.5! 9.0 2.560.4 3.560.3 4.260.4 5.260.4 5.460.3 7.560.22 Checkerboard~8.3! 6.5 2.060.4 2.760.4 3.160.3 3.660.4 3.860.4 5.960.13 Cellular ~12! 4.5 2.360.4 3.160.4 3.360.5 3.760.4 3.860.4 4.160.34 Cellular ~8! 3.7 1.960.3 2.260.4 2.560.4 2.860.4 3.060.3 3.460.3

834Yu. D. Kropotov and I. Z. Kremen

rethfi-

re-ial-en-ly

nt

aliant

e-

r,

s

ro-

rs

SNR.6 For the sensitivity threshold of a neural network, theis the same uncertainty. Our experiments have shownthe network discriminates a texture with sufficient condence ifOmr /Ob exceeds 3.5.

FIG. 5. High-frequency spatial noise filtering by the network~sector mod-ule!. ~a! noisy checkerboard input textures~spatial frequency 12.5! ~b! noisycellular input textures~spatial frequency 8!; ~c! responses of the network tonoisy checkerboard textures;~d! responses of the network to noisy cellulatextures;~e! response intensity scale. Model parameters are the same aFig. 4.

835 J. Opt. Technol. 66 (9), September 1999

at

VII. CONCLUSION

The experimental data of the current studies lets usgard our canonical cortical module as a distinctive spatfrequency filter that can map different frequencies and oritations to different module areas and can reliabdiscriminate different textures in conditions of significanoise of the input signal.

1J. D. Kropotov, I. Z. Kremen’, and V. A. Ponomarev. ‘‘A realistic neurnetwork simulating a visual system and its usage in tasks of invarimage description,’’ Opt. Zh.65, No. 9, 40~1998! @J. Opt. Technol65,716 ~1998!#.

2D. H. Hubel,Eye, Brain and Vision~W. H. Freeman, New York, 1988;Mir. Moscow, 1990!.

3D. Marr, Vision: A Computational Investigation into the Human Reprsentation and Processing of Visual Information~W. H. Freeman, SanFrancisco, 1982; Radio i svyaz’, Moscow 1987!.

4N. N. Krasil’nikov, Theory of Image Transmission and Perception~Radioi svyaz’, Moscow, 1986!.

5B. K. P. Horn,Robot Vision~M.I.T. Press, Cambridge Mass, 1986; MiMoscow, 1989!.

6M. M. Miroshnikov, Theoretical Principles for Optoelectronic Device~Mashinostroenie, Leningrad, 1983!.

This article was published in English in the original Russian journal. Repduced here with stylistic changes by the Translation Editor.

for

835Yu. D. Kropotov and I. Z. Kremen