6
Designs and devices for optical bidirectional associative memories Clark C. Guest and Robert TeKolste The bidirectional associative memory (BAM) is a powerful neural network paradigm that is well suited to optical implementation. The BAM is heteroassociative (of which autoassociative operation is a special case) and is guaranteed to converge to a stable final state regardless of the connection weight matrix used. The BAM is placed in a conceptual framework that facilitates comparison with other neural network models. Variations on the BAM such as the bidirectional optimal memory (BOM), the competitive BAM (CBAM), and the adaptive BAM (ABAM)indicate some of the interesting directions this simple structure may evolve, leading in a natural progression toward the power of a model such as the Carpenter-Grossberg ART. The simplicity of the BAM invites uncomplicated optical implementations. BAM designs based on optical matrix-vector multipliers (MVMs) and on volume holographic connections are presented. Spatial light modulator (SLM) device designs currently under development to support the MVM BAMs are given. 1. Introduction Neural networks are parallel distributed systems for processing information in a nonalgorithmic way. Typically they are configured as associative memories. Autoassociative memories can be used for pattern completion, error correction, and pattern classifica- tion. Heteroassociative memories perform mappings from one set of patterns onto another set and may be regarded as a generalization of the autoassociative case where both sets of patterns are the same. The bidirectional associative memory (BAM) intro- duced by Koskol is a heteroassociative structure and a generalization of the feedback autoassociative struc- ture used by Hopfield 2 that was originally analyzed by Cohen and Grossberg. 3 Through its bidirectional character it allows available information about exist- ing input conditions and expected higher-level pat- terns to be used in determining a stable classification state. It is the simplest form of a hierarchical neural network, and thus amenable to physically simple im- plementations; yet through its bottom-up/top-down functionality it is an approach to the power of the Carpenter-Grossberg adaptive resonance model (ART). 4 Optics is regarded as a promising technology for neural networks because of its ability to economically provide massively parallel interconnections. The BAM structure is particularly attractive for optical implementation since its simplicity offers uncompli- cated implementations yet it exhibits powerful pro- cessing capabilities. 5 Furthermore, as will be illus- trated in later sections, optical systems provide The authors are with University of California, San Diego,Depart- ment of Electrical & Computer Engineering, La Jolla, California 92093. Received 10 August 1987. 0003-6935/87/235055-06$02.00/0. © 1987 Optical Society of America. naturally the bidirectional connections required for the architecture. Section II will briefly review the structure and prop- erties of the BAM, as well as some interesting varia- tions on its basic structure. Next, the BAM will be placed in context with other neural net paradigms within a unifying conceptual framework. The precur- sors to optical BAM architectures will then be re- viewed, and it will be shown that several existing opti- cal neural net implementations have a BAM-like structure. Next implementations of BAMs based on optical matrix-vector multipliers will be given, fol- lowed by a presentation of devices under development at UCSD to support these systems. Finally, optical BAM designs based on volume holography will be pre- sented. 11. Bidirectional Associative Memory Principles The bidirectional associative memory (BAM) is the simplest form of a multilayer neural net. It consists of two fields of processing elements, which will be re- ferred to as the A field and the B field in this paper. Each element receives inputs from all elements in the opposite field, but none from elements in its own field, as shown in Fig. 1. A description of the BAM that puts it in context with other network architectures is given in Sec. III. Each of the neural processing elements forms a weighted sum of its inputs. A threshold is applied to this sum to give the output value of the element. The threshold may be a sharp step threshold as in the McCulloch and Pitts model, 6 or it may be a more gradual, monotone increasing, sigmoidal func- tion. Inputs to the BAM system are brought in as extra input lines to the processing elements in one or both fields (typically both). System outputs are taken as the input lines of some or all of the processing elements. BAM operation begins with the presenta- tion of initializing values on the input lines. The state of the processing elements may subsequently be up- 1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5055

Designs and devices for optical bidirectional associative memories

  • Upload
    robert

  • View
    213

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Designs and devices for optical bidirectional associative memories

Designs and devices for optical bidirectional associativememories

Clark C. Guest and Robert TeKolste

The bidirectional associative memory (BAM) is a powerful neural network paradigm that is well suited tooptical implementation. The BAM is heteroassociative (of which autoassociative operation is a special case)and is guaranteed to converge to a stable final state regardless of the connection weight matrix used. TheBAM is placed in a conceptual framework that facilitates comparison with other neural network models.Variations on the BAM such as the bidirectional optimal memory (BOM), the competitive BAM (CBAM),and the adaptive BAM (ABAM) indicate some of the interesting directions this simple structure may evolve,leading in a natural progression toward the power of a model such as the Carpenter-Grossberg ART. Thesimplicity of the BAM invites uncomplicated optical implementations. BAM designs based on opticalmatrix-vector multipliers (MVMs) and on volume holographic connections are presented. Spatial lightmodulator (SLM) device designs currently under development to support the MVM BAMs are given.

1. Introduction

Neural networks are parallel distributed systems forprocessing information in a nonalgorithmic way.Typically they are configured as associative memories.Autoassociative memories can be used for patterncompletion, error correction, and pattern classifica-tion. Heteroassociative memories perform mappingsfrom one set of patterns onto another set and may beregarded as a generalization of the autoassociative casewhere both sets of patterns are the same.

The bidirectional associative memory (BAM) intro-duced by Koskol is a heteroassociative structure and ageneralization of the feedback autoassociative struc-ture used by Hopfield2 that was originally analyzed byCohen and Grossberg.3 Through its bidirectionalcharacter it allows available information about exist-ing input conditions and expected higher-level pat-terns to be used in determining a stable classificationstate. It is the simplest form of a hierarchical neuralnetwork, and thus amenable to physically simple im-plementations; yet through its bottom-up/top-downfunctionality it is an approach to the power of theCarpenter-Grossberg adaptive resonance model(ART).4

Optics is regarded as a promising technology forneural networks because of its ability to economicallyprovide massively parallel interconnections. TheBAM structure is particularly attractive for opticalimplementation since its simplicity offers uncompli-cated implementations yet it exhibits powerful pro-cessing capabilities. 5 Furthermore, as will be illus-trated in later sections, optical systems provide

The authors are with University of California, San Diego, Depart-ment of Electrical & Computer Engineering, La Jolla, California92093.

Received 10 August 1987.0003-6935/87/235055-06$02.00/0.© 1987 Optical Society of America.

naturally the bidirectional connections required forthe architecture.

Section II will briefly review the structure and prop-erties of the BAM, as well as some interesting varia-tions on its basic structure. Next, the BAM will beplaced in context with other neural net paradigmswithin a unifying conceptual framework. The precur-sors to optical BAM architectures will then be re-viewed, and it will be shown that several existing opti-cal neural net implementations have a BAM-likestructure. Next implementations of BAMs based onoptical matrix-vector multipliers will be given, fol-lowed by a presentation of devices under developmentat UCSD to support these systems. Finally, opticalBAM designs based on volume holography will be pre-sented.

11. Bidirectional Associative Memory Principles

The bidirectional associative memory (BAM) is thesimplest form of a multilayer neural net. It consists oftwo fields of processing elements, which will be re-ferred to as the A field and the B field in this paper.Each element receives inputs from all elements in theopposite field, but none from elements in its own field,as shown in Fig. 1. A description of the BAM that putsit in context with other network architectures is givenin Sec. III. Each of the neural processing elementsforms a weighted sum of its inputs. A threshold isapplied to this sum to give the output value of theelement. The threshold may be a sharp step thresholdas in the McCulloch and Pitts model,6 or it may be amore gradual, monotone increasing, sigmoidal func-tion. Inputs to the BAM system are brought in asextra input lines to the processing elements in one orboth fields (typically both). System outputs are takenas the input lines of some or all of the processingelements. BAM operation begins with the presenta-tion of initializing values on the input lines. The stateof the processing elements may subsequently be up-

1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5055

Page 2: Designs and devices for optical bidirectional associative memories

dated one at a time in a randomly ordered fashion, orthe fields may update all their elements simultaneous-ly in alternation. In our experience, the latter methodprovides faster convergence with no degradation inperformance. In either case, the system quickly con-verges to a resonant state with the patterns in bothfields remaining constant for all further iterations.

It has been rigorously proved1 that a stable finalstate results regardless of the set of interconnectionweights used. This is to be contrasted with otherfeedback associative memory structures that require asymmetric matrix of connection weights. This guar-antee of stability opens the way for investigation ofvery general learning algorithms.

The BAM is trained to associate pairs of vectors.For each vector pair, the outer product is taken. Thesum of these outer product matrices gives the set ofconnection weights used in the BAM.

The basic BAM structure suggests many interestingvariations. These variations follow two directions:modified training laws, and additional structured con-nections.

One variation of the BAM that we have investigatedreplaces the weight matrix obtained from vector outerproduct training with optimal associative mappingmatrices. The forward and backward connection ma-trices are not transpose pairs in this case, but arederived using the Greville algorithm as indicated byKohonen7 to give the best mapping in the least-squaressense for the forward and backward directions. Be-cause the matrices are not transpose pairs, the entireconnection matrix is not symmetric and rigorous proofof the system stability is not available. However, ex-amination of these systems has not once encounteredan oscillatory situation, leading to the strong suspicionthat the systems can be guaranteed stability and aproof is being sought. The advantage of the direction-al optimal memory BOM is in memory capacity. Atrial system having eight elements in one field and tenin the other can store five nonorthogonal patterns asstable states. This represents a considerable improve-ment over conventional autoassociative memories thattypically have a storage capacity of only 15% of thenumber of neural elements. However, the BOM, likethe BAM and other associative systems, exhibits spuri-ous stable states that do not correspond to trainedpatterns. Their effect is to decrease the radii of con-vergence (in the Hamming sense) for the stored states.In no case, though, has a stored state proved to beunstable.

Spurious states often take the form of complementsof the desired state; a problem the BOM shares withthe BAM and the Hopfield network. These comple-ment states can be circumvented by making the neuralelement fields one element longer than the vectors tobe memorized. Then training vectors are encoded aschanges of state between adjacent neural elements, i.e.,a 1 in the nth position of a training vector requires thestates of the nth and (n + 1)th elements in the neuralfield to be opposite in sign, a 0 in the training vectorrequires they be of the same sign. The state of the first

I I

Reid A

Reld B

( (VX S SE~~~~~~Emert9

; 0~~~~~~~~~en

Fig. 1. The BAM is composed of two fields of processing elements.Weighted connections exist between the fields, but not within them.

neural element in a field is arbitrary and may be chosento alternate for successive training vectors. Duringrecall, a Boolean EXCLUSIVE-OR operation is appliedbetween neighboring elements in the output field torecover the data, whether the stored state or its com-plement has been recalled.

An alternative learning prescription for the BAM isto make it continuously adaptable. Kosko proposesthat the adaptive BAM (ABAM) use the signal Hebblaw8 with dwij/dt = - kw1j + S(ai)S(bj), where wij is theconnection weight from the ith element of the A fieldto the jth element of the B field, a and b are thecurrent states of the elements, k is a constant, andS( ) is a sigmoidal function.

Another variation on the BAM theme introducescompetition within the fields through untrained in-hibitory interconnections.9 Stability is guaranteedwith symmetric inhibitory connections. This intra-field competition provides pseudonormalization of ac-tivation patterns and contrast enhancement of reso-nant patterns. As will be explained in Sec. VI, theeffect of these inhibitory connections may be moreefficiently implemented by a single global signal.

Ill. Conceptual Framework

With the proliferation of neural net architectures,the problem of placing them in a common context forpurposes of comparison becomes formidable. Thissection presents the BAM and its variations in a com-mon format with several other neural net architecturesto facilitate insight into their similarities and differ-ences. The presentation concentrates on the struc-ture of the connection matrix for each architecture.Issues such as weight update laws and neuron firingrules must be considered independently.

The common features uniting many neural net ar-chitectures are the multiplication of an input vector bya matrix of weights, followed by the application of anonlinear threshold to the components of the productvector. Many network architectures feed the result ofthe threshold operation back to the input of the systemto institute an iterative or dynamic mode of operation.The structure of most neural net paradigms can besummarized with the aid of diagrams such as those inFig. 2.

The model that Hopfield has used2:' 0is shown in Fig.2(a). The connection matrix is represented by the

5056 APPLIED OPTICS / Vol. 26, No. 23 / 1 December 1987

Page 3: Designs and devices for optical bidirectional associative memories

a)"Hopfield Model" b)Two Independent Systems c)Hierarchical System

ADO 0~~~~~~~~~~~~~~~~~~~~10

d) Three leuel hierarchy e)BRM f)CBRM

Fig. 2. Structure of the connection matrix for several neural net-work models is compared: (a) the Hopfield model, (b) two indepen-dent systems, (c) two connected systems, (d) hierarchical systemwith connections between adjacent levels (D,E) and nonadjacent

levels (), (e) the BAM, (f) the CBAM.

block labeled S, and the threshold units are represent-ed by the block labeled T. The outputs of the thresh-old units are fed back in parallel to the inputs of theconnection matrix. The connection matrix that Hop-field prescribes is a symmetric, zero-diagonal matrix.This system performs autoassociative recall but isknown to have low storage capacity for randomly cho-sen vectors and produces spurious states that do notcorrespond to training vectors.

A multilevel or hierarchical system can be represent-ed by partitioning the connection matrix and thethreshold element vector. To start, two independentnetworks can be represented on the same diagram as inFig. 2(b). These systems may in general be of differ-ent size, and the threshold elements may have differ-ent input-output and temporal responses if desired.Introducing the connection matrices C12 and C21 in Fig.2(c) combines the two independent systems into asingle hierarchy. The matrix C12 mediates the con-nection of outputs from system 1 to the inputs ofsystem 2, and C21 provides feedback from system 2 tosystem 1. This concept may be extended to a hierar-chy containing any number of levels, as indicated inFig. 2(d). Figure 2(d) shows the case when communi-cation paths exist that in some instances skip overlevels in the hierarchy.

In distinction to the hierarchical system, only theinterconnection portions of the matrix are nonzero forthe BAM, as shown in Fig. 2(e). The specification forthe BAM requires that C12 is the transpose of C21.

The BOM can be represented by the structure usedfor the BAM except the C12 and C21 matrices do nothave a transpose relationship, but are calculated inde-pendently as optimal mappings using Greville's algo-rithm.

The CBAM adds intrafield connections to the basicBAM structure. Each processing element has an in-hibitory connection to every other element in its fieldas shown in Fig. 2(f).

IV. Predecessors to the Optical BAM

Optical implementations of content-addressableand associative memory have been studied since theearly 1960s.1"-4 The majority of optical neural net-works currently being studied fall into one of two cate-gories: matrix-vector multipliers (MVMs)15-18 andholographic correlators.19-21 Within each of these cat-egories, BAM-like structures have already been inves-tigated, although not for their properties as BAMs.Athale et al.22 have demonstrated a small autoassocia-tive system of the MVM type that can be extended toBAM operation, as will be explained in Sec. V. Lead-ing holographic correlator systems at Hughes19 andNorthrop20 use bidirectional propagation of lightthrough a hologram to implement the memory matrix.Both systems use a phase conjugate resonator with onephase conjugate mirror (PCM) providing threshold-ing. By making both PCMs nonlinear, either systemmay implement a holographic BAM. HolographicBAM system designs that do not utilize PCMs will bepresented in Sec. VII.

V. Optical Matrix-Vector Multiplier BAM

Optical BAMs using an MVM approach are similarin design (see Fig. 3) to the first optical neural netsystem presented by Farhat and Psaltis.15 The BAMdiffers from usual MVMs in that there are sources anddetectors on both sides of the connection matrix. Thetwo fields of this optical BAM are implemented aslinear arrays of paired detector and source devices.The light falling on a detector is converted to an elec-trical signal, amplified, passed through a thresholdcircuit, and ultimately drives its associated opticalsource. Section VI will present devices currently un-der development to provide these functions in a mono-lithic package. The connection matrix is implement-ed as photographic film for fixed connections, or as a 2-D electrically addressed spatial light modulator fortrainable connections. This BAM MVM has the ad-vantage of not requiring long electrical feedback pathsfrom the detectors to the sources; all electrical connec-tions are local. As with all optical MVMs, bipolarinput values and connection weights must be handled

Detector-SourcePair ArrayDetector-Source

Pair Array

Fig. 3. This BAM implementation based on an optical matrix-vector multiplier uses linear arrays of paired detector and sources forthe neural element fields. Light passes through the connection

matrix in both directions.

1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5057

Page 4: Designs and devices for optical bidirectional associative memories

by doubling the number of elements, wavelength mul-tiplexing, or some other scheme.

If the detectors, sources, and associated circuitry areto be fabricated in a single device, it may be desirableto have both fields of the BAM in the same plane. Asystem architecture that allows this and cuts systemvolume by 50% is shown in Fig. 4. A spatial lightmodulator that operates in the reflective mode mayconveniently be used to implement the interconnec-tion matrix. Note, light passes twice through the con-nection matrix mask in this configuration, so the singlepass mask transmittance must be the square root of thedesired value. Also, to account for the center elementwhere the fields of the BAM intersect, an opaque crossis included in the connection matrix.

An even more compact implementation of the BAMis possible if devices such as those shown in Fig. 5 canbe used. The device consists of an array of singleelement line detectors paired with optical modulatorelements of a similar form. Light falling on a detectorcauses its associated modulator to become more trans-parent. Two such device arrays, oriented orthogonal-ly as shown in Fig. 6, may be used to implement anoptical BAM. The connection matrix is a transparen-cy placed between the two devices. Light is intro-duced onto both faces of the resulting assembly. Thepossibility of using linear device arrays of this form toimplement sandwich MVMs was realized indepen-dently by Athale.2 3

VI. Device Development for the MVM BAM

High performance optical BAM implementationswill benefit from the availability of devices specializedfor the particular system requirements. UCSD has along-standing program of SLM research24 that may beapplied to production of optical BAM devices. Theapproach is to use hybrid systems with silicon detec-tors and circuitry and PLZT modulators. Threemethods are under current investigation: sputteringPLZT onto a silicon substrate, recrystallizing amor-phous silicon deposited on PLZT, and depositing bothsilicon and PLZT onto a sapphire substrate. Many ofthe techniques learned will carry over to systems thatutilize new high-speed low-power modulator devices,such as multiple quantum wells or nonlinear organicmaterials. At present, a linear array of sixteen paireddetectors and sources that can be used in the system ofFig. 3 has been produced using a flip-chip bondingtechnique. Work is under way to produce an arraywith 128 pairs in a hybrid device.

An array of the type shown in Fig. 5 is also underdevelopment. A simple first approach is to sputterstripes of amorphous silicon onto a PLZT wafer. Ametallization pattern then allows each silicon stripe toact as a photoconductive detector that applies an in-tensity dependent voltage across the neighboringPLZT modulator region. The inherent sin2 character-istic of electrooptic modulation provides the nonlinearresponse required of the neural processing elements.

The SLM devices for both the imaging MVM BAMand the sandwich MVM BAM may be easily modified

Source-DetectorPair Arrays

ConnectionMatrix

Lens

Mirror

Fig. 4. This optical matrix-vector multiplier BAM system is foldedwith a mirror to allow both BAM fields to lie in the same plane.

opticalModulator

OpticalDetector

Silicon DriveCircuitry

Fig. 5. This spatial light modulator device consists of alternatingstripes of silicon photodetectors and electrooptic modulators. Thesignal from each detector is amplified and thresholded by silicon

circuitry that then drives the associated modulator.

Linear Modulator Connection Linear ModulatorArray Matrix Array

Light E d 1 2 & 0 Ly w g F , In~~~~~~~put

Fig. 6. Two line modulator arrays can be used with a connectionmatrix transparency to implement a sandwich form of optical ma-

trix-vector multiplier BAM.

to provide the intrafield competition to implement aCBAM. Rather than implementing a large number ofinhibitory connections, a single global signal, schemat-ically indicated in Fig. 7, may be used to achieve thesame effect. This means of controlling the number ofactive units in a field has been successfully employedin computer simulations of neural nets.25

The most interesting abilities of neural network sys-tems stem from the adaptation of their connectionweight matrices. SLM devices specialized to provideadaptable optical connections are also being contem-plated. One possible design is shown in Fig. 8. The

5058 APPLIED OPTICS / Vol. 26, No. 23 / 1 December 1987

Page 5: Designs and devices for optical bidirectional associative memories

configuration is most easily understood if the deviceshown is imagined as the matrix mask between thetwo-line modulator arrays in the sandwich BAM ofFig. 6, although it will work for other systems as well.The device will be fabricated on a transparent sub-strate so that light may reach it from both sides. Asingle element of the matrix is shown in the figure.The area covered by the element is partitioned intofour regions. The first region lies between a modula-tor in the A field and a detector in the B field, thesecond region between the modulator in the B field andthe detector in the A field, the third region betweenmodulators for both fields, and the fourth between thedetectors of both fields. The first two regions will beoccupied by modulator elements that determine theBAM connection strengths in the forward and reversedirections, respectively. The third region will containtwo detectors, one sensitive to light coming from the Afield and the other to light coming from the B field.The fourth region, where no light falls, can containanalog or digital circuitry that uses signals from thedetectors to determine the transmittance of the for-ward and backward modulator elements. If sufficientroom is available, the circuitry can incorporate multi-ple memory locations, thereby allowing the connectionarray to rapidly switch between different matrices.This function could be used to allow a connection arrayto implement a much larger virtual array through timemultiplexing.

VII. Holographic BAMs

Holographic implementations of neural networksare attractive for several reasons. First, volume hol-grams have potentially very high information storagecapacity; the theoretical limit is in excess of 1012 bits/cm3 of material.26 27 Current systems fall far short ofthis limit, but still appear to offer greater capacity thancan be achieved with 2-D connection transparenciesused in MVMs. Second, if photorefractive crystals orother real-time holographic recording media are used,the potential exists for systems with innate adaptabil-ity. Also, connection weights are stored in distributedform, making the systems highly fault tolerant. Final-ly, holographic systems are a more natural way toperform neural processing on image data, meaning lesspreprocessing and reformatting of data are requiredthan for MVMs. For these reasons, holographic im-plementations of BAMs are being considered.

The BAM system design shown in Fig. 9 uses areflection volume hologram to implement the proces-sor interconnections. The two BAM fields exist sideby side on a single device. The device provides ampli-fication of an input image followed by thresholding. Aliquid crystal light valve, a microchannel spatial lightmodulator, or two-wave mixing in a photorefractivecrystal are candidate implementations for this func-tion. Light from the two BAM fields is polarizedorthogonally so that they may follow different pathsaround the system. Light from the output side of theA field is reflected from the hologram and passes to theinput side of the B field. Light from the output side of

GlobalCompetitionSignal Line

Detector

Modulator

Fig. 7. Single signal line carrying a voltage proportional to thenumber of active elements can be used to produce the same effect as

inhibitory connections between all elements in a field.

Line Adaptaion X llBackward

Field R

Modulejoh Maskr..............

Modulator Det~~oltor

Field B

Fig. 8. This element of a spatial light modulator device contains allthe components needed to implement an adaptive BAM connection

matrix.

Mirror Mirror

VoumeReflectionHologram

Fig. 9. Interfield connections in this BAM implementation areimplemented with a volume reflection hologram.

the B field is reflected from the hologram and passes tothe input side of the A field. Note first that the beamsused for reading the holographic gratings are not in aconfiguration to write new reflection gratings. Thusthe operation of learning is decoupled from data pro-cessing. This is not a liability since the innate adapta-tion characteristic of the holographic material is sel-dom what is desirable from an algorithmic standpoint.For learning to occur in this system, separate beamsmust be brought in to the other side of the hologram;their intensity and duration may be adjusted to give acontrolled learning characteristic. The second point

1 December 1987 / Vol. 26, No. 23 / APPLIED OPTICS 5059

Page 6: Designs and devices for optical bidirectional associative memories

ThresholdDevice

Confocal Spherical Mirrors

Fig.10. Volume transmission hologram is used in this BAM design.

to note is that the system does not require PCMs or anysort of resonant cavity. This can be expected to im-prove system stability and reduce system complexityand power consumption.

A design for a BAM using a transmission hologram isshown in Fig. 10. Again, some type of nonlinear opti-cal amplifier is used for the BAM fields. In this casethe hologram reading beams are in a configuration towrite new gratings, provided the polarizations of thebeams are aligned. The confocal mirror configurationis used only as a convenient way of performing Fouriertransform operations and folding the optical path; theoptical cavity is not used as a resonator.

The two holographic BAM configurations presentedabove are given as illustrative examples of the formthat holographic BAMs might take. Numerous prac-tical details need to be worked out before holographicsystems are practical. Obvious details missing fromthe systems above include practical implementation ofthe nonlinear amplifier and coding of the data fields toallow an appreciable fraction of the storage capacity ofthe volume hologram to be utilized.

Vil. Conclusions

The BAM is a powerful neural network paradigmthat is well suited to optical implementation. TheBAM is heteroassociative (of which autoassociativeoperation is a special case) and is guaranteed to con-verge to a stable final state regardless of the connectionweight matrix used. It has been placed in a conceptualframework that facilitates comparison with other neu-ral network models. Variations on the BAM such asthe BOM, the CBAM, and the ABAM indicate some ofthe interesting directions this simple structure mayevolve, leading in a natural progression toward thepower of a model such as the Carpenter-GrossbergART. The simplicity of the BAM invites uncompli-cated optical implementations. BAM designs basedon optical MVMs and on volume holographic connec-tions have been presented. SLM devices designed tosupport the MVM BAMs are being designed and built.These devices will allow optical implementations toevolve toward more powerful systems as the BAMevolves.

References

1. B. Kosko, "Bidirectional Associative Memories," IEEE Trans.Syst. Man Cybern. SMC (1987), in press.

2. J. J. Hopfield, "Neural Networks and Physical Systems withEmergent Collective Computational Abilities," Proc. Natl.

Acad. Sci. U.S.A. 79, 2554 (1982).3. M. A. Cohen and S. Grossberg, "Absolute Stability of Global

Pattern Formation and Parallel Memory Storage by Competi-tive Neural Networks," IEEE Trans. Syst. Man Cybern. SMC-13, 815 (1983).

4. G. A. Carpenter and S. Grossberg, "A Massively Parallel Archi-tecture for a Self-Organizing Neural Pattern Recognition Ma-chine," Comput. Vision Graphics Image Process. 37, 54 (1987).

5. B. Kosko and C. Guest, "Optical Bidirectional Associative Me-mories," Proc. Soc. Photo-Opt. Instrum. Eng. 758, (1987), inpress.

6. W. S. McCulloch and W. Pitts, "A Logical Calculus of the IdeasImminent in Nervous Activity," Bull. Math. Biophys. 5, 115(1943).

7. T. Kohonen, Self-Organization and Associative Memory(Springer-Verlag, New York, 1984).

8. B. Kosko, "Adaptive Bidirectional Associative Memories,"Appl. Opt. 26, 4947 (1987).

9. B. Kosko, "Competitive Adaptive Bidirectional Associative Me-mories," in Proceedings, International Conference on NeuralNetworks, San Diego (June 1987).

10. J. J. Hopfield, "Neurons with Graded Response have CollectiveComputational Properties Like Those of Two-State Neurons,"Proc. Natl. Acad. Sci. U.S.A. 81, 3088 (1984).

11. G. R. Knight, "Holographic Associative Memory and Proces-sor," Appl. Opt. 14, 1088 (1975).

12. M. Sakaguchi, N. Nishida, and T. Nemoto, "A New AssociativeMemory System Utilizing Holography," IEEE Trans. Comput.C-19, 1174 (1970).

13. V. N. Morozov, "Associative Parallel Search Memory," Sov. J.Quantum Electron. 8, 1 (1978).

14. C. C. Guest and T. K. Gaylord, "Truth-Table Look-Up OpticalProcessing Utilizing Binary and Residue Arithmetic," Appl.Opt. 19, 1201 (1980).

15. N. H. Farhat, D. Psaltis, A. Prata, and E. Paek, "Optical Imple-mentation of the Hopfield Model," Appl. Opt. 24, 1469 (1985).

16. A. D. Fisher and J. N. Lee, "Optical Associative ProcessingElements with Versatile Adaptive Learning Capabilities,"Technical Digest of OSA Topical Meeting on Optical Comput-ing, vol. 11, pp. 137-140, March 1987.

17. N. H. Farhat, "Architectures for Optoelectronic Analogs of Self-Organizing Neural Networks," Opt. Lett. 12, 448 (1987).

18. Y. Owechko, G. J. Dunning, E. Marom, and B. H. Soffer, "Holo-graphic Associative Memory with Nonlinearities in the Correla-tion Domain," Appl. Opt. 26, 1900 (1987).

19. T. Jannson, C. Karagaleff, and H. Stoll, "PhotorefractiveLiNbO 3 as a Storage Medium for High-Density Optical NeuralNetworks," J. Opt. Soc. Am. A 3(13), P64 (1986).

20. E. G. Paek and D. Psaltis, "Optical Associative Memory UsingFourier Transform Holograms," Opt. Eng. 26, 428 (1987).

21. D. Z. Anderson and M. C. Erie, "Resonator Memories and Opti-cal Novelty Filters," Opt. Eng. 25, 434 (1987).

22. R. A. Athale, C. B. Friedlander, and B. G. Kushner, "AttentiveAssociative Architectures and Their Implications to OpticalComputing," Proc. Soc. Photo-Opt. Instrum. Eng. 625, 179(1986).

23. R. A. Athale, BDM Corp.; personal correspondence (May 1986).24. S. C. Esener, J. H. Wang, T. J. Drabik, M. A. Title, and S. H. Lee,

"One-Dimensional Silicon/PLZT Spatial Light Modulators,"Opt. Eng. 26, 406 (1987).

25. R. Hecht-Nielsen, "Nearest Matched Filter Classification ofSpatiotemporal Patterns," Appl. Opt. 26, 1892 (1987).

26. P. J. van Heerden, "Theory of Optical Information Storage inSolids," Appl. Opt. 2, 393 (1963).

27. T. K. Gaylord, "Digital Data Storage," in Handbook of OpticalHolography, H. J. Caulfield, Ed. (Academic, New York, 1979),pp. 379-414.

5060 APPLIED OPTICS / Vol. 26, No. 23 / 1 December 1987