IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993

process is sampled at random transition times of the input sequence, giving rise to a random walk process. A bmi TJC-RS for which E[U] = E[V] is said to have zero average jitter, as the expected value of the timing jitter is zero (although the variance increases with time).

There is a straightforward generalization of the above capacity results to nonbinary memory less increments (dmi) TJC-RS. For a dmi TJC-RS with A levels, log( A-I) must be added to the capacity per run. Another direct generalization is to channels for which the output runlengths depend on both the input runlengths and the channel symbol that makes up the runs: P*(vl'u, x) . Then, flU; V) is to be replaced by f(U;VIX), with Px(O) = Px(l) = 1/2. For TIC that introduce amplitude uncertainties, there is a probability that successive runs by chance gct the same amplitude, and thus merge into a single run. Such channels are no longer run synchronous.

Computation of the capacity of the bmi TJC-RS is a special case of computation of "capacity per unit cost" as defined by Verdu [15]. Generalized Arimoto-Blahut algorithms for the computation of capacity formulas f(X; Y)/E[b(X)] exist in the literature (see [15]).

VII. CONCLUSION

We conclude that a timing jitter channel that models timing jitter between the transmitter and receiver as a random walk allows for exact determination of the channel capacity. This holds true even in the, from a practical point of view, very interesting case in which the innovations of the timing jitter depend on the lengths of the input runs. In OnT approach, the Baggen-Wolf channel is a timing jitter channel with memory, whose capacity remains an open problem. Another interesting topic for further research is obtained by the combination of time and amplitude uncertainty. Such channels are no longer run synchronous.

ACKNOWLEDGMENT

The author would like to acknowledge the very illuminating intro­duction to timing jitter by S. Baggen and J. Wolf [2]. Furthermore, the author would like to thank F. Willems of the Eindhoven University of Technology, The Netherlands, for interesting discussions on the subject as well as his reference to the recent work of Verdu [15]. The author would also like to thank his colleague I. van Tilburg and an anonymous referee for useful comments on earlier versions of the manuscript.

REFERENCES

[1] C. Shannon and W. Weaver, The Mathematical Theory ofCommunica­tion. Urbana, IL: Univ. of Illinois Press, 1949.

[2] S. Baggen and J. Wolf, "An information theoretic approach to timing jitter," in 11th Symp. Inform. Theory in the Benelux, Noordwijkerhout, The Netherlands, Oct. 1990, pp. 174-180.

[3] F. Willems, "Totally asynchronou, Slepian-Wolf data compre,sion," IEEE Trans. Inform. Theory, vol. 34, pp. 35--44, Jan. 1988.

[4] S. Verdu, "Capacity region of the symbol-asynchronous Gaussian mUltiple-access channel," IEEE Trans. Inform. Theory, vol. 35, pp. 733-51, July 1989.

[5] __ , "Multiple-access channels with memory with and without frame synchronism," IEEE Trans. Inform. Theory, vol. 35, pp. 605-19, May 1989.

[6] T. Berger and D. Tufts, "Optimum pulse amplitude modulation; Part II: Inclusion of timing jitter," IEEE Tran.,. Inform. Theory, vol. IT -13, pp. 209--16, 1967.

1067

[7] P. Vogel, "PAM-systems including timing jitter; A game-theoretical approach," Arch. Elektron. Uebertr.techn., vol. 40, no. 3, pp. 1653-1658, June 1986.

[8] Y. Tankik, "Comments on 'Pulse shape, excess bandwith, and timing sensitivity in PRS,'" IEEE Trans. Commun., vol 37, Aug. 1989.

[9J G. Agrawal and T. Shen, "Power penally due to decision-time jitter in optical communication systems," Electron. Lett., vol. 22, no. 9, pp. 450--451, Apr. 1986.

[10] A. Hekstra, "The capacity of the discrete memory less timing jitter chan­nel and its capacity in the case of weak synchronisation," Netherlands PIT Res. Rep. 975190, Nov. 1990.

[111 __ , "The capacity of the discrete memoryless timing jitter channel and its capacity in the case of weak synchronisation," 12th Symp. Inform. Theory in the Benelux, Vcldhoven, The Netherlands, May 1991, pp. 9-16.

[12] R. Gallager, Information Theory and Reliable Communiation. New York: Wiley, 1968.

[13 J R. Blahu� Principle, and Practice of Information Theory. Amsterdam, The Netherlands: Addison-Wesley, 1987.

l14] I. Csiszar and J. Komer, Coding Theorems for discrete memoryless system.. New York: Academic Press, 1981.

[15] S. Verdli, "On channel capacity per unit cost," IEEE Trans. Inform. Theory, vol. 36, pp. 1019-1030, Sept. 1990.

Majority Coset Decoding

Iwan M. Duursma

Abstract-In general, the basic algorithm lor decoding of aIgebraic­geometric codes does not correct up to the designed minimum distance. Pellikaan formulated a nonconstructive procedure to overcome this, More recently, Feng aud Rao suggested a constructive procedure for a particular class of codes. Their procedure for the general case is formulated,

Index Terms-Algebraic-geometric codes, algebraic decoding, coset decoding.

I. INTRODUCTION

Justesen et al. [4] observed that the Peterson algorithm could be generalized to codes from plane algebraic curves. As was shown by Skorobogatov and Vliidul, the restriction to plane curves and to a particular class of codes was not essential. Thus a basic algorithm for the decoding of an arbitrary geometric code is available. It corrects up to (d' - 1)/2 - g/2 errors. Improvements of this result either are not constructive and of high complexity [6], [9] or do not correct up to the designed minimum distance in general [1], [7]. Recently, Feng and Rao [3] formulated a procedure to decode up to the designed minimum distance with the same complexity as the basic algorithm.

We show that the procedure can be applied to an arbitrary algebraic geometric code. In our setup, a reduction step is added to the basic algorithm. In case the basic algorithm fails, a majority scheme is used to obtain an additional syndrome for the error vector. Thus, a strictly smaller coset containing the error vector is obtained. In this way, the basic algorithm is applied to a decreasiug chain of cosets and after

Mannscript received May 13, 1992; revised September 20, 1992. This work was supported by NWO, through Stichting Mathematisch Centrum.

The author is with the Department of Mathematics and Computing Science, Eindhoven University of Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands.

IEEE Log Number 9206379.

0018-9448/93$03.00 © 1993 IEEE

1068

finitely many steps the coset will be small enough for successful application of the basic algorithm. We call the procedure Majority Coset Decoding (in short MCD).

The repeated applications of the basic algorithm in the procedure can be carried out with one common set of data. For this, Feng and Rao [3] presented a scheme for the computations. The scheme is referred to as Modified Fundamental Iterative Algorithm (in short MFIA). In the last section, we point out that the computations can be carried out with the MFIA Thus the decoding procedure has the complexity of the basic algorithm, that is O(t2n + In).

Although based on the same idea, our procedure does not compare immediately with [3] when applied to the same codes. Our formulation involves square matrices of size t + g, whereas in [3] matrices of size2t + 9 are used. In [2], we presented our procedure in the setup of [3], as to establish the relation.

II. COSET DECODING In the following CI = CdD. Gil C F; is a fixed residue code

[5], [8]. Thus,P = {H, P2, . • • , Pn} is a set of n rational points on a nonsingular curve X / Fqn, D = H + P2 + . , . + Pn and Gl is a divisor defined over Fq with support disjoint from P. The codewords of C1 are of the forrn(resP, (1)), resP2 (1/), ... ,resPn (1)), for 1) E O(G1 - D). With 9 the genus of the curve;\:', the code CI has designed distanced7 = dcg(GtJ- (2g - 2). The dual code ct is equal to the functional code CL(D, G1), with codewords of the form (h(P1),h(I'2),··· ,h(Pn», for hE L(GIl.

For a rational point Poo (t P, let Go = G1 - Poo and let G2 = G1 + poo. We have an extension of residue codes

Co = Cn(D, Go) :J CI = Cn(D, G1) :J C2 = Cn(D, G2)'

(19)

Let e be a vector with

wt(e) ::; (d� -1)/2. (20)

We formulate a coset decoding procedure with respect to the extension of codes C1 :J C2 : for a given Yl E e + C, we show how to obtain Y2 E e + C2. Note that with condition (2) this is well defined. In case Y2 '" e the procedure can be repeated, till eventually the error vector is obtained. With a combination of the procedure and known algorithms the number of repetitions required can be bounded by the genus 9 of the curve (Remark 4), We may clearly assume that

C1 '" C2 and L(Gz) f. L(GtJ,

where the second assumption follows from the first. As with de­coding of linear codes in general, syndromes are crucial. We define syndromes as linear maps to the constant field.

is

Definition 1: With a vector Y = (YI, Y2, . . . • Yn) E FR we associate a one-dimensional syndrome Si (Y), for i = 0,1,2,

Si(Y): L(G;) ---> Fq,

h ...... L y1h(Pj). 1=1

Lemma 1: The one-dimensional syndrome is a coset invariant, that

S;(y) = S,(e) {o} y E c + Ci, i = 0,1,2.

Proo!, From the definition

Si(Y - e) = ° {o} Y - e E CL(D, G;).l.

The result follows with CL(D, G,) = Ci.l. (24)

o

IEEE '!'RA1\ISACTIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993

Lemma 2: For U E CI \C2 and for h E L(G2)\L(Gt) we have S2(u)(h) '" O.

Proof:G2 = G1 +Poo implies I(G2)-I(Gd ::; 1 and L(G2) =

L(Gt) ffi (It). With the lemma, we have

u E CI/\ u (t C2 '* SI(U) = 0 /\ S2(U) '" 0

and '* S2(u)(h) '" O. o The following is immediate from the lemmas.

Corollary 1: Let U E C1 \C2 and let YI E c + C1. There exists a unique A such that YI - AU E e + C2. For h E L( G2), it satisfies

.\S2(u)(h) = S2(YI)(h) - S2(e)(h),

where S2(u)(h) f. 0, for h E L(G2)\L(Gt}.

By Corollary 1, it suffices for coset decoding to find a functionh E L(G2)\L(G,) with S2(e)(h) = O. We give a procedure to obtain functions!, 9 such that h = ! 9 will do.

III. Two-DIMENSIONAL SYNDROMES

Definition 2: For a vector c with (2) we consider the cosets e + C; with syndromes Sire), for i = 0,1,2. With a divisor F, that has support disjoint from P, we associate a two-dimensional syndrome

S, (F), for i = 0,1,2,

S,(F) : L(F) x L(G, - F) ---> Fq. (j,g) ...... Si(e)(jg).

Let lI.-, (F) be defined as the subspace of L(F) with

f E K,(F) {o} 'if 9 E L(Gi - F): Si(F)(f,g) = o.

Lemma 3: The containments

I�-I (F + Pool :J lI.-o(F), KI (F + Pe>e) :J 11.-2 (F + Poo),

Eo(F) :J I{I(F). and K2(F + Pool :J I{t(F).

define factor spaces of dimension at most one. Proo!, The inclusion relations are immediate from the

definition. In particular, Ko(F) = Kt(F + Pool n L(F). Since E I (F + P oo) C L (F + P"" ) , the intersection reduces the dimension by at most one. Similar for K2(F + Pool :J KI(F). For the inclusion Ko(F) :J KI(F), we observe that I'I(F) is obtained by applying at most one linear condition to I{o(F). Similar for I'I(F+Poo):J I'2(F+Poo). 0

The inclusion relations may be depicted as follows Et(F+ Poo)

/ � Eo(F) I'2(F + Pool

� / E1(F)

As the next lemma explains, we are interested in the situation where the following conditions are satisfied:

]{I (F + Poo) '" Ko(F), (A1)

Ko(F) = J{I (F), (A2)

L(Gl - F) '" L(G1 - F - Pool. (A3) E1(F+Pcc)= l{2(F+Poo), (B.l)

K2(F + r=) '" K1(F). (8.2)

Let (A) {o} (A.1) /\ (A.2) /\ (.4.3). Also let (B) {o} (B.1) /\ (B.2). Corollary 2: The conditions satisfy

(A.l) /\ (B.l) {o} (A.2) /\ (B.2). Proo!, Immediate from Lemma 3. o

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 39, NO, 3, MAY 1993

Lemma 4 (Main Lemma): Let the functions I, 9 satisfy

1 E Kl(F + P=)\Ko(F), and

9 E L(G] - F)\L(GI - F - P=),

Then,

fg E L(G2)\L(Gd,

and

Proof: From t E L(F + P oo) and 9 E L(GI - F), we obtain Ig E L(GI + P=) = L(G z) , For Ig � L(Gl) it suffices to consider the pole order -v=(fg) at P= and

-voo(fg) = -v=(f) - v=(g)

= ordp=(F + P=) + ordp=(Gl - F)

> ordp=(GJ). Note that 9 E L(Gz - F - P=). With f E K2(F + P=), we have

S2e(fg) = Sz(F + P=)(f, g) = O.

By the remark following Corollary 1, we can choose h = f 9 with t, 9 as in the lemma, provided that conditions (AI), (A.3), and (B.l) hold. With the corollary the conditions (A) and (B) need to be fulfilled. In the decoding situation we cannot determine K z (F + P oo ) and we are unable to verify (B). To overcome this we use a majority scheme.

IV. A MAJORITY SCHEME

Remark 1: Let the notation be as in Definition 2. Let the divisor Pe = 2:, .;to Pj' With Definition 1 and Definition 2, we see, for i = 0,1,1.

1 E L(F - Pel V 9 E L(G, - F -Pel => Si(F)(f,g) = o.

Lemma 5: Consider the conditions

L(F + Poo - Pel 1= L(F - Pel, L(Gl - F -Pe) 1= L(GI - F- Poo - Pel·

(C.l)

(C.2)

1069

The next theorem may be read with l' = O. The case l' > 0 is relevant only in reducing the computations when coset decoding is applied more than once.

Theorem 1 (Main Theorem): Let Co :J Cl :J C2 be an extension of residue codes as in (1). Let the genus 9 2: 1. Let the numbers t, r 2: 0 satisfy 2t + l' + 1 S deg(Gd - 2g + 2. For a vector e of weight wt(e) S t, we consider the cosetse+Ch for j = 0,1,2. Let Fa be an arbitrary divisor of degree t with support disjoint from D. Also, we define F, = Fa + iPoo, for i = 0,1"" , 2g - 1. Let

1= {r, r + 1,···, 2g - 2},

Is = {i E II (A) 1\ (B), for F = F;},

I'B = {i E I I (A) 1\ ...,(B). for F = F,}.

Then, at least one of the following holds:

L(GI - F2g-1 -Pe -1'Poo) i= O.

L(Fr - Pel i= O.

#Is 2: #I'B + 1.

Proof.' The assumption on wt( e) yields

deg(F2g_l - Pel ;::: 2g - 1,

deg(Gl - FrPe) 2: 2g � 1.

Let the sets 10 and IO be defined as

Ie = {i E II (C.I) 1\ (C.2). for F = Fd, I� = {i E I HC.2), for F + Fi},

I� = {i E I 1...,(C.l) 1\ ...,(C.2), for F = Fd.

(3)

(4)

(5)

(6) (7)

By Corollary 2 and Lemma 5, we have 10 C Is and I'B C 1'0. Thus, for (5) to hold, it suffices to prove

#10;::: #I� + 1. (8)

We may assume that (3) and (4) do not hold. In that case, we have

I(GI - F2g-l -Pel S r,

I(Fr - Pel = O.

Combination with (6) and (7) yields

I(F2g-l -Pel -1(Fe - Pel 2': g,

I(GI - Fr -Pel -1(GI - F2g-l - Pel ;::: 9 - r.

With the conditions (Al)-(A.3) and (B.1)-(B.2) as in the previous Also, section the following holds:

(C.I) => (A.1) 1\ (B.2), (C.2) => (A.2) 1\ (B.2) 1\ (A.3).

(C.l)

(C.2)

Proof.' Let (C.1) hold. For (AI) and (B.2), respectively, it suffices to give functions II E Kl (F + P=)\L(F) and h E Kz(F + P=)\L(F), respectively. With the remark, a function 1 E L(F + Poo - Pe)\L(F - Pel will do in both cases. Let (C.2) hold. Recall that Go = Gl - Poo. We may write L(Gl - F) = L(Go - F)+L(Gl - F - Pel. From Remark 1, we obtain Ko(F) C Kl (F). This proves (A2). For (B.l). we use Gl =

G2 - Poo and we write L(G2 - F -Pool = L(GI - F - Pool + L(Gz - F - Poo - Pel to obtain Kl(F+ Poo) C K2(F+ Pool· The condition (A3) follows immediately with Pe 2: o. D

#1 - #I� = #{i E II (C.l)} + #{i E II (C.2)} - #Ie,

2: (2g - r ) -#10.

With #1 = (2g -1 - r) , we obtain (8).

V. ApPLICATION

D

In the following remarks, we show the application of the theorem to decoding. Let Yl E e + Cl be given. If (3) or (4) holds, we can apply the basic algorithm. If both (3) and (4) fail, we can apply majority coset decoding.

Remark 2 (Basic Algorithm): We recall the basic algorithm [7, Theorem 1]: an error pattern e for the residue code Cf). (D, G) can be corrected if the following are satisfied for a divisor F:

L(F -Pel i= 0 and deg(G - F -Pel > 2g - 2. (9)

1070

In that case, we define a syndrome S(F) and a subspace K(F) C L(F) as in Definition 2 and the following reverse of Remark 1 holds

S(F)(J,g) = 0, 'if g E L(G - F) :=} f E L(F - Pel.

An error locator function f E L(F - Per can thus be obtained by solving for f E K(F)'. In the situation of Theorem 1, we have (6), (7), and the conditions (9) are satisfied for

G = G1 - rPexo, F = G1 - F2g-1 - rPoo, if (3) holds.

G = G1, F = Fr, if (4) holds.

Remark 3 (Majority Coset Decoding): If neither (3) nor (4) holds, the theorem tells us that (5) will hold. With the notation as in the theorem the decoding proceeds as follows. The set 1.4. =

{i E I I (Al, for F = Fi} is determined. We have IA = Ia U IE. With the possibly wrong assumption that IA = Ia application of Lemma 4 and Corollary 1 yields a vector Y2 = Yl - .xu, one for each i E IA. The unique vector Y2 with Y2 E e + C2 is obtained for i E la. By (5), this vector will occur with the highest multiplicity.

Remark 4 (Computations): by the previous remarks, we have an effective procedure to decode arbitrary algebraic geometric codes up to the designed distance. With the basic algorithm, one has to solve for 1 E f{ (F)* for suitable G, F. With majority coset decoding one has to solve for 1 E f{1 (F + Pool\Ko(F) in Lemma 4 for a number of G1, F. These computations dearly dominate the complexity of the procedure. They consist of solving a system of linear equations. We show that it suffices to consider one homogeneous system of linear equations Sx = O.

Let Cfl(D, G) be a code with deg(G) = 2g - 2 + 2t + 1. By replacing G with G - P=, if necessary, we may assume that deg(G) is odd. Let e be an error pattern of weight wt(e) -::; t. Let Fo be a divisor of degree t as in Theorem 1. The decoding procedure starts with G1 = G and, if necessary, continues with G1 = G+rPoo, r = 1,2, . . ·,g. For r = g, condition (4) in Theorem 1 holds and the basic algorithm will yield the error vector. Thus, the decoding procedure terminates after at most g repetitions. For eaeh G1 the following two-dimensional syndromes are considered

5o(F;) : L(Fi) x L(Go - Fi) --t 1�, 51 (F,) : L(Fi) X L(G1 - Fil ...."Fq

5t(Fi + P=) : L(F; + Pool x L(G1 - Fi - Pool"'" Fq, 5AFi + P=) : L(F; + Pool x L(Gz - F, - Pool"'" Fq,

where i = r, r + 1" " , 2g - 2, for G1 = G + I·P=. All syndromes are clearly compatible. They are restrictions of a map

5: L(Fzg-d x L(G - Fa) ...., Fq.

This map has a representation by a square matrix S of size t + g. To obtain compatible representations for the syndromes we choose the bases for L(F2g_tl alld L(G - Fo) as follows. Let

Also, let

L(Fo) = (/J,h,"',/I), L(G - Hg-l) = (gl ,g2,'" ,g",).

L(F2g-d = L(Fo) + (11+1,1/+2,"',1'+9)'

L(G - Fo) = L(G F2g-tl + (!7m+1, 9m+2,"', .9'+g),

such that

-V=(gm+l) < -V=(gm+2) < ... < -v=(gt+g),

-V=(fl+tl < -Voo(/I+2) < ... < -voo(ft+9)'

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993

The functions 1 E I\I(F + Poo)\Ko(F) and f E K(Fr correspond to (partial) relations among the rows of the matrix S. Note that by nature of the procedure some entries of S only become known in the course of the procedure, but all computations use known entries. Applying Gaussian elimination to the matrix S gives the partial relations as intermediate results. Thus, the overall complexity can be shown to be not larger than one application of the Gaussian elimination algorithm to the matrix S. For this, Feng and Rao formulated the Modified Fundamental Iterative Algorithm [3].

The definition of the matrix S has complexity O«i + g)�Tl) in general. The Gaussian elimination has complexity O(i + g)3) and for the overall complexity we obtain O(t"n + In), which is similar to the basic algorithm (of course the constant will be larger).

ACKNOWLEDGMENT

The author would like to thank Dr. G. L. Feng for discussions on the subject.

REFERENCES

[1] I. M. Duursma. "Algebraic decoding using special divisors," IEEE Trans. Inform. Theory, vol. 39, pp. 694-698, Mar. 1993.

[2] __ , "On the decoding procedure of Feng and Rao," presented at Proc. Algebraic and Combinut. Cuding Theury, Tamayo, Bulgaria, 1992.

[3] G. L. Feng and T. R. N. Rao, "Decoding algebraic geometric codes up to the designed minimum distance," IEEE Trans. Inform. Theory, vol. 39, pp. 37-45, Jan. 1993.

[4] J. Justesen, K. J. Larsen, H. E. Jensen, A. Havemose, and T. H¢holdt, "Construction and decoding of a class of algebraic geometric codes," TEEE Trans. Tnform. Theory, vol. 35, pp. 811-821, July 1989.

[5] J. H. van Lint and G. van der Geer, Introduction to Coding Theory and AlgebraiC Geometry. Basel: Birkhauser Verlag, 1988.

[6] R. Pellikaan, "On a decoding algorithm for codes on maximal curves," IEEE Trans. Inform. Theory, vol. 35, pp. 1228-1232, Nov. 1989.

[7] A. N. Skorobogatov and S. G. Vliidu!, "On the decoding of al, gebraic-geometric codes," IEEt.· Trans. Inform. Theory, vol. 36, pp. 1051-1060, Sept. 1990.

[8] M. A. Tsfasman and S. G. Vliidu(, Algebraic Geometric Codes. Dar· drecht: Kluwer, 1991.

[9] S. G. VHidu!, "On the decoding of algebraic-geometric codes over F q for q 2: 16," 1t.·EE TrailS. Inform. Theory, vol. 36, pp. 1461-1463, Nov. 1990.

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 39, NO.3, MAY 1993

An Application of the Hopfield Model to Huffman Codes

Francesco Fabris and Giacomo Della Riccia

Abstract-The discrete neural network model due to Hopfield (1982), and his following developments, allow to tie the dynamical evolution of a neural network to a quaptity-the energy of the network-which is monotonically decreasing up to a minimum stable point. A network of this kind is able to "solve" efficiently some useful computational problems. An application of the neural Hopfield model to the source (optimal) coding problem is shown; this application gives the structure of a Hopfield network, which is able to calculate the lengths of the codewords that must he associated to a source alphabet so as to minimize their average length. The case of a stationary and memoryless block-to-variable-Iength (B-VL) source coding is discussed.

Index Terms-Source coding, lIuffman codes, adaptive coding, neural networks, parallel computation,

I. INTRODUCTIO�

We shall consider the continuous model [2], in which N "formal neurons," similar to that of Fig. 1, are connected by means of appropriate "weights" Tij (conductances); Tij connects the output of the j th neuron with the input of the i th neuron. The V;' s are the state variables (voltages, for example), while the Ii'S are constant biases (currents, for example) which are independent from the V; 's; the Ii'S are injected in the input node. Pi is the input resistance of the amplifier gi(A,U,), and C, is a capacity which simulates the synaptic delay. The velocity of the network evolution essentially depends on the time constant T = Pi Ci. The input-output relation gi (A,ui) of the amplifiers is sigmoidal, with a slope that depends on the parameter Ai (see Fig. 2). Under the hypotheses Tij = Tji, and Ai tends to infinity (for each i), it can be proved (see [2]) that the quantity

1 N IV N

[= -zLLTi,VjV; - LUi;, i=lj=l i�l

(1 )

called the energy of the network, always decreases in time until it reaches a minimum (relative or absolute) at which the evolution stops.

The stable state variables VI, V2,···, V N that we can observe at the end of the evolution may be interpreted as the "solution" of the computational problem that is implicit in the [ structure and in the parameters Tij, Ii. In other words, whenever there is a problem which can be expressed in terms of the minimization of a function similar to (1), a possibility does exist to build a Hopfield network which "solves" that problem, stopping its evolution on a stable point which is the "solution." Nevertheless, we have difficulties due to the presence of false minima for E, i.e., minima that are not the desired solutions; we refer the reader to [I] and [2] for a detailed discussion.

Several optimization problems have been introduced in the Hop­field model context: in [1] and [5] to build a content addressable memory; in [4] to obtain solutions for the traveling salesman problem;

in [3] and [6] to realize an AID converter; and in [3] again to implement a signal decision circuit and to solve linear programming

problems.

Manuscript received January 2, 1992; revised August 20, 1992. This work was supported by the Mimistrero dell'Universitii e della Ricerco Scientifica. This work was presented in part at the Eurocude 92, Udine, Italy, October 1992.

The authors are with the Dipartimento di Matematica e Informatica, Universit. degJi Studi di Udine, via Zanon 6, 33100, Udine, Italy.

IEEE Log Number 9207219.

1071

Ii

VI Til Ttl

V2 Ti2

VN TiN

Fig. 1. Structure of the I10pfieIil neuron.

Vi-gi(A-Ui)

o Ui

Fig. 2. Sigmoidal response of the amplifier g, (AU,).

In the following, we shall introduce a network which is able to give optimal solutions for block-to-variable-length source coding, i.e., to implement a real-time Huffman encoder. The main advantage is that we skip the execution of the Huffman algorithm, since the network gives us the solution in one step only. This can conveniently be exploited to perform an adaptive Huffman coding, i.e., a coding which follows the variations of the source probability distribution.

II. THE BLOCK�TO-VARIABLE-LENGTH SOURCE CODING PROBLEM

Let us consider an information source § with alphabet A = { aI , a2,' .. , a I< }. Each "letter" Ui has an a priori probability Pi to be generated. We suppose p, > 0 and Ei Pi = 1 for the probability distribution (PD), P = {PI, P2,'" ,PI(}. Let § be stationary and memoryless; the block-to-variable-lenglh (B-VL) source coding problem can be described in the following way (sec [7]): we want to realiL:e a translation from the letters of the primary alphabet A to the codewords of a secondary alphabet B (binary, for example) such that:

1) the code be uniquely decodable, i.e., the primary sequence of letters obtained from the secondary sequence of codewords is unique;

2) the codewords W" W2," ', WI{ have length It, 12,'" ,Ix such that the average length

K

1= LP,/, (2) i=l

be reduced to a mmImum. This problem has an optimal solution Zr, l�, .. . , lj{ which can be found by using the well-kflown Huffman algorithm [8]. By application of the Kraft inequality in the Huffman binary case, we can obtain the following equality [7]:

K

LTI: = 1. (3) i=l

0018-9448/93$03.00 © 1993 IEEE