300 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 1, JANUARY 1996

to maximize the free distance, even when the number of nearest neighbors is large. The performance of the new codes was compared to the best previously known-codes via computer simulation. It was shown that significant real coding gains can be achieved with large constraint length codes. Since the additional coding gain in going from v = 6 to v = 8 is large and the increase in complexity is moderate, the new v = 7 and v = 8 linear codes may be good choices for a V.34 bis modem standard.

A class of nonlinear trellis codes that can achieve full 90” rotational invariance was described and good codes from this class were found by exhaustive search. Simulation results show that a v = 8, fully rotationally invariant, 4D nonlinear trellis code achieves 0.3-dB real coding gain compared with the v = 6 ITU-T code at a BER of lor5. Moreover, it is only four times more complex than the U = 6 code when Viterbi decoding is used. Since the 0.4-dB real coding gain of the v = 6 code over the v = 4 code is only achieved with 16 times more complexity, the new v = 8 code would appear to be a good choice for a V.34 bis modem standard.

ACKNOWLEDGMENT

The authors wish to thank G. D. Fomey Jr. for his continuing interest, suggestions, and criticisms that have contributed to the de- velopment of this work. They also thank L. C. Perez, who contributed an important insight regarding the conditions for rotational invariance.

REFEENCES

G. Ungerboeck, “Channel coding with multileveUphase signals,” IEEE Trans. Inform. Theory, vol. IT-28, pp. 55-67, Jan. 1982. L. F. Wei, “Rotationally invariant convolutional channel coding with expanded signal space-Part 11: Nonlinear codes,” IEEE J. Select. Areas Commun., vol. SAC-2, pp. 672-686, Sept. 1984. A. R. Calderbank and N. J. A. Sloane, “Four-dimensional modulation with an eight state trellis code,” AT&TTech. J:, vol. 64, pp. 1005-1018, May-June 1985. -, “An eight-dimensional trellis code,” Proc. IEEE, vol. 74, pp. 757-759, May 1986. L. F. Wei, “Trellis-coded modulation with multidimensional constel- lations,” IEEE Trans. Inform. Theory, vol. IT-33, pp. 483-501, July 1987. S. A. Tretter, “An eight-dimensional 64-state trellis code for transmitting 4 bits per 2-D symbol,” IEEE J. Select. Areas Commun., vol. 7 , pp. 1392-1395, Dec. 1989. S. S. Pietrobon and D. J. Costello Jr., “Trellis coding with multidi- mensional QAM signal sets,” IEEE Trans. IMorm. Theory, vol. 39, pp. 325-336, Mar. 1993. S. S. Pietrobon, R. H. Deng, A. Lafanechere, G. Ungerboeck, and D. J. Costello Jr., “Trellis-coded multidimensional phase modulation,” IEEE Trans. Inform. Theory, vol. 36, pp. 63-89, Jan. 1990. G. D. Forney Jr. and L. F. Wei, “Multidimensional constellations-Part I: Introduction, figures of merit, and generalized cross constellation,” IEEE J. Sel. Areas Commun., vol. 7, pp. 877-892, Aug. 1989. G. D. Fomey Jr., “Multidimensional constellations-Part E. Voronoi constellations,” IEEE J. Sel. Areas Commun., vol. 7, pp. 941-958, Aug. 1989. A. R. Calderbank and L. H. Ozarow, “Nonequiprobable signaling on the Gaussian channel,” IEEE Trans. Inform. Theory, vol. 36, pp. 726-740, July 1990. G. D. Fomey Jr., “Trellis shaping,” IEEE Trans. Inform. Theory, vol. 38, pp. 281-300, Mar. 1992. R. Laroia, N. Farvardin, and S. Tretter, “On optimal shaping of mul- tidimensional constellations,” IEEE Trans. Inform. Theory, vol. 40, pp. 1044-1056, July 1994. A. R. Khandani and P. Kabal, “Shaping multidimensional signal spaces-Part I: Optimum shaping, shell mapping and part 11: Shell-

addressed constellations,” IEEE Trans. Inform. Theory, vol. 39, pp. 1799-1819, Nov. 1993. F. R. Kschichang and S. Pasupathy, “Optimal shaping properties of truncated polydiscs,” IEEE Trans. Inform. Theory, vol. 40, pp. 892-903, May 1994. G. Lang and F. Longstaff, “A leech lattice modem,” IEEE J. Select. Areas Commun., vol. 7, pp. 968-973, Aug. 1989. M. V. Eyuboglu, G. D. Fomey Jr., P. Dong, and G. Long, “Advanced modulation techniques for V.fast,” European Trans. Telecom., vol. 4, pp. 243-256, 1993. F. Q. Wang, L. C. Perez, and D. J. Costello Jr., “On the perfor- mance of codedshaped modulation,” in Proc. 1994 IEEE Int. Symp. on Information Theory (Trondheim, Norway, June 1994), p. 329. AT&T, “A new 4D @-state rate 4/5 trellis code,” Contribution D19, CCI’IT Study Group 14, Geneva, Switzerland, Aug. 1993. General Datacom, “A new 4D 64-state rate 4/5 trellis code,” Con- tribution D25, CCITT Study Group 14, Geneva, Switzerland, Aug. 1993. British Telecom, “Code choice for V.fast,” Contribution D9, CCITT Study Group 14, Geneva, Switzerland, Aug. 1993. M. D. Trott and J. P. Sarvis, “A family of 64-state rate 4/5 4D codes,” Tech. Note, MIT, Aug. 1993. G. D. Fomey Jr., “Geometrically uniform codes,” IEEE Trans. Inform. Theory, vol. 37, pp. 124-1260, Sept. 1991. -, private communication, Sept. 1993. -, “Coset codes I Introduction and geometrical classification,” IEEE Tram. Inform. Theory, vol. 34, pp. 1123-1151, Sept. 1988. ITU-T, “V.3LA modem operating at data signalling rates of up to 28,800 bit/s for use on the general switched telephone network and on leased point-to-point 2-wire telephone-type circuits,” Sept. 1994. F. Q. Wang and D. J. Costello Jr., “Erasurefree sequential decoding of trellis codes,” IEEE Trans. Inform. Theory, vol. 40, pp. 1803-1817, Nov. 1994. R. M. Fano, “A heuristic discussion of probabilistic decoding,” IEEE Trans. Inform. Theory, vol. IT-9, pp. 64-74, Apr. 1963. F. Q. Wang and D. J. Costello Jr., “Probabilistic construction of large constraint length trellis codes for sequential decoding,” IEEE Trans. Commun., vol. 43, pp. 2439-2448, Sept. 1995.

New Binary Covering Codes Obtained by Simulated Annealing

L. T. Wille, Member, IEEE

Abstrac t4ew binary covering codes of radius 1, obtained by simulated annealing, are presented. These constructions establish that K(9,l) 5 62 and K(12,l) 5 380.

Index Tem--Binary codes, coverings, covering radins, simulated an- nealing.

I. INTRODUCTION This correspondence is concerned with finding upper bounds on

K(n , R), the mnimum cardinality of any binary code of length n and with covenng’radlus R. The general problem of determining bounds on this quantity (and the related problem for q-ary codes) has been the subject of many recent papers (see [I] and references therein). A comprehensive table with upper and lower bounds on K(n, R) may be found in [2], although this did not contain all the published bounds

Manuscnpt received September 1, 1994; revised June 26, 1995’. The author is with the Department of Physics, Flonda Atlantx University,

Pubhsher Item Idenhfier S 0018-9448(96)00021-1 Boca Raton, FL 33431 USA.

0018-9448/96$05.00 0 1996 IEEE

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 1, JANUARY 1996 301

[3]. More recently, Ostergkd and Hiimaliiinen circulated a technical report [4] with a number of subsequent improvements.

The purpose of the present correspondence is to list the codewords establishing that K ( 9 , l ) 5 62 (a result published in [3] but omitted from later overviews of upper bounds) and also to present a newly discovered code demonstrating that K( 1 1 , l ) 5 380, which improves on the previous published upper bound of 382 (found by an unknown author in a Swedish football pool magazine and published in [5]). The same bound (li( 1 1 , l ) 5 380) has also been discovered by Exoo (personal communication, quoted in [4]). Attention is also drawn to the fact that the upper bound l i ( 1 0 , l ) 5 120, usually credited to Ostergird [6], was in fact previously published by the present author in [3]. All of these upper bounds were obtained by the simulated annealing method [7].

11. ALGORITHM The simulated annealing technique, introduced by Kirkpatrick et al.

[7] to solve complicated (often, NP-complete) optimization problems, has been successfully used in code design and related problems [3], [6], [8]-[12]. To implement this method one needs to formulate the problem at hand as the optimization of a certain cost function whose global minimum (or one of its global minima) one wants to find. A “greedy” algorithm would start from an initial guess at a solution and proceed downhill on the cost function hypersurface, by making small changes to the current configuration, until no further improvements are possible. The main drawback of this approach is that it tends to get stuck in local minima. The central idea of the annealing method is to avoid trapping in local minima by accepting “uphill” moves with a temperature-dependent probability given by a Boltzmann factor exp (-LICIT), where A C is the change in cost. Simulations start at an initial temperature T, selected such that the system can easily move from one local minimum to another. After a number of iterations, N I T , the temperature is decremented, either logarithmically, according to T + xT, or linearly, according to T + T - AT, until the system is frozen into a final state. Details of the method may be found elsewhere [13], [14].

Because of the need to use a finite value for NIT there is no guarantee that simulated annealing will find the global minimum on every attempt. The results presented here were typically obtained as the best out of many runs. It was also observed that there was a point of diminishing returns as far as increasing NIT was concerned, i.e., rather than spending the computer resources on a single run with a large NIT, it is advantageous to perform several runs (with different random number seeds) with a smaller NIT. Appropriate values need to be established by trial and error. Because these calculations tend to be very time-consuming care must be taken that the computer programs are efficient. For example, the repeated calculation of the Boltzmann factor can be avoided by the use of lookup tables.

The algorithm used in the present work is similar to that in our earlier publications [3], [9] and starts from a partial covering of the space Fz with the number of uncovered codewords taken as the cost function. This implementation is not the only one possible. While it is the most direct execution of the ideas of Kirkpatrick et al. [7], it becomes unwieldy for n values that are too large. Thus a number of alternative approaches based on a partitioning and reduction in size of the problem have been proposed. Van Laarhoven et al. [lo] used a theorem due to BIokhuis and Lam [15] to reduce the size of the optimization problem, an approach subsequently also employed by Ostergird [12]. Koschnick [16] used the same theorem in conjunction with a randomized algorithm, very similar to simulated annealing. In other work, Ostergkd [6] employed an ingenious line of attack starting from a mixed covering of IF: IF; (found by simulated

TABLE I BOUNDS ON l i (n , R) FOR R = 1

(New results obtained by simulated annealing are in boldface.)

n

9

bounds

54’ - 62

10 10Sb - 12W

1 1 177d - 192e

12 342f - 380

a = 1171

b=[19]

c = frst published in 131; also discovered by Osterghd 161 (see also [SI).

d = [l]

e = 1171, earlier found by Virtakallio (see 151).

f = [18]

TABLE I1 CODEWORDS FOR CODE SHOWING I i ( 9 , l ) 5 62

~~ - ~

000000101 000001010 000010011 000011100

000100001 000100100 000101010 000111111

001001010 001010110 001011001 001100111

001101010 001101101 001110000 010000110

010001001 010010000 010011111 010101111

010110010 010110101 010111000 011000011

011001100 011010101 011100000 011110101

011111011 011111110 100000000 100001111

100010110 100011001 100100101 100101010

100110110 100111001 101000000 101000001

101000100 101001111 101011111 101101010

101110011 101111100 110000011 110001100

110010101 110011010 110100011 110101100

110110000 110111111 111001111 111010010

111010101 111011000 111011010 111100110

111101001 111110101

annealing) followed by a mapping of the elements of IF4 into binary codewords.

111. NEW COVERING CODES Table I lists the current best values for the upper and lower bounds

on K ( n , 1) with the results presented here in boldface. The lower bound on K ( 9 , l ) was established by Cohen et al.

[17], while the previous best upper bound K ( 9 , l ) 5 64 follows immediately by a direct sum construction from the exact result K ( 8 , l ) = 32 obtained by van Wee [18] using improved sphere bounds. Employing the simulated annealing algorithm with T, = 2.0, logarithmic cooling with x = 0.995, and NIT = 1000 iterations per temperature step, near T E 0.44, a covering of IF; with 62 codewords and their neighbors within a sphere of radius 1 was found, a result first announced in [3]. The corresponding codewords are listed in Table 11.

The lower bound on l i ( l 0 , I ) was found by Zhang [19]. The upper bound of 120 was first established by the present author in

302 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 42, NO. 1, JANUARY 1996

TABLE I11 CODEWORDS (IN HEXADECIMAL) FOR CODE SHOWING K(12,1) 5 380

014 01F 026 02C 032 043 044 049 04A 050

054 061 062 068 07D 080 082 08D 098 OA7

OB1 OBE OCO OC4 OD0 ODE OE7 O F E 106 108

113 125 139 141 142 148 14F 160 173 17E

188 195 1AO 186 1BA 1BE 107 1D9 1EA 1FW

1F4 201 205 20E 211 237 238 242 248 25F

260 266 276 296 2 9 8 2A4 ZAA 2BD 2 W 203

2EE 2F5 2F8 31A 31D 322 32F 334 340 355

35A 361 36C 379 387 38C 390 3A9 363 3C6

3CB 3Dc 3E1 3F2 3FF 401 405 411 4l.A 426

43D 442 448 458 460 46E 477 48E 497 448

464 4BB 4CF 4D5 4E9 4F2 4FC 50F 51C 52A

530 531 540 556 55D 565 568 578 584 592

599 5A3 SAD 5C3 5CC 5DA 5% 5F1 5 W 600

604 605 606 610 611 615 623 629 6% 647

65C 660 611 61A 683 689 69C 6A1 6AF 6B2

6C1 6CA 6D6 6D9 6E3 6E4 6FF 703 709 716

721 72C 738 749 74E 753 762 774 781 78A

79F 786 765 768 7C5 700 7E8 7F7 7PB 7FE

807 8OA 819 820 835 840 844 850 857 86F

87A 884 893 8119 866 8BA 8BE 8CO 8CB 8DD

8EC 8F1 8F6 90D 910 91E 923 92E 93C 93F

956 95C 966 969 975 981 98E 99C 99F 9AC

9AF 902 9% 9BA 9BD 9C5 9CE 9DZ 9B3 9F8

9FE A01 A12 A1C A26 A2D A3B A4B A59 A65

A73 A7C A88 A8F A95 AA3 AB0 AC7 AD4 ADA

AE2 AE9 AFF BO4 BOB 817 828 031 B3E 843

B4D 656 B6A 670 682 899 B9E BA5 BAE BE€

BC8 BD1 BE4 BF7 BFB CO1 COC C16 C2F C33

C38 C4D C5E C63 C74 C79 C8B C90 C9D W

CA5 CC6 CD3 CD8 CE5 CEA CPF DO2 D15 DlB

924 929 935 D47 D4A D51 D6C D72 D87 D88

D9E DAE DB1 DBC Dc9 DD4 DE0 DF7 DPB EO0

E01 E05 Ell E1F E2A E34 E4B E52 E55 E66

E68 E86 E9A EAC EB7 EB9 E 1 ECC E F O EF7

EFB EFE FOE P18 F27 F32 F3D F44 F58 F5F

F61 F6F F7D FIE 680 F93 F94 FAO FAB W F

FC2 FCF FDD FDF FED FEF FF3 FF6 FFA

[3] using a direct application of the annealing algorithm (T, = 2.0, x = 0.995, NIT = 1000). Lock-in into this minimum occurred near T z 0.36. Ostergird [6] later published the same upper bound, which he found by applying the theorem discussed above to a mixed covering obtained by simulated annealing. It is interesting to note that Osterg5rd states that he was unable to find a covering by a direct annealing approach. Presumably this is because the cooling used by this author proceeded too rapidly.

For K ( 1 1 , l ) the lower bound was obtamed by van Wee [l], while the upper bound was determined by Cohen et al. 1171, although an earlier reference exists to a Finnish football pool magazine (see [5]). This upper bound was reproduced, but no improvements on it were fopnd using the annealing algorithm with control parameters similar to those employed for the other simulations mentioned here.

For K(12; l ) the lower bound is from van Wee [18], while the previous published best upper bound was 382 (established in a Swedish football pool magazine as quoted in [SI), which is a slight improvement over 384 obtained by a direct sum construction on the upper bound for II( 11,l). Using simulated annealing with linear cooling and T, = 0.60, AT = 0.01, and NIT = 60000 iterations, at a temperature of T z 0.45, a code with 380 words was found which

leads to a covering of the space Fi2. The corresponding codewords are listed in Table 111. The same bound was established by Exoo (unpublished, quoted in [4]) using simulated annealing.

ACKNOWLEDGMENT

The author wishes to thank I. Honkala and P. Ostergird for useful correspondence.

REFERENCES

[l] G. J. M. van Wee, “Some new lower bounds for binarv and ternary covering codes,” IEEE Trans. Inform. Theory, vol. 39, pi. 1422-1424, July 1993. Z. Zhang and C. Lo, “Linear inequalities for covering codes: Part E-Triple covering inequalities,” IEEE Trans. Inform. Theory, vol. 38, pp. 1648-1662, Nov. 1992. L. T. Wdle, “Improved binary code coverings by simulated annealing,” Congressus Numerantium, vol. 73, pp. 53-58, Jan. 1990. P. R. J. Osterg&d and H. 0. HBinalXinen, “New upper bounds for bi- narykmary mixed covering codes,” Helsinki University of Technology, Ser. A: Res. Rep. 22, Mar. 1993. H. HMi2inen and S. Rankinen, “Upper bounds for football pool problems and mixed covering codes,” J. Comb. Theory A, vol. 56, pp.

P. R. J. Ostergkd, “A new binary code of length 10 and covering radius 1,” IEEE Trans. Inform. Theory, vol. 37, pp. 179-180, Jan. 1991. S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, ‘‘Optimization by simulated annealing,” Science, vol. 220, pp. 671-680, 1983. A. A. El Gamal, L. A. Hemachandra, I. Shperling, and V. K. Wei, “Using simulated annealing to design good codes,” IEEE Trans. Inform. Theory, vol. IT-33, pp. 116-123, Jan. 1987. L. T. Wille, “The football pool problem for 6 matches: A new upper bound obtained by simulated annealing,” J. Comb. Theory A, vol. 45,

P. J. M. van Laarhoven, E. H. L. Aarts, J. H. van Lint, and L. T. Wille, “New upper bounds for the football pool problem for 6, 7, and 8 matches,” J. Comb. Theory A, vol. 52, pp. 304-312, 1989. P. R. J. Ostergtkd, “Further results on (L, t)-subnormal covering codes,” IEEE Trans. Inform. Theory, vol. 38, pp. 206-210, Jan. 1992. -, “New upper bounds for the football pool problem for 11 and 12 matches,” J. Comb. Theory A, vol. 67, pp, 161-168, 1994. P. J. M. van Laarhoven and E. H. L. Aarts, Simulated Annealing: Theory and Applications. Dordrecht, The Netherlands: Kluwer, 1988. C. R. Reeves, Ed., Modem Heuristic Techiques for Combinatorial Problems. Oxford, U K Backwell Sci. Pub., 1993. .A. Blokhuis and C. W. H. Lam, “More coverings by rook domains,” J. Comb. Theory A , vol. 36, pp. 240-244, 1984. K. U. Koschnick, “A new upper bound for the football pool problem for nine matches,” J. Comb. Theory A, vol. 62, pp. 162-167, 1993. G. D. Cohen, A. C. Lobstein, and N. J. A. Sloane, “Further results on the covering radius of codes,” IEEE Trans. Inform. Theory, vol. IT-32, pp. 68M94, Sept. 1986. G. J. M. van Wee, “Improved sphere bounds on the covering radius of codes,” IEEE Trans. Inform. Theory, vol. 34, pp. 237-245, Mar. 1988. 2. Zhang, “Linear inequalities for covering codes: Part I-Pair covering inequalities,” IEEE Trans. Inform. Theory, vol. 37, pp. 573-582, May 1991.

84-95, E991.

pp. 171-177, 1987.