INFOR~/[ATION AND CONTROL 21, 117--133 (1972)

Forward-Error Correction with Decision Feedback*

GEORGE I . DAVlDA

Department of Electrical Engineering, University of Wisconsin-Milwaukee, Milwaukee, WI 53201

AND

SUDI-IAKAR M . REDDY

Department of Electrical Engineering, University of Iowa, Iowa City, Iowa 52240

A new coding s cheme is presented. In this s cheme a given t -e r ror -de tec t ing code is u sed for t r ansmiss ion over a noisy channel . W h e n t or less errors are detected, a single r eques t is m a d e via a feedback channe l for t r ansmis s ion of addi t ional r e d u n d a n c y to enable the decoder to correct t or less errors. I t is s h o w n to be super ior to forward error correction in rate and probabi l i ty o f error. For low channe l errors the resu l t ing rate is h igher t h a n tha t of re t rans- miss ion wi th no significant difference in the probabi l i ty of error.

I . INTRODUCTION

Two basic schemes which can be used for error control are forward-error correction and retransmission (Peterson, 1961). In forward-error correction, sufficient redundancy is sent with the information to allow the decoder to correct certain error patterns. In retransmission, an error detecting code is used to request, via a feedback channel, retransmission of an erroneous block (Benice and Frey, 1964; Park, 1969). I t has been shown that for low- error probabilities, the rate for retransmission is higher than that of forward- error correction (Benice and Frey, 1964). We consider a combined strategy.

* T h e research repor ted in this paper was suppor t ed in par t by the Nat ional Science F o u n d a t i o n gran ts G K - 2 9 9 0 and GK-10025 . T h i s paper is par t of a disser ta t ion s u b m i t t e d to the D e p a r t m e n t of Electrical Engineer ing , Un ive r s i ty of Iowa, Iowa City, in part ial fu l f i l lment of the r equ i r emen t s of the Ph .D . degree.

117 Copyright © 1972 by Academic Press, Inc. All rights of reproduction in any form reserved.

118 DAVIDA AND REDDY

II . NEW CODING SCHEME

Let (n, k) be a code 9f block length n, k information symbols per block and r = n - k redundant symbols per block, which can correct t or less errors. Suppose that it is possible to puncture this code, that is, remove some of the redundant symbols, and obtain an (n, , k) code, with r 8 ~ ns - - k redundant symbols, which can only detect t or less errors, as shown in Fig. 1, where r ' = r - - r s is the number of redundant symbols which are removed. These r ' symbols are initially stored at the transmitter. The ( n s , k ) code is then used for transmission over a noisy channel, and when t or less errors are detected, a single request is made, via a noiseless feedback channel, for transmission of the remaining redundant symbols.

I_.,

E i ~ .

1l. s -1d

r I

FIG. 1. Puncturing a code.

L J

FIG. 2. Typical .received sequence.

The feedback channel need not be delayless as in some feedback com- munication schemes (Berlekamp, 1968). A typical received sequence is shown in Fig. 2, where (ki, r~i ) is the i-th block received and rl ' is the set of remaining redundant symbols, had (kl, rsl) block been in error, and z is the delay in the two channels. This delay will determine the number of storage elements at the decoder,

THEOREM 1.

ERROR CORRECTION WITH FEEDBACK 119

The expected value of the rate R for the proposed scheme it

~(R) = k/(n~ + P # ) , (1)

where PD is the probability of detecting an error.

Pro@ Let

M = the number of words to be transmitted,

n = the length of the full code,

then

E(number of blocks of length n) = M P 9 ,

E(number of blocks of length ns) = M(1 - - P9),

E(total number of symbols transmitted) = MnP9 -t- Mns(1 -- PD), number of information symbols = Mk.

Hence

E(R) = Mk/[MnPD + Mn~(1 -- PD)] = k/[nPo + n~(1 - - PD)].

Substituting ns -]- r' for n, we get

E(R) = a/(n~ + P y ) . Q.E.D.

Consider the Binary Symmetric Channel. Assume that bounded distance decoding is used, that is, only e or less errors are corrected. We then have the following theorem:

THEOREM 2. The probability of correctly decoding a block of k bits for the proposed scheme is higher than that of forward-error correction by exacly APe, where

i=e+l

Proof. Let

P~I = probability of correctly decoding k bits for the proposed scheme,

P~2 = probability of correctly decoding k bits for forward error correction.

120 DAVIDA AND REDDY

Then

i=0

• _ e - - i y t . .

+ °~"~-~ ~ ~ v ~-~ [,~' + ~ ' ~ < . . .+ (~ ~' ~)~ v'-~+q + .- 2

i=O

A p e = q,~ + n s p q ~ _ , q / + n s p q , _ l ( r , p q ~ , _ l ) + n~(n~ - - 1)p,q,_~(q~,) + .. . 2

- - [q° + "Pq°-" ~ n(n 2 - - 1) p~q._~ + ...]

= qn, _ q , @ n~pqn~+r'-I __ npqn-1

+ [ , / + n~Jn~ .-- k)] p~¢~+~,_,, n(n -- 1) p ,¢_ , + ... 2 2 "

Combining terms, we get

A P , = q"" - - q" - - r 'pq "-a r ' ( r ' - - 1) p2q, -~ + .. . 2

APe = qn"[1 - - q~' - - r 'pq ~'-1 + ""]

i = O

, f "

/ = e + l

Q.E.D.

ERROR CORRECTION W I T H FEEDBACK 121

The above result can be seen by observing that whenever the n~ bits are received error-free, certain uncorrectablc error patterns are avoided, as shown in Fig. 3.

O0 . . . . O 0 I I 0 I 0 - - - I I

/

11 r 5

FIG. 3. Avoided error patterns.

EXAMPLE 1. Let us look at the shortened Hamming (6, 3) single-error- correcting binary code, whose parity check matrix H is

H = 1 1 0 1 . 0 1 0 0

F rom the top row we note that the first check bit forms a check on all the information positions. Thus, if we were to send the 3 information symbols and the first check bit, the decoder can detect all single errors. When a single error occurs the remaining two parity bits can be requested and, if received error-free, the single error can be corrected. The expected value of the rate for this code is

E(R) = 3/(4 + 2PD), where

i.e.,

Thus

Also

P , = P[1 error] q- P[3 errors],

PD = 4pq 3 + 4P~q.

E(R) = 3/4(1 + 2pq ~ + 2p3q).

APe ~ ( 2~ i 6 i : i=2 i ) p q -- = p2qa.

I I I . PUNCTURING

All the Hamming single-error-correcting codes can first be shortened and then punctured to yield a single-error-detecting code. The parity check

1 2 2 DAVIDA AND REDDY

matrix for binary codes, with n = 2 m -- 1, consists of all possible nonzero

H =

m-tuples.

"i 1 "" 1 1 0 0 ... 1 0 1 ... ! 0 1 0 . . . .

. , .

, . .

. . .

0 1 "" 1 0 0 0 ... 1 l< m >

.

0

m .

For convenience we assume that the last m positions are the redundant positions. I f we drop all the columns in the information positions, which begin with zero, the first check bit will form a parity check on the remaining information positions and we thus obtain a single-error-detecting code with one parity bit whose parity check matrix H ' is

H ' = [ 1 1 1 1 ... 1 ;

,C ~ 8 - - >

where ns = 2 m-1. Initially, only one parity check is sent, and when a parity failure is detected, the remaining (m - - 1 ) checks are sent and then correction is carried out.

Other codes that can be punctured include the maximal shift register codes (Solomon and Stiffler, 1965), uniform convolutionai codes (Robinson, 1969), Reed-Solomon codes (Reed and Solomon, 1960), and a class of extended BCH codes (Davida, 1970). The Golay (23, 12) code is next shown to be puncturable. This, however, does not appear to be a general result. Consider t hepa r i t ycheekma t r i xH:

" 1 1 0 1 0 0 0 1 1 1 1 1 0 1 1 1 0 1 1 1

H = 1 1 1 1 0 1 1 0 0 1 0 0 1 1 0 1 0 1 1

1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 " 0 1 0 0 0 1 1 1 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0 0 0 1 1 1 0 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 1 0 1 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 1

23

ERROR CORRECTION WITH FEEDBACK 123

Dropping column 7, we H ' is

get the shortened (22, 11) whose parity check matrix

- 1 1 0 1 0 0 ! 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 0 0 1 1 1 0 0 1 0 0 0 0 0 0 0 0 ~ 1 0 1 1 0 1 0 0 1 1 ' 1 0 0 1 0 0 0 0 0 0 0 1 1 0 1 1 0 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 1 1 1 0 1 1 1 0 0 0 1 0 0 0 0 1 0 0 0 0 0

H ' = 1 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 1 0 0 0 0 . 1 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 0 0 1 1 1 1 1 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 0 1 1 0 1 0 0 0 0 0 0 0 0 1 0 1 1 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0 1 1 0 1 0 0 0 1 1 0 1 1 0 0 0 0 0 0 0 0 0 0

Rows 1, 2, 3, 5 and 8 Of H' ~ o m the extended Hamming (16, 11) binary code which can detect 3 or less errors. The parity check matrix H ~ %r the (16, 11) code is [i0000000i] 1 1 0 1 0 0 1 1 1 0 0 1 0 0

H " = 0 1 1 0 1 0 0 1 1 1 0 0 1 0 . 1 1 0 1 1 1 0 0 0 1 0 0 0 1 0 0 1 1 1 1 1 0 1 0 0 0 0 0

Thus, we can initially store 6 parity che& bits and transmit with a rate of 11/16 instead of 11/22. When 3 or less errors are d a e a e d we can request the remaining symbols and proceed to correction.

IV. ALTERNATE SCHEMES

T h e above scheme naturally leads one to consider the following variations, of which B is much easier to implement since it avoids the necessity of puncturing codes, which may not be readily accomplished:

A. Let an (n, h) code which can correct t or fewer errors be punctured to an (n~, h) code which can correct e errors and detect t errors (e < t). The expected value of the rate for this scheme will be higher than the above mentioned one in cases where e or less errors are quite likely resulting in a high request rate for additional information. The remaining redundancy will be requested only when the number of errors exceeds e.

643/eI/2-3

124

Then

B.

DAVIDA AND REDDY

E(R) = k/(ns q- r'PD). (3)

Let (n 1 , k) be a code which can correct e errors and detect t errors, and (n2, k) be a different code which can correct t errors. Initially, the (nl , k) code is used. When more than e errors occur, the redundant symbols, computed according to the (ne, k) code, are requested and the redundant symbols at the (n I , k) code are discarded. For this case,

E(R) = k/In 1 + eD(n2 -- k)]. (4)

EXAMPLE 2. Consider the (5, 4) single-error-detecting binary code with one parity bit whose parity check matrix H~ is

H 1 - [1 1 1 1 1],

and the Hamming (7, 4) single-error-correcting binary code whose parity check matrix H e is

H ~ = 0 1 1 0 ~ .

1 0 1 0 0

Initially, the (5, 4) code is used, and when a single error is detected, a request is made to send the parity symbols for the (7, 4) code and the parity bit f rom the (5, 4) code is discarded. Then

E(R) = 4/(5 q- 3PD),

P g = (~) Pq4 + (53) P3q2 + (55) p5

: 5pq~ + lOpZq~ + pS;

therefore

E(R) = 4/(5 + 15pq 4 + 30pSq z + 3p5).

Using two codes, one for detection and the other for correction is more readily accomplished since the two codes are not necessarily related. This leaves one with greater flexibility in choosing the two codes.

V. COMPARISON WITH RETRANSMISSION

Let us consider the simple retransmission scheme where an (n, k) code is used for error detection over a noisy channel. When an error is detected,

ERROR CORRECTION W I T H FEEDBACK 125

a request is made via a noiseless feedback channel for the repetition of that erroneous block. We will consider here the scheme where the number of repetitions is unlimited. Consider the Binary Symmetr ic Channel.

THEOREM 3. The expected value of the rate for the simple retransmission scheme is

E(R) ~- (1 - - eD)h/(n~ + 1),

where PI) is the probability of detecting an error.

Proof. Let

M ~ the number of words transmitted;

then

E(number of blocks repeated once) = MPD,

E(number of blocks repeated twice) ~ MPD 2,

E(number of blocks repeated N times) = MPD N,

E(total number of bits transmitted) = M(ns @ 1) @ M(n~ q- 1) PD

+ M(n~ + 1) PD ~ "- 1

= M(n~ + l) I - - P D

1 = M(n~ q- 1) 1 - - PD

M • E ( R ) = M ( n ~ + 1 ) [ 1 / ( 1 - - P D ) ] = k (1 - - P~)/(n~ + 1).

THEOREM 4.

P~oof.

Q.E.D.

The probability of error for the simple retransmission scheme is

P~ = (1 - - PD -- q~)[1/(1 - - PD)]"

P, ~ P(undetected error) + PD[P(undetected error)]

q- (PD)n[p(undetected error)] q- ---,

V~ = (I - - PD - - q~)(1 + PD q- PD 2 -k "") 1

= - P o - ¢ o ( 1 - P o ) Q.E.D.

126 DaVIOa AND REDDY

The expected values of the rate for the shortened and punctured Hamming codes for both the proposed scheme and retransmission are shown in Figs. 4-6 for values of n s of 4, 128, and 512. The word error probabilities are shown at selected points. The corresponding word error probabilities are shown in Figs. 7-9. In the case of the proposed scheme, when an error is detected, the remaining r ' parity bits are requested (only once), whereas in retrans- mission the whole code-word is repeated. There is no significant difference in the probability of error between the two schemes. Significant difference does exist in the rate, where the proposed scheme.is superior.

When two codes are used, that is, one for detection and the other for correction, retransmission has a lower probability or error, as shown in Figs. 10-12. But this is achieved at a significant reduction in the rate, as in Figs. 13-15, where again the word error probabilities are shown at selected points.

1,0

,80

ILl .60

.40

.20

O

i 0 ~ 0

X X

- : ( 4 ~ 3 ) : r ' = 2

~'-.0

~D

- - - - - - P R O P O S E D e J C H E M E x o)

p- RETR/~ .NSMISSION

, 1 , , I , , I . , i . . I . , I , , I . . i - 5 - 4 - 3 - 2 - | ~o-7 : 6 s ~o ,o io io Io I P

FIG. 4. Comparison of the proposed scheme (single-error correction) with retransmlssion (single-error detection). The word-error probabilities are shown at selected points.

ERROR CORRECTION WITH FEEDBACK 127

W I-- <~ n.-

1,0

. 8 0

. 6 0

. 4 0

.~0

U2 8112T}, r=7 ,

(128 ,127)

N IO

X

o- .~ - o - - - 6 - - . o

%

o~ ..;

- - - - - PROPOPOSED SCHEME

RETRANSMISSION

, I , . I . . I . . I . . I . . I , . I . . I

,~7 ~;6 65 ,;4 ,;3 , ~ I~1 , p

FIG. 5. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection). The word-error probabilities are shown at selected points.

VI . CONCLUSIONS

As expected, the above scheme performs better than forward-error correction. It was seen that the probability of error for the proposed scheme is slightly higher than the simple retransmission, but that it has significantly higher rate. Since the number of requests for additional redundant bits is limited to one, the overflow problem has been considerably reduced.

I t is expected that for equal rate or equal probability of error the proposed scheme will perform better than the retransmission scheme in the probability of error and the rate, respectively.

hO

. 0 0

I.l..J , 6 0 I - -

. 4 . 0

.ZO

L~IZ,6I I~I , ~ ' , l ) .-. o ..: ..:

{ 5 1 2 ~ - - ; 0

l0

- - - P R O P O S E D S C H E M E

R E T R A N S M I S S I O N

FIO. 6. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection). The word-error probabilities are shown at selected points.

CE 0 BE re" LLi

LL. 0 >.. I - -

._l

CO

CO 0 0:: 0..

DE 0

| , 0 m

- - - - - - P R O P O S E D S C H E M E

/ ~ R E T R A N S M I S S i O N

• I . • I . • I . • I • • i . I • • I , • I

FIc. 7. Comparison of the proposed scheme (single-error correction) with retransmission (single-error detection).

cr O r~ r~ L~

LL 0 >- t--- -n

<~ m O

n

c~ rY O

1.0

' T

[ ~ " i I , . I , , I , . I , , I i , I , . I , I I

td 7 ~d~ ,o -5 jo -4 ,d ~ ,d ~ jo "~ I. P

Fie . 8. Error probabil i ty for the proposed scheme (single-error correction) and retransmission (single-error detection).

" 9 O no a: , 63

Lfl

I,L 0

>- I---

,6 ~ m <~ m O r r

~- 1 6 9

l ' Y

O

1.0

.I, , I, .I ...... I, ,I, . ~ . . I . ~

j6 r io "~ j6 5 , ~ i~ 3 ~6 2 16 j I. P

FIG. 9. Error probabil i ty for the proposed scheme (single-error correction) and re t ransmission (single-error detection).

hO

n . 0

O~ W

0 >_ l~ c

. J

m

m Jd ~ 0 r~. n

r'~ Ir L~ '~

0

. . . . PROPOSED SCHEME

RETRANSMISSIO N

/ /

t I

I

l

I

t I

(I~,H) / I

(2~11) I I

i I I

!

'1. t i

I

/ I

. I

I I

I I

o

. I . ~ . , . I { . J l , . I , . i . . I , . ,

,d r ,o-~ ,d 5 t d ~ ,d J ,d ~ 10 -I ~ p

FIc. 10. Comparison of the proposed scheme (3-error correction) wi th retrans- mission (3-error detection).

1,0

r r O tic t r l0 3

W

ta. O >_ 10 ~

I-- _I

m

cn

0 n~ [I.

n" 0

-i$ I0

- - - - - P R O P O S E D SCHEME / /

' RETRANSMISSION / /

I , | I I I / /

(128j12o) I /

• • . • | . • • • • . i i |

L6r i~ ~5 16+ ,63 ~62 i~i I.

FIo. 11. Comparison of the proposed scheme (3-error correction) with retrans- mission (3-error detection).

1.0

r r 0 -3

I0 rr" LLI

LL -CD 0 Io >,.. I ' - ._1 ( 3 3 - s

IO ( 3 3 0

C3 It) (3:: 0

. 1 5 I O

. I I ,

i// (612~5 02). / /

- - - - - P R O P O S E D 5 C H E N E

RE T R A N S M ~ S S ~ O R

, I , . I . . I , . I , . I . , 1 , . I , , I

t67 166 165 I ff~ I d3 i 62 i 6~ I. p

FIC. 12. Comparison of the proposed scheme (3-error correction) with retrans- mission (3-error detection).

I.O

L I J ! ' -

o r

. 8

. 4

. 3

b-

' O

t O ( I 6 , i l ) , ( 2 6 , 1 1 ) .~. _

t67 io-e 1if5 164 ,-~ -2 -~ I0 I0 I0

I

I P

FIG. 13. Comparison of the proposed scheme (3-error correction) with retrans- mission (3-error detection). The word-error probabilities are shown at selected points,

LLI

l - -

n.*

h O

.8

.6

. 4

.2

I O

(12811g0) x - f,144~12 O) ~ ' o

.. :

i

'" RETR& NS MISS|ON ~, "m" 0

, I . , I , , I , , I . , I , . I , ~ , . . - J - - " ~ I

Id 7 16 e ~6 s 16 4 ~6 ~ 16 2 Io "~ i. P

FIG. 14. Comparison of the proposed scheme (3-error correction) with retrans- mission (3-error detection). The word-error probabilities are shown at selected points.

N I O (512~502) X

1.0 _ 153Z~502) N

,,,~.=:< .,~'6 ix ,; . , . , l . , , . . , . , , . . ~ ; : ~ ~,

i~7 I(~6 ib5 i~4 16~ |~Z i~I I p FIG. 15. Comparison of the proposed scheme (3-error correction) with retrans-

mission (3-error detection). The word-error probabilities are shown at selected points.

ERROR CORRECTION WITH FEEDBACK 133

ACKNOWLEDGMENT

The authors would like to thank Professor John P. Robinson for the numerous discussions and helpful suggestions. They would also like to thank the reviewers for correcting errors in an earlier version of the manuscript.

RECEIVED: M a y 6, 1971

REFERENCES

1. BENICE, R. J. AND FREY, A. H. (1964), An analysis of retransmission systems, IEEE Trans. Commun. Technology CTI2, 135-145.

2. BERLEKAMP, E. R. (1968), Block coding for the binary symmetric channel with noiseless delayless feedback, in "Proceeding of the Symposium on Error Correction Codes at the University of Wisconsin," Wiley, New York.

3. BEt~LEI~AMP, E. R. (1968), "Algebraic Coding Theory," McGraw-Hill, New York. 4. DAVmA, G. I. (1970), Multiple extension of the minimum distance of BCH codes,

in Proceedings of the Eighth Annual Allerton Conference on Circuit and System Theory.

5. FANO, R. M. (1963), "Transmission of Information," M1T Press, Cambridge, MA. 6. GraVEN, P. E. (1961), Feedback communication systems, in "Lectures on Com-

munication Systems Theory" (E. J. Bagdady, Ed.), McGraw-Hill, New York. 7. MASSEY, J. L. (1963), "Threshold Decoding," M I T Press, Cambridge, MA. 8. PARK, J. H. (I969), Information rates and errors using decision feedback, IEEE

Trans. Commun. Technology CT17, 20-24. 9. PETERSON, W. W. (1961), "Error Correcting Codes," M I T Press, Cambridge,

MA. 10. RBED, I. S. AND SOLOMON, G. (1960), Polynomial codes over certain finite fields,

J. Soc. Ind. Appl. Math. 8, 300-304. 1l. ROBINSON, J. P. (1969), Punctured uniform codes, IEEE Trans. Information

Theory IT15, 149-152. 12. SCHALKWlJK , J. P. M. AND KAILATH, T. (1966), A coding scheme for additive

noise channels with feedback: 1. No bandwidth constant, IEEE Trans. Information Theory IT12, 172-182.

13. SCHALKWlJK, J. P. M. (1966), A coding scheme for additive noise channels with feedback: II. Bandlimited signals, IEEE Trans. Information Theory IT12, 183-189.

14. SOLO~ON, G. AND STIFF~R, J. J. (1964), Algebraically punctured cyclic codes, Information and Control 8, 170-179.