Single-Sample Robust Joint Source–Channel Coding: Achieving Asymptotically Optimum Scaling of SDR Versus SNR

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012 1565

Single-Sample Robust Joint Source–Channel Coding:Achieving Asymptotically Optimum

Scaling of SDR Versus SNRMahmoud Taherzadeh and Amir Keyvan Khandani

Abstract—In this paper, we consider the problem of zero-delay(encoding a single-source sample) robust joint source–channelcoding over an additive white Gaussian noise channel. We proposea new scheme that, unlike previously known coding schemes,achieves the optimal scaling of the source signal-to-distortionratio (SDR) versus channel signal-to-noise ratio (SNR). Also,we propose a family of robust codes, which together maintain abounded gap with the optimum SDR curve (in terms of decibel).To show the importance of this result, we derive some theoreticalbounds on the asymptotic performance of a widely used class ofdelay-limited hybrid digital–analog (HDA) coding schemes basedon superposition of analog and digital components. We show that,unlike the delay-unlimited case, for this class of delay-limited HDAcodes, the asymptotic performance loss is unbounded (in termsof decibels). Although the main focus of this paper is on uniformsources, it is also shown that the results are also valid for a moregeneral class of well-behaved distributions.

Index Terms—Analog coding, hybrid digital–analog (HDA)coding, robust joint source–channel coding, scaling of signal-to-distortion ratio (SDR) versus signal-to-noise ratio (SNR), succes-sive refinement, zero-delay coding.

I. INTRODUCTION

I N MANY applications, delay-limited transmission ofanalog sources over an additive white Gaussian noise

(AWGN) channel is needed. Also, in many cases, the exactsignal-to-noise ratio (SNR) is not known at the transmitter andmay vary over a wide range of values. Two examples of thisscenario are transmitting an analog source over a quasistaticfading channel and/or multicasting it to different users (withdifferent channel gains).

Manuscript received May 26, 2008; revised October 03, 2010; accepted Feb-ruary 02, 2011. Date of current version February 29, 2012. This work was sup-ported by Nortel Networks and by the Natural Sciences and Engineering Re-search Council of Canada (NSERC), and by Ontario Centers of Excellence(OCE). The material in this paper was presented in part at the 2007 IEEE In-ternational Symposium on Information Theory and the 41st Annual Conferenceon Information Sciences and Systems, Baltimore, MD, 2007.

M. Taherzadeh was with the Department of Electrical and Computer Engi-neering, University of Waterloo, ON N2L 3G1, Canada. He is now with CienaCorporation, Ottawa, ON K2H 8E9, Canada (e-mail: [email protected]).

A. K. Khandani is with the Department of Electrical and Computer Engi-neering, University of Waterloo, Waterloo, ON N2L 3G1, Canada (e-mail: [email protected]).

Communicated by A. Grant, Associate Editor for Communications.Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TIT.2011.2177579

Without considering the delay limitations, digital codescan theoretically achieve the optimal performance in theGaussian channel. Indeed, for the ergodic point-to-point chan-nels, Shannon’s source–channel coding separation theorem[1], [2] ensures the optimality of separately designing sourceand channel codes. However, for the case of limited delay,several articles [3]–[7] have shown that joint source–channelcodes have a better performance as compared to the separatelydesigned source and channel codes (which are called tandemcodes). Also, digital coding is very sensitive to the mismatch inthe estimation of the channel SNR.

For any fixed digital code, the end-to-end distortion of thedigital joint source–channel code is upper bounded by thequantization noise of the corresponding quantizer and the SDRbecomes saturated for a high SNR. To avoid the saturation ef-fect of digital coding, various analog and hybrid digital–analog(HDA) schemes are introduced and investigated in the past[8]–[22]. Among them, examples of 1-D to 2-D analog mapscan be found as early as the works of Shannon [8] and Kotel-nikov [9] and different variations of Shannon–Kotelnikov maps(which are also called twisted modulations) are studied in [10],[11], and [19]. Also, in [14] and [15], analog codes based ondynamical systems are proposed. Although these codes canprovide asymptotic gains (for a high SNR) over simple repeti-tion codes, they suffer from a threshold effect. Indeed, when theSNR becomes less than a certain threshold, the performance ofthese systems degrades severely. Therefore, design parametersof these methods should be chosen according to the operatingSNR, resulting in sensitivity to SNR estimation errors. Also,although the performance of the system is not saturated forthe high SNR values (unlike digital codes), the scaling ofthe end-to-end distortion is far from the theoretical bounds.Theoretical bounds on the robustness of joint source–channelcoding schemes (for the delay-unlimited case) are presented in[23]–[25].

To achieve better signal-to-distortion (SDR) scaling, a codingscheme is introduced in [26] and [27], which uses repetitionsof a binary code to map the digits of the infinite binary expan-sion of samples of the source to the digits of a transmit vector.This scheme achieves a SDR scaling better than linear coding,but cannot achieve the optimum SDR scaling by using a singlemapping. Subsequently, in [28], the authors extend the resultsof [26] and [27] to the Gaussian source and to noninteger valuesof bandwidth expansion.

In this paper, we address the problem of robust jointsource–channel coding, using delay-limited codes. In partic-ular, we show that the optimum slope of the SDR curve can be

0018-9448/$31.00 © 2012 IEEE

1566 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 3, MARCH 2012

achieved by a single mapping. The rest of the paper is organizedas follows.

In Section II, the system model and the basic concepts arepresented. In Section III, we present an analysis of the previousanalog coding schemes, and their limitations. In Section IV, weintroduce a class of joint source–channel codes, which have aself-similar structure, and achieve a better asymptotic perfor-mance, compared to the other minimum-delay analog and HDAcoding schemes. The asymptotic performance of these codes,in terms of the SDR scaling, is comparable with the schemepresented in [26], but with a simpler structure and a shorterdelay. We investigate the limits of the asymptotic performanceof self-similar coding schemes and their relation with the Haus-dorff dimension of the modulation signal set. In Section V, wepresent a single mapping that achieves the optimum slope of theSDR curve, which is equal to the bandwidth expansion factor.Although this mapping achieves the optimum slope of the SDRcurve, its gap with the optimum SDR curve is unbounded (interms of decibel). In Section VI, we construct a family of robustmappings, which individually achieve the optimum SDR slope,and together, maintain a bounded gap with the optimum SDRcurve. We also analyze the limits on the asymptotic performanceof a widely used class of delay-limited HDA coding schemesbased on superposition of analog and digital components.

II. SYSTEM MODEL AND THEORETICAL LIMITS

We consider a memoryless uniform source with zeromean and variance , i.e., . Also, the samples ofthe source sequence are assumed independent with identical dis-tributions (i.i.d.). Although the focus of this paper is on a sourcewith uniform distribution, as it is discussed in Appendix C, theasymptotic results are valid for all distributions, which have abounded probability density function (pdf).

The transmitted signal is sent over an AWGN channel, withthe power constraint . The problem is to map the one-di-mensional signal to the -dimensional channel space (where

is the bandwidth expansion factor) such that the effect of thenoise is minimized. This means that the data , ,are mapped to the transmitted vector . Atthe receiver side, the received signal is , where

is the AWGN with variance . We useto denote base 2, i.e., , unless specified other-

wise (base of the logarithm does not affect the validity of mostof the derivations).

As an upper bound on the performance of the system, weuse Shannon’s theorem on the separation of source and channelcoding. By combining the lower bound on the distortion of thequantized signal (using the rate-distortion formula) and the ca-pacity of parallel Gaussian channels with the noise variance

, we can bound the distortion . On onehand, we have

(1)

and on the other hand (see, e.g., [29, p. 568])

(2)

where is the differential entropy of the source. Combining(1) and (2), for any source with finite differential entropy, wehave

(3)

(4)

(5)

(6)

where is a constant number, depending on thedistribution of the source.

Let be the variance of the source distribution (e.g., in thiscase, ). The SDR can be defined as andSNR is . In this paper, we are mainly interested in the asymp-totic slope of the SDR curve versus SNR (in terms of decibel),or in other words, 1. The previous discussionshows that this slope cannot be greater than . The aim of thispaper is to achieve this optimum slope by using a single-sourcesample and by relying on a single joint source–channel map-ping.

III. CODES BASED ON DYNAMICAL

SYSTEMS AND HDA CODING

Previously, two related schemes, based on dynamical sys-tems, have been proposed for the scenario of delay-limitedanalog coding:

1) shift-map dynamical system [14];2) spherical shift-map dynamical system [15].

These are further explained in the following.

A. Shift-Map Dynamical System

In [14], an analog transmission scheme based on shift-mapdynamical systems is presented. In this method, the analog data

are mapped to the modulated vector , where

(7)

(8)

where is an integer number, . The set of modulatedsignals generated by the shift map consists ofparallel segments inside an -dimensional unit hypercube. In[15], the authors have shown that by appropriately choosing theparameters for different SNR values, one can achieve theSDR scaling (versus the channel SNR) with the slope ,for any positive number . Indeed, we can have a slightly tighterupper bound on the end-to-end distortion as follows.

Theorem 1: Consider the shift-map analog coding system,which maps the source sample to an -dimensional modulatedvector. For any noise variance , we can find parameter

such that for the shift-map scheme with the parameters, the distortion of the decoded signal is bounded as

(9)

1This definition of slope is also used in other relevant works, e.g., see [28].

TAHERZADEH AND KHANDANI: SINGLE-SAMPLE ROBUST JOINT SOURCE–CHANNEL CODING 1567

Fig. 1. Shift-map modulated signal set for � � � dimensions and � � �.

where depends only on . Note that for , which isassumed throughout this paper2 (unless specified otherwise),

is a number greater than 1.Proof: See Appendix A.

Also, we have the following lower bound on the end-to-enddistortion.

Theorem 2: For any shift-map analog coding scheme and anynoise variance , the output distortion is lower boundedas

(10)

where depends only on .Proof: See Appendix B.

B. Spherical Shift-Map Dynamical System

In [15], a spherical code based on the linear systemis introduced, where is the -dimensional modulated

signal and is a skew-symmetric matrix, i.e., . Thisscheme is very similar to the shift-map scheme. Indeed, withan appropriate change of coordinates, the previous modulatedsignal can be represented as

(11)

for some parameter .If we consider as the modulated signal generated by the

shift-map scheme with parameters in (8), then (11) canbe written in the vector form as

(12)

The relation between the spherical code and the linearshift-map code is very similar to the relation between

2Note that we are interested in high channel SNR values.

phase-shift keying (PSK) and pulse-amplitude modulation(PAM). Indeed, the spherical shift-map code and PSK modu-lation are, respectively, the linear shift-map and PAM modula-tions, which are transformed from the unit interval tothe unit circle.

For the performance of the spherical codes, the same resultas Theorem 1 is valid. Indeed, for any parameters and , thespherical code asymptotically has a saving of or 5.17 dBin the power. This asymptotic gain results from transformingthe unit-interval signal set (with length 1 and power ) to theunit-circle signal set (with length and power 1). However, thespherical code uses dimensions (compared to dimensionsfor the linear shift-map scheme).

For both these methods, for any fixed parameter , the outputSDR asymptotically has linear scaling with the channel SNR.The asymptotic gain (over the simple repetition code) is ap-proximately (because the modulated signal is stretchedapproximately times)3. Therefore, a larger scaling pa-rameter results in a higher asymptotic gain. However, by in-creasing , the distance between the parallel segments of themodulated signal set (i.e., the segments in Fig. 1) de-creases, where each one of these segments is mapped from oneof the subintervals of the source. The minimum distancebetween these segments is approximately and for the lowSNRs (when the noise variance is larger than or comparableto ), jumping from one segment of the modulated signal setto another one becomes the dominant factor in the distortionof the decoded signal, which results in a poor performance inthis SNR region. Thus, there is a tradeoff between the gainin the high-SNR region and the critical noise level, which isfatal for the system. By increasing the scaling parameter , theasymptotic gain increases, but at the same time, a higher SNRthreshold is needed to achieve that gain. In [30], the authors

3The exact asymptotic gain is equal to the scaling factor of the signalset, i.e., � � � � � � �� for the shift map and

� � � � � � � � for the spherical shift map.


have combined the dynamical-system schemes with LDPC anditerative decoding to reduce the critical SNR threshold. How-ever, overall behavior of the output distortion is the same for allthese methods. Also, in [31] and [32], a scheme is introduced forapproaching arbitrarily close to the optimum SDR, for coloredsources. However, it is not delay-limited and it only achievesthat robustness for the bandwidth expansion of 1.

The shift-map analog coding system can be seen as a vari-ation of a HDA joint source–channel code. Various types ofsuch hybrid schemes are investigated in [16], [17], [18], [24],and [33]. Indeed, for the shift-map system, we can rotate themodulated signal set such that all the parallel segments of itbecome aligned in the direction of one of the dimensions. Inthis case, by changing the support region of the modulated set(which is a rotated -dimensional cube) to the standard cube,we obtain a new similar modulation that is HDA and has almostthe same performance. In the new modulation, the informationsignal is quantized by points in an -dimensionalsubspace and the quantization error is transmitted over the re-maining dimension.

Regarding the scaling of the output distortion, the perfor-mance of the shift-map scheme, with appropriate choice of pa-rameters for each SNR, is very close to the theoretical limit. Infact, the output distortion scales as4 , insteadof being proportional to . However, for any fixed set of pa-rameters, the curve of SDR versus SNR (in decibel) is saturatedby the unit slope (instead of ). This shortcoming is an inherentdrawback of schemes like the shift-map code or the sphericalcode (which are based on dynamical systems). Indeed, in [23],it is shown that no single differentiable mapping can achieve anasymptotic slope better than 1. This paper addresses this short-coming.

There are some other analog codes in the literature, which usedifferent mappings. Analog codes based on the two-dimensionalShannon map [20], [21], [22], or the tent map [14] or the schemein [34] are the examples of these codes. However, all these codesshare the shortcomings of the shift-map code.

IV. JOINT SOURCE–CHANNEL CODES

BASED ON FRACTAL SETS

In this section, we propose a coding scheme, based on thefractal sets, that can achieve slopes greater than 1 (for the curveof SDR versus SNR).

Scheme I: For the modulating signal , , weconsider the binary expansion of

(13)

Now, we construct as

(14)

(15)

(16)

where is the base- expansion.5

4Note that �� is a number greater than 1.5In this paper, we define the base-� expansion, for any real number � � �

and any binary sequence �� , as � � � � � � � � � � � .

Note that in this coding scheme, a single map is used for allranges of the SNR, and the design parameter is not dependenton the SNR. The choice of this design parameter affects theperformance of the code and its effect on the performance ischaracterized by the following theorem.

Theorem 3: In the proposed scheme, for any and noisevariance , the output distortion is upper bounded by

(17)

where depends only on , and .Proof: Consider as the Gaussian noise on the th

dimension

(18)

(19)

Now, we bound the distortion, conditioned onfor . If the th digit of and

are different

(20)

(21)

(22)

(23)

Therefore, if for , the first digits of

and are the same, where . Now, by

considering

(24)

(25)

Therefore, for , the first

digits of can be decoded without any error, andhence, the first

bits of the binary expansion of can be reconstructed perfectly.In this case, the output distortion is bounded by

(26)

(27)


Fig. 2. Boxes of size � and their intersections with the decoding regions.

where depends only on and . By combining the upperbounds for the two cases, noting that

(28)

(29)

(30)

Note that in the previous derivations, we have used averaging oftwo cases and the fact that the squared error is always less than1 for the first case (source is bounded between and ),and the probability of the second case is less than 1.

According to Theorem 3, for any , we can construct amodulation scheme that achieves the asymptotic slope of

(for the curve of SDR versus SNR, in terms of decibel). Asexpected (according to the result by Ziv [23]), none of thesemappings are differentiable. More generally, in [23], the authorhas shown the following.

Theorem 4 ([23, Th. 2]) : For the modulation mapping, define

If there are positive numbers and , such that

(31)

then there is constant , such that

(32)

Fig. 3. Output SNR (or SDR) for the first proposed scheme (with � � � and3) and the shift-map scheme with � � �. The bandwidth expansion is � � �.

In Scheme I, by decreasing , we can increase the asymptoticslope . However, it also degrades the low-SNR performance ofthe system. This phenomenon is observed in Fig. 3.

In Scheme I, the signal set is a self-similar fractal [35], wherethe parameter , which determines the asymptotic slope ofthe curve, is the dimension of the fractal. There are differentways to define the fractal dimension. One of them is the Haus-dorff dimension. Consider as a Borel set in a metric space,and as a countable family of sets that covers it. We define

, where the infimum isover all countable covers that diameter of their sets are notlarger than . The -dimensional Hausdorff space is definedas . It can beshown that there is a critical value , such that for , thismeasure is infinite, and for , it is zero [35]. This criticalvalue is called the Hausdorff dimension of the set .

Another useful definition is the box-counting dimension. Ifwe partition the space into a grid of cubic boxes of size , andconsider as the number of boxes, which intersect the set ,the box-counting dimension of is defined as

(33)

It can be shown that for regular self-similar fractals, the Haus-dorff dimension is equal to the box-counting dimension [35].Intuitively, Theorem 3 means that in Scheme I among theavailable dimensions, only dimensions are effectively used.Indeed, we can show that for any modulation set6 with the box-counting dimension , the asymptotic slope of the SDR curveis at most .

Theorem 5: For a modulation mapping , if the mod-ulation set has the box-counting dimension , then

(34)

Proof: We divide the space into boxes of size . Con-sider as the number of cubic boxes that cover . We

6Modulation set is the set all possible modulated vectors.


divide the source signal set into segments of length .Consider as the corresponding -dimensionaloptimal decoding regions (based on the MMSE criterion), and

as their intersection with the cubes (seeFig. 2). The total volume of these sets is equal to thetotal volume of the covering boxes, i.e., . Thus, at least,half of these sets (i.e., of them) have volume less than

. For any of these sets, such as and any box, the volumeof the intersection of that box with the other sets is at least

. For any point in the correspondingsegments of the set , the probability of decoding to a wrongsegment is lower bounded by the probability of a jump to theneighboring sets in the same box. Because the variance of theadditive Gaussian noise is per each dimension, and for sucha jump, the squared norm of the noise at most needs to be(square of the diameter of the box), the probability of such ajump to the neighboring sets can be lower bounded as

(35)

(36)

where is the pdf of the noise vector .Now, for these segments of the source, consider the sub-

segments with the length at the center of them. Whenthe source belongs to one of these subsegments, wrongsegment decoding results in a squared error of at least

. Thus, for these sub-

segments whose total length is at least , atleast with the probability , we have a squared error

that is not less than . Therefore

(37)

where only depends on the bandwidth expansion . Onthe other hand, based on the definition of the box-countingdimension

(38)

By using (37) and (38)

(39)

It should be noted that Theorem 5 is valid for all signal sets,not just self-similar signal sets. As a corollary, based on the factthat the box-counting dimension cannot be greater than the di-mension of the space [35], Theorem 5 provides a geometric in-sight into (1).

Another scheme based on self-similar signal sets and the infi-nite binary expansion of the source is proposed in [26] and [27],which similar to the scheme proposed in this section, achievesan SDR scaling better than linear coding, but cannot achieve theoptimum SDR scaling. The scheme presented in [26] is based

on using repetitions of a ( , ) binary code to map the digitsof the infinite binary expansion of samples of the source tothe digits of an -dimensional transmit vector. This schemeshares the shortcoming of Scheme I. In [26], the bandwidth ex-pansion factor is and the SDR asymptotically scales as

, instead of the optimum scaling .The main difference between Scheme I and the scheme pro-posed in [26] is that in Scheme I, the delay is minimum (it usesonly one sample of the source for coding), but in [26], the delayis , and the the ratio between the SDR exponent and the op-timum SDR exponent is dependent on the delay (it is ), i.e., toincrease it, one needs to increase the length of the binary code,which results in increasing the delay.

The idea of using the infinite binary expansion of thesource, for joint source–channel coding, can be traced back toShannon’s paper [8], where shuffling the digits is proposed forbandwidth contraction (i.e., mapping high-dimensional data toa signal set with a lower dimension). For bandwidth expansion,space-filling self-similar signal sets have been investigated in[13]; however, the SDR scaling of those schemes are not betterthan linear coding. The reason is that when we use a self-similarset to fill the space, the squared error caused by jumping toadjacent subsets dominates the scaling of the distortion. Toavoid this effect, we need to avoid filling the whole space. Thisresults in losing dimensionality for self-similar sets, whichresults in suboptimum SDR scaling (as investigated in thissection). To avoid this drawback, we need to consider signalsets, which are not self-similar, as proposed in the next section.

V. ACHIEVING THE OPTIMUM ASYMPTOTIC SDRSLOPE USING A SINGLE MAPPING

Although Scheme I can construct mappings that achieve thenear-optimum slope for the curve of SDR (versus channel SNR),none of these mappings can achieve the optimum slope . Toachieve the optimum slope with a single mapping, we slightlymodify Scheme I. The main drawback of Scheme I is that weneed to have explicitly larger than 2 to provide enough spacingbetween different layers (to avoid the possibility of having largesquared errors caused by small noise vectors). As discussed inthe previous section, these spacings result in wasting a portion ofthe available dimensions (the effective dimension of the signalset is ). To avoid this drawback, we need to progressivelyreduce the wasted portion in finer layers and, at the same time,make sure that the gap between adjacent subsets is large enoughto control large errors, which result from jumping to adjacentsubsets.

Scheme II: For the modulating signal , consider. We construct ; see, for example,

(40)–(42), shown at the bottom of the next page. The differencebetween this scheme and Scheme I is that instead of assigningthe ( )th bit to the signal , the bits of the binary expan-sion of are grouped such that the th group ( )consists of bits and is assigned to the th dimension. In de-coding, we find the point in the signal set, which is closest tothe received vector . If , thefirst bits of can be decoded error free(for ), which include bits of thesource .


Theorem 6: Using the mapping constructed by Scheme II,for any noise variance , the output distortion is upperbounded by

(43)

where and only depend7 on .Proof: Similar to the proof of Theorem 3, we consider two

cases and apply a simple averaging of the two. Let be theGaussian noise on the th dimension and assume that is se-lected such that

(44)

The probability that is negligible.Indeed

(45)

(46)

(47)

(48)

where ( ) is due to

for ,

and ( ) is due to (44), and ( ) is due to .

On the other hand, when , for

, ; hence, the firstbits of are error free, which in-

clude bits of the source . Thus, the first

7Throughout this paper, � � � � � � � are constants, independent of � (they maydepend on � ).

bits of can be decodederror free. Now

(49)

(50)

(51)

(52)

where depends only on . Therefore, by using the assump-tion (44)

(53)

(54)

Consequently, the output distortion is bounded by

(55)

(56)

(57)

(58)

It should be noted that in this proof, the assumption of havinga uniform distribution is not used, and the previous proof is validfor any source whose samples are in the interval . InAppendix C, we extend the scheme proposed in this section toother sources, which are not necessarily bounded.

(40)

(41)

(42)


VI. APPROACHING A NEAR-OPTIMUM ASYMPTOTIC SDRSLOPE BY DELAY-LIMITED CODES

In [24], a family of HDA source–channel codes are proposed,which together can achieve the optimum SDR curve and each ofthem only suffers from the mild saturation effect (the asymptoticunit slope for the curve of SDR versus SNR). However, theirapproach is based on using capacity-approaching digital codesas a component of their scheme. In [25], it is shown that for anyjoint source–channel code that touches the optimum SDR curveat a certain SNR point, the asymptotic slope cannot be betterthan 1.

In this section, we consider the problem of finding a familyof delay-limited analog codes, which together have a boundedasymptotic loss in the SDR performance (in terms of decibel).Results of Section III show that none the previous analog codingschemes (based on dynamical systems) can construct such afamily of codes. In this section, we also show that the specificclass of HDA source–channel coding schemes under consider-ation can achieve this goal.

In HDA source–channel coding schemes under considera-tion, to map an -dimensional source to an -dimensionalsignal set, the source is quantized by points, which are sentover dimensions and the residual noise is transmittedover the remaining dimensions. In other words, the regionof the source (which is a hypercube for the case of a uniformsource) is divided into subregions . These subre-gions are mapped to parallel subsets of the -dimensionalEuclidean space, , where is a scaled version of

with a factor of .

Theorem 7: Consider a HDA joint source–channel scheme(as defined earlier), which maps an -dimensional uniformsource (inside the unit cube) to parallel -dimensional sub-sets of an -dimensional Euclidean space ( ), with apower constraint of 1. If the decoding of digital and analog partsare done separately, for any noise variance , the outputdistortion is lower bounded by

(59)

where depends only on and .Proof: See Appendix D.

Using (59) to compute the SDR slope establishes the desiredresult.

Now, we construct two families of delay-limited analogcodes, which, by a proper choice of parameters according to thechannel SNR (i.e., a specific member of each family is selectedfor a given range of channel SNR), offer a bounded asymptoticloss in the SDR performance (in terms of decibels).

Type I—Family of Piecewise Linear Mappings: For any, for , we construct an analog code

as follows.

For , whererepresents the fractional part, we construct as

(60)

First, we show that , for . By using thefact that the value of the bits are at most 1, and

(61)

(62)

(63)

Therefore, noting that , by an appropriate shift(e.g., modifying the transmitted signal set as ), thetransmitted power per dimension can be bounded by a constant,i.e., . Next, we show that the proposed scheme has a boundedgap (in terms of decibel) to the optimum SDR curve.

Theorem 8: In the proposed family of mappings (Type I),with noise variance , if we use the joint source–channelcode for , the output distortion isupper bounded by

(64)

where depends only on .Proof: The signal set consists of segments of length

, where each of them is a subsegment of the source region(the unit interval), scaled by a factor of .

For any , if the corresponding information bitsof and differ in the th position, then

. Therefore, the probability that the first error occursin the th bit ( , where ) of isbounded by

(65)and it results in an output squared error of at most

. Therefore, by consideringthe union-bound over all possible errors, we obtain

(66)


Now, by using and , we have

(67)

It is worth noting that in the proposed family of codes, foreach code, the asymptotic slope of the SDR curve is 1 (as weexpected from the fact that for each code, the mapping is piece-wise differentiable). We can mix the idea of this scheme withScheme II of the previous section, to construct a family of map-pings where for each of them, the asymptotic slope is , andtogether, they maintain a bounded gap with the optimal SDR(in terms of decibels).

Type II—Family of Robust Mappings: For, we construct as

Note that similar to the previous code construction,, hence the power of the transmitted signal is bounded.

Theorem 9: In the proposed family of mappings (Type II),there are constants , independent of and (areonly dependent on ), such that for every integer , ifwe use the modulation map , we have the following.

1) For ,

(68)

2) For any ,

(69)

Proof:1) Similar to the proof of Theorem 8, the probability that the

first error occurs in the th bit ( )of is bounded by and it results in anoutput squared error of at most , and when there isno error in the first bits, the squared error is

. Therefore, by considering the union bound over allpossible errors, we have

Similar to the proof of Theorem 8, by usingand , we have

2) Consider as the Gaussian noise on the th channel andassume that is selected such that

(70)

The probability that is negligible.[It is bounded by .]

On the other hand, when , the firstbits of can be decoded error free

( ), which include bits of .Thus, the first bits of can be decoded error free.Now, similar to the proof of Theorem 6

(71)

(72)

(73)

Therefore, by using assumption (70)

(74)

(75)

Therefore, the output distortion is bounded by

(76)


(77)

(78)

VII. SIMULATION RESULTS

In Fig. 3, for a bandwidth expansion factor of 4, the perfor-mance of Scheme I (with parameters and 4) is comparedwith the shift-map scheme with . As we expect, for theshift-map scheme, the SDR curve saturates at slope 1, while thenew scheme offers asymptotic slopes higher than 1. For the pro-posed scheme, with parameters and , the asymp-totic slope is, respectively, and(as expected from Theorem 3). Also, we see that the proposedscheme provides a graceful degradation in the low-SNR region.It should be added that the shift-map scheme, with a certain pa-rameter, is designed to be optimum for a certain SNR value,but in the proposed scheme, we do not optimize the code fordifferent SNR ranges. While most SNR-optimized codes (in-cluding simple digital codes) can beat the proposed scheme ina narrow SNR region, the proposed scheme has a better overallperformance in different ranges of SNR. For the sake of com-parison, Fig. 3 also includes the SDR-versus-SNR curve corre-sponding to inequality (5) as an upper bound. It is easy to showthat the actual curve will be below this bound witha gap (in the SDR) limited to , or 1.53 dB. There is a sig-nificant gap between this bound and the curves correspondingto delay-constraint cases (shift map and our proposed schemes).This gap is primarily due to the lack of optimum channel coding.The overall performance gap is (to a large extent) unavoidable.The corresponding SNR gap is similar to the SNR gap observedin comparing a modulation scheme (with limited dimension) tothat of a capacity achieving channel code. Ideally, the perfor-mance of the delay-constraint cases should be compared witha bound, which captures the constraint on source coding di-mension, as well as (indeed more importantly), the constrainton channel coding dimension. However, finding an appropriatebound to capture both these limitations is outside the scope ofthis study.

Fig. 4 shows the performance of Scheme II for di-mensions. As shown in the figure, the asymptotic exponent ofthe SDR is close to the optimum value of 4, i.e., the band-width expansion ratio. The fluctuations of the slope of the curveis due to the fact that groups of consequent bits are assignedto each dimension, and for different ranges of the SNR, er-rors in different dimensions become dominant (e.g., for SNRvalues around 40–50 dB, the error in the second layer of bitsof becomes dominant in the overall squared error). By mod-ifying Scheme II and assigning groups of bits of length

(instead of ) to the th dimension,we can slightly improve the performance in the middle SNRrange. Asymptotic exponents of the SDR in both variations ofScheme II are the same. Note that the SDR scaling of scheme IIis the same as Scheme I, with the difference that in Scheme II,it is possible to optimize the performance for a specific range ofSNR (by choosing an appropriate code among a family of ro-bust source–channel codes).

Fig. 4. Performance of Scheme II for � � � dimensions: (a) scheme in-troduced in Section V and (b) other variation of Scheme II, when groups of� � � � �� bits are considered.

VIII. CONCLUSION

To avoid the mild saturation effect in analog transmission(i.e., achieving the optimum scaling of the output distortion),one needs to use nondifferentiable mappings (more precisely,mappings that are not differentiable on any interval). Two non-differentiable schemes are introduced in this paper. Both theseschemes, which are minimum-delay schemes, outperform thetraditional minimum-delay analog schemes, in terms of scalingof the output SDR. Also, one of them (Scheme II) achieves theoptimum SDR scaling with a simple mapping (it achieves theasymptotic exponent for the SDR versus SNR).

APPENDIX APROOF OF THEOREM 1

The set of modulated signals consists of parallel seg-ments, where the projection of each of them on the th dimen-sion has the length ; hence, each segment has the length

. By considering the distance oftheir intersections with the hyperspace orthogonal to the th di-mension (which is at least ) and the angular factor of thesesegments, respecting to the -axis, because , we canbound the distance between two parallel segments of the mod-ulated signal set as (see Fig. 1)

(79)

First, we consider the case of . Consider

. The probability of a jump to a wrong

segment (during the decoding) is bounded by

(80)


(81)

By using

(82)

On the other hand, each segment of the modulated signalset is a segment of the source signal set, stretched by afactor of (its length ischanged from to ). There-fore, assuming the correct segment decoding, the averagedistortion is the variance of the channel noise divided by

:

(83)

(84)

(85)

where is the estimate of , and is independent of andand only depends on . Now, because and

are bounded by 1

(86)

(87)

(88)

On the other hand, for

(89)

(90)

(91)

(92)

Therefore, by combining these two bounds together, weobtain

(93)

APPENDIX BPROOF OF THEOREM 2

We consider the following two cases.

Case 1. : Each segment of the modulated

signal set is a segment of the source signal set, scaled by a factorof , and hence

(94)

(95)

(96)

(97)

Case 2. , for : In this

case, we bound the output distortion by the average distortioncaused by a large jump to another segment. Let be the additivenoise in the first dimension and be the modulatedvector corresponding to the source sample .

For any point in the interval(for ), when , for any

point , the received point is closerto than . Therefore, the decoded signal is

. Thus, in this case, the squared error is at least. Therefore, the average distortion is lower bounded by

(98)

(99)

(100)

(101)

By using

(102)

(103)

By combining the bounds (for two cases), and noting that

(104)

(105)

Note that this theorem is about an upper bound on the perfor-mance of the shift-map scheme, in other words, a lower boundon the distortion for various choices of parameters. To have ageneral lower bound, (104) is computed as the minimum of thelower bounds of different cases. This theorem states that nomatter how we choose the design parameters of the shift-mapscheme, the distortion is lower bounded by (10).


APPENDIX CCODING FOR UNBOUNDED SOURCES

Consider as an arbitrary memoryless i.i.dsource. We show that the results of Section V can be ex-tended for nonuniform sources, to construct robust jointsource–channel codes with a constraint on the averagepower. Without loss of generality, we can assume the vari-ance of the source to be equal to 1. For the source sample

, we can write it as , where is an in-teger, , and .Now, we construct the -dimensional transmission vector as

,where are constructed using (42) in Section V. Let

be the distortion conditioned on correct decoding of .Similar to the proof Theorem 6, we can show that is upperbounded by

(106)

where and depend only on .Now, we bound the distortion , for the case that is

not decoded correctly. Since is constructed by Scheme II(in Section V), is between 0 and , and hence,

. To have an error of , the am-plitude of the noise on the first dimension should be greaterthan , and hence, its probability is bounded by .

When , the overall squared error is lower boundedby

(107)

Therefore, by using the union bound for all values of , thedistortion is lower bounded by

(108)

(109)

(110)

Thus,To finish the proof, we only need to show that the average

transmitted power is bounded. For , the transmittedpower is bounded as . For

(111)

(112)

Thus, using the Cauchy–Schwarz inequality,

(113)

(114)

APPENDIX DPROOF OF THEOREM 7

We consider two cases for the scaling factor .

Case 1. : Eachsubset of the modulated signal set is the scaled version of a seg-ment of the source signal set by a factor of . Hence, we canlower bound the distortion by only considering the case that thesubset is decoded correctly and there is no jump to adjacent sub-sets

(115)

(116)

(117)

Case 2.

for : In this case, we bound the outputdistortion by the average distortion caused by a jump to anothersubset. Without loss of generality8, we can consider .Hence, . First, we show that there are two constants

and (independent of and ) such that the probability ofan squared error of at least is lower bounded by

(118)

By considering the power constraint, the maximum distance ofeach source sample to its quantization point is upper boundedby

(119)

We can partition the -dimensional uniform source tocubes of size .

We consider as the union of the quantization regions whosecenter is in the th cube ( ). Because the decoding ofdigital and analog parts are done separately, the -di-mensional subspace (dedicated to send the quantization points)can be partitioned to decoding subsets, corresponding to re-gions . If we consider , the intersectionsof these decoding regions, and the -dimensional cubeof size 4 centered at the origin, at least of them have volume

less than .

This volume is less than the volume of an -dimen-sional sphere of radius . Thus, for any point inside

with this property, the probability of being decoded to awrong subset is at least equal to the probability that theamplitude of the noise is larger than the radius of that sphere(i.e., ). This probability is lower bounded by

. Now, for thecubes corresponding to these subsets, we consider points insidea smaller cube of size , with the same center.

8For � � � � , the distortion � is larger than � (the distortion for

� � ), and hence,� � � � � � �� , and� dependsonly on � .


For these points, at least with the probability , decoder findsa wrong quantization region, where the distance between itscenter and the original point is at least .

Hence, the final squared error is at least

.Because at least half of the subsets have the mentioned

property, the overall probability of having this kind of pointsas the source is at least , and in transmitting these points,with a probability which is lower bounded by , the squarederror is at least . Therefore, the distortion is lowerbounded by

(120)

Finally, by considering the minimum of (117) and (120), weconclude

(121)

REFERENCES

[1] C. Shannon, “A mathematical theory of communication,” Bell Syst.Tech. J., vol. 27, p. 379–423, 623–656, Jul., Oct. 1948.

[2] C. Shannon, “Coding theorems for a discrete source with a fidelity cri-terion,” in Proc. IRE Natl Conv. Rec., Mar. 1959, pp. 149–163.

[3] E. Ayanoglu and R. M. Gray, “The design of joint source and channeltrellis waveform coders,” IEEE Trans. Inf. Theory, vol. 33, no. 6, pp.855–865, Nov. 1987.

[4] N. Farvardin and V. Vaishampayan, “Optimal quantizer design fornoisy channels: An approach to combined source–channel coding,”IEEE Trans. Inf. Theory, vol. 33, no. 6, pp. 827–838, Nov. 1987.

[5] N. Farvardin, “A study of vector quantization for noisy channels,” IEEETrans. Inf. Theory, vol. 36, no. 4, pp. 799–809, Jul. 1990.

[6] N. Phamdo, N. Farvardin, and T. Moriya, “A unified approach totreestructured and multistage vector quantization for noisy channels,”IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 835–850, May 1993.

[7] M. Wang and T. R. Fischer, “Trellis-coded quantization designedfor noisy channels,” IEEE Trans. Inf. Theory, vol. 40, no. 6, pp.1792–1802, Nov. 1994.

[8] C. E. Shannon, “Communication in the presence of noise,” Proc. IRE,vol. 37, pp. 10–21, 1949.

[9] V. A. Kotel’nikov, The Theory of Optimum Noise Immunity. NewYork: McGraw-Hill, 1959.

[10] J. M. Wozencraft and I. M. Jacobs, Principles of Communication En-gineering. New York: Wiley, 1965.

[11] U. Timor, “Design of signals for analog communications,” IEEE Trans.Inf. Theory, vol. 16, no. 5, pp. 581–587, Sep. 1970.

[12] C. M. Thomas, C. L. May, and G. R. Welti, “Design of signals foranalog communications,” IEEE Trans. Inf. Theory, vol. 16, no. 5, pp.581–587, Sep. 1970.

[13] S.-Y. Chung, “On the construction of some capacity-approachingcoding schemes,” Ph.D. dissertation, Massachusetts Inst. Technol.,Cambridge, MA, 2000.

[14] B. Chen and G. W. Wornell, “Analog error correcting codes based onchaotic dynamical systems,” IEEE Trans. Inf. Theory, vol. 46, no. 7,pp. 1691–1706, Jul. 1998.

[15] V. A. Vaishampayan and I. R. Costa, “Curves on a sphere, shift-mapdynamics, and error control for continuous alphabet sources,” IEEETrans. Inf. Theory, vol. 49, no. 7, pp. 1691–1706, Jul. 2003.

[16] M. Skoglund, N. Phamdo, and F. Alajaji, “Design and performanceof VQ-based hybrid digital–analog joint source–channel codes,” IEEETrans. Inf. Theory, vol. 48, no. 3, pp. 708–720, Mar. 2002.

[17] W. Wang, F. Alajaji, and T. Linder, “Design of VQ-based hybrid dig-ital–analog joint source–channel codes for image communication,” inProc. Data Compress. Conf., Mar. 2005, pp. 193–202.

[18] M. Skoglund, N. Phamdo, and F. F. Alajaji, “Hybrid digital–analogsource–channel coding for bandwidth compression/expansion,” IEEETrans. Inf. Theory, vol. 52, pp. 3757–3763, Aug. 2006.

[19] F. Hekland, “On the design and analysis of Shannon–Kotelnikov map-pings for joint source–channel coding,” Ph.D. dissertation, NorwegianUniv. Sci. Technol., Trondheim, Norway, 2007.

[20] H. Coward and T. A. Ramstad, “Quantizer optimization in hybrid dig-ital–analog transmission of analog source signals,” in Proc. Int. Conf.Acoust. Speech Signal, 2000, pp. 2637–2640.

[21] T. A. Ramstad, “Shannon mappings for robust communication,” Telek-tronikk, vol. 98, no. 1, pp. 114–128, 2002.

[22] X. Cai and J. W. Modestino, “Bandwidth expansion Shannon mappingfor analog error-control coding,” in Proc. 40th Annu. Conf. Inf. Sci.Syst., Mar. 2006, pp. 1709–1712.

[23] J. Ziv, “The behavior of analog communication systems,” IEEE Trans.Inf. Theory, vol. 16, no. 5, pp. 1691–1706, Aug. 1970.

[24] U. Mittal and N. Phamdo, “Hybrid digital–analog (HDA) jointsource–channel codes for broadcasting and robust communications,”IEEE Trans. Inf. Theory, vol. 48, no. 5, pp. 1082–1102, May 2002.

[25] Z. Reznic, M. Feder, and R. Zamir, “Distortion bounds for broadcastingwith bandwidth expansion,” IEEE Trans. Inf. Theory, vol. 52, no. 8, pp.3778–3788, Aug. 2006.

[26] N. Santhi and A. Vardy, Analog codes on graphs 2006 [Online]. Avail-able: arxiv.org/abs/cs/0608086

[27] N. Santhi and A. Vardy, “Analog codes on graphs,” in Proc. IEEE Int.Symp. Inf. Theory, Jul. 2003, p. 13.

[28] K. Bhattad and K. R. Narayanan, “A note on the rate of decay of mean-square error with SNR for the AWGN channel,” in Proc. Annu. AllertonConf. Commun. Control Comput., Sep. 2007, pp. 332–335.

[29] R. G. Gallager, Information Theory and Reliable Communication.New York: Wiley, 1968.

[30] I. Rosenhouse and A. J. Weiss, “Combined analog and digital error-cor-recting codes for analog information sources,” IEEE Trans. Commun.,vol. 55, no. 11, pp. 2073–2083, Nov. 2007.

[31] Y. Kochman and R. Zamir, “Analog matching of colored sourcesto colored channels,” in IEEE Int. Symp. Inf. Theory, Jul. 2006, pp.1539–1543.

[32] Y. Kochman and R. Zamir, “Approaching�� in colored jointsource/channel broadcasting by prediction,” in IEEE Int. Symp. Inf.Theory, Jun. 2007, pp. 2231–2235.

[33] S. Shamai (Shitz), S. Verdu, and R. Zamir, “Systematic lossy source/channel coding,” IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 564–579,Mar. 1998.

[34] K. Bhattad and K. R. Narayanan, “A note on the rate of decay ofmean-squared error with SNR for the AWGN channel,” IEEE Trans.Inf. Theory, vol. 56, no. 1, pp. 332–335, Jan. 2010.

[35] G. A. Edgar, Measure, Topology and Fractal Geometry. New York:Springer-Verlag, 1990, ch. 6.

Mahmoud Taherzadeh was born in Borujen, Iran, in 1977. He received theB.Sc. and M.Sc. degrees in electrical engineering from Sharif University ofTechnology (SUT), Tehran, Iran, and the Ph.D. degree from the University ofWaterloo, Canada, in 1999, 2002, and 2008, respectively. His research interestsare communication theory and coding, especially coding and decoding in mul-tiple-antenna systems, and joint source–channel coding. He is currently workingat Ciena Corporation, Ottawa, ON, Canada (optical communications).

Amir Keyvan Khandani received the M.A.Sc. degree from the University ofTehran, Tehran, Iran, and the Ph.D. degree from McGill University, Montreal,QC, Canada, in 1985 and 1992, respectively. After graduation, he worked for 1year as a Research Associate with INRS Telecommunication, Montreal. In 1993,he joined the Department of Electrical and Computer Engineering, Universityof Waterloo, Waterloo, ON, Canada, where he is currently a Professor.

Documents

Single-Sample Robust Joint Source–Channel Coding: Achieving Asymptotically Optimum Scaling of SDR Versus SNR