Extended Ziv–Zakai Lower Bound - AMiner · PDF fileThese bounds relate the MSE in the estimation problem to the probability of error in a ... a lower bound on the MSE can be obtained

c 199x IEEE. Personal use of this material is permitted. However, permis-sion to reprint/republish this material for advertising or promotional purposesor for creating new collective works for resale or redistribution to servers orlists, or to reuse any copyrighted component of this work in other works mustbe obtained from the IEEE .

1

624 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 2, MARCH 1997

Extended Ziv–Zakai Lower Boundfor Vector Parameter Estimation

Kristine L. Bell, Member, IEEE, Yossef Steinberg, Yariv Ephraim,Fellow, IEEE,and Harry L. Van Trees,Life Fellow, IEEE

Abstract—The Bayesian Ziv–Zakai bound on the mean squareerror (MSE) in estimating a uniformly distributed continuousrandom variable is extended for arbitrarily distributed contin-uous random vectors and for distortion functions other thanMSE. The extended bound is evaluated for some representativeproblems in time-delay and bearing estimation. The resultingbounds have simple closed-form expressions, and closely predictthe simulated performance of the maximum-likelihood estimatorin all regions of operation.

Index Terms— Parameter estimation, performance bounds,mean square error.

I. INTRODUCTION

L OWER bounds on the minimum mean square error(MSE) in estimating a set of parameters from noisy

observations are widely used for problems where the exactminimum MSE is difficult to evaluate. Such bounds providethe unbeatable performance of any estimator in terms of theMSE. They can be used to investigate fundamental limits of aparameter estimation problem, or as a baseline for assessingthe performance of a specific estimator. In this paper, wefocus on the derivation of good, computable lower boundsand demonstrate their application.

We are primarily interested in applying bounds to signalparameter estimation problems in which the source signalis a nonlinear function of the unknown parameter. Typicalunknown parameters include the time-delay, frequency, phase,or bearing of a signal. In these types of problems, severaldistinct regions of operation can be observed. In the asymptoticregion, which is characterized by long observation time and/orhigh signal-to-noise-ratio (SNR), the MSE is small. In theapriori performance region where observation time and/or SNRare very small, the observations provide little information andthe MSE is close to that obtained from the prior knowledgeabout the problem. Between these two extremes, there maybe an additional ambiguity region, or simply a transition

Manuscript received January 6, 1995; revised July 15, 1996. The materialin this paper was presented in part at the 1994 IEEE International Symposiumon Information Theory, Trondheim, Norway, June 1994 and at the 1994 IEEEIMS Workshop on Information Theory and Statistics, Alexandria, VA October1994.

K. L. Bell is with the Department of Applied and Engineering Statistics,George Mason University, Fairfax, VA, 22030-4444 USA.

Y. Steinberg is with the Department of Electrical and Computer Engineer-ing, Ben-Gurion University of the Negev, P.O. Box 653, Beer-Sheva 84105,Israel.

Y. Ephraim and H. L. Van Trees are with the Department of Electrical andComputer Engineering, George Mason University, Fairfax, VA 22030-4444USA.

Publisher Item Identifier S 0018-9448(97)00630-5.

region. We are interested in bounds which closely characterizeperformance in all regions and accurately predict the locationsof the thresholds between regions.

The most commonly used bounds include the Cramer–Raobound (CRB) [1]–[3], the Barankin bound (BB) [4], theZiv–Zakai bound (ZZB) [5]–[8], and the Weiss–Weinsteinbound (WWB) [9], [10]. The Cram´er–Rao and Barankinbounds belong to the family of deterministic “covarianceinequality” bounds [11], which treat the parameteras anunknown deterministic quantity, and provide bounds on theMSE in estimating any selected value of the parameter, say

. Other bounds in this family include the Bhattacharyya [12],Hammersley–Chapman–Robbins [13], [14], Fraser–Guttman[15], Kiefer [16], and Abel [17] bounds. The CRB is generallythe easiest to evaluate, but it does not adequately characterizeperformance outside of the asymptotic region. The Barankinbound is tighter than the CRB, and has been used for locatingthresholds and for analyzing ambiguity region performance. Itis more difficult to evaluate, however, and requires optimiza-tion over a set of “test points.”

The major drawback of local bounds is that the estimatorsto which they apply must be restricted in some manner.Otherwise, the trivial bound of zero can be obtained, forexample, when the estimator is chosen to be a constant equalto . The CRB and BB require estimators to be unbiased.This restriction limits the applicability of the bounds sincebiased estimators are often unavoidable. For example, in thecommonly encountered situation of a parameter whose supportis finite, no estimators can be constructed which are unbiasedover the entire parameter space (see, e.g., [5]). Furthermore,local bounds do not take into account anya priori informationabout the parameter space.

The extent to which the bounds are impacted by theseconsiderations is illustrated in Fig. 1 for a typical case inbearing estimation. The details of this example can be foundin [18]. In this problem, the parameter of interest is thebearing, or direction-of-arrival, of a signal observed by anarray of sensors. Thea priori parameter space is limited toa finite interval by physical considerations. In Fig. 1(a), theCramer–Rao bound, Barankin bound, and the MSE of themaximum-likelihood estimator (MLE) obtained from simula-tions are plotted versus SNR. In the asymptotic region, bothbounds are equal and closely predict the performance of theMLE. The MSE of the MLE exhibits a sharp increase when theSNR drops below the asymptotic region threshold, then flattensout as the size of the estimation errors becomes limited by

0018–9448/97$10.00 1997 IEEE

BELL et al.: EXTENDED ZIV–ZAKAI LOWER BOUND FOR VECTOR PARAMETER ESTIMATION 625

(a)

(b)

Fig. 1. MSE boundsand MLE performance versus SNR for bearing estima-tion. (a) Local Cramer–Rao and Barankin bounds. (b) Global Ziv–Zakai andWeiss–Weinstein bounds.

the size of thea priori parameter space. The Barankin boundprovides a reasonably good indication of the location of theasymptotic region threshold. However, both bounds exceed theMSE of the MLE in the low SNR region. This behavior canbe attributed both to the violation of the unbiased estimatorassumption, and to the lack ofa priori information in thebounds. Similar observations were made in [17].

The Ziv–Zakai and Weiss–Weinstein bounds are Bayesianbounds which assume that the parameter is a random variablewith known a priori distribution. They provide bounds on theglobal MSE averaged over thea priori probability densityfunction (pdf). There are no restrictions on the class of esti-mators to which they apply, and they incorporate knowledgeof the a priori parameter space via thea priori distribution.The ZZB was originally derived by Ziv and Zakai in [5],and improved by Seidman [6], Chazan, Zakai, and Ziv [7],and Bellini and Tartara [8]. The Bellini–Tartara bound is thetightest in the group [8], [19]. These bounds relate the MSE inthe estimation problem to the probability of error in a binarydetection problem. The WWB is a member of a family of

Bayesian bounds derived from a “covariance inequality” prin-ciple. This family includes the Bayesian Cramer–Rao bound[20], Bayesian Bhattacharyya bound [20], Bobrovsky–Zakaibound (for parameter estimation) [21], and the more generalfamily of bounds proposed by Bobrovsky, Mayer–Wolf, andZakai [22].

The ZZB and WWB, and the global MSE of the MLE ob-tained from simulations are plotted for comparison in Fig. 1(b)for the same bearing estimation problem. These bounds aretight and reliable in all regions of operation. In addition, theyprovide a better indication of the asymptotic region thresholdthan the Barankin bound. However, because they bound theglobal MSE, Bayesian bounds can be strongly influencedby the performance at the parameter values which have thelargest errors. They are most useful when performance isnearly the same for all values of the parameter, as is thecase in the example shown. Bayesian bounds can also becomputationally complex. Evaluation of the WWB involveschoosing test points and inverting a matrix, while the ZZBrequires evaluation of several integrals, often numerically.

While the CRB, BB, and WWB are applicable to estimationof vectors of parameters, the ZZB is applicable only toestimation of uniformly distributed scalar random variables.Nevertheless, the ZZB has been used to analyze a varietyof nonlinear signal parameter estimation problems in [5]–[8]and [23]–[31]. In this paper, the Bellini–Tartara form of theZZB is extended to arbitrarily distributed continuous vectorrandom variables, and to distortion functions other than MSE.The extended bound is then analyzed for some representativeproblems in time-delay and bearing estimation which couldnot be handled previously by the ZZB. The evaluation isstraightforward, and the resulting bounds have simple closed-form expressions which are good for all regions of operation.Additionally, simulation results similar to those shown inFig. 1 are provided to verify that the bound closely predictsthe performance of the MLE.

The organization of the paper is as follows. In SectionII, the ZZB is extended to arbitrarily distributed continuousvector parameters, and to other distortion functions. In SectionIII, some properties of the extended bound are developed.Examples are given in Section IV, and concluding remarksare given in Section V.

II. EXTENDED ZIV–ZAKAI BOUND

The original Ziv–Zakai bound [5] and the improvements in[6]–[8] were all derived for the special case of a continuousuniformly distributed scalar random variable. Ziv and Zakaialso derived a version for discrete scalar parameters witharbitrary probability mass functions in [32]. In this paper, wefocus on continuous parameters. We begin by generalizing theBellini–Tartara form of the ZZB for continuous scalar randomvariables with arbitrarya priori pdf’s. The derivation useselements from [7], [8], and [32], but is organized differently.This formulation provides the framework for further gener-alizations of the bound to arbitrarily distributed continuousvector random variables, and to distortion functions other thanMSE.


A. Scalar Parameters

Consider estimation of a continuous scalar random variablebased upon noisy observations . We assume that

is a subset of the real line and thathas a priori mean, variance , and pdf . Let denote the

conditional pdf of given , and denote the jointpdf of and . For any estimator , the estimation error is

. We are interested in lower-bounding the MSE

(1)

where expectation is taken over bothand .The derivation starts from the identity [33, p. 24]:

(2)

Since both and are nonnegative, a lower boundon the MSE can be obtained by lower-bounding .This is done as follows:

(3)

(4)

(5)

Now let and , where is fixed. This yields

(6)

Multiplying and dividing by ,

(7)

Now consider the detection problem defined by

(8)

with

(9)

and the suboptimal decision rule in which the parameter isfirst estimated and a decision is made in favor of its “nearestneighbor”

Decide if

Decide if(10)

The term in square brackets in (7) is the probability of error forthis suboptimal decision scheme. It can be lower-bounded bythe minimum probability of error obtained from the likelihoodratio test, . Therefore,

(11)

This inequality can be improved using a technique from [8].Since is a nonincreasing function of, it can bemore tightly bounded by applying a “valley-filling” function tothe right-hand side of (11). For any function , the function

is a nonincreasing function of obtained by fillingin any valleys in , i.e., for every , is givenby [25]:

(12)

The valley-filling function is illustrated in Fig. 2. After apply-ing the valley-filling function to the right-hand side of (11),we have

(13)

Substituting (14) into (2) gives the final bound

(14)

When is uniformly distributed on , the bound (11)on reduces to Kotelnikov’s inequality [34, p. 91],[8], and the MSE bound (14) becomes

(15)

which is the Bellini–Tartara bound [8]. The Chazan–Zakai–Zivbound [7] is obtained by omitting the valley-filling function.


Fig. 2. Valley-filling function.

Note that when is uniform, the prior probabilities of thetwo hypotheses in the detection problem in (9) will be equal to

for all possible combinations of parameter values. Whenis not uniform, the prior probabilities vary as a function of

the a priori pdf.

B. Vector Parameters

Now consider estimation of a -dimensional continuousvector random parameter based upon noisy observations

. We now assume that is a subset of , and thathasa priori mean , covariance matrix , and pdf .Let denote the conditional pdf of given , and

denote the joint pdf. For any estimator , theestimation error is , and the error correlationmatrix is defined as

(16)

where expectation is taken overand . We are interested inlower-bounding for any -dimensional vector . Ofspecial interest is the case whenis the unit vector with a onein the th position. This choice yields a bound on the MSE ofthe th component of .

When estimation performance for only one component of, say , is of interest, there are several approaches which

may be taken to obtain a bound. One approach is to firstfind the joint pdf by averaging overthe remaining “nuisance” parameters, then apply theextended scalar ZZB in (14) to the resulting scalar problem.A second approach is to find the conditional pdf ofandgiven the nuisance parameters, ,apply the extended scalar ZZB in (14) to theconditionalscalarproblem, and then average the result over the nuisance param-eters. A third approach is to bound the matrix formfor all the parameters jointly, then chooseto obtain the desired bound. It was shown in [22] that ingeneral, bounds obtained using the first approach (averaging)are always as least as good as bounds obtained using thesecond approach (conditioning). Bounds obtained from thethird approach (joint bounding) may be better or worse than theother alternatives depending on the problem. In this section,the extended scalar ZZB is generalized for vector parameters

to a joint bound of the third type. In Section III, it will beshown that bounds obtained from the vector ZZB are at leastas good as bounds obtained using the conditioning approachwith the scalar ZZB.

The derivation of the vector bound begins by replacingwith in (2):

(17)

Again, can be lower-bounded as follows:

(18)

(19)

(20)

Letting and , then multiplying and dividingby gives

(21)

(22)


For a given , if is chosen to satisfy

(23)

then (22) becomes

(24)

Now consider the detection problem defined by

(25)

with

(26)

The term in square brackets in (24) represents the proba-bility of error for this detection problem using the followingsuboptimal decision rule:

Decide if

Decide if(27)

The detection problem and suboptimal decision rule are illus-trated in Fig. 3. The constraint means that on ,lies on the hyperplane perpendicular to the-axis defined by

(28)

The decision regions are separated by the hyperplane

(29)

which is also perpendicular to the-axis and passes throughthe midpoint of the line connecting and . A decisionis made in favor of the hypothesis which is on the same sideof the separating hyperplane as the estimate .

Lower-bounding the suboptimal probability of error by theminimum probability of error obtained from the likelihoodratio test yields

(30)

Fig. 3. Vector parameter binary detection problem.

which is valid for any satisfying (23). The tightest boundis obtained by maximizing over subject to this constraint.Performing the maximization and applying the valley-fillingfunction gives

(31)

Substituting (31) into (17) yields the final vector bound

(32)

The vector bound relates the MSE to the probability of errorin a detection problem in which the prior probabilities varywith and . The prior probabilities are equal for all andonly in the special case when is uniform. In many casesit can be quite difficult to evaluate the probability of error, andhence the extended ZZB, when the hypotheses are not equallylikely. In the next section, a weaker but more practical boundis derived in which prior probabilities in the detection problemare always equal, regardless of the form of .

C. Equally Likely Hypotheses

We consider the same continuous vector parameter estima-tion problem as in the previous section. The derivation ofthe new bound begins with the identity in (17) and proceedsthrough (21). The expression in (21) is then lower-bounded by

(33)


Multiplying and dividing by and assuming , (33)becomes

(34)

The term in square brackets is the probability of error in thesuboptimal nearest neighbor decision scheme in (27) for thedetection problem in (25) when . Itcan be lower-bounded by the minimum probability of error forequally likely hypotheses, . Maximizing over

, and applying the valley-filling function yields

(35)

and

(36)

When thea priori pdf is uniform, the equally likelyhypotheses bound (36) and the general bound (32) are thesame. In Section III, we will show that the equally likelyhypotheses bound is weaker than the general bound when

is not uniform. Even though it is weaker in theory, theequally likely hypotheses bound is often much easier to workwith in practice. This bound will be used in the examples inSection IV.

D. Other Distortion Functions

The vector bounds (32) and (36) can be further generalizedfor a wide class of distortion functions in addition to MSE.Consider a distortion function which is symmetric, non-decreasing, and differentiable with and derivative

. The average distortion is

(37)

By the fundamental theorem of calculus

(38)

(39)

where is the indicator function which is equal to one whenits argument is true. Substituting (39) into (37) yields

(40)

(41)

(42)

Making the substitution

(43)

For vector parameters, this becomes

(44)

Since is nondecreasing for , is nonnegativefor , and we can lower-bound the integral in (44) bybounding using either (31) or (35). Note thatwhen , and (43) reduces to (17) asexpected.

III. PROPERTIES

Properties of the extended Ziv–Zakai bound (EZZB) forvector random parameters derived in the previous section willbe developed here. In the analysis, the following probabilityof error expressions will be needed:

(45)

and

(46)

(47)

(48)

We also make use of properties of monotonic ellipticallycontoured distributions in some of the results. A vector


is said to have an elliptical distribution with parametersand if its density function has the form [35, p. 34]:

(49)

for some function , where is a normalizing constantand is positive-definite. The argument of in (49)is the equation of an ellipsoid centered at, whose shapeis defined by the matrix . We will be restricting atten-tion to monotonic elliptical distributions in which is amonotonic, nonincreasing function of . The multivariateGaussian and distributions are members of this family,as are Gaussian mixture densities [35]. In the Appendix,it is shown that monotonic elliptical distributions have thefollowing properties.

Property A1:

(50)

Property A2:

(51)

We now develop properties of the EZZB.Property 1: The general bound (32) and the equally likely

hypotheses bound (36) are equal when thea priori pdf isuniform, otherwise the general bound is tighter.

Proof: To prove this, we show that the integrand in theintegral over in (32) is greater than the corresponding termin (36), i.e.,

(52)

Using (47) and (45), we obtain

(53)

(54)

(55)

Equality is obtained when for all values ofand for which . This is satisfied when

is constant over , i.e., has a uniforma priori pdf.Property 2: When has a monotonic ellipticala priori

distribution, both the general bound (32) and the equally likelyhypothesis bound (36) converge to in the region of verylow observation time and/or SNR.

Proof: Let EZZB( ) denote the general bound in (32),i.e.,

EZZB

(56)

As observation time and/or SNR become very small, theobservations become useless. In the estimation problem, theminimum MSE estimator converges to thea priori mean

, and the corresponding error correlation matrix convergesto . Any lower bound on , including EZZB ,therefore cannot exceed . In the detection problem, theoptimal decision rule tends to always favor the hypothesiswith the greatest prior probability. Therefore, the probabilityof error approaches the minimum of the prior probabilities, i.e.,

(57)

(58)

By the dominated convergence theorem, (57) can be substi-tuted into (56) to obtain a limit on EZZB(). Then using (50),(51), we have

(59)

(60)

(61)

(62)

The valley-filling function is trivial in (61) because the terminside the brackets is nonincreasing in. Since EZZB( ) isboth upper- and lower-bounded by as observation timeand/or SNR decrease, it must converge to . The prooffor the equally likely hypothesis bound is identical.

Property 3: The general bound (32) coincides with theminimum MSE, achieved by conditional mean estimator,

, when thea posteriori distribution is a mono-tonic elliptical distribution with parameters and , where

does not depend on.Proof: Let denote the error correlation matrix for

the conditional mean estimator . This is the optimalcorrelation matrix, therefore EZZB . To showthat the bound is equal to under the stated conditions,we proceed as in the proof of Property 2, using (48), (50),and (51):

(63)


(64)

(65)

(66)

(67)

(68)

(69)

Again, since EZZB is upper- and lower-bounded by, then EZZB .

This property holds, for example, when the parameter ofinterest is a multivariate Gaussian parameter, and the observa-tions are a linear function of the parameter plus multivariateGaussian noise. The Bayesian Cramer–Rao bound is also equalto the minimum MSE in this case [20, p. 84].

Property 4: When the probability of error for equally likelyhypotheses is only a function of the offset between thehypotheses, and not a function ofitself, i.e.,

the equally hypotheses bound (36) simplifies to

(70)

where

(71)

Furthermore, when has a monotonic elliptical distribution,Property 2 implies that

(72)

These simplifications will be used in the examples.Property 5: When a bound on the MSE of one component

of is desired, the vector bound (32) is always at least aslarge as the bound obtained by conditioning the scalar bound(14) on the remaining parameters, then averaging overthose parameters, when valley-filling is omitted.

Proof: Assume that the component of interest is.Choosing yields

(73)

The tightest bound is achieved by maximizing oversubject tothe constraint . A weaker bound is obtained by simplychoosing . This gives

(74)

(75)

(76)

The last expression is the expected value of the conditionalscalar bound without valley-filling.

Property 6: Other multiple parameter bounds such as theWeiss–Weinstein, Cram´er–Rao, and Barankin bounds can beexpressed in terms of a matrix,, which is a lower boundon the correlation matrix in the sense that for an arbitraryvector , . The Ziv–Zakai formulation doesnot automatically produce a bound matrix, however, it ispossible in some cases to manipulate the bound to have thisform. The bearing estimation problem considered in SectionIV is one example. Even when the EZZB does not have thematrix form, it conveys the same information. In both cases,bounds on the MSE of the individual parameters and any linearcombination of the parameters can be obtained. The matrixform has the advantage that the bound matrix only needs tobe calculated once, while the nonmatrix form must in principalbe recomputed for each choice of.

IV. EXAMPLES

In this section, the EZZB is evaluated for some repre-sentative problems in time-delay and bearing estimation. Wefirst revisit the problem of estimating the time delay of a


finite-bandwidth pulse considered in [5] and [7]. The resultsof [7] are generalized for nonuniforma priori distributions.Furthermore, when the pulse is narrow with respect to thea priori parameter space, a simpler closed-form expressionthan is found in [7] is derived, as well as expressions for thethresholds separating the operating regions.

We then consider the problem of estimating the two-dimensional bearing of a narrowband planewave signalincident on a planar array of sensors. The scalar ZZB wasapplied to the problem of estimating the one-dimensionalbearing using a uniformly spaced linear array in [26]–[29].Analytical expressions for certain segments of the ZZB,as well as threshold expressions, were derived. We allowfor arbitrary planar array configurations, and develop anexpression which applies to all regions of operation. In bothexamples, simulation results are provided to demonstrate thatthe bounds closely predict the performance of the associatedMLE’s.

A. Time-Delay Estimation

In this problem, the observed signal consists of adelayed version of a known signal and additive noise

:

(77)

where is the unknown delay to be estimated. The signal isassumed to be a finite-bandwidth pulse whose shape is known,and the noise waveform is a sample function of a zero-mean,stationary, white Gaussian random process with double-sidednoise spectral density .

The delay is assumed to be a random parameter witha priori mean , variance , and pdf , which isdefined on the interval . To simplify the analysis, isrestricted to be symmetric and nonincreasing in . Theseconditions define the scalar equivalent of a monotonic ellipticaldistribution, and are satisfied by a wide class of distributionswhich includes the uniform distribution. Initially, it is assumedthat the pulse is narrow with respect to the length of theapriori parameter space.

To bound the MSE for this problem, we will use thesimplified equally likely hypotheses bound given in (70), withvalley-filling omitted.1 For scalar parameters, it has the form

(78)

where

(79)

When is symmetric and nonincreasing in ,is a nonincreasing function of with and .From (72), satisfies

(80)

1Omitting the valley-filling function produces a simpler, but theoreticallyweaker bound. In this problem it turns out that the functionA(h)P el

min(h) is

nonincreasing inh, and valley-filling has no effect on the bound. Therefore,valley-filling is eliminated at the outset to reduce complexity in the derivation.

The probability of error in deciding between two equallylikely signals and is given by

(81)

where is the signal energy

(82)

is the correlation of

(83)

and

(84)

The bound (78) can be evaluated by first lower-boundingthe probability of error in the same manner as in [5] and[7]. Let denote the Fourier transform of . ApplyingParseval’s theorem to (82) and (83)

(85)

(86)

Using in (86), canbe upper-bounded by

(87)

where is the angular bandwidth of the pulse

(88)

Furthermore, since

(89)

can then be lower-bounded by using the minimum of(87) and (89) in (81). This yields

(90)

Substituting (90) into (78) yields

(91)The first integral is evaluated using (80). To evaluate thesecond integral, note that when the pulse is narrow with respect


to the length of thea priori parameter space, will be largeand will be small compared to . For small in theinterval , . With this approximation,the second integral in (91) can be evaluated using integrationby parts:

(92)

(93)

(94)

(95)

where is the incomplete gamma function defined by

(96)

and . Substituting (80) and (95) into (91) yieldsthe final bound

(97)

This bound depends on thea priori distribution only through, and on the pulse shape only through. It is valid when

the pulse is narrow with respect to. We have found that thiscondition is satisfied when the product is greater than

. This expression is considerably simpler than the bound in[7], which is derived without restrictions on the pulsewidth.

To analyze the bound, note that

1) As and

2) As and .

Therefore, in the region where is small, the bound isdominated by the first term in (97) and it approaches theapriori covariance . This performance can be achieved byignoring the observations and always estimatingby the apriori mean . In the asymptotic region, where islarge, the bound is dominated by the second term in (97) andthe bound approaches , which is the inverse of theFisher information for this problem [5], [7]. This is the sameasymptotic value as predicted by the local Cramer–Rao boundfor each value of . Note that the EZZB provides a boundon the global MSE, while the CRB provides a bound on thelocal MSE for a specified value of. The bounds coincidebecause asymptotic performance characterized by the Fisherinformation is the same for all .

The thresholds to the asymptotic anda priori performanceregions can be found using techniques from [27], [28]. The

threshold delimiting thea priori performance region canbe defined as the value of where the bound dropsapproximately 3 dB below thea priori performance level.Since the bound is dominated by the first term in (97) in thisregion, this occurs when

(98)

Letting denote the value of at the threshold to thea priori performance region, we have

(99)

The threshold to the asymptotic region occurs at the pointwhere the bound is essentially equal to the asymptotic valueof . This can be defined as the value of atwhich the right-hand side of (97) exceeds by a smallamount, say 10%. In this region, , therefore, thethreshold occurs when

(100)

Letting denote the value of at the threshold to theasymptotic region, can be found numerically from the largerof the two solutions to the equation

(101)

In Fig. 4 the bound (97) is plotted versus for. Also shown are the thresholds predicted by (99) and

(101), and the MSE of the MLE obtained from simulationswhen is a raised cosine pulse, and is uniformlydistributed on . Each point on the MSE curve representsan average of 2000 trials with the delaydrawn randomlyfrom the uniform prior distribution. For this example, we have

(102)

otherwise

(103)

(104)

(105)

When , the pulsewidth is approximately one-tenth of thea priori interval . In this example, the boundclosely predicts the performance of the MLE over all regions,and the predicted thresholds are also accurate.

When the pulse is not narrow with respect to, the EZZBcan be calculated using a more exact expression for in(93) than the approximation . In order to proceed, theform of and hence must be specified. To evaluatethe bound, can be approximated by a polynomial in


Fig. 4. Extended Ziv–Zakai bound (EZZB) and simulated MLE performancefor raised cosine pulse for�2�2

�= 100. The prior distribution of� is uniform

on [0; T ]. All values are normalized by thea priori variance of� .

using a Taylor-series expansion of appropriate order tosufficiently characterize over the interval . Thesecond integral in (91) can then be expressed in closed formas a sum of incomplete gamma functions of higher order. Forexample, if is the uniform density

(106)

then has the form

(107)

and the first-order Taylor-series expansion of is equal tothe exact expression (107). This results in a bound with oneadditional term

(108)

which agrees with the bound derived in [7].

B. Bearing Estimation

We next consider the problem of estimating the two-dimensional bearing of a narrowband planewave signalincident upon an -sensor planar array, as illustrated inFig. 5. The sensor locations are defined by the matrix

(109)

The th column of , denoted by , represents the locationof the th sensor in the -plane. The source’s bearing hastwo components which may be expressed in either polar or

Fig. 5. Geometry of the single-source bearing estimation problem using aplanar array.

Cartesian coordinates

(110)

(111)

We shall use Cartesian coordinates, therefore the vectoristhe parameter of interest. Note that in the special case of alinear array with sensors along the-axis or -axis, only onecomponent of the bearing, , can be estimated.

The source is assumed to generate a narrowband signalcentered about a known frequency. At each sensor, theobserved waveform consists of a delayed version of the sourcesignal and additive noise. The time delay at theth sensor(relative to the origin of the array) is given by

(112)

where is the speed of propagation.Letting , , and denote the complex envelopes

of the source signal, the additive noise at theth sensor, andthe received signal at theth sensor, respectively, we have

(113)

Employing vector notation, the vector of receivedsignals may be expressed as

(114)

where

and

The vector is the “array response” or “steering vector”

(115)

It is assumed that the source and noise waveforms are sam-ple functions of uncorrelated, zero-mean, stationary Gaussianrandom processes. The observed data consists of a set ofsnapshots taken at times , which are sufficientlyspaced so that successive samples of the source and noise


waveforms are uncorrelated. Under these assumptions, thesignal samples, , are independent and identicallydistributed (i.i.d.) zero-mean, complex Gaussian random vari-ables with variance , and the noise samples, ,are i.i.d. zero-mean, complex Gaussian random vectors withcovariance .

It is assumed that is a random parameter with mean ,covariance matrix , anda priori pdf , which is definedon a subset of the unit disc. For this analysis, is requiredto be circularly symmetric about thea priori mean, , and tobe nonincreasing in . With these restrictions,will have a monotonic circular distribution, which is a specialcase of a monotonic elliptical distribution. Evaluation of thebound for this problem is similar to the previous example, butmore involved. The analysis is carried out in [18], and theresults summarized here. The final bound is

(116)

The matrix denotes the Fisher information matrix, whichhas the form

(117)

where is an increasing function of the number of sensorsand the SNR

(118)

and is the curvature matrix for the array beam patternat the peak of the main lobe. It can be expressed in terms ofthe sensor locations as follows:

(119)

where denotes the vector of ones and .The bound has a matrix form which is very similar to the

bound derived in the time-delay example. It is a decreasingfunction of the product , thus SNR and observation timecan be traded off to some extent to achieve a desired levelof performance. In the region where is small, the boundmatrix approaches thea priori covariance matrix . Thisperformance can be achieved by ignoring the observationsand always estimating by the a priori mean . In theasymptotic region when is large, the bound approachesthe inverse Fisher information matrix .

The thresholds to the asymptotic anda priori regions can bedefined as in the previous example. Heredenotes the valueof at the threshold to thea priori performance region, andis given by

(120)

In this problem, the value of at the threshold to theasymptotic region will be a function of the vectorand will

Fig. 6. Extended Ziv–Zakai bound (EZZB) and simulated MLE performancefor ux using a 16-element circular array with1

2-wavelength spacing for

N = 100. The prior distribution ofuuu is uniform on the unit disc withcovarianceRRRuuu =

1

4III. All values are normalized by thea priori variance

of ux.

be denoted by . It can be found numerically from thelarger of the two solutions to the equation

(121)

In Fig. 6, the bound (116) is plotted versus SNR. Alsoshown is the MSE of the MLE obtained from simulations,and the thresholds predicted by (120) and (121). The arrayis a 16-element circular array with -wavelength spacing,and the prior distribution of the two-dimensional bearingis uniform on the unit disc. Each point on the MSE curverepresents an average of 2000 trials with the source DOAdrawn randomly from the uniform prior distribution. Fig. 6shows the element of the bound matrix, which is abound on the MSE in estimating the-component of .Because the array is symmetric with respect toand , theperformance in estimating theand components of will bethe same. For this example, the explicit bound closely predictsthe performance of the MLE in all regions, and the predictedthresholds are also accurate.

V. CONCLUDING REMARKS

The Ziv–Zakai lower bound on the MSE in estimatinga set of random parameters from noisy observations hasbeen extended for arbitrarily distributed continuous vectorsparameters, and for a large class of distortion functions inaddition to squared error. Two versions were derived, bothof which relate the MSE in the estimation problem to theprobability of error in a binary detection problem. In thefirst (general) version, the prior probabilities in the detectionproblem vary as a function of thea priori pdf of the parameter.In the second version, the prior probabilities remain fixed andequally likely. Although the second version was shown to beweaker in theory, it is generally much easier to use in practice.


To demonstrate the straightforward application of the bound,the equally likely hypothesis version was evaluated for twoproblems in time-delay and bearing estimation. The resultingbounds had simple closed-form expressions which were goodfor all regions of operation. Expressions for the thresholdsseparating the regions were also derived, and simulation resultswere provided to show that the bounds closely predict theperformance of the MLE in all regions.

APPENDIX

PROPERTIES OFMONOTONIC ELLIPTICAL DISTRIBUTIONS

The vector is said to have an elliptical distributionwith parameters and if its density function has theform [35, p. 34]:

(A1)

for some function , where is a normalizing constantand is positive-definite. The argument of in (A1)is the equation of an ellipsoid centered at, whose shape isdefined by the matrix . We will be restricting attention tomonotonic elliptical distributions in which is a mono-tonic, nonincreasing function of . Monotonic ellipticaldistributions have the following properties.

Property A1:

(A2)

Proof: The region where is the regionwhere

(A3)

which simplifies to

(A4)

This is illustrated in Fig. 7 for a two-dimensional. Therefore,

(A.5)

(A.6)

(A.7)

(A.8)

Fig. 7. Elliptically contoured densities.

Property A2:

(A9)

Proof: When is multivariate Gaussian, is thecovariance matrix, and [20, pp. 98–99]

(A10)

The probability in (A10) is a monotonically decreasing func-tion of , therefore, it is maximized by the value ofwhich minimizes subject to . The solution is

(A11)

When is not Gaussian,does not have the simple form in (A10), and the optimumisnot necessarily the same as in (A11). However, the left-handside of (A9) can be lower-bounded by choosingto have thisform. This gives

(A12)

ACKNOWLEDGMENT

The authors wish to thank Prof. S. Shamai (Shitz) for nu-merous discussions throughout this work, and the anonymousreviewers whose comments greatly improved the presentation.

REFERENCES

[1] R. A. Fisher, “On the mathematical foundations of theoretical statistics,”Phil. Trans. Roy. Soc., vol. 222, p. 309, 1922.

[2] H. Cramer, Mathematical Methods of Statistics.Princeton, NJ: Prince-ton Univ. Press, 1946.

[3] C. R. Rao, “Information and accuracy attainable in the estimation ofstatistical parameters,”Bull. Calcutta Math. Soc., vol. 37, pp. 81–91,1945.


[4] E. W. Barankin, “Locally best unbiased estimates,”Ann. Math. Stat.,vol. 20, pp. 477–501, 1949.

[5] J. Ziv and M. Zakai, “Some lower bounds on signal parameter estima-tion,” IEEE Trans. Inform. Theory, vol. IT-15, no. 3, pp. 386–391, May1969.

[6] L. P. Seidman, “Performance limitations and error calculations forparameter estimation,”Proc. IEEE, vol. 58, pp. 644–652. May 1970.

[7] D. Chazan, M. Zakai, and J. Ziv, “Improved lower bounds on signalparameter estimation,”IEEE Trans. Inform. Theory, vol. IT-21, no. 1,pp. 90–93, Jan. 1975.

[8] S. Bellini and G. Tartara, “Bounds on error in signal parameter es-timation,” IEEE Trans. Commun., vol. COM-22, pp. 340–342, Mar.1974.

[9] A. J. Weiss, “Fundamental bounds in parameter estimation,” Ph.D.dissertation, Tel-Aviv Univ., Tel-Aviv, Israel, 1985.

[10] E. Weinstein and A. J. Weiss, “A general class of lower boundsin parameter estimation,”IEEE Trans. Inform. Theory, vol. 34, pp.338–342, Mar. 1988.

[11] E. L. Lehmann,Theory of Point Estimation. New York: Wiley, 1983.[12] A. Bhattacharyya, “On some analogues of the amount of information

and their use in statistical estimation,”Sankhya Indian J. Stat., vol. 8,pp. 1–14, 201–218, 315–328, 1946.

[13] J. M. Hammersley, “On estimating restricted parameters,”J. Roy. Stat.Soc. (B), vol. 12, pp. 192–240, 1950.

[14] D. G. Chapman and H. Robbins, “Minimum variance estimation with-out regularity assumptions,”Ann. Math. Stat., vol. 22, pp. 581–586,1951.

[15] D. A. S. Fraser and I. Guttman, “Bhattacharyya bounds without regu-larity assumptions,”Ann. Math. Stat., vol. 23, pp. 629–632, 1952.

[16] J. Kiefer, “On minimum variance estimators,”Ann. Math. Stat., vol. 23,pp. 627–629, 1952.

[17] J. S. Abel, “A bound on mean-square-estimate error,”IEEE Trans.Inform. Theory, vol. 39, no. 5, pp. 1675–1680, Sept. 1993.

[18] K. L. Bell, Y. Ephraim, and H. L. Van Trees, “Explicit Ziv-Zakai lowerbound for bearing estimation,”IEEE Trans. Signal Processing, vol. 44,pp. 2810–2824, Nov. 1996.

[19] E. Weinstein, “Relations between Belini–Tartara, Chazan–Zakai–Ziv,and Wax–Ziv lower bounds,”IEEE Trans. Inform. Theory, vol. 34, no.2, pp. 342–343, Mar. 1988.

[20] H. L. Van Trees,Detection, Estimation, and Modulation Theory, Part I.New York: Wiley, 1968.

[21] B. Z. Bobrovsky and M. Zakai, “A lower bound on the estimation error

for certain diffusion processes,”IEEE Trans. Inform. Theory, vol. IT-22,no. 1, pp. 45–52, Jan. 1976.

[22] B. Z. Bobrovsky, E. Mayer-Wolf, and M. Zakai, “Some classes of globalcramer-rao bounds,”Ann. Stat., vol. 15, pp. 1421–1438, 1987.

[23] P. N. Misra and H. W. Sorenson, “Parameter estimation in poissonprocesses,”IEEE Trans. Inform. Theory, vol. IT-21, no. 1, pp. 87–90,Jan. 1975.

[24] S. Bellini and G. Tartara, “Correction to ‘Bounds on error in signalparameter estimation,’”IEEE Trans. Commun., vol. COM-23, p. 486,Apr. 1975.

[25] S. Bellini, “An upper bound on the performance of pulse modulationsystems,”IEEE Trans. Commun., vol. COM-26, pp. 1352–1355, Sept.1978.

[26] A. J. Weiss and E. Weinstein, “Composite bound on the attainable mean-square error in passive time delay estimation from ambiguity pronesignals,” IEEE Trans. Inform. Theory, vol. IT-28, no. 6, pp. 977–979,Nov. 1982.

[27] , “Fundamental limitations in passive time delay estimation—PartI: Narrow-Band Systems,”IEEE Trans. Acoust. Speech, Signal Process-ing, vol. ASSP-31, pp. 472–486, Apr. 1983.

[28] E. Weinstein and A. J. Weiss, “Fundamental limitations in passive timedelay estimation—Part II: Wide-Band Systems,”IEEE Trans. ASSP, vol.ASSP-32, pp. 1064–1078, Oct. 1984.

[29] A. J. Weiss, “Bounds on time-delay estimation for monochromaticsignals,”IEEE Trans. Aerosp. Electron. Syst., vol. AES-23, pp. 798–808,Nov. 1987.

[30] S. C. White and N. C. Beaulieu, “On the application of the Cramer–Raoand detection theory bounds to mean square error of symbol timingrecovery,” IEEE Trans. Commun., vol. 40, pp. 1635–1643, Oct. 1992.

[31] K. L. Bell, Y. Ephraim, and H. L. Van Trees, “Comparison of theChazan–Zakai–Ziv, Weiss–Weinstein, and Cramer–Rao bounds for bear-ing estimation,” inProc. Conf. on Information Sciences and Systems(Baltimore, MD, 1993).

[32] M. Zakai and J. Ziv, “A generalization of the rate-distortion theory andapplication,” inInformation Theory New Trends and Open Problems, G.Longo, Ed. New York: Springer-Verlag 1975, pp. 87–123.

[33] E. Cinlar,Introduction to Stochastic Processes.Englewood Cliffs, NJ:Prentice-Hall, 1975.

[34] V. A. Kotel’nikov, The Theory of Optimum Noise Immunity.NewYork: McGraw-Hill, 1959.

[35] R. J. Muirhead,Aspects of Multivariate Statistical Theory.New York,:Wiley, 1982.

Documents

Extended Ziv–Zakai Lower Bound - AMiner · PDF fileThese bounds relate the MSE in the estimation problem to the probability of error in a ... a lower bound on the MSE can be obtained