MIMO outage capacity_Large deviations approach.pdf

RESEARCH PROJECT REPORT, MASTER SAR, 2013-2014 1

MIMO Outage Capacity: Large Deviations AnalysisMohamed Abouzrar, Graduate Student, Supelec

Abstract—We are considering a MIMO system underan asymptotic regime, for which, we are investigating theoutage and error probabilities. Our main focus will beon a research work [1], where the authors present a newapproach to calculate the probability density function andthe outage probability of the MIMO mutual information.This approach, namely Large Deviations, shows interestingresults, especially in non Gaussian tails, where otherapproaches fail to provide a good approximation of theoutage probability. We have simulating the main resultsand comparing them to those provided by the authorsof [1]. In a second time, we studied the decoding errorprobability from [2], of our system, the Gallager boundwas used to get a tight upper bound. The large deviationsapproach was introduced in the calculus of the errorprobability.

Keywords—Large deviations principal, MIMO outageprobability, Error probability, Gallager bound.

I. INTRODUCTION

THE present paper is a report of our researchproject. We will consider a MIMO system with

large number of transmitting and receiving antennas,such that the ratio stays finite. Our aim is to derive theoutage and error probabilities, starting from two works[1] and [2].

In the next section we will introduce some tools thatwe found it useful to manipulate the subject, namelyWishart’s matrix, Large Deviations Principal, MIMOoutage capacity and Gallager upper bound. In the twofollowing sections, we will present our summary results,especially the derivation of the outage probabilityand then using it to enhance an upper bound of errorprobability.

II. USEFUL TOOLS

A. Random matrices: Wishart’s Matrix

The study of random matrices is started with the workof the statician J. Wishart back to 1928, he was essen-tially interested on the behavior of covariance matriceswith i.i.d entries, his work provided an expression of the

The author was working under the supervision of Mr. R. Couillet,Telecom Departement, Supelec (Gif-sur-Yvette)

joint probability of the entries of such covariance matrixwhen the entries are identical CN(0, 1).

What about a formal definition of random matrix ? Letthe triple (Ω,F , P ) be the probability space so that Fis a sigma-algebra of subsets of Ω and P a probabilitymeasure on (Ω,F) , MatN (F) denote the space of N-by-N matrices with entries in F, i.g F = C, then arandom matrix XN is defined to be a measurable mapfrom (Ω,F) to MatN (F ).

The main interest of random matrices is in theireigenvalues, recall that eigenvalues of XN are the rootsof the characteristic polynomial P (z) = det(zIN−XN ),therefore the eigenvalues are functions of the entries ofXN . We will use essentially Hermitian matrices, as aconsequence of perturbation theory of normal matrices,the real eigenvalues λi(XN ) are continuous functionsof XN , i.e since XN is a random matrix then theeigenvalues are random variables. Thus, we will reducethe complexity of the problem from considering aO(N2) of random variables to just N random variables,namely the eigenvalues.

The first asymptotic considerations was developed bythe physician E. Wigner, the result is will known asWigner’s theorem. A Wigner matrix XN ∈ MatN (F)is any Hermitian matrix whose entries are independentand identically distributed, except for the Hermitianconstraints. Let LN = N−1

∑Ni=0 δλi(XN ) denote the

empirical measure of the eigenvalues of XN , Wigner’stheorem affirm that LN converges weakly, in probability,to the semicircle distribution σ defined as follows:

σ(x) =1

2π

√4− x21|x|≤2 (1)

Then, the theorem assert that for any continuousfunction f, with real support, and ∀ε > 0:

limN→∞

P(|〈LN , f〉 − 〈σ, f〉| > ε) = 0

Lets go back to the beginning of this section, andpresent some useful results related to Wishart matrix.Let Y be an M-by-N rectangular matrix with Gaussianentries over F, then XN = Y HY is a Wishart matrix,square covariance matrix. The joint distribution of thereal eigenvalues λi(XN ) is defined as follows, in casewhere F = C:


P (λi(XN )) ∝∏i

λM−Ni

∏i<k

(λi − λj)2e−∑Ni=1 λi (2)

A physical interpretation of this expression is knownas 2D Coulomb gas method:

P (λi(XN )) ∝ e−E(λi(XN )) (3)

such that E is the energy of the system seen as particlesconfined to a line each with a position defined by aneigenvalue:

E(λi(XN )) =

N∑i=1

(λi−(M−N) log λi)−∑i 6=j

log |λi − λj |

(4)Therefore

P[λi ≤ t . . . λN ≤ t] = P[λmax ≤ t] =ZN (t)

ZN (∞)(5)

Where

ZN (t) =

∫ t

−∞. . .

∫ t

−∞

∏i

dλi exp[−E(λi(XN ))]

A result, due to Marcenko and Pastur back to 1967,concerning the asymptotic behavior of the empirical mea-sure of the eigenvalues assert the convergence weakly toMarcenko-Pastur distribution given by:

fMP (x) =1

2πx

√(x+ − x)(x− x−) (6)

s.t:

x± = (1±√M

N)

B. Large Deviations Principal

The theory of Large Deviations is concerned with thestudy of the probabilities of very rare events, especiallywith the rates at which this probabilities decay as anatural parameter in the problem varies. If we havesequence of probability distributions Pn on (X ,B), acomplete separable metric space ( Polish space) X withits Borel sigma-field B, we say that it satisfies a LargeDeviation Principle (LDP) with rate I(x) if the followingproperties hold:• The function I(x) ≥ 0 is lower semicontinuous

and the level sets Kl = x : I(x) ≤ l arecompact for any finite l.

• For any closed set C ⊂ X we have

limn→∞

sup1

nlogPn[C] ≤ −infx∈CI(x) (7)

• For any open set G ⊂ X we have

limn→∞

inf1

nlogPn[G] ≥ −infx∈GI(x) (8)

A consequence of the definition is the followingtheorem,

Theorem 1: Let Pn satisfy LDP on X with rate I andF : X → R a bounded continuous function. Then

limn→∞

1

nlog

∫enF (x)dPn = supx[F (x)− I(x)] (9)

An other consequence, very important for our case,concerning the probability distribution Qn defined by

Qn(A) =

∫A enF (x)dPn∫X enF (x)dPn

(10)

for Borel subsets A ⊂ X . Then Qn satisfies an LDP onX as well with the new rate function J given by

supx∈X [F (x)− I(x)]− [F (x)− I(x)] (11)

This theorem was used in [1], as we will see hereafter,where was mentioned to be Varadhan’s lemma.

C. MIMO outage capacity

An other concept to be clarified is the channel capacityfor single-user communications, we focus in the Shannontheoretic sense. For time-invariant channel, the capacityis defined as the maximum mutual information betweenthe channel input and output, it was proved by Shannon’stheorem that the capacity is the maximum data rate thatcan be transmitted over the channel at arbitrary smallerror probability.

When the channel is time-variant, the definition of thecapacity depends on the channel knowledge at both sidesof the channel. If the Channel State Information (CSI), i.ethe instantaneous channel gains, are perfectly known atboth sides of the channel, then the ergodic capacity is de-fined as the maximum mutual information averaged overall channel states. When the CSI is known only at thereceiver, the channel coefficients are typically assumedto be jointly Gaussian, so the channel is specified by thefirst two moments, i.e the mean and the covariance. Inthis case the ergodic capacity defines the rate that can beachieved, however the transmitter can send data at a ratethat cannot be supported by all channel states, thereforean outage is declared, by the receiver, in poor channelstates.

On the other hand, concerning the non-ergodicchannel, the case in which the channel is chosenarbitrarily in the beginning of the communication and


remain constant during the uses of the channel, itwas claimed by Telatar [4] that the maximum mutualinformation is in general not equal to the channelcapacity and the Shannon capacity is zero. In this case,we talk about a tradeoff between outage probability andsupportable rate.

Consider a transmitter with M transmit antennas anda receiver with N antennas, the channel can be presentedby N-by-M matrix H. The received signal y is equal toy = Hx + z , where x is the transmitted signal vectorand z is AWGN vector, normalized so that its covariancematrix is the identity matrix. In the following we givethe expression of the channel capacity, from [5], in eachcase that was discussed above:• When channel is constant and known perfectly at

the transmitted and the receiver, where Q is theinput covariance matrix:

C = maxQ:Tr(Q)=P

log∣∣IN + HQHH

∣∣ (12)

• For fading MIMO channel with perfect CSI at boththe transmitter and the receiver:

C = EH

[max

Q:Tr(Q)=Plog∣∣IN + HQHH

∣∣] (13)

• For fading MIMO channel with perfect CSI at thereceiver only:

C = maxQ:Tr(Q)=P

EH

[log∣∣IN + HQHH

∣∣] (14)

• And for the non-ergodic channel, the outage prob-ability is given by:

Pout(R,P ) = infQ≥0:Tr(Q)≤P

P[log∣∣IN + HQHH

∣∣ < R]

(15)

D. Gallager Upper Bound

In most of cases we can not compute directly theprobability, that is why the bounds are an important toolsto derive a closer value, upper or lower bound, usuallyupper bounds are more interesting. We will see the trivialbound, known as Union Bound, then we will get tothe Gllager bound as a generalization of Bhattacharyyabound, supposing a random coding and ML decoding.

Union bounds are based on the trivial inequality whichstates that the probability of a union of events is upperbounded by the sum of the probabilities of the individualevents. Let a set of signals s1, s2, . . . , sN, the UnionBound on the probability of error given the kth was sentis,

P (E/sk) ≤∑i 6=k

P (Ei,k/sk) (16)

for orthogonal signals, |s1 − sk| =√

2Es , i 6= k,

P (E) ≤ (N − 1)Q(√Es/N0)

It turns out that (15) turns to be an equality if these eventsare disjoint, otherwise, it could be very loose bound. Thelooseness of the Union Bound comes from the fact thatintersections of half-spaces related to codewords otherthan the transmitted one, are counted more than once.

We shall now present the Bhattacharyya Bound, herealso we assume that sk was sent, then

Ei,k = r : p(r/si) > p(r/sk)

we define the indicator function,

φ(r) =

1 , r ∈ Ei,k0 , otherwise

Thus,

P (Ei,k/sk) =

∫Ei,k

p(r/sk)dr (17)

=

∫allr

p(r/sk)dr (18)

using the fact that,

∀r ∈ Ei,k, φ(r) ≤ 1 ≤

√p(r/si)

p(r/sk)

we get the Bhattacharyya Bound,

P (E/sk) ≤∫r

∑i 6=k

√p(r/si)p(r/sk)dr

A generalization of this bound could be deduced byusing the fact that,

Ek ⊂ Ek =

r :∑i 6=k

[p(r/si)

p(r/sk)

] 1

1+µ

hence we get the Gallager Bound,

P (E/sk) ≤∫r(p(r/sk))

1

1+µ

∑i 6=k

((p(r/si))1

1+µ

µ dr ,with µ ≥ 0

(19)Assuming a random coding, we shall obtain the Gal-

lager upper bound of the average error probability:

¯P (E) ≤ e−N(E0(µ,P )−µR)

where

E0(µ, P ) = − logEH

∫y

[∫xP (x)p(y/x,H)

1

1+µdx

]1+µdy


III. OUTAGE PROBABILITY: LD APPROACH

A. Summary of The Approach

The authors of [1] are looking for an analytic expres-sion for the probability distribution function and outageprobability of the MIMO mutual information, by usingLarge Deviations theory techniques.

The previous approaches have shown that mutualinformation, tends asymptotically to a Gaussian behaviorfor large number of receiver antennas (N). This approx-imation is acceptable around the mean of the mutualinformation, i.e ergodic capacity, however it fails tocapture the tails of the distribution especially when therate drop bellow the half of the ergodic capacity. Thereis much variants of Gaussian approximation, that can besummarized in two methods, the large N fixed SNR onthe one hand, and the large SNR fixed N limit on theother hand, both failing to produce quantitative results forthe outage probability outside their respective regions.

Since the tails of the mutual information PDF arevery important, because they match the rate region ofvery low outage probability, where one would want tooperate. Then, one still needs an other approaches thatgive more relevant results on the mutual informationPDF for arbitrary SNR and rate. That is the goal of [1]by using a Large Deviations analysis.

We will focus on i.i.d Gaussian noise and input, theMIMO channel model:

y = Hx + z

the mutual information for a given channel matrix H,

IN = log |I + ρHHH| (20)

where ρ is the signal to noise ratio, H is a MxN whoseelements are independent N (0, 1/N), such that the Ntransmitting and M receiving antennas are large, and β =N/M stay finite. We shall write (20) in terms of λk theeigenvalues of the Wishart matrix HHH,

IN =

N∑k=1

log(1 + ρλk) (21)

We shall now use the distribution given at (2), but beforethat, we will use an asymptotic expression of the energyterm (4),

E(p) =

∫xp(x)dx − (β − 1)

∫p(x)log(x)dx(22)

−∫∫

p(x)p(y) log |x− y|dxdy (23)

Then, the probability density function of the normal-ized mutual function IN/N can be written as (5),

PN (r) =ZN (r)

ZN(24)

this expression corresponds to (10), by applying theVaradhan’s lemma (11), we get

limN→∞

1

N2logPN (r) = E0 − E1(r) (25)

whereE0 = inf

p∈XE(p) (26)

E1(r) = infp∈Xr

E(p) (27)

and p is a probability density, such that

X = p : the expectation exists and∃ε > 0,

∫|p(x)|1+ε <∞

Xr = p ∈ X :

∫p(x) log(1 + ρx)dx = r

Now, to get the expression of (27), it suffices to resolvethe following optimization problem,

Min E(p)Subject to p ∈ Xr

(28)

Than we can deduce (26) by simply vanishing theLagrangian multiplier corresponding to the constraint,∫

p(x) log(1 + ρx)dx = r

The derivation of the solution is given in details at theappendix C in [1].

The evaluation of E0 seems to be relatively easier,because it takes into account just one constraint,

E0 =∆2

32+a

2− log ∆− β − 1

2log(a∆)

− ∆

2

[G(

0,a

∆

)+β − 1

2G( a

∆,a

∆

)] (29)

where ∆ , b − a , the function G(x, y) =1π

∫ 10

√(t(t − 1)) log(t+x)t+x dt which is computed by [6],

and [a, b] is the support of the distribution p. Whenβ = 1, E0 = 3/2.

The evaluation of E1(r) is relatively difficult, this timewe have to take into account two constraint, hence theresult will depend on β. I will focus on the case whereβ = 1, because its the case that we have simulated. Sincethe rate is non-negative or null, a ≥ 0, then we shouldsplit this case itself into two cases :


• a = 0,

p(x) =

√b− x

2π(1 + ρx)√x

(1 + ρx− kρ√

1 + ρx

)(30)

such that:k =

b/2− 2

1− 1/√

1 + ρx

• a > 0,

p(x) =ρ

2π

√(b− x)(x− a)

1 + ρx(31)

where a and b are the solution of the equation:

x2 − (2k+ 4− 2

ρ)x+ k2 − 1

ρ2− 2k + 4

ρ+

2

ρ= 0

which leads to, a = (√k + 1 − 1)2 − ρ−1 and

b = (√k + 1 + 1)2 − ρ−1

B. The evaluation of the outage probability

In this section, we will derive the expression of theoutage probability defined as follows,

Pout(r) = P(IN/N < r) (32)

To do so, we will use the probability density of thenormalized mutual information given by (25).

Firstly, we compute the normalized factor of PN , itsuffices to evaluate it at r ∼ rerg, rerg is the ergodicaverage of IN/N , where the Guassian approximation isvalid, which leads to

PN (r) ≈ N√2πverg

e−N2(E1(r)−E0) (33)

To obtain Pout, we will need this following lemma,

Lemma 1 (Watson’s lemma): Suppose f(t) = O(eat)as t → ∞ and in some neighborhood of t = 0 , f(t)can be expanded as

f(t) = tα

[n∑k=0

aktk +Rn+1(t)

], 0 < t < τ, α > −1

where |Rn+1(t)| < Atn+1 for 0 < t < τ . Then

F (s) =

∫ ∞0

e−stf(t)dt

has the asymptotic expansion

F (s) ∼n∑k=0

akΓ(α+ k + 1)

sα+k+1+O(s−(α+k+1)), s→∞

Therefore:

• When r < rerg,

Pout(r) ≈Q

(N |E′

1(r)|√E′′

1 (r)

)e−N2

(E1(r)−E0−

E′21 (r)

2E′′1

(r)

)√E′′1 (r)verg

(34)• and when r > rerg

Pout(r) ≈ 1−Q

(N |E′

1(r)|√E′′

1 (r)

)e−N2

(E1(r)−E0−

E′21 (r)

2E′′1

(r)

)√E′′1 (r)verg

(35)Proof:

C. Simulations and Discussion

The trivial approximation of the outage probability isthe Gaussian approximation which is valid in the regionwhere r ≈ rerg , such that rerg is the ergodic average ofnormalized mutual information,

rerg = (1− β) log u+ β log(1 + u) + u−1 − 1

The ergodic variance is given by,

verg = − log

(1− (1− u)2

βu2

)where,

u =1

2(1 + +ρ(β − 1) +

√(1 + ρ(β − 1))2 + 4ρ

Then we obtain,

PGout = 1−Q(Nr − rerg√verg

)(36)

To simulate the outage probability given by (34) and(35) for the case where β = 1 , we have derived all theequations needed, and they are presented at Appendix A.

Some of the simulation results are given bythe following figures Fig. 1, Fig. ??, and Fig. 3.We have two remarks, first our results, especiallyfor PLDoutcurves, differ from those obtained in [1].Nevertheless, our results do not contradict the calculationapproach, because we wait from the LD approach togive us an accurate expression of the outage probabilitynear the small errors, which is showed in our simulations.

We can say that the authors of [1] are successfullyobtained a good results even for higher error probability.However, we are far from an exact analytical expressionof the outage probability, because a rigorous treatmentof the large-deviations region was not pursued.


IV. DECODING ERROR PROBABILITY

We will use the outage probability (34) and (35) andthe Gallager bound (19), to derive an upper bound of thedecoding error, assuming a random coding.

[2] claim that a decoding error occurs if either thechannel matrix is atypically ill-conditioned, which leadsto outage, noise is atypically large, or some codewordsare atypically close to each other.

We shall write,

Pe(r) = Poutage(r)Pe/outage(r) + (1− Poutage(r))Pe/no outage(r)≤ Poutage(r) + Pe/nooutage(r)

(37)

Let now derive an upper bound of Pe/nooutage(r) byusing the Gallager bound (19) and maximizing over theparameter µ,

Pe/nooutage(r) ≤ exp(−NE(r)) (38)

whereE(r) = max

µ∈[0,1]E0(µ)− µr (39)

and,

E0(r) = − logEH det

(I +

ρ

1 + µHHH

)−µ= − logEH

[N∏k=1

(1 +

ρ

1 + µλk

)−µ] (40)

Fig. 1: Outage probability: MIMO 2x2 , ρ = −10dB

Using Jensen’s inequality,

Fig. 2: MIMO 2x2 , ρ = 10dB

Fig. 3: MIMO 3x3 , ρ = 10dB

E0(r) ≤ EH

− log

[N∏k=1

(1 +

ρ

1 + µλk

)−µ]

= µEH

N∑k=1

log

(1 +

ρ

1 + µλk

)= µEH(I ′N )

Where I ′N is exactly a mutual information given asignal-to-noise ratio ρ

1+µ .

In the following, we assume

E0(r) ≈ µEH(I ′N )


this means simply that we are interested in some asymp-totic region where the mutual information behaves as anaffine function.

Therefore,

Pe/no outage(r) ≤ minµ∈[0,1]

e−EH(µNI′N )+µNr

≤ minµ∈[0,1]

EH

[e−µNI

′N

]eµNr

≤ minµ∈[0,1]

EI′N[e−µNI

′N

]eµNr

≤ minµ∈[0,1]

∫e−µNI

′NdPN e

µNr

eventually it still need to be expansed and thenchecked.

V. CONCLUSION

To sum up, we have considered a MIMO system in anasymptotic regime and we attempt to obtain the outageand error probabilities. This project is split into two mainfocuses, first reviewing the work [1] and simulating someof its results, and second starting from a work on errorprobability [2] in order to use the results of [1] and theGallager bound to get some enhancement. From the firstwork, we obtained simulation result that differ from theones given by [1], but our simulations show that theLD approximation works as it should be at small errors.Actually, the second one still need some works to obtainan analytic expression of tight upper bound of decodingerror.

APPENDIX ASIMULATION EQUATIONS

We will present the equations used to simulate theoutage probability (34) for the case when β = 1 ,where [a, b] is the rate support and k is the Lagrangianmultiplier of the rate constraint:

When a = 0,∂k

∂r=

1

B

∂k

∂r=

1

AB

s.t

A =2(ρb+ 1)

3

2 − 3ρb+ 4ρ− 2

4√

1 + ρb(2 + ρb− 2√

1 + ρb

B = 2 log1 +√

1 + ρb

2− 1

2log(1 + ρb)

where k and b are the solution of the followingequation, for a given r:

ρb3 + (4ρk2 − 4k + 16ρ− 8 + 16ρk)bb

+ (1− 4kρ− 8ρ)b2 + 16k + 16 = 1

The first two derivatives of E1(r) are as follows:

∂E1

∂r=k

2+

1

B

[r

2− b

8− log

1 +√

1 + ρb

2

]+

1

AB

[1

8ρ+

3b

16− 1

b− k

8− kρ

2(1 + ρb+√

1 + ρb)

]

and

∂2E1

∂r2=

1

B− 1

AB

[1

4+

ρ

2(1 + ρb+√

1 + ρb)+

ρ

2(1 + ρb)

]

+1

AB

[3

16+

1

b2+

kρ2(2√

1 + ρb+ 1)

2(ρb+ 1)3

2 (1 +√

1 + ρb)2

]

− C

AB3

[r

2− b

8− log

1 +√

1 + ρb

2

]−AC +BD

(AB)3

[1

8ρ+

3b

16− 1

b

k

8− kρ

2(1 + ρb+√

1 + ρb)

]

Where,

C =ρ2b

2(1 + ρb)(1 +√

1 + ρb)2

and

D =−32(1 +

√1 + ρb)− ρb(b(b+ 12) + 32

√1 + ρb+ 48

8ρb3ρ(1 + ρb)3

2

When a > 0:

a = (√k + 1− 1)2 − ρ−1 ; b = (

√k + 1 + 1)2 − ρ−1

and k is obtained for a given r.

∂k

∂r=

1

log(1 + 1

k

)


Then

∂E1

∂r=r − log ρ+ 1

2

2 log(1 + 1

k

)+k − 1

2

+log k

log(1 + 1

k

)and,

∂2E1

∂r2=

1

log(1 + 1

k

)+r − log ρ+ 1

2

(k2 + k) log3(1 + 1

k

)+

1

2k log2(1 + 1

k

)+log k

(k2 + k) log3(1 + 1

k

)

REFERENCES

[1] P. Kazakopoulos et al, Living at the Edge: A Large DeviationsApproach to the Outage MIMO capacity , Information Theory,IEEE Transactions, 2011.

[2] S. Ray et al, On Error Probability for Wideband MIMO Chan-nels , Conference on Information Sciences and Systems, TheJohns Hopkins University, March 16-18, 2005.

[3] I. E. Telatar, Capacity of multi-antenna Gaussian channels ,European Transactions on Telecommunications, Vol. 10, No. 6,pp. 585-595, 1999.

[4] L. Zheng and D. N. C. Tse, Diversity and multiplexing: Afundamental tradeoff in multiple antenna channels, InformationTheory, IEEE Transactions, 2003.

[5] A. Goldsmith et al, Capacity Limits of MIMO channels, IEEEJournal, 2003.

[6] Y. Chen and S. M. Manning, Some eigenvalues distributionfunctions of the Laguerre ensemble, J. Phys. A: Math. Gen, vol.29, pp. 7561-7579, 1996.

[7] Greg W. Anderson et al, An Introduction to Random Matrices,Cambridge University Press, 2010.

[8] Romain Couillet and Merouane Debbah, Random Matrix Meth-ods for Wireless Communications, Cambridge University Press,2011.

[9] A. Dembo and O. Zeitouni, Large Deviations Techniques andApplications, Springer, 2ed 2010: .

[10] S. R. S. Varadhan, Large Deviations, 2010,www.math.nyu.edu/faculty/varadhan.

[11] W. A. Pearlman , The Gallager and Bhattacharyya Bounds,Orthogonal Signal and Random Coding Bounds, 2005,http://www.cipr.rpi.edu/ pearlman/.

Documents

MIMO outage capacity_Large deviations approach.pdf