The Capacity of the Relay Channel - isr.umd.edu · Summary Solved an problem posed by Cover and...

Cover’s Open Problem:“The Capacity of the Relay Channel”

Ayfer Ozgur

Stanford University

Advanced Networks Colloquia SeriesUniversity of Maryland, March 2017

Joint work with Xiugang Wu and Leighton Pate Barnes.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 1 / 26

Father of the Information Age

Claude Shannon (1916-2001)

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 2 / 26

The Bell System Technical JournalVol. XXVII J Illy, 1948 No.3

A Mathematical Theory of CommunicationBy c. E. SHANNON

IXTRODUCTION

T HE recent development of various methods of modulation such as reMand PPM which exchange bandwidth for signal-to-noise ratio has in-

tensified the interest in a general theory of communication. A basis forsuch a theory is contained in the important papers of Nyquist! and Hartley"on this subject. In the present paper we will extend the theory to include anumber of new factors, in particular the effect of noise in the channel, andthe savings possible due to the sta tistiral structure of the original messageand due to the nature of the final destination of the information.

The fundamental problem of communication is that of reproducing atone point either exactly or approximately a message selected at anotherpoint. Frequently the messages have meaning; that is they refer to or arecorrelated according to some system with certain physical or conceptualentities. These semantic aspects of communication are irrelevant to theengineering problem. The significant aspect is that the actual message isone selected from a set of possible messages. The system must be designedto operate for each possible selection, not just the one which will actuallybe chosen since this is unknown at the time of design.

If the number of messages in the set is finite then this number or anymonotonic function of this number can be regarded as a measure of the in-formation produced when one message is chosen from the set, all choicesbeing equally likely. As was pointed out by Hartley the most naturalchoice is the logarithmic function. Although this definition must be gen-eralized considerably when we consider the influence of the statistics of themessage and when we have a continuous range of messages, we will in allcases use an essentially logarithmic measure.

The logarithmic measure is more convenient for various reasons:1. It is practically more useful. Parameters of engineering importance

1 Nyquist, H., "Certain Factors Affecting Telegraph Speed," Belt System Tectmical J OUT-

nal, April 1924, p, 324; "Certain Topics in Telegraph Transmission Theory," A. I. E. E.TI aIlS., v. 47, April 1928, p. 617.

2 Hartley. R. V. L.. "Transmission oi Information.' Belt System Technical Journal, July1928, p. .'US.

379

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 3 / 26

“A method is developed for representing any communication systemgeometrically...”

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 4 / 26

AWGN Channel

Transmitter Receiver

P

Capacity

C = log

(1 +

P

N

)

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 5 / 26

Converse: Sphere Packing

pnN

pn(P + N)

pnN

pnN

Y sphere

0

Xn(1)

Xn(2)

Xn(3)

noise sphere

# of X n ≤

∣∣∣Sphere(√

n(P + N))∣∣∣

∣∣∣Sphere(√

nN)∣∣∣

.=

2n2log 2πe(P+N)

2n2log 2πeN

= 2n2log(1+ P

N )

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 6 / 26

Achievability: Geometric Random Coding

Pr(∃ false X n) ≤ |Lens|∣∣∣Sphere(√

nP)∣∣∣× 2nR

.=

2n2log 2πe PN

P+N

2n2log 2πeP

× 2nR

= 2−n2log(1+ P

N ) × 2nR

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 7 / 26

The story goes...

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 8 / 26

Cover’s Open Problem

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 9 / 26

Gaussian case

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

C (∞) =1

2log

(1 +

2P

N

)

Achievability: C ∗0 =∞.

Cutset-bound (Cover and El Gamal’79):

C ∗0 ≥1

2log

(1 +

2P

N

)− 1

2log

(1 +

P

N

).

Potentially, C ∗0 → 0 as P/N → 0.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 10 / 26

Main Result

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

Theorem

C ∗0 =∞

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 11 / 26

Upper Bound on the Capacity

0 0.1 0.2 0.3 0.7 0.8 0.9 12.5

2.6

2.7

2.8

2.9

3

3.1SNR = 15 dB

Cut-set boundC-FOld boundNew bound

0.4 0.5 0.6 C0 (bit/channel use)

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 12 / 26

Cutset Bound

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

nR ≤ I (X n;Y n, In) + nεn

= I (X n;Y n) + I (X n; In|Y n) + nεn

= I (X n;Y n) + H(In|Y n)︸ ︷︷ ︸≤nC0

−H(In|Y n,X n)︸ ︷︷ ︸≥0

+nεn

≤ n(I (X ;Y ) + C0 + εn)

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26

Cutset Bound

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

nR ≤ I (X n;Y n, In) + nεn

= I (X n;Y n) + I (X n; In|Y n) + nεn

= I (X n;Y n) + H(In|Y n)︸ ︷︷ ︸≤nC0

−H(In|X n)︸ ︷︷ ︸≥0

+nεn

≤ n(I (X ;Y ) + C0 + εn)

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 13 / 26

pnN

Xn

Typical set of Zn/Y n

✓ In-th bin

Zn

Y n

If H(In|X n) = 0, then H(In|Y n) = 0.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

pnN

Xn

Typical set of Zn/Y n

✓ In-th bin

Zn

Y n

If H(In|X n) = 0,

then H(In|Y n) = 0.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

pnN

Xn

Typical set of Zn/Y n

✓ In-th bin

Zn

Y n

If H(In|X n) = 0, then H(In|Y n) = 0.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 14 / 26

In = f(Zn)

Source

Relay

Destination

W1 ⇠ N (0, N)

W2 ⇠ N (0, N)

Xn Y n

Zn

C0

R ≤ I (X n;Y n) + H(In|Y n)︸ ︷︷ ︸≤?

− H(In|X n)︸ ︷︷ ︸=−n log sin θn

+nεn

Goal:

In = f (Zn)− Zn − X n

︸ ︷︷ ︸H(In|X n)=−n log sin θn

−Y n

︸ ︷︷ ︸H(In|Y n)≤?

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 15 / 26

H(In|X n) 6= 0

Multiple bins

Xn

pnN

Typical set of Zn

# of bins =? P(each bin) =?

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 16 / 26

From n- to nB- Dimensional Space

X,Y,Z, I : B-length i.i.d. from {(X n(b),Y n(b),Zn(b), In(b))}Bb=1.

If H(In|X n) = −n log sin θn, then for any typical (x, i)

p(i|x).

= 2nB log sin θn ,

P(Z ∈ A(i)|x).

= 2nB log sin θn

pnBN

x

Typical set of Z/Y

Pr.= 2nB log sin ✓n

ith bin Ax(i)

|Ax(i)| .= 2nB( 1

2 log 2⇡eN sin2 ✓n)

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 17 / 26

Isoperimetric Inequalities

Isoperimetric Inequality in the Plane (Steiner 1838)

Among all closed curves in the plane with a given enclosed area, thecircle has the smallest perimeter.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 18 / 26

Isoperimetric Inequalities

Isoperimetric Inequality on the Sphere (Levy 1919)

Among all sets on the sphere with a given volume, the spherical caphas the smallest boundary, or the smallest volume of ω-neighborhoodfor any ω > 0.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 19 / 26

Blowing-up Lemma

|Ax(i)| .= 2nB( 1

2 log 2⇡eN sin2 ✓n)

Isoperimetric Inequality on the Sphere + Measure Concentration:

P(Z ∈ blow-up of Ax(i)|x) ≈ 1.

⇓P(Y ∈ blow-up of Ax(i)|x) ≈ 1.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 20 / 26

Geometry of Typical Sets

n-dimensional space:Almost all (X n,Y n,Zn, In)

pnN

XnY n

Zn ! In

nB-dimensional space:Almost all (x, y, i)

pnBN

z ! i

xy

⇡

2� ✓n

Information Inequality: (Wu and Ozgur, 2015)

H(In|Y n) ≤ n(2 log sin θn +√

2 log sin θn ln 2 log e).

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 21 / 26

A new approach

Ax(i)

Y

Ax(i)

Y

Control the intersection of a sphere drawn around a randomly chosen Yand Ax(i).

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 22 / 26

Easy if Ax(i) is a spherical cap

pnBN

z0

y0x

✓n

Cap(z0, ✓n)

Cap(y0,!n)

Cap(z0, ✓n) \ Cap(y0,!n)

!n

|Cap(z0, θn) ∩ Cap(Y, ωn)| .= 2nB( 12log 2πeN(sin2θn−cos2 ωn))

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 23 / 26

Strengthening of the Isoperimetric Inequality

Strengthening of the Isoperimetric Inequality:

Among all sets on the sphere with a given volume, the spherical cap hasminimal intersection volume at distance ω for almost all points on the spherefor any ω > π/2− θ.

Proof: builds on the Riesz rearrangement inequality.

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 24 / 26

A Packing Argument

Given Y,# of I ≤

∣∣∣Sphere(Y,√nBN4sin2 ωn

2

)∣∣∣

2nB( 12log 2πeN(sin2θn−cos2 ωn))

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 25 / 26

Summary

Solved an problem posed by Cover and named “The Capacity ofRelay Channel” in Open Problems in Communication andComputation, Springer-Verlag, 1987.

Developed a converse technique that significantly deviates fromstandard converse techniques based on single-letterization and hassome new ingredients:

I TypicalityI Measure ConcentrationI Isoperimetric Inequality

Ayfer Ozgur (Stanford) “The Capacity of the Relay Channel” March’17 26 / 26