10/11/2016

In nµ(× , TuesdayIdea :

xn Size of the typical set : 2

to

ftp.yy#.i.if" "t" " " points

sina.ba znm" *

=

# of bells = ZNIHIXYcovering

"

Strong typical set :

Tl={xn : 1 # lxlxnl - Rail { EnM¥ }

Properties of Strong typical set :

1. If X

"

is ii. d. ~P× then P[El" ]→1

2. Tenet '

!

3. If ( xn, ynletemlx,

'll thenXNETEMCX ) and ynete "lY)

⇒ Th"lx , 'll e AEYX,

H

proof of 3. Given III x. ylxn,yn) - Pxylxy )|{ EPYH.IQ

ltlxlxntpxaltyntlxiylxniynl-

yEP×y(x.g) | { III( x. ylxn,y" ) - Pxylxy ) |

÷epxhtia.ge#a=9IixFr4. If MET

"

n+¥yglxi)E[t. e) Epxglx ',

HHERYHD ( same proof as 2.

)

5' IEMI e [ 4- e) IN " " '

,znl " '× ' - e) ]

1 T ↳ inherits ( use 2. )

/ large enough , ↳ same as weak typialset counterpart .

combine properly 1 and 2

.

6. It H,Y " ) i. i. d. ~ BR, ,

Pflxn, yyete

' " ] e fltgzfttxixttsel,

z.nl#xnt3el ]T nlayeenogh

Uk 1,

2 and 5.

7. Conditionally typical set

Te "lYH={ yn :L xn.ynk.TL" }

↳

7. a.) It Y~lI,Pyx

,⇒ ,

and xne I"

p[ yne I "( Ylxn)] → 1 as n→ x FE '

> e

(This is the main reason why we switched to Jd

"

simplex of distributions (Fath's')

. • = B' It.it#ritcx1xy

(Type class )) ↳weak typical set

strong typical .

ypeclass : Tp

" '. { xn : Icxlxnj = Pan }

Both Tel"

and AY'

are unions of type classes.

Proof of 7.a.

*"¥YxYxn"T=ki÷xl¥⇒H{ lxi,Yit

1 ( Yi

=y)

→ Ep,,×⇒H Yi 8) HEY

. use lliixi * H

= Pyixlylxl

⇒ p[tYy×Y!YI > Pyndyniluet] - 0 Veto

⇒ P[tlxylxnyn) 3 Pylxly 'D Pxlxl ( it E) ( they, ) ] → 0

Let E be smallenough

st . ( tie ) (ttffx,)< ttEµ÷x

,

7. b. ' '

ITEM

(

YlxnHIefxejznlHMHt24.z-nlNHH-24jtnlayeenoyhPnofnoteilflxn.ynleIHCYxnyjPyyxngn1xyefz-nHMxty4znHMxt2dYPxnynCxn_yygz-nHCXiYlzyPxnixnYktpxnlxnznlE47.c.1fxnETmandYni.i.d.nPylPIYnEEMCy1xnDef@yznlItxiDt4ePznlIlxiyt4eYTnlageen.gh

Tholds forAE'

as well.

7. c .* Given Pxyz,

If * ,ynHI"

and

ZwiIIPzilyi.glPfZneTeMlz1xn.ynjTEL@eyznlIHgiHHt4dz.nCIHIHH-49]

Note : If X . Y - Z"

aetoyn

IPIEEEYYZIX;ynD→1

R . D theorem ( converse )

Suppose Fn,

f. gat role R sit

.E[d( x

" ,In) ]fD ⇒ Xn - M - Yin MEER ]

NR ? Mm )

7I( xn ; In) D. PI.

.

and

:M¥eI(xinnlxiyd

IILX; ; Exit )

in 3¥,

I( × , ;fn ) because XIIXI"

↳3¥

,

Ilxii.

Let Tnunif { 1, ... ,n

} \I. Ilxiixil

THYME )=nIlx

, ;xIH )

3 NICK.

; xnt )/ ×t±T " Kay

.

r.

Lastly ,

define X= Xt,

i. It,

notice X~B

also" Etdlx ,x7]=E[ Eu

,

,n×nx¥(×' ID ]= ELKE

,HXYIY ]

{ D

113/2016

Thursday

Let IR,

D) be in the interior :

""

¥l÷* -

dlx ,xy]

⇒ Fpxyx st.

Ilxix )< R

ELHKIDCD

Codebook : Derive RYKKTZPXKI Pxyxlilx )

C = { Mm) }m2? where In (ml are ii. d. ~ Pxn Fm and independent .

Encoder : choose E > 0,

find

anym sit

,

( XIXTMDEI"Transmit m (eomnlgr

HIM !)↳

observed.

If error,

chooseany

m .

↳complexity is in encoder ( of Channel

codingthm pmef ,

where complexity is in deader)

Decoder : Reconstruct XTM )