-
8/13/2019 Tutorial 1_EEL710
1/3
EEL710_2013 (Prof. R. Bose) TUTORIAL 1 07/08/2013
# Date & time of submission: 23/08/2013 (1:00 AM).#
Solutions to practice problems are not to be submitted but they
cover some important topics.
Problem 1: (Inequalities).Let X, Y, !" # $e %o&!t r!"o'
r&$es. Proe t*e foo+&!&!e-&t&es !" f&!"
o!"&t&o!s for e-&t.
() (X, Y #) (X #).
($) I (X,Y4 #) I (X 4 #).() (X,Y,#) 5 (X,Y) 6 (X,#) 5 (X).
(") I (X4#Y) I (#4 YX) 5 I (#4 Y) I (X4 #).
As ! 99&t&o! of so'e of t*e &!e-&t&es of
&!for't&o! t*eor, o!s&"er ! ""&t&e
!o&se
f"&! *!!e Y : X; #, +*ere # &s ""&t&e !o&se,
; &s r!"o' r&$e re9rese!t&!f"&!, !" # !" ; re
&!"e9e!"e!t of e* ot*er !" of X. Yo ! re t*t t*e
!'$ers 1 !" =1 se" +&t* t*e 9ro$$&&t&esPX(+1)
:PX(=1) : 1/2. T*e ot9ty &s t*e s'
of t*e &!9tx !" ! &!"e9e!"e!t !o&se r!"o' r&$e #
+&t* t*e 9ro$$&&t "e!s&t pZ(z) :1/> for 2z2
!"pZ(z) : 0 ese+*ere. I! ot*er +or"s, t*e o!"&t&o!
9ro$$&&t "e!s&t of
y o!"&t&o!e" o!x &s &e! $PY|X (y|x) : 1/> for
=2 6 =? 6 2 !"PY|X (y x) : 0 ese+*ere.
() @&!" !" s
-
8/13/2019 Tutorial 1_EEL710
2/3
EEL710_2013 (Prof. R. Bose) TUTORIAL 1 07/08/2013
(a) @&!" t*e 9&t of t*&s *!!e &f Z1!" Z2re
%o&!t !or' +&t* or&!e 'tr&?
(b) H*t &s t*e 9&t for : 0, : 1, : 51
Problem 5: (Entropy) I!for't&o! T*eor, Fo"&! !" Fr9tor9*
$ R. Bose, 9ro$e' !o
2.10 (&) J (&&).
Problem 6: (Information Conveyed)A s&!e !$&se" "&e
&s tosse" o!e. If t*e fe of t*e "&e
&s 1,2,3, or >, ! !$&se" o&! &s tosse" o!e.
If t*e fe of t*e "&e &s K or , t*e o&! &s
tosse"t+&e. @&!" t*e &!for't&o! o!ee" $ot t*e fe of
t*e "&e $ t*e !'$er of *e"s o$t&!e".
Problem 7: (Exponential oise Channel) Yi =Xi +Zi ! +*ereZi
&s &.&." e?9o!e!t& "&str&$te"
!o&se +&t* 'e! .Ass'e t*t +e *e 'e! o!str&!t o! t*e
s&! (&.e., EXi ). *o+
t*t t*e 9&t of s* *!!e &sC=o 1
.
Problem 8: ("uffman code) A D sore ot9ts t*e foo+&! e&*t
s'$os M, $, , ", ef, , *N +&t* res9et&e
9ro$$&&t&es M0.3, 0.2, 0.1K, 0.1, 0.1, 0.08, 0.0K,
0.02N. Deter'&!e t*e
ff'! o"e !" eff&&e! $ t
-
8/13/2019 Tutorial 1_EEL710
3/3
EEL710_2013 (Prof. R. Bose) TUTORIAL 1 07/08/2013
'X=")o,1
" 1" )o,
1
1" 1"'Y
+*ere Y &s ! e!se'$e ofM = 1 9o&!ts a1... , aM1+&t*
9ro$$&&t&esPY(a) :PX(a)*(1") 41)M1 . *o+ t*t
'X"o 1"1"o 11"1" oM1
!" "eter'&!e t*e o!"&t&o! for e-&t.
PP4:A DF &s e" ""&t&e 'o"o$ &f &t *s t*e
&!9t !" ot9t 9*$et 0, 1....,$=1
!" t*e ot9ty &s &e! &! ter's of t*e &!9tx !"
!o&se r&$e Z $ y=x z . T*e !o&ser&$e t
