8/3/2019 ITC_Unit_I
1/31
Unit 1PART B
ote that the answer for the questions is indicated via the page number given alongside. Match the page numbersto the scanned pages given below to get the answer.
1. What is Entropy? Explain the properties and types of Entropy? (1-7 to 1-11)
2. Explain about the relationship between Joint and Conditional Entropy. (1-7)
3. The Joint Probability Matrix is given as
nd all the Entropies and Mutual Information.Prove that the Upper bound on Entropy is given as Hmaxlog2M.Here M is the number of messages
mitted by the source. (1-11)Prove that H(X,Y)=H(X/Y)+H(Y) (1-72)
=H(Y/X) +H(X)(i) A channel has the following Channel matrix.
P(Y/X) = (1-70)
)Draw the Channel diagram.)If the source has equally likely outputs, Compute the probabilities associated with the channel
utputs for P=0.2 (6 marks)i)Two BSCs are connected in cascade as shown in the figure.
a)Find the Channel Matrix of the resultant channel. (1-74)
)Find P (Z1) and P (Z2), if P(X1) =0.6, P(X2) =0.4(i)Prove that the Mutual information of the channel is Symmetric.
I(X, Y) =I(Y, X) (6 marks) (1-87)
i)Prove that the mutual information is always positiveI(X, Y) 0 (6marks) (1-89)
Prove the following relationships: I(X,Y)=H(X)-H(X/Y) (1-87)
b) I(X,Y)=H(Y)-H(Y/X) (1-89)(i) Consider the Binary Symmetric Channel shown in the figure. (1-91)
P(X1) = P
P(X2) = 1-P
alculate H(X),H(Y),H(Y/X) and I(X,Y)i)Prove the following (1-90)
1
2
Z1
Z2
Y
Y2
0.8 0.7
0.3
0.3
0.8 0.7
0.2
0.2
Y1
X2
X1
Y2
1-
1-
8/3/2019 ITC_Unit_I
2/31
(X,Y)=H(X)+H(Y)-H(X,Y)0. (a)Find the Mutual Information and Channel capacity for the channel shown in the figure. Givenat P(X1) =0.6 and P(X2) =0.4 (6 Marks) (1-94)
)A Binary Channel Matrix is given as:
etermine H(X), H(X/Y), H(Y/X) and Mutual Information I(X, Y). (1-98)
Y1
X2
X1
Y2
=
1= =
1- =
8/3/2019 ITC_Unit_I
3/31
8/3/2019 ITC_Unit_I
4/31
8/3/2019 ITC_Unit_I
5/31
8/3/2019 ITC_Unit_I
6/31
8/3/2019 ITC_Unit_I
7/31
8/3/2019 ITC_Unit_I
8/31
8/3/2019 ITC_Unit_I
9/31
8/3/2019 ITC_Unit_I
10/31
8/3/2019 ITC_Unit_I
11/31
8/3/2019 ITC_Unit_I
12/31
8/3/2019 ITC_Unit_I
13/31
8/3/2019 ITC_Unit_I
14/31
8/3/2019 ITC_Unit_I
15/31
8/3/2019 ITC_Unit_I
16/31
8/3/2019 ITC_Unit_I
17/31
8/3/2019 ITC_Unit_I
18/31
8/3/2019 ITC_Unit_I
19/31
8/3/2019 ITC_Unit_I
20/31
8/3/2019 ITC_Unit_I
21/31
8/3/2019 ITC_Unit_I
22/31
8/3/2019 ITC_Unit_I
23/31
8/3/2019 ITC_Unit_I
24/31
8/3/2019 ITC_Unit_I
25/31
8/3/2019 ITC_Unit_I
26/31
8/3/2019 ITC_Unit_I
27/31
8/3/2019 ITC_Unit_I
28/31
8/3/2019 ITC_Unit_I
29/31
8/3/2019 ITC_Unit_I
30/31
8/3/2019 ITC_Unit_I
31/31
Recommended