Introduction to Communications Slides

7/30/2019 Introduction to Communications Slides

1/75

Chapter 1: Introduction Chapter 3: Probability, Random Variables, Random Processes

A First Course in Digital CommunicationsHa H. Nguyen and E. Shwedyk

February 2009

A First Course in Digital Communications 1/75
http://find/


2/75


Analog and Digital Amplitude Modulations

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

tAnalogmessage

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

t

AM

signal

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

tDigitalmessage

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

t

BASKsignal

http://find/


3/75


What is Digital Communication?

x(t)

(a)

0

(c)

Ts0

t

(b)

(d)

Ts

0

x(t)

x(t) x(t)

t

t

t

http://find/


4/75


Why Digital Communications?

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

tTransmittedAM

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 52

0

2

t

ReceivedAM

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

tTran

smittedBASK

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 52

0

2

t

ReceivedBASK

http://goforward/http://find/http://goback/


5/75


Why Digital Communications?

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

tTransmittedAM

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 55

0

5

t

ReceivedAM

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 51

0

1

tTran

smittedBASK

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 55

0

5

t

ReceivedBASK


Ch I d Ch P b b l R d V bl R d P
http://find/


6/75


Regenerative Repeater in Digital Communications

Propagation distance

Original pulse Some distortion DegradedSeverely

degraded Regenerated

21 3 4 5

Digital communications: Transmitted signals belong to afinite set of waveforms

The distorted signal can be

recovered to its ideal shape, hence removing all the noise.Analog communications: Transmitted signals are analogwaveforms, which can take infinite variety of shapes Oncethe analog signal is distorted, the distortion cannot be

removed.A First Course in Digital Communications 6/75

Ch 1 I d i Ch 3 P b bili R d V i bl R d P
http://find/


7/75


Block Diagram of a Communication System

Source

(User)Transmitter Channel Receiver

Sink

(User)

Synchronization

Transmitter Receiver

Source

Encoder

Channel

EncoderModulator

De-

modulator

Channel

Decoder

Source

Decoder

(a)

(b)

Note: Synchronization block is only present in a digital system.A First Course in Digital Communications 7/75

Ch te 1 I t d ti Ch te 3 P b bilit R d V i bles R d P esses
http://find/


8/75


Digital vs. AnalogAdvantages:

Digital signals are much easier to be regenerated.Digital circuits are less subject to distortion and interference.

Digital circuits are more reliable and can be produced at alower cost than analog circuits.

It is more flexible to implement digital hardware than analoghardware.

Digital signals are beneficial from digital signal processing(DSP) techniques.

Disadvantages:

Heavy signal processing.

Synchronization is crucial.

Larger transmission bandwidth.

Non-graceful degradation.A First Course in Digital Communications 8/75

Chapter 1: Introduction Chapter 3: Probability Random Variables Random Processes
http://find/http://goback/


9/75


Introduction

The main objective of a communication system is the transfer

of information over a channel.Message signal is best modeled by a random signal: Anysignal that conveys information must have some uncertainty init, otherwise its transmission is of no interest.

Two types of imperfections in a communication channel:

Deterministic imperfection, such as linear and nonlineardistortions, inter-symbol interference, etc.Nondeterministic imperfection, such as addition of noise,interference, multipath fading, etc.

We are concerned with the methods used to describe andcharacterize a random signal, generally referred to as arandom process (also commonly called stochastic process).

In essence, a random process is a random variable evolving intime.


http://find/


10/75


Sample Space and Probability

Random experiment: its outcome, for some reason, cannot bepredicted with certainty.

Examples: throwing a die, flipping a coin and drawing a card

from a deck.Sample space: the set of all possible outcomes, denoted by .Outcomes are denoted by s and each lies in , i.e., .A sample space can be discrete or continuous.

Events are subsets of the sample space for which measures oftheir occurrences, called probabilities, can be defined ordetermined.


http://find/


11/75


Example of Throwing a Fair Die

Various events can be defined: the outcome is even number ofdots, the outcome is smaller than 4 dots, the outcome is morethan 3 dots, etc.


http://find/


12/75


Three Axioms of Probability

For a discrete sample space , define a probability measure P on as a set function that assigns nonnegative values to all events,denoted by E, in such that the following conditions are satisfied

Axiom 1: 0 P(E) 1 for all E (on a % scaleprobability ranges from 0 to 100%. Despite popular sportslore, it is impossible to give more than 100%).

Axiom 2: P() = 1 (when an experiment is conducted therehas to be an outcome).

Axiom 3: For mutually exclusive events1

E1, E2, E3, . . .wehave P (

i=1 Ei) =

i=1 P(Ei).

1The events E1, E2, E3,. . . are mutually exclusive ifEi Ej = for all

i = j, where is the null set.A First Course in Digital Communications 12/75

http://find/


13/75

p p y, ,

Important Properties of the Probability Measure

1. P(Ec) = 1 P(E), where Ec denotes the complement of E.This property implies that P(Ec) + P(E) = 1, i.e., somethinghas to happen.

2. P(

) = 0 (again, something has to happen).

3. P(E1 E2) = P(E1) + P(E2) P(E1 E2). Note that iftwo events E1 and E2 are mutually exclusive thenP(E1 E2) = P(E1) + P(E2), otherwise the nonzerocommon probability P(E1 E2) needs to be subtracted off.

4. If E1 E2 then P(E1) P(E2). This says that if event E1is contained in E2 then occurrence of E1 means E2 hasoccurred but the converse is not true.




14/75

Conditional Probability

We observe or are told that event E1 has occurred but are

actually interested in event E2: Knowledge that of E1 hasoccurred changes the probability of E2 occurring.

If it was P(E2) before, it now becomes P(E2|E1), theprobability of E2 occurring given that event E1 has occurred.

This conditional probability is given by

P(E2|E1) =

P(E2 E1)P(E1)

, if P(E1) = 00, otherwise

.

If P(E2|E1) = P(E2), or P(E2 E1) = P(E1)P(E2), thenE1 and E2 are said to be statistically independent.

Bayes rule

P(E2|E1) = P(E1|E2)P(E2)P(E1)

,


http://find/


15/75

Total Probability Theorem

The events

{Ei

}ni=1 partition the sample space if:

(i)ni=1

Ei = (1a)

(ii) Ei Ej = for all 1 i, j n and i = j (1b)If for an event A we have the conditional probabilities{P(A|Ei)}ni=1, P(A) can be obtained as

P(A) =n

i=1

P(Ei)P(A|Ei).

Bayes rule:

P(Ei|A) = P(A|Ei)P(Ei)P(A)

=P(A|Ei)P(Ei)

nj=1 P(A|Ej)P(Ej)

.


http://find/


16/75

Random Variables

R

1

4

3

2

4( )x 1( )x 2( )x3( )x

A random variable is a mapping from the sample space tothe set of real numbers.

We shall denote random variables by boldface, i.e., x, y, etc.,while individual or specific values of the mapping x aredenoted by x().


http://find/


17/75

Random Variable in the Example of Throwing a Fair Die

R

52 3 41 6

There could be many other random variables defined to describethe outcome of this random experiment!


http://goforward/http://find/http://goback/


18/75

Cumulative Distribution Function (cdf)

cdf gives a complete description of the random variable. It isdefined as:

Fx(x) = P( : x() x) = P(x x).

The cdf has the following properties:1. 0 Fx(x) 1 (this follows from Axiom 1 of the probability

measure).2. Fx(x) is nondecreasing: Fx(x1) Fx(x2) if x1 x2 (this is

because event x() x1 is contained in event x() x2).3. Fx() = 0 and Fx(+) = 1 (x() is the empty set,

hence an impossible event, while x() is the wholesample space, i.e., a certain event).

4. P(a < x b) = Fx(b) Fx(a).


http://find/


19/75

Typical Plots of cdf I

A random variable can be discrete, continuous or mixed.

0.1

0 x

( )F xx

(a)


http://find/


20/75

Typical Plots of cdf II

0.1

0 x

( )F xx

(b)

0.1

0 x

( )F xx

(c)


http://find/


21/75

Probability Density Function (pdf)

The pdf is defined as the derivative of the cdf:

fx(x) =dFx(x)

dx.

It follows that:

P(x1

x

x2) = P(x

x2)

P(x

x1)

= Fx(x2) Fx(x1) =

x2

x1

fx(x)dx.

Basic properties of pdf:1. fx(x) 0.2.

fx(x)dx = 1.

3. In general, P(x A) = A

fx(x)dx.

For discrete random variables, it is more common to definethe probability mass function (pmf): pi = P(x = xi).

Note that, for all i, one has pi 0 and i

pi = 1.


http://find/


22/75

Bernoulli Random Variable

0x

1 0x

1

p1

1

(1 )p( )p

( )F xx

( )f xx

A discrete random variable that takes two values 1 and 0 withprobabilities p and 1 p.Good model for a binary data source whose output is 1 or 0.

Can also be used to model the channel errors.




23/75

Binomial Random Variable

0x

2

05.0

10.0

20.0

15.0

25.0

30.0

4 6

( )f xx

A discrete random variable that gives the number of 1s in asequence of n independent Bernoulli trials.

fx(x) =n

k=0

n

k

pk(1p)nk(xk), where

n

k

=

n!

k!(n k)! .


http://find/


24/75

Uniform Random Variable

0x

ab

1

a b 0x

a b

1

( )F xx( )f xx

A continuous random variable that takes values between aand b with equal probabilities over intervals of equal length.

The phase of a received sinusoidal carrier is usually modeledas a uniform random variable between 0 and 2. Quantizationerror is also typically modeled as uniform.


http://find/


25/75

Gaussian (or Normal) Random Variable

0x

0x

12

2

1

2

1

( )f xx

( )F xx

A continuous random variable whose pdf is:

fx(x) = 122

exp(x )

2

22

,

and 2 are parameters. Usually denoted as N(, 2).Most important and frequently encountered random variablein communications.


http://find/


26/75

Functions of A Random Variable

The function y = g(x) is itself a random variable.

From the definition, the cdf ofy can be written as

Fy(y) = P( : g(x()) y).Assume that for all y, the equation g(x) = y has a countable

number of solutions and at each solution point, dg(x)/dxexists and is nonzero. Then the pdf ofy = g(x) is:

fy(y) =i

fx(xi)

dg(x)dx x=xi,

where {xi} are the solutions of g(x) = y.A linear function of a Gaussian random variable is itself aGaussian random variable.


http://find/


27/75

Expectation of Random Variables I

Statistical averages, or moments, play an important role in the

characterization of the random variable.The expected value(also called the mean value, first moment)of the random variable x is defined as

mx = E{x}

xfx(x)dx,

where E denotes the statistical expectation operator.

In general, the nth moment ofx is defined as

E{xn}

xnfx(x)dx.

For n = 2, E{x2} is known as the mean-squared value of therandom variable.


http://find/


28/75

Expectation of Random Variables II

The nth central moment of the random variable x is:

E{y} = E{(x mx)n} =

(x mx)nfx(x)dx.

When n = 2 the central moment is called the variance,

commonly denoted as 2x:

2x = var(x) = E{(x mx)2} =

(x mx)2fx(x)dx.

The variance provides a measure of the variablesrandomness.

The mean and variance of a random variable give a partialdescription of its pdf.


http://find/


29/75

Expectation of Random Variables III

Relationship between the variance, the first and secondmoments:

2x = E

{x2

} [E

{x

}]2 = E

{x2

} m2x.

An electrical engineering interpretation: The AC power equalstotal power minus DC power.

The square-root of the variance is known as the standarddeviation, and can be interpreted as the root-mean-squared(RMS) value of the AC component.


http://find/


30/75

The Gaussian Random Variable

0 0.2 0.4 0.6 0.8 10.8

0.6

0.4

0.2

0

0.2

0.4

0.6

t(sec)

Signalamplitude(

volts)

(a) A muscle (emg) signal



(b) Hi t d df fit
http://find/


31/75

1 0.5 0 0.5 10

0.5

1

1.5

2

2.5

3

3.5

4

x (volts)

fx(x)(1/volts)

(b) Histogram and pdf fits

Histogram

Gaussian fit

Laplacian fit

fx(x) =1

22x

e (xmx)

2

22x (Gaussian)

fx(x) =a

2ea|x| (Laplacian)



G i Di ib i (U i i )
http://find/


32/75

Gaussian Distribution (Univariate)

15 10 5 0 5 10 150

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

fx(x)

x=1

x=2

x=5

Range (kx) k = 1 k = 2 k = 3 k = 4P(mx kx < x mx kx) 0.683 0.955 0.997 0.999Error probability 103 104 106 108

Distance from the mean 3.09 3.72 4.75 5.61



E l
http://find/


33/75

Example

The noise voltage in an electric circuit can be modeled as aGaussian random variable with zero mean and variance 2.

(a) Show that the probability that the noise voltage exceeds somelevel A can be expressed as Q A .

(b) Given that the noise voltage is negative, express in terms ofthe Q function the probability that the noise voltage exceedsA, where A < 0.

(c) Using the MATLAB function Q.m available from the courses

website, evaluate the expressions found in (a) and (b) for2 = 108 and A = 104.



S l i
http://find/


34/75

Solution

fx(x) = 12

e (x

)2

22 (=0)= 12

e x2

22 .

The Q-function is defined as:

Q(t) =1

2

t

e2

2 d

The probability that the noise voltage exceeds some level A is

P(x > A) =

A

fx(x)dx

= 12

Ae

x222 dx

(= x )=(d=dx

)

12

A

e22 d

= Q

A

.



Gi th i lt i ti th b bilit th t th i


35/75

Given the noise voltage is negative, the probability that the noisevoltage exceeds some level A < 0 is:

P(x

> A|x

< 0) =

P(x > A , x < 0)

P(x < 0) =

P(A < x < 0)

P(x < 0)

=QA

Q(0)Q(0)

=QA

121/2

= 2Q

A

1.

0x

( )f xx

22

1

0x

( | 0)f x


36/75

With 2 = 108 and A = 104,

P(x >

104) = Q

104

108 = Q(

1) = 0.8413

P(x > 104|x < 0) = 2Q(1) 1 = 0.6827.



Example
http://find/


37/75

Example

The Matlab function randn defines a Gaussian random variable of

zero mean and unit variance. Generate a vector of L = 106

samples according to a zero-mean Gaussian random variable x withvariance 2 = 108. This can be done as follows:x=sigma*randn(1,L).

(a) Based on the sample vector x, verify the mean and variance of

random variable x.

(b) Based on the sample vector x, compute the probability thatx > A, where A = 104. Compare the result with thatfound in Question 2-(c).

(c) Using the Matlab command hist, plot the histogram ofsample vector x with 100 bins. Next obtain and plot the pdffrom the histogram. Then also plot on the same figure thetheoretical Gaussian pdf (note the value of the variance forthe pdf). Do the histogram pdf and theoretical pdf fit well?


http://find/


38/75

Solution

L=10^6; % Length of the noise vector

sigma=sqrt(10^(-8)); % Standard deviation of the noiseA=-10^(-4);

x=sigma*randn(1,L);

%[2] (a) Verify mean and variance:

mean_x=sum(x)/L; % this is the same as mean(x)

variance_x=sum((x-mean_x).^2)/L; % this is the same as var(x);

% mean_x = -3.7696e-008

% variance_x = 1.0028e-008

%[3] (b) Compute P(x>A)P=length(find(x>A))/L;

% P = 0.8410



%[5] ( ) Hi d G i df fi
http://find/


39/75

%[5] (c) Histogram and Gaussian pdf fit

N_bins=100;

[y,x_center]=hist(x,100); % This bins the elements of x into N_

% and returns the number of elements in each con

% while the bin centers are stored in x_center.

dx=(max(x)-min(x))/N_bins; % This gives the width of each bin.

hist_pdf= (y/L)/dx; % This approximate the pdf to be a constant

pl(1)=plot(x_center,hist_pdf,color,blue,linestyle,--,li

hold on;

x0=[-5:0.001:5]*sigma; % this specifies the range of random vari

true_pdf=1/(sqrt(2*pi)*sigma)*exp(-x0.^2/(2*sigma^2));

pl(2)=plot(x0,true_pdf,color,r,linestyle,-,linewidth,1

xlabel({\itx},FontName,Times New Roman,FontSize,16);

ylabel({\itf}_{\bf x}({\itx}),FontName,Times New Roman,Fo

legend(pl,pdf from histogram,true pdf,+1);

set(gca,FontSize,16,XGrid,on,YGrid,on,GridLineStyle,


http://find/


40/75

5 0 5

x 104

0

500

1000

1500

2000

2500

3000

3500

4000

x

fx

(x)

pdf from histogram

true pdf



Multiple Random Variables I
http://find/


41/75

Multiple Random Variables I

Often encountered when dealing with combined experimentsor repeated trials of a single experiment.

Multiple random variables are basically multidimensionalfunctions defined on a sample space of a combinedexperiment.

Let x and y be the two random variables defined on the samesample space . The joint cumulative distribution function isdefined as

Fx,y(x, y) = P(x x,y y).Similarly, the joint probability density function is:

fx,y(x, y) =2Fx,y(x, y)

xy.



Multiple Random Variables II
http://find/


42/75

Multiple Random Variables II

When the joint pdf is integrated over one of the variables, oneobtains the pdf of other variable, called the marginal pdf:

fx,y(x, y)dx = fy(y),

fx,y(x, y)dy = fx(x).

Note that:

fx,y(x, y)dxdy = F(, ) = 1Fx,y(, ) = Fx,y(, y) = Fx,y(x, ) = 0.



Multiple Random Variables III
http://find/


43/75

Multiple Random Variables III

The conditional pdf of the random variable y, given that the

value of the random variable x is equal to x, is defined as

fy(y|x) =

fx,y(x, y)

fx(x), fx(x) = 0

0, otherwise.

Two random variables x and y are statistically independent ifand only if

fy(y|x) = fy(y) or equivalently fx,y(x, y) = fx(x)fy(y).

The joint moment is defined as

E{xjyk} =

xjykfx,y(x, y)dxdy.



Multiple Random Variables IV
http://find/


44/75

Multiple Random Variables IV

The joint central moment is

E{(xmx)j(ymy)k} =

(xmx)j(ymy)kfx,y(x, y)dxdy

where mx = E

{x

}and my = E

{y

}.

The most important moments are

E{xy}

xyfx,y(x, y)dxdy (correlation)

cov{x,y} E{(x mx)(y my)}= E{xy} mxmy (covariance).



Multiple Random Variables V
http://find/


45/75

Multiple Random Variables V

Let 2x and 2y be the variance ofx and y. The covariance

normalized w.r.t. xy is called the correlation coefficient:

x,y =cov{x,y}

xy.

x,y indicates the degree of linear dependence between two

random variables.It can be shown that |x,y| 1.x,y = 1 implies an increasing/decreasing linear relationship.If x,y = 0, x and y are said to be uncorrelated.

It is easy to verify that ifx and y are independent, thenx,y = 0: Independence implies lack of correlation.

However, lack of correlation (no linear relationship) does notin general imply statistical independence.



Examples of Uncorrelated Dependent Random Variables
http://find/


46/75

Examples of Uncorrelated Dependent Random Variables

Example 1: Let x be a discrete random variable that takes on

{1, 0, 1} with probabilities {14 , 12 , 14}, respectively. Therandom variables y = x3 and z = x2 are uncorrelated butdependent.

Example 2: Let x be an uniformly random variable over[

1, 1]. Then the random variables y = x and z = x2 are

uncorrelated but dependent.

Example 3: Let x be a Gaussian random variable with zeromean and unit variance (standard normal distribution). Therandom variables y = x and z = |x| are uncorrelated butdependent.Example 4: Let u and v be two random variables (discrete orcontinuous) with the same probability density function. Thenx = u v and y = u+ v are uncorrelated dependent randomvariables.



Example 1
http://find/


47/75

p

x {1, 0, 1} with probabilities {1/4, 1/2, 1/4} y = x

3

{1, 0, 1} with probabilities {1/4, 1/2, 1/4} z = x2 {0, 1} with probabilities {1/2, 1/2}my = (1)14 + (0)12 + (1)14 = 0; mz = (0)12 + (1)12 = 12 .The joint pmf (similar to pdf) ofy and z:

0

1

1

1

12

1

4

1

4

y

zP(y = 1, z = 0) = 0P(y = 1, z = 1) = P(x = 1) = 1/4P(y = 0, z = 0) = P(x = 0) = 1/2

P(y = 0, z = 1) = 0

P(y = 1, z = 0) = 0

P(y = 1, z = 1) = P(x = 1) = 1/4Therefore, E{yz} = (1)(1)14 + (0)(0)12 + (1)(1)14 = 0 cov{y, z} = E{yz} mymz = 0 (0)1/2 = 0!



Jointly Gaussian Distribution (Bivariate)
http://find/


48/75

y ( )

fx,y(x, y) =1

2xy

1 2x,yexp

1

2(1 2x,y)

(x mx)22x

2x,y(x mx)(y my)xy

+(y my)2

2y

,

where mx, my, x, y are the means and variances.

x,y is indeed the correlation coefficient.

Marginal density is Gaussian: fx(x) N(mx, 2x) andfy(y) N(my, 2y).When x,y = 0 fx,y(x, y) = fx(x)fy(y) randomvariables x and y are statistically independent.Uncorrelatedness means that joint Gaussian random variablesare statistically independent. The converse is not true.

Weighted sum of two jointly Gaussian random variables is alsoGaussian.



Joint pdf and Contours for x = y = 1 and x,y = 0
http://find/


49/75

p x y x,y

32

10

12 3

2

0

2

0

0.05

0.1

0.15

x

x,y

=0

y

fx,y

(x,y)

x

y

2 1 0 1 22.5

2

1

0

1

2

2.5



Joint pdf and Contours for x = y = 1 and x,y = 0.3
http://find/


50/75

p y ,y

32

10

12

3

2

0

2

0

0.05

0.1

0.15

x

x,y

=0.30

y

fx,y

(x,y)

a crosssection

x

y

2 1 0 1 22.5

2

1

0

1

2

2.5



http://find/


51/75

y ,y

32

10

1 23

2

0

2

0

0.05

0.1

0.15

0.2

a crosssection

x

x,y

=0.70

y

fx,y

(x,y)

x

y

2 1 0 1 22.5

2

1

0

1

2

2.5



http://find/


52/75

32

10

12 3

2

0

2

0

0.1

0.2

0.3

0.4

0.5

x

x,y

=0.95

y

fx,y

(x,y)

x

y

2 1 0 1 22.5

2

1

0

1

2

2.5



MultivariateGaussian pdf
http://find/


53/75

Definex = [x1,x2, . . . ,xn], a vector of the means

m = [m1, m2, . . . , mn], and the n n covariance matrix Cwith Ci,j = cov(xi,xj) = E{(xi mi)(xj mj)}.The random variables {xi}ni=1 are jointly Gaussian if:

fx1,x2,...,xn(x1, x2, . . . , xn) =1

(2)n det(C)

exp

1

2(x m)C1(x m)

.

If C is diagonal (i.e., the random variables {xi}ni=1 are alluncorrelated), the joint pdf is a product of the marginal pdfs:Uncorrelatedness implies statistical independent for multipleGaussian random variables.



Random Processes I
http://find/


54/75

Real number

Timetk

x1(t,

1)

x2(t,

2)

xM

(t,M

)

.

.

.

1

2

M

3

j

x(t,)

t2

t1

.

.

.

.

.

.

. . .

x(tk,)

.

.

.

A mapping from a sample space to a set of time functions.



Random Processes II
http://find/


55/75

Ensemble: The set of possible time functions that one sees.

Denote this set by x(t), where the time functions x1(t, 1),x2(t, 2), x3(t, 3), . . . are specific members of the ensemble.

At any time instant, t = tk, we have random variable x(tk).

At any two time instants, say t1 and t2, we have two different

random variables x(t1) and x(t2).Any relationship between them is described by the joint pdffx(t1),x(t2)(x1, x2; t1, t2).

A complete description of the random process is determined by

the joint pdf fx(t1),x(t2),...,x(tN)(x1, x2, . . . , xN; t1, t2, . . . , tN).The most important joint pdfs are the first-order pdffx(t)(x; t) and the second-order pdf fx(t1)x(t2)(x1, x2; t1, t2).



Examples of Random Processes I
http://find/


56/75

(a) Thermal noise

0 t

(b) Uniform phase

t0



Examples of Random Processes II
http://find/


57/75

t0

(c) Rayleigh fading process

t

+V

V

(d) Binary random data

0

Tb



Classification of Random Processes


58/75

Based on whether its statistics change with time: the processis non-stationary or stationary.

Different levels of stationarity:Strictly stationary: the joint pdf of any order is independent ofa shift in time.Nth-order stationarity: the joint pdf does not depend on thetime shift, but depends on time spacings:

fx(t1),x(t2),...x(tN)(x1, x2, . . . , xN; t1, t2, . . . , tN) =

fx(t1+t),x(t2+t),...x(tN+t)(x1, x2, . . . , xN; t1 + t, t2 + t , . . . , tN + t).

The first- and second-order stationarity:

fx(t1)(x, t1) = fx(t1+t)(x; t1 + t) = fx(t)(x)

fx(t1),x(t2)(x1, x2; t1, t2) = fx(t1+t),x(t2+t)(x1, x2; t1 + t, t2 + t)

= fx(t1),x(t2)(x1, x2; ), = t2 t1.A First Course in Digital Communications 58/75


Statistical Averages or Joint Moments
http://find/


59/75

Consider N random variables x(t1),x(t2), . . .x(tN). Thejoint moments of these random variables is

E{xk1(t1),xk2(t2), . . .xkN(tN)} =x1=

xN=

xk11 xk22 xkNN fx(t1),x(t2),...x(tN)(x1, x2, . . . , xN; t1, t2, . . . , tN)

dx1dx2 . . . dxN,

for all integers kj 1 and N 1.Shall only consider the first- and second-order moments, i.e.,E{x(t)}, E{x2(t)} and E{x(t1)x(t2)}. They are the meanvalue, mean-squared value and (auto)correlation.



Mean Value or the First Moment
http://find/


60/75

The mean value of the process at time t is

mx(t) = E{x(t)} =

xfx(t)(x; t)dx.

The average is across the ensemble and if the pdf varies withtime then the mean value is a (deterministic) function of time.

If the process is stationary then the mean is independent of tor a constant:

mx = E{x(t)} =

xfx(x)dx.



Mean-Squared Value or the Second Moment
http://find/


61/75

This is defined as

MSVx(t) = E{x2(t)} =

x2fx(t)(x; t)dx (non-stationary),

MSVx = E{x2

(t)} =

x2

fx(x)dx (stationary).

The second central moment (or the variance) is:

2x(t) = E[x(t) mx(t)]

2

= MSVx(t) m2x(t) (non-stationary),

2x = E

[x(t) mx]2

= MSVx m2x (stationary).



Correlation
http://find/


62/75

The autocorrelation function completely describes the power

spectral density of the random process.Defined as the correlation between the two random variablesx1 = x(t1) and x2 = x(t2):

Rx(t1, t2) = E{x(t1)x(t2)}

=x1=

x2=

x1x2fx1,x2(x1, x2; t1, t2)dx1dx2.

For a stationary process:

Rx() = E{x(t)x(t + )}=

x1=

x2=x1x2fx1,x2(x1, x2; )dx1dx2.

Wide-sense stationarity (WSS) process: E{x(t)} = mx forany t, and Rx(t1, t2) = Rx() for = t2 t1.



Properties of the Autocorrelation Function
http://find/


63/75

1. Rx() = Rx(). It is an even function of because thesame set of product values is averaged across the ensemble,regardless of the direction of translation.

2. |Rx()| Rx(0). The maximum always occurs at = 0,though there maybe other values of for which it is as big.Further Rx(0) is the mean-squared value of the random

process.3. If for some 0 we have Rx(0) = Rx(0), then for all integers

k, Rx(k0) = Rx(0).

4. If mx = 0 then Rx() will have a constant component equalto m2x.

5. Autocorrelation functions cannot have an arbitrary shape.The restriction on the shape arises from the fact that theFourier transform of an autocorrelation function must begreater than or equal to zero, i.e., F{Rx()} 0.



Power Spectral Density of a Random Process (I)
http://find/


64/75

Taking the Fourier transform of the random process does not work.

x2(t,

2)

xM

(t,M

)

x1(t,1)

t

t0

0

0

0

0

0

|X1(f,1)|

|X2(f,

2)|

|XM

(f,M

)|

f

f

f.

.

.

.

.

.

.

.

.

.

.

.

Timedomain ensemble Frequencydomain ensemble

t



Power Spectral Density of a Random Process (II)
http://find/


65/75

Need to determine how the average power of the process is

distributed in frequency.Define a truncated process:

xT(t) =

x(t), T t T

0, otherwise.

Consider the Fourier transform of this truncated process:

XT(f) =

xT(t)ej2ftdt.

Average the energy over the total time, 2T:

P =1

2T

TTx2T(t)dt =

1

2T

|XT(f)|2 df (watts).



Power Spectral Density of a Random Process (III)
http://find/


66/75

Find the average value ofP:

E{P} = E

1

2T

T

Tx2T(t)dt

= E

1

2T

|XT(f)|2 df

.

Take the limit as T :

limT

12T

TT

Ex2T(t)

dt = lim

T1

2T

E|XT(f)|2 df,

It follows that

MSVx = limT

1

2T

T

T Ex

2

T(t)

dt

=

limT

E|XT(f)|2

2T

df (watts).



Power Spectral Density of a Random Process (IV)
http://find/


67/75

Finally,

Sx(f) = limT

E|XT(f)|2

2T

(watts/Hz),

is the power spectral density of the process.

It can be shown that the power spectral density and theautocorrelation function are a Fourier transform pair:

Rx() Sx(f) =

=Rx()ej2fd.



Time Averaging and Ergodicity
http://find/


68/75

A process where any member of the ensemble exhibits the

same statistical behavior as that of the whole ensemble.All time averages on a single ensemble member are equal tothe corresponding ensemble average:

E{xn(t))

}=

xnfx(x)dx

= limT

1

2T

TT

[xk(t, k)]ndt, n, k.

For an ergodic process: To measure various statistical

averages, it is sufficient to look at only one realization of theprocess and find the corresponding time average.

For a process to be ergodic it must be stationary. Theconverse is not true.



Examples of Random Processes
http://find/


69/75

(Example 3.4) x(t) = A cos(2f0t +), where is arandom variable uniformly distributed on [0, 2]. This processis both stationary and ergodic.

(Example 3.5) x(t) = x, where x is a random variable

uniformly distributed on [A, A], where A > 0. This processis WSS, but not ergodic.(Example 3.6) x(t) = A cos(2f0t +) where A is azero-mean random variable with variance, 2A, and isuniform in [0, 2]. Furthermore, A and are statistically

independent. This process is not ergodic, but strictlystationary.



Random Processes and LTI Systems
http://find/


70/75

Linear, Time-Invariant

(LTI) System

Input Output

( ) ( )h t H f ( )tx ( )ty

, ( ) ( )m R S f x x x

, ( ) ( )m R S f y y y

, ( )R x y

my = E{y(t)} = E

h()x(t )d

= mxH(0)

Sy(f) = |H(f)|2 Sx(f)Ry() = h() h() Rx().



Thermal Noise in Communication Systems
http://find/


71/75

A natural noise source is thermal noise, whose amplitude

statistics are well modeled to be Gaussian with zero mean.

The autocorrelation and PSD are well modeled as:

Rw() = kGe||/t0

t0(watts),

Sw(f) =2kG

1 + (2f t0)2(watts/Hz).

where k = 1.38 1023 joule/0K is Boltzmanns constant, Gis conductance of the resistor (mhos); is temperature indegrees Kelvin; and t0 is the statistical average of timeintervals between collisions of free electrons in the resistor (onthe order of 1012 sec).



(a) Power Spectral Density, Sw

(f)

N0/2
http://find/


72/75

15 10 5 0 5 10 150

f(GHz)

Sw

(f)(watts/Hz)

0.1 0.08 0.06 0.04 0.02 0 0.02 0.04 0.06 0.08 0.1 (picosec)

Rw

()(watts)

(b) Autocorrelation,Rw

()

0

N0/2()

White noise

Thermal noise

White noise

Thermal noise



The noise PSD is approximately flat over the frequency rangeof 0 to 10 GHz let the spectrum be flat from 0 to :
http://find/


73/75

Sw(f) =N0

2(watts/Hz),

where N0 = 4kG is a constant.

Noise that has a uniform spectrum over the entire frequencyrange is referred to as white noise

The autocorrelation of white noise isRw() =

N02

() (watts).

Since Rw() = 0 for = 0, any two different samples ofwhite noise, no matter how close in time they are taken, areuncorrelated.

Since the noise samples of white noise are uncorrelated, if thenoise is both white and Gaussian (for example, thermal noise)then the noise samples are also independent.



Example
http://find/


74/75

Suppose that a (WSS) white noise process, x(t), of zero-mean andpower spectral density N0/2 is applied to the input of the filter.

(a) Find and sketch the power spectral density andautocorrelation function of the random process y(t) at theoutput of the filter.

(b) What are the mean and variance of the output process y(t)?

L

R( )tx ( )ty



R 1
http://find/


75/75

H(f) =R

R +j2f L=

1

1 +j2fL/R.

Sy(f) =N02

1

1 +2LR

2f2

Ry() = N0R4L

e(R/L)||.

0 0

2

0N

(Hz)f (sec)

L

RN

4

0

( ) (watts/Hz)S fy

( ) (watts)R y

http://find/

Documents

Introduction to Communications Slides