ECE 145B / 218B, notes 2: Mathematics of Electrical Noise · Thermal Noise as a Random Variable R The noise voltage has variance kT/C. / / 2 / 2 / 2 at temperature T has mean energy

class notes, M. Rodwell, copyrighted 2012

ECE 145B / 218B, notes 2: Mathematics of Electrical Noise

Mark Rodwell

University of California, Santa Barbara

[email protected] 805-893-3244, 805-893-3262 fax


Strategy

ive.comprehens isbook sder Zeil'Van .literature in theor

web),(on the notes class noisemy in found becan detail More

rates.error ns,correlatio signal

functions,n correlatio densities, spectral SNR,

ofn calculatiocorrect for sufficient backround give

:Strategy

detail.in subject thisdevelop toclass in this not time is There


Topics

Math:

distributions, random variables, expectations,

pairs of RV, joint distributions, covariance and correlations.

Random processes, stationarity, ergodicity, correlation functions,

autocorrelation function, power spectral density.

Noise models of devices:

thermal and shot noise.

Models of resistors, diodes, transitors, antennas.

Circuit noise analysis:

network representation. Solution.

Total output noise. Total input noise. 2 generator model.

En/In model. Noise figure, noise temperature.

Signal / noise ratio.


random variables


The first step: random Variables

function. ondistributiy probabilit theis )(

)(}{

is and between lies y that probabilit The

. valueparticulara on takes variablerandoma ,experiment an During

2

1

21

21

xf

dxxfxxxP

xxx

xX

X

x

x

X

fX(x)

xx1

x2


Example: The Gaussian Distribution

Gaussian. theofthat

toclose onsdistributiy probabilit have effects smallmany of sum thefrom

arising processes random physical rem*,limit theo central* theof Because

).( deviation standard theand )( mean eshortly th define willWe

2exp

2

1)(

:ondistributi Gaussian The

2

2

2

2

x

xx

X

x

xxxf

fX(x)

xx

~2x


Mean values and expectations

dxxfxXEX

X

dxxxfXEXX

dxxfxgxgE

X

X

X

)(

of valueExpected

)(

X of Value Mean

)()()(

X variablerandom theof g(X) functiona of nExpectatio

222

2


Variance

variance theofroot square the

simply is X of deviation standard The

)(

valueaverage its from

deviation square-mean-root its is X of varianceThe

2222

2

x

XX

x

dxxfxxxXExX


Returning to the Gaussian Distribution

fX(x)

xx

~2x

clear. benow should

2exp

2

1)(

:ondistributi Gaussian thedescribing notation The

2

2

2

xx

X

xxxf


Variance vs Expectation of the Square

n.expectatio theof square theminus

square theof nexpectatio theis varianceThe

2

2

2

222

22

22

22

22

xX

xxxX

xXxX

xxXX

xXxXxX

X

X


Pairs of Random Variables

D

C

B

A

XY

XY

dxdyyxfDyCBxAP

yxf

),(} and {

),( ondistributijoint by the described isbehavior joint Their

y.andx valuesparticular specific on takes

Yand X variablesrandom ofpair a ,experiment an In

. variablesrandom of pairs understandfirst must we

processes, random understand To


Pairs of Random Variables

D

C

Y

D

C

XY

B

A

X

B

A

XY

dyyf

dxdyyxfDyCP

dxxf

dxdyyxfBxAP

)(

),(} {

:for Ysimilarly and

)(

),(} {

defined be alsomust onsdistributi Marginal


Statistical Independence

expected.generally not is This

t.independenlly statistica be tosaid are variablesthe

, )()(),(

wherecase theIn

yfxfyxf YXXY


Expectations of a pair of random variables

. and for similarly ...and

)( ),(

ofn Expectatio

)(),(

: ofn Expectatio

),(),(),(

is Y and Y variablesrandom theof Y)g(X,function a ofn expectatio The

2

2222

2

YY

dxxfxdxdyyxfxXXE

X

dxxxfdxdyyxxfxXE

X

dxdyyxfyxgyxgE

XXY

XXY

XY


Correlation between random variables

values.mean zero have or Y X

either if same theare covariance and ncorrelatio that Note

is Yand X of covariance The

),(•

is Yand X of ncorrelatio The

yxR

yxyXYxXYEyYxXEC

dxdyyxfxyXYER

XY

XY

XYXY


Correlation versus Covariance

careful. Be

ons.distributi lconditiona

calculate e.g. we whenreturn can valuesmean nonzero But,

ably.interchang terms two theuse to

analysis noisecircuit in common thereforeisIt

.covariance toequal thenis nCorrelatio

mean. zero have then variablesrandom The

component. varying- time thefrom bias) (DC valuemean the

separateusually wecurrents, and voltages with workingare weWhen


Correlation Coefficient

.covariance and ncorrelatio between

gy terminoloin confusion (standard) theNote

/

is Yand X oft coefficien ncorrelatio The

YXXYXY C


Sum of TWO Random Variables

n.correlatio of role theemphasizes This

2

means zero have both and If

2

2)(

: variablesrandom twoof Sum

222

22

2222

XY

XY

CYEXEZE

YX

RYEXE

YXYXEYXEZE

YXZ


Pairs of Jointly Gaussian Random Variables

jiiii

n

YYXXXY

XYYX

XY

xxExxEx

XXX

yyyyxxxx

yxf

scovariance and , variances, means

ofset a by specified is which

),,,(

vector random GaussianJointly a have can wegeneral, In

. variablesof #larger a toextended be can definition This

)())(()(

)1(2

1exp

12

1),(

:GaussianJointly are Yand X If

21

2

2

2

2

2

2


Linear Operations on JGRV's

number.any of JGRVsfor holdsresult The

function. Gaussiana produces

functions Gaussian 2 of nconvolutio

because arisesresult theproof; without stated is This

Gaussian.Jointly also are and Then

and

define weif and Gaussian,Jointly are Yand X If

WV

dYcXWbYaXV


Probability distribution after a Linear Operation on JGRV's

2

2

2

2

2

2

22

22

22222222

22222222

)())(()(

)1(2

1exp

12

1),(

. and of ondistributijoint thecalculatenow can We

))(()(

)(

))((

)(2

)(2

and

WWVVVW

VWWV

VW

VW

W

V

wwwwvvvv

wvf

WV

YdXcYbXaYEbdXYEbcadXacE

WVbdYXYbcadacXE

WVdYcXbYaXEWVVWEC

YdXcXYEcdYEdXEcWWE

YbXaXYEabYEbXEaVVE

YdXcWYbXabYaXEVEV

tedious details


Why are JGRV's Important ?

circuits).(linear

systemslinear in npropagatio noise of nscalculatio simplifies vastly This

found.simply be always can functions ondistributi n,informatio thisWith

. variancesand ns,correlatio means, of

track keep tosufficient isit ,operationslinear tosubjected sJGRV' With

:conclusionclear a is but there tedious wasslidelast theon math The


Uncorrelated Variables.

ceindependenimply does eduncorrelat s,JGRV'For

ce.independenimply not does ncorrelatio Zero

n.correlatio zero implies ceIndependen

)()(),(

:tindependenlly Statistica

0

:edUncorrelat

yfxfyxf

C

YXXY

XY


Summing of Noise (Random) Voltages

V1

V2

R

add. do voltagesnoise random theof values timeousinstantane The

included. bemust termoncorrellatia -

-addnot do generators random two theof powers noise The

12

11

2111

12

11

12

11

21

)(1

P variablerandoma isresistor thein dissipatedpower The

resistor R the toapplied are voltagesTwo

2

221

2

1

2

2

2

1

2

2

2

1

2

2

2

1

2

221

2

1

2

21

2121

21

21

VR

VVR

VR

RV

RV

R

VRR

VR

VR

CR

VR

VVVVR

VVR

PPE

VVVV

VV

VV


Shot Noise as a Random Variable

count. theof valuemean theapproachescount theof varianceThe

,1 and ,1 ,1 supposeNow

)( and

, photons received of # thecalling --- so t,independenlly statistica

is each of sion transmis, them)of ( photonsmany sendnow weIf

so and

. photons received of # thecall and photon, one Send

.y probabilit ion transmisshasfiber The

2

222

1

22

1

2

1

22

111

1

1

1

N

MppM

ppMMMpNEMNE

N

M

ppNNEpNEpNNE

N

p

N

NN

N


Thermal Noise as a Random Variable

R

kT/C. variancehas voltagenoise The

/

2/2/

2/

hence kT/2,energy mean has T tureat tempera

systema of freedom of degreet independenany amics, thermodynFrom

resistor. via theroom the

withmequilibriu thermalsestablisheit :power no dissipate can C

heat. of form thein room thehenergy wit exchange can R

T tureat tempera room) (a warm reservoir""a withmequilibriu in isresistor The

. resistor Ra toconnected is Ccapacitor A

2

2

CkTV

kTCV

kTE


random processes


Random Processes

v(t)is outcome particular The

).( is process random The

time.of function randomour is This

random.at sheet oneout Pick

space. sampley probabilit the

called is can garbage This

can. garbagea into Put them

time. vs. voltageof functions of

paper, of sheets separate on graphs, ofset a Draw

tV


dttvgtvgA

tvdtvftvgtvgE

t

gg

gdxxfxgxgE

V

X

))(())((

:over time function

theof average an define also can We

))(())(())(())((

: timeparticular someat space sample

over the average an define can we

,definition process randomour With

space. sample over the is average the where, of * valueaverage* theis

)()()(

X variablerandoma of g(X) functiona of nexpectatio theof definion theRecall

1111

1

Time Averages vs. Sample Space Averages


space" sample over the variationrandom"

toequivalent

time" with variationrandom"

made have wesense, some In

))(())((

space sample lstatistica over the averages

toequal over time averages

has process random ErgodicAn

1 tvgAtvgE

Ergodic Random Processes


Gaussian.jointly be not)might (or might They

on.distributiy probabilitjoint some have )( and )(

).( and )( valueshas )( process random the

and at times samples timeWith

21

21

21

tVtV

tVtVtV

tt

Time Samples of Random Processes

v(t)

t

v(t1 )

v(t2 )

t1

t2


Random Waveforms are Random Vectors

v(t)

tt1

t2

tn

class. thisof scope thebeyondmostly is This

geometry. and analysisvector

using signals random analyze thuscan We

vector.randoma into process randomour convert we

,... samples timespaced-regularlypick weif and

d,bandlimite is signal randoma if

theorem,sampling sNyquist' Using

1 ntt


)()(orderslower

)(),()(),(

:tystationariorder 2

orderslower ..and

)(),...,(),()(),...,(),(

:tystationariorder N

time.ithnot vary w do process stationarya of statistics The

11

2121

nd

2121

th

tVfEtVfE

tVtVfEtVtVfE

tVtVtVfEtVtVtVfE nn

Stationary Random Processes

v(t)

t

v(t1 )

v(t2 )

t1

t2


Restrictions on the random processes we consider

We will make following restrictions to make analysis tractable:

The process will be Ergodic.

The process will be stationary to any order: all statistical properties are independent of

time. Many common processes are not stationary, including integrated white noise and 1/f

noise.

The process will be Jointly Gaussian. This means that if the values of a random process

X(t) are sampled at times t1, t2, etc, to form random variables X1=X(t1), etc, then X1,X2,

etc. are a jointly Gaussian random variable.

In nature, many random processes result from the sum of a vast number of small

underlying random processses. From the central limit theorem, such processes can

frequently be expected to be Jointly Gaussian.


Variation of a random process with time

For the random process X(t), look at X1=X(t1) and X2=X(t2).

21212121 ),(•2121

dxdxxxfxxXXER XXXX

To compute this we need to know the joint probability distribution. We have assumed a

Gaussian process. The above is called the Autocorrellation function. IF the process is

stationary, it is a function only of (t1-t2)=tau, and hence

RXX E X t X t

this is the autocorrellation function. It describes how rapidly a random voltage varies

with time….

PLEASE recall we are assuming zero-mean random processes (DC bias subtracted). Thus

the autocorrellation and the auto-covariance are the same


Variation of a random process with time

Note that RXX 0 E X t X t X2

gives the variance of the random process.

The autocorrelation function gives us variance of the random process and the correlation

between its values for two moments in time. If the process is Gaussian, this is enough to

completely describe the process.

)(xxR

)(xxR

Narrow autocorrelation:

Fast variation

Broad autocorrelation

Slow variation


Autocorrelation is an Estimate of the Variation with Time

If random variables X and Y are Jointly Gaussian, and have zero mean,

then knowledge of the value y of the outome of Y results in a best estimate of X as

follows:

E X Y y X Y y RXY

Y2 y

"The expected value of the random variable X, given that the random variable Y has value

y is ..."

Hence, the autocorrellation function tells us the degree to which the signal at time t is

related to the signal at time t

A narrow autocorrelation is indicative of a quickly-varying random process


Power spectral densities

The autocorrellation function describes how a random process evolves with time.

Find its Fourier transform:

djRS XXXX )exp(

This is called the power spectral density of the signal.

Remembering the usual Fourier transform relationships, if the power spectrum is broad,

the autocorrellation function is narrow, and the signal varies rapidly--it has content at high

frequencies, and the voltages of any two points are strongly related only if the two points

are close together in time.

If the power spectrum is narrow, the autocorrellation function is broad, and the signal

varies slowly--it has content only at low frequencies and the voltages of any two points are

strongly related unless if the two points are broadly separated in time.


Power spectral densities

Rxx(tau)

tau

Rxx(tau)

tau

time

time

x(t)

x(t)

Sxx(omega)

omega

Sxx(omega)

omega


Power Spectral Densities

density" spectralpower " term,for the ionjustificat theis This

power. theus give lldensity wi spectralpower the

gintegratin thenprocess, theinpower thecalled is if So,

2

1)0(

ly,specifical

)exp(2

1

thatso holds, transforminverse The

)exp(

function ationautocorrel theof ransform Fourier tthe

isdensity spectralpower that theRecall

2

2

X

XXXXX

XXXX

XXXX

dSR

djSR

djRS


Correlated Random Processes

djSR

djRS

tYtXER

XYXY

XYXY

XY

)exp(2

1 thereforeand

)exp(

:follows asdensity spectral-crossa have They will

processes theof function oncorrellati-cross theDefine

Y(t).and X(t) processes random woConsider t

related.lly statistica be can processes Two


Single-Sided Hz-based Spectral Densities

dfjRjfS

dffjjfStXtXER

djRjS

djjStXtXER

XXXX

XXXX

XXXX

XXXX

)2exp(2~

)2exp(~

2

1)()(

DensitiesSpectral based- HzSided-Single

)exp(

)exp(2

1)()(

DensitiesSpectral Sided-Double


Single-Sided Hz-based Spectral Densities- Why ?

.frequency the toclose lying sfrequencieat

bandwidth signal of per Hz

power signal of Watts hedirectly t is ~

~~

2

1~

2

1Power

, bandwidth theinpower signal The

? notation Why this

high

low

high

low

low

high

highlow

f

jfS

dfjfSdfjfSdfjfS

ff

XX

f

f

XX

f

f

XX

f

f

XX


Single-Sided Hz-based Cross Spectral Densities

XYdf

djfS

dfjRjfS

dffjjfStYtXER

djRjS

djjStYtXER

XY

XYXY

XYXY

XYXY

XYXY

as writtenoften also is ~

)2exp(2~

)2exp(~

2

1)()(

DensitiesSpectral Cross based- HzSided-Single

)exp(

)exp(2

1)()(

DensitiesSpectral Cross Sided-Double


Example: Cross Spectral Densities

jfSjfSjfSjfS

jSjSjS

jSjSjSjSjS

RRRR

tYtXtYtXER

tYtXtV

XYYYXXVV

XYYYXX

XYXYYYXXVV

YXXYYYXX

VV

~Re2

~~~densities spectral sided-single in Or,

Re2

)()()()(

)()()(

*

X

RY

V

P=V2/R


Example: Cross Spectral Densities

relevant. isdensity spectral-cross that theNote

R.in dissipatedpower (expected) total thegives

relevant) is bandwidthever (over whatfrequency respect to withgIntegratin

...~

, and between bandwidth thein And

/0/)()(

valueexpected has /)( Power The 2

high

low

f

f

VV

highlow

VV

dfjfSP

ff

RRRtVtVE

RtVP X

RY

V

P=V2/R


Our Notation for Spectral Densities and Correlations

)2/(2)(~

)()()(

)()(

)2/(2)(~

)()(density spectral cross

)()()()()()(function lationcrosscorre

)2/(2)(~

)()()(

)()(

)2/(2)(~

)()(density spectralpower

)()()()()()(function ationautocorrel

)(),()(),(frequency of function

)()( timeof function

Outcome ProcessRandom

*

*

jSjfS

jyjxjS

RjS

jSjfS

RjS

tytvARtYtXER

jSjfS

jvjvjS

RjS

jSjfS

RjS

tvtvARtVtVER

jvjfvjVjfV

tvtV

xyxy

xy

xyxy

XYXY

XYXY

xyXY

vvvv

vv

vvvv

VVVV

VVVV

vvVV

FF

FF

)()()()( and )()()()(

processes ergodic stationaryFor

** jyjxjSjSjvjvjSjS xyXYvvVV

. tesimply wri can we),(or )(her clear whetit makescontext When vjvvtvv


Example: Noise passing through filters & linear electrical networks

Vin(t) h(t) Vout(t))()()(

)()()(

)()()()()(

)()()(

)()()(

)()()()()()(

So

)()()( ),()(any for then

),( function transfer and )( response impulse hasfilter theIf

**

2

2

***

jSjhjS

jSjhjS

jvjvjhjvjv

jSjhjS

jSjhjS

jvjhjvjhjvjv

jvjhjvtvtv

jhth

inininout

inininout

ininoutout

ininoutout

VVVV

vvvv

inininout

VVVV

vvvv

ininoutout

inoutoutin

densities. spectral based- Hzsided-single tochange to trivialisIt

Documents

ECE 145B / 218B, notes 2: Mathematics of Electrical Noise · Thermal Noise as a Random Variable R The noise voltage has variance kT/C. / / 2 / 2 / 2 at temperature T has mean energy