@G. Gong 1

Chapter 5. Signal Representation and Detection (Chapter 3 in the text book)

Chapter 5. Signal Representation and Detection (Chapter 3 in the text book)

Consider digital transmission systems that are, in general, M-ary.

1. Conversion of Waveform to Vector Space 2. Geometric Interpretation of Signals3. Demodulation4. Response of Bank of Correlators to Noisy Input5. Detection of Signals in Noise6. Probability of Error for Signals in AWGN7. Matched Filters and Signal-to-Noise Ratio (SNR)

Maximization

Message sink

}{ im

}1,...,1,0{ −∈ Mmi

Vector encoder Modulator

To Channel:

(one message each Tm s)

(one signal each Ts s)

n(t)

( )iNiii sss ,...,, 21=sim

},...,2,1|)({ Mitsi =

)(tsi

DemodVector detector

( )iNiii rrr ,...,, 21=rim̂

Decision: minimum the prob. of error Figure 1.

)()()( tntstr ii +=

channel:AWGN +

)}({ tri

Ts

Message sink

mM

m TR /)(log2=

@G. Gong 3

iM

mPp ii allfor 1}emitted { ==

Message source:One message symbol mi each Tmseconds, there are m different symbols and all of them occur equally likely, ( )iNiiii sssm ,...,, 21=sa

Vector encoder:Mapping a symbol to a real-valued vector of dimension N ≤ M,

∫ ∞

)(2

)( and 0)]([ 0 τδτN

RtnE n ==

Waveform channel:

LTI system, bandwidth accomodates si(t) without distortion, and noise is added.

where n(t) is an additive white Gaussian noise.

sii Tttntstr

A. Review of Inner product (Euclidean) space

1. Conversion of Waveforms to Vector Space Representation

1. Conversion of Waveforms to Vector Space Representation

Inner product:

V∈>< βααββα , ,, , *(1) Symmetry:

)0 iff 0( 0 , ==≥>< ααα(2) Positive definite:

CbaVbaba ∈∈>=+< , ,,, ,,,, 212121 βααβαβαβαα

(3) Linearity:

Consider V, a linear space over the complex number field C.

@G. Gong 6

B. Signal Space Representation

∫=><T

dttytxtytx0

)()( )(),(

Definitions.

Inner product of two real-valued signal x(t) and y(t) defined on the interval [0, T] is defined by

0)( )()(),()(2/1

0

22/1 ≥=⎥⎦⎤

⎢⎣⎡=>=< ∫ Edttxtxtxtx

T

Norm (or length) of a signal x(t) is defined by

Energy of the signal

@G. Gong 7

Orthogonal set: A set of N signals waveforms is called orthogonal iff

{ }Njtj ≤≤1|)( φ

⎩⎨⎧

≠=

>=<jijic

tt jji ,0 ,

)(),( φφ

∫ ⎩⎨⎧

≠==

Tji ji

jidttt0 ,0

,1)()( φφ

If , unit energy, then the signal set is called orthonormal, i.e.,

1 , i.e. , 1 =∀= jj Ejc{ } )( tjφ

).(on

)( of projection theis which gives )()( :Note0

t

tssdttts

j

iijT

ji

φ

φ∫

C. Gram-Schmidt Orthogonalization Process

∑=

≤≤==N

jjiji TtMitsts

1

0 , ..., ,2 ,1 , )()( φ

{ }Mitsi ≤≤1|)(

{ } MNNjtj ≤≤≤ where1|)(φ

∫>==<T

jijiij dtttsttss0

)()()(),( where φφ

Any signal si(t) in a set of M energy signals can be represented by a linear combination of a set of Northonormal functions as

(1)

@G. Gong 9

Thus, using the set of the basis functions, each signal si(t) maps to a set of N real numbers, which is a N dimensional real-valued vector

( )iNiii sss ,...,, 21=s

( )⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

=

)(

)()(

,...,,)( 21

21

t

tt

sssts

N

iNiii

φ

φφ

M

The representation (1) has the following matrix form:

This is a 1-1 correspondence between the signal set (or equivalently, the message symbol set) and the N dimensional vector space.

@G. Gong 10

)()()(),( where

)()(

such that ,...,1 and 0for

functions basis lorthonorma :)}({

0

1

∫

∑

>==<

=

=

@G. Gong 11

Procedure

energy.unit has )( and where

)()()(

1

111

111111

tEs

tstEts

φ

φφ

=

==⇒

∫==

=

=

TdttsE

Etstgtgt

tstg

0

211

1

1

1

11

11

)( )(

length)(unit )()( )(

)(direction )( )(

φ

Step 1.∫=

Tdtttss

0 1221)()( φ

Step 2. Compute:

))()(( )( )( )( 1212122 ttgtststg φφ ⊥−=

)(direction 0 )(),( 12 =>

2212

221

2212

0

222

2

)()(

sE

ssE

dttgtgT

−=

+−=

= ∫

Compute the norm of g2(t):

)( 0

222 ∫=

TdttsE

)()()(

2

22 tg

tgt =φ

Set

2212

1212 )()(sE

tsts−

−=

φ

1)(||)(||0

222 == ∫

Tdttt φφ

We have

0 )()( and 0 21

=∫T

dttt φφ

∫=T

dtttss0 1221

)()( φ

Step 2.

Compute:

))()(( )( )( )( 1212122 ttgtststg φφ ⊥−=

)(direction 0 )(),( 12 =>

@G. Gong 13

)(direction )()()(1

1∑−

=

−=n

jjnjnn tststg φ

∑

∑−

=

−

=

−

−=

1

1

2

1

1)()(

n

jnjn

n

jjnjn

sE

tsts φ

11, )(),( ,...,n- jttss jnnj =>=< φStep n. Compute:

length)(unit )()()(

tgtgt

n

nn =φ

∑−

=

−=1

1

2||)(||n

jnjnn sEtg ))()((

0

22 ∫==T

nnn dttstsE

)(),...,(byfor accountedalready

)( of components

11 tt

ts

n

n

−φφ

@G. Gong 14

.0)( gives that )(signalany skipjust , case In this ).(),...,(

ofn combinatio aby for accountedalready not component no has )( because 0)( So,

0)()(that happen may It

processed. are )(),...,( signals all until proceeded is procedure This

11

1

1

1

=

=

=−

−

−

=∑

ttstt

tst

tsts

tsts

nn

n

nn

n

jjnjn

M

φφφ

φ

φ

@G. Gong 15

)(1 ts

t0

1

2

)(2 ts

t0

12

1-1

0

1

31-1

)(3 ts

t

)(4 ts

t0

1

3

Example. A set of four waveform is illustrated as below. Find an orthonormal set for this set of signals by applying the Gram-Schmidt procedure.

@G. Gong 16

30 , 2/)()( 11 ≤≤= ttstφ

Solution.

Step 1. 3 , 2 1 )()( 2

0

3

0

210

211 ===== ∫∫∫ TdtdttsdttsE

T

Step 2.

0 )11( 1 2

1

)()()()(

2

0

2

0

3

0 120 1221

=⋅−+=

==

∫∫

∫∫dtdt

dtttsdtttssT

φφ )( )( 12 tts φ⊥⇒

)211)( (

2/)( /)()(

0

222

2222

=+==

==

∫T

dttsE

tsEtstφSet

@G. Gong 17

)(2)(||)(||

)()( 233

33 ttstg

tgt φφ +== )( and )( of

n combinatiolinear a is )( , i.e.

31

4

ttts

φφ

Step 3. Step 4.

2)()(

0)()(

0 2332

0 1331

−==

==

∫∫

T

T

dtttss

dtttss

φ

φ

)(2)()( 233 ttstg φ+=⇒

{ } 1)()( 2/10

233 == ∫

Tdttgtg

1 ,0

2)()(

4342

0 1441

==

== ∫ss

dtttssT

φ

0)()(2)()( 3144 =−−=⇒ tttstg φφ

))()()()( ( 3214 tstststs ++=

@G. Gong 18

2

21

0 t

)(1 tφ

t

)(3 tφ

20 3

1

2

21

0 t

)(2 tφ

21

−1

Thus is an orthonormal set. })( ),( ,)({ 321 ttt φφφ

2. Geometric Interpretation of Signals2. Geometric Interpretation of Signals

A. Signal Sets and Signal Constellations

Definition 1. Let be a signal set,be an orthonormal set and

{ }MitsS i ≤≤= 1|)( { }Njtj ≤≤1|)(φ

∑=

≤≤==N

jjiji TtMitsts

1

0 , ..., ,2 ,1 , )()( φ

RdtttsttssT

jijiij ∈>==< ∫ )()()(),( where 0 φφ

Then NiNii Rsss ∈= ),,,( 121 Ls

where R is the real number set.

is called the vector representation of the signal si(t) and {si, i = 1, … , M} is called a signal constellation of the signal set S.

Remark. RN is the set consisting of all vectors of dimension N whose entries are taken from the real number set R, which is a Euclidean space of dimension N. Thus si is a point in the Euclidean space RN.

@G. Gong 20

Example 1. Find the vector representation and the signal constellation of the signal set in the example of Section 1.

Solution. From the example in Section 1, we have

)0,0,2()(2)()( 111111 =⇒== sttsts φφ

)0,2,0()(2)( 222 =⇒= stts φ

)1,2,0()()(2)( 3323 −=⇒+−= sttts φφ

)1,0,2( )()(2

)()()()(

4

31

3214

=⇒+=

++=

stt

tstststsφφ

)(1 tφ

)(2 tφ

)(3 tφ

2

1s

2s

4s3s

2

@G. Gong 21

t

)(1 tφ

0

1

1

20 t

)(3 tφ

3

1

)(2 tφ

20 t1

1

( ) ( )( ) ( )1,1,1 , 1,1,1

, 0,1,1 ,0,1,143

21=−=

−==ss

ss

Remark. The basis function set is not unique, thus t he vector representation of a signal set is not unique. For example, the following set is another basis function set for the signal set in the above example. Under this set, the vector representation of the signals are

},...,1|)({ Njtj =φ

)(1 tφ

)(2 tφ

)(3 tφ

1s2s

4s

3s

1

1

1

@G. Gong 22

Example 2. (a) Manchester encoding (this method is used in many commertial disk storage product and also in what is known as 10BT or Ethernet. )

Let

⎪⎩

⎪⎨⎧ ≤≤=

otherwise 02/0for 2)(1

Tt /Ttφ⎪⎩

⎪⎨⎧ ≤≤=

otherwise 02for 2)(2

Tt T/ /Ttφ

Then is an orthonormal set. The Manchester encode is as follows.

)}(),({ 21 tt φφ

)()()( 211 ttts φφ −= )1,1(1 −=⇒ s

)()()( 212 ttts φφ +−= )1,1(2 −=⇒ s

Remark.The data rate equals one bit per T seconds, for a data transfer rate into a disk of 24 Mbytes/s or 192 Mbps, T = 1/(192Htz), for a data rate of 10Mbps in “Ethernet”, T = 100ns.

@G. Gong 23

Figure 2. Manchester (Ethernet) basis functions and waveforms

TT/2 t

T/2T/1

TT/2 t

T/2T/1

)(2 ts

)(1 ts

TT/2 t

T/2T/1

TT/2 t

T/2T/1

)(2 tφ

)(1 tφ

)(1 tφ

)(2 tφ

1s

2s Figure 3. Manchester signal constellation

@G. Gong 24

Example 2. (b). Binary phase-shift keying (BPSK) (used in some satellite and deep-space transmissions).

⎪⎪⎪

⎩

⎪⎪⎪

⎨

⎧≤≤⎟

⎠⎞

⎜⎝⎛ +

=otherwise0

0 if4

2cos2

)(1

TtT

tT

t

ππ

φ

⎪⎪⎪

⎩

⎪⎪⎪

⎨

⎧≤≤⎟

⎠⎞

⎜⎝⎛ −

=otherwise0

0 if4

2cos2

)(1

TtT

tT

t

ππ

φ

Then is an orthonormal set (verify this). BPSK signals are )}(),({ 21 tt φφ

Tt

Tπ2sin2−=)()()( 211 ttts φφ −= )1,1(1 −=⇒ s

)()()( 212 ttts φφ +−= )1,1(2 −=⇒ sTt

Tπ2sin2=

@G. Gong 25

TT/2 t

T/2T/1

TT/2 t

T/2T/1

TT/2 t

T/2T/1

)(2 tφ

TT/2 t

T/2T/1

)(2 ts

Figure 4. BPSK basis functions and waveforms

)(1 ts)(1 tφ

)(1 tφ

)(2 tφ

1s

2s Figure 5. BPSK signal constellation

@G. Gong 26

Remark. Important phenomenon: BPSK have the same constellations as Manchester, shown in Figure 3, although the basis functions differ from the previous. The common vector space representation (i.e., signal constellation) of the Ether net and BPSK examples allows the performance of a detector to be analyzed for either systems in the same way, despite the gross differences in the overall systems.

In either of the systems in Example 2, (a)-(b), a more compact representation of the signals with one basis function is possible (leave it as an exercise).

@G. Gong 27

Example 3. (ISDN -2B1Q) For the quaternary signaling, we have N = 1 and M = 4. The integrated services digital network (ISDN) transmits 2bits/s and uses a basis function that is the Nyquist pulse shape

TtTt

Tt ≤≤⎟

⎠⎞

⎜⎝⎛= 0 ,csin1)(1φ

where 1/T = 80khz. The signal constellation is as follows.

)(1 tφ-3 -1 1 3

Figure 6. 2B1Q signal constellation

(2 bits are transmitted using one 4 level (or quaternary ) symbol every Tsecond, hence the name 2B1Q.)

Remark. Telephone companies, R = 1.544Mbps. A method, known as HDSL (high-bit-rate digital subscriber lines) uses 2B1Q with 1/T = 392 kHz. Thus it transmits a data rate of 784 kbps on each of two phone lines for a total of 1.568 Mbps (1.544 Mbps plus 24 kbps overhead).

@G. Gong 28

B. Properties of Inner Products

We have mapped a signal to a vector in the linear space RN, i.e.,

∑=

>=<N

kjkikji ss

1

, ss

ii ts sa)( The inner product of si and sj is defined as

Theorem 1 (invariance of inner product). >>==<T

jiji dttstststs0

)()()(),(

Note that )()(,

ttss rrk

kjrik φφ∑=

∫>=⇒<T

ji tsts0

)(),( dtttss rrk

kjrik )()(,

φφ∑ dtttss rrk

k

T

jrik )()(, 0

φφ∑ ∫= ∑= k jkikss

⎩⎨⎧

≠=

=∫ rkrk

dtttT

rk 01

)()(0

φφsince>=< ji ss ,

@G. Gong 29

Remark. We introduce the notation of the Kronecker delta function:

⎩⎨⎧

≠=

=jiji

ij 01

δ

Thus the orthonormal set can be characterized as},...,1|)({ Njtj =φ

ij

T

ji dttt δφφ =∫0

)()(

>=

@G. Gong 30

Property 1. (The squared distance between two signals) The (Euclidean) squared distance between two signals is defined as si and sj

22 |||| jiijd ss −=

Then ∑∫ −=−=−=k

jkikT

jijiij ssdttstsd2

0

222 )())()((|||| ss

Or equivalently, ∑−+=k

jkikjiij ssd 2||||||||222 ss

(3)

(4)

Definition 2(average energy). The average energy of a signal constellation is defined by

∑=

==M

iiiS PEE

1

2 }{||||||][|| sss

@G. Gong 31

{ }im { } is { })(tsisource Vectorencoder { }Njj ≤≤1|

modulator φ

transmitter

{ }Mi ,...,1∈

( )iNii ss ,...,1=s

×

)(tNφ

×

)(1 tφ

iNs

2is

1is

×)(2 tφ

+ )(tsi

...

Figure 6. Implementation of generating the signal set { })(tsi

@G. Gong 32

3. Demodulation3. Demodulation

The computation of the coordinates of the received signal is accomplished by the parallel bank of multiplier-integrators. Each multiplier-integrator combination is referred to as a correlator. The overall structure is called a correlative demodulation (or correlative receiver). See Figure 1 (for simplicity, in the following, we will write y(t) = ri(t)).

)()()( tntntn pr +=Remark. The noise cannot, in general, be represented in the vector space. But we always can have the decomposition: where nr(t) can be represented by a vector under the N basis functions and np(t) may be ignored in making the decision as to which signal was sent.

@G. Gong 33

Figure 1. Correlative demodulator

)(2 tφ

)(1 tφ

)(tNφ

)(ty

1y

2y

Ny

∫T

dt0

∫T

dt0

∫T

dt0

×

×

×

)n vectorobservatio(y

∫=T

ii dyy 0 )()( ττφτ

An interesting phenomenon:

Tti dtTy

=

∞

∞−∫ +−= ττφτ )()(

TtitTty

=−∗= )( )( φ

Thus a correlator can be implemented by a matched filter. The component of the received waveform y(t) along the ith basis function is equivalently the convolution of the waveform y(t) with a filter at output sample time T. This is so called the matched-filter demodulation, which is “matched” to the corresponding demodulator basis functions. See Figure 2.

)( tTi −φ

)(tiφ

)(ty iy )(ty)(tiΓ

ii yT =Γ )()( tTi −φ=∫

bT dt0×

Matched filterCorrelator

)( )()( tTtyt ii −∗=Γ φ

∫∞

∞−= ττφτ dy i )()(

@G. Gong 35

Figure 2. The matched-filter demodulator

)(ty

)(1 tT −φ 1y

)( tTN −φ Ny

)(2 tT −φ2y

@G. Gong 36

4. Response of Bank of Correlators to Noisy Input4. Response of Bank of Correlators to Noisy Input

AWGN channel

+)(tsi )()()( tntstr ii +=

MiTt

,...,2,10=

≤≤

)(tn

2/0Nwhere n(t) is white Gaussian noise with zero mean and power spectral density

A. Decision Vector

)(tjφ

)(tri

∫T

dt0

×

Figure 3. jth correlator (similar to the jth matched filter)

jij

Tji

Tjiij

ns

dtttnts

dtttrr

+=

+=

=

∫∫

)()]()([

)()(

0

0

φ

φ

∫=T

jiij dtttss0

)()( φ

∫=T

jj dtttnn0

)()( φ

where

@G. Gong 38

Since nj is a Gaussian random variable and sij is constant, then rij is a Gaussian random variable.

Proof.

Property 1. The variable rij is a Gaussian random variable whose mean and variance are given by

ijij srE =][2

02 Nij =σ

kjrrCov ikij ≠= for 0)( ,

Furthermore, rij and rik are uncorrelated, therefore, independent

and

ijjijjijijr snEsnsErEij =+=+== ][][][μ

then,0][0)]([ and )()( Since0

=⇒== ∫ jT

jj nEtnEdtttnn φ(a)

)])([(),(ikij rikrijikij rrErrCov μμ −−= )])([( ikikijij srsrE −−= ][ kjnnE=

⎥⎦⎤

⎢⎣⎡ ⋅= ∫∫

T

k

T

j duuundtttnE 00 )()()()( φφ ∫ ∫=T

nkjT

dtduutRut0 0

),()()( φφ

∫ ∫ −=T

kjT

dtduututN

0 00 )()()(

2δφφ

2)()(

20

00

jkT

kjN

dtttN

δφφ == ∫

If , then kj ≠ 0),( =ikij rrCov

If , then 2),( 02

NrrCov ijijijr ==σ kj =

Thus are mutually uncorrelated (i.e., any two of them are uncorrelated). Since these random variables are Gaussian random variables, then they are independent.

{ } 1|)( Njtrij ≤≤

@G. Gong 40

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

iN

i

i

i

r

rr

M2

1

Define r

.2/ varianceand mean with variablesrandomGaussian t independen are ,...,2,1 , where

0NsNjr

ij

ij =

B. Likelihood Functions of AWGN

For simplicity, in the following, we will write

)( )( ii mts ⇒

),,,( 21 Ni XXX L== rX

),,,( 21 Nxxx L=xand is a sample of X, i.e., xi is a sample of Xi. Next, we investigate the conditional probability density function (pdf) of X given that the signal was transmitted. We denote it as

)|(| iM mf i XX

@G. Gong 41

)|(1

|∏=

=N

jijMX mxf ij (by independence)

2

)(exp

21

12

2

∏= ⎥

⎥

⎦

⎤

⎢⎢

⎣

⎡ −−=

N

j X

Xj

X j

j

j

x

σ

μ

σπ ∏= ⎥

⎥⎦

⎤

⎢⎢⎣

⎡ −−=

N

j

ijj

Nsx

N1 0

2

0

)(exp1

π

)(1exp)( 1

2

0

2/0

⎥⎥⎦

⎤

⎢⎢⎣

⎡−−= ∑

=

−N

jijj

N sxN

Nπ

)|,...,,( )|( 21|,...,,| 21 iNMXXXiM mXXXfmf iNi ΔXX

This conditional pdf is defined by

)|(| iM mf i XX

Thus we have

MiN

N iN ,...,2 ,1 , 1exp)( 2

0

2/0 =⎥

⎦

⎤⎢⎣

⎡−−= − sxπ

(recall that x = ri )

(1)

@G. Gong 42

process.making-decision in theerror ofy probabilit theminimize

wesuch that way ain , symbol tted transmi theof ˆestimatean to from mapping a perform tohave we

, n vector observatio eGiven th : problem Detection

ii

i

i

mmr

r

/1)( mmP i =If the symbol mi occurs equally likely, i.e., , minimize Pe is equivalent to maximize the likelihood function.

The function represented by (1) is called the likelihood function of the AWGN.