Detection: chapter 3 Statistical detection theory I Example of detection

Detection: chapter 3

Statistical detection theory I

Natasha Devroye

[email protected]

http://www.ece.uic.edu/~devroye

Spring 2010

Example of detection

Example of estimation

Goals

• infer value of unknown state of nature based on noisy observations

Mathematically, optimally

model of hypothesis

H

NatureTransmission /

measurementProcessing

Noise

phenomenon

experiment

the ``truth’’

model of observationor transmission

process

decision rule

estimation function

mapping from

observation space

to decisions/

estimates

Detection example 1: digital communications

Source Encoder Channel Decoder Destination

Noise

10001010100010

Detect?

Detection example 2: Radar communication

Send

Receive

Hypothesis

Hypothesis

Detect?

Further examples

• Sonar: enemy submarine!

• Image processing: detect and aircraft from infrared images

• Biomedicine: cardiac arryhthmia from heartbeat sound wave

• Control: detect occurrence of abrupt change in system to be controlled

• Seismology: detect presence of oil deposit

Difference between detection and estimation?

• Detection:

• Estimation:

Discrete set of hypotheses

Right or wrong

Continuos set of hypotheses

Almost always wrong - minimize error instead

Estimation example 1: communications

• Pulse amplitude modulation (PAM)

Analog source Sampler Transmitter

Receiver?

Estimation example 2: Radar

Send

Receive

Hypothesis

Hypothesis

Estimate?

Our methods

• Will treat everything generally, with a unified mathematical representation

• Bias towards Gaussian noise

• Examples mainly drawn from communications / radar

Aside: “Classical” vs. “Bayesian”

• Hypotheses/parameters are fixed, non-random

• Hypotheses/parameters are treated as random variables with

assumed priors (or a priori distributions)

Classial

Bayesian

Course outline

ECE 531: Detection and Estimation

University of Illinois at Chicago, ECE

Spring 2010

Instructor: Natasha Devroye, [email protected]

Course coordinates: Tuesday, Thursday from 2-3:15pm in TH 208 (Taft Hall).

Office hours: Monday from 4-5:30pm in SEO 1039, or by appointment

Welcome to ECE 531! This course is a graduate-level introduction to detection and estimation theory,

whose goal is to extract information from signals in noise. A solid background in probability and

some knowledge of signal processing is needed.

Course Textbook: Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory, by

Steven M. Kay, Prentice Hall, 1993 and (possibly) Fundamentals of Statistical Signal Processing,

Volume 2: Detection Theory, by Steven M. Kay, Prentice Hall 1998.

Other useful references:

Harry L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, II, III, IV

H. Vincent Poor, Introduction to Signal Detection and Estimation

Louis L. Scharf and Cedric Demeure, Statistical Signal Processing: Detection, Estimation, and Time

Series Analysis

Carl Helstrom, Elements of Signal Detection and Estimation. It's out of print, so here's my pdf copy.

Notes: I will follow the course textbooks fairly closely, using a mixture of slides (highlighting the

main points and with nice illustrations) and more in-depth blackboard derivations/proofs in class. I

will post a pdf version of the slides as they become ready here, but the derivations will be given in

class only.

Topics: Estimation Theory:

General Minimum Variance Unbiased Estimation, Ch.2, 5

Cramer-Rao Lower Bound, Ch.3

Linear Models+Unbiased Estimators, Ch.4, 6

Maximum Likelihood Estimation, Ch.7

Least squares estimation, Ch.8

Bayesian Estimation, Ch.10-12

Detection Theory:

Statistical Detection Theory, Ch.3

Deterministic Signals, Ch.4

Random Signals, Ch.5

Statistical Detection Theory 2, Ch.6

Non-parametric and robust detection

Grading: Weekly homeworks (15%), Exam 1 = max(Exam1, Exam 2, Final) (20%), Exam 2 =

max(Exam 2, Final) (20%), Project (15%), Final exam (30%).

http://www.ece.uic.edu/~devroye/courses/ECE531/

1 of 3 1/11/10 8:50 PM

Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory, by Steven M. Kay, Prentice Hall, 1993

Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory, by Steven M. Kay, Prentice Hall 1998.



Spring 2010














Series Analysis





class only.








Detection Theory:









1 of 3 1/11/10 8:50 PM

Estimation: General Minimum Variance Unbiased Estimation

• Bias: (expected value of estimator - true value of data)

• MVUE:

Estimation: Cramer-Rao lower bound

• Lower bound on variance of ANY unbiased estimator!

• Usage:

• assert whether an estimator is MVUE

• benchmark against which to measure the performance of

an unbiased estimator

• feasibility studies

• Depends on?

NatureTransmission /

measurementProcessing

Noise

Estimation: linear models

• What’s a linear model and why is it useful?

• What can be said?

• Best Linear Unbiased Estimators (BLUE)

Estimation: Maximum Likelihood Estimation

• Alternative to MVUE which is hard to find in general

• Easy to compute - very widely used and practical

• What is the MLE?

• Properties?

Estimation: Least Squares

• Alternative estimator with no general optimality properties, but nice and

intuitive and no probabilistic assumptions on data are made - only need a

signal model

• Advantages?

• Disadvantages?

Estimation: Bayesian Estimation

• Parameter to be estimated is assumed to be random, according to some prior

distribution which models our knowledge of it

• Bayesian Minimum Mean Squared Error (MMSE):

• Applications to Gaussian noise / linear model

Estimation: Bayesian Estimation

• General risk functions - arbitrary “cost” functions

• Maximum a posteriori (MAP) estimation

• Linear MMSE: constrain estimator to be linear - very practical

Course outline



Spring 2010














Series Analysis





class only.








Detection Theory:









1 of 3 1/11/10 8:50 PM

Fundamentals of Statistical Signal Processing, Volume 1: Estimation Theory, by Steven M. Kay, Prentice Hall, 1993

Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory, by Steven M. Kay, Prentice Hall 1998.



Spring 2010














Series Analysis





class only.








Detection Theory:









1 of 3 1/11/10 8:50 PM

Detection: Statistical Detection Theory

• Binary hypothesis testing

Hypothesis

Hypothesis


• Binary hypothesis testing


Detection: Deterministic Signals

• How to detect known signals in noise?

• The famous matched filter!

Xx[n]

T(x)

s[n]

• Generalized matched filter

• > 2 hypotheses

Detection: Random Signals

• What if s[n] is random?

• Key idea behind estimator-correlator:

• Linear model simplifies things again...

Detection: Statistical Decision Theory II

• model for the pdfs under 2 hypotheses are unknown

• Uniformly most powerful test

• Generalized likelihood ratio test

• Bayesian approach

• Wald test

• Rao test

Detection Theory

Binary hypothesis testing

Binary hypothesis testing

Neyman-Pearson hypothesis testing

Example 1

Useful problem 2.1

Example 2

Example 3

Deflection coefficient

Receiver Operating Characteristics (ROC)

[different terminology - can you map it to ours?]

Bayesian risk

Bayesian risk

Bayesian risk example

Multiple hypothesis testing

Multiple hypothesis testing

Multiple hypothesis testing example

Documents

Detection: chapter 3 Statistical detection theory I Example of detection