Embed Size (px)
Citation preview
Estimation: chapter 2+3
Minimum variance unbiased estimation + the CRLB
Natasha [email protected]://www.ece.uic.edu/~devroye
Spring 2011
• Find a few estimators
Estimation: a first example
• Estimate the DC level, A, of a signal given noisy measurements x[0], x[1], ... x[N-1] where
x[n] are samples of this!
• Compare their performance { • mean?• variance?• pdf?
Estimation: a first example
• Estimators of the DC level, A
Estimation: definitions
Estimation: definitions
How would you pick a ``good’’ estimator?
Vector versions....
Minimum variance unbiased estimation
Why?
So?
Minimum variance unbiased estimation
Give a counter-example! (b1pg.20)
The Cramer-Rao Lower Bound
• the CRLB give a lower bound on the variance of ANY UNBIASED estimator
• does NOT guarantee bound can be obtained
• IF find an estimator whose variance = CRLB then it’s MVUE
• otherwise can use Ch.5 tools (Rao-Blackwell-Lehmann-Scheffe Theorem and Neyman-Fisher Factorization Theorem) to construct a better estimator from any unbiased one - possibly the MVUE if conditions are met
The Cramer-Rao Lower Bound (CRLB)
• Use?
• Intuition?
The Cramer-Rao Lower Bound (CRLB)
CRLB examples
CRLB proof
CRLB T or F
What is I(θ)?
• Why “information”?
• non-negative• additive for independent observations
Vector form of the CRLB
Vector form of the CRLB
Vector form of the CRLB examples
What can we conclude?
CRLB for transformations
Vector CRLB for transformations
Example of vector CRLB with transformation
CRLB for General Gaussian Case
• When observations are Gaussian and one knows the dependence of the mean and covariance matrix on the unknown parameters, we know the closed form of the CRLB (or Fisher information matrix):
CRLB for Gaussians examples
Example: range estimation
!"#$
%&'(
)*+
,)+)+)+ !"
#"
! $$##%#"#&
!
!!
!
"#$$"%#!"!#$
-.&/0%12345607833 "+#)333333/9/:7.939;7.3#&<,($"=
>7?71;73>7@67?219/8333 "+#'( !!)
A7&05.73-1%73B76&C8 !!
!"#$%&'()*$+,-.*
D&/E61%127E
FG1273H&5001&/
#$"
"+#)
#$
"+#''( !!)D4I
J3K%L
&+#)
/%0$,1$!2"3
)'*(*
#$45
678$9$:)('.$6;<&=)<&,($/>,?*.=
Continuous time signal model
!"#$
%&'(
)*+
,)+)+)+ !"
#"
! $$##%#"#&
!
!!
!
"#$$"%#!"!#$
-.&/0%12345607833 "+#)333333/9/:7.939;7.3#&<,($"=
>7?71;73>7@67?219/8333 "+#'( !!)
A7&05.73-1%73B76&C8 !!
!"#$%&'()*$+,-.*
D&/E61%127E
FG1273H&5001&/
#$"
"+#)
#$
"+#''( !!)D4I
J3K%L
&+#)
/%0$,1$!2"3
)'*(*
#$45
678$9$:)('.$6;<&=)<&,($/>,?*.=
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
Example: range estimation
Discrete time signal model
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
%&'()*+,-*./+! 0+#"1!"2*3
#4$5+0+#6$!7
89+:;<=*+
>&?22<&@+AB<2*
C&.+"! " !"#
#DD#DE547654 #$%#!$ %$$#$&$' ( !&
)"
#$%&'(&$)%*
&#"1!
!*!
#"! !"1!
+,-&'(&$)%*
"#.! "! " !"
#
''(
'')
*
#++%
#%++%#!
#++
$
#54
#5476
#E54
54
%$+$$#
+$$$$#$&
$$$#
$'
(
(((
(
&
&4$F&(5G ;&2++ @B@HI*.B+2&'()*2+2=&.=<@J+&=+$(
/0123&45678067'1&-9:&,7210;&<'=3;
!"#$
%&'()*+,-*./+! 0+#"1!"2*3
#4$5+0+#6$!7
89+:;<=*+
>&?22<&@+AB<2*
C&.+"! " !"#
#DD#DE547654 #$%#!$ %$$#$&$' ( !&
)"
#$%&'(&$)%*
&#"1!
!*!
#"! !"1!
+,-&'(&$)%*
"#.! "! " !"
#
''(
'')
*
#++%
#%++%#!
#++
$
#54
#5476
#E54
54
%$+$$#
+$$$$#$&
$$$#
$'
(
(((
(
&
&4$F&(5G ;&2++ @B@HI*.B+2&'()*2+2=&.=<@J+&=+$(
/0123&45678067'1&-9:&,7210;&<'=3;
Example: range estimation
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
%&'()**+,(-.)/0)10(2345(16-7+.(8&1(-9:/)+(;(<=%>
!!
!!
"
# $#
"%
# "$#
"%
#
"
#
&&'
())*
+
,
,#
&&
'
(
))
*
+
,
,
#
&&'
())*
+
,
"$,#
&&'
())*
+
,
,-
#
?
@
@
#@
@
# @
@
#
?
@
@
ABAB
ABCDE
AFG)1B
!
" "#
!"
"" "#
!"
"" $
$
%
" $
$
$
#
#&
#
#&
"&"&
$
$ $
$
$
..
/
/
.
/
/
./
/
H+7:(9/I()/0(J66*(
/&/KL61&(.61M-
NO*+&9.(2)+P7+7-QQQ R-6()**1&O9M).9&/>(/$'('$ "
$
ST6/(0&(PT)/:6(&8(G)19)U+6-QQ
!"#$%&'()*+")*,#&-!./
Example: range estimation
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
!
"
#
!
"
#
"#
$
%&&
&!
'
$%&
&
&!
'
%&&
&! !!!
!!! "#
$%&
'((
)
"#
$%&
'((
)
"#
$%&
'((
*
+
%
&
%
&
%
&
&
'(
&"
#
'(
&"
'(#')*+,(
,-
-../012.+03412.3+567826.2.0+449 +33,:;<2./02=>267?18,+49
!)!"
! %&&!$%
& '(
. /
. /
!!
!
0
01
0
01
)
+
%&()
%(()(
'
$
$
%(()(
'
$
#
!
!
"
#
!
#!
&
&&
%
&&
'(
'(&
&"
#
'(&
&"
#')*+,(
2
2- @A2AB1:,10
C2D+,.1*+4
D+,.1*+4. /
!
!0
01
0
01)%&()
%(()(*+,!
&
&&
'(
'(&2
!"#$%"&'&()&*"'+,-".
*+,! 6.2EFGH2IJK22(LM'-2?>312:N2EHOFK
/'%0"&1+2$*'2$3%&4/5(&673%289
Example: range estimation
Taken from http://www.ws.binghamton.edu/fowler/fowler%20personal%20page/EE522.htm
!"#$
%&'()*+,-&-*'.-/&*0-*/11'2-*/+*+,-*3456*7(*+,-*.-8/9:
;<&-=#
;>2/1< ?
?!"#
$
%&'( !"#
@7*)-+*+,-*3456*7(*+,-*!"#$%A*B&-*C+1/(&DE*7D*F/1G&H*1-&B8+:*
;<GI"
;>2/1< ?
?
?
!"#%&'(
)(
!"
$*(+%
(*(+%
$( ##
>
?
> $$%
&''(
)
*
*+ 0'+,* ( J*)#
$,"*?
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
L7*+,-*3456*+-88&*B&A
K 3,77&-*&')(/8*0'+,*8/1)-*%!"#
K P(&B1-*+,/+*LM4*'&*8/1)-
K 6-++-1*7(*M-/1Q9"8/1)-*+/1)-+&
K O,'=,*'&*Q-++-1R**
S T7BQ8-*+1/(&G'++-.*-(-1)9R
S T7BQ8-*4NL*Q/(.0'.+,R
&"#$%'()*+,"*+-#'.&/0'12-#*34
!"#$
%&'()*+,-&-*'.-/&*0-*/11'2-*/+*+,-*3456*7(*+,-*.-8/9:
;<&-=#
;>2/1< ?
?!"#
$
%&'( !"#
@7*)-+*+,-*3456*7(*+,-*!"#$%A*B&-*C+1/(&DE*7D*F/1G&H*1-&B8+:*
;<GI"
;>2/1< ?
?
?
!"#%&'(
)(
!"
$*(+%
(*(+%
$( ##
>
?
> $$%
&''(
)
*
*+ 0'+,* ( J*)#
$,"*?
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
3456*'&*'(2-1&-89*F17F71+'7(/8*+7:
K LM4*N-/&B1-
K 4NL*6O*N-/&B1-
L7*+,-*3456*+-88&*B&A
K 3,77&-*&')(/8*0'+,*8/1)-*%!"#
K P(&B1-*+,/+*LM4*'&*8/1)-
K 6-++-1*7(*M-/1Q9"8/1)-*+/1)-+&
K O,'=,*'&*Q-++-1R**
S T7BQ8-*+1/(&G'++-.*-(-1)9R
S T7BQ8-*4NL*Q/(.0'.+,R
&"#$%'()*+,"*+-#'.&/0'12-#*34
