Identi–cation of Interconnected Systems by Instrumental ...staff.iiar.pwr.wroc.pl/grzegorz.mzyk/ect2012slides.pdf · Intro Single-elements Interconnected systems Hammerstein system

Intro Single-elements Interconnected systems Hammerstein system NARMAX system Summary

Identification of Interconnected Systemsby Instrumental Variables Method

Grzegorz Mzyk

Institute of Computer Engineering, Control and RoboticsWrocław University of Technology

Poland

3th-May-2012


Structure of the presentation

1 Identification of single-element systems—MISO linear static element—SISO linear dynamic elementLeast squares (LS) method and instrumental variables(IV) method

2 Interconnected linear static systems—LS-based estimate and limit properties— IV-based estimate and limit properties—generation of instrumental variables

3 Nonlinear dynamic block-oriented systems—Hammerstein system—NARMAX system












MISO linear static block

(1)x(2)x

( )sx

y*a

z

Figure: MISO linear static block

a∗ =

a∗1a∗2...a∗s

Assumptions:

Ez = 0, varz < ∞x (i ), z — independent !!!

XN =

xT1xT2...xTN

=x (1)1 x (2)1 .. x (s)1x (1)2 x (2)2 .. x (s)2...

.........

x (1)N x (2)N .. x (s)N


MISO linear static block (continued)

YN =

y1y2...yN

, ZN =z1z2...zN

Measurement equation

YN = XNa∗ + ZN

Model

YN (a) = XNa

Least squares criterion∥∥YN −YN (a)∥∥22 → mina

Normal equation

XTNXNa = XTNYN

Uniqueness of the solution

rankXN = s

LS estimate

aN=(XTNXN

)−1XTNYN = X

+NYN

aNp.1→ a∗, as N → ∞


FIR linear dynamics

kyku kvkε

)( 1−qB

Figure: Linear dynamic object MA(s)

vk = b∗0uk + ...+ b

∗s uk−s

yk = vk + εk

yk = b∗0uk + ...+ b

∗s uk−s + zk

zk = εk


FIR linear dynamics (2)

ku

ky*b

kz1ku −

k su −

Figure: MA object

b∗ =

b∗0b∗1...b∗s

Assumptions:

Ez = 0, varz < ∞{uk} , {zk} — independent !!!

ΦN =

φT1

φT2

...φTN

=u1 u0 .. u1−su2 u1 .. u2−s...

.........

uN uN−1 .. uN−s

YN = ΦNb∗ + ZN bN=

(ΦTNΦN

)−1ΦTNYN


IIR linear dynamics

kyku ( )( )

1

1

B q

A q

−

−

kvkε

Figure: Linear dynamic object ARMA(s,p)

vk = b∗0uk + ...+ b

∗s uk−s + a

∗1vk−1 + ....+ a

∗pvk−p

yk = vk + εk

yk = b∗0uk + ...+ b

∗s uk−s + a

∗1yk−1 + ....+ a

∗pyk−p + zk

zk = εk − a∗1εk−1 − ...− a∗pεk−p


IIR linear dynamics (2)

ku

ky*θ

kz1ku −

k su −

1ky −

k py −

Figure: ARMA object

θ∗ =

b∗0b∗1...b∗sa∗1a∗2...a∗p

Ez = 0, varz < ∞

uk−i , zk — independentyk−i , zk — correlated !!!


IIR linear dynamics (3)

ΦN =

φT1

φT2

...φTN

=u1 u0 · · · u1−s y0 y−1 · · · y1−pu2 u1 · · · u2−s y1 y0 · · · y2−p...

......

......

......

...uN uN−1 · · · uN−s yN−1 yN−2 · · · yN−p

YN = ΦN θ∗ + ZN θN=

(ΦTNΦN

)−1ΦTNYN


Instrumental variables approach

θIVN =

(ΨTNΦN

)−1ΨTNYN

Consistency conditions(a) dimΨN = dimΦN , i.e. ΨN = (ψ1, ...,ψN )

T ,dimψk = s + p + 1(b) PlimN→∞

( 1N ΨT

NΦN)exists and is not singular

(c) PlimN→∞( 1N ΨT

NZN)= 0

θIVN

p→ θ∗, as N → ∞

Generation of the instruments ψk

1) ψk = (uk , uk−1, ..., uk−s , uk−s−1, ..., uk−s−p)T

2) ψk =(uk , uk−1, ..., uk−s , y k−1, ..., y k−p

)T , where y k−i —model output (e.g. L.S.)


Instrumental variables approach

θIVN =

(ΨTNΦN

)−1ΨTNYN

Consistency conditions(a) dimΨN = dimΦN , i.e. ΨN = (ψ1, ...,ψN )

T ,dimψk = s + p + 1(b) PlimN→∞

( 1N ΨT

NΦN)exists and is not singular

(c) PlimN→∞( 1N ΨT

NZN)= 0

θIVN

p→ θ∗, as N → ∞

Generation of the instruments ψk

1) ψk = (uk , uk−1, ..., uk−s , uk−s−1, ..., uk−s−p)T

2) ψk =(uk , uk−1, ..., uk−s , y k−1, ..., y k−p

)T , where y k−i —model output (e.g. L.S.)


Interconnected systems

1u

1y

nuiu

11,ba

nn ba ,ii ba ,

H

nyiy

1x

nxix

1δ iδ nδ

1ξ

iξ

nξ

Figure: Interconnected MIMO linear static system


Identification of i-th element

yi = aixi + biui + ξ i (i = 1, 2, ..., n)

xi = Hiy + δi

YiN = (ai , bi )WiN + ξ i

YiN = [y (1)i , y (2)i , ..., y (N )i ]

WiN = [w (1)i ,w (2)i , ...,w (N )i ], where wi = (xi , ui )T ,


Least squares based approach

(al .s .i , bl .s .i ) = YiNWTiN

(WiNW

TiN

)−1WiN = [w

(1)i , w (2)i , ..., w (N )i ]

wi = (xi , ui )T , where xi = Hiy = xi − δi .

The estimation error

(al .s .i , bl .s .i )− (ai , bi ) = ΘiNWiN

(WiNW

TiN

)−1does not tend to zero, as N → ∞.


Instrumental variables estimate

(ai .v .i , bi .v .i ) = YiNΨTiN

(W TiNΨT

iN

)−1ΨiN = [ψ

(1)i ,ψ

(2)i , ...,ψ

(N )i ]

ψ(k )i =

(ψ(k )i ,1 ,ψ

(k )i ,2

)TTheoremThe optimal instruments with respect to the value of

Q(ΨiN ) = ‖∆(ΨiN )‖ = λmax(

∆(ΨiN )∆T (ΨiN ))

has the form

ψ∗i = w i = (x i , ui )T , where x i = E (xi |u) = HiKu.


Hammerstein system

ku kykz

kw( )⋅μ { } 0i iγ ∞

=kv

Figure: Hammerstein system

A1: |uk | 6 umax, ∃ p.d.f. ν(u)A2:

|µ(u)| 6 wmaxA3:

∞

∑i=0|γi | < ∞

A4: µ(u0) is known for someu0 and γ0 = 1

A5:

zk =∞

∑i=0

ωi εk−i

{εk} — i.i.d. process,independent of {uk}, E εk = 0,|εk | 6 εmax{ωi}∞

i=0 —unknown,∑∞i=0 |ωi | < ∞


Nonparametric regression

Regression function

E (yk |uk = u) = µ (u)

Kernel estimate

µ(u) =N

∑k=1

K (u − ukh(N)

) · yk

/N

∑k=1

K (u − ukh(N)

)

Orthogonal estimate (wavelet-based)

µ (u) =2M−1∑n=0

αMnϕMn (u) +K−1∑m=M

2m−1∑n=0

βmnψmn (u)

αMn =k

∑l=1

yl∫ ulul−1

ϕMn (u) du βmn =k

∑l=1

yl ·∫ ulul−1

ψmn (u) du


Parametric knowledge

ku kykz

kw( )*,cuμ { } 0i iγ ∞

=

kv

Figure: Hammerstein system (parametric model of the static nonlinearity)

given µ(u, c), such that µ(u, c∗) = µ(u), wherec∗ = (c∗1 , c

∗2 , ..., c

∗m)

T —true parametersµ(u, c) —differentiable with respect to cfor each u ∈ [−umax, umax] it holds that

‖5cµ(u, c)‖ 6 Gmax < ∞, c ∈ O(c∗)c∗ is identifiable, i.e. there exits the sequence u1, u2, ..., uN0such that

µ(un, c) = µ(un, c∗), n = 1, 2, ...,N0 ⇒ c = c∗


Nonlinearity estimation

QN0(c) = ∑N0n=1 [wn − µ(un, c)]

2 c∗ = argminc QN0(c)

Stage 1:wn,M = RM (un)− RM (0)

Stage 2:

QN0,M (c) =N0

∑n=1

[wn,M − µ(un, c)]2 → min

c


Limit properties

TheoremIf

RM (un) = R(un) +O(M−τ) in probability, as M → ∞

then

cN0,M = c∗ +O(M−τ) in probability, as M → ∞.


Two-stage identification of linear dynamics

ku kykz

kw( )μ ( )( )

1

1

B q

A q

−

−

kv

kε

{ } 0i iω ∞

=

Figure: Hammerstein system (parametric model of the linear component)

vk = b0wk + ...+ bswk−s + a1vk−1 + ....+ apvk−pθ = (b0, b1, ..., bs , a1, a2, ..., ap)T

ϑk = (wk ,wk−1, ...,wk−s , yk−1, yk−2, ..., yk−p)T

yk = ϑTk θ + zk , zk = zk − a1zk−1 − ...− apzk−pYN = ΘN θ + ZN , ΘN = (ϑ1, ..., ϑN )

T , ZN = (z1, ..., zN )T


Nonparametric instrumental variables

θ(IV )N ,M = (Ψ

TN ,M ΘN ,M )

−1ΨTN ,MYN

where

ΘN ,M = (ϑ1,M , ..., ϑN ,M )T

ϑk ,M = (wk ,M , ..., wk−s ,M , yk−1, ..., yk−p)T

ΨN ,M = (ψ1,M , ..., ψN ,M )T

ψk ,M = (wk ,M , ..., wk−s ,M , wk−s−1,M , ..., wk−s−p,M )T


Limit properties

TheoremIt holds that

θ(IV )N ,M → θ in probability

as N,M → ∞, provided that NM−τ → 0. In particular, forM ∼ N (1+α)/τ, α > 0, the asymptotic convergence rate is∥∥∥∥θ

(IV )N ,M − θ

∥∥∥∥ = O(N−min( 12 ,α)) in probability.


Optimal instruments

∆(IV )N (ΨN ) , θIVN − θ∗ Z ∗N ,

1√NZN

zmax

Q (ΨN ) , max‖Z ∗N‖2≤1

∥∥∥∆(IV )N (ΨN )∥∥∥22

TheoremIt holds that

limN→∞

Q(ΨN ) > limN→∞

Q(Ψ∗N ) with probability 1

where Ψ∗N = (ψ∗1,ψ

∗2, ...,ψ

∗N )T , and

ψ∗k = (wk , ...,wk−s , vk−1, ..., vk−p)T .


NARMAX system

ku ky

kz

kw( )µ { }n

ii 0=γ

( )η { }p

jj 1=λkw'

kv

kv'

Figure: The NARMAX system

Λ = (λ1, ..,λp)T

Γ = (γ0, ...,γn)T

c = (c1, ..., cm)T

d = (d1, ..., dq)T


Over-parametrization method

θ = (γ0c1, ...,γ0cm , ...,γnc1, ...,γncm ,

λ1d1, ...,λ1dq , ...,λpd1, ...,λpdq)T

= (θ1, ..., θ(n+1)m , θ(n+1)m+1, ..., θ(n+1)m+pq)T

φk = (f1(uk ), ..., fm(uk ), ..., f1(uk−n), ..., fm(uk−n),

g1(yk−1), ..., gq(yk−1), ..., g1(yk−p), ..., gq(yk−p))T

yk = φTk θ + zk YN = ΦN θ + ZN


Two-stage estimate

Stage 1. Instrumental variables

θ(IV)N = (ΨT

NΦN )−1ΨT

NYN

Θ(IV)Λd , and Θ(IV)

Γc of the matrices ΘΛd = ΛdT and ΘΓc = ΓcT

Stage 2. Singular value decomposition (S.V.D.)

Θ(IV)Λd =

min(p,q)

∑i=1

δi ξ i ζTi Θ(IV)

Γc =min(n,m)

∑i=1

σi µi νTi

Λ(IV)N = sgn(ξ1[κξ1 ])ξ1

Γ(IV)N = sgn(µ1[κµ1])µ1

c (IV)N = sgn(µ1[κµ1])σ1 ν1

d (IV)N = sgn(ξ1[κξ1 ])δ1 ζ1


Theorem

If det{EψkφTk } 6= 0 and Eψkzk = cov(ψk , zk ) = 0 then it holdsthat

Λ(IV)N → Λ

Γ(IV)N → Γ

c (IV)N → c

d (IV)N → d

with probability 1 as N → ∞.


Summary

1 Consistent estimates under correlated excitations anddisturbances

2 Problem decomposition with the use of nonparametricmethods

3 Broad class of models (non-linear-in-parameters static blocks+ I.I.R. linear filters)


Summary





Summary




Documents

Identi–cation of Interconnected Systems by Instrumental ...staff.iiar.pwr.wroc.pl/grzegorz.mzyk/ect2012slides.pdf · Intro Single-elements Interconnected systems Hammerstein system