Linear multivariable systems : preliminary problems inmathematical description, modelling and identificationCitation for published version (APA):Hajdasinkski, A. K. (1980). Linear multivariable systems : preliminary problems in mathematical description,modelling and identification. (EUT report. E, Fac. of Electrical Engineering; Vol. 80-E-106). TechnischeHogeschool Eindhoven.

Document status and date:Published: 01/01/1980

Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)

Please check the document version of this publication:

• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.

If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne

Take down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.

Download date: 06. Aug. 2021

LINEAR MLLTIVARIABLE SYSTEMS

Preliminary Problems in Mathematical Description, Modelling and Identification.

by

A. K. H ajdasinski

E I N D H 0 V E NUN I V E R SIT Y 0 F TEe H N 0 LOG Y

Department of Electrical Engineering

Eindhoven The Netherlands

LINEAR MULTIVARIABLE SYSTEMS.

Preliminary Problems in Mathematical Description, Modelling and Identification.

By

A.K. Hajdasinski

TH-Report 80-E-106 ISBN 90-6144-106-4

Eindhoven

April 1980

Contents

1.

1.1.1.

1. 2.

1.2.1.

1.2.2.

1.2.3.

1.2.4.

1. 3.

2.

2.1.

2.1.1.

2.1.2.

2.2.

2.2.1.

2.2.2.

2.3.

2.3.1.

2.3.2.

2.3.3.

2.4.

2.4.1.

2.4.2.

Acknowledgements

Abstract

ii

Introductory informations

Preliminaries - definitions of some important notions

Mathematical models commonly used for the multivariable dynamical system description

Transfer function matrix model

Decomposition of the transfer matrix and classification of the multivariable dynamical systems

State space representations

Nonuniqueness of the state space equations

controllability and observability in multivariable dynamical systems

Basic structures of the multivariable dynamical systems and canonical forms

Definition of the order of the multivariable dynamical system

Advantages and disadvantages of the transfer function matrix models

iv

v

1

1

12

13

19

21

27

28

31

31

34

Advantages and disadvantages of the state space models 35

Observable and controllable canonical forms for the state 36 space models

Canonically observable form

Canonically controllable form

Innovation state space models

Optimum estimation and conditional expectation

Optimum estimation and orthogonal projection

The discrete-time innovation problem

Generation of canonical forms from Hankel matrices

The Ho-Kalman minimal realization algorithm

The minima realization algorithm with the use of singular value decomposition of the Hankel matrix

38

42

45

46

51

54

59

62

64

3.

3.1.

3.2.

3.3

3.4.

3.5.

3.5.1.

3.5.2.

3.5.3.

3.6.

3.6.1.

3.6.2.

3.6.3.

3.7.

4.

4.1.

4.2.

4.3.

iii

Identification of the multi variable dynamical system structure

Estimation of structural invariants - Guidorzi's method

Order test based on the innovation-approach to the state space modelling-Weinert - Tse's method

Structural identification proposed for the transfer function matrix model of the MIMO system - Furuta's approach

Miscellaneous order test

Akaike's FPE (final prediction error) and AIC (Akai~s

maximum Information criterion) as the order test for MIMO Systems

Statistical predictor identification - Final Prediction Error Approach

Akaike's maximum Information Approach

Concluding remarks

Structural identification based on the Hankel model

Behaviour of the error function

Behaviour of the determinant of the Hankel Matrix

Singular value decomposition of the Hankel Matrix

Conclusions and remarks

Multivariable system identification

The Tehter's minimal partial realization algorithm

Gerths's algorithm

The approximate Gauss - Markov scheme with the singular value decomposition minimal realization algorithm

References

72

72

79

86

90

91

92

104

114

115

115

116

117

123

124

126

130

134

141

iv

ACKNOWLEDGEMENTS

This report, being a part of the project "Identification in MIMO

Systems", has been written with the kind help and financial support

of the Samenwerkingsorgaan between Katholieke HQ2eSchool' Tilbur2 and

Technische Hogeschool Eindhoven.

The author feels honoured to express his acknowledgements to

Ir. A.J.W. van den Boom, the project leader, who took the responsibility

of co-ordination, discussion and correction of this report.

The author also feels indebted to Mrs. Barbara Cornelissen, whose

devotion in typing this report within a short time, including lunches,

wins appreciation and gratitude.

The writing of this report in a relatively short time was also possible

due to the generosity of the author's wife and son, who were left --alone

for four months.

Present address of the author:

Dr. Ing. Andrzej K. Hajdasifiski, G16wne Biuro Studi6w i Projekt6w G6rniczych, Plac Grunwaldzki 8/10, 40-950 KATOWICE (Poland)

v

Abstract

This report contains a partial knowledge about linear multivariable

systems. It starts with very simple concepts from multivariable

system theory, and closes with some proposals of further research

in the field of MIMO systems identification.

Selected subjects were mainly discussed, however, forming a comprehensive

set. The choice was certainly subjective, but presented methods were

either applied with good experience or convincing records about their

application were found.

There is, however, an exception which still needs further research,

namely the Akaike FPE method, which intuitively is quite obvious, but

practically never well explained.

This work deals with subjects to be found in generally available

literature, but also (this is a subjective feeling) with subjects which

are presented in an artificially complicated way (e.g. innovation

approach) or which are mainly authors' studies (e.g. Markov parameters,

. order tests for MIMO systems).

- 1 -

1. INTRODUCTORY INFORMATION

The notion of the multi variable dynamical system has appeared in literature

and in practice as a natural evolution of the scalar dynamical system being

the very first approximation of real processes. R.W. Brocket and

H.H. Rosenbrock in their foreword to the series" Studies in Dynamical

Systems" have written: "During the last twenty years there has been

a progressive increase in the complexity and degree of interconnection of

systems of all kinds. The reasons are clear: recent progress in communication

data processing, and control have made possible a much greater degree of

coordination between the parts of a system than ever before."(+) Such a

development demanded new techniques, new mathematical models and methods

suitable for handling more complex and intercorrelated tasks of the agregated

systems. A quick development of the multi input - multi output systems theory

had to go in line with a very advanced mathematical apparatus application

and unavoidable incorporation of digital computers and numerical methods. In

this study we will try to give a comprehensive description of selected problems

being of particular interest for a system designer. For the rest of already

tremendously imposing material we will refer .to an extended bibliography.

1.1. Definition of a multivariable dynamical system

The definition of the multivariable (multi input - multi output system)

dynamical system as proposed by Wolovich (1974), Niederlinski (1974) and

Rosenbrock (1970) is as follows:

Definition The multivariable system is the system having more than one

input and one output and inputs to this system may influence more than one

output at a time.

(+) Published by Thomas Nelson, London.

- 2 -

The block diagram of such a system is shown in Fig. 1.

u, --- .. --...... ,-- .,.,.--:::---........ -~ ........ ~,.",

-::...~_ ...c._--- <...---~~ . , .-:::- _ >0 -. ~ ......... , , -::::- -- - - ---"':!...

t".!. i..

Further we will consider only a relatively simple class of multivariable

dynamical systems, namely those which are linear, time invariant and finite

dimensional.

1.1.1. Preliminaries - definitions of some important notions

Definition 2 For the multivariable dynamical, linear, time invariant and

finite dimensional system having p inputs u,(k) .••.•• u (k) (forming the p

input vector ~(k)) and q outputs Yl(k) ...•• Y (k) (forming the output vector q

y(k». Here is defined the q x p matrix !(z). called the transfer matrix

(being considered the rational matrix of the argument z) fulfilling the

following condition:

l.(z)

where

l.(z)

!(z)~(z)

y (z) ,

y (z) q

~(z) u (z) ,

u (z) p

(1 )

- 3 -

and l.(z) , .o!.(z) are the "z" transforms of l.(k) and u(k) respectively, under

zero initial conditions. (see also Zadeh L.A., C.A. Desoer (1963),

Eykhoff P. (1974), Rosenbrock H.H.(1970), Niederlinski (1974), Schwarz H.

(1971), Wolovich (1974~.

The coordinates of the .o!.(k) vector can be both control variables and

disturbances, while the coordinates of the l.(k) vector are the output variables.

Further there will be considered only such linear systems, for which outputs

are linearly independent i.e. the outputs cannot be described as a linear

combination of remaining ones. This simply means that the inputs and the

outputs must fulfil the following condition:

q ~ p

This condition is always fulfilled while the rank of the ~(z) matrix is

equal to q.

{rank {~(z)} = q}<:> {q ~ p}

(2)

(3)

Definition 3 The aharacteristia polynaminal w(z) of the strictly proper+)

or proper transfer matrix !(z) is defined as the least Common Denominator of

all minors\nE(z), having by the greatest power of "z" the coefficient equal to

one. (see also Wo10vich (1974), Schwarz, (1971), Rosenbrock (1970».

Proceeding further with definitions we have to define the degree of a

parametric (po1ynomina1) matrix and the state representation of the multivariab1e

dynamical system.

Defini tion 4 The degpee 6{K(z)} of the strictly proper or proper transfer

matrix ~(z) is defined as the degree of its characteristic polynominal.

- 4 -

(Practically it is the smallest number of shifting elements necessary to

model the dynamics of this system).

+),. a multivariable system is called "a proper system" or its transfer

matrix is called "a proper transfer matri;x:"

if

lim ~(z) '" 0

z + 00

2. a multivariable system is called "a striatZy propel' system" or its

transfer matrix is called "a striatZy proper tronsfer matrix"

if

lim ~(z) 0

z + 00

3. a multivariable system is called "an impl'oper system" if at least for

one component of a transfer matrix it holds that the degree of a

nominator is greater than that of a denominator.

Definition 5 For the multivariable, linear, time invariant, dynamical system,

the state of the system of an arbitrary time instant k = k is defined as a o

minimal set of such numbers Xl(k), X2(k), ••••• X (k) the knowledge o 0 " 0

of which, together with the knowledge of the system model and inputs for

k ~ k is sufficient for determination of the system behaviour for k ~ k o 0

X(k) = - 0

X2 (k ) • 0

Xl(k) o

X (k ) n 0

is called the state vector, and members Xl(k ) .•••• X (k ) are

o n 0

called state variables.

- 5 -

(see also DeRusso P.II., Roy R.J., Close Ch. M. (1965), Kalman R.E.,

Falb P.L., Arbib M.A. (1969), Rosenbrock H.H. (1970), Schwarz (1971),

Wolovich (1974), Niederlinski (1974) and many other~.

Defini tion 6 The set of difference equations

~(k + I) = ~~(k) + !~(k)

where ~(k + I) - is a (n xl) state vector

~(k) - is a (p x I) input vector

is called the state equation, while the set of difference equation

where ~(k) - is a (q x I) output vector

is called the output equation.

Definition 7 The triplet of matrices {~t !t £} is defined as the

reaZization of the dynamical, linear, time invariant, multivariable system.

Defini don 8 The number of state variables "nit in the state equation is

defined as the dimension of the state vector or the state space and also

denoted as the dimension of the complete system.

Definition 9 Any polynominal fez)

fez)

for which holds

f(~) = Ak + Cl~k-t C A2 CAe AO ~ + ••••••• + k-2- + k-l- + k- = 0/

(4 )

(5)

(6)

(7)

- 6 -

is called the annihilating polynominal of the ~ matrix.

Investigating various properties of multivariable system the following

Lemma drawn from the Cayley-Hamilton Theorem can be of great help.

Lemma The characteristic polynominal of the ~ matrix - WA(z) is one

of the annihilating polynominals of the A matrix.

Definition 10 The polynominal f(z) of the smallest. nonequal zero.

degree k. fulfilling definition 9. is called the minimal polynominal of the

A matrix.

Definition II The matrix coefficient ~ = C AkB for k = 0.1.2 •..••. is

referred to as the k-~ ,Markov Parameter of the system defined by the

realization {~.~. ~}. (see also Ho B.L •• Kalman R.E. (1966). Schwarz (1971).

Gerth. W. (1971). Tether A.J. (1970). Hajdasiiiski A.K. (1976. 1978).

Hajdasinski A.K .• Darnen A.A.H. (1979».

Definition 12 The following description of the multivariable dynamical

system is referred to as the Hankel model (H - model) of this system.

y MUll + !!J<~ ~(i) = 0 for i < 0 (8)

where

1. '''l ~(o) S -properly

y U = dimensioned block vector containing

1.(1) ~(lL initial conditions

1.(5) ~(2 )

- 7 -

MO ....... 0 -0

= M -0

Generalized Toeplitz Matrix

H --k

M -2

M -0

M -1

M -:1

M -0

o

M ••••• ~ •••• -1 -1

M :i. •••• ~ ••••••

M ; ••• '~+1 ••••

Generalized Hankel Matrix

and ~ - for k = 0,1,2, ...• are the Markov parameters of the considered

system.

For a rigorous -derivation and more facts about the H-model, the reader is

referred to: Schwarz H. (1971), Gerth, H. (1971), Hajdasiiiski A.K. (1976),

Hajdasiiiski A.K. (1978).

In aiming for equivalency conditions for different types of models of multi-

variable systems, we must pass through two fundamental theorems and the

definition of the order of the multivariable dynamical system.

Theorem 1 The sequence of Markov parameters {~} for k = 0,1,2,3, ••.•..

has a finite dimensional realization {~, !, C} if and only if there are an

integer r and constants ai such that:

M --r + j =t: o'(i) M

r + j - i for all j ~ 0

i = 1

where r is the degree of the minimal polynominal of the state matrix A

(assuming we consider only minimal realizations).

(9)

- 8 -

Remark: Theorem 1 is called the reatizabitity criterion and the r is

called the reatizabitity index.

The proof of this theorem is to be found in the Ho L.B •• Kalman R.E •• (1966).

Schwarz H •• (1971). Kalman R.E •• Falb P.L •• Arbib M.A •• (1969).

Theorem 2 If the Markov parameters sequence {~} for k = 0.1.2 ••••. has

a finite dimensional realization {A. ~. ~}. with realizability index r. then

the minimal dimension n in the state space (also of the realization) for o

this realization fulfils

rank [.!!..-] = n o

where n - minimal state space dimension o

and

[M.] - (q x p) matrix -:l

n ~ r x min (P.q) o

H -r

M M -<> -I

M M -I -.

M M -r-l -r

......

......

M -r-I

M -r

- the Hankel Matrix (finite)

( 10)

(II)

The most correct proof of this theorem is to be found in Schwarz H •• (1971).

Remark: From linear dependence of Markov parameters it follows that

n o

(12)

- 9 -

Discussion of the Chapter 1.1.

With the aid of these 12 definitions, I Lemma and 2 Theorems it is possible

now to find a link between different types of multivariable system

descriptions. There will be no ~igorous mathematical derivation presented

here. To visualize this link we will draw a block scheme showing inter­

dependence of different type models. The arrows in this scheme show only

possible direct links (fig. 2).

From this scheme we learn that while from the state space description there

is a straighforward way to get the transfer matrix !(z), the reverse

procedure must be completed employing the realization theory, which is a lot

more complicated.

OTHER REALIZATIONS

~ Ho-Kalman STATE SPACE

REALIZATION - ~, ~, ~ !(z)

DESCRIPTION

t MARKOV

PARAMETERS

l , H

MODEL

fig. 2. Interdependence of different

type models

- 10 -

On the contrary, knowing Markov parameters, it is equally easy to get any

required form of description. For the sake of modelling, Markov parameters

can be derived as easily from the state space description as from the

transfer matrix. Obviously Markov parameters are also used in the H - model.

Example I Let us consider a simple two input - two output system described

in the following transfer matrix ~(z):

u (z) 1

y (z)

:1 ~(z) :1 u~(~z)~--1----------1----~y (z)

2 2

- 1.0 z - 0.5

= (z - 0,8) 2(z - O,2)(z - 0,8)

I .0 1.0

(z - 0,4) (z - 0,4)

the characteristic polynominal w(z) of the strictly proper matrix !(z) is:

W(z) (z - 0.2)(z - 0.4)(z - 0.8)

the degree of the ~(z) is:

Markov parameters for this system are:

M = --Q

[

-1.0 0.0]

1.0 1.0

!:!I = [-0.8 0.6]

0.4 0.4

M """"2

-0.64

0.4

0.6

0.4

M = [-0.512

-3

0.064

M. = -1.

- II -

0.504 1 0.064

. . . . and so on •

One of the possible realizations of this system is:

0.2 0.0

A = 0.0 0.4

0.0 0.0

0.0 -1.0

B 1.0 1.0

1.0 1.0

c = [

1.0

0.0

0.0

1.0

-1.0 0.0

1.0 1.0

-1.0 0.0

1.0 1.0

H = 0.6 -. -O.B

0.4 0.4

0.0

0.0

O.B

-1.0 I 0.0

= M -0

-O.B

0.4

-0.64

0.4

rank H = 2 -I

0.6 rank H

0.4 -.

0.6

0.4

c 3

- 12 -

rank H = rank H = rank H -3 -" -" + N

N > 2

Thus r - realizability index = 2

n o

- dimension of the realization = 3

Coefficients of the minimal polynominal are: a1

f ",.in (~) = A2 - A + 0.16 I =

0.16

Remark: It is made evident now that not every annihilating polynominal of

A (~z.W(~) = (~- 0.2)(~ - 0.4)(~ - 0.8» must be of minimal order. In

this case the characteristic polynominal of !, being one of the annihilating

polynominals for ~, is of the .3-rd order while the minimal order is 2.

M . -,,+J

for j ~ 0

1.2. Mathematical models commonly used for the multivariable dynamical system description

Solving problems in multivariable dynamical systems requires implementation

of quite a huge and advanced mathematics: theory of sets, matrix algebra

and analysis with special attention payed to polynominal matrices and

functions of matrices, theory of linear spaces, theory of limiting processes,

advanced mathematical analysis, some topics in functional analysis, theory

of differential e.quations, complex analysis, Laplace and "z" transform

techniques and many supplementary topics from related disciplines.

It is not possible to give a review of even selected problems and the only

possibility is to direct a reader to references.

Unfortunately there does not exist any comprehensive publication treating a

- 13 -

subject of "Mathematical methods ll for multivariable systems entirely. Moreover,

the best references are with literature concerning the control problems,

because the control engineering science was stimulating the development of

certain methematical disciplines.

In this report we will attempt to give the most intuitive and simple

description of rnultivariable systems.

The practical applications show that sometimes sitnpler models may better

serve the control tasks than very sophisticated ones. This always is a

compromise between achievable accuracy, "common sense" and a scientifically

formal approach.

Extensive references for further readings will also be given here.

1.2.1. Transfer function matrix models

The main interest will be focused on the discrete-time systems. However,

it seems to be useful to start with the continuous-time, linear systems and

generalize derived results using the "z" transform concept.

Assuming that there are given: !(s) - the transfer function matrix and ~(s) -

the Laplace's transform of the input vector, it is always possible to find the

output vector Z(t) with the zero initial conditions.

and using the convolution integral: t J !(t - T)u(T)dT

o

I.( t)

where

( 13)

( 14)

is the (q x p) "weighing matrix" (see Zadeh L.A., Polak E. (1969), Schwarz H.

- 14 -

(1971), Niederlinski (1974~. The weighing matrix has an interesting

physical interpretation for columns of this matrix can be interpreted as

impulse responses to the separately applied input Dirac pulses.

if

t

1.( t)

o

o

u (T) I

0. (T) •

where Uk(T) = 0

A k f i, i = 1.2 - - P

k (t - T)k (t - T) ... k .(t - T) ... k (t - T) 0 II 12 I' IP

k (t - T)k (t - T) .•• k .(t - T) ... k (t - T) 0 21 22 21 2P

k. (t - T)k. (t - T) .•• k .. (t - T) .•• k. (t - T) O.(T) • 11 12 11 1p

o

k (t - T)k (t - T) ••• k .(t - T) ••• k (t - T) 0 ql q2 q1 qp

k .(t - T) I'

k .(t - T) 2'

k .(t - T) q1

o.(T)dT = k.(t) • -:L

i - th. column of

the ~(t)

dT =

( 15)

- 15 -

Example 2:

As an example to start with, we can consider a simple two input and two

output system of the level control.

,,~ Q. [,;:'J Qz.r-w-\] VI.

I M-

-------- -- ---- --

h, hI.

! A~[m~l Q ...

. A,,[m'] QIA

-

h

J

The task is to maintain levels h and h at a certain 2

range, manipulating

valves v and v such that the volumetric flows Q and Q can be properly I 2 I 2

adjusted. In order to propose a control algorithm, we have to find a model

- 16 -

of the given system, having the following block diagram:

Q. ~ ..

Writing linearized mass balance equations

llQ 1

llQ 2

llQ 1 1

llQ 21

A 1

= A 2

dt

dllh 2

dt

dllh llQ + llQ - llQ a A ---

II 21 dt

VI

111

where llQ, llQ, llQ ,llQ ,llQ, llh , llh ,llh are small deviations of 1 2 II 12 1 2

variables around the working points.

assuming

Q = f(h)

AQ = ~ a~~h) ) h

• llh = h

0

( af(h)

) h =

a a ah 0

h 0

llQ = a llh 0

also llQ = B llh 1 1 1 0 1

llQ = B llh 21 20 2

. .60,(5)

AQz.(S)

finally,

- 17 -

after applying the Laplace transform:

llQ (s) = A sllh (s) + S llh (s) I I I I 0 I

llQ (s) = A sllh (s) + fl llh (s) 2 2 2 20 2

llQ (s) = fl llh(s) + fl llh (s) - Asllh(s) I 0 I 20 2

a llh(s) + Asllh(s) o fl llh (s) + fl llh (s)

1 1 20 2

The block diagram of this dynamical process is following:

-I ,.,

A.s + (3.0 (3.0 ~ As T <:i..o

~h2(!»

~"o r r--

-1 I A'l.S + (310

Ant!»

An2.(~1

- 18 -

and a

i\.b(s) 0

= (8~: 'S+I)(f-'S+I) 0

i\.h (s) 2

0.0

introducing

820

k =--. k =-- k • a I I a 12 0 0

AI A2 T =-- ·T = T

I 8 2 B I 0 H

we get

!(s) !(S) !!.(S)

where k

I I = ---------

(I + 5 T )( I + s T) I

0.0

is the transfer matrix, and

k I I

22

-1.u..... a

i\.Q (s) 0 I

(~S+I~' S+I) 820 a

i\.Q (S)

'V~~ 2

At '" .... 1 ?S;o i\.b(s)

= !(s) =

820 i\.b (s) 2

A i\.Q (s) .. . I = !!.(s) • CXo i\.Q (5)

2

k 12

(I + sT )(1 + sT) 2

k 22

(I + sT) 2

k 12 -t -t

!(t) = (Te - - T e -) T I T

-t -t (Te - - T e -)

T 2 T T - T I T I

0.0

- T 2

k -t eT" 22 2

2

- 19 -

is the weighting matrix of the dynamical system

(I + sT1)(1 + sT2 )(1 + sT)

3

1.2.2. Decomposition of the transfer matrix and classification of the multivariable dynamical systems

Following Niederli~ski (1974) we will decompose the input vector ~(s) into

q - control inputs and p - q - disturbing inputs:

u (s) 1

~(s) = u (s) 2

(q x I)

. u (s)

q

u q + 1(9) (p - q) x I

u (9) p

which leads to the following relation:

1.( s) Q(s)] [~(s) 1 !.(s)

= t(s) ~(s)

u (9) p

+ Q(s)z(s)

t(s) + q x q transfer matrix of control inputs

Q(s) + q x (p - q) transfer matrix of disturbances

Following Niederlinski(1974) and Iserrnann (1977) we will classify the multi-

variable dynamical systems in the following way:

1. stable multivariable dynamical systems - Le. all those systems for which

(16)

- 20 -

all poles of the transfer function matrices lie in the left half plane of

the complex variable "s" and there are no poles on the imaginary axis of the

plane.

2. nonstab1e mu1tivariab1e dynamical systems - i.e. all those not fulfilling

the stability definition.

3. minimum-phase m.d.s. - i.e. all those for which all zeros of the

determinant det{!(s)} are in the left half plane of the complex variable "s".

4. non-minimum phase m.d.s. - if at least one zero of the det{!(s)} appears

in the right half plane of the complex variable "s".

It will be noticed that the mu1tivariab1e system may be non-minima1 phase.

However, each of its components p .. (s) (i,j, = 1,2, .... q) is the minimal 1J

phase object. The non-minima1 phase objects, are much more difficult for

handling than minimal phase ones.

Another classification due to Nieder1inski (1974) and Isermann (1977) is made

according the internal couplings of the multivariable dynamical systems:

I. m.d.s. with a negative coupling - i.e. such a system for which

det !(o) 1 - < 0

rlpu(O) (17)

i=1

II. m.d.s. with a positive coupling - i.e. such a system for which

det !(o)

1 - > o (18) q

TI P .. (0) 11

i~l

- 21 -

III. m.d.s. with a zero coupling - i.e. such a system for which

det !(o)

= 0 (19) q

[1 P .. (o) 11

i=1

!f.d.s. with a positive coupling resemble very much simple input-output systems

with a positive feedback, inclining to monotonic instability.

Another classification, credited to MacFarlane (1970)"Ostrowski(1952) and

Rosenbrock (1970), is suggested according to a very convenient stability criterion

by MacFarlane (1970):

a) m.d.s. with a dominant main diagonal - i.e. systems for which

q

\ P ii (jld) \ > L: P ik (jW)\

k=1 k,&i

i=I,2, .... q

i=1,2, .... q

(20)

(21)

b) m.d.s. without a dominant main diagonal - i.e. all those for which (20)

or (21) does not hold.

For further readings, the reader is referred to: Schwarz H. (1971),

Rosenbrock H. (197Q), MacFarlane A. (1970), Zadeh, Desoer (1963), Wolovich'

(1974), Kalman, Falb, Arbib (1969), Niederlinski (1970), Rij nsdorp (1961,

1971), Isermann (1977).

1.2.3. State space representations

Let us start with a definition.

Definition 5 in the chapter 1.1.1. is valid for both the continuous and discrete

time systems. The physical meaning of this definition is that the dynamical

- 22 -

system has a certain "memory" which contains information about influence

of past events on the present and the future. For the continuous time

systems definition 6 changes into the set of equations

d ~ (t) A x (t) + !~(t)

dt

assuming x(t ) = x(o) - initial conditions. - 0 -

A solution to this problem is

~(t)

t

A(t - t ) r e- 0 ~(o) + J

t o

l.( t) Ce _A (t - to ).!( 0) + erA (t - T) () ( ) J e- ! ~ T dT + !!. ~ t

t o

For the discrete time systems following definition 6 we have:

~(k + I) A x (k) + !~(k)

l.(k) = ~ ~ (k) + ~ ~(k)

where matrices {A, B, C, n} from eq. (22) are not the same as from

eq. (25) (26)

Solutions to (25) and (26) are:

x (N) N !!. ~(o) +

N - I ~ AN - k - IB u (k)

k = 0

N - I :leN) N = ~!!. ~(o) + c L:: AN-k-IBu() () k + ~ ~ N

k = 0

(22)

(23)

(24)

(25)

(26)

(27)

(28)

- 23 -

Using equations (22) and (25),(26) it is easy to find relations between state

equations and transfer functions for both types of systems:

a) continuous-time m.d.s.

applying the "Laplace" transform and combining two operator equations:

assuming

, y(s) = £ (s.!. - ~t ~ ~(s) + £(s.!. - ~) ~(o) + Q. ~(s) (29)

for the "zero" initial conditions are required for the transfer function def-

inition, ~(o) = 0

-' !(s) = £(s.!. -~) ~ + Q., where.!. = [Unitary matrix] (30)

of dimension (n x n)

b) discrete-time m.d.s.

applying the "z" transform and combining eq. (25)(26)

, -' y(z) = £(z.!. -~) ~ ~(z) + z £(z.!. - A) ~(o) + Q. ~(z) (31 )

and again for the "zero" initial condition

(32)

UlS\

L.1.l'L)

- 24 -

The block scheme of the multivariable dynamical system described in terms of

state equations is as follows:

--") B

Example 3:

equations:

dlW 1

dt

dfIV 2

dt

dV

dt

choosing t.V , 1

t.Q =

t.Q = 11

D -

;(): ~ I~ (4) -

~ 6

~(Sl :!lzl

- ----

lr ....

c -X -

':J.(,S ) ) 'itz

Reconsidering example 2, we can write the following set of

t.Q - t.Q t.V = A t.h 1 1 1 1 1 1

= t.Q - t.Q t.V = A t.h 2 21 2 2 2

= t.Q + t.Q - t.Q t.V Mh 1 1 2

lW and t.v as state variables and writing 2 3

Ct tN

0-

A

t.V B 1 10---

A 1

- 25 -

!W l>Q ~ 8 2

21 HA 2

we get:

deW I B,O = - --!W, + l>Q, dt A,

dl>V2 ~ - 820

dt -- l>V 2 A2 + l>Q2

t. I I' 'I

dl>V 810 820 () I,

l>V , + V - _0_ l>V f:

= --, A;""-dt A, A 'I; '1

[ "'j. [ I'

l>V l> l>V I l> l> '"] ).

l>h c __

x· u • za A tN2 l>Q2 lIh2 , = ..§!..L r

lIh2 A2 lIV

b. . /:;.V. - A. 0 0 I:Ni ~ 0 Aq.

~ AV1 = 0 _is. 0 tN,. T 0 1 ~Qa. i A~

h .J2v 0<.0 ,

AV -- AV 0 0 Ai AI. A

[!~J [ 0 0 1 ] 6V, 1 = 0 1 0 AVa.

AV

-NA. 0 0

[~ ~l A= 0 -A./A~ 0 B=

-AjV'A. J3m/A2. -oly'A

~ =( 0 0 ~ ] 0 1

~ ,. Q

- 26 -

Assuming that first q inputs to the system are the control quantities and

remaining p - q are the disturbing quantities, we get the following state

equation.

d .! (t) A x (t) + l1!p '!!Q ] [~(t)l !.( t)

dt

1.( t) = ~.! (t) +[!!.p !!.q] [~(t)l

z(t)

=

= C x (t) +.!!p ~(t) + !!.q ~(t)

.!!p(n x q), ~(n x (p - q)), ~(q x q), Qq(q x (p - q)) which corresponds to

the transfer matrix of control inputs

1

~(s) = ~(!s -!) .!!p + !!.p

and transfer matrix of disturbances

(33)

(34)

(35)

(36)

The matrix (!s -!) is called the characteristic matrix of A and the polynominal

(37)

is the charateristic polynominal of A.

Example 4

For the dynamical system from examples 2 and 3 we have

W(s) (s + ~ ) (s + ~) (s + Clo )

A

- 27 -

WA(s) - calculated as the Characteristic Polynominal of A

W(s)

Discussion

=

)(s + ~)(s +

A2

Ct o

A

)

The degrees of the W(s) and WA(s) are the invariants of the multivariable

dynamical system. The degree of the WA(s) is equal to the dimension of the

state space (see def. 8). The degree of W(s) is the degree of the transfer

matrix ~(s) (see def. 4). In general, W(s) I WA(s) and 0 WA(s) I 0 W(s) .

This will be discussed in the sequel.

1.2.4. Nonuniqueness of the state space equations

It is sometimes difficult to find a state space description which has a physical

interpretation. This happens while estimating the state space equations basing

an experimental input-output data without a prior knowledge of the physical

structure of the considered object. This is due to nonuniqueness of the state

space equations.

It can be proven that for a m.d.s. it is possible to find and infinite number

of state equations.

Let

!.(t) = T x (t) (38)

where! (n x n) is any nonsingular matrix. Thus equations (22) or (25)(26) will

result in

d x (t)

dt

_1 TAT x - --- (39)

- 28 -

~(t) = C T x (t) + ~~ (t) (40)

This new set of state equations results in the same transfer function matrix

as equations (22):

1 I 1

~ !(!s -! ~!) T B + D = =

= =

= I

~(~s -~) B + D = !(s) (41 )

The relation A -I

.!. !:..!. is called the "similarity relation" and there is

as special type equivalence between matrices ~ and A. For more details see:

DeRusso,P.M.,Roy R.J., Close Ch. M. (1965), Rosenbrock H.H. (1970), Wolovich

W.A. (1974), Birkhoff G., MacLane S. (1965), Gantmacher (1959) and many others.

1.3. Controllability and observability in multivariable dynamical systems

The multivariable dynamical system described by means of the state equation

!(t) = ~ ~(t) + ~P 'y'(t)

is sain to be COMPLETELY CONTROLLABLE i~ Kalman's sense if given any initial

state x(t ) there exist such the control vector _v(t), which will drive the - 0

m.d.s. to the final state ~(tf) = ~ for finite (tf - to)'

(42)

The necessary and sufficient conditions for complete controllability in Kalman's

sense are usually formulated in the following way: Kalman R.E., (1960),

Chen C.T., Desoer C.A., Niederli~ski A. (1966), Kuo B.C. (1970), Wolovich W.A.

(1974), Niederlinski A. (1974):

- 29 -

Theorem 3

The necessary and sufficient condition for the multivariable dynamical system

to be controllable in Kalman's sense is that the block matrix (n x qk)

(43)

where K ~ n is the degree of the minimal polynominal of ~, has the rank equal n.

(Alternative conditions are also discussed by Kuo B.C. (1970), Paul C.R. and

Kuo Y.L. (1971), Zadeh A.L., Desoer C.A. (1963) and many others.

The proof of this theorem can be found in KalmanR.E. (196~, Kuo B.C. (1970),

DeRusso P.M., Roy R.J., Close Ch. M. (1965).

From condition (43) it follows that the complete controllability in Kalman's

senSe is the property of the pair of matrices (~, ~) and does not depend on

the way outputs are produced by the system.

The multivariable dynamical system described by means of the state equation (42)

and the output equation

1.( t) £. x (t)

is said to be completely observable in Kalman's sense if given any input ~(t)

and output 1.(t) for to ~ t ~ t f is sufficient to determine the initial state

~(to) for a finite interval [to' tfJ.

The necessary and sufficient conditions for complete observability in Kalman's

sense are usually formulated in the following way.

Theorem 4

The necessary and sufficient condition for the multivariable dynamical system

(44)

to be completely observable in Kalman's sense is that the block matrix (n x qk)

(45)

- 30 -

where k ~ n is the degree of the minimal polynominal of !, has the rank equal n.

(Alternative conditions are also discussed by Kuo B.C. (J970), Paul. C.R. and

Kuo Y.L. (1971».

In the classical analysis of control systems, transfer functions are often

used for the modelling of linear time-invariant systems. Although

controllability and observability are concepts of modern control theory, they

are closely related to the properties of the transfer function. The following

theorem gives the relationship between controllability and observability and

the pole zero cancellation of a transfer function.

Theorem 5

If the input-output transfer function of a linear system has pole-zero

cancellation, the system will be either not state controllable or unobservable,

depending on how the state variables are defined. If the transfer function of

a linear system does not have pole-zeros cancellation, the system can always

be represented by dynamic equations as a completely controllable and observable

system. Kuo B.C. (1975). (This is an excellent reference for further readings

for everyone who wants to gain more information about dynamical systems being

considered in a very physical way).

Concepts of controllability and observability in the Kalman's sense are rather

useless while dealing with noisy systems. Also for some theoretical systems

those concepts can cause misunderstandings. An example will be given while

discussing a closely related concept of the multivariable system order.

- 31 -

2. Basic structures of the multivariable dynamical systems and canonical forms

Three main types of models incorporated in multi variable dynamical systems

design, analysis and identification have been described in brief. According

to very specific properties of each of these models, slightly deeper insight

must be done into their structural properties and utility possibilities for

different types of tasks. The concept of the multivariable dynamical system

order will be of an essential importance for the following part of the text.

2.1. Definition of the order of the multivariable dynamical system

In trying to model the reality, one has to answer first a question: what type

of applications for this model is considered?, thus facing the problem

of the "structure" choice for this model. This structure presents a desired

type of relations between inputs and outputs. However, two additional steps

must be performed - those are: demarcation of the "degree of complexity" for

this model (corresponds to the order determination) and parametric estimation

of the chosen model being of the pre-estimated degree of complexity

Hajdasifiski - Damen (1979) •

This procedure in most practicaZ cases is an iterative seeking for the modeZ

order and inaorporated set of parameters, matching it to reaZ data according

to a given optimaZity criterion and comparing with resuZts of previous runs.

- 32 -

input data output data

- - OBJECT )

choice of structures I'

choice of models

• > complexity

d a t a

test (O'yeSII 'llllll/mlq

• decisions

parameter

~ estimation ~

'F'T('1\'I'T""

mOdel output computation

~ .J ~ ~ ~ no

MODEL . ~ f", ")( ATCHIN y-

~ yes

FINAL MODEL

Definition 13 The order of the multivariable dynamical system will be defined

as the minimal number of Markov-parameters necessary and sufficient to re-

construct the entire realizable sequence of Markov -parameters. Hajdasinski,

Damen (1979) •

- 33 -

Remark It means that the system order is equal to the realizabilty index

fir" - see relation (9) Theorem J ..

This definition favours the Hankel model description and it really is the

intention for this type of model to give the most general possibilities of

descriptiveness without causing any ambiguity. Having a properly described

model complexity it is very easy to generate proper structures in the state

space and a properly structured transfer function matrix.

For example for the state space description, the multivariable system order

can be defined as the degree of the minimal polynominal. For the transfer

function matrix description, however, the order definition in the general case

is not possible.

Defini tion 14 The dimension of the multivariable dynamical system is defined

as the number "n" being equal to the rank of the H - Hankel matrix for this r

system, where fir" is the order of the system.

Remark Compare def. 14 with the theorem 2, relations (10) and (II).

Alternatively for the state space description it is the dimension of the state

matrix A. And again for the transfer matrix description there does not exist

a unique definition of the system dimension. Only in the cases when all poles

1n elements of the transfer matrix are either different or "equal and common"

the dimension can be determined as the degree of this matrix (see definition 4.)

To illustrate this, the following example shows the case of equal but distinct

or non-common poles. Hajdasinski - Damen (1979).

0.0 K(z)

(z - 0.8)(z - O,25)(z - 0.5)

0.0 (z - 0.8)(z - 0.6)(z - 0.01)

- 34 -

Dimension of this system is n = 6, but degree 0 k(z) 5 because the equal

poles z = 0.8 are noncommon and they refer to different state variables.

However, in practical cases it will be seldom that distinct poles have exactly

the same value i.e. are equal. Nevertheless, when poles are given up to a

certain accuracy (for example numerically evaluated poles) it may be difficult

to decide whether they are really distinct or not. This problem is also one

of the drawbacks of the transfer matrix description and one more argument for

the state space and Hankel description, where this ambiguity never arises.

2.1.1. Advantages and disadvantages of the transfer function matrix models

The following features of transfer function matrix models are worth noticing.

I. the transfer matrix description is a unique description of the multi-

variable dynamical system given a unique ordering of inputs and outputs.

It means that there exists one and only one transfer function matrix k(z)

for a given order of inputs and outputs.

II. the transfer function matrix has a very easy physical interpretation for

elements K .. (z) of K(z) are transmitances between y.(k) outputs and u.(k) 'J - 'J

inputs of the considered system.

III. the transfer function matrix description is not very economical for the

analog computer modelling.

IV. the transfer function matrix is very inconvenient for digital modelling.

V. the transfer function matrix may not encounter all dynamical properties of

the system (see 2.1.).

VI. from knowledge of the transfer function matrix it is difficult to derive

state space equations, but it is fairly easy to derive the Hankel model.

- 35 -

For further comments see: DeRusso P.M.,Roy R.J., Close Ch. M. (1965),

Wolovich (1974), Kuo B.C. (1975), Isermann (1977), Niederlinski (1972, 1974).

2.1.2. Advantages and disadvantages of the state space description

The following features of the state space models are worth noticing:

I. the state space model is a non-unique description of the multivariable

dynamical system given an unique ordering of inputs and outputs (see 1.2.4.)

II. in a general case there is a Zaak of physiaaZ interpretation for the state

space model.

III. the state space model is more eaonomiaaZ for analog modelling than the

transfer function matrix model.

IV. the state space model is very aonvenient for digital modelling

V. the state space model enaounters aZZ dynamiaaZ properties of the system

being modelled.

VI. the state space model provides equaZZy easy transformation into the transfer

function matrix as into the'Hankel model.

- 36 -

2.2. Observable and controllable canonical forms for the state space models

This chapter will start with the phase canonical forms for single input -

single output controllable systems. The generalization for multi input-

multi output systems will be then easier. The major results in the field

of canonical forms are due to Kalman R.E. (1963), Lue.berger G.D. (1966),

Mayne D.Q. (1972a, 1972b) Popov V.M. (1972).

Assuming that there is given the transfer function K(z):

y(k) b + bIZ + + biz m-I

K(z) 0 m-= =

u(k) a2 z2 n-I n a + alZ + + ••••• + a 1 z + z

0 n-

for m ,n, and there is no pole-zero cancellation, the corresponding state

space equations in the minimaZ canonicaZ form may take the following form:

o o

X2 (k + 2) o o

Xn

_1 (k + 1 ) 0 0 0

x (k + u

1 ) -a 0

-al -a2

y(k) = lbo b l b2 •••••• b m-I

o

o

.... 1

.. .. -a n-I

•••• 0 •• I • 0

XI (k)

x2(k)

xn

_1 (k)

x (k) n

]

+

xI(k)

x (k) n

o

o u(k)

o

Equations (47) are referred to as the "phase-vanabZe canonicaZ form" and

(46)

(47)

- 37 -

state variables x (u) are called the "phase variables". i

The state A o o •... 0

o o o

o o o ..•• I

is called the "F:t'obenius matm" or the "companion matm of the poly-

nominaL ao + a z + ...... . u-l u" + a 1 z + z • u-

Another phase canonical form will be

~

)1.,(\(+1) 0 0 0 0 - a. A

1 0 0 0 -0... x~( k +1)

=

><,,)\(+4) 0 0 1 0 - a...-1 A

X" (\1.+1) 0 0 0 1 - a ".\

\jl k) :: [ 0 0 . ·0 0 o 1 ]

~

)(, (It.) b.. Xl tic.) b,

+ u'l'-'\

b"'_1 0

A

X "t\C.) 0

A

)(. (k) • )(l l \()

The development of phase canonical form for single output - single input

system was an attractive area of research for two main basic reasons:

(I) simplicity of derivation

(2) a convenient starting point for r.ertain control design problems.

The canonical forms for MIMO systems are even more important than

for 5I50 systems. The canonical form will be defined as t~s-

(48)

- 38 -

formation of the state vector to a new coordinate system in which

the system equations take a particular simple form". (see

Niederlinski - Hajdasinski (1979)).

Unlike the SISO case, the corresponding canonical forms for

multivarible systems are not unique. Among the most used canonical

forms, the canonically observable and controllable forms are of the

greatest importance.

2.2.1. Canonically observable form

Consider the discrete completely observable multivariable system represented

by state equations:

~(k +

x.(k)

Let H =

I)

h T -1

h T -:z

h T --q

H x

I x(k) + ~ u(k)

(K)

Constructing the vec.to..,. sequences

and selecting them in the following order

h -2

, .....

.....

retaining a vector (FT)sh. in (52) if and only if it is independent from - -1

(49)

(50)

(51 )

(52)

- 39 -

T . all previously selected ones and all the vectors (F )J h . (0 < j < s) have

- -1

already been selected. Let v , v ,v .... v be the numbers of vectors I 2 3 q

selected from the first, second and ----- q-th sequence in (51);

h , FTh , -2 -~

~~-----------v.--------

FROM THIS SEQUENCE ::;'V I

FROM THIS SEQUENCE ~ v 2

Tv' Thus the vectors (F ) Jh. are therefore linearly dependent on previously - -J

selected ones (because of the choosing procedure and because v. is counted J

from zero).

The complete onservabi1ity of the system implies that

v + v + •••• + v = n I 2 q

where n - is the dimension of the considered system.

. 1 . TT Let us now construct the nons1ngu ar matrIx _

h -q

and use the T as a transformation matrix for the similarity transformation

(see 1.2.4.) providing a new state vector x:

x Tx

Remark: The T is a nonsingular (n x n) matrix which consists of linearly

nondependent columns, each product

v + v ..... v = n. Lue~berger I 2 q

The state equations are:

~(k + I) =!! (k) + B u (k)

(FT)ih . - -J

(1967).

is the vector and

(53)

(54)

(55)

x.(k) C x (k) (56)

- 40 -

where matrices ~,~, and £ are of the following form

A = T F T- I - -- {A .. }

-1J i, j = I, 2, ..... , q

and (v. x v.) matrices A .. show the structure ~ ~ -],,1.

A .. = -11

o

o

a .. I 11,

.••••• a. . . 1.1,Vl.

which is exactly as already discussed for 8180 systems Frobenius matrix.

Because of the order followed in the selection of vectors in (51) and

of the consequent structure of !, in every matrix A .. -1]

at most v. + 1 1

elements are non-identically zero if j < i and v. if j 1

number v .. is given by 1J

L: min (v. , 1 v .. =

1J

(v., v.) 1 J

v. - 1 ) for j < i J

for j > i

and the (v. x v.) matrices A .. are of the type 1 J -].J

A .. = -1J

Further more

o .•..•.•...•......•.•.. 0

o ..............•....•.. 0

a .. 1J,

a.. 0 ...... lJ ,v ....... 0 1J

> i. Thus the

o ...... 0 0 ..................... 0

= o o o

0 ........... : .... : ..... 0 o .... 0

t • (v + I) 1

t (v + '" +v 1 + I)

1 q-

(57)

(58)

(59)

(60)

(61)

- 41 -

b b •••• b

B = T G does not possess 11 12 IP (62)

any special structure

b b •••• b nl n2 np

It should be noticed that the structure of the canonically observable couple

(!, £) is completely determined by v. - indices which are called Kponecke~ ~

invariants. These are structuraZ invariants of the dynamical system. (see

Luenberger D.G. (1967), Guidorzi R.P. (1973), , Popov V.M. (1972),

Niederli~ski, Hajdasifiski (1979). A single-output, single-input system has

only one structural invariant, namely the system order r or the system

dimension n (which are related to each other by the relation n = rank Hr -

see Theorem 2, relation (10».

The transformation! decomposes the original system (!, ~) into q inter-

connected subsystems having a structure guaranteeing the complete

observability of the jth subsystem from the jth output component. This

properly justifies the name "canonically observable form". (Niederlinski-

Hajdasinski (1979».

Remark: The dependent vectors (FT)vi h. are a linear combination, with - -~

coefficients given by the ith significant row of ! matrix, of

previously selected ones i.e. as demonstrates Popov (1972).

i-I

=L: j=1

min (v. ,v .-~) + ~ 1 J

~ aij'k k=1

min(v. ,v.) ~ 1 J

L- aij,k k=1

(See also Niederlinski - Hajdasifiski (1979».

Remark: Given Kroneckey invariants vi, the number of aij.k invariants

is well determined with the following properties (Niederlinski

Hajdasifiski (1979».

( 1 ) of v. and a .. k invariants are independent, i.e. for any ~ 1J,

the set

private integers v. satisfying v + v + •••• + van ~ 1 2 q

- 42 -

and any numbers aij,k eR' with properly ranging indices (i,j,k),

there exists a pair of matrices (F,H) whose invariants are

precisely the above integers vi and -numbers aij ,k.

(2) the set of invariants vi and aij,k are aompZete, i.e. if for two

pairs of matrices (!,~) and (!~) of the same dimension the

invariants are respectively equal, there exists a nonsingular matrix

such that F = T F T- 1 and H =

(3) the set of vi and aij,k invariants in the emaZZeet set of parameters

determining a canonical form. It is impossible to construct a

canonical form having the universality properly with less parameters

than the number of aij,k parameters, with the vi parameters det-

ermining their position in the canonical form.

2.2.2. Canonically controllable form

For the controllable couple (G,F) from equations (49), (50) also exists

a canonically controllable form derived by writing the G matrix in the form:

G [ . " ] = ,g,: g ~ ... :g-1 t """"2 •• -p

(64)

and constructing the sequence

P , g , .... g., Fll ., F" ..... Fa.., F2", ,F2,. ... F2Q. , ... (65)

.... ,..., -p -:!:' -~ -~I' - ~1 -!L~ - !.p

Similarly, as before, the choice of linearly independent vectors !s.s.j can be

made and again n ,n ..... n are the numbers of vectors selected from 1 2 P

sequences.

(66)

- 43 -

Then the vectors Fnl~. are linear combinations of previously chosen - L

s F a. and following relations hold: - .£J

n + n + ••••• + n = n '2 P

(n· - are Kronecker invariants of the controllable form), l

n; F 0.' - 11'

(67)

(68)

where ai.j.k - are invariant parameters corresponding to the· aq.k.' The sets

of the {n t} and {a tj.k.} invariants share all properties of the sets of

{Vi) and {a:i.j.k J invariants,

Introducing the new state vector z = !~, with

-I R = ,IF IF I"'IF

[

I I n,-I I I np-I ] ~, I -~, I - ~.. ,- ~,p

the system equations (49) (50) become

Z(k) ~ !.(k)

where A R F R- 1 fA, .} -LJ

i,j = J,2, •.•. p

with o •••.... 0

n. x n. matrices A .. L L -n

I. 1 -nL -

... 0 •••••• 0 and n. x n. matrices A •• =

L J -LJ 0 •••••• 0

o ...... 0

a .. n,

aii ,

a .. 1J ,1

a .. 1J ,2

n. L

ai . 0 .. J, LJ

(69)

(70)

(71 )

(72)

(73)

(74)

and

n .. 1J

Further

B

and c

c

{" min

R G =

- 44 -

min(n. , 1 ) for j < i n. -1 J

(n .• n. ) for j > i 1 J

0 • •••••• 0 .. 1

0 0 • •••••• a

o O ••••••• 0

o · ..•... 0 ~ n + I 1

•••• 0 · ...... . • • • • • • • • • • • • • 0

o o ...•... 1 ~ n + •••.• +n + 1 1 p-l

o O ••••••• 0

H R- I does not reflect any specific structure

c C 1 1 1 2.

(75)

(76)

(77)

It can be noticed that the transformation (69) has decomposed the original

couple (!. ~) into p interconnected subsystems having a structure

guaranteeing the complete controllability of the ith subsystem by the ith

input component.

This property justifies the name "aanoniaaZly controllable form". (see

Niederlinski. Hajdasifiski (1979».

Remark: The observable and controllable canonical forms are only two

among many possible canonical forms, but also very important ones.

- 45 -

As it will be discussed later, these two canonical forms play

an important role in determination of the multivariable system

structure.

For further reading in the subject of canonical forms see

Denham (1974), Mayne (1972a,b), Niederli~ski, Hajdasifiski (1979).

2.3. Innovation state space models

The essence of the innovation approach to the state space modelling can be

expressed in words of Kailath T. (1968 ):

" ....•.• the innovation approach is first to convert the observed process

to a white-noise process, to be called the innovation process, by means of

+ a causal and causally invertible linear transformations. The point is that

the estimation problem is very easy to solve with white-noise observations.

The solution to this simplified problem can then be re-expressed in terms of

the original observations by means of the inverse of the original '~hitening"

filter .•... "

The given observations are recorded in the form

l.(k)

E{!. (k)}

~(k) + !.(k) k 0,1,2 •..•

Q - expected value of the !.(k)

E{!. (k)!. T (l)} !(k)Okl - covariance matrix for !.(k)

(!.(k) - white noise) k=l Kronecker del ta

kfl

(79)

(80)

(81 )

+ The external properties of a physical system can frequently be characterized by an operator relation of the form

g T f (78)

- 46 -

{z(k)} a zero-mean finite-variance process, i.e.

T ~ E{~(k)~ (k)} < 00 (82)

Let us introduce further the linear least-squares estimate of

what is the meaning of the last, it is necessary to solve the problem of the

optimum estimation and conditional expectation. Also, for the sake of the

innovation derivation we will need a theorem of the orthogonal projection.

To solve the final problem, we will refer to sub-chapters 2.3.1. and 2.3.2.

2.3.1. Optimum estimation and conditional expectation

The innovation approach to multivariable dynamical systems plays a still more

and more important role in modelling identification and the state reconstruction.

+ in which g and f are real n-vector valued functions of t, such that gTf

represents the total instantaneous power entering the system. (For example

an electromechanical system, the jth component oft might be voltage or force

at jth accessible point of the system, in which case g would be the

corresponding n-vector of currents or velocities. The notions of energy and

causality playa central role in the study of physical systems. Let us agree

to say that! is causaZ if for every 0 >- 00, If, and !f2 agree on the

interval (-00,0) whenever the n-vector-valued functions f, and f2 are

permissible inputs to the system fepresented by 1 and fl and £2 agree on

(-00,0). Let us say also that T is passive if for all

real x and permissible f. It has been proven that under some weak assumptions,

if an operator T is linear and passive, it is also causal.

- 47 -

For many other reasons, such as simplification of derivations, equivalence

to the Kalman filter without necessity to solve the Riccatti equation,

easier implementation of the identifiability notion, to mention but a feW,

we will show a very formal derivation of the innovation state space model.

This requires more complicated mathematics and a few notions which may seem

to be complicated at first sight. However, it will profit in deeper

understanding of the more general properties of the state space models and

related to it realizations.

Let {Z(o), X(I) .••••. ~(k - I)} be a set of measured random variables of

y (i). We can determine in principle, the probability of z.(j) (where z.(j) 1 1

is the j ith component of the ~(j)) assuming value Vi or smaller for

i = 1,2, ..... n.

This follows from the definition of the conditional probability function.

Considering a random variable z of a random process, given the actual value

n with which the random variable y has occurred, the conditionaZ p~obabiZity

density function of z, given y is denoted by p(~\n), and is defined by the

following relationship:

In general, the conditional p~obabiZity density of z, given

~(O) = no, ~(I) = n 1

is denoted by p(~\nOl n1···nk _ I)'

~(k - I) = n\l._1

The conditionaZ p~obabiZity dist~bution function of z given

yeo) no, .•..• , ~(k - I) = nl( _ I

(83)

- 48 -

is written

a Tl ) a k -I

(84)

Defining the joint probability distribution fUnction of the random variables

z (j), z (j) ..... Z (j) as the probability of the simultaneous occurrence I 2 n

of:

z.(j) ~ ~ , z (j) ~ ~ •••• z (j) ~ ~n 1. 1 2 2 n

(85)

and denoting it P(~ ,~ , •••• ~ ), and defining the ma~inal probability I 2 n

distribution function of random variables z, y(O), y(l) •••• 1(k- I)

Pr(z ~ ~, Z(O) ~ Tl o ' ••.•• Z(k - I) ~ Tlk _ I) = P(~.Tlo'Tll····Tlk _ I) (86)

the conditional probability distribution function is written

(87)

which is nothing but a Bayesian relation.

Let us use the expression

Z(k - I)} (88)

to describe the probability that the random variable z.(j) will be less than 1

or equal to some value ~. simultaneously for i = 1,2, •.• n, at the time j, 1

given measurements y(O), y(l) •••• y(k - I). This expression of the conditional

expectation distribution function is clearly also a statistical estimate of the

- 49 -

random variable ~(j), given information on ~(j). This statistical estimate

is represented by i(j k - I), which is assumed to be a fixed vector whose

elements are known whenever ~(O), ~(I) ••.•• ~(k - I) are known.

The general estimate i(jlk - I) will be different from the actual random

variable ~(j), which is unknown. For the purpose of comparison, it is

desirable to define an error function vector e,

~(j) = -'!.(j I k - I) + ~(j) (89)

Let us define, according to the statement made,that we are looking for the

linear least-squares estimate of ~(j), the loss function as L(~) = ~T(j)~(j).

We want to show that minimization of the average or expeated vaZue of L(~) is

equivalent to minimizing aonditionaZ expeatation. Given the random variable

vector ~(j) = ~(j) + ~(j), where ~(j) is the actual signal vector and ~(j) is the

noise vector and 1.(0),1.(1) .•••• ~(k - I) are k given measured random vectors.

It is desired to find the best estimate of ~(j) at j ~ k or j < k from

knowledge of the measured random variables. (j > k - smoothing (interpolation),

j = k - filtering, j > k - predicitng).

The problem is to find the optimum estimate z (jlk - I), given the measurement -opt

on ~(O), ...•• Z(k - I), so that the expected value of the loss function L(~)

is minirunized.

E!{~T(j) ~(j)1 Z(O),~(I) '" ~(k-J) }.

!{[~(j) - iOlk - 1)]T[~(j) - i(jlk - I)~Z(O), ~(1) ""Z(k - I)} =

= min (90)

Performing operations under the expectation operator we get

!{~T(j)~(j)1 ~(O)'Z(I) .. ". ~(k - I)} = E{~T(j)~(j)1 1.(0), 1.(0 ... Z(k - I)}-

- E{zT(jl k - 1)~(j)1 1.(0),1.(1) ..... Z(k - I)} -E{~T(j)i(jlk-l) ~(O)'Z(I) .. ~(k-I)}+

(91 )

- 50 -

Since ~(jlk - I) is a known constant vector when Z(O), Z(I) .••. Z(k - I)

are given, we have:

E{l(j)e(j)lz(O) .... Z(k - I)} = E{~.T(j)~(j)lz(O) .. 'Z(k - I)}-

- i(jlk - I)E{~(j)lz(O)""Z(k - I)}- E{~T(j)lz(o),Y(I)""Z(k - I)} +

+ zT(jlk - I)z(jlk - I) - -. (92)

Taking the partial derivative of the last equation with respect to

~(jlk - I) and set it equal to O.

A

d Z.<jlk-1) =

.-a Zn<jlk-1)

d I "'I' " l o ~ljl"-1) l Z lj'k-1)~q'k-1)j =

... :: 2. ~(j' k-1)

(%)

- 51 -

and

(98)

This shows that the optimum estimate of ~(j) in the sense of the minimum

least squares error is the conditional expectation of ~(j).

2.3.2. Optimum estimation and orthogonal projection

Consider the set of random vectors {~(j)} j = 0.1.2 •...• where ~(j) is an

n-vector. The set of all linear combinations of these random vectors of the

form:

i = a A. z(i) -,,-

(99)

(where A. is an n x n matrix with constant coefficients) forms a vector space -1

which is denoted Z(j).

Similarly. we will call the vector space generated by the set of vectors

k - I

L: Y(k - I) ( 100)

i = 0

The dimension of the matrix B. with constant elements is n x q. If k - J ~ j, -1 -

the vector space Y(k - I) is a subspace of the vector space Z(j).

Now let us consider that the random vector ~(j) is composed of two components.

that is:

~(j) = :!(jlk - I) + l(jlk - I) (10 I)

where ~(jlk - 1) is the orthogonaZ projection of ~(j) on the subspace Y(k - I).

- 52 -

The vector ~(j\k - I) is the component which is orthogonal to the subspace

Y(k - I)(i(jlk - I) 1 Y(k - I). that is. orthogonal to every vector in

Y(j). Now we shall prove the following theorem:

Theorem 6 The orthogonal projection ~(j\k - I) of ~(j) is that

Proof:

vector in Y(k - I) which minimizes the loss function

Let ~ be any vector n x I in the vector space Y(k - I).

Forming the conditional expectation

lo(k - I)}

and substituting (101) into (102) we get

Eq~U) - £f [~(j) -~] I lo(O). lo(I) .... • lo(k - I)} =

E{iTUlk - l)iU Ik - I) + iTU Ik - 1)[iU Ik - I) - f] +

+ [~(jlk - I) - ~JTiUlk - I) +

+ [fOlk - I) -~n~.(jlk - I) - §] I lo(O).loW .... x.(k - I)}

Since ~(jlk - I) - ~ is the vector which is in Y(k - I). and

I) is orthogonal to Y(k - I) and therefore also to

i(j/k - I) - ~. equation (103) can be written:

Eq~(j) -~IT[~(j) - ~11x.(0).x.(1) .... lo(k - I)} =

= E(~.T(jlk - I)i(jlk - 1)\ lo(O).lo(I) ..... x.(k - I)} +

(102)

(103)

+ E{["i(jlk - I) - i1T!E(jlk - I) - i]Ix.(O).X.(I) ..... X.(k - I)} (104)

Because the term E{lUlk - l)iUlk - I) I x.(O).x.(I) '" x.(k - I)}

will remain always positive definite. it is apparent that to

=

Remark I:

Remark 2:

- 53 -

minimize (104), we must have ~ equal z(J'\k - I) = Z (J'\k - I) - - -opt '

so that the minimum loss function is:

E{[~(j) - ~pt(j\k - I)1TI~(j) - ~Pt(j\k - 1)]\ ~(O),~(I) ••• y(k-I)}

E{iT(jlk - I)i(j\k - I) I ~(O),~(I) ..... ~(k - I)}

Q.E.D;

Relation (101) is of a great importance for consideration.

It shows that ~(jlk - I), which is an optimal estimate of

~(j), can be expressed as a difference between a fixed vector

(for a measured ~(O), ~(I), ...•. ~(k - I) and the correcting

term i (j I k - I)

i(j\k - I) = ~(j) - i(jlk - I)

To find the correcting term l(j\k - I) is due to the Kalman

filtering theory and the innovational approach. (106) - is

called a "correction equation".

Based on the discussion carried out in these last two sections,

we can now summarize:

(105)

(106)

Z (j\k - I) = OPTIMAL ESTIMATE OF =.(j) GIVEN ~(O) .... ~(k - I) = -opt

Kalman filter approach ~ =

Innovation approach -= E{~(j) I ~(O), l.(I) .... ~(k - I)}

= ~(jlk - I) of =.(j) on Y(k - I)

- 54 -

2.3.3. The discrete-time innovation problem

Let us return to equations (79)(80)(81)(82).

Z(k) = ~(k) + ~(k) K=O,l,2 •••..

E{~(k)} = Q ; E{~(k)} = 0

E{~(k)VT(k)} = ~(k)6kl T tr E{~(k)~ (k)} < 00

Let us define the innovation proce88 by

where as z(k\k - I) = z (k\k - I) - the linear least squares estimate - -opt

(79)

(80)

(81)

(82)

(107)

~(k) given {Z(l) O't~k - I} (108)

Thus we can calculate

E~(k)} = E{y(k)} - E{z(klk - I)} = ~

~(k) = Z(k) - ~(k\k - I) K X(k) - ~(k/k - I) =

~(k) + ~(k) - ~(klk - I) = I(klk - I) + z(klk - I) -

- ~(klk - I) + ~(k) = ~(k) + l(k/k - I)

thus for k > 1

E~(k~1t)} = E{~(k)lU,)} + E{~(k) 1(.9. 19. - I } +

+ E{l(klk - l)lT(9.19. - I)} + E{l(klk - 1)~T(9.)}

= E{~(k)~T(9.)} + E{l(klk - 1)[~(9.) + ~(9.)1T} =

= E{~(k)l (9.)}

k > 9,

the same can be shown for k < ~.

(109)

(110)

( I I I )

- 55 -

For k = £ we have

E~(k)~T(£)} = E{~(k)~T(k)} + 2E{~(k)i(klk-l)} + E{i(klk-l)iT(klk-l)}

E{~(k)~T(k)} + E{i(klk-l)iT(klk-I)}

covariance of the covariance matrix of the error noise in the estimate i(klk-I)

~(k)oik

E{i(klk-l)iT(k\k-I)} = E{[~(k) - ~(k\k-I)1(z(k) - ~(k\k-l)]T)

= P (k) -z

So that~(k) like ~(k) is white but with a different varaiance.

Remark: The ~02:t!n~o~s-tim~case can be approached by a limiting procedure

(I 12)

(I 13)

in which R(k) becomes indefinitely large, while P (k) remains finite, -z

so that the variance of~(k) and ~(k) are the same. (see Kailath T.

(1968).

Let us refer to our initial equation:

and let

~(k + I) = ~~(k) + !~(k)

r(k) = £ ~(k) + y(k)

~(k) = £~(k)

E{~(k) ~T(£)} = g(k)oki; E{~(k)~T(i)} = ~(k)oki

Assuming that the matrix [~(.) + ~(.)1-1 exists, for £ ~ k we get

~(k + I) = ~(k + II k) + ~(k + II k), where ~1!!.

Let an optimal estimate of ~(k + I) be

k ~(k + II k) = L. gi (k)!: (£)

£ = 0

(114)

( 115)

( 116)

( 117)

so that

- 56 -

k ~(k + I) - ~(k+I\k) = L g~(k)!:.(~)

~ = 0

according to the projection theorem, Multiplying both sides of (118) by

/(k) we get:

k T - 4. g~ (k)!:.(~)1!. (k) ~ ·.0

( 118)

( 119)

E{~(k + 1)1!.T(k) - R(k + l{k)~T(k)} = ~{~(k + 1)~T(k)} (120)

according to the projection theorem:

thus

and

k

L ~ D °

k = L: g~ (k) [Kz (~) + !(~)~~I< D

~ D 0

= g~ (k) (Kz (~) + ~(R.)]

(121 )

( 122)

(I23)

Putting (123) into (117) we get

k !.(k + Ilk) = L. E{~(k + 1)1!.T(~)} [Kz(~) + !(~) rl}!.(R.) (124)

~ = 0

Rearranging (124) in the way

k-I R(k + 11k) = L E{~(k + l)iO,) [~(~) + !(R.)rl!:.(R.) +

~=O

+ (125)

- 57 -

and substituting

and k-I E{~(k + l)lU,)} [~(R,) + ~(R.)]-Il:!.(R,) A x(k\k - I) L: =

R.=O

because k-I

l)i(R.)} r~W + ~(J/,)rll:!.(J/,) L: E{~(k + =

J/,=O

=

k-I ~ E{~ x(k)l:!.T(R.) + !u(k)l:!.T(J/,)} [~(R.) + ~(J/,)]-Il:!.(R.) = R.=O

k-I E{~ x(k)l (J/,) [ ~ (R.) + ~(R.)rl T

= L l:!.(J/,) = (for E{~ l:!. }

£=0 k-I

A L: E{~(k)l(J/,)} [~(R.) + ~(J/,)rl!:.(R.) = ~!(k\k - I)

R.=O

we finally get

form (107)

~(k + 1\ k) = ~ ~(k\k - I) + .!S.(k)l:!.(k) }

Z(k) = £ x(klk - I) + l:!.(k)

l:!.(k) = Z(k) - !(k\k - I) - innovation equation

!(klk - I) = ~~(klk - 1)

= Q

Equations (128), (129) constitute the innovation modeZ of the state space •

.!S.(k) is called the Kalman filter gain matrix, and to evaluate it we have to

consider

E{~(k)l (k)} = E~ ~(k) + ! ~(k) J [~T (kl k - I) £T + l (k) J} T T

= ~ E{~(k) ~ (klk - I)} £ +! ~(k) =

= A!(k)~T + !.~(k)

(J 26)

( 127)

)

(128)

(J 29)

(130)

- 58 -

where

~(k) ~ E{~(klk - I) ~T(klk - I)}

using relation y(k) = ~(k) + ~(k) and the orthogonal projection of

~(t) it is straightforward to show

so

!(k) = ~[~(k)£T + l! ~(k)][£ ~(k)£T + !(k)rl

Finally it can be shown (see Kalman R.E. (1960). Falb P.L. (1967).

Kalman R.E .• Bucy R.S. (1961) for example).

~(k + I) = ~ ~(k)~T + Q(k)

~(k)

(131 )

(132)

( 133)

(134 )

(135 )

Equations (128) and (134)(135) define the so called discrete-time Kalman filter.

An innovation approach to least squares estimation of the state has many

advantages and is broadly discussed and used in the modern system theory. Many

identification methods refer to this model, which incorporates less parameters

than ordinary state space model to identify from the same set of input-output

data. (it will be demonstrated in the next chapter). Also some order test

algorithms refer to this method. The complete derivation of the innovation

model was presented in order to introduce the reader to more advanced, however

intuitional1y quite simple, mathematical formalisms, which are not to be

avoided when studying literature on the multivariable system identification.

The following references are also warmly suggested to enlarge experience in

state space models handling: Jazwinski (1970). Kailath T. (1968). Kailath T.

Frost P. (1968). Kailath T. (1970). Kailath T •• Geesey R.A. (1971). Kailath T .•

Geesey R.A. (1973). Eykhoff (1974). Aesnaes H.B •• Kailath T. (1973). Gevers M ••

Kailath T. (1973). Meh,.a R.K. (1971). Akaike H. (1973). Akaike H. (1974a).

Akaike (1974b). Niederli~ski. A. Hajdasinski A.K. (1979) and others.

- 59 -

2.4. Generation of canonical forms from Hankel matrices

State equations for the purely conceptual noiseless system deliver the

already discussed transfer function matrix:

-I !.(z) = ~(z ~ -~) !

where !(z) is assumed to be proper or strictly proper.

According to Schwarz (1971), Ho-Kalman (1966), Hajdasinski (1976)

!.(z) can be described in the form of the exponentional expansion:

!.(z) f= i=O

M. -, i+1

z

where M. are already known Markov parameter matrices -, i M. = CAB - (q x p)

-1 ---

see Ho-Kalman (1966), Gantmache~ (1959).

Alternatively, applying the z-transform to the state and output equations,

( 136)

(137)

(138)

we may describe this system by means of the so-called "weighting sequence".

(See Hajdasinski (1978».

with B 13 = x - "-Q -0

= C Ak x (0) +

k-I

=~i!.o + 2: i=O

M. u(k - i-I) -, -

M. u(k - i-I) -J. -

=

(139)

It is very easy and intuitievely simple to define the equivalence of dynamical

systems in terms of Markov-parameters.

Definition 15 Two dynamical systems

are said to be equivalent if and only if their Markov-parameters

fulfill the following condition (Ho-Kalman (1966»,

for i = 0,1,2,3, ..... (140)

- 60 -

With the knowledge of Markov-parameters it is possible to construct the finite

Hankel matrix Hr - (qr x pr), which is a submatrix of the double infinite

Hankel matrix form relation (8).

M M ...... H --0 -1 = -r

M M ...... -1 -2

M M -r-1 -r

M r-1

M -r

M -2 r -2

It is very easy to notice that Hr can be expressed as a product of the

observability and controllability matrices (see Ho-Kalman (1966».

W = L~; ~ ~! ~2~ ••.• ~r-1~ 1 v

H -r

[~T i ~T~Ti (~T)2~T .•••• (~T)r-1l]

= C A B

C A

C :. ....

observability Matrix

(141 )

( 142)

( 143)

(144)

From (144) it is obvious that a row (column) of ~r is dependent if and only if

it corresponds to a dependent row (column) of the observability (controllability)

matrix (see Niederlinski, Hajdasiii.ki (1979». This means that there exists

equivalence between canonically observable and canonically controllable forms

(described in 2.2.1. and 2.2.2.) and forms which can be generated from the

Hankel matrix. This is stated in the following theorem by Candy, Warren and

Bullock (1978).

Theorem 7

- 61 -

1. If the rows of the Hr matrix are examined for predecessor

independence, then ~th (dependent) row, where ~ = i + qv., 1

i = 1,2, .... q is given by

l-.f rnin(Vi ,vj-d

L L a.'i,\( yj .. ",k

i"~ k~O

where a .. k and v. are invariant parameters for canonically 1. J , l.

observable couple (~,f).

2. If the columns of the Hr matrix are examined for predecessor

independence, then the ~th (dependent) column, where

~ = i + pn., i = 1,2 •.• p is given by 1

-l-1 W\i~(V\~,nj-d

= I. L Q'j,d j ... pI( 1-1;1 '.0 -

where a .. k and n. are invariants for the canonically controll-1J , 1.

able couple (~,~),

Candy, Warren and Bullock (1978) present an algorithm for the transformation of

the Hr matrix to a form enabling the simultaneous determination of the v. and 1

n. invariants by inspection. This algorithm may be treated as the determination 1

of the canonical structure of the cenceptual, nondist~bed system, having the

advantage that both sets of Kroneck~indices are formed in parallel.

An infinite number of canonical forms can be derived from the Hankel matrix ~r,

depending on the decomposition of the latter. No completely satisfactory

research has been made in order to reflect the hidden possibilities of the

Hankel canonical forms. As an illustrative example the minimaZ reaZization

algorithm by Ho-Kalman will be presented as a method using a "general"

decomposition of the Hankel matrix and the minimal realization algorithm

- 62 -

with use of the singular value decomposition as an illustration of a

particularly interesting decomposition of the Hankel matrix, giving

possibilities for interpretation of the noisy experiements in view of

least squares fit to a given data set.

3.4.1. The Ho-Kalman minimal realization algorithm

The minimal realization problem is referred to as finding the triplet of

matrices {~,~,~} using the external data like input-output signals or the

transfer function matrix. This problem is very well known in the system

theory, and quite extensively worked out at present. There is a huge number

of minimal realization algorithms, but still the originelwork by Ho-Kalman

(1966) shows the majority over another method. Let us present the Ho-Kalman

algorithm in the form of the following theorem:

Theorem 8 For an arbitrary, finite dimensional, linear, dynamical system,

given the input-output map, the canonical realization exists in

the following form:

Let q - be the number of outputs

p - the number of inputs

n - dimension of the realization

I. There is defined the following matrix:

k x J/, matrix [~~-k ] if k < J/, EJ/,

k J/, matrix

[~J (145 ) =k x

if k > J/,

k x k matrix [~ ] if k J/,

- 63 -

2. Choose r such that the relation

M • = -r+J

for all j ~ 0 (146)

holds.

3. Find a nonsingular matrix P(q x q ) and a nonsingular matrix - r r

where

P H Q --r-

H = -r

Q(p x p ) such - r r

=

Ik -"k

Opr-n

n

On Opr-n

qr-n qr-n

M -0

•.•.••..• •• M 1 -r-

MI' ••••••••• M2 -r- - r-

=

oH = -r

En qr

Epr

n

~l ••••••••.•.

M ••••••••••• -r

4. A canonical realization of the considered system is given by:

qr A = E P(OH).9, En

--n - -r ."r

B = Eqrp H EP --n --r."r

C =

M -r

(147)

shifted (148) Hankel Matrix

M -2r;'1

(149)

- 64 -

The proof of this theorem can be found in Ho B.L., Kalman R.E. (1966),

Kalman R.E., Arbib, Falb (1967), Schwarz H. (1971), Hajdasinski (1976).

In the Ho-Kalman algorithm the Hankel matrix decomposition is non-unique

and depends on the choice of the transformation matrices P and ~. Kalman

(1967) suggested the upper and lower triangular structure for the (!, ~)

or (~,!) respectively. For this sake the Andree (1951) algorithm was

perfectly suited. But it can be shown (Hajdasinski (1976) that by a different

choice P and ~ the phase-canonical form of the state equations can be achieved.

However, in general, when state variables are not well defined, it is not possible.

3.4.2. The mini~al realization algorithm with the use of Singular Value Decomposition of the Hankel matrix

The following decomposition of the Hankel matrix will be called the Singular

Value Decomposition (s.v.d.) (see Hajdasinski - Damen (1979)):

where

u - (r~ x n)

V - (r p x n)

H = -r

D is the n x n diagonal matrix

D = diag. (01. 02 ••••• • ,8 ) n

cr., for i = 1,2, ..... ,n are called the singular values 1

matrix consisting of n orthogonal columns u .• J

UTU I --n

matrix consisting of n orthogonal columns v j ,

VTV I --n

(ISO)

(151 )

It can be easily demonstrated that the Moore-Penrose inverse of Hr given the

- 65 -

s.v.d. (150) is:

as the other hand from (147) we have.

P H Q = En Epr - --r-" -qr ~

by definition P H HtH Q J - -r -r--r-"

P H Q i; P H Q --------r-~ '---r-"

J J

thus

Comparing (152) and (155)

=

v

J

J

A canonical realization of the considered system is given by

A D-IUT(OH )V - - ---T-

B = D-IUTU D VTEP = VTEP -----pr pk

C =

( 152)

(153)

(154)

( 155)

(156)

( 157)

(158)

Once the (s.v.d.) of the ~r is given, the realization (158) is unique. This

canonical form plays a dominant role in indentification of both the order and

parameters of the noise corrupted dyncamical multivariable systems, and will

be more broadly explained in chapters 3 and 4.

- 66 -

More facts about the realization and minimal realization theory employed for

the multivariable system identification are to be found in: Ackermann J.E.

Bucy R.S. (1971), Ackermann J.E. (1972), Anderson B.D.O. (1977), Anderson B.H.,

Brasch F.M. Jr., Lopresti P.V. (1975), Audley D.R., Rugh W.J. (1973a), Audley

D.R., Rugh W.J. (1973b), Barraud A., de Larminat P. (1973), Bingulac S.P.

Djorovic M. (1973), Bingulac S.P. (1976), Budin M.A. (1971), Budin M.A. (1972a,

1972b), Dickinson B.W. Morf M. Kailath T. (1974), Dickf.nson B.W. Kailath T.

Morf M. (1974) Faurre P., Marmorat J.P. (1969), Faurre (1976), Furuta K. (1973),

Gerth W. (1971), Gopinath B. (1969), Gupta R.D., Fairman F.W. (1973),

Hajdasifiski A.K. (1976), Hajdasifiski A.K. (1978), Hajdasifiski A.K. (1979),

Hajdasifiski A.K. Darnen A.A.H. (1979), Ho-KaZman R.E. (1966), Kalman R.E.(1963)

Kalman R.E., Falb P.L., Arbib M.A. (1969), Lal M. Singh H., Parthasarthy R. (1975),

Mayne D.Q. (1968), Rissanen J., Kailath T. (1972), Rissanen J. (1974), Roman J.R.,

Bullock T.E. (1975), Rossen R.H. Lapidus L. (1972), Rozsa P. Sinha N.K. (1974),

Shmask Y. (1975), Shiek L.S. (1975), Silverman L.M., Meadows H.E. (1969),

Silverman L.M. (1971), Sinha N.K. (1975), Sinha N.K. Sen A. (1976), Rozsa P.

Sinha N.K.(1975), Tether A. (1970), ThamQ.C. (1976), Zeiger H.P. McEwen J. (1974).

Example 5 Let us consider a two input, two output dynamical system, whose

realization {!,~,!! } is:

0 -I -I 0 -I

0 0 0 0

[: 0 0

:] -5 -I -2 G = H 0 F = -2 -3 I -I 0

0 0 0 0 0

-I 0 -2

constructing

H =

[~ 1

- 67 -

where h = -,

o o o

h -2

o o o

o

and performing the test for linear independence according to relation (52)

we get

v = 3, v = 2, V -1 2 12

v = 3 2 ,

so the transformation matrix TT will be (see (54»

FT 0 0 -2 0 -1 (!.T) 2

0 -5 0

-1 -3 0

-1 -1 0 0

0 0 -2 -2

thus

TT = 0 0 0 0 T =

0 0 0

0 0 0 0

0 0 0

0 0 0 0

2 -2 8 -1 0

5 -5 11 -8

4 -3 4 0 -3

2 -1 0 0

-1 9 -2 2

0 0 0

0 0 0 0

0 0 0

0 0 0 0

0 0 0 0

- 68 -

-I -I 0 0 0 T =

0 0 0 0

0 0 -I 0

0 0 0 0

0 0 0 0

Now the canonical form may be completed:

0 0 I 0 0 A I A

A = T F T- I I -11 I -12 I = ,

--- 0 0 I 0 0 I I I (see (58), (60»

-2 -3 -3 I 2 -I ---+----I ----------+------ I I

0 0 0 I 0 I I I I A I A I

-I 2 \-1 -2 -21 I """"'22 I

B = T G • o o

-I 0

o

o o o =

(see (61» o o

The realization {~,!,£,} is a canonically observable realization of the considered

system, and is equivalent (in the sense of Definition 15) to the realization

- 69 -

Example 6 Let us consider a two input, two output dynamical system,

characterized by the following series of Markov-parameters:

{M.L -1 1 0, ) ,2 ....

={[I.O 0.0

M -0

o.oJ; [0.8

1.0 0.0

0.21 ;[0.64

0.6 0.0

0.28];[0.512

0.36 0.0

0.296];

0.216

[0.4096

lO.O 0.28 1 0.129~' [

0.32608

0.0

0.24992 1;

0.07776 [

0.260864

0.0

0.215488) ••••• }

0.46626

M -. M -5

M -6

I. Applying the Ho-KaUnan.algorithm we will derive the realization

It is sufficient to consider H for -1

det H = I -1

det H = ° -,

rank H 2 -1

rank H = 2 -,

thus the system is of the first order and has the dimension n = 2. In such

a case it is trivial to find ~ and g matrices, which are:

P a I = [:' :,1 P-2

.9- a I = [:, : 1 q-2

q

and a = p

aq

- 70 -

The Ho-Kalman algorithm (149) provides us with

A = a (oH) a = aH p -I q -I

B a H = a I P-I p--,

c = H a -I q

a I q-2

~I = [0.8 0.2]

0.0 0.6

Eigenvalues: A 1

0.8

A - 0.6 2

Transfer function matrix of this system is:

!5.(z) -1

= ~(!z -~) ! = 0.2

(z - 0.8) (z - 0.8)(z - 0.6)

0.0 (z - 0.6)

II. Applying the s.v.d. realization algorithm for example for H • we get -. a following set of singular values:

0 = 2.5701194 1

0 = I. 4734138 2

0 = 8.6947604 . 10- 12 ;; 0 3

04

= 0

thus if it is clear that the system dimension ~ = 2, r - order = I.

Set of equations (158) provides us with:

~

[ 0.834575 -0.068194 ] Eigenvalues: A ~ 0.8 A =

-0.011896 0.565417 A = 0.6 2

'il' [ -0.568066 -0.259609 ]

-0.288067 0.772964

c = [- 1. 5041 70

-0.560574

- 71 -

-0.505194]

1.105450

Realizations {!,!,£} and {!,!,~} are equivalent in the sense of Definition 15.

Realization{!,!,f} delivers exactly the same !(z) as the realization {!'!'£}.

The great advantage of the s.v.d. realization is its uniqueness, and an easy

test for the system order.

-72-

3. Identification of the structure of the multivariable dynamical

systems.

The whole chapter will be based on the definitions 13 and 14, and on

structural properties of dynamical models for MIMO systems as described

in chapter 2. The task of the structural identification it is to determin

the suitable "comple~ity " of the chosen model. Thus in view of definitions

13 and 14 it will be determination of the order -r , or ( and) the minimal

dimension - n of the considered system ( MIMO ).

Such a posing of the problem is possible only for some strictly conceptual

systems having both a finite order and a finite dimension. In the real ,

noisy systems identification, however, one cannot search for any exact rand

n , because due to the noise these are systems of infinite order and dimension.

The only goal which we can aim at is to find a reasonably simple, and resulting

in good outputs of the model, a finite dimensional approximation of the real

system - i.e.to find finite estimates of the rand n.

3.1. Estimation of structural invariants - Guidorzi's method

The Guidorzi's method is based on the canonically observable form

of the state equations - see 2.2.1 , relations (56),(57),(58),(59),(60),(61),(62).

According to Guidorzi, the structural identification is defined as:

Definition 16 The structural identification of a multivariable system it is

the determination of the set of integers VI •••• V ,( Kronecker q

invariants) defining the structure of the couple ( !, ~ ) from

input-output relations, without the intermediate construction of

a parametric model.

-73-

Thus the Guidorzi's method requires the input-output description of the

dynamical system, which would use the Kronecker invariants.

(159)

where

X.T(k) [ y 1 (k) , y 2 (k) , ..• , yq(k)l

.!!.T(k) [ u 1 (k) , u2

(k) , ••. , uq (k) 1

~(z) [ f" ,., ... f""'] Q(z) = [ j" 'd ... 1""'1

Pql(Z) Pqq(Z) qql(Z) qqp(Z)

From the j-th component of the canonically observable form it can be written:

( x(k).d~f x.(k) ) 1 1

v. 1+1) r

v. 1+2) J-

= y. (k) J

zy. (k) J

2 Z y. (k)

J

T - zb - (Vl+ ••• +V. 1+I)u(k) r -

...............................................................

V. ) J

... + v. I)u(k) -r -

v. 2 b T (k) - z J - _ (v 1 + ... + v. + 1 ).!!. J-I

Thus the canonically observable state equation can be rewritten as:

(160)

(161 )

where

::. (z)

max i

1 •••••••• a z •••••••• a 'VI-I

z •••••••• 0 o 1 ...... 0 Oz •••••• 0 · ........ . · ........ . o ........ 1 • •••••••• z

, 'v -I O ••••••• z q

( v. ) L

-74-

w =

1(z) =

o 4 ••••••••••••• 0 ••• 0 ••• O. 0

!?.r 0 .•....•......•....••. 0

T ' T b. O I .... b l 0 ....... 0 O. 0 -Vl- -

o •••.•..•••...•.•••.•.••. 0

bT ~-'V + 1 0 •••••••••••••••• 0

q

'T b I •••• -<1-

T ' b -<1-V +1

q

I

zl T

o ....... 0

Finally the relation between ~(k) and ~(k) is found to be:

Comparing coefficients of (159) and (162) we obtain

p .. (z) LL

p .. (z) LJ

v. L = z

- a.. z LJ , v, .

LJ

v .. -1 LJ

and for q .. (z) it is necessary to construct a matrix ~ = M B LJ

( 162)

( 163)

(164)

where

'" B

M =

-75-

-a ll ,2 -a ll ,3 - ...

-a I I , V 1

I

-aql ,2 ..• -aql,v I • • q

-a

-a

o

o o

q 1, V I o .... ~ ............. 0

o •...•....••....... 0

and comparing coefficients we get:

q .. (z) 1J + •.. +

II, VI

...

•• , +

-a lq ,2 .•. -alq,VI . q

o

-a· . . . . . . . . . .. 0 I q, vI o q

o .......•.......•. 0

-a qq,vql

.........

J ••••••••••••••••••

(165)

(166)

From relations (162),(163),(159) it is seen that the set of indices

V , v, .•. , V can be deduced by inspection or from the knowladge of A or 1 2 q

of K(z); also from the parametric standpoint! and !(z) are equivalent.

Matrix C can be directly written if the Kronecker invariants are known. See

also Guidorzi(1973),Bonivento c,Guidorzi R(1971), Bonivento c,Guidorzi R (1972).

Guidorzi considers the matrix of input-output data given by:

-76-

YI (k) Y I (k+ I) Y (k) Y q (k+ I) ul

(k) ... u (k) '" q p

Y I (k+ I) YI (k+2) Y (k+ I) q Yq (k+2) ul

(k+I) .. u (k+I) •• p

Y I (k+N) Y I (k+N+ I ) •. Y (k+N) q yq(k+N+I) .• ul

(k+N) •• u (k+N) •• p

= [2:.1 (k) 2:.1 (k+ I) ... 2:.1 (k+N) , 2:.2 (k) I.2 (k+ I) .. 'lq (k) ... ~I (k) •. • ~(k) ... ] (167)

From relations (162),(161) and(166) the following relation between y.(k) and -1

u.(k) can be derived: -1

v . q S1

Y (k+v ) = ~ ~ a .. y.(k+j-I) + s s . 1 • I S1,] 1

1= J=

and V = \) ss s

v

!:-E i= 1 j = 1

a(V + •• • +v I+j)ui (k+j-I) I s-

Relation (168) shows clearly the linear dependence between present and

former samples of outputs and between present outputs and former inputs.

If we look closely to the relation (167), this property may very easy be

forecasted. Each subsystem in(167) is the Hankel matrix and relation (168)

has the form of the realizability criterion. It first is due to Guidorzi

(1971),that this property has been discovered letting determination of

VI ••••••••• Vq by selecting of nonsingular matrices being products of

these output-input vectors :

(168)

L. (y.) -1 -J

- i elements (169)

L.(u.) = [u.(k) ..• u.(k+i-I)] -1 -J -J -J

- i elements (170)

Constructi~g following! and ~ matrices:

R (8 1,8Z, ... ,8 ) p+q

-77-

(171)

(17Z)

d im{ S (81

, ••• , 8 )} q+p (81+ 8Z+ ••. + 8q+p )x(8

1 + 8Z + ••• +8q+p )

(173)

To complete the structural identification we need to built up a sequence

of increasing dimension matrices :

l(Z,I,I, ••. ,I); l(Z,Z,I, ••. ,I); •.• ; l(Z,Z, ••• ,Z); •.•

and selecting nonsingular ones, find indices v. while a singular matrix is 1

found. Suppose S(ll,lZ, ..• ,l ) is a singular matrix and let 1. is the index - q+p 1

increased by one , with respect to previous nonsingu1ar matrix in the sequence.

Then v. = 1.-1 and v .. = 1. for j = I,Z, ••• ,q for i ; j • This ends the 1 1 1J J

structural identification.

The Guidorzi's method it is the first coherent approach to the structural

and parametric identification. For 11 really" multi variable systems with a high

number of inputs and outputs, however, the selecting procedure for S matrices

increases complexity of already complicated algorithm. In noisy cases the

structural identification produces higher dimensions then the realizations

based for example on the Hankel model or on partial minimal realizations - see

Tether(1970), Anderson(1977),Roman,Bullock(1975).

Also for noisy cases the Guidorzi's method requires a priori knowladge of

a covariance matrix of the noise. In noisy cases the structural ( and parametric)

identification , according to Guiclorzi, must be performed on the base of

following relations :

Y!"(k) = y. (k) + dey. (k)) J J J

(174)

-78-

'* u. (k) J

u. (k) + d(u. (k» J J

(175)

where 1*) are for the noisy signals.

Assuming that the covariance matrix of the noise vector

N (d) = prob lim..l..( !!, !!,T) (176) N->oo N

it is possible to find

and

... R

T R + n

s .. = (t)T(~*)

(177 )

(178)

Thus for the structural identification we need the II compensated estimates "

of Rand S Assuming that input and output noises are II zero mean " white

noises and that ~(d) - the estimate of ~(d) can be found, the compensated

estimate of S is

(179)

Discussion. The main difficulty for application of this method lies in the fact ,

that practically ~ will never be a singular matrix and the

structural identification will become very unreliable and tedious , unless

the additional constraints are imposed on the problem. It also excludes the

existence of another essumption about the noise nature which were previosly

made.

-79-

3.2. Order tests based on the" innovation approach" - Tse-Weinert's

order test.

where

Tse and Weinert (1975) started from the following model

~(k+l) = ~~(k) + ~(k)

Z(k) = £~(k) + ~(k) (180)

~(k) and !(k) are zero-mean Gaussian noises

with covariances:

E{_W(k)_w(j)T} - W 8 . - kJ

It means that there is considered a multivariable dynamical system for which the

input signal is stabilized. and we observe deviations from the steady state

caused by the state noise - w(k) • Unknown parameters are {x .n.A.C.W.V.D.}= {y}. o -----

The objective is to estimate {r}. using output data {!}N= {Z(I).Z(2) •...• Z(N) }.

As it is seen, a part of the objective is to estimate the system dimen8ion 0,

( called by tse and Weinert - the order of the MIMO system ) and only this part

of the algorithm will be discussed here. The set of parameters {y} is quite

extended and requires a large amount of data. As it is pointed out by Tse and

Weinert. {y} may not be identifiable and there fore the innovation reprezentation

is proposed :

supposing that

-x(k+ I , k) - is the predicted state vector ( the conditional mean at time -

k+l. given the estimate ~(k) at time k )

~(k+ I) - corrected state vector at time k+1

K - steady state Kalman filter gain

we get

-80-

-x(k+llk) = ~ ~(k) - one step prediction equation (181 )

~(k+l) = ~(k+llk) + !(L(k+l) - £~(k+llk» - the correction eq. ( 182)

L(k) = £ ~(klk-I) + v(k) - output equation (183)

~(k) = L(k) - £~(klk-I) - innovation equation ( or " zero mean (184)

innovation process)

From equations (181) ~ (184) we get

~(k+llk) A x(klk-I) + ~!~(k) ( 185)

y(k) = £ ~(klk-I) + ~(k) (186)

The set of equations (185) and (186) is called the innovation model of the dynamical

multivariable 8Y8tem. ( see references to chapter 2.3.3. of this report)

Additional assumptions are

A - is stable

(~,£) - is observable

(~,!) - is controllable

9. - is the unknown covariance matrix of ~(k)

B = A K - is the optimum gain

-dim {~(klk-I) } - is finite but unknown

The canonically observable form of (J8,;) and (186) is applied. If vi again

denote the Kronecker indices such that

q

LVi = n

i= I (187)

-81-

from the observability matrix and from the definition of the Kronecker indices

there follows already described the observable canonical form (56),(57),(58)-(62),

and this implies an existence of a unique set of {a .. k} such that for i = 1,2, •• q 1J.

T vi c. A -1 -

T c. -1

= i v.-I ~~ j=1 k=O

T k a .. k c. A 1J, -J-

c: is the i th row of C matrix. -1

if

if

Introducing the covariance matrix of states - S , -ot

v. > 1

o

v. = 0 1

- - - -r T T T { ~(k+I' k)l(k+llk)} = ~{~(klk-I)~\k(k-I)}~ + ~h:.(k)~ (k)}! +

( 188)

(189)

(190)

+ M ;(klk-I)l(k) }!T + !{~(k);h\k-1)}~T (191)

Taking expectations of both sides of (191) we get

S = A S AT -x --x- + ( 192)

for ~(k) was assumed to be a zero-mean noise

Defining

(193)

and considering

X.<k) £ ~(klk-I) (194)

-82-

finally we get:

!(O) = c s CT + Q (195) --oc-

!(a) = C Aa- 1S ( 196)

s = A S CT + !Q ( 197) --oc-

denoting as r .. (a) the ij th element in _R(a) , and as s. the j th column ~ ~

in S , using (188) from (195),(196) we get :

r .. (v. + , ) 1J 1

where T = 1,2, ...

v. 1 T A ' A'- = c. S.

-1- - -J

~~ 1=1 k=O

i-I v -I

2: ± 1=1 k=O

Basing on (196) and (198) we get:

r .. (v.+,) 1J 1

Thus for i=1

defining

T !.I

v -1 ~.J-. ~ ~ ail,k r 1j (k+,) 1=1 k=O

v -I ± all,k rlj(k+T) k=O

T k ,-1 a' l k c1 A As. 1, - - - -J

T k ,-I a' l k c1 A A s. 1., - - - -J

v. > 0 1

v. = 0 1

T = 1,2,3, •...

v.> 0 1

(198)

V.= 0 1

( 199)

(200)

-83-

rlj(l) r lj (2) r Ij (k)

r lj (2) r Ij (3) r lj (k+l)

!.I (k) (20 I)

r I j (k) r lj (2k-l)

Remark:

This is worth a notice, that rlj(cr) are the Markov parameters of the I st

subsystem in the canonical repre.entation, treated as the multi-input/single-

output subsystem. Thus (200) is merely the peaZiaabiZity aPitepion for the

first subsystem, and (201) is the Hankel matrix of the first subsystem. In such

a case the Hankel model can be applied :

(202)

Remark:

The Tse-Weinert's order test is nothing but a Hankel matrix determinant test

which will be described for a more general case later in this chapter.

If d I (k) def I det !.I (k) I =

t d I (k) > 0 for k = J,2,3, .•. v1 (203)

d I (k) 0 for k > vI

thus if !(cr) were exactly known, vI could be found by testing dl(k) for

k=I,2,3, .•• until dl(i) = 0 and then vl=i-1 ,which completes the order test.

Since only L! J N is avaliable ,!(cr) must be estimated the following way :

!(cr) = -N-

N-cr ~ ~(k+cr) ~T(k) k=1

(204)

-84-

~(cr) is a strongly consistent estimate of ~(cr) for a stable A and N+oo

Replacing !I(k) by !I(k) • we can perform the order test.

If the first sharp decrease in dICk) occurs at time k = k* • then vI is

chosen as vI = k*-I ( in the original paper by Tse - Weinert (1975) it is

vI = k* • but there is a different meaning of what is the" point where the

sharpest decrease occurs II In Tse and weinert's work it is always one point

before the decreased value of dICk) occured). ~

d,l"')

I --- -,---, ____ + __ ...l __

, ' , I I , , ,

I , , , , I I

-:... -=--..:-..:.=:::-=.-_ -:. T"_-=-=, -:.. __ ~ 1 :z. 3 ... I

Behaviour of the dICk)

6

The estimate of ~I is found from

~I

According to the order test criterion

k* = 4 => v = 3 1

(according to Tse-Weinert

k* = v = 3 ) 1

(Z05 )

(206)

~l is a strongly consistent estimate of ~1 ' provided vI = vI .For i=2.3, ... q,

v. and S.are computed in analogous manner. For i=Z. T = 1.2 •... (VI+VZ) 1 -1

E..2

.!2(k)

-85-

: r 2 ' (I) ... r2' (k) • J J • :r2 ,(2) ... r 2 ,(k+l) ___________________________ + __ 1 __________ 1 ________ _

r1j(v1+l)

r1j

(v1+2)

r 1j (2v 1)

r 1j (2v t 1)

• • • • · ' • • •

• • • • r 1j (2v

1+k-l) :r

2j(v1+k) •• r

2j(v

1+2k-l)

and again for the estimate .!2(k) of !2(k) we get:

and v2 is found testing d2

(k).

Remarks:

(207)

(208)

The method by Tse and Weinert is a combination of the innovation approach to

the state space with the canonical reprezentation of the innovation model,

which decomposes the original model into q interconnected subsystems having

a structure quaranteeing the complete observability of the j th subsystem

from the j th output component. Each of such subsystems is driven by the white

noise. The stochastic realization theory is applied to determin orders of

subsystems ( orders understood as dimensions)

The method is more convincing then the Guidorzi's method and delivers

very good results of the structural test.

The concept of the stochastic minimal realization of the sequence of the output

autocovariance coefficients was proposed by Rissanen(1974).

-86-

3.3. Order tests proposed for the transfer matrix model of the MIMO system

Furuta's approach.

The main idea of the Furuta's approach is to identify coefficients

appearing in the transfer matrix of the considered system, assuming a certain

degree of the Common Denominator of the transfer matrix, This identification

is performed employing the criterion function J.

J =

where

-t. i=1

r .u(k-i) -1-

(209)

~(k) = ~(k) + ~(k) - (p x 1) dimensional input measurement (210)

vector.

2(k) deterministic input (p x 1)

!'.!(k) stochastic input (p x 1)

E (~(k» = 0 (211 )

~(k) = x,(k) + !!.(k) - (q x 1) dimensional output measurement (212)

vector

x,(k) - deterministic output (q x 1 )

!!.(k) - stochastic output (q x 1 )

E (!!. (k» = 0 E(!!.(k)!!.T(t» = RI c (213) -- kt

E(~(k)!!.T(t» 0 (214)

Furuta assumes further the following model of the transfer function matrix

!.(z ,~)

Q(z,~)

r. -1

T ~il

-87-

----'-_ [ Il z -I +

Q(z,~)

+ ••• +

(q X p) matrix ~ .. 1]

(q x I) vector

(215)

which corresponds with (209). Using (215) and (209) the vector of parameters

is identified minimizing J with respect to ~.

The parameter estimation procedure is described in Furuta(1973). Assuming

the transfer function matrix has already been identified, the nonminimal

realization of the transfer function matrix can be found as :

~(k+1 ) = ~(k) + £':: (k)

Rn (216) I.(k) = ~(k) ~(k) E

where 0 •..... 0 -a I

R,:;. !~ I -I

F = G = b 0

0 .... 0 I -a I II

1-

H = [Q " .. . ,Q, .!.J

This is a very well known Frobenius canonical form ( see for ex. Hajdasinski(1976»,

and it is seen Q(z,~) - is the anihilating polynomial of F. From this form

Ho and Kalman(1966) extracted a controllable and observable realization,

-88-

called the minimal ~ealization. It is based on the Hankel matrix, which can be

derived as the product of the controllability and observability matrices.

w

(217)

H = [vT w1 -n --

(218)

Furuta introduces the concept of ~ - practical controllability and

observability, which gives a solution to the order test and further to the

£ - minimal realization :

Definition 17 The state x of the system (216) is said to be ~- practically

controllable ( observable ) if ~ is element in the space

spanned by the eigenvectors of ~T(yyT) corresponding to the eigenvalues

larger then ~,W and V denote controllability and observability matrices.

Remark It is easily seen that seeking for eigenvalues of the ~~T(~ ~T)

matrix is equivalent to the diagonal decomposition of the HT matrix, ...., which is only slightly different from the singular value decomposition of the

Hankel matrix.

Definition 18 The system represented by the states which are ~- practically

controllable and observable is said to be ~- practically

minimal realization.

The order test is made by comparison of eigenvalues of the ~~T , and

truncation of the selection matrix consisting of normal eigenvectors of

~ ~T, such that consists only of these eigenvectors which correspond to

eigenvalues larger than c.

-89-

w.(i =I, ..• ,n ) - normal eigenvectors of W WT -1 0

If £- is chosen as

v > £ > vn +1 n 0 0

where

n 0

2: v. 1

i= 1 -= n

L: v. 1

i=1

v.( i = I, ••• ,n) - eigenvalues of W wT 1

(219)

(220)

(221)

then the order test is considered as compl.eted, and £ - minimal realization

of the system is found as :

{ F -0

Remark

G -0

H = H s } -0

(222)

This refers to the notion of the numepiaal pank of the Hankel matrix

which will be discussed while considering the 8inguZaP value deaompo-

8ition of the Hankel matPix.

The Furuta's method is rather inefficient and incorporates an unnecessary step

of the transfer matrix identification with arbitrarily chosen ~. This method,

however, pointed out the way to the more advanced methods of the approximate

minimal realization and to the eigenvalue problem in determination of the system

order.

-90-

3.4. Miscellaneous order tests. The pattern recognition method by

Thiga and Gough.

The approach presented by Thiga and Gough is direct in the sense the

model parameters are not estimated at each 'step. The test is based on

a measure of the linear dependence(op independence) of features displayed by

each model opdeP.

This method is totally empirical and assumes identification of the order

(understood as a degree of the transfer function denominator) of individual

subsystems, such that the final system is described in form of the transfer

function matrix.

G = [gij 1 i=1,2, ... ,q j = 1,2, ... ,p (223)

where g .. correspond to the linear differential or difference equations 1J

n. m n.> m i=I,2 ... q

L: L: 1

d.ky. (t) c j J1.U j (t) 1 1

k=O J1. = 0 d ik " o j=1 ,2 ... p

and d ik and c j J1. are differential operators

or

y. (k) 1

9.=0

b. o u.(k-£) -J'" J

J1.=0

a. o y.(k-£) 1", L

k 0,1,2, ... , N

The order test is completed when all subsequent n. are found. 1

(224)

(225)

Authors proposed to solve the order recognition problem by testing a single-

valued function of pattePn, which will reflect a decision surface f(x) = £,

-91-

where E - is the treshold of pattern recognition. For the sake of the order

test for a noisy system, such a function must serve two following tasks :

I. help to spread the cluster of the patterns in the two classes further

apart in the pattern space.

2. reduce the dimensionality of the pattern space by combining dimensions

The input-output cross-correlation function is proposed as a one for the

training of the system :

R (k) uy =

T

M ± j=1

u(j) y(j + k) (226)

Authors are reporting quite a succes in the order discrimination during an

experiment carried on for a set of II representative systems " chosen from

amongst the vast possible combinations of characteristic roots of low-pass

filters, up to seventh order. The machine was learned to extract from such

patterns purported characteristics of a given system order and type of roots.

This type of approach was tested with different types of processes proving

its usefulness.

3.5. Akaike's FPE ( final prediction error) and AIC ( Akaike's maximum Information

Criterion) as order tests for MIMO systems.

Akaike has proposed two new approaches to the order determination and

consequently parameter estimation. These two methods - FPE and AlC are

assymPtoticall~equivalent in final results, as the distribution of the equation

error (for the AR model - considered as the prediction error ) converges to the

Gaussian one.

These two methods employ much more statistical properties of given measurements,

than some 11 a priori II assumptions about its nature . However one general

-92-

assumption, which in practical cases almost always holds, is the assymptotic

Gausseness of the time series samples • To get some more insight into these

two methods, let us study some important passages in their derivations, however

no claim for completeness is being made.

3.5.1. Statistical predictor identification - Final Prediction Error

Approach.

Let us consider first a single input-single output system and assume that the

output of this system is a stationary and ergodic process Y(n). In practice

Y(n) is given as a function of the recent values of Y(n) and the the structure

or the parameter of the function is determined. There is considered the situation

where the structure is identified using an observation of a process X(n) and,

using the structure, the prediction is made with another process Y(n) which is

independent of X(n), but with one and the same statistical property as X(n).

The FPE is defined as the mean square prediction error

FPE of X(n) - 2 = E {(X(n) - X(n)) } (227)

for X(n) being the predictor of X(n)

When the process X(n) is stationary and the predictor Y(n) of Y(n) is linear

and given by

M

Y(n) = ~ ~(m) Y(n-m) + ~(O)

m=1

where ~(m) is a function of X(n)

Using (227) and (228) we have

FPE of ~(n) = a2(M) +

M ~ ~=O

(228)

(229)

where

(i(M)

-93-

M

E {(yen) - L: "M(m) Y(n-m)

m=1

2 - "M(O) ) }

where "M(m) denotes the member of the set of parameters {a(m)} giving.

in the sense of mean squares, the best linear predictor ie.

M

min E {(yen) - ~ a(m) Y(n-m) - a(0»2}

I a (m) ( m=1

ll"M(m) ~(m) - "M(m) m 0.1.2 .... M

and

VM+ 1 (i.m) = E(Y(n-i)Y(n-m» i.m 1 ,2 , .•• M

VM+1(0.m) VM+ 1 (m. 0) = E(Y(n» m 1,2, ••. M

VM+1(0.0) =

(230)

(231 )

(232)

(233)

(234 )

In the relation (229) all components containing expectations of products

of noncorrelated quantities are neglected.

It is seen that FPE is composed of two components: prediction for a given M­

a2(M) • and the second due to statistical deviation of ~(m) from "M(m).

Behaviour of the FPE .namely decrease of a2(M) for increasing M and increase

of the second term for increasing M. suggests that there exists an optimum

for a certain M. which is nothing but the order of the autoregressive model(228). 'TPE

x "'F-'\>E. :E x

. * G'l(IM)

0--0 '[~~-

z..

-94-

This idea is worked out further in Akaike(1970). Following this idea we get:

M

X(n) = L a(m) X(n-m) + a(O) + c(n)

m=1

(235 )

where c(n) are the samples of the" white noise" • uniformly distributed and

E(£(n)) o (236)

If there is a collection of data avaliable. { X(n) ; n = -M+l. -M+2 •••• N}.

the parameter ~(m) is defined as the least squares estimate of a(m) ( ~(m)

is the parameter of the predictor (228) and we are going to find it basing

on observations of X(n)).

Defining

where

C (m.i) xx =--N

n=1

N

x = m

N L. X(n-m)

n=1

M

C (m.i) a..(m) xx M = -- L N n=1

M

(237)

m=O,I,2, •.• M (238)

=> L. Cxx(m.i) ~(m) M M

_1_~ ( L ~(m)X(n-m) - ~(m) Xm) ( X(n-i) - Xi )

N n=1 m=1 m=1

M M

L C (m.i) a..(m) xx M 1 '"" -= -.L-, (X(n) N

n=1

(239)

m=1

-95-

where

M N

L 2: ~(m)X(n-m) m=1 n=1

(240) N

Thus for N large enough we can write :

M

L e (m,"') ~(m) = e (0,"') xx xx (241 )

m=1

{ '" = 1,2,3, ... ,M for

m = 0, I ,2 , •• • ,M

according to (237) and (238). Relation (241) can be rewritten in the form

e (I , I) e (2, I ) xx xx

e (1,2) e (2,2) xx xx

e (I,M) e (2,M) xx xx

or

if e 1S nonsingular -xx

~= e -Ie -xx -M

e (M,I) xx

e (M,2) xx

e (M,M) xx

if e is singular or ill conditioned -xx

~= + c e -xx -M

~(I)

~(2)

~(M)

e (0,1) xx

e (0,2) xx

e (O,M) xx

where + C - is the Moore-Penrose pseudo inverse of C . -xx -xx

(242)

(243)

(244 )

(245)

-96-

The" zero" coefficient ~(O) is estimated basing on (235)

M

~(O) Xo - .L. ~(m) Xm m=1

Following the definition given previously - (228) using (246), we get

M

yen) =2: m=1

~(m) ( Y(n-m) - Xm ) - x o

the relation for the predictor yen) of yen).

Assuming that yen) is generated by relation

M

yen) L a Yen - m) + a + 6(n) m 0

m=1

where

6(n) has exactly the same statistical properties

M

as E(n), we get

M

(246)

(247)

(248)

yen) - Yopt(n) = 6 (n) - L lI~(m)y(n-m) - ( lIXo - L ~(m)lIXm )

for a m

m=1 m=1

~(m) are coefficients of the optimal predictor Yopt(n).

(249)

Relation (249) is somewhat abstract and refers to statistical properties of

the optimal predictor. It is assumed here,the process yen) is stationary and

ergodic. thus defining the following:

lI"M(m) ~(m)

yen) yen) - E(Y(n» (250)

lIX~ X~ - E(X(n»

-97-

basing on (247) and(248) ,(249) we get two relations

M

yen) - Y (n) opt

M

o(n) - (L. M

~(m)Y(n-m) -~ ~(m)Y(n-m) - 2. ~(m)E(Y(n-m) m= I m= I m=1

M

+ L ~(m)E(Y(n-m)) ) - (X - E(X(n» ) + o

m=1

M

+ Z ~(m)( Xm - E(X(n»

m= I

from (249)

M

yen) - yen) = o(n) - L ~(m) (Y(n-m) - Xm) -

m=1

M

ao = Xo - L ~(m)Xm m=1

thus

M ~

yen) - yen) 8(n) -~ ( ~(m) - am)Y(n-m) =

m=1

M

= o(n) - Z « ~(m) - am(m» + ( ~(m) - am» Y(n-m) =

m=1

M

= o(n) - L. lI~(m) Y(n-m) +

m=1

M

2. (~(m) - am) Y(n-m)

mel

a Y(n-m) + a m 0

(251 )

(252 )

(253)

(254)

From (251) and (254) we see that ( yen) - yen»~ will statistically be equal

( yen) - Y t(n» if : op

+

M z:. ( "M(m)

m=1

? - a ) Y(n-m) =

m

-98-

M

L (~(m) - "M(m) ) y(n-m)

m=1 M

( Xo - E(X(n))) + ~ ~(m)(Xm - E(X(n)) )

m=1 (255)

From (255) it is seen that statistically left and right sides are equivalent

if am ~ "M(m) , thus (249) is proven. Taking into account independence of

yen) of 6"M(m) and 6X ~ , we get :

M M

FPE of yen) E( yen) - yen) )2 = (52 + L Z E(6"M(m)6"M(~)Rxx(~-m) +

m=1 ~=I

where

+ E( 6X o

M

2: ~(m)6Xm)2 m=1

R (~- m) xx

2 E( X(n-~)X(n-m) ) - (E( X(n) )

(256)

(257)

For the sake of numerical solution we have to be interested in assymptotic

properties of the FPE, which hopefully can be a lot simpler to handle than

statistical evaluation of all subsequent quantities of the relation (256).

For this sake quantities 6Xo and 6"M(m) have been introduced.

For the asymptotic evaluation of the FPE we will need the following theorem

Theorem 9: Under the assumption of the stationarity and ergodicity of X(n),

the limit distribution iN 6X =.(J;i1 (X - E(X(n)) ) and o 0

IN' 6"M(m) =.fN1 ( ~(m) - "M(m)) for m - 1,2, ••. M,

when N tends to infinity, is (M+I) - dimensional Gaussian with zero mean and

the varianae matrix :

-I !lr.!

where

and

M

1 ... L "M(m)

m=1

-99-

!M - (M x M) matrix of R(i,m)

o - denotes a zero column vector

R (i - m) xx

( this theorem is a special case of the limit theorem as presented in the

book of Anderson T. w. (1971) ).

From the ergodicity of the process it is clear that C (i,m) converges to xx

R (i- m) as N+ 00 , with probability one. Thus !. is a consistent estimate xx -M

of ~ ( a vector of parameters of the optimal predictor) with convergence

with probability one.

From (244) we have

- (i) ( -I

C (O,i) (258) = C (m, i» ( ~ -xx -xx

- (i) ~(i) ... (C (m,i) )-I( C (i» (259) ~ -xx -EX

where N

fsx(i) N- I 2: e(n)( X(n-i) - Xi ) (260)

n=1

1I~(t) = ( C (m, i) )-I(C (t) ) (261) -xx -ex

From the consistency of C (m,i) it is concluded that that the limit distribution xx

of iN lIKo and iN'1I"M is identical to that {N'lIKo and -iN''\;ICex

- 2 Now,instead of taking the expectation of ( yen) - yen»~ as it was done in (256),

the conditional expectation of (Y(n) - Yen) )2 for a given X(n) is considered.

The last is denoted by

E{ ( yen) - yen) /1 X(n) }

-100-

From the independency of Y(n) and X(n) it comes

M M

- 2 a 2

+ """ """ E{(Y(n) - Y(n)) / X(n) } = ~ ~ m=1 ~=I

M

+ (Mo - L ~(m)Mm)2 (262)

m=1

Again in the limit the difference between-{N'Mo and{N'Mm are stochastically

vanishing ( m= 1,2, ... ,M) and Akaike (1970) demonstrates that:

N { E( ( Y(n) - Y(n) )2/ X(n) ) - a2 } (263)

has a limit distribution with expectation equal to (M+I)a2 , thus

N { ( FPE )M - a2 } =(M+I)a

2 (264)

(265)

where (FPE)M stands for an asymptotic evaluation of FPE.

Now remains to find an apropriate estimate of a2 • From the ergodicity of X(n)

it is concluded that

S(M) c (0,0) xx

M

-2: ~=I

a..(~)c (O,~) 1'I xx (266)

is a consistent estimate of a2 • It is shown further in Akaike(1970) that

the expression

M + I )-IS(M) ( I - N (267)

is the best estimate of a2 while N+oo. Thus the final Akaike's FPE criterion is

-101-

(268)

The order M is chosen which minimizes(FPE)M' thus completing the order test.

For the sake of MIMO models identification. the order test can be performed in

the very similar way. The only assumption to be made is that the multivariable

system is described by the following autoregressive model :

where

M

!(j) = L A Y(j-m) -- +

m=1

y(j)

!(j) = y(j) !;!(j)

A -0

+ ~(j )

Y I (j)

Y2(j)

y q (j)

A (p+q) x (p+q) matrix of parameters -m

!;!(j)

~o - (p+q) x vector of initial conditions

-

!!:(j) - (p+q) x I random vector satjsfying the relations

E{!!:(j) } = Q.

E{ ~(j)!T(j_m) } = 0 for m >1

o. S Jm -

ul (j)

u2 (j)

u (j) p

Given a set of observed data (!(j) ; j a 1.2 ..... N ). where Yi(j) ( i

will in general denote a i th component of !(j).

I. Define Y.(j) ( i = 1.2 •••• k. k=p+q)( j = 1.2 •••• N ) by 1

Y i (j) Y i (j)

(269)

(270)

1.2 ... q+p)

(271)

where

y. = 1

1 N

-102-

II. For 01 = 0,1,2, •.. L , where L is maximally allowable order of the model

(272)

and should be generally kept below N/5k, define the(i,t) th element of

(k x k) matrix

C (i,t) -01 N

C as: -01

N-m

L )\ (j+m»)\ (j)

j=1

(273)

III. Basing on (270)(271)(272) and definition of ~ the sample covariance matrix

of residual error can be computed by the following recursive formulae:

M

~(M) = s;, L A (M) CT

-01 -m 01=1

where A (M) can be evaluated using a recursive algorithm. -01

~(M)

!l. (M)

Q(M)

~(M)

A (M+ I) -01

M

~+I - 2: A (M) C -01 -M+I-m

01=1 M

C - 2: B (M) C -0 -m -m

01=1

~(M) {g(M)}-1

{~(M) T}{~{M)}-I

m i,2, ..• ,M

(274 )

(275)

(276)

(277 )

(278)

(279)

-103-

~m(M+I) = Q(M) for m = M+I

B (M+I) -m ~m(M) - ~(M) ~+I_m(M) for m = 1.2 •...• M

B (M+ I) -m

§,(O)

~(M) for m M+I

_Q(O) = C -0

(280)

(281 )

(282)

(283)

IV. Subsequently the order test must be performed according to the (FPE)M criterion.

which in this case takes the form

FPC(M) = 1\ ~qxq (M) II (I + M~+ I ) q. ( I -Mk+1

N (284)

where ~qxq(M) denotes the qxq submatrix in the upper left corner of §,(M).

Adopting the value of M which gives the minimum of FPE(M) for ( M = 1.2 .... L).

as the order of the model (269). the necessary matrices of coefficients can be

found by relations (274) - (283) • n . II - denotes a determinant .

This completes the order identification in terms of Akaike's FPE(M) for MIMO

systems.

-104-

3.5.2 Akaike's maximum Information Criterion approach

Akaike has proposed a new approach to the order determination and

parameter estimation , based on the FPE statistics which was a simplified

concept of the broader concept of AIC ( Akaike's Information Criterion-

(1972),(1975) ). The AIC criterion is built on the extension of the maximum

likelihood principle , very well known and broadly applied everywhere,

where the computing time factor does not playa main role. A brief and elegant

explanation of the classical maximum likelihood method, the reader can find

in Eykhoff(1974).

Considering y being the output vector and the components of L random

variables {y(I), ... ,y(k)} (for 5150 systems for the time beeing), the

joint probabiZity function for L will be :

p(y(I),oo.,y(k); b ) = p( l; b ) (285)

where

k L depends on ~ , the vector of parameters.

This is our "a priori" knowladge; a posteriori knowledge encompases values

of the random variable as measured. From this, an estimate ~ of the b can

be determined.

To distinguish it from relation(285), the joint probability function

k of L and ~ is called the likelihood function and denoted as L

L{ y(I),oo.,y(k); ~} (286)

Fer the sake of convenience,lnL is usually considered and the logarithmic

function , as the monotonic function, has the maximum at the same value of

~ as L.( Eykhoff(1974». This value of ~ can be obtained by solving:

-105-

(287)

8 - is called a maximum likelihood estimate of ~. In the classical theory

of estimation it is usually assumed that observations in ~k are independent

so that

k k p{~ ;~ } L p(y(i) ;~) (288)

i=1

and k k n L{~ ;f } L(y(i) ;f) (289)

i=1

k k

R,nL{~ ;f} L R,nL(y(i) ;f) (290)

i=1

To discuss properties of the Akaike's method it is necessairy to discuss

some properties of the maximum likelihood method.

A

Let us define a bias ~(£) in the estimator ~, . . k

gLven the observat10ns ~ :

6(,,-J" ~-El~('i~)I\2) ~ \e- \&l'i<)\,('2.I<.JI1)d';!~ " and the error covariance matrix !(b) of the estimator f2

In general, it is not possible to compute ei ther L'l (~) or ~(b).

(291)

( 2.92)

However, for

any unbiased estimator, we have the following Cramer-Rao lower bound.

-<

R(~) > ~~. (~) (2.93)

-106-

1~d"1~-E U~,L"pll{'~)\~J ~

~ E lU~ lnpl'i\ £) 1~b \.\'\pl'i"i E) T'\ ~} (2.'l~)

is the Fisher information matrix.

Relation (293) is equivalent to

which also means that every diagonal element of !(b) must be no smaller than -,

the corresponding element of 1~' (Ill

Thus the Cramer-Rao lower bound provides a lower bound on the accuracy to which

any component of b can be estimated. The technical assumptions required in the

derivation of the lower bound do not include any assumptions of linearity or

Gaussianess. Thus the bound is well-suited for nonlinear problems such as the

parameter identification problem for dynamical systems. It is also worth noticing

that6(b) and !(b) and 6 k(b) depend on b. ":1..

Some asymptotic properties of the maximum likelihood estimate will also be very

interesting. The asymptotic properties are concerned with the limiting

behaviour as the number of observations becomes infinite. The first assumption

to be made is the identifiability condition.

(2%)

This assumption means that no two parameters lead to observations with identical

probabilistic behaviour. It is obvious that if the identifiability condition is

violated for some pair ~1' ~, of parameters, then ~, and ~, cannot be dis-

tinguished no matter how many observations are made.

Assuming independent observations, identifiability and additional technical

assumptions, the assymptotic results are as follows:

-107-

I. Consistency

( 2.~7)

2. ASymptotic unbiasedness

3. ASymptotic normality

4. Asymptotic efficiency

In other words, as the number of processed observations becomes infinite, the

maximum likelihood estimate converges to the true value of ~, and the A

parameter estimate error is asymptotically normally distributed (~- ~) I

wi th covariance matrix d-t (9.) , so that the Cramer-Rao lower bound is

asymptotically tight.

The independence assumption implies an additive form for the information matrix

(2. 9 B)

is the information matrix for a single observation.

In terms of the asymptotic covariance matrix, we see that:

(2.~9)

-108-

This was introductory information. Let us return to the AlC-criterion. /'.

Given a set of estimates ~ of the vector of parameters ~ of a probability

distribution with density function k p(Z ;~), we will choose the one which will

give the maximum of the expectation from the tn (logarithm) of the likelihood

function, which will be by the definition:

l300)

where ~ - is the random variable following the distribution with the density A

function p (y;~) and is independent of 0 . For the purpose of discrimination between the two probability distributions

with density functions p.(y) (i = 0,1), all the necessary information is 1

contained in the likelihood ratio T(y) = Thus Akaike suggests,

instead of the classical maximum likelihood principle, maximization of

information theoretic quantity, which is given by definition:

Relation (301) is very well known as the Kullback-Leibler's mean value of

information for discrimination k A k'

between p{Z I~) and p{y I~) and can be

interpreted as the distance between the two distributions.

EW(~l~\~l)= ~ ~('iI~)lnp(~10)cl.y -

- ) 1"('-2.12 )lnpl'i I~)cli ~

('Oo~)

-109-

Following the concept of the measure of information, Akaike defines the

"mean amount of information to be discriminated pel' obsel'Vation" by

IIp,,po,cp) = fqJ( \'1(Y)) Poly)dy ,3(3) J \'0('1) --

where ~ must be "properly" ch-;'sen, and by d1. demonstrates the measure with respect

to which p.(y) are defined. Considering a parametric situation, where 1 _

probability densities are specified by a set of parameters b in the form

, The quantity defined by (303) will be denoted !(~,~,~) with p (1.) = p(1.I~)

, and po(I,) p(.ll /2.). To find the most "proper" (suitable) form of the

A

function, Akaike analysed the sensitivity of I(~,~~) to the derivation from A

/'J to b.

It is assumed that either p(.ll ~) and~(r) are the regular functions.

l?,04 )

('~os)

-110-

and assuming validity differentiation unde~ integral sign as well as that

• which implies that

In the same way it can be shown that

t '!.08)

which gives

Comparing (309) with (298) we see that integral

.L 1c:ly .. t{ l'o) ll!.'1\ - d~ll.'M-f_l~ LA --

D" (h - -is the <l,m)th element of Fisher's information matrix.

-111-

Summarizing we get:

" Relation (311) shows that~(I) must be different from zero if ll~ I~.~)

ought to be sensitive to small variations of b. Also the relative sensitivity

of I(b,~,(f» is high when l4?l<l I is large. -The possible forms ofq)(r) are e.g. - q)l~)

loge r, (r-I) and r

To restrict further the form of ~(r), the increase of information is considered,

by increase of N independent observations of Y.

For this caSe

l'!>it)

thus

-112-

" From (313) it is seen that IN(~'~'¢) is not very much influenced by the

increase of information. It can be also seen from evaluation of (313) from

(312) that the only quantity concerned with a final result is

t.J

d n ptl.Jli)l~) 3bL

This last relation shows very clearly that taking into account the In from

the density function p(y(i) ~), (314) will be fulfilled in a natural way:

and thus this observation suggests the choice of~(r) = lnr for the definition

of the amount of information, - this simply leads to the Kullback-Leibler

definition of information.

Remark: Any other definition ofcp(r) will be useful only ifq? is not vanishing.

For the purpose of the estimation Akaike proposes the following loss

and the risk

tlQ.\ ~) " E Z Wl~~)~ functions, which are based on the Kullback-Leibler formula. Akaike pestulates

that when N independent realizations y(i) (i = 1,2, ••.• N) of! are available,

(-2) times the sample mean of the ~n-likelihood, will be a consistent estimate /'

ofW(b,[':,), -- \-l I'

(-L) L. 9.M. ( pll.Jl'j

• Q» ) c=< \ ~l~U),~)

-113-

" thus, at least for large values N, the value~ which will give the maximum of

I-l " LiM (e l '1Li

" f:) ) i>' p('1L\I,~)

'" will nearly minimize W(~,~).

The whole idea of Akaike is to, instead of considering a single estimate of ~,

to consider estimates corresponding to various possible restrictions of the

" distribution. This whole idea can be simply realized by comparing R(~.~ or t\ .-

W(~,~) if possible, for various ~ 's and taking the one with the minimum of

" ..-R(~,~) or W(~,~) as a final choice.

This approach may be viewed as a natural extension of the classical maximum

ZikeZihood principZe. The only problem in applying this extended principle in

" '" practical situations is how to get a reliable estimate of R(~,~) and W(~,~)

'" " Akaike gives a procedure which enables finding estimates of !(~.~) and W(~~)

which are used later for estimation of the order of the model and the vector ,...

of parameters ~.

Assuming bk

(k = 0,1,2, ••.. L) - components of the vector~. Akaike is looking A

for 0>k (k = 0, I ,2, ...•. L) being maximum likelihood estimates of bk

. Considering

the situation where the results y(i) (i = 1,2, ..•.. N) are obtained as

independent N observations, ~k will be the value of bk which gives the maximum \J

of the likelihood function fl p(y(i)lbk).

i=1

Thus Akaike suggests that

( ~2.0)

-114-

A "

as an estimate of W~, f!> k). In Akaike' s paper (1972) it is precisely proven

that the

is the good estimate of E tW(~ ,~,,))(where ~ is the estimate of f':J k according to

Euclidean norm), at least in cases where N is sufficiently large and Land k

are relatively large integers. Since we are only concerned with finding out

'" the f':> k which will give the minimum of r(!!>l (\>k)' we have only to compute:

( '!.2.z.)

('l>B)

Relation (322) is the essence of the Akaike's method.

3.5.3. Concluding remarks

As it was proven by Soderstrom (1977) Akaike's FPC criterion and Akaike's AIC

criterion are asymptotically equivalent. However, for the MIMO systems the

evidence of equivalence of FPC(M) and AIC(M) is not proven. From a practical

point of view it would seem that FPC(M) should be singular and less time

consuming. Soderstrom (1977) once again points out that it is not the case

and in view of his work FPE(M) and AIC(M) are equivalent for sufficiently large

observation samples.

-115-

3.6. Structural identification based on the Hankel model

3.6.1. Behaviour of the error function with respect to the number of Markov-parameters in the Hankel model

The error function for the Hankel model of the multivariable dynamical system

is defined as:

(324)

where W is the gain scaling matrix which is positive definite. Quantities

Y and Yare calculated according to the Hankel model (see chapter I).

It can be easily shown that the error function V (s), where "s" is the number -w

• AT T of different Markov-parameters encountered ln Y = N S , rapidly decreases

- """1Il

in value when the model reaches the proper order i.e.

Entier(~ + I)

where r is the estimate of the order r. Then "s" is also the number of

(325)

different Markov-parameters contained in the Hr matrix. Behaviour of the error

function is not always clear. Sometimes it is difficult, at first sight, to

decide for an optimal s. Then the difference ratio

K I Vw(i+l) - Vw(i + 2)1

/Vw(i) - Vw(i + I) 1

gives the clearer picture of changes.

for i = (s-2),(s-I), s, ....

(326)

Difficulties will occur in cases with a relatively high noise power and in the

case when eigenvalues of the state matrix of the identified system are close

to each other. Another possibility, in cases where the order of the system is

smaller than 3~ a direct decrease rate of the error function can be used.

D I1V (k)

w V (k) - V (k + I) w w

(327)

11k

-116 -

This order test may be compared with the error function order test for 5I50

sytems as presented by A. J. W. van den Boom. A.W.M. van den Enden (1973).

and more details can be found in A. Hajdasifiski (1979).

3.6.2. Behaviour of the determinant of the Hr ~rT matrix

This order test is based on the property. that the rank of the Hankel matrix

Hr is equal to n if

n r"}. ____ _

min (P.q)

(which means that when contructing the sequence of matrices {~r} for r = 1.2.3 •••.

and checking rank {~r} for r = 1.2 ••••.•• the proper r is already found while

for

n r} ____ _ + rank {~r} = n). (328)

min (P.q)

For purely deterministic systems having finite realization. we may find this way

a real nand r. Increasing the Hankel matrix dimension as long as singularity

is not detected. we find the rank n. the r can be deduced from structural

properties of the ~r.

In cases where q 1 p, it is necessary either to check all minors in subsequent

Hankel matrices. or (which was found to be easier) to check the singularity of

the HrHrT

matrix.

Talking about the rank of the matrix, we understand this as the "numerical

rank" defined as follows:

Definition 19 Let N(n) be a set of ordinal numbers {1.2.3 ••.•• } and R(O) a

set of reals. The (k x k ) matrix Hk of the rank min(k x k ). has a "numerical q p p q

rank" (s.o.n) with respect to the spectral norm if: 2

-n = inf {rank~: II~ - ~II 2'i d. rank ~ IS N(n)

~ = sup { 0 :U~ - ~ II 2 ~ 0 :::;. rank ~ ~ n} cSeR(cS) (329)

where the

-117-

spectral norm of the Hk matrix

/I~1J2 = 1I~~ae

sup ~ x n e

x - any (k x I) vector 6Rkp p

II' lie - Euclidean norm

is:

The method again incorporates observation of changes in the behaviour of

T det {~r} or {~r~r}. The rapid decrease in the value of the det{~r} or

det {~r~rT} indicates a proper r. Then for the chosen r, a test for the

dimensionality of the state space must be performed.

(330)

Usually this order test is a very convincing one, and when applied together with

the test for the error function behaviour, it appears to be a very efficient one.

In cases where this order test would be confusing, it is again desireable to

look for the ratio test:

dedH.} det{~~T} K

-J, K (331 ) = or =

dedH. I}I -1+ I ded~+I~+IT}1 For more details see: Haj das iiiski (1980) , Isidori (1972).

3.6.3 Singular value decomposition of the Hankel matrix

The singular value decomposition of the Hankel matrix is a very efficient order

test which can be combined with the new realization algorithm (A. Hajdasiiiski,

A.A.H. Damen (1979». The whole procedure is based on the following:

Having exact Markov-parameters, it is always possible to find k > r - the system

order for which (see relation (150)

D - is the n x n diagonal matrix. (332)

are

-118-

D = diag. (0 ,0 , ..... 0) 1 2 n

o. for i 1

1,2, •••• n are called singular values

U - is the (k~ x n) matrix consisting of n eigenvectors of the

~~T Le. !!J!kT

= I --n

v - is the (kp x n) matrix consisting of n eigenvectors of the

T . T, V D2VT

!4c~, 1.e. ~ ~ =

vTv .. I . a. ={>\ , where A. for i = I ,2, . .. n -n' 1 1

eigenvalues of ~T!4c or !!J!kT. In the ideal case, the singular value

decomposition performed on the ,!!k, delivers the Itn" - the system dimension equal

to the number of nonzero singular values.

By a simple deduction the r can be found from structural properties of the Hankel

matrix.

In the noisy cases an easy test can be performed to discriminate between which

singular values are substantial and which can be neglected by comparing their

rate of decrease. The decision concerning which singular values can be

neglected will depend on the accuracy we will impose on our model.

Two types of approach can be proposed:

I. The relative error of the least squares fit on the Hankel matrix.

Defining the Euclidean norm of the matrix as:

2

\I !4c 1\ e = (333)

we can prove the following theorem: (Kam J.J. van der, Damen A.A.H. (1978),

Golub G.H., Reinsch C. (1970».

Theorem 10:

-119-

Given a s.v.d. for a (k x k ) matrix Hk: q p

D = diag. (0,,02 ••••• 0 min (k x k ) q p

the (kq x kp) matrix ~ of the rank ~ ~ min (kq

x kp

) and such that

1\ ~ - ~ \I e' is given the following:

u

where

U - contains the first q columns of U -~ ;J

v - contains the first D columns of V -g ;J

D - diag. (01.02 •••••• 0 .. \ -3 ~

(334)

Remark: Thus setting the smallest (min (k .k ) - s) singular values to zero q p

in the s.v.d •• through the relation (17). we obtain the best. in the least

squares sense. approximation of the ~ matrix. being of a smaller rank than

~.

To decide which singular values can be neglected the absolute error criterion

min(k .k )

z= q P2

O. J

o < £ « 1

or the relative error criterionr' __ ~~ __ ~~ __ -' min(k .k ) 1

2::: QP2

... 1 O. 1=~+ 1

min(k .k ) ~ q p

L-:. 0.> i=I L

~t o < 8 < 1

(335)

(336)

-120-

may be applied.

II. The numerical rank approach

Let us start with the following Lemma.

Lemma I: Let 01) O2 ~ ., ~ 0min(k ,k ) be the nonzero singular value of the q p

matrix~, then ~ has a numerical rank (E,o,n)2 iff:

(337)

The definition of the numerical rank and relations (329),(330) together with

Lemma 1 provide another criterion of the absolute error of the approximate

realization based on~. Assuming the upper bound - ° and lower bound - E of

the absolute error, through relations (337) and (334) ,we obtain the least

square optimal approximation of the ~ matrix - ~, where

rank ~ = numerical rank {~} = (E,o,n)2 (338)

Assuming that 2k Markov-parameters are estimated, it is possible to construct

the following Hankel and shifted Hankel matrices - ~ and O~. Performing

the s.v.d. there also is found a vector of singular values

where s = min(k ,k). The relative error of the least squares fit or the q p

numerical rank absolute error (337) decides which singular values may be

neglected~ In such a way the estimate fi of the system dimension is determined.

Example 7. Let us compa~e the last three order tests for the system having the

following properties:

0.2 0.0 0.0 0.0 -1.0

.!(k+l) = .!(k) T ~(k) 0.0 0.4 0.0 1.0 1.0

0.0. 0.0 0.8 1.0 -1.0

Z{k) = [ 1.0

0.0 -1.0J ~(k)

0.0 1.0 0.0

, , , , , , , , , , , , , , , , , ,

-121-

for this dynamical system r = 2, n = 3. The output of the dynamical system

is affected by the noise which is generated from the white gaussian noise by

the filter:

!.F, (k + I) = I o. I

0.0

E.(k) /

1 .0

0.0

0.0

!.F,(k) 0.7

0.0/ !.F,(k)

1.0

+

/

1.0

0.0

0.0 I I .0

• F,(k) -

Thus the measured output Zm(k) = Z(k) + E.(k).

The intensity of the simulated noise - n(k) is 10% of the output signal

amplitude - Z(k).

Ideal and estimated Markov-parameters are the following:

M ~1 ~? ~1 ~" ~5 -0

Ideal Markov -1.0 0.0 0.8 0.6 1-0.64 0.6 -0.512 O. -0.41 0.4( 1-0.33 0.32

parameters 1.0 1.0 0.4 0.4 0.16 O. I 0.064 0.01 0.026 0.02 0.01 0.01

Estimated 1-0.99 0.0 0.79 0.59 1-0.61 0.5 -0.47 0.4 1-0.35 0.38 0.25 0.28

Markov param. 1.0 1.0 .403 0.4 p.162 0.1' ~.07 0.07 p.02S 0.03 .015 0.01

and values bf the V as the function of Markov param. number - s, are: w

" , , s , "T - I , ,

V =tr(Y-Y) (Y-Y) , , , , W ,

2 , 3 ,

4 5 , 6 , , , , , , , , , , ,

I , I , , , , , , , Estim. Mark. par.:0.50606 :0.26831 :0.15562 :0.09177 :0.04948 :0.00326 , , , ,

I I , , , , , , , , Ideal Mark. par. :0.3855 :0.2011 : 0.0602 :0.0422 :0.0271 :0.0000

, , , , , , , , , , , , , , ,

-122-

O,S ------

1

)t---- -)I ideal M, par. ------)(

1\ O,?>

-----~--\--o C> estim. M. par.

0,2. 1 \1 ----------~ -----1-----t'~ ---------~----+--~-~-- -------r=----...j---- ~*- : ---== -= =- =. - =-= -=-- F ==- -=----~ -=-~- ~ #-==-~ -=:.

0,1

s

At first sight there seems to be some confusion as to which s should be chosen.

One may choose s = 2 or s = 3. For example for:

r Entier s 1 ) s = 2; = (- + 2 2

s = 3; r = Entier (~ + 1) = 2 2

for r = 2, n = 3 is sufficient to provide a good approximation of the original

system.

Performing the determinant test on the H matrix we get: -r

4.0 O,~

I I I I I I I I I I I

o,oOsst ____ L ______ _ QOOI __ ---1 __ _

1 2. !>

-123-

we have exactly the same answer: r = 2, fi = 3.

For the s.v.d. order test we choose the ~. matrix and we get the following set

of singular values:

al = 2.7707; a2 =

-3 8.1052.10 ;

-I -2 1.7185; a,· 3.5826.10 ; a. = 3.709.10 ;

a6 = 3.8196.10-3; a7 = 2.8425.10-3 ; a. = 1.0788.10-3 •

It is seen that:

a, > 0.1 > 0.05 > a.

thus the choice fi = 3 assures the numerical rank of the ~. matrix equal to:

(E,o,fi)2 = (0.05; 0.1; 3)2

and provides the relative error with respect to the Euclidean norm

tt~.-~Ile

U ~. II e

= 0.03829

The system order r > fi min(p,q)., thus it is the smallest integer fulfilling

r> 1.5. And again, we have as the result of the order test r = 2, which is

sufficient to provide fi = 3.

It is demonstrated then, that all three order tests provide us with the same

result r = 2 and fi = 3.

Conclusions

There exists already a quite extended class of order tests. Certainly this

choice of order tests mentioned here is one of many possible choices. However,

it shows quite well the mainstream in the methodology of model building.

The Guidorzi's order test has been applied for many practical problems

providing quite satisfactory results.

-124-

However, in many cases it was possible to find lower order approximations of

investigated systems. Such a comparison between the Guidorzi ' s method,

Ho-Kalman's algorithm and the partial minimal realization by Tether (Tether A.

(1970)) was made by A. Krause (1976), showing that it was always possible,

for noisy dat~, to find a lower order model for Tether's and Ho-Kalman's methods,

whereas for Guidorzi's methods a higher order model was found.

Tse-Weinert's methods, being a special case of the determinant test for the

Hankel matrix, gives very good results. Examples can be found in an original

publication by Tse and Weinert, (1975).

Furuta's approach is rather unnecessarilly complicated, but leading to quite

good results for low dimensional model.

Akaike's methods have already been discussed. They are very interesting from

a theoretical point of view. However, in practical applications demand a large

amount of data.

Order tests proposed for the Hankel model are rather simple and efficient.

Especially good results are with use of the s.v.d. method. This is, however,

the beginning of research in this field and many unexpected events may occur

during application of these methods in practice.

4. Multivariable system identification

There has already been much said about identification and parameter estimation.

However, the general definition of the identification has not yet been given.

According to Zadeh (1962), identification is the determination, on the basis of

input and output, of a system within a specified class of systems, to which the

system under test is equivalent.

-125-

This short definition covers a huge range of knowledge about the modelling and

model building basing an input/output data. A good survey of problems

concerning modelling and identification of single input - single output systems

has been written by Fasal K.H.,and Jorgel H.P. as the theoretical lecture for

the 5th IFAC Symposium on Identification and Parameter Estimation - Darmstadt

1979. For there is no substantial difference in general classification of

MlMO and SISO systems it is fair to quote this tutorial lecture as a good

reference.

Of our particular interest, there is an expePimentaZ anaZysis of a process,

which in general is considered just as the "identification". If the structure

of the model is known in advance, or at least can be estimated (see the previous

chapter), parametPic methods can be used for identification of the mathematical

model. In cases where the structure cannot he estimated non-parametric

techniques have to be applied.

Among the various goals of identification the below mentioned are most

frequently quoted:

deeper insight into the nature of the process

prediction of the dynamical behaviour of the process

verification of theoretical models

computation (or better estimation) of non-measurable variables

design of the controller

optimization of the-process.

Most of all, our interest will be attached to discrete time modeZs which will

be split into stochastic modeZs and detenninistic modeZs. However, the

deterministic models, as purely conceptual ones, usually serve as the prelim-

inary information about the process.

A survey of methods for identification of multivariable systems has recently /

been written by Niederlinski A. and Hajdasifiski A. (1979). This paper discusses

-126-

multi variable systems structures, parametric and non-parametric identification

techniques for mu1tivariab1e systems.

In this report we will present only the outline of a few of the most important

techniques, which are related to already discussed order tests. Also more

attention will be given to the Markov-parameter approach.

It should also be noted that the parametric identification of canonical models

by Guidorzi and Tse-Weinert has already been presented in chapters 3.1 and 3.2,

(however briefly), the Akaike's information criterion approach has also been

presented in quite SOme detail. Thus the following algorithms, as in the

illustrative examples, will be discussed further;

Tether's minimal partial realization algorithm

Gerth's algorithm

The Approximate Gauss-Markov Scheme with the s.v.d. Minimal Realization

algorithm

4.1. The Tether's minimal partial realization algorithm

The problem of minimal partial realization is formulated as follows:

1. Given an infinite sequence of Markov-parameters {M.},.which does not 1

have a finite dimensional realization, it is desired to find such a finite

dimensional realization so that its first No Markov-parameters are

correspondingly equal to the first No Markov parameters of the first

sequence, or

2. Given a finite sequence of Markov-parameters M , M, ..•. M __ I' find a -0 - ~o-

finite - dimensional realization whose first No Markov-parameters are

correspondingly equal to the Markov-parameters of the given finite sequence.

-127-

The second alternative is much more suited to the purpose of identification

and gives a powerful tool for handling the noisy case. It does not, however,

incorporate any stochastic considerations, assuming that only some extraction

of the Markov-parameters from the noise corrupted systems is prior to the

minimal partial realization procedure. The following two definitions are

useful:

Definition 20: {~,!,~} is said to be a partial realization of order{Nolof the

sequence {M.}, if an only if M. = C AiS holds for i = 0,1,2, -1 -l. - --

N - 1. o

Definition 21: {~,!,~} is said to be a partial minimal realization of order

{No)if and only if the dimension of A is minimal among all other {~;!;~I}

satisfying definition 20.

Tether (1970) proved that for every finite series of Markov-parameters {M.}, -1

i=O,J,2, .•.. No 1 there exists an extension sequence {~o+l""'}' for

which a completely controllable and observable partial realization exists via

the Ho-Kalman's algorithm.

It can also be demonstrated that among all possible partial realizations must

exist at least one minimal partial realization which is unique if and only if

the extent ion sequence is unique. Defining

M !:!.1 ........ !:!.r,' -~l N

-0

I (339)

Mi ~

~2.' ••••••• ~,

!:!.r,-I .•...•.•.... !:!.r,' + N - 2

-128-

it is possible to prove that given a finite sequence of r x m matrices

{M , M1 , •••••• M. I} satisfying ~ - ~o-

rank ~l ,N rank ~'+I ,N rank ~',N+I

for some N,N' such that N'+ N = N the extension of the sequence o

(340)

{M , M' ....... M.. I} to {M ••••• M.. I' •••• M k I} with 0 ... k < '" for which --0 - ~o- -0 ~o- ~o+ -

rank !!(m' ,m) = rank ~l ,N

where ro 1+ m = N + k, is unique. o

This result proven by Tether (1970) leads to the following reaZizabiZity

criterion:

(341 )

Let {~, ~l, •••.•. ~O-I} be an arbitrary finite sequence of q x p real matrices.

let H. . for i + j ... N be a corresponding Hankel matrix. Then a minimal partial 1; J 0

realization {A, !, ~} given by the Ho-Kalman algorithm is unique and realizes

the sequence up to and including the Noth term if and only if there exists

positive integers Nl and N such that:

I. N' + N = N o

2. rank ~lN rank ~'+I ,N rank ~l ,N+I

(342)

(343)

If the realizability criterion is satisfied then the Ho-Kalman algorithm can be

applied with

n = rank ~l ,N

using for the Ho-Kalman algorithm ~'N and a~' N' , The resulting minimal partial realization is unique because the extension

(344)

k sequence ~k = ~!! for k No,No + 1, generated by the realization satisfies

rank .!!--l + :...~ j, N+i = rank~, ,N (345)

for all i,j ?O

-129-

If 1 and 2 are not satisfied, then a minimal partial realization - if it exists -

may not be unique, for in order to use the Ho-Kalman algorithm new matrices

{~o' .•...• ~o-I} must be found until

rank ~1+I,M = rank ~I,M+1 = rank ~l M , (346)

where Ml + M = Po These matrices can be partially or completely arbitrary.

since (~,!,~) are functions of the sequence {~o .••• ~O-I} they may also not

be unique.

One of the most important results of Tether (1970) is the determination of a

lower bound for the minimal partial realization dimension. This is formulated

as the following Lemma.

Lemma 2 Let (~,!,~) be a partial realization where first No Markov-parameters

are equal to the given sequence {M , Ml , ••••• M I}' Then the dimension of --0 - ~o-

the minimal partial realization satisfies the following inequality:

No n ~ n(No) = L

j=1

rank H. N 1 . J' 0+ -J

No

2: j=1

rank H. N • -.J, O-J

This lower bound can be achieved for a "suitably chosen extension!!. This

(347)

"suitable choice lt is the subject of the fundamental theorem of minimal partial

realization given by Tether (1970). In order to state this theorem, it is

necessary to formulate one more definition and one more lemma.

Definition 22: Let NI(No) equal the first integer such that every row of the

block row [~l (No) ...... M ] is linearly dependent on the rows of the 410-1

Hankel matrix ~l(No), No-NI(No)' let N(No) equal the first integer such that

every column of the block column [~(NO) .....• ~0_11T is linearly dependent

on the columns of the Hankel matrix ~O-N(No),N(NO)'

Lemma 3 Let n(No), NI(No) and N(No) be integers defined previously. Then

any extension {~o'~o+1 ....• } of {~o, ~l ••••• ~o-I} whose realization

-130-

achieves the minimal lower bound n(No) for its dimension also satisfies

rank ~(No),N(No) = rank ~l(No)+I,N(No) =

= rank ~l(No),N(No)+1 (348)

for that extension.

From this Lemma follows the Minimal Partial Realization Theorem, which is central

in Tether's algorithm:

Theorem II Let {~, ~1 •••••• ~0~1} be a fixed partial sequence of q x p

matrices with real coefficients and let n(No),N(No) and N1 (No) be integers

defined previously. Then:

I. n(No) is the dimension of the minimal partial realization

2. N(No) and N1 (No) are the smallest integers such that (348) holds

simultaneously for all minimal extensions

3. There is a minimal extension of order P(No) = N(No) + N1 (No) for which

n(No) is the dimension of the realization computed by the Ho-Kalman

algorithm, but which in general is not unique.

4. Every extension which is fixed up to P(No) is uniquely determined thereafter.

It must be stressed that Tether's algorithm does not explicitly recognize the

stochastic nature of experimentally determined Markov-parameters. Nevertheless

it proves useful for real-life noisy and incomplete data because it can

generate an approximate model which agrees with the data. A number of successful

applications of this algorithm have been described (see Rossen-Lapidus (1972),

Hajdu (1969), Roman (1975)).

4.2. Gerth's algorithm

Tether's algorithm was assuming the availability of Markov-parameters. However,

they must be estimated. Therefore, an asymptotically unbiased and efficient

estimation of Markov-parameters is an essential step for the successful

identification of multivariable systems via the realization theory.

-131-

Gerth (1971,1972) proposed a multistage procedure for the initial estimation

of Markov-parameters and further refinement of the finite series {M.} in a way -:L

that assures the linear dependence of the extension series. It works on a

noise-corrupted data set of input-output pairs, assumes a ppeseZeated degree r

of the minimal polynominal and applies the Ho-Kalman algorithm to determine

a realization.

Gerth's algorithm consists of two separate steps:

I. Estimation of the finite series of Markov-parameters {N.} for i c O.I.Z •••• k -1

2. Determination of the minimal polynominal coefficients ai and matrices M. --:L

for i = 0.1 ••••• r - I. which are as "close as possible" to N .• for the -1

)

same i and specify an infinite realizable sequence via the relation

r

Z i.:~

"'i'M' • <\;1 _ )-t. (349)

with ~) = -a . being the coefficients of the minimal polynominal with a = I. ~1 r

The Gerth's algorithm assumes r - the order equal n - the dimension of the

realization. Assuming further that:

1. initial conditions are zero

2. ncises corrupting the signals are white and zero mean

3. number of samples is sufficiently large

and using the Hankel model and the least-squares method. the algorithm results

in the estimate of (k + I) Markov-parameters:

N (350)

-132-

where NT ~ l~' ~" ...... ~1 and matrices; and Yare defined by

yT = [~(~), ~(~ + I), .•••• ~(~ + m)l

i(i) =[~ (i), ~ (i) ••.•.. y (id " q

S ""1Il

~(£ - I) u(£ + m - I)

~(~ - k - I) ••• u(£ + m - k - I)

~T(i) = [u, (i), u2 (i) .••.• up(i)]

(351 )

(352)

Using an input excitation {u.} = {I,O,O •••.• } for each of the p x q partial >

systems and using the Hankel model (8), the excitation matrices for these partial

systems will be

T '0 -UJ"

w .. J> (353)

o

for j = 1,2, ••••• q, £ = J, ••••• p, where Wj~ is the individual weighting factor

for every partial system.

For every partial system the following Hankel matrix is constructed from the

elements of {~-i} for i = 0, I, ..... k.

N '0 -o,jJV

. ~r-I,j£

N '0 ...... N '0 -jlJ" k-r oJ "

(354)

-133-

where N .. , is the jR,-th element of N .. The vector containing elements of 1,JN -~

{N.} for i = r, r + I, '" k is defined as: -1

N ." N I ., ....... Nk ., r,]J<. r+ ,]N ,]JV (355)

The estimation of the minimal polynominal coefficients is also performed using

the least-squares method and the following loss function.

obtained from (349), with the coefficients vector

and

h T --r

H

-a , I

vT = [v T ,V T ....... ~pT]

-11 -12 ."

T -u diag[T TTl ""'-'11 ' -U11.'····· -uqp

The minimization of a gives:

(356)

(357)

(358)

(359)

(360)

(361)

It can be proven that the least-squares estimation of the minimal polynominal

coefficients for a given degree r is unique if and only if rank ~ = r. Next

the finite sequence {M.}, for i = 0,1, .•••. r-I, must be determined using again -1

least-squares estimation for computing.

min m. , -J"

k

2: i=o

[ y(i).,(h ,M) - Y(i).,(N)12

J" --r - J" - (362)

-134-

for j = 1,2, •••• p, R. = 1,2, •••• q, where y(i).,(h M) is the output sequ~nce J'" -r-

generated for the jR.-th partial system by T ., for the already determined -UJ'"

h but unknown {M.}, and y(i).,(N) - is the output sequence generated for the -r -J.. Jx. -

same partial system and the same excitation, but for known {N.}. The solution -1.

to (362) is given by:

where

with

(G T T T. GT)-I G T T T , ., n., - -U-UJ'- - - -UJ "'-u -J '"

nT

., =[N ." N ." ..... Nk .,] - J'- o,J'" I oJ'- oJ'"

G

E ~ 0 I - I

== ---I r-I I

I -r-I I

I

k-r+1 1 •••••• R e - -r

~l ~T =[ 0 .... 0 .... 1. ... 0]

r-th position

(363)

(364)

(365)

(366)

(367)

(368)

If all T ., are nonsingular, the sequence {M.}, i = O,I ••••• r-l, with a given -UJ'" -,

vector h is always unique and can be used to determine the minimal realization -r

using the Ho-Kalman's algorithm.

Extensions of Gerth's idea can be found in papers by Hajdasifiski (1976,1978).

4.3. The approximate Gauss-Markov scheme with the singular value decomposition minim.al =ealization algorithm

Using the Hankel model (8) and relations (351) (352), defining the noise vector

ET = [.~(O, ~(2.+ I)

~T(i) = [ e1 (i) e

2 (i)

(369)

(370)

-135-

and

(371 )

the block vector containing estimates of the first k + 1 Markov-parameters

(372)

the block vector containing the remaining Markov-parameters and the matrix

of initial conditions ~ , the Hankel

~( -k-2) ............. ~( -k+m-2)

~( -k-3) ............. u( -k+m-3) ... ~(-I )

o

o •....•............• u,l-iJ

model, for the noisy system will be following:

Y

Assuming that input samples {u.} and the noise samples {e.} are mutually -1. -1

uncorrelated stationary processes and that E{~(i)} = 0, an estimate of the

(373)

(374)

first k + 1 Markov-parameters can be found minimizing the error function Vw

(see Hajdasinski (1976)(1978)(1979), Niederlinski-Hajdasinski (1978)).

where

Y S T N -1tl -

(375)

(376)

is the estimate of the Y. Thus applying the well known formalism, an expression

for a minimal trace of Vw is found:

= 0;

N (377)

-136-

Expressing N in terms of E

N (378)

it is seen that the term (5 W 5 T)-I 5 WE is the bias of the estimate (377). -m---m -m--

This expression depends on properties of the noise E and asymptotically vanishes

when E{e(i)} o and there is no correlation between samples of E and 5 . - """1ll

Also for the zero initial conditions the second term in the expression (377)

vanishes. In case E{!} f 0, for this type of model i.e. (374), it is very

easy to estimate the bias of the estimate - see Hajdasifiski (1978).

Considering the following expression:

as an accuracy criterion, it follows that:

E{(~ - ~)(~ -

Substituting E{E ET} = -I Rand W = R we get:

see also Hajdasifi.ski (1976 ,1978,1979)

-I Thus choosing ~ =! , this results in an efficient estimate of the Markov-

parameters, since it is easy to show that:

E{(M - N)(M - N)T} - - - - R

(379)

(380)

(381 )

(382)

This way of weighting resembles very much the classical Gauss-Markov estimation

for 5ISO systems.

Defining

T n

-137-

n. T = [e.(t), e.(9.+I), ...... e.(9.+m)]

-1 1 1 1

the noise covariance matrix will take the form:

T = E{E, E, }

It is seen that the trace of g is the following sum of squares:

m+1 trg 2:

i= I j = I

(383)

(384)

(385)

(386)

Writing the R matrix in an explicit form it can be seen that the trace of g

is equal to the trace of R. (Hajdasifiski (1978».

trg trR (387)

Treating the! matrix as the covariance matrix of a hypothetical sinie-input

single-output noise filter it is possible to reconstruct this noise and its

covariance matrix. This "composite" noise model is but a mathematical fiction,

having no strictly physical interpretation. According to (386) and (387),

minimization of the (379), which appears in the minimal tr!, results in the

minimal trace of the noise covariance matrix, attaining in this way the main

goal of an efficient estimation.

If all noises corrupting a multi variable system are stationary, the R matrix

has also a very simple structure:

R =

S 2

s 2 1

S 2 m

S 2 S 2 •••• 1 2

S 2 •••••••••

s2 m-I

S 2 S 2 m-I m

S 2 rn-2

S 2 m-I

(388)

-138-

In practice the R matrix must be estimated using a finite number of samples,

taking as 52 and 52 m

m + 1 q S2 = L ~

-2(.) (389) e i J m +

j i = 1

m-k+1 q

52 = 2: L. e. (j) e. (j+k) (390) 1 1

n m-k+1 j = 1 i = 1

where e.(.) are standing for estimates of e.(.). 1 1

Thus only the first part of the ~ matrix, being the estimate of !, can be

computed with a sufficient accuracy, for in an explicit method there are only

m + 1 input-output and residual error samples available. Higher ordered elements

in R will be increasingly less accurate. As it is demonstrated by Hajdasifiski

(1978), the quality of the ~ matrix estimation plays a key role in the convergence

of Markov-parameters estimates. The realization theory once again helps to get

improved results, providing meaDS to reconstruct the composite noise covariance

matrix. For a detailed derivation of this result the reader is referred to

Hajdasifiski (1978).

Treating E as the set of specially arranged samples of a one-dimensional noise

with the covariance matrix!, it can be assumed that this noise is generated

by a colouring filter from a hypothetical white noise. In such a caSe an

estimate R of R is given by the following expression:

-139-

R ~ee(o)

~ ee(l)

'I' ee (I ) . • • •• ~ ee (m)

~ ee(m-I)

~ ~ ee(m).............. ee(o)

where E is the estimate of the multivariable noise, calculated using the

residual error of the L.S. estimation of Markov-parameters.

(391 )

Assuming that the noise e(k) is generated from s(k) (white noise) by a moving-

average filter,

e(k) -I

+ C(z ) s(k) = (C a

(392)

a very interesting decomposition of the R matrix can be found. From (391) it

follows

with

Since

for

e = £~

~ = ~ (k) ~ (k + I) ....• ~ (k + T [T T T

[ T T T

~ = ~ (k-v) ~ (k-v+l) ..••• ~ (k

C v-I C v-2 C 0 ...... 0 0

C = 0 C v-I C C ...... 0

1 0

o ..........•..... C I ••••••••• C v- 0

E{ii T} C cTa2

E{~~T} = a2 1

(393)

(394)

(395)

(396)

(397)

(398)

Then with accuracy to a constant factor the composite noise covariance matrix

is

R T

aC C (399)

-140-

However, for the Gauss-Markov estimation it is only necessary that the

weighting matrix W is similar to the covariance matrix R.

w (400)

Knowing a finite number of initial elements in the R matrix and assuming

they are sufficiently exact, it will be possible to find, via the realization

theory, the remaining elements and reconstruct the full rank covariance matrix

R.

A comparison of three methods for Markov-parameters estimation is presented by

Hajdasifiski (1978). Sequential algorithms are also available at present. A

general background can be found in Hajdasifiski (1976). Another approach to

the identification with Markov-parameters is presented by Rissanen,Kailath

(1972h Anderson, Brasch, Lopresti (1975).

Having identified Markov-parameters {M.}, the minimal realization algorithm ~

with the s.v.d. of the Hankel matrix was proposed by Hajdasifiski and Damen (1979).

Application of the last is straighforward using results derived in the chapter

3.4.2. - (150) - (158). For the numerical example see Example 6.

- 141 -

References

Aasnaes, H.B. and T. Kailath (1973) AN INNOVATION APPROACH TO LEAST-SQUARES ESTIMATION - Part VII: Some applications of vector autoregressive-moving average models. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 601-607.

Ackermann, J.E. and R.S. Bucy (1971) CANONICAL MINIMAL REALIZATION OF A MATRIX OF IMPULSE RESPONSE SEQUENCES. Inf. & Control, Vol. 19(1971), p. 224-231.

Ackermann, J.E. (1972) ON PARTIAL REALIZATIONS. IEEE Trans. Autom. Control, Vol. AC-17(1972), p. 381.

Akaike, H. (1970) STATISTICAL PREDICTOR IDENTIFICATION. Ann. Inst. Statist. Math. (Tokyo), Vol. 22(1970), p. 203-217.

Akaike, H. (1971) AUTOREGRESSIVE MODEL FITTING FOR CONTROL. Ann. Inst. Statist. Math. (Tokyo), Vol. 23(1971), p. 163-180.

Akaike, H. (1972) USE OF AN INFORMATION THEORETIC QUANTITY FOR STATISTICAL MODEL IDENTIFICATION. In: Proc. 5th Hawaii Int. Conf. on System Sciences, Honolulu, 11-13 Jan. 1972. Ed. by A. Lew. North Hollywood, Calif.: Western Periodicals, 1972. P. 249-250-.­

Akaike, H. (1973) INFORMATION THEORY AND AN EXTENSION OF THE MAXIMUM LIKELIHOOD PRINCIPLE. In: 2nd Int. Symp. on Information Theory, Tsahkadsor, Armenia, USSR, 2-8 Sept. 1971. Ed. by B.N. Petro v and F. Csaki. Budapest: Akad8miai Kiad6, 1973. P. 267-281.

Akaike, H. (1974a) STOCHASTIC THEORY OF MINIMAL REALIZATION. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 667-674.

Akaike, H. (1974b) A NEW LOOK AT THE STATISTICAL MODEL IDENTIFICATION. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 716-723.

Akaike, H. (1975) MARKOVIAN REPRESENTATIONS OF STOCHASTIC PROCESSES BY CANONICAL VARIABLES. SIAM J. Control, Vol. 13(1975), p. 162-173.

Anderson, B.D.O. (1977) AN APPROACH TO MULTIVARIABLE SYSTEM IDENTIFICATION. Automatica, Vol. 13(1977), p. 401-408.

Anderson, 5.H., F.M. Brasch, Jr. and P.V. Lopresti (1975) THE SEQUENTIAL CONSTRUCTION OF MINIMAL PARTIAL REALIZATIONS FROM FINITE INPUT-OUTPUT DATA. SIAM J. Control, Vol. 13(1975), p. 552-571.

Anderson, T.W. (1971) TH~ STATISTICAL ANALYSIS OF TIME SERIES. New York: Wiley, 1971. Wiley Series in Probability and Mathematical Statistics

- 142 -

Andree, R.V. (1951) COMPUTATION OF THE INVERSE OF A MATRIX. Amer. Math. Monthly, Vol. 58(1951), p. 87-92.

Audley, D.R. and W.J. Rugh (1973a) ON THE H-MATRIX SYSTEM REPRESENTATION. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 235-242.

Audley, D.R. and W.J. ~ (1973b) A GENERAL FOUNDATION FOR LINEAR SYSTEM REALIZATION. In: Proc. 7th Annual Princeton Conf. on Information Sciences and Systems, Princeton, 22-23 March 1973. Princeton, N.J.: Department of Electrical Engineering, Princeton University, 1973. P. 223-227.

Barraud, A. and P. de Larminat (1973) MINIMAL REALIZATION AND APPROXIMATION OF LINEAR SYSTEMS FROM NORMAL OPERATING RECORDS. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 953-956.

Bingulac, S.P. and M. Djorovic (1973) ESTIMATION OF STATE VARIABLE MODELS USING INPUT-OUTPUT DATA. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 995-1004.

Bingulac, S.P. (1976) IDENTIFICATION OF MULTIVARIABLE DYNAMIC SYSTEMS. In: Identification and System Parameter Estimation. Proc. 4th IFAC Symp., Tbilisi, USSR, 21-27 Sept. 1976. Part 3. Edited by N.S. Ra.jbman. Amsterdam: North-Holland, 1978. P. 1979-1988.

Birkhoff, G. and S. MacLane (1965) A SURVEY OF MODERN ALGEBRA. 3rd ed. New York: Macmillan, 1965.

nonivento, G. and R. Guidorzi (1971) CANONICAL INPUT-OUTPUT DESCRIPTION OF LINEAR MULTI VARIABLE SYSTEMS. Ricerche di Automatica, Vol. 2( 1971), p. 72-83.

Bonivcnto, C. and R. Guidorzi (1972) APPLICATION OF AN IDENTIFICATION METHOD TO DISTILLATION COLUMNS. In: Proe. 6th A~~ual Princeton Conf. on Information Sciences and Systems, Princeton, 23-24 March 1972. Princeton, N.J.: Department of Electrical Engineering, Princeton University, 1972. P. 185-189.

Bruni, C., A. =sidori and A. Ruberti (1969) THE ORDE1~ OF LINEAR CONTINUOUS SYSTEMS DESCRIBED BY THE IMPULSE RESPONSE ~~TRIX. Alta Frequenza, Vol. 38(1969), p. 108-116, No.2 (Feb.), English edition No.1.

- 143 -

Budin, M.A. (1971) MINIMAL REALIZATION OF DISCRETE LINEAR SYSTEMS FROM INPUT-OUTPUT OBSERVATIONS. IEEE Trans. Autom. Control, Vol. AC-16(1971), p. 395-401.

Budin, M.A. (1972a) A NEW APPROACH TO SYSTEM IDENTIFICATION AND STATE ESTIMATION. IEEE Trans. Syst., Man & Cybern., Vol. SMC-2(1972), p. 396-402.

Budin, M.A. (1972b) CANONICAL FORMS FOR MULTI-INPUTjMULTI-OUTPUT MINIMAL SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-17(1972), p. 554.

Candy, J.V., M.E. Warren and T.E. Bullock (1978) REALIZATION OF INVARIANT SYSTEM DESCRIPTIONS FROM INFINITE MARKOV SEQUENCES. IEEE Trans. Autom. Control, Vol. AC-23(1978), p. 93-96.

Chen, C.T., C.A. Desoer and A. Niederlinski (1966) SIMPLIFIED CONDITIONS FOR CONTROLLABILITY AND OBSERVALITY OF LINEAR TIME-INVARIANT SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-11(1966), p. 613-614.

Denham, M.J. (1974) CANONICAL FORMS FOR THE I9ENTIFICATION OF MULTIVARIABLE LINEAR SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 646-656.

DeRusso, P.M., R.J. Roy and Ch.M. Close (1965) STATE VARIABLES FOR ENGINEERS. New York: Wiley, 1965.

Dickinson, B.W., T. Kailath and M. Morf (1974) CANONICAL MATRIX FRACTION AND STATE-SPACE DESCRIPTIONS FOR DETERMINISTIC AND STOCHASTIC LINEAR SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-19 (1974), p. 656-666.

Dickinson, B.W., M. Morf and T. Kailath (1974) A MINIMAL REALIZATION ALGORITHM FOR MATRIX SEQu&~CES. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 31-38.

Eykhoff, P. (1974) SYSTEM IDENTIFICATION: Parameter and state estimation. London: Wiley, 1974.

Falb, P.L. (1967). INFINITE-DIMwNSIONAL FILTERING: The Kalman-Bucy filter in Hilbert space. Inf. <I: Control, Vol. 11(1967), p. 102-137.

- 144 -

Fasol, K.H. and H.P. Jorgl (1979) MODELLING AND IDENTIFICATION. In: Tutorials on System Identification. Presented at the 5th IFAC-Symposium on Identification and System Parameter Estimation, Darmstadt, 24-28 Sept. 1979. Ed. by R. Isermann. Darmstadt: Institut fUr Regelungstechnik, Technische Hochschule Darmstadt, 1979. P. 1-13.

Faurre, P. and J.P. Marmorat (1969) UN ALGORITHME DE REALISATION STOCHASTIQUE. C.R. Acad. Sci. Paris Ser. A, Vol. 268(1969), p. 978-981.

Faurre, P. (1970) IDENTIFICATION PAR MINIMISATION D'UNE REPRESENTATION MARKOVIENNE DE PROCESSUS ALEATOIRE. In: Symposium on Optimization, Nice, 29 .June -5 July 1969. Fn. by A.V. Balakrishnan et all Berlin: Springer, 1970. (Lecture Notes in Mathematics, Vol. 132.) P. 85-107.

Faurre, P.L. (1976) STOCHASTIC REALIZATION ALGORITHMS. In: System Identification: Advances and case stUdies. Ed. by R.K. Mehra and D.G. Lainiotis. New York: Academic Press, 1976. (Mathematics in Science and Engineering, Vol. 126.) P. 1-25.

Furuta, K. (1973) AN APPLICATION OF REALIZATION THEORY TO IDENTIFICATION OF MULTI VARIABLE PROCESS. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 197~. P~t 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 939-942.

Gantmacher, F.R. (1959) THE THEORY OF MATRICES. Vol. 1 and 2. New York: Chelsea, 1959. Original Russian ed.: Teorija matric 1, 2. Moskva 1954.

Gerth, W. (1971) ON THE CONSTRUCTION OF LOW DRDER LINEAR STATE-VARIABLE MODELS FROM MEASURED DATA. In: Proc. 2nd IFAC Symp. on Multivariable Technical Control Systems, Dusseldorf, 11-13 Oct. 1971. Vol. 1. Ed. by H. Schwarz. Amsterdam: North-Holland, 1971. Paper 1.2.1 (15 pages).

Gerth, W. (1 972) ZUR IDENTIFIKATION UND MINIMALREALISIERUNG VON MEHRGROSSENVBERTRAGUNGS­SYSTEMEN DURCH MARKOVPARAMETER. Ph.D. Diss. Technische Universitat Hannover, 1972.

Golub, G.H. and C. Reinsch (1970) SINGULAR VALUE DECOMPOSITION AND LEAST SQUARES SOLUTIONS. Numerische Mathematik, Vol. 14(1970), p. 403-420.

Gevers, M.R. and T. Kailath (1973) AN INNOVATIO~;S APPROACH TO LEAST-SQUARES ESTIMATION - Part VI: Discrete-time i~nov&tions and recursive estimation. IEEE Trons. Acto" .. Control, Vol. AC-18(1973), p. 588-600.

Gopinath, B. (1569) ON THE IDENTIFICATION OF LINEAR TIME-INVARIANT SYSTEMS FROM INPUT-OUTPUT DATA. Bell Syst. Tech. J., Vol. 48(1969), p. 1101-1113.

- 145 -

Guidorzi, R.P. (1973) CANONICAL STRUCTURES IN THE IDENTIFICATION OF MULTI VARIABLE SYSTEMS. In: Identification and System Parameter Estimation. Proc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. Eykhoff. Amsterdam: North-Holland, 1973. P. 943-952.

Gupta, R.D. and F.W. Fairman (1973) AN EXPLICIT APPROACH TO MINIMAL REALIZATION FROM SYSTEM MARKOV PARAMETERS. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 52-54.

Ha,jdasinski, A.K. (1976) A MARKOV PARAMETER APPROACH TO IDENTIFICATION OF MULTIVARIABLE DYNAMICAL SYSTEMS. Eindhoven: Group Measurement and Control, Department of Electrical Engineering, Eindhoven University of Technology, 1976. Technical Report.

Ha,jdasiftski, A.K. (1978) THE GAUSS-MARKOV APPROXIMATED SCHEME FOR IDENTIFICATION OF MULTIVARIABLE DYNAMICAL SYSTEMS VIA THE REALIZATION THEORY: An explicit approach. Eindhoven: Department of Electrical Engineering, Eindhoven University of Technology, 1978. TH-Report 78-E-88

Hajdasinski, A.K. and A.A.H. Damen (1979) REALIZATION OF THE MARKOV PARAMETER SEQUENCES USING THE SINGULAR VALUE DECOMPOSITION OF THE HANKEL MATRIX. Eindhoven: Department of Electrical Engineering, Eindhoven University of Technology, 1979. TH-Report 79-E-95

Ha,jdasinski, A.K. (1979) IDENTIFICATION OF MULTIVARIABLE NOISE CORRUPTED SYSTEMS WITH THE AID OF THE REALIZATION THEORY. In: Preprints 5th IFAC Symp. on Identification and System Parameter Estimation, Darmstadt, 24-28 Sept. 1979. Vol. 1. Ed. by R. Isermann. Oxford: Pergamon, 1979. P. 437-444.

Hajdasinski, A.K. (1980) ESTIMATION OF THE ORDER OF MULTIVARIABLE DYNAMICAL SYSTEMS. Journal A, Vol. 21(1980), p. 21-32.

Hajdu, L.P. (ed.) (1969) N.EW DEVELOPMENTS IN SYSTEM STATE ESTIMATION AND ANOMALY DETECTION. Palo Alto, California: Systems Control, Inc., 1969. Contract 14-03-79968 (Final Report)

Ho, B.L. and R.E. Kalman (1966) EFFECTIVE CONSTRUCTION OF LINEAR STATE-VARIABLE MODELS FROM INPUT-OUTPUT FUNCTIONS. Regelungstechnik, Vol. 14(1966), p. 545-548.

Is ermann , R. (1977) DIGITALE REGELSYSTEME. Berlin: Springer, 1977.

- 146 -

Isidori, A. (1972) ALGORITHMS FOR ORDER EVALUATION IN LINEAR SYSTEM THEORY. Ricerche di Automatica, Vol. 3(1972), p. 78-96.

Jazwinski, A.H. (1970) STOCHASTIC PROCESSES AND FILTERING THEORY. New York: Academic Press, 1970. Mathematics in Science and Engineering, Vol. 64.

Kailath, T. (1968) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part I: Linear filtering in additive white noise. IEEE Trans. Autom. Control, Vol. AC-13(1968), p. 646-655.

Kailath, T. and P. Frost (1968) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part II: Linear smoothing in additive white noise. IEEE Trans. Autom. Control, Vol. AC-13(1968), p. 655-660.

Kailath, T. (1970) THE INNOVATIONS APPROACH TO DETECTION AND ESTIMATION THEORY. Proc. IEEE, Vol. 58(1970), p. 680-695.

Kailath, T. and R.A. Geesey (1971) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part IV: Recursive estimation given lumped covariance functions. IEEE Trans. Autom. Control, Vol. AC-16(1971), p. 720-727.

Kailath, T. and R.A. Geesey (1973) AN INNOVATIONS APPROACH TO LEAST-SQUARES ESTIMATION - Part V: Innovations representations and recursive estimation in colored noise. IEEE Trans. Autom. Control, Vol. AC-18(1973), p. 435-453.

Kalman, R.E. (1960) ON THE GENERAL THEORY OF CONTROL SYSTEMS. In: Automatic and Remote Control. Proc. 1st Int. Congress of the IFAC, Moscow, 27 June-7 July 1960. Vol. 1. Ed. by J.F. Coales. London: Butterworth, 1961. P. 481-492.

Kalman, H.E. and R.S. Bucy (1961) NEW HESULTS IN LINEAR FILTERING AND PHEDIGTION THEORY. Trans. ASME Ser. D. J. Basic Engrg., Vol. 83(1961), p. 95-108.

Kalman, R.E. (1963) MATHEMATICAL DESCRIPTION OF LINEAR DYNAMICAL SYSTEMS. J. SIAM Cor:trol, Vol. 1 (1962/63), p. 152-192.

Kalman, R.S. (1967) ALGEBRAIC ASPECTS OF THE THEORY OF DYNAMICAL SYSTEMS. In: Differedial Equations and Dynamical Systems. Ed. by J.K. Hale and J.P. LaSalle. New York: Academic Press, 1967. P. 133-146.

- 147 -

Kalman, R.E., P.L. Falb and M.A. Arbib (1969) TOPICS IN MATHEMATICAL SYSTEM THEORY. New York: McGraw-Hill, 1969. International Series in Pure and Applied Mathematics.

Krause, A. (1976) METHODS FOR IDENTIFICATION OF MULTIVARIABLE LINEAR SYSTEMS. M.Sc. Thesis. Gliwice Polytechnic Institute, 1976. (In Polish)

Kuo, B.C. (1975) AUTOMATIC CONTHOL SYSTEMS. Jrd ed. Englewood Cliffs, ~.J.: Prentice-Hall, 1975. P. 156-157.

Kuo, B.C. (1970) DISCRETE-DATA CONTROL SYSTEMS. Englewood Cliffs, N.J.: Prentice-Hall, 1970.

Lal, M., H. Singh and R. Parthasarathy (1975) A MINIMAL CANONICAl REAlIZATION ALGORITHM FOR IMPULSE RESPONSE MATRIX USING MOMENTS. Proc. IEEE, Vol. 63(1975), p. 538-540.

Luenberger, D.G. (1966) OBSERVERS FOR MULTI VARIABLE SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-11(1966), p. 190-197.

Luenberger, D.G. (1967) CANONICAL fORMS FOR LINEAR MULTIVARIABLE SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-12(1967), p. 290-293.

MacFarlane, A.G.J. (1970) RETURN-DIFFERENCE AND RETURN-RATIO MATRICES AND THEIR USE IN ANAlYSIS AND DESIGN OF MULTIVARIABLE FEEDBACK CONTROL SYSTEMS. Proc. Inst. Electr. Eng. (London), Vol. 117(1970), p. 2037-2049.

Mayne, D.Q. (1968) COMPUTATIONAl PROCEDURE FOR THE MINIMAL REALISATION OF TRANSFER-FUNCTION MATRICES. Proc. Inst. Electr. Eng. (London), Vol. 115(1968), p. 1363.

Mayne, D.Q. (1972a) PARAMl,TRIZATION AND IDENTIFICATION OF' LINEAR MULTI VARIABLE SYSTEMS. In: Stability of Stochastic Dynamical Systems. Proc. of the Int. Symp., University of Warwick, 10-14 July, 1972. Ed. by R.F. Curtain. Berlin: Springer, 1972. Lecture Notes in Mathematics, Vol. 294. P. 56-61.

Mayne, D.Q. (1972b) A CANONICAL MODEL FOR IDENTIFICATION OF MULTIVARIABLE LINEAR SYSTEMS. IEF£ Trans. Autom. Control, Vol. AC-17(1972), p. 728-729.

Mehra, H.I<:. (1971) ON-LINE IDENTIFICATION OF LINEAR DYNAMIC SYSTEMS WITH APPLICATIONS TO KALMAN FILTERING. IEEE Trans. Aut am. Control, Vol. AC-16(1971), p. 12-21.

- 148 -

Mehra, R.K. (1972) AN INNOVATIONS APPROACH TO MAXIMUM LIKELIHOOD IDENTIFICATION OF LINEAR AND NONLINEAR DYNAMIC SYSTEMS. In: Froc. 5th Hawaii Int. Coni'. on System Sciences, Honolulu, 11-13 Jan. 1972.~Ed. by A. ~. North Hollywood, Calif.: Western Periodicals, 1972. P. 252-254.

Niederlinski, A. (1970) ANALYSIS AND DESIGN OF 2-VARIABLE INTERACTING CONTROL SYSTEMS USING INVERSE POLAR PLOTS. Proc. Inst. Electr. Eng. (London), Vol. 117(1970), p. 20%-2059.

Niederlinski, A. (1971a) DIE VERALLGEMEINERUNG DER ZIEGLER-NICHOLSSCHEN REGELN FUR qEKOPPELTE ZWEI~ UND MEHRFACHREGELUNGEN. Regelungstechnik und Frozess-Datenver­arbeitung, Vol. 19(1971), p. 291-295.

Niederlinski, A. (1971b) SOME NEW MEASURES OF INTERACTION FOR LINEAR MULTIVARIABLE PLANTS AND THE APPLICATIONS. In: Froc. 2nd IFAC Symp. on Multivariable Technical Control Systems, Dusseldorf, 11-13 Oct. 1971. Vol. 1. Ed. by H. Schwarz. Amsterdam: North-Holland, 1971. Paper 1.3.5 (13 pages).

Niederlinski, A. (1971c) TWO-VARIABLE DISTILLATION CONTROL: Decouple or not decouple. AIChE J., Vol.17(1971), p. 1261-1263.

Niederlinski, A. (1971d) A HEURISTIC APPROACH TO THE DESIGN OF LINEAR MULTIVARIABLE INTERACTING CONTROL SYSTEMS. Automatica, Vol. 7(1971), p. 691-701.

Niederlinski, A. (1972) CONDITION FOR STRUCTURAL MONOTONIC INSTABILITY OF INTERACTING LINEAR MULTI VARIABLE CONTROL SYSTEMS WITH INTEGRATING CONTROLLERS. Proc. Inst. Electr. Eng. (London), Vol. 119(1972), p. 255-257.

Niederlinski, A. (1974) UKbADY WIELOWYMIAROWE AUTOMATYKI (Multivariable Control Systems). Warszawa: Wydawnictwo Naukowo-Techniczne, 1974. (In Polish)

Niederlinski, A. and A.K. Hajdasiriski (1979) MULTIVARIABLE SYSTEM IDENTIFICATION - A SURVEY. In: Freprints 5th IFAC Symp. on Identification and System Parameter Estimation, Darmstadt, 24-28 Sept. 1979. Vol. 1. Ed. by R. Isermann. Oxford: Pergamon, 1979. P. 43-76.

Ostrowski, A.M. (1952) NOTE ON BOUNDS FOR DETERMINANTS WITH DOMINANT PRINCIPAL DIAGONAL. Froc. Amer. Math. Soc., Vol. 3(1952), p. 26-30.

Paul, C.R. and Yen-Long Kuo (1971) CONTROLLABILITY AND OBSERVALITY OF MULTIVARIABLE SYSTEMS. IFEE Trans. Autom. Control, Vol. AC-16(1971), p. 207-209.

- 149 -

Popov, V.M. (1972) INVAI(lMT m:~3CIUrTlON OF lINl';JIH, TIMe:-INVAlUANT CONTHOLLABU: ~YSTEMS. SIAM J. Control, Vol. 10(1972), p. 252-264.

Rijnsdorp, J.E. (1961) SINGLE AND MULTIPLE LOOP CONTROLS. In: Handbook of Automation, Computation, and Control. Vol. 3. Ed. by E.M. Grabbe, S. Ramo and E.D. Wooldridge. New York: Wiley, 1961. Chapter 10 (42 pages).

Ri,insdorp, J .E. (1965) INTERACTION IN TWO-VARIABLE CONTROL SYSTEMS FOR DISTILLATION COLUMNS, I and II. Automatica, Vol. 3(1965/66), p. 15-52.

Rijnsdorp, J.E. (1971) MULTIVARIABLE SYSTEMS IN PROCESS CONTROL. In: Proc. 2nd IFAC Symp. on Multivariable Technical Systems, Dusseldorf, 11-13 Oct. 1971. Vol. 4. Ed. by H. Schwarz. Amsterdam: North-Holland, 1971. P. 133-161.

Rissanen, J. and T. Kailath (1972) PARTIAL REALIZATION OF RANDOM SYSTEMS. Automatica, Vol. 8(1972), p. 389-396.

Rissanen, J. (1974) BASIS OF INVARIANTS AND CANONICAL FORMS FOR LINEAR DYNAMIC SYSTEMS. Automatica, Vol. 10(1974), p. 175-182.

Roman, J.R. and T.E. Bullock (1975) MINIMAL PARTIAL REALIZATIONS IN A CANONICAL FORM. IEEE Trans. Autom. Control, Vol. AC-20(1975), p. 529-533.

Rosenbrock, H.H. (1970) STATE-SPACE AND MULTI VARIABLE THEORY. London: Thomas Nelson, 1970. Studies in Dynamical Systems

Rossen, R.H. and L. Lapidus (1972) MINIMUM REALIZATIONS AND SYSTEM MODELLING: Part I: Pundamental theory and algorithm. Part II: Theoretical and numerical extensions. AIChE J., Vol. 18( 1972), p. 673-684 (part I) and 881-892 (part II).

Rozsa, P. and N.K. Sinha (1974) EFFICIENT ALGORITHM FOR IRREDUCIBLE REALIZATION OF A RATIONAL MATRIX. Int. J. Control, Vol. 20(1974), p. 739-751.

Rozsa, P. and N.K. Sinha (1975) MINIMAL REALIZATION OF A TRANSFER FUNCTION MATRIX IN CANONICAL FORMS. Int. J. Control, Vol. 21(1975), p. 273-284.

Schwarz, n. (1971) MEHRFACHREGELUNGEN. Band 2. Berlin: Springer, 1971.

Shamash, Y. (1975) MODEL REDUCTION USING MINIMAL REALIZATION ALGORITHMS. In: Proc. 9th Convention of Electrical and Electronics Engineers in Israel, Tel-Aviv 22-24 April 1975. New York: IEEE, 1975. Paper A-3-4 (7 pages). '

- 150 -

Shieh, L.S., Y.J. Wei and R. Yates (1975) MINIMAL REALIZATIONS OF TRANSFER-FUNCTION MATRICES BY MEANS OF MATRIX CONTINUED FRACTIONS. Int. J. Control, Vol. 22(1975), p. 851-859.

Silverman, L.M. and H.E. Meadows (1969) EQUIVALENT REALIZATIONS OF LINEAR SYSTEMS. SIAM J. Appl. Math., Vol. 17(1969), p. 393-408.

Silverman, L.M. (1971) REALIZATION OF LINEAR DYNAMICAL SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-16(1971), p. 554-567.

Sinha, N.K. (1975) MINIMAL REALIZATION OF TRANSFER FUNCTION MATRICES: A comparative study of different methods. Int. J. Control, Vol. 22(1975), p. 627-639.

Sinha, N.K. and A. Sen (1976) ON LINE IDENTIFICATION OF MULTIVARIABLE SYSTEMS. In: Identification and System Parameter Estimation. Proc. 4th IFAC Symp., Tbilisi, USSR, 21-27 Sept. 1976. Part 3. Edited by N.S. Ra.jbman. Amsterdam: North­Holland, 1978. P. 1501-1510.

Soderstrom, T. (1977) ON MODEL STRUCTURE TESTING IN SYSTEM IDENTIFICATION. Int. J. Control, Vol. 26(1977), p. 1-18.

Tether, A.J. (1970) CONSTRUCTION OF MINIMAL LINEAR STATE-VARIABLE MODELS FROM FINITE INPUT-OUTPUT DATA. IEEE Trans. Autom. Control, Vol. AC-15(1970), p. 427-436.

Tham, Q.C. (1976) A MINIMAL POLYNOMINAL FROM THE MARKOV PARAMETERS OF SYSTEM. IEEE Trans. Autom. Control, Vol. AC-21(1976), p. 133-134.

Thiga, R.S.A. and N.E. Gough (1974) ARTIFICIAL INTELLIGENCE APPLIED TO THE DISCRIMINATION OF THE ORDER OF MULTI VARIABLE LINEAR SYSTEMS. Int. J. Control, Vol. 20(1974), p. 961-969.

Tse, E. and H.L. Weinert (1975) STRUCTURE DETERMINATION AND PARAMETER IDENTIFICATION FOR MULTIVARIABLE STOCHASTIC LINEAR SYSTEMS. IEEE Trans. Autom. Control, Vol. AC-20(1975), p. 603-613.

van den Boom, A.J.W. and A.W.M. van den Enden (1973) THE DETERMINATION OF THE ORDER OF PROCESS- AND NOISE DYNAMICS. In: Identification and System Parameter Estimation. Froc. 3rd IFAC Symp., The Hague/Delft, 12-15 June 1973. Part 2. Ed. by P. EYkhoff. Amsterdam: North-Holland, 1973. P. 929-938.

- lSI -

van der Karn, J.J. and A.A.H. Darnen (1978) OBSERVABILITY OF EI"ECTRICAL HEART ACTIVITY STUDIED WITH THE SINGULAR VALUE DECOMPOSITION. Eindhoven: Department of Electrical Engineering, Eindhoven University of Technology, 1978. TH-Report 78-E-81

Wolovich, W.A. (1974) LINEAR MULTI VARIABLE SYSTEMS. Berlin: Springer, 1974. Applied Mathematical Sciences, Vol. 11.

Zadeh, L.A. (1962) FROM CIRCUIT THEORY TO SYSTEM THEORY. Proc. Inst. Radio Engrs (New York), Vol. 50(1962), p. 856-865.

Zadeh, L.A. and Ch.A. Desoer (1963) LINEAR SYSTEM THEORY: The state space approach. New York: McGraw-Hill, 1963. McGraw-Hill Series in System Science

Zadeh, L.A. and E. Polak (1969) SYSTEM THEORY. New York: McGraw-Hill, 1969. Inter-University Electronics Series, Vol. 8.

Zeiger, H.P. and A.J. McEwen (1974) APPROXIMATE LINEAR REALIZATIONS OF GIVEN DIMENSION VIA HO'S ALGORITHM. IEEE Trans. Autom. Control, Vol. AC-19(1974), p. 153.

EINDHOVEN UNIVERSITY OF TECHNOLOGY THE NETHERLANDS DEPARTMENT OF ELECTRICAL ENGINEERING

Reports:

93) Duin, C.A. van DIPOLE SCATTERING OF ELECTROMAGNETIC WAVES PROPAGATION THROUGH A RAIN MEDIUM. TH-Report 79-E-93. 1979. ISBN 90-6144-093-9

94) Kuijper, A.H. de and L.K.J. Vandamme CHARTS OF SPATIAL NOISE DISTRIBUTION IN PLANAR RESISTORS WITH FINITE CONTACTS. TH-Report 79-E-94. 1979. ISBN 90-6144-094-7

95) Hajdasinski, A.K. and A.A.H. Damen

96)

REALIZATION OF THE MARKOV PARAMETER SEQUENCES USING THE SINGULAR VALUE DECOMPOSITION OF THE HANKEL MATRIX. TH-Report 79-E-95. 1979. ISBN 90-6144-095-5

Stefanov. B. ELECTRON MOMENTUM TRANSFER CROSS-SECTION IN CESIUM AND RELATED CALCULATIONS OF THE LOCAL PARAMETERS OF Cs + Ar MHD PLASMAS. TH-Report 79-E-96. 1979. ISBN 90-6144-096-3

97) Worm, S.C.J. RADIATION PATTERNS OF CIRCULAR APERTURES WITH PRESCRIBED SIDELOBE LEVELS. TH-Report 79-E-97. 1979. ISBN 90-6144-097-1

98) Kroezen, P.H.C. A SERIES REPRESENTATION METHOD FOR THE FAR FIELD OF AN OFFSET REFLECTOR ANTENNA. TH-Report 79-E-98. 1979. ISBN 90-6144-098-X

99) Koonen, A.M.J. ERROR PROBABILITY IN DIGITAL FIBER OPTIC COMMUNICATION SYSTEMS. TH-Report 79-E-99. 1979. ISBN 90-6144-099-8

100) Naidu, M.S. STUDIES ON THE DECAY OF SURFACE CHARGES ON DIELECTRICS. TH-Report 79-E-I00. 1979. ISBN 90-6144-100-5

101) Verstappen, H.L. A SHAPED CYLINDRICAL DOUBLE-REFLECTOR SYSTEM FOR A BROADCAST-SATELLITE ANTENNA. TH-Report 79-E-I0l. 1979. ISBN 90-6144-101-3

102) Etten, W.C. van THE THEORY OF NONLINEAR DISCRETE-TIME SYSTEMS AND ITS APPLICATION TO THE EQUALIZATION OF NONLINEAR DIGITAL COMMUNICATION CHANNELS. TH-Report 79-E-I02. 1979. ISBN 90-6144-102-1

103) Roer, Th.G. van de ANALYTICAL THEORY OF PUNCH-THROUGH DIODES. TH-Report 79-E-I03. 1979. ISBN 90-6144-103-x

104) Herben. M.H.A.J. DESIGNING A CONTOURED BEAM ANTENNA. TH-Report 79-E-104. 1979. ISBN 90-6144-104-8

EINDHOVEN UNIVERSITY OF TECHNOLOGY THE NETHERLANDS DEPARTMENT OF ELECTRICAL ENGINEERING

Reports:

105) Videc, M.F. STRALINGSVERSCHIJNSELEN IN PLASMA'S EN BEWEGENDE MEDIA: Een geometrisch­optische en een golfzonebenadering. TH-Report 80-E-105. 1980. ISBN 90-6144-105-6

106) Hajdasinski, A.K. LINEAR MULTIVARIABLE SYSTEMS: Preliminary problems in mathematical description, modelling and identification. TH-Report 80-E-106. 1980. ISBN 90-6144-106-4

107) Heuvel, W.M.C. van den CURRENT CHOPPING IN SF6. TH-Report BO-E-l07. 19BO. ISBN 90-6144-107-2

108) Etten, W.C. van and T.M. Lammers TRANSMISSION OF FM-MODULATED AUDIOSIGNALS IN THE B7.5 - lOB MHz BROADCAST BAND OVER A FIBER OPTIC SYSTEM. TH-Report 80-E-l08. 19BO. ISBN 90-6144-108-0